ontario association of children's aid societies,

Originally developed by Simcoe Children's Aid Society in l990, the Intervention Spectrum is an eligibility determination tool that at least l5 Ontario CASs either use, or are in the process implementing. This article follows the development of the Spectrum past its genesis (see OACAS Journal, 38(1), l994) to its current status: the most comprehensive eligibility framework found to date within the North American child welfare field This paper reviews the literature on eligibility determination in Child Protection Services and describes the research project designed to establish the reliability, and validity of the Spectrum.

The Intervention Spectrum is an assessment framework developed to assist child protection professionals to make reliable and accurate judgments about whether a case is appropriate for child protection services (CPS). It is not a risk assessment tool nor is it based on social science research or empirical findings. The Spectrum is intended as an eligibility or gatekeeping tool to aid CPS staff at that first point of contact.

The Spectrum is unique for a number of reasons. It is a taxonomy or range of reasons for Child Protection Services that is grounded in the Child and Family Services Act (CFSA) and rated by level of severity.

As a comprehensive taxonomy, it provides agencies, their communities and funders with a common language that can promote clarity and understanding. If benchmarks, definitions, and descriptions of eligibility have a high level of similarity across the province, then databases can be developed and shared across communities. Furthermore, if CPS agencies are required to cut services then the Spectrum's taxonomy can allow agencies to be precise regarding the areas and intervention levels that can no longer be serviced.

No other eligibility tool found in North America to-date has matched the Spectrum's comprehensiveness. Furthermore, the Spectrum was developed by the field, which enhances its relevancy, utility and credibility. The Spectrum is easy-to-use and reflects the realities of child welfare by incorporating a broad range of Reasons for Service.

Minimum intervention levels are delineated for each Reason for Service. A minimum intervention level refers to the lowest level of severity within a Reason for Service that meets mandated requirements, thereby justifying action by the Child Protection Services agency. In other words, any referral that falls into a "most" and/or "medium" severity description within each Reason for Service means it lies above the minimum intervention level and receives CPS. Conversely, any referral placed into a "least" and "no" severity description falls below the minimum intervention level and does not receive Child Protection Services, but may be referred for community services. There is considerable network support for the minimum intervention levels as they were set by the Spectrum. Research Project in consultation with the nine CAS agencies using the Spectrum. The stated minimum intervention levels advance both clarity and communication within and across CPS agencies and communities. Some local determination of these levels can be made through agency and community consultation.

For Child Protection Services agencies, the decision to accept a report is not only a critical first step in agency compliance with the CFSA, but is an important decision in the allocation of diminishing funding and reduced resource availability. CPS agencies can no longer tolerate inefficiencies and many now need to review their screening practices. The Spectrum promises to bring clarity and accuracy and consistency to an area in child welfare that has not been well addressed in the field, policy, or research.

A review of the literature suggests that determination of eligibility for Child Protection Services cases and accompanying minimum intervention levels are relatively new areas of need and development in child welfare. Policy statements concerning eligibility state that CPS professionals must meet legislative and policy requirements. However, there are at least two key reasons why using clinical judgment alone to determine Child Protection Services eligibility has been problematic.

First, it is well documented that interpretation of the child protection mandate, including eligibility, will vary widely given the influence of factors such as workload pressures, availability of staff, minimal training, high turnover, and community factors (Leck and Ballantyne, I994; Wells, Stein, Fluke and Downing, 1989). The result of sole reliance on clinical judgment has meant a lack of consistency and accuracy in determining CPS eligibility. This point is illustrated in a recent review of Rhode Island cases by the Children's Resource Center ( I 993); they found that between I 5% to 25% of high risk cases were not opened for Child Protection Services, but many low risk families received CPS.

The second reason is an absence of legislative standards, agency guidelines and clinical criteria to assist Child Protection Services professionals regarding the factors to use in determining eligibility (Leck et. al., 1994). Although the need for a screening instrument seems intuitively logical, a review of the North American literature on eligibility determination in CPS finds some dominant factors have impeded development of such a framework.

A continuum of approaches to eligibility determination is the norm within Child Protection Services. At one end of the scale are CPS agencies that open and investigate most cases to avoid risk of making an error in judgment. Then there are Child Protection Services agencies that recognize screening as a distinct phase, but do not implement a framework or criteria, thus maintaining the sometimes faulty clinical judgement as the sole determinant for eligibility decisions. Further along the continuum are CPS agencies that view eligibility as a distinct phase and have developed tools and/or clear guidelines, but from the premise that it is just to screen out false reports and some low risk cases (Borone, Adoms, & Toomon, I9BI). Finally, it appears that only a few Child Protection Services agencies treat eligibility as a distinct and critical stage. They have developed a relevant instrument that reflects the mandated requirements coupled with clear criteria so as to increase the consistency and accuracy of screening decisions, and to avoid missing cases that are eligible.

 To date, the search for CPS eligibility tools has located only three in North America. The two American tools, the Washington Risk Assessment Mode) - Screening for Sufficiency Questions (WRAM) and the Navy Family Advocacy Program - Eligibility Decision, are similar in that they require the Child Protection Services worker to answer four short questions, centred around presence of imminent risk, allegation of abuse and child's age and location. The third and by far the most comprehensive eligibility instrument is the Spectrum.

First generation studies on eligibility determination in CPS focused on trying to establish common outcomes (i.e. substantiation or referral/reoccurrence) or ordering cases based on criteria (i.e. source of referral, amount of information, and presence of injury). However, these methods did not prove to be reliable or comprehensive enough in determining eligibility (Washington State, Department of Social & Health Services, 199I ).

Another barrier limiting the development of an eligibility framework has been the focus on the development and use of risk assessment models, child well-being scales, and structured decision-making systems. In particular, structured risk assessment tools have been used to determine eligibility. This is an inappropriate use of the tool as risk assessment instruments are designed to predict future risk of abuse or neglect. In fad, the ability of such tools to accurately predict future maltreatment has been seriously questioned (Wold & Woolverton, I990).

Many factors, either independently or in combination, have hindered development of a theoretical or clinical Child Protection Services eligibility framework. However, the ramifications for inconsistent and varied eligibility decisions by CPS professionals have striking similarity regardless of the jurisdiction. It means that other professionals as well as the general public will often view child protection workers as incompetent, biased, and unfair (Alter, / 985). Furthermore, providing Child Protection Services to cases that are not eligible contributes to diminishing CPS resources. The greatest concern of course is missed cases, where children who are in need of protection and are eligible for Child Protection Services do not get screened in.

In the final analysis, an eligibility framework is needed to increase the consistency and accuracy of screening decisions. This alone will likely improve CPS agencies' credibility and reputation. It appears from the search to date that the Spectrum is the most comprehensive eligibility determination tool in North America. While outcome and process data from the Child Protection Services agencies currently using the Spectrum suggests it has much promise, empirical substantiation has yet to occur. Therefore, MCSS has funded, through a grant to the OACAS, the Intervention Spectrum Research Project. The primary purpose of this research is to test the reliability and validity of the Spectrum.

In November 1994, a survey was completed by 51 of the 53 Ontario CAS agencies on their use of the Spectrum. The survey indicated that 16% (8) CAS agencies were using the Spectrum, 33% ( 17) were implementing or planning to implement the Spectrum, 37% ( 19) were interested in the Spectrum's use, and 14% (7) had no plans to use the Spectrum.

The 8 CAS agencies using the Spectrum indicated they believed that the Spectrum identified at least 90% of CPS cases under the CFSA. Limitations with it were usually linked to limitations associated with the CFSA, although the cost and difficulty for agencies to implement new instruments when faced with decreased funding and increased referrals were mentioned. Setting of the minimum intervention levels for five of the eight agencies was achieved through an internal and community consultation process. Short and long term effects from the community process were noted as improved community relations, more appropriate referrals and greater understanding from the community regarding the CAS mandate. Increased quantity and quality of referral information from the professional community were identified, especially with those cases that are rated near the minimum intervention levels.

Of agencies implementing the Spectrum, all expected the consistency of eligibility determination to improve. They also expected that use of the Spectrum would result in improved community relations (67%), more consistent decisions across the CAS network regarding eligibility (67%), and more appropriate referrals (50%). Besides clarifying actual and intended use, the survey was able to identify initial and ongoing impressions plus highlight areas for refinement and research.

Refinement of the Spectrum began in early I 995 and involved those agencies that were using the Spectrum. Referred to as the "Spectrum Users' Group", they provided suggestions to improve the Spectrum's format, wording, use, and descriptions, as well as identify any gaps, and overlap. The findings from the analysis of the minimum intervention levels of the various versions of the Spectrum were of particular interest. The analysis indicated that the agencies using the Spectrum varied at most only one intervention level in 2 I of the 25 Reasons for Service (84%). Only 4 categories ( I 6%), "Emotional Abuse", "Unattended Mental, Emotional and Developmental Condition", "Parent/Child Conflict" and "Child Misconduct", varied by two levels. In consultation with the Spectrum Users' Group, consensus was reached for all minimum intervention levels in the I 995 Spectrum. This collaborative, inclusive approach spawned many positive benefits and most importantly resulted in a Spectrum that has credibility, support, and relevancy within the Ontario CAS network.

The next step in the research process is critical and involves testing the reliability and validity of the Spectrum. This will occur from Spring 1996 to Spring 1997. Five CAS agencies were selected to participate: Huron, Frontenac (Kingston), Sudbury, Jewish Family and Child Service, and Toronto CCAS. Selection of this purposive sample was based on variance in criteria such as locale (north, west, east and central) regions), size (small, medium and large agencies), intake model (generalist, specialist and combination), type of cases (rural, urban inner-city), cultural groups (Native, English, French, and multi-cultural), degree of community consultation process (from none to some to extensive) and finally, interest in the research and willingness to use the Spectrum.

Reliability testing involves having the intake workers rate each selected case twice over a three week period (intrarater reliability). As well, a second worker will rate each of the selected cases after the first rater and will be unaware of the first rating (interrater reliability). Cases will be monitored for a maximum of six months. The Spectrum appears to be leading the field in the design of comprehensive eligibility instruments for Child Protection Services, so validity assessment through traditional means of comparison to other standardized tools - is not readily available. However, for content validity an expert panel of two judges, two lawyers, and two CPS workers will assess the Spectrum in terms of how closely it reflects the CFSA's intent. Furthermore, for predictive validity certain variables will be tracked for a maximum of six months to test whether higher severity ratings will be associated with greater resource use and more negative events. Variables include: how soon resources are referred within each case; the number of internal and external resources assigned to the case; whether the case is opened to Family or Children's Services; the number of agency worker hours spent on the case; and the number of placements within each case type. Convergent validity will be explored by having two of the research agencies review a pool of earlier cases which had been assessed using their traditional approach and compare this with a rating of the Spectrum.

Three of the five research agencies have elected to incorporate a community and staff consultation process related to introducing the Spectrum. This process involves asking both Child Protection Services staff and professional community members to identify, using the Spectrum and the CFSA, what they perceive to be the current minimum intervention levels for their CPS agency. As well, they will be asked to note what change there should be in these levels, based on their experience and perceptions.

It is hoped that this data will not only inform how and where different professional groups assess child welfare eligibility, but will provide an opportunity for the views of the professional community to be heard on the issue of current as well as future legislation.

The Spectrum holds much promise for its many stakeholders. It is an innovative approach to increase the consistency and accuracy of eligibility decisions in child welfare, delineate minimum intervention levels, and facilitate more appropriate reporting by the community. Long term benefits may accrue such as improved community relations, analysis and reporting of social needs, more focused Child Protection Services training, and enhanced management of scarce CPS resources.

Many challenges are associated with the development of tools for Child Protection Services that are relevant, accurate and effective. Research is crucial to ensure the reliability and validity of any instrument. The intent of this research is to address the ability of the Spectrum to achieve what it was designed for - to determine CPS eligibility consistently and accurately so children who are in need of protection receive Child Protection Services.

Purchase of the lntervention Spectrum is available through the Ontario Association of Children's Aid Societies.

Deborah Goodman is a Doctoral Candidate at the Faculty of Social Work, University of Toronto, and is the research assistant/project manager for the Intervention Spectrum Research Project.

Dr. Robert MacFadden is Associate Professor at the Faculty of Social Work, University of Toronto, and is principal investigator for the Intervention Spectrum Research Project.