About the Catalogue

Use of the Catalogue

The indicators

The quantity and quality of evidence is inconsistent across the indicators. For some very well researched areas such as literacy and re-notification, the task for the reviewers was to sift through a vast number of potentially useful interventions to identify those most likely to be suitable in the Victorian context. In other areas it was more difficult to find strategies supported by evaluation evidence, and it was necessary to recommend interventions built on evidence-based principles that appear to be producing promising results.

This particularly applies to areas such as increasing participation in kindergarten and absenteeism. Where the evidence is limited, this is clearly indicated in the literature review and efforts have been made to ensure that the recommended strategies, although not ‘gold standard’, have other qualities likely to make them useful for Best Start, such as good documentation and cultural appropriateness. Indeed, because of the nature of evaluation methods, some of the most innovative strategies are not supported by evidence at level one or level two but may nevertheless be appropriate for Best Start. These issues are discussed further in the companion volume, the technical report.

The literature reviews

Catalogue entries for each indicator are preceded by a literature review, which provides essential background information to guide the selection and implementation of the strategies. It is strongly recommended that users of the catalogue read the literature reviews.

The format of the reviews is as follows:

• a background section that provides key information about the indicator, including a definition of terms, where necessary
• a broad description and summary of the available literature (evidence base) relevant to the indicator
• the recommended strategies are described in detail, with information about their implementation and evaluation. This section also provides information about other promising interventions that did not make the final cut in this version of the catalogue
• a discussion of issues relevant to the selection and implementation of evidence-based strategies.

The evidence table

The evaluation framework included in the catalogue is based on the following criteria for the strength of supporting evidence:

1. well-supported practice — evaluated with a prospective randomised controlled trial
2. supported practice — evaluated with a comparison group and reported in a peer-reviewed publication
3. promising practice — evaluated with a comparison group
4. acceptable practice — evaluated with an independent assessment of outcomes, but no comparison group (such as pre and post-testing, post-testing only, or qualitative methods) or historical comparison group (such as normative data from standardised tests)
5. emerging practice — evaluated without an independent assessment of outcomes (such as formative evaluation, service evaluation conducted by host organisation).

These categories were based on several classification systems, most notably the one used by the California Evidence- Based Clearinghouse for Child Welfare (www.cachildwelfareclearinghouse.org, downloaded 7/9/06). The basis for this classification system is described in the companion volume, the technical report. Where interventions could potentially fall into two different categories (such as a rigorous but non-independent study), they were assigned to the highest relevant level of evidence.

In addition, the framework provides information on the following indicators of quality:

• replication — has the intervention been implemented and independently evaluated at more than one site?
• documentation — are the content and methods of the intervention well documented (such as provider training courses and user manuals) and standardised to control quality of service delivery?
• theoretical basis — is the intervention based upon a well-accepted theory or developed from a continuing body of work in its field?
• cultural reach — has the program been trialed with people in disadvantaged communities, Indigenous people or people from culturally and linguistically diverse backgrounds?

Cost-effectiveness was originally included in the framework but was later removed because this type of information was available for very few of the recommended strategies.

As the aim was to produce a catalogue of evidence-based strategies, the literature reviews and catalogue entries necessarily focused on interventions that had been rigorously evaluated. In some cases, where evidence was limited, it was necessary to include interventions for which there is relatively little evidence of effectiveness. The reasons for including these interventions are clearly indicated in the relevant sections of the report and catalogue. The focus on evidence means that some established practices that have not yet been evaluated were not able to be included in the catalogue. We would strongly encourage those who have developed early intervention programs for the Victorian context to consider evaluating them using robust methods in order that they may be included in future editions of the catalogue.

The catalogue entries

For each of the indicators, some evidence-based interventions suitable for implementation in the Victorian context have been identified and described.

The catalogue is not intended as a self-contained manual for implementation of the strategies. Rather, it provides evidence that is suitable, relevant and achievable within local resources and constraints. Contact details are provided so that sites can seek further information, including manuals and other documentation and advice on implementation. The catalogue includes the following elements:

• a brief report on the literature that supports the particular intervention
• an explanation of the evidence behind the strategy (why the strategy works)
• an explanation of the population group on whom the strategy could be expected to work
• an explanation of where the strategy might be expected to work
• contact details where sites can obtain further information.

 

More information

For more information on the development of the catalogue, including the search strategies and evidence evaluation framework, consult the companion document, the technical report. For more information about a recommended strategy, contact the person named on the catalogue entry (where possible, this is someone involved in the implementation rather than the evaluation), consult the website, or obtain and read the original sources, such as journal articles and reports (a full reference list is provided at the end of the literature review for each indicator). For more information about the Best Start program, see Best Start.