Self-selected search Also, predefined search

Selecting Content

Dr. Beverly Showers: Iowa Professional Development Technical Assistance Seminar Series Training Materials, October 2003

The following narrative is a record of one person's approach to selecting staff development content for an identified need. Although there are other processes that legitimately can be taken when selecting staff development content, certain principles apply to all such searches. The principles are:

• Selection is a Critical Decision

  The selection of appropriate content for a district or school staff development program is one of the most critical decisions to be made. If the content does not have a solid research base, the district/school risks considerable expenditure of time, resources, and effort on learning, implementing and evaluating something that does not yield the desired effects.

• Match Student Achievement Goals to Content

  Finding a good match between district/school goals for student achievement and content likely to achieve those goals is not a simple process. The profit motive often drives the claims made by commercial interests for their products and personal ideologies can affect both the research undertaken and the results obtained by researchers as they pursue evidence for their beliefs. Unfortunately, there is not a simple index one can consult that states, "If your ninth-grade students are struggling with Algebra I, the three most powerful remedies are "x," "y," and "z."

• Invest Time in Searching for Appropriate Content

Investing time in the search for appropriate staff development content is time well spent. Spending the time and resources to investigate the research-based options that address your students' needs for improvement greatly increases the likelihood that a district/school staff development process will be successful.

Example of Process for Selecting Content

I. What is the district/school goal for improving student learning?

After analyzing its student achievement data, the Mid-Continent School District discovered a distressing pattern in its reading scores: total reading scores were declining through the grades. Thus, while 81% of elementary students were deemed proficient (using the state's criterion for proficiency), 65% of middle school students and 61% of high school students were scoring at the proficient rate. Closer examination of the data revealed that many of their students were struggling with higher-order comprehension tasks, or what NAEP defines as "the ability to interrelate ideas and make generalizations." (Campbell, Hombo, & Mazzeo, 2000)  When special education, low socioeconomic status and English language learner subgroups were examined, the trend of declining scores was even more pronounced.

The Mid-Continent School District set Annual Yearly Progress goals for reading, using the state's trajectories (as negotiated with the federal government under No Child Left Behind guidelines). Its middle and high schools, however, set goals to rapidly increase the numbers of students able to engage successfully in higher-order comprehension tasks.

II. Is there scientifically based research on teaching higher-order comprehension skills to secondary students?

What is available to address this need? Entering the research base in any specific area can be a bit overwhelming at first. Mid-Continent needed a plan that would enable it to identify choices in curriculum and instruction that addressed its need and had strong evidence supporting its efficacy in the area they wanted to improve.

[Although this example was generated by a single individual working alone, it is strongly suggested that a committee (three to six members) work together to study the research base and generate options for consideration by the staff. Thus, one or two people can search data bases, one can locate and copy relevant articles, and one or two can read and summarize the articles. Dividing the labor makes this a much easier task.]

Look at the work of others who share your agenda.

One way to enter the research base without being swamped by the sheer volume of published material is to begin with the work of others who have already begun the work of reviewing research in a given area. Mid-Continent started with three sources:

• The Iowa Content Networks (with its links to other reviews of research);
  
• Reading Research Quarterly (the primary research publication of the International Reading Association); and
  
• Review of Educational Research (a journal published by the American Educational Research Association that is devoted entirely to reviews of research on specific topics).
  

[A general note in terms of process: Try to get a general feel for a body of work, rather than going immediately for the "one right answer." Assume such a search is going to take a couple of days, and consider it time well spent if an entire staff is going to then invest a year of their time and energy studying, learning, implementing and evaluating the product of the search. When reading reviews, also mark promising references that you may want to read in full.]

Mid-Continent stopped here to summarize general findings and trends. At this point it appeared that several instructional strategies had strong research support for teaching advanced comprehension skills to adolescents (e.g., inductive strategies, activating prior schema, reciprocal teaching, independent reading with student choice of books, vocabulary teaching strategies, think alouds, and collaborative discourse.) It appeared that, given the multiplicity of student learning preferences in any classroom and the prior learning histories of struggling adolescent readers, a successful intervention needed to incorporate a variety of powerful instructional strategies.

The following sources provided additional information for Mid-Continent to consider before making a decision.

Educational Laboratories and Centers.  The federal government funds educational laboratories and centers around the country, many attached to universities. It is the mission of these labs and centers to conduct research in education. The web site www.ed.gov/prog_info/Labs/ links to all the federally funded educational laboratories. The site provided links to several labs currently conducting research in reading and provided some very useful reading.

At the Johns Hopkins Center for Social Organization of Schools, there is a very useful review of research on both reading and math for high school freshmen struggling with those subjects. The full text of the article provided a very useful summary of the needs of such students as well as the remedies available.

[Note: When reviewing articles that have not gone through a review process be aware that there may be mistakes or omissions. When authors are cited in text but missing in references, go to ERIC or EBSCO to find the reference.]

Publishers websites.  Some publishers hire researchers to evaluate the impact of their programs on students. Although some dismiss any research conducted by a publisher or commissioned by a publisher, judge such research on a case-by-case basis (e.g., examine the quality of an actual piece of research before drawing conclusions about the strength of its findings).

An example of such research is on the web site at www.scholastic.com on its secondary reading program READ 180. After reading the information on their web site, the publisher was asked for the technical report of the initial study conducted on this program and was willing to send the report.

ERIC and EBSCO.  ERIC and EBSCO are data bases which list references (and sometimes, full text) for articles published in educational journals, conference presentations, reports published by foundations, labs and centers, etc. Both data bases provide abstracts of reports and articles listed. When full text is not available on EBSCO, the reference can be found in ERIC and the document needed can be ordered or located in a college library.

The size of these data bases are simultaneously their strength and weakness – if one is imprecise when requesting information, a search can yield thousands of articles, many of which are neither research nor on your topic. Nevertheless, these are invaluable resources for identifying research in a given topic.

Summary.  Fewer programs exist at the secondary level than at the elementary level. There are, however, many studies of effective instructional strategies in this area. That left Mid-Continent School District with the choice of choosing from a few already developed secondary reading programs or developing one by learning a combination of powerful instructional strategies.

The benefits of choosing an already developed program were obvious. Someone else has already gone through the trial and error of combining various strategies and programs and testing the entire program to determine its effectiveness. A second benefit is the relative ease of getting training in one place for a developed program. However, as mentioned earlier, there are not as many choices at the secondary level.

III. Identify options

The next step for Mid-Continent School District was to summarize the findings of their search and to collect the relevant articles/studies before presenting options to a decision making body. The presentation to the group might begin like this: "We have identified three programs and six strategies (plus some promising practices) that are strongly supported by research. Secondary students have repeatedly improved their reading comprehension skills when taught with these programs and strategies. We have divided you into six teams – three teams will each get the best report of a program, and each of the other three teams will get the best study for each of two strategies. Before our next meeting, analyze your reports/articles and come prepared to summarize the findings and make recommendations for the faculty (or committee or board)."

IV. Get information on costs (for training and materials) and availability of trainers

Once the decision making body had ranked its choice, the top three choices were selected, and information was collected on costs for materials and training. This information assisted the decision-making group in making a final decision.

V. Insert your choice into the "Select Content" section of the Iowa Professional Development Model and begin!