Methods
Coding and Analysis

Methods

Great emphasis was placed on the development of codes that could be applied with high levels of inter-rater reliability. Coding Procedures

Once we had developed a list of what to code we began developing the specific coding procedures. Coding procedures were developed by a group of four code developers, all of whom had participated in the initial viewing and discussion of the 27 field test tapes. One of the developers was from Germany, one from Japan, and two from the United States. Three—one from each country— were doctoral students in either psychology or education, all with classroom teaching experience. The fourth member of the team was a doctoral student in applied linguistics, also American, who helped us work through the technical issues involved in coding classroom discourse.

Some codes were used to indicate frequency of events, others to indicate the durations of various kinds of activities. In all cases, great emphasis was placed on establishing the inter-rater reliability of the codes before putting them to use. Once codes were defined and coding procedures specified, reliability was checked between the code developers. Only after the code developers reached at least 80 percent agreement among themselves on independent judgments of the same code did coding begin. Actual coding was done by a separate group of coders who were trained by the code developers to implement the codes. Coding of the main study tapes did not begin until coders were reliable with each other and with the code developers on at least 80 percent of their judgements.

[Page 3 of 3]
Bottom Bar