| Great
emphasis was placed on the development of codes that
could be applied with high levels of inter-rater
reliability. |
Coding
Procedures Once we had developed a list of what to
code we began developing the specific coding procedures.
Coding procedures were developed by a group of four code
developers, all of whom had participated in the initial
viewing and discussion of the 27 field test tapes. One of
the developers was from Germany, one from Japan, and two
from the United States. Threeone from each
country were doctoral students in either psychology
or education, all with classroom teaching experience. The
fourth member of the team was a doctoral student in
applied linguistics, also American, who helped us work
through the technical issues involved in coding classroom
discourse.
Some codes were used to indicate frequency of events,
others to indicate the durations of various kinds of
activities. In all cases, great emphasis was placed on
establishing the inter-rater reliability of the codes
before putting them to use. Once codes were defined and
coding procedures specified, reliability was checked
between the code developers. Only after the code
developers reached at least 80 percent agreement among
themselves on independent judgments of the same code did
coding begin. Actual coding was done by a separate group
of coders who were trained by the code developers to
implement the codes. Coding of the main study tapes did
not begin until coders were reliable with each other and
with the code developers on at least 80 percent of their
judgements.
|