Intercoder Reliability

To establish our intercoder reliability we first conducted coding training with (10) thought segments from our original avatar coding assignment containing (288) segments. We used the thought segments from our original avatar assignment since no other codes were available. With two coders involved in the process we each coded separately and came together to refine our coding instrument. Through the refining of our coding we reached an agreement on several codes to calibrate our coding more accurately. The intercoder reliability level that was calculated for each variable in the coding training on our first attempt was 90% in agreement, and a rating of substantial, so we moved onto the pilot test.

The next step in conducting our intercoder reliability was in a pilot test. By selecting (30) random thought segments from our (288) segments that were different from those used in our code training, we were able to have a different look at the

We felt it was important to take another (50) and complete them independently of the pilot test. We each coded these (50) independently and brought them together to evaluate reliability levels. Throughout the process we had several coding disagreements that we had to solve. Since there was only two of us we chose to discuss and resolve the disagreement by explaining our view on the segments. With each point of disagreement we were able to come to an agreement, and chose the code that best fit the segment. For the coding of the full sample, the intercoder reliability level that was calculated for each variable on our first attempt found that we were 90% in agreement, and a rating of substantial. The total amount of time that it took us to go through the various reliability tests, the training, pilot, and full sample was approximately 2 hours, with an additional hour to calculate and reference our reliability