When to Use Scott’s π or Krippendorff's α, If Ever?

Page **18** of **36**

1d. The other coder draws a marble randomly from the same urn, notes marble’s color, and puts it

back into the urn.

1e. If both draw black, each reports positive; if both draw white, each reports negative; in either

case they do not look at the object being coded (Assumption 2). Only when one draws a black

and the other draws a white will they code objectively, at which point they may honestly agree

or disagree, and report accordingly (Assumption 3).

1f. The two coders calculate the average of positive cases and the average of negative cases

they’ve reported. If one of the two averages has reached the predetermined quota, they stop

random drawing and coding, and report the remaining objects in such a way that both averages

meet the quota (Assumption 1). If neither average has reached the quota, they go to Step 1g.

1g. The coders repeat Step 1b and the subsequent steps until all objects are "coded."

The three assumptions also portray the following Krippendorff Scenario:

2a. Two coders set a quota for the black and white marbles, decide the total number of marbles

(*N*

*m*

), and fill the urn accordingly (Assumptions 1 and 2).

2b. They take an object to be coded (Assumption 2

b

).

2c. One coder draws a marble randomly from the urn, notes marble’s color, and puts it aside

without placing it back into the urn (Assumption 2

b

).

2d. The other coder draws another marble randomly from the same urn, notes marble’s color, and

puts it aside without placing it back into the urn (Assumption 2

b

).

2e. If both draw black, each reports positive; if both draw white, each reports negative; in either

case they do not look at the object being coded (Assumption 2

b

). Only if one draws a black

and the other draws a white will they code the object objectively, at which point they may

honestly agree or disagree, and report accordingly (Assumption 3).

2f. The two coders calculate the average of positive cases and the average of negative cases

they’ve reported. If one of the two averages has reached the predetermined quota, they stop