| | | Reader 1 |
|---|
Positive | Negative |
|---|
Reader 2 | Positive | 19,818 | 156 |
Negative | 1 | 25 |
| | | Cohen κ | 0.240 |
| | | Information agreement | 0.580 |
- Tables 3 and 4 have the same Cohen κ (i.e., 0.5), but the number of accordant diagnoses in the former (i.e., 73.4%) is significantly lower than those of the latter (i.e., 99.4%). Information agreement properly measures this difference and assigns the values 0.311 and 0.638 to Tables 3 and 4, respectively. On the contrary, Tables 4 and 5 report quite similar scenarios, as they differ for less than 1% of the total diagnoses. However, the Cohen κ of the former (0.5) is much higher than that of the latter (0.240) whereas their information agreement (0.638 and 0.58, respectively) do not significantly deviate. In all these cases, the information agreement exhibits a behavior more consistent to the common sense of agreement than Cohen κ
