Chi-Square Test

Date: 04/24/2002 at 09:32:20
From: Dave Edwards
Subject: Probability

Four chiropractors examine a patient. Each can allocate the patient to 
a category, 1, 2, or 3. I have calculated by percentage the number of 
times the chiropractors agree for any combination of 2, 3, and all 
4. I need to compare this finding to that of chance probability of 
agreement. How do I do this? 

I have made grid diagrams of all possible outcomes, but there must be 
a smarter way. Hope you can help. Many thanks. 

Dave.

Date: 04/24/2002 at 10:21:23
From: Doctor Mitteldorf
Subject: Re: Probability

The statistical test you're looking for is called chi-square. You can 
start reading about it here:

   Calculating and Interpreting Expected and Chi-Square Tables
   http://mathforum.org/library/drmath/view/52756.html 

Here's a response that I wrote to another question on this subject:


You've just been hired as an expert in a gambling fraud case. The 
plaintiff has accused the Slippery Fingers Gambling Casino of using 
loaded dice. You are handed a six-sided die, and your assignment is 
to report to the court whether it's loaded or fair.

You roll it and get a 2. Obviously that doesn't tell you anything.  
You roll it again and get a 1. Still, you can't say anything. But 
suppose you roll the die lots of times - then you should be able to 
tell. Say you roll it 600 times. Your expected results are:

1- 100
2- 100
3- 100
4- 100
5- 100
6- 100

If you're off by a little, you're not going to say it was a loaded 
die. But if you're off by a lot, you'll tell the court, "Guilty as 
charged!"  For example, your intuition tells you that if you get

  Expected   Actual
1- 100        105
2- 100         97
3- 100         89
4- 100        108
5- 100        106 
6- 100         95
   
that seems about normal, but if your results are

  Expected   Actual
1- 100        142
2- 100         94
3- 100         90
4- 100         96
5- 100         91
6- 100         87
  
then for sure you're going to say there were too many 1's. But how do 
you make this scientific and objective? Can you attach a number to how 
unlikely you think it is that the die was really straight? 

That's what chi square is all about. To calculate a chi square 
statistic, first construct two more columns: The third column is the 
difference (actual-expected), and to get the fourth column you 
multiply the third column by itself and divide by the first column: 

4th column = chi square = (actual-expected)^2 / (expected)

  Expected   Actual  Difference   Chi square
1- 100        142         42         17.64
2- 100         94         -6           .36
3- 100         90        -10          1.00
4- 100         96         -4           .16
5- 100         91         -9           .81
6- 100         87        -13          1.69
                                    --------
                                     21.66

Add up the chi squares and you get 21.66. That's your measure of how 
far off from the expected your results are.

Now comes the part that's a little mysterious. Look in a table for 
what you expect chi square to come out to. You have six numbers, but 
they had to add up to 600 no matter what. In other words, if you only 
had 5 numbers, the 6th would be determined. So you really have only 
5 independent numbers in your system, or 5 "degrees of freedom." You 
look in a table of chi square for 5 degrees of freedom, and find your 
number 21.66 corresponds to a probability of 0.0006.  

So you report back to the judge: "Your Honor, an honest statistician 
will never report anything with certainty. But I can tell you from the 
experiment I did that there are only 6 chances in 10,000 that that die 
was true."

For an on-line chi square calculator, go to SurfStat.australia:

   http://www.anu.edu.au/nceph/surfstat/surfstat-home/tables.html 

- Doctor Mitteldorf, The Math Forum
  http://mathforum.org/dr.math/