RSS

In Tests We Trust. Should We? (Part 2)

17 May

Yesterday, we visited Dr. Filomena (an imaginary doctor), got tested, and were told we have Burgdorfer syndrome (an imaginary disease), which is treated with permanent Tamoxifolate (an imaginary drug). Dr. F told us that the disease, BS, afflicts about 1% of the population and that the test she used to diagnose it is 90% accurate. Based on our positive test result, she says we have a 90% chance of having the disease. Is she right?

Well, no. Here’s why.

Consider a population of 1,000,000 people. If 1% of the population have the disease, that’s 10,000 people. We can tabulate the situation like this:

Well 990,000
Sick 10,000
Total 1,000,000

Now lets test the million people using the same 90% accurate test that Dr. F used. Based on the accuracy rate of the test, 90% get accurate results, 10% get a false result. Considering the people who get a positive test result, we can add another column to the table:

Population Positive Test
Well 990,000 99,000 (10%)
Sick 10,000 9,000 (90%)
Total 1,000,000 108,000

So, 10% of the well people get (incorrect) positive test results, indicating they have the disease when they don’t. On the other hand, 90% of the sick people get correct positive test results. They have the disease and the test says they do. Since the test is 90% accurate, it’s correct for 90% of the sick people and incorrect for 10% of the well people.

Now let’s add the column for negative test results.

Population Positive Result Negative Result
Well 990,000 99,000 (10%) 891,000 (90%)
Sick 10,000 9,000 (90%) 1,000 (10%)
Total 1,000,000 108,000 892,000

In the negative results column, 90% of the well people get a correct negative test result while 10% of the sick folks get an incorrect negative test result (the test says they don’t have the disease even though they do).

Looking at the positive column, we see that of all the people who get a positive test result (meaning the test says they have the disease), the vast majority of them are in the “Well” row and don’t, in fact, have the disease at all. Even with a positive result from a test that is 90% accurate, for a disease that affects 1% of the population, your chances of actually having the disease are about 1 in 12, or 8.3%.

As the incidence of the disease in the population increases, the chances of the test being accurate about you increase as well. Increasing the accuracy rate of the test also increases the chance of the test being accurate about you. However, even if we take the test accuracy all the way up to 99%, while holding the incidence rate of the disease in the population steady at 1%, a positive test result only indicates a 50% chance of having the disease.

So, should you take the Tamoxifolate?

I’d want a lot more assurance that I have an accurate diagnosis first. Stay tuned for another kick in the head tomorrow.

Advertisements
 
Leave a comment

Posted by on 2011/05/17 in statistics

 

Tags: , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: