![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
A short mathematical rant
Jun. 24th, 2005 02:01 pmThe readers of Paulos and, indeed, anyone capable of adding two and two, probably have a greater knowledge of the laws of probability than most of the UK's medical profession. It was Paulos who demonstrated the medical profession's irredeemable stupidity when he set this poser (or something like it).
There is a one in a million chance of someone having a rare and fatal disease that may develop later in life. A test is found to establish whether this disease will develop. There is one chance in a hundred that a false positive will be given. If a person tests positive for this disease, an operation is available to prevent it developing, but there is a one in 10 chance that this operation will result in death. If a person tests positive for this disease, should you operate?
Nearly all the doctors questioned said, yes, you should operate.
But look at the numbers again. Suppose one million people are tested for this disease. One of them is likely to really have it. But, since the test generates a false positive one time in 100, 10,000 people will yield a positive result. In fact, if you test positive for this disease, there is still only a one in 10,000 chance that you have it. But, if the doctors had their way, you would undergo an operation from which 1,000 of these people would die, utterly unnecessarily. By not operating, one person would eventually die from the disease.
Well, this is not just a hypothetical joust. Professor Sir Roy Meadow testified at the trial of Sally Clark that the chances were one in 73 million that both the deaths of Mrs Clark's babies were genuine cot deaths.
How did our knight and professor come by this number? Well, he took the fact that about one in 8,400 babies dies a cot death, and he squared it. And as a result he came up with a number which implied that there was an utterly minuscule chance that a woman was innocent of killing one of her babies. So, incompetent mathematics by a fool resulted in a woman spending many years in prison for a crime that she almost certainly did not commit.
Indeed, the mathematics were incompetent on many levels.
1) No-one suggests that if you have a cot death, that there is only a one in 8,400 chance that you are innocent of murder. So, the mathematics here should have been one in 8,400 that the second death was pure chance.
2) The assumption is that cot death is like throwing a dice, where the two events are independent. Even the work which Meadows used to get his spurious probability emphasized that this was not the case. Just as the fact that a house has been burgled makes it more likely that the house will be burgled again, if one child dies from a cot death -- the causes of which are still not completely understood, then the likelihood of a second cot death increases, but by how much, no-one knows. Even this fool should have known this instinctively. If you had told him that there had been a history of twins in a family, and that only about 1 in 60 births resulted in twins, would he say that the chance of a mother having a second set of twins was one in 3,600? I doubt it.
3) As was also pointed out, the extreme rarity of the event made the statistics unreliable.
Meadow is only charged with serious professional misconduct. Perhaps he is innocent of that. All he is guilty of is being a fool. But when they let fools into positions of authority who are allowed to give "expert" testimony on matters about which they know nothing (in this case, maths) innocent people suffer. Are there no lawyers out there who can, by the use of just a few simple questions, illustrate the flaws evident in nearly all probability-based testimony used in the courts today?
It gets even worse with DNA. If I hear of one more "billion-to-on" chance being bandied about -- a number concocted almost out of thin air and based once again on the incorrect assumption that blocks of DNA occur independently of each other, rather than in rigidly constructed sequences, then I shall scream. Within a decade, dozens if not hundreds of people are likely to be released on the grounds that their convictions are unsafe, when it is pointed out that the DNA samples used were unreliable proof.
Depressing.
There is a one in a million chance of someone having a rare and fatal disease that may develop later in life. A test is found to establish whether this disease will develop. There is one chance in a hundred that a false positive will be given. If a person tests positive for this disease, an operation is available to prevent it developing, but there is a one in 10 chance that this operation will result in death. If a person tests positive for this disease, should you operate?
Nearly all the doctors questioned said, yes, you should operate.
But look at the numbers again. Suppose one million people are tested for this disease. One of them is likely to really have it. But, since the test generates a false positive one time in 100, 10,000 people will yield a positive result. In fact, if you test positive for this disease, there is still only a one in 10,000 chance that you have it. But, if the doctors had their way, you would undergo an operation from which 1,000 of these people would die, utterly unnecessarily. By not operating, one person would eventually die from the disease.
Well, this is not just a hypothetical joust. Professor Sir Roy Meadow testified at the trial of Sally Clark that the chances were one in 73 million that both the deaths of Mrs Clark's babies were genuine cot deaths.
How did our knight and professor come by this number? Well, he took the fact that about one in 8,400 babies dies a cot death, and he squared it. And as a result he came up with a number which implied that there was an utterly minuscule chance that a woman was innocent of killing one of her babies. So, incompetent mathematics by a fool resulted in a woman spending many years in prison for a crime that she almost certainly did not commit.
Indeed, the mathematics were incompetent on many levels.
1) No-one suggests that if you have a cot death, that there is only a one in 8,400 chance that you are innocent of murder. So, the mathematics here should have been one in 8,400 that the second death was pure chance.
2) The assumption is that cot death is like throwing a dice, where the two events are independent. Even the work which Meadows used to get his spurious probability emphasized that this was not the case. Just as the fact that a house has been burgled makes it more likely that the house will be burgled again, if one child dies from a cot death -- the causes of which are still not completely understood, then the likelihood of a second cot death increases, but by how much, no-one knows. Even this fool should have known this instinctively. If you had told him that there had been a history of twins in a family, and that only about 1 in 60 births resulted in twins, would he say that the chance of a mother having a second set of twins was one in 3,600? I doubt it.
3) As was also pointed out, the extreme rarity of the event made the statistics unreliable.
Meadow is only charged with serious professional misconduct. Perhaps he is innocent of that. All he is guilty of is being a fool. But when they let fools into positions of authority who are allowed to give "expert" testimony on matters about which they know nothing (in this case, maths) innocent people suffer. Are there no lawyers out there who can, by the use of just a few simple questions, illustrate the flaws evident in nearly all probability-based testimony used in the courts today?
It gets even worse with DNA. If I hear of one more "billion-to-on" chance being bandied about -- a number concocted almost out of thin air and based once again on the incorrect assumption that blocks of DNA occur independently of each other, rather than in rigidly constructed sequences, then I shall scream. Within a decade, dozens if not hundreds of people are likely to be released on the grounds that their convictions are unsafe, when it is pointed out that the DNA samples used were unreliable proof.
Depressing.
