I came here after reading “I’ve got nothing to hide”. I believe that there is one crucial argument against data mining (and “stop and search”) which has been missed and is widely misunderstood. It is based on the same piece of mathematics which makes universal health screening a bad idea WHEN YOUR TEST ISN’T VERY GOOD. Basically – let’s assume 1 in a million people is an active terrorist (or has some rare condition), and that your technique is good enough to spot 80% of these people. That sounds like you should do it! However if 0.1% of the population are picked up as “potential terrorists” (or false positive to have the desease) then applying this data minimg will cause about 1000 innocent people to be arrested for every terrorist spotted (or patient unnecessarily treated). Not only is this a bad use of resources, it potentially radicalises and recruits for the terrorists. So if 1% of those falsely accused now believe they HAVE to take up arms against the government you just made the problem 10 times worse. In the clinical case if treatment side effects occur in 1% of cases you have a similar result of decreasing overall health.

Ah – if only we could increase mathematical literacy, perhaps we could avoid each generation of politicans and policemen thinking this was a good idea, until the public backlash they don’t understand forces them to abandon it as impractical.

Regards

JIBC