Science 101: Cherry Picking & Black Swans
July 26, 2012. When I am asked to testify as an expert witness at a hearing, I am asked to submit a written document that will accompany my oral testimony. The question I address in my expert testimony is, “What scientific evidence do we have that this form of energy (low frequency electromagnetic fields, radio frequency radiation, or whatever) is harmful below guidelines?”
But that is not the question adjudicators want to hear. They want scientists to present a review of ALL literature so they can decide for themselves even though they are not qualified to address that question–no matter how brilliant they may be–if they don’t understand the scientific method. There is a disconnect between the legal system and the scientific method and weight–of–evidence and falsifiability are two areas where the legal system fails to understand science.
Journalists often make the same mistake and label scientists as being biased or having preferences when they present information showing that something is harmful without presenting the same number of studies showing that something is safe.
Unfortunately, policy makers fall into the same category. They just don’t get it! And–because they don’t get it–we have a lag in critical policy decisions that need to be made in a timely fashion. The result is that guidelines remain non-protective for much longer than necessary.
One key that gives this away are statements using the “c-words.” What are “c-words”? Conclusive, consistent, convincing often placed before the word “evidence” and preceded by the word “no”.
A typical statement might be, “We have no conclusive, consistent, convincing evidence that bla-bla-bla is harmful below guidelines.” As soon as you hear these words you recognize that evidence does exist but the person making this statement doesn’t hold that evidence in high regard. That person seldom expands by indicating what kind of evidence would be classified as conclusive, consistent or convincing, because if that evidence were available s/he would be in a quandary.
Science has a way of dealing with this “confusion” (another c-word) and that is the concept of falsifiability coined by Sir Karl Popper, one of the leading and most influential philosophers of science in the 20th century.