How to properly read a scientific paper–Adolescent brain tumours and mobile phones.
August 17, 2011. Scientific documents published in peer-reviewed journals are intended to be read by scientists with specific areas of specialization. A layperson, a journalist and even a scientist–who specializes in a different field–may find reading and comprehending such a document difficult. Critiquing such documents is what we teach university students. Once they learn how to decipher a scientific paper and decompose a study, they no longer need to rely on the opinion of others about that document. Teaching students how to think for themselves is one of the roles of a university professor.
I recently read Mobile phone use and brain tumors in children and adolescents: A multicenter case-control study, published in June 2011 in the Journal of the National Cancer Institute. What is in the abstract of this publication and what has been quoted by the press is not a fair and honest representation of the findings of this study.
If you want to know what is really happening with cell phones and cancer – you need to learn how to read between the lines. That is what scientists do. They go beyond the abstract and read the document. They examine the results and compare their own conclusions to those of the authors’. Some authors who are dealing with controversial, contentious, politically charged issues or those who have a bias–financial or otherwise–may deemphasize a finding to minimize backlash directed at them or their study. But, if the authors are honest, the document will provide the truth–sometimes buried between the lines–for other scientists to find. The truth is out there. You just need to learn how to find it.
Let’s examine the Röösli study and see what it really says. Click here for PDF of study.
The results in the abstract of this study, which is what most people read, state the following (emphasis and comments in square brackets [] are mine):
- Regular users of mobile phones were not statistically significantly more likely to have been diagnosed with brain tumors compared with nonusers (OR = 1.36; 95% CI = 0.92 to 2.02).
- Children who started to use mobile phones at least 5 years ago were not at increased risk compared with those who had never regularly used mobile phones (OR = 1.26, 95% CI = 0.70 to 2.28).
- In a subset of study participants for whom operator recorded data were available, brain tumor risk was related to the time elapsed since the mobile phone subscription was started [no odds ratio is provided] but not to amount of use.
- NO increased risk of brain tumors was observed for brain areas receiving the highest amount of exposure.
Based on these results it was natural for the authors to conclude the following:
The absence of an exposure-response relationship either in terms of the amount of mobile phone use or by localization of the brain tumor argues against a causal association.
Any parent, doctor, policy maker, journalist reading this would conclude that cell phone use by children–based on the conditions in this study–is unlikely to be harmful and indeed may be “safe.”
But is this what the study showed or is the abstract misleading by deemphasizing adverse outcomes?
*****
Here is what the abstract failed to say about risk to children and adolescents:
RESULTS:
1. If you use a cell phone at least once a week for at least 6 months you are classified as a regular user. This limited exposure, both in terms of amount and duration, dilutes the results and favors a “no effect” as shown in Table 2.
2. Based on records from your cell phone provider, if you use the phone for more than 2.8 years your chance of getting a brain tumour goes up by 115% and the longer you use it the greater the risk.
3. Those who use a cell phone have an increased risk of developing a tumour on the side of the head and a reduced risk of developing a tumour in the center of the head. This may be an artifact or it may be real. More studies are necessary.
4. For tumors on the side of head (same or opposite side held to a cell phone) the following was found:
a. The longer your subscription the greater the risk of lateral brain tumours. For subscriptions beyond 4 years the increased risk is between 274% to 300%.
b. There is evidence that tumor detection for “time since first use” is much shorter for children than for adults (more than 10 years) with statistically significant findings between 3.3 and 5.0 years (227% increased risk).
c. The more cumulative time you spend on calls the greater your risk of developing a tumour on the side of your head. Above 144 hours the risk increases to 519%.
d. The more calls made, the greater the risk of lateral tumours. More than 2638 calls and the risk increases by 191% to 482%.
CONCLUSION: The increased risk for brain tumours, on the side of the head for children and adolescents with such short duration and limited total calls, needs to be taken seriously as more young people are using cell phones and the longer they use them the greater their risk of developing a brain tumour.
RECOMMENDATION: Keeping the phone away from the head and minimizing your time using a phone is highly recommended for all mobile phone users. For more recommendations click here.
*****
Let’s have a look at the tables in this document and then you can decide which abstract, their’s or the one above, best summarizes the data.
Tables 2 to 5 provide the odds ratio (OR) and the confidence interval (CI). First we have to understand what these mean.
OR is the odds ratio or the ratio of observed to expected results–in this case–for brain tumors. The “observed” value is based on those who use cell phones and the “expected” value is based on those who do not use cell phones (referent population). Note the referent population OR is set at 1.0. An OR above one means a greater risk (harmful relationship) and an OR below one means a lower risk (beneficial relationship).
However, for this OR to be statistically significant, the CI (or confidence interval) must be above one for greater risk and below one for a reduced risk. This is a quick way of determining if an OR is statistically significant. Scientists do some fancy calculations (logistic regression models) that give similar results. See figure below for the different types of OR and CI possible and what they mean. Often a 95% CI is used, which means confidence that results will be within the interval indicated 95% of the time. In other words there is a 5% probability that the results are due to chance. Those are pretty good odds! If the value is greater than 5% it is rejected by convention.
Figure 1. Odds ratio and 95% confidence intervals.
Table 3. Odds ratio (OR) and 95% confidence intervals (CI) for stratified analyses.
In Table 3, most of the ORs are above one, which indicates an increased risk, but these are not statistically significant (i.e. scientists don’t have confidence in the values) and hence it may not be necessary to mention them in the abstract.
However, one of the values was statistically significant (OR 1.92; CI 1.07 to 3.44) indicating a 92% increased risk for tumors in other than the cerebellum, temporal and frontal lobes (highlighted). Indeed, this was important enough to be mentioned in the paper (see quote below) but not in the abstract:
We found no elevated risk among regular users of mobile phones when we looked at the parts of the brain with the highest radio frequency exposure, that is, the temporal and frontal lobes and the cerebellum (Table 3). On the other hand, we did find a statistically significant odds ratio for tumors in the parts of the brain with the lowest exposure to radiation among regular users of mobile phones (OR = 1.92; CI = 1.07 to 3.44).
Perhaps the authors decided this did not make sense because one might expect the highest risk of cancer to be in locations with the highest radiation exposure (adjacent to the mobile phone) as was shown in the INTERPHONE study (2010).
Had this been the ONLY statistically significant increased risk in the entire paper it would make sense not to bring attention to it in the abstract because it may have been an anomaly due to error of some sort, but . . . let’s look at other results.
Table 4. Comparison of analyses with operator-recorded and self-reported mobile phone use.
Once again, most of the ORs were above one (indicating an increased risk) but were not statistically significant. The exception was for “time since first subscription based on operator-recorded use” (highlighted). When this was compared with the referent (never a regular user) the OR 2.15 (115% increased risk) was statistically significant (CI 1.07 to 4.29). The trend for “operator recorded time since first subscription” was also statistically significant with a probability of 0.001 or with a 0.1% probably that this was due to chance (highlighted). This was mentioned in the abstract although the OR was not provided. If the OR is not provided it is difficult to determine the magnitude of the risk and difficult to quote as no value is provided. So the authors were signaling that there is a problem.
Table 5. Association between brain tumors and mobile phone use by side of phone use.
The data in Table 5 cannot be ignored.
In Table 5, all of the ORs were above one for both ipsilateral (same side of the head) and contralateral (opposite side of the head) tumours. In this table, many more of the results were statistically significant (highlighted) and several showed a statistically significant trend with increasing exposure measured as duration of subscription (years), cumulative duration of calls (hours) and very close to significance (P=0.06) with cumulative number of calls (see table below, based on Table 5 in original document).
Based on Table 5. Association between brain tumors and mobile phone use by side of phone use.
The table above, indicates an increased risk for either or both ipsilateral and contralateral tumors for more than 937 cumulative calls; for more than 36 hours of cumulative calls; for more than 4 years cumulative duration of subscriptions and between 3.3 and 5.0 years since first use. These ORs ranged from 2.66 to 6.19 (166% to 519% increased risk). Why were these statistically significant ORs not mentioned in the abstract?
The fact that contralateral tumours have a higher OR than ipsilateral tumors is less of a concern if side of head recall was unreliable as the authors state below:
“ . . . subjects’ statements about which side of the head they preferred to hold the mobile phone near during its use are often considered unreliable . . . “
Whether you use the phone on the left side or the right side of the head, the side of the head is going to be exposed and the exposure is likely to be higher than in the center of the head. So the fact that contralateral tumour risks were high may not be an anomalous result as they may not necessarily be contralateral tumours (due to recall error).
The other interesting observation is that the ORs for tumours in central or unknown locations had ORs below one and some of these were statistically significant (blue box, Table 5). So if lateral tumors are compared with central or unknown tumors the ORs are quite different.
Contrary to the conclusions–that the results in this document do not support a causal association between mobile phone use and brain tumours–the results in this publication are disturbing as they indicate an increased risk of brain tumours in children and adolescents after relatively short periods of exposure (much shorter than for adults).
Indeed several scientists (Devra Davis, Lloyd Morgan, Ronald Herberman) have openly criticized this study, presumably after they went through a similar deconstruction and interpretation of the results. Click here for pdf. Sam Milham submitted a response to the Journal of the National Cancer Institute, which may or may not be published. Click here for pdf. Others have lauded this report and I can’t help but wonder if they read it.
This is a very important paper and needs to be read properly.
When I provide papers such as this to my students for their own evaluation their eyes glaze over as they look at the tables. Once they learn how to interpret the data in the tables they are often astounded by the discrepancy between their interpretation and those of the authors. Sometimes this is their first lesson that, “you should not believe everything you read, even if it is peer-reviewed.”
The abstract of this cell phone study does not adequately present the key findings in the study. The omission of adverse results may be due to a number of things including funding bias. Unfortunately, who funds the study can influence the experimental design and the interpretation of the results (Huss et al. 2007; click here for pdf). This was evident in the INTERPHONE study. It is becoming such a concern that journals are now requesting that authors declare all types of conflict of interest including funding sources.
Under the heading of “funding” primarily government research council grants are identified. However, in the “notes” immediately beneath “funding”, sources of industry or private funding received by the authors are provided and these include Mobile Manufacturers’ Forum, GSM Association, TellaSonera, Ericsson AB, Telenor, three Swiss mobile phone network operators, ENERGI.DK, two mobile telephone operators (not identified), and COWI consultants. TeliaSonera, Telenor (Sonofon), TDC, Telia and Hi3G supplied mobile phone operator data.
Despite the authors’ assurance that,
“The funders did not have any involvement in the design of the study; the collection, analysis, and interpretation of the data; the writing of the article; or the decision to submit the article for publication.”
and that
“Industry funding was given under agreements that the studies be given scientific independence”
it is difficult to explain why well qualified scientists who have been doing research for decades and who work for reputable institutions presented only positive outcomes that were favorable to the mobile phone industry and failed to include adverse outcomes in their abstract.
I contacted Dr. Röösli and asked him some questions. His response is provided below (August 16, 2011):
Thank you very much for your interest in our study. All coauthors contributed to the writing of the manuscript. We reached consensus in the interpretation of the results and we have also noted in the abstract positive findings (” In a subset of study participants for whom operator recorded data were available, brain tumor risk was related to the time elapsed since the mobile phone subscription was started but not to amount of use.”)
The laterality analyses we did not find consistent. Biologically it makes no sense to have increased ipsi AND contralateral risk and at the same time having decreased risk for centrally located tumors or for mobile phone users without a preferred head side. This is a clear indication of recall bias.
As this is quite a complex issue we have discussed it in the paper but not mentioned in the abstract as word count for abstract is quite restrictive.
I can’t help but wonder why one sentence stating that, “There was statistically significant increased risk for lateral tumours and a decreased risk for centrally located tumours,” was not mentioned.
Was it word count, funding, or something else? Perhaps the authors veiled the truth so as not to cause mass panic and lawsuits.
Studies that do not adequately present the key findings in the abstract do a disservice to society. They perpetuate misinformation, generate doubt, fuel controversy, and delay changes to policy.
Journalists beware . . . if you want to do a honest job reporting on a study you must read more than the abstract and the press package provided.