Thank you for a great post. This tenacity and attention to detail is why I read all your posts. (and Brian's)
I totally agree on going beyond the abstract. I find a significant number of abstracts to be purposely misleading. This might be because of the antivax nature of my interests, as studies that objectively demonstrate anti-covid-vax results might state a pro-vax conclusion that allows them to be published.
I usually read the title, abstract, figures and usually the whole article. If the article mentions vaccinated vs unvaccinated control group my interest is piqued and I always look at all data tables.
There were a couple of articles that contained data that allowed some unintended (by authors) conclusions and such things are always newsworthy.
I also often struggle with graphs or figures, which use abbreviations that I cannot understand, or do not explain what they are actually displaying etc. I am glad that I am not alone. I thought I was.
I do suffer from attention deficit and am constantly distracted by people and that is terrible for attentive article reading, so I struggle in this department but try to at least make sure I understood what the article is saying. Sometimes I feel to be intentionally being confused.
This is where I am grateful to people who dig extra deeply into articles, like you do. It is very refreshing.
The M&M section is something that I usually ignore due to not having lab knowledge.
Abstracts have severe character/word limits which leads to very precise wording which can often be misleading (see Brian’s example in his comment). I personally want to understand the materials/methods and I want to compare the data to the purported conclusions. Ask yourself very basic questions when going through M&M: why are they using these cells, why are they measuring this signal and what is the timeframe being used for data collection, what are they using as controls (do using these controls make sense ... what else could be going on).
Also be aware that a lot of the statistical work is outsourced so there can be some misunderstanding between researchers & statisticians.
But yes, don’t be afraid to give it a go - we all have to start somewhere! Honestly I’m amazed at some of the stupid things I’ve been asked to do in the lab, umm... this is not proving your point!!
What I would say on "Methods" is that they are indispensable if human subjects are involved. There is often no other part of the paper that actually reveals "what the study is showing." So it's different than a purely molecular study where methods is usually just a list of specialist techniques, and often a black box that prevents identifying whatever the authors might have done to distort their own results.
"Title → Abstract → Introduction → Discussion → Results → Methods" well, oops, haha. I only start with abstract if it's an old paper and I want to rule out actually reading it. Otherwise straight to methods or supplemental, then discussion (as a better version of the abstract), and then Results. Hate introductions with my life.
The retracted Efimenko paper asserts standard tropes I've seen many times at the FLCCC: "We are really concerned about this problem because the patients may start taking or demanding this medication from their physicians, which can potentially be harmful." Nonsense. Billions of doses of the drug have been taken, with a safety profile better than aspirin.
"This misrepresentation of the study may lead to a huge public health problem, since Ivermectin is a medication that is not FDA approved for COVID treatment, and currently has proven to be ineffective in clinical trials, which are truly the gold standard to evaluate the efficacy of a medication." Again, nonsense. This is all part of the effort to scare practitioners and patients, to centralize the control of medical practice with federal authorities. In a pandemic, a legitimate doctor uses their experience and training to figure out treatments. Observational trials have a roughly equivalent value as RCTs under the circumstances, and studies have shown the results tend to be in the same direction as RCTs. This is Pharma propaganda.
I remember reading an article not long ago where the abstract was completely misleading when one got to the discussion and conclusion. Of course, people were quoting the abstract which if I remember correctly supported the injections but the abstract was completely at odds with the results and conclusions. I wasn’t the only one who noted this which I appreciated. I thought I was hallucinating or just plain stupid.
One recommendation I have for the layperson wanting to read studies and research papers is to build up fluency and familiarity with statistics and statistical methods. Becoming comfortable with things like confidence intervals goes a long way towards being more comfortable in critically thinking about the material.
The original COVID inoculation trial write-ups are an excellent example. Pfizer's 95% efficacy for the original monovalent shot was predicated on an incidence of infection in the control group that was not only statistically insignificant within the study population (<200 out of ~17,000), but was a fraction of the prevailing infection rate in the country overall -- going by the numbers even back then one of the best ways to avoid COVID was to be in the control group of a Pfizer study.
The better one is with statistics the more accessible the details become, in my view.
Thank you for a great post. This tenacity and attention to detail is why I read all your posts. (and Brian's)
I totally agree on going beyond the abstract. I find a significant number of abstracts to be purposely misleading. This might be because of the antivax nature of my interests, as studies that objectively demonstrate anti-covid-vax results might state a pro-vax conclusion that allows them to be published.
I usually read the title, abstract, figures and usually the whole article. If the article mentions vaccinated vs unvaccinated control group my interest is piqued and I always look at all data tables.
There were a couple of articles that contained data that allowed some unintended (by authors) conclusions and such things are always newsworthy.
I also often struggle with graphs or figures, which use abbreviations that I cannot understand, or do not explain what they are actually displaying etc. I am glad that I am not alone. I thought I was.
I do suffer from attention deficit and am constantly distracted by people and that is terrible for attentive article reading, so I struggle in this department but try to at least make sure I understood what the article is saying. Sometimes I feel to be intentionally being confused.
This is where I am grateful to people who dig extra deeply into articles, like you do. It is very refreshing.
The M&M section is something that I usually ignore due to not having lab knowledge.
Thank you for an amazing post
Great post. I would add to this the importance of the supplementary indexes where sometimes key results are hidden w/o comment.
Also Conflict of Interest and Funding sections
There is a lot. That’s why discussion is so very important.
Abstracts have severe character/word limits which leads to very precise wording which can often be misleading (see Brian’s example in his comment). I personally want to understand the materials/methods and I want to compare the data to the purported conclusions. Ask yourself very basic questions when going through M&M: why are they using these cells, why are they measuring this signal and what is the timeframe being used for data collection, what are they using as controls (do using these controls make sense ... what else could be going on).
Also be aware that a lot of the statistical work is outsourced so there can be some misunderstanding between researchers & statisticians.
But yes, don’t be afraid to give it a go - we all have to start somewhere! Honestly I’m amazed at some of the stupid things I’ve been asked to do in the lab, umm... this is not proving your point!!
What I would say on "Methods" is that they are indispensable if human subjects are involved. There is often no other part of the paper that actually reveals "what the study is showing." So it's different than a purely molecular study where methods is usually just a list of specialist techniques, and often a black box that prevents identifying whatever the authors might have done to distort their own results.
"Title → Abstract → Introduction → Discussion → Results → Methods" well, oops, haha. I only start with abstract if it's an old paper and I want to rule out actually reading it. Otherwise straight to methods or supplemental, then discussion (as a better version of the abstract), and then Results. Hate introductions with my life.
The retracted Efimenko paper asserts standard tropes I've seen many times at the FLCCC: "We are really concerned about this problem because the patients may start taking or demanding this medication from their physicians, which can potentially be harmful." Nonsense. Billions of doses of the drug have been taken, with a safety profile better than aspirin.
"This misrepresentation of the study may lead to a huge public health problem, since Ivermectin is a medication that is not FDA approved for COVID treatment, and currently has proven to be ineffective in clinical trials, which are truly the gold standard to evaluate the efficacy of a medication." Again, nonsense. This is all part of the effort to scare practitioners and patients, to centralize the control of medical practice with federal authorities. In a pandemic, a legitimate doctor uses their experience and training to figure out treatments. Observational trials have a roughly equivalent value as RCTs under the circumstances, and studies have shown the results tend to be in the same direction as RCTs. This is Pharma propaganda.
I remember reading an article not long ago where the abstract was completely misleading when one got to the discussion and conclusion. Of course, people were quoting the abstract which if I remember correctly supported the injections but the abstract was completely at odds with the results and conclusions. I wasn’t the only one who noted this which I appreciated. I thought I was hallucinating or just plain stupid.
https://igorchudov.substack.com/p/jama-vaccine-shedding-in-breast-milk
One recommendation I have for the layperson wanting to read studies and research papers is to build up fluency and familiarity with statistics and statistical methods. Becoming comfortable with things like confidence intervals goes a long way towards being more comfortable in critically thinking about the material.
The original COVID inoculation trial write-ups are an excellent example. Pfizer's 95% efficacy for the original monovalent shot was predicated on an incidence of infection in the control group that was not only statistically insignificant within the study population (<200 out of ~17,000), but was a fraction of the prevailing infection rate in the country overall -- going by the numbers even back then one of the best ways to avoid COVID was to be in the control group of a Pfizer study.
The better one is with statistics the more accessible the details become, in my view.