13 Comments
author

Well, I apparently didn't set up Abstract #3 Poll's well. Hopefully no one looked up the answers before giving their response! πŸ˜‰

The majority of responses for all 3 polls were correct:

1. Generated

2. Original: https://www.nejm.org/doi/10.1056/NEJMoa2212270

3. Generated

I removed the end line from the generated abstracts which included where the original article was. I also removed the line at the end of the conclusion for the original abstract which pointed to a VA clinical trial. This one was a bit more contentious since my prior article mentioned that the clinical trial links for the generated article tended to lead to dead ends, and by removing it in the original I sort of removed that context clue that could have helped.

In general, it appears that ChatGPT can't do well will numbers.

Abstract #1

Note that Abstract #1 uses Confidence Intervals but with percentages. I'm unsure about this method but it's usually the case that CI is given in odds ratios, and that's what the authors of the original list:

"... IRR of individuals who received an mRNA vaccine within 120 days from the last dose was 0.03 (95% CI, 0.03-0.04; P < .001), whereas IRR of individuals who received an adenoviral vector vaccine after 120 days was 0.21 (95% CI, 0.19-0.24; P < .001)."

It could be that ChatGPT did not understand how to process IRR, and so maybe did what it could with percent differences, although remember here that random numbers were used. Also, note here that the original abstract listed different ratios for each vaccine platform, which ChatGPT didn't and appeared to have collected under the umbrella of "vaccines".

Abstract #2

This comes from the NEJM. I took to removing the last sentence which links to a clinical trial:

"(Funded by the Veterans Affairs Cooperative Studies Program; ClinicalTrials.gov number, NCT02185417.)"

What's strange here is that ChatGPT actually gave a completely different conclusion. Where the NEJM article suggests no difference between the two drugs, ChatGPT instead argued that one was more beneficial:

"In this large, randomized, controlled trial, chlorthalidone was superior to hydrochlorothiazide in reducing the incidence of major cardiovascular events in adults with hypertension. These findings support the use of chlorthalidone as a preferred treatment option for hypertension."

Talk about actually giving out bad information! This issue is far more egregious than just making up numbers alone since the fake numbers created a wrong conclusion.

Abstract #3

For this one, the abstract came from an article in Nature. This is why the formatting is in paragraph form, so I hope no one suggested that this was fake due to this format (remember that ChatGPT was told to format the generated abstract in the style of the specified journal).

Here, the abstract constructed is actually close to the original. It's possible that the formatting of Nature helps for ChatGPT to link information together in a more cohesive manner. Take a look at the original and see for yourself if there are egregious differences:

https://www.nature.com/articles/s41591-022-02126-1

"Immune checkpoint inhibitors have shown efficacy against metastatic triple-negative breast cancer (mTNBC) but only for PD-L1positive disease. The randomized, placebo-controlled ALICE trial (NCT03164993) evaluated the addition of atezolizumab (anti-PD-L1) to immune-stimulating chemotherapy in mTNBC. Patients received pegylated liposomal doxorubicin (PLD) and low-dose cyclophosphamide in combination with atezolizumab (atezo-chemo; n = 40) or placebo (placebo-chemo; n = 28). Primary endpoints were descriptive assessment of progression-free survival in the per-protocol population (>3 atezolizumab and >2 PLD doses; n = 59) and safety in the full analysis set (FAS; all patients starting therapy; n = 68). Adverse events leading to drug discontinuation occurred in 18% of patients in the atezo-chemo arm (7/40) and in 7% of patients in the placebo-chemo arm (2/28). Improvement in progression-free survival was indicated in the atezo-chemo arm in the per-protocol population (median 4.3 months versus 3.5 months; hazard ratio (HR) = 0.57; 95% confidence interval (CI) 0.33–0.99; log-rank P = 0.047) and in the FAS (HR = 0.56; 95% CI 0.33–0.95; P = 0.033). A numerical advantage was observed for both the PD-L1positive (n = 27; HR = 0.65; 95% CI 0.27–1.54) and PD-L1negative subgroups (n = 31; HR = 0.57, 95% CI 0.27–1.21). The progression-free proportion after 15 months was 14.7% (5/34; 95% CI 6.4–30.1%) in the atezo-chemo arm versus 0% in the placebo-chemo arm. The addition of atezolizumab to PLD/cyclophosphamide was tolerable with an indication of clinical benefit, and the findings warrant further investigation of PD1/PD-L1 blockers in combination with immunomodulatory chemotherapy."

Expand full comment

I laughed at your comment about AI characters dissolving into the scenery. πŸ˜… That would be a hurdle to get over.

Thanks for linking to my stack. πŸ’• I appreciate JJ's streams and perspective; the biology really clears away some significant cobwebs over the truth of these past 3 years, and what might be coming in the future. I get to revisit my love of biology too, which I studied in college.

Expand full comment
author

You can find videos of AI George trying to sit where AI Jerry is and they just fold into one another on the couch, like an AI singularity.

I haven't looked too much into Couey's work aside from what Brian has mentioned in his Substack. I think the idea is interesting, and having information presented in such a way that you don't sacrifice much of the quality while also making it accessible to readers is a really important skill; it's certainly something I lack!

Expand full comment

>This is a huge difference than just injecting mRNA into cows (which still wouldn’t be good if it comes into practice) since you’d have to provide an explanation for how the mRNA from an injection can become packaged into exosomes in milk.

Hasn’t this already been proven with mothers who’ve been vaccinated and passed it on to their babies through breastfeeding?

Expand full comment
author

One thing is figuring out whether all of this is being done sans any cow- take milk and add in these milk exosomes, and I suppose that's the argument people are making over this topic, but again we run into territories of uncertainty in stability of these products.

Expand full comment
author

That was based off of this study:

https://jamanetwork.com/journals/jamapediatrics/fullarticle/2796427

The data was very limited and it doesn't appear other people have corroborated these findings so far. However, in these women breast milk was obtained hours-5 days after vaccination. There's also no information on what effects the picogram/mL concentrations would have on infants and the researchers are just assuming that it's "safe".

The problem is that we wouldn't know how feasible this process would be in cows- you'd probably have to vaccinate cows in close proximity to milk collection, and then you wouldn't even know how much would travel to the milk, and even at that point packaging into exosomes would seem unnecessary if the mRNA is already there- but we don't have any information on this stability in milk, or if the mRNA would even be packaged into exosomes (note that the size of the RBD-spike used in the Zhang, et al study was more than 600 bp. Size is likely to contribute on exosome formation to some degree).

Expand full comment

No matter how you look at the possibility of introducing the spike protein via injecting cows & pigs, which Bill Gates is planning to do this or next month, or somehow figuring out how to introduce it into the fruits and vegetables &/or milk, the entire idea is outrageous & no more gain of function research should ever be allowed on this. Humanity is at stake here. More than enough damage has already been done.

Expand full comment
author

One can argue that the entire situation is ridiculous while arguing the feasibility of these concepts. I wouldn't want mRNA vaccinated livestock, or these products getting into my food supply without my consent. I can also argue that there are issues going from a conceptual model to arguing that this is being placed into our milk. We can hold two ideas without them being contradictory to one another.

Expand full comment

I can’t copy and paste links into a reply so I had to make a new comment.

That’s the huge problem we have at this point. We know that roughly 1/3 of RNA and exosomes from bovines survive processing.

Whether mRna from a vax or if the cow reverse transcribes it into their dna and it’s passed through the milk and survives processing is yet to be determined.

From 2012:

Bovine milk contains microRNA and messenger RNA that are stable under degradative conditions

we concluded that the RNA in milk are highly stabilized and could be resistant to industrial processing.

We therefore investigated the miRNA and mRNA in infant formulas that are commercially available in Japan, and we found both types of RNA.

https://www.sciencedirect.com/science/article/pii/S0022030212004973

Expand full comment
author

In the link they noted that the levels found in infant formula were lower than raw milk. The problem is that a lot of the concerns are just as speculative as the actual concept. Like I said I can see a better model in them trying to add these to milk, but then that still runs into so many QA issues and we wouldn't even know if the LNPs would remain stable in that environment.

I think the article you linked provides an interesting thought in the matter, but it raises the question of translatability to the mRNA vaccines.

Expand full comment

I am glad you brought up the point able raw milk. I had been wondering about that

Expand full comment

I'm more concerned about combining mRNA with the flu shot. I plan to ask my Dr. This fall. If I can't get a straight answer I won't take the flu shot.

Expand full comment

I appreciate you further discussion on the Missouri bill. After reading the original post, I wasn’t sure what you focus was. I appreciate the clarification.

Expand full comment