The Protective Effect
A new study purports to show that Covid vaccines have “Long Covid efficacy.” It has “some” limitations.
A study trawling the TriNetX record set finds that the Covid vaccinated who were PCR positive or diagnosed with Covid-19 between September 2020 and September 2021 were less likely to have new-onset illnesses in the 3 months afterward (implying, let’s say, 40 - 50% Long Covid efficacy).
This may be evidence (just about the first) of Long Covid efficacy, or it may be an artifact of the study design (a poorly controlled observational trawl through health records, with no breakdown of rates for low-comorbidity patients or women vs men), or of a biological paradox in Covid vaccine adverse events, or all three.
The study gives the reader little to work with in deciphering which one is the case.
The headline results
Comorbidity-matched (generally high-comorbidity) Covid-vaccinated patients had lower rates of new-onset illnesses and symptoms in the 28 (1-28) and 90 days (as in between 29 - 90, in my reading) after PCR-positive or diagnosed Covid-19:
Since I like raw numbers, I have highlighted the raw numbers. (If you imagine moving the decimal one to the left in the parenthesized values, this reflects the percentage, e.g. 4.96% and 9.29% for respiratory symptoms.)
Note that the bigger relative risk reductions in the “90 day” (29-90?) reflect lower absolute rates but come out to about the same number of apparent prevented events per 1,000 in both time windows. Additionally, since all these rates are baseline-inclusive, the apparent relative risk reduction and percentage of events prevented as a result of SARS-CoV-2 would ostensibly be even greater than calculated, for all diagnoses. It could be 100% for all we know.
Notably, this improvement carries over not just for the more high-stakes diagnoses that top the list, which might reflect generic severe efficacy (less spread of the virus to organs like the heart or kidneys, to offer a simplistic theory), but to the lifestyle-impairing symptoms that are more commonly associated with Long Covid.
Thus, this study appears to be breaking ground in demonstrating that the Covid vaccines are effective in reducing experiences of Long Covid symptoms after infection.
Limitations Galore
If you anticipate deriving value from this study analysis, please drop a few coins in your fact-barista’s tip jar.
The dataset
TriNetX is a big set of electronic health records. These records are generated when any of about 70 million patients show up to any of 57 US healthcare centers within the network. Some of these records are PCR tests for SARS-CoV-2 which turn out to be positive. Some of these records are for receipt of Covid vaccines. Others, obviously, for everything else under the sun. Researchers can query TriNetX for these records, and use them to make studies about what happens when patients have positive PCRs, or “presumed Covid,” as well as diagnosed Covid-19.
For whatever reason, studies using TriNetX don’t drive a lot of headlines, even though it might offer a better window into “normal Covid outcomes” than datasets in which patients are older (the VA) or which potentially over-weight California and New York (Kaiser-Permanente, or the CDC’s network). TriNetX and the also-not-oft-used N3C might be more representative of “normal” outcomes than these. But it’s hard to say. Ultimately they’re just a big collection of records.
For any discrete record, say “new diagnosis of hypertension,” the question of “is this individual more likely to come seek a diagnosis” is unknowable. Likewise for PCR-positivity. Different individuals might be less likely to seek a test, or interact with the specific healthcare centers in the dataset for the same. Likewise for receipt of Covid-vaccines. For this single, simple reason, any “conclusions” using the TriNetX data set are fundamentally inconclusive. They can be evidence in favor of a conclusion, but they do not shut the book.
Low hits for “breakthrough” infection
Something is off, for example, with the amount of hits for double-Covid-vaccination 7 or more days before PCR positive in the results obtained for the present study.
Despite trawling every PCR positive in TriNetX between mid-September 2020 and 2021, only 22,225 were found 7 or more days after a record of a “full” Covid vaccine course. This is even though the dataset had records for “at least one” Covid vaccine for 300,117 patients in March, midway through the study’s chosen window (already a likely undercount). If I were the authors, I would have presumed that Covid-vaccination is being under-recorded (lots of Covid-vaccinated in the “unvaccinated” PCR-positive set) and called off the study.
I wasn’t the authors. They pressed on. They note, innocently, that likely under-recording of Covid-vaccination should mean that any of their observed differences in outcomes are also understated. I disagree. There are likely two groups of Covid vaccinated patients in the dataset - those likely to have their injection recorded in the set, and those unlikely - and we have no idea what drives this difference in likelihood, and if it carries an increased rate of post-PCR-positive illnesses as baggage. What if likelihood for being both Covid-vaccinated and PCR-positive corresponds with a higher rate of previous infection, whether recorded in TriNetX or not (the authors do not even seem to mention if prior infections were taken into consideration for either group)?
Ultimately the authors are evaluating outcomes for a biased set (not just Covid-vaccinated, but recorded as Covid-vaccinated and infected). An uncharacteristically unhealthy bias.
Unhealthy user bias
The extent to which Covid-vaccinated Americans are likely to be healthier than the unvaccinated is unclear, and likely varies by geography. In California, for instance, the reporting concerning elderly, heavily-immigrant, unvaccinated admissions for Covid-19 in inland counties strongly supports a healthy user bias.
Such a bias could drive differences in rates of severe outcomes, implying that the apparent severe efficacy of the Covid vaccines is an illusion. I would certainly grant that to be the case in California or any datasets that depend heavily on it. But in the TriNetX database, the Covid-vaccinated and infected are older and more co-morbid vs the relatively young “unvaccinated” patient set (“partially vaccinated” should have been excluded, per the Supplementary data description). Covid-vaccinated infected feature an incredible 47% preexisting hypertension rate, 23% diabetes, and 38% history of cancer or other neoplasms.
This unusual skew might be the result of trends in the geographic areas covered by the network, but again strongly suggests that “likely to be recorded as Covid-vaccinated and infected” requires a wheel-cart and a butler to get its bias-baggage out of the hotel.
Naturally, the authors have to correct for this by some means in order to fulfill their should-have-been-called-off attempt to measure the impact of the Covid vaccines, rather than of the higher comorbidities, on long-term outcomes of infection. But this is actually impossible. The thing they need to correct for - the bias - is the thing they are trying to measure the effect of. Still, they use all the comorbidities as a proxy, fishing through the much larger “unvaccinated” infected set to find 25,225 matched “controls.”
No breakdown of sub-groups
This matching creates another hazard. Do the results they find - lower rates of long term outcomes of infection for the Covid-vaccinated - hold for all risk groups, or does it only apply to the “high-overall-comorbidity” patients they are now comparing head-to-head? Fortunately, the authors aren’t idiots. They aren’t a bunch of dumb-dumbs. They know that it’s important to see if the rates of long term outcomes are only different in this high-risk group, and not for an easy-to-segment subset of lower-risk Covid-vaccinated and control patients. Duh.
No. They do not show the rates for lower risk patients.
Nor the rates by acute infection outcome (outpatient, hospitalized, or ICU), to confirm, for example, that non-hospitalized Covid vaccinated had lower rates than non-hospitalized controls (and so, the apparent lower rates only apply if you are hospitalized during infection; or vice versa). Which is essentially the most important question of all, when it comes to whether the Covid vaccines make a difference to long-term symptoms. If the answer is, “not if you weren’t at risk of severe outcomes” or “not if you weren’t hospitalized” (whether it was the Covid vaccine that kept you out of the hospital or not), then what actual good is “Long Covid efficacy”?
What about sex? Women have been found to be more at risk of Long Covid diagnosis than men, but did the apparent protection of the Covid vaccines extend to women, or was it primarily driven by a paradoxical improvement in the lower-risk sex? Wouldn’t patients want this information, if using this study to inform their healthcare decisions?1
Or how about, just for fun, run the study’s matching in reverse, excluding Covid-vaccinated infections until the demographics of this set matches the control, even if it results in a less robust sample - just to see what might have come out?
Nope.
It is as if the authors put off doing their big science project until the night before. A shoebox and some tissue paper isn’t going to win the class prize. But really, this lack of granularity seems intentional. “Look, everyone, this experimental injection must be protecting some of you - so why don’t you go on and take a bet that you’ll be the lucky winner!”
But I have to say, I’m actually less annoyed at the likely propagandistic nature of this design choice than the mere fact that the authors had this data, and so I could be looking at it, but can’t, because eh, they didn’t want to publish it.
A Biological Paradox? (I think unlikely)
Because the authors are interested in new-onset illnesses, they take care to exclude patients with a past history of each given diagnosis. “History,” of course, meaning before infection - which for the Covid-vaccinated, of course, occurred after Covid-vaccination.
Does this “history,” in the Covid-vaccinated group, include new diagnosis between Covid-vaccination and infection? The authors could have presented this data. They do not.
Naturally, it would be true that those who are not excluded are still experiencing lower rates of these diagnosis after infection, vs. the so-called control group. But if exposure to the spike protein tends to prompt these “new diagnosis” in individuals who are in some manner (beyond their list of generic co-morbidities) “primed” beforehand, then Covid-vaccination would act as a filter which removes these primed individuals from the Covid-vaccinated set (by prompting the relevant before-infection-diagnosis and exclusion). So even if it would seem that the authors had controlled for comorbidities, they had not controlled for this biological “priming” (controlling for this would require filtering out the “primed” by exposing them to the spike protein in advance of infection, which would require Covid-vaccinating the unvaccinated in advance, which is a paradox).
However, the reader should note that this seems like a weak theory, or at best only part of the story. The authors don’t exclude anyone in the “common Long Covid symptom” analysis, and yet the Covid-vaccinated still outperform the control - but notably, to a lesser degree:
So even if this paradox is at work, the apparent “Long Covid efficacy” here is still driving at least some of the lower rate (in this inappropriately un-granular view of things). But what exactly is this “protective effect” - the Covid vaccines, or the bias for being recorded as Covid-vaccinated and infected in the TriNetX dataset?
Fundamentally Inconclusive
As stated in the overview of the dataset, the most that can be said about this study is that it is a (likely bias-distorted) element of evidence in favor of a conclusion. It must be weighted against others. As regards “Long Covid efficacy,” the story here continues to be a mixed bag.
An overview of this bag was provided by the UKHSA earlier this year. Apart from a somewhat dispensable, survey-based study in Israel, none of the prior studies offer what I would call very strong support for the effect of Covid-vaccination on reducing long-term symptoms (and so, the TriNetX study is the first one, and it is strong but flawed).
Also unclear, is whether Omicron has altered the landscape in this regard, leading to lessened “protection” (if there really was any) or even a negative effect (via disease enhancement, or a non-paradoxical priming of spike-related illness).
Also unclear, is whether the “protection” is worth the cost, or just what the cost is, in terms of Covid-vaccine-induced long-term symptoms. Again, the authors of this paper could have measured and presented outcomes following TriNetX recorded Covid-vaccination itself; they chose not to.
And so on both these counts we are left, again, in the realm of anecdotes. The blogger who goes by “DoorlessCarp” recently highlighted a telling example, from author Rose George, in DC’s super-post about Long Covid therapeutics:3
George, 52 years old and until now an avid marathoner, takes to the Guardian to detail the months of debilitating symptoms that she has experienced since developing “Long Covid” in January of this year (when Delta infection was possible, but BA.1 Omicron was more likely).4
I ran 30 miles a week. I could turn up to a 20-mile fell race on inadequate training and run it, thoughtlessly. I did yoga, weight training and cycling. I had a low resting heart rate and strong biceps. For a 52-year-old menopausal woman, I was in extremely good shape.
But then on 3 January I fell ill with a sore throat, then flu-like weakness, a cough that hasn’t left me since, and a constant and persistent headache that is resistant to every painkiller. In the months since, I have been not ill, but not well. I have days of feeling fine, and then I don’t. As a runner, I can say that long Covid feels like the wall at mile 18 in a marathon, when suddenly your energy has gone, and you feel like a different person and you don’t know why.
Except, in George’s case, her initial illness was not accompanied by a positive test for SARS-CoV-2.
My long Covid is suspected by my GP, since I never actually tested positive, but many on the forums had only mild infections and are still suffering.
The wording, “never tested positive,” further suggests that testing occurred, though maybe well after the actual acute phase of infection (whereas for my own presumed Omicron infection in January, I would use the construction “presumed, because I did not test”).5
And so a role of SARS-CoV-2 in George’s illness isn’t even known; she may be in the smaller, historically precedented group of individuals who experience this condition for myriad other reasons, including other infections.
On the other hand - and what an “other hand” it is - she may simply be experiencing harms from the Covid vaccine she received in November, as DoorlessCarp revealed through some twitter-snooping:
This could be a direct or indirect harm. Potentially, innate immune suppression from the Covid-vaccines could make recipients vulnerable to novel infections or reversals of dormant infections that lead to the same symptoms described as Long Covid; implying that George’s injection in November left her susceptible to the onset of these symptoms resulting from a different bug. Again as highlighted by DoorlessCarp, George’s testing revealed either depletion (as in engagement against infection) or suppression of her innate immune response:
Some measure of innate suppression, leading to opportunistic infection, may also be the case for Long Covid after infection; but it would mean that the “protection” afforded by the Covid vaccines (if still valid in the Omicron era) is achieved by incurring exactly the same risk as infection. Spike protein is not safer in every random corner of body than in the respiratory tract.
For now, whether the Covid vaccines prevent or aggravate the mysterious phenomenon of Long Covid, and to what degree for either, remains an open question. Perhaps it will do so forever.
If you derived value from this post, please drop a few coins in your fact-barista’s tip jar.
Even if I, like Illich, would argue that personal risk can never really be inferred from such statistical endeavors. It’s pulling teeth to do footnotes on mobile web by the way. I sure regret not downloading the new SubstackTM app. download your SubstackTM app before it’s too late everybody.
(link anchor)
“DoorlessCarp.” “Therapeutics for Long Covid & Transfection Sequalae.” (2022, February 16 (updated May 2).) DoorlessCarp’s Scientific Literature Review.
George, Rose. “I was a marathon runner with killer biceps – long Covid has stopped me in my tracks.” (2022, May 1.) The Guardian.
See “Author has the thing post.”
Brian,
Can you clarify or confirm something for me, please.
With any multi-author scientific paper, is the author who contributed the most named last?
Very good dissection of the study design! Wow!
Could you please have a look and second-opinion this post of mine as suggested by Igor Chudov?
https://live2fightanotherday.substack.com/p/mrna-jabs-you-havent-seen-nothing