Right, I have always been among those making this case. But McKernan's experiment is very valuable because it wasn't clear whether maybe they had improved things by now (i.e. there's no way that product was consistent in the initial roll-out). Apparently no.
I don't think this is corrected for in Figure 2: Vaccine effectiveness against coronavirus (COVID-19) hospitalisation and death involving COVID-19, by age group
If you ask me that is a considerable Healthy User Bias!?
Most interesting is perhaps jump in "vaccine effectiveness" against non-Covid mortality between 2nd and 3rd dose. Note ~20% of double-dosed did not receive 3rd dose.
I'm just now really getting to this report. There are so many things to dislike about the design!
With the data they had at hand they should have attempted a matched retrospective cohort so the unvaccinated start being at risk on the same dates as the vaxxed enter different buckets. Instead, it's hazard ratio from March 2021 for the unvaccinated, which just sets them up for the Delta wave firing squad. *There is no way that they are retrospectively not going to have a big wave of death 3 months after they start being at risk.* This includes deaths from the virus that don't get coded as "Covid-19." And even if the big wave is 3 months in, March itself is already riding a wave from post-Alpha deaths so the shorter buckets still get a handicap especially since the dead can't get vaxxed (i.e. all short buckets are just showing survivorship bias).
It's all meaningless. I guess it's making me rethink my endorsement of Horne, et al. since it might have the same problems. *edit: nvm Horne et al can't have that problem because it deliberately lines the start dates up for each age group.
But "not on deathbed bias" and "healthy user bias" are more like cousins than identical twins. You can get a huge reduction from "baseline" death rate in any intervention if you simply don't apply intervention to people you think will die soon, and likewise a huge increase from baseline in the non-intervened. But that wouldn't exclude other reasons to forgo the intervention.
So the rate of all cause death isn't necessarily measuring "health" of the users because, due to removing and/or baking-in predicted deaths, all cause death no longer tells you anything about the *survivors* in each group.
It's therefore not necessarily damning that deathbed bias isn't corrected for when calculating severe efficacy. The problem with the virus is not exclusively that it is killing those who would die anyway - what you really care about is the people it kills who wouldn't have died anyway, and whether the intervention reduces this. What I would point out about Figure 2 is that the 18-39 group looks exactly like what I would predict for any young age group - there is no severe efficacy because severe outcomes are too rare to protect against, so instead you just have "putting off the inevitable" which is strictly driven by temporary infection efficacy. This is why you go negative when only measuring performance after 3 months, because this group is catching up with infection.
But in older groups, despite the fact that catching up is happening, there still seems to be efficacy in the > 3 months bucket. The only problem here is that you don't know if this is an artifact of deathbed bias again - i.e., you want to measure protection performance of the people who weren't going to die anyway, but you don't have a control group for that.
Subtracting the "fully adjusted" values in Figure 3 (effectiveness against non-Covid mortality) from those in Figure 2 gives the following "corrected" vaccine effectiveness against COVID-19 mortality:
Eh... I wouldn't really want to do anything without the raw numbers. That's why I like the other ONS tables, you get deaths and person years and can ignore the adjusted stuff. I don't endorse the vaccine report (they even flubbed the 1 day < 21 group, as if to intentionally discredit the other table!)
At least this part was news to me: (speaking of one FOI'd source)
"Note that every death up until 21 June 2021 was recorded as unvaccinated simply because hospitals in this Trust were using the NIMS system for classifying deaths which was not up and running until then"
I have no critique to really offer there. All data on vaxxed severe illness / hospitalization ratios that combines early 2021 with later periods is pretty much nonsense, because the vaxxed were not at risk (due to mostly not existing yet) in early 2021.
Does this mean the conclusion - vax reduces severe disease - is incorrect? No, because the reverse of bad data is not automatically true. And yes, almost all the data is bad. This doesn't have to do with manipulation so much as biases and accidents of timing (i.e. unvaxxed met the virus in Delta form, vaxxed mostly in Omicron; or, when you roll out the initial doses or boosters in the middle of waves, you create survivorship bias where all severe outcomes happen to the uninjected).
Maybe the Israel dashboard was the only good source on severe efficacy in the end (i.e. maybe it was good, but who really knows if it was; it was my source at the time because it was simultaneously showing zero infection efficacy). But I'll note again that Kirsch's own readers' anecdotes matched the party line, despite the same readers being part of this ecosystem saying the party line was a lie for over a year https://unglossed.substack.com/p/the-american-unvaccinated-holocaust
There is a key point you have missed about my ghost analysis. (Ghost = not in ONS sample but in NIMS). The analysis focuses on deaths in the vaccinated population. The unvaccinated denominator does not detract from the findings of this analysis.
Yes, the difference in mortality rates for the vaccinated in ONS and NIMS is not huge. The issue is that if deaths are being misclassified when there is a low percentage of mismatching to the vaccine database, then deaths which were in fact in vaccinated people end up being classified as deaths in the unvaccinated. A tiny proportion of misclassified deaths in the vaccinated can have a huge impact on the overall unvaccinated mortality because of the disproportionate sizes of those two populations.
The reason that the ghost rates matter is not because of the denominator it is because they expose the problem with the *numerator*. Analysis of all cause deaths showed the vaccinated ghosts had a very high mortality rate. https://drclarecraig.substack.com/p/deaths-among-the-ghost-population
How can the same population have one of the highest, or the highest mortality rate compared to other groups for most conditions and yet the lowest for covid? It makes no sense and certainly doesn't look like a human induced bias.
The ONS said (for 2011) they had a ~5% mismatch rate where people who were on the census did not have a record on the NIMS database that they could match to. There are all sorts of errrors, ambiguity or out of date information that could lead to this problem. If we assume a similar problem for matching death certificates to NIMS then deaths could be misclassified. The ONS said that any failure to match a death certificate to a vaccine record resulted in the assumption that the death had been in an unvaccinated person. (Note the ONS make no mention of deaths that were not matched to a NIMS record - because if they did not match they were assumed to be unvaccinated).
For a death to be classed as 'in the vaccinated' (in table 5) requires a successful match between the death certificate and a vaccine record in NIMS. For a death to be classed as unvaccinated requires no match. Therefore, any deaths in the vaccinated with different details to their NIMS entry would be misclassified as unvaccinated. Imagine a death of a vaccinated person who was in the ONS sample (in table 2). Let's say their NIMS entry used a nickname or had an error in it. The death certificate would not match to a vaccine record but it could still match to the census data. They would then be included as a death in an unvaccinated person.
Any bias this creates should be the same for the vaccinated sample in the ONS sample and those outside of it and for covid as well as all cause deaths. There must therefore be an additional issue to explain the high all cause ghost vaccinated mortality and the low covid ghost vaccinated mortality.
The big difference between covid and all cause deaths is the proportion that occur in hospital (71% vs 44%). It is fair to assume that deaths in hospital are more likely to have had their NHS record cleaned up and for those certifying to produce a certificate that is identical to the NHS record. Any cleaning up of the NHS record would automatically result in a more accurate NIMS record as they are related.
If we make very plausible assumptions that there is a difference in hospitalisation rates for the vaccinated and unvaccinated then the mortality rate anomalies can be replicated almost entirely. The assumptions are simply that
a) around 5% of death certificates do not match perfectly to the NIMS database for deaths outside of hospital
b) this falls to only 4% for deaths in hospital
c) deaths outside of hospital are slightly more likely to be vaccinated
The latter is fair given that 21% of deaths outside hospital are in care homes where it is virtually compulsory to be vaccinated.
The only difference in mortality rates between groups that this model cannot reproduce is the finding of a higher mortality rate in the vaccinated ghosts than the unvaccinated population in some groups. There must be another explanation for that.
I whole heartedly agree with your call to ask for more than just data.
Here's what I plan to ask from the ONS:
1. A reproduction of their calculations after accounting for a potential 5% error rate in matching of death certificates in the vaccinated i.e. assuming that an equivalent proportion of the unmatched were in fact vaccinated not assuming they were all unvaccinated.
2. The proportion of hospital deaths that were matched as vaccinated compared to the proportion of non-hospital deaths.
3. An estimate of the death certificate mismatch rate. For example, a sample of death certificates could be manually matched to NIMS to see the difference between more generous matching criteria and their automated matching.
Any suggestions from anyone as to other pertinent questions would be welcome.
Thanks - it will take a while to have a worthwhile response to such a detailed comment.
Off-hand it wouldn't seem surprising if many different primary causes of death influence the "ghost" death rate, perhaps due to speed of turnaround or perhaps likelihood of hospitalization-associated record cleanup. This might be more an artifact of how causes are "tagged" to deaths in the invisible ghost bucket rather than how a cause of death influences likelihood of being in PHDA/ONS. Once someone is in, they are in. Likewise, regarding your first post, "The difference between the population estimates really matters because people who die turn up in records but when people are alive they can remain hidden from the records." wouldn't seem to be a problem as long as no one can actually move in and out of PHDA.
This remains my understanding of the design, and I don't see anything in the raw numbers that suggests otherwise (eg no unvaxxed age groups reach a point where deaths stop dwindling due to a constant dark matter death pool).
A similar off-hand thought is that I see where a problem could exist with "excluded vaxxed more likely to die, some portion of excluded vaxxed deaths not in fact excluded but included without vax status (and here the small N would no longer be trivial because they are landing in a lower py denominator)," but this doesn't lead to easy predictions of what we would see in excluded vax death rates; i.e. should they be higher (because dying more) or lower (because actually being included as unvaxxed). Without a clear prediction, the higher excluded vax death rates can't be considered a signal for a problem here. Nor would such a vaxxed death leak necessarily outweigh biases against recording unvaxxed deaths (just because the death rate is high doesn't mean there are zero immortal unvaxxed in PHDA/ONS).
Hopefully I can reply in a less back of napkin fashion soon.
Your response remains very focused on the denominator which is not the issue I am really concerned about here. It's mistakes in the numerator that are causing the problem. Misallocation of a fraction of deaths would not result in a "constant dark matter death pool" because it will always be in proportion to the overall deaths.
Nice. Once again you provide an important balancing contribution to the debate. Rigorous peer review and scrutiny are what good (citizen) science demands. I'll keep reading both sides of "our side" and theirs. Thanks, Brian!
Yup. Healthy User Bias explains it. Fact is, there are far too many non-covid excess deaths two years into the vaccination program. While the ONS study compares deaths to unvaxxed, It would also be useful to compare death rate of vaxxed to the pre-COVID baseline. While this also would suffer from a HUB, a higher rate in the vaccinated compared to baseline would be a conservative indicator the vaccinated death rate is elevated.
Then of course, still to be answered is the reason “Long Covid “ or the Vax , or a synergistic combination
This is why I think examining the relationship between "catch-up effect" and the pivots in vaxxed ACM are illuminating. So, yes, probably an effect of post-breakthrough virus harms, either sudden cardiac death from the virus, or just under-diagnosed acute pneumonia / ARDS i.e. Covid-19.
Given the complexity of the manufacturing process for the gene-therapy treatment for SARS-CoV-2 it seems the likelihood they didn't fuck up is low:
https://anandamide.substack.com/p/pfizer-and-moderna-bivalent-vaccines
Right, I have always been among those making this case. But McKernan's experiment is very valuable because it wasn't clear whether maybe they had improved things by now (i.e. there's no way that product was consistent in the initial roll-out). Apparently no.
Hey Brian, check out Figure 3 from latest ONS drop:
Figure 3: Reduction in risk of non-COVID-19 death by vaccination status shows the importance of adjusting for health-related factors
Reduction in risk of non-coronavirus (COVID-19) death by vaccination status compared with unvaccinated, England, 21 March 2021 to 20 March 2022 https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/bulletins/covid19vaccineeffectivenessestimatedusingcensus2021variablesengland/31march2021to20march2022
I don't think this is corrected for in Figure 2: Vaccine effectiveness against coronavirus (COVID-19) hospitalisation and death involving COVID-19, by age group
If you ask me that is a considerable Healthy User Bias!?
Most interesting is perhaps jump in "vaccine effectiveness" against non-Covid mortality between 2nd and 3rd dose. Note ~20% of double-dosed did not receive 3rd dose.
I'm just now really getting to this report. There are so many things to dislike about the design!
With the data they had at hand they should have attempted a matched retrospective cohort so the unvaccinated start being at risk on the same dates as the vaxxed enter different buckets. Instead, it's hazard ratio from March 2021 for the unvaccinated, which just sets them up for the Delta wave firing squad. *There is no way that they are retrospectively not going to have a big wave of death 3 months after they start being at risk.* This includes deaths from the virus that don't get coded as "Covid-19." And even if the big wave is 3 months in, March itself is already riding a wave from post-Alpha deaths so the shorter buckets still get a handicap especially since the dead can't get vaxxed (i.e. all short buckets are just showing survivorship bias).
It's all meaningless. I guess it's making me rethink my endorsement of Horne, et al. since it might have the same problems. *edit: nvm Horne et al can't have that problem because it deliberately lines the start dates up for each age group.
But "not on deathbed bias" and "healthy user bias" are more like cousins than identical twins. You can get a huge reduction from "baseline" death rate in any intervention if you simply don't apply intervention to people you think will die soon, and likewise a huge increase from baseline in the non-intervened. But that wouldn't exclude other reasons to forgo the intervention.
So the rate of all cause death isn't necessarily measuring "health" of the users because, due to removing and/or baking-in predicted deaths, all cause death no longer tells you anything about the *survivors* in each group.
It's therefore not necessarily damning that deathbed bias isn't corrected for when calculating severe efficacy. The problem with the virus is not exclusively that it is killing those who would die anyway - what you really care about is the people it kills who wouldn't have died anyway, and whether the intervention reduces this. What I would point out about Figure 2 is that the 18-39 group looks exactly like what I would predict for any young age group - there is no severe efficacy because severe outcomes are too rare to protect against, so instead you just have "putting off the inevitable" which is strictly driven by temporary infection efficacy. This is why you go negative when only measuring performance after 3 months, because this group is catching up with infection.
But in older groups, despite the fact that catching up is happening, there still seems to be efficacy in the > 3 months bucket. The only problem here is that you don't know if this is an artifact of deathbed bias again - i.e., you want to measure protection performance of the people who weren't going to die anyway, but you don't have a control group for that.
Does this approach make sense?:
Subtracting the "fully adjusted" values in Figure 3 (effectiveness against non-Covid mortality) from those in Figure 2 gives the following "corrected" vaccine effectiveness against COVID-19 mortality:
********
UPDATE: The approach above is wrong!
Northwoods describes the correct approach below:
Eh... I wouldn't really want to do anything without the raw numbers. That's why I like the other ONS tables, you get deaths and person years and can ignore the adjusted stuff. I don't endorse the vaccine report (they even flubbed the 1 day < 21 group, as if to intentionally discredit the other table!)
Thanks, you're right!
What says ye to it?
https://wherearethenumbers.substack.com/p/claims-the-unvaccinated-were-at-higher
At least this part was news to me: (speaking of one FOI'd source)
"Note that every death up until 21 June 2021 was recorded as unvaccinated simply because hospitals in this Trust were using the NIMS system for classifying deaths which was not up and running until then"
I have no critique to really offer there. All data on vaxxed severe illness / hospitalization ratios that combines early 2021 with later periods is pretty much nonsense, because the vaxxed were not at risk (due to mostly not existing yet) in early 2021.
Does this mean the conclusion - vax reduces severe disease - is incorrect? No, because the reverse of bad data is not automatically true. And yes, almost all the data is bad. This doesn't have to do with manipulation so much as biases and accidents of timing (i.e. unvaxxed met the virus in Delta form, vaxxed mostly in Omicron; or, when you roll out the initial doses or boosters in the middle of waves, you create survivorship bias where all severe outcomes happen to the uninjected).
Maybe the Israel dashboard was the only good source on severe efficacy in the end (i.e. maybe it was good, but who really knows if it was; it was my source at the time because it was simultaneously showing zero infection efficacy). But I'll note again that Kirsch's own readers' anecdotes matched the party line, despite the same readers being part of this ecosystem saying the party line was a lie for over a year https://unglossed.substack.com/p/the-american-unvaccinated-holocaust
Thank you Brian - an enjoyable read.
There is a key point you have missed about my ghost analysis. (Ghost = not in ONS sample but in NIMS). The analysis focuses on deaths in the vaccinated population. The unvaccinated denominator does not detract from the findings of this analysis.
Yes, the difference in mortality rates for the vaccinated in ONS and NIMS is not huge. The issue is that if deaths are being misclassified when there is a low percentage of mismatching to the vaccine database, then deaths which were in fact in vaccinated people end up being classified as deaths in the unvaccinated. A tiny proportion of misclassified deaths in the vaccinated can have a huge impact on the overall unvaccinated mortality because of the disproportionate sizes of those two populations.
The reason that the ghost rates matter is not because of the denominator it is because they expose the problem with the *numerator*. Analysis of all cause deaths showed the vaccinated ghosts had a very high mortality rate. https://drclarecraig.substack.com/p/deaths-among-the-ghost-population
However, the opposite was true for covid deaths. https://drclarecraig.substack.com/p/how-many-ghosts-die-with-covid
How can the same population have one of the highest, or the highest mortality rate compared to other groups for most conditions and yet the lowest for covid? It makes no sense and certainly doesn't look like a human induced bias.
The ONS said (for 2011) they had a ~5% mismatch rate where people who were on the census did not have a record on the NIMS database that they could match to. There are all sorts of errrors, ambiguity or out of date information that could lead to this problem. If we assume a similar problem for matching death certificates to NIMS then deaths could be misclassified. The ONS said that any failure to match a death certificate to a vaccine record resulted in the assumption that the death had been in an unvaccinated person. (Note the ONS make no mention of deaths that were not matched to a NIMS record - because if they did not match they were assumed to be unvaccinated).
For a death to be classed as 'in the vaccinated' (in table 5) requires a successful match between the death certificate and a vaccine record in NIMS. For a death to be classed as unvaccinated requires no match. Therefore, any deaths in the vaccinated with different details to their NIMS entry would be misclassified as unvaccinated. Imagine a death of a vaccinated person who was in the ONS sample (in table 2). Let's say their NIMS entry used a nickname or had an error in it. The death certificate would not match to a vaccine record but it could still match to the census data. They would then be included as a death in an unvaccinated person.
Any bias this creates should be the same for the vaccinated sample in the ONS sample and those outside of it and for covid as well as all cause deaths. There must therefore be an additional issue to explain the high all cause ghost vaccinated mortality and the low covid ghost vaccinated mortality.
The big difference between covid and all cause deaths is the proportion that occur in hospital (71% vs 44%). It is fair to assume that deaths in hospital are more likely to have had their NHS record cleaned up and for those certifying to produce a certificate that is identical to the NHS record. Any cleaning up of the NHS record would automatically result in a more accurate NIMS record as they are related.
If we make very plausible assumptions that there is a difference in hospitalisation rates for the vaccinated and unvaccinated then the mortality rate anomalies can be replicated almost entirely. The assumptions are simply that
a) around 5% of death certificates do not match perfectly to the NIMS database for deaths outside of hospital
b) this falls to only 4% for deaths in hospital
c) deaths outside of hospital are slightly more likely to be vaccinated
https://drclarecraig.substack.com/p/the-ons-have-a-faulty-sorting-hat
The latter is fair given that 21% of deaths outside hospital are in care homes where it is virtually compulsory to be vaccinated.
The only difference in mortality rates between groups that this model cannot reproduce is the finding of a higher mortality rate in the vaccinated ghosts than the unvaccinated population in some groups. There must be another explanation for that.
I whole heartedly agree with your call to ask for more than just data.
Here's what I plan to ask from the ONS:
1. A reproduction of their calculations after accounting for a potential 5% error rate in matching of death certificates in the vaccinated i.e. assuming that an equivalent proportion of the unmatched were in fact vaccinated not assuming they were all unvaccinated.
2. The proportion of hospital deaths that were matched as vaccinated compared to the proportion of non-hospital deaths.
3. An estimate of the death certificate mismatch rate. For example, a sample of death certificates could be manually matched to NIMS to see the difference between more generous matching criteria and their automated matching.
Any suggestions from anyone as to other pertinent questions would be welcome.
Thanks - it will take a while to have a worthwhile response to such a detailed comment.
Off-hand it wouldn't seem surprising if many different primary causes of death influence the "ghost" death rate, perhaps due to speed of turnaround or perhaps likelihood of hospitalization-associated record cleanup. This might be more an artifact of how causes are "tagged" to deaths in the invisible ghost bucket rather than how a cause of death influences likelihood of being in PHDA/ONS. Once someone is in, they are in. Likewise, regarding your first post, "The difference between the population estimates really matters because people who die turn up in records but when people are alive they can remain hidden from the records." wouldn't seem to be a problem as long as no one can actually move in and out of PHDA.
This remains my understanding of the design, and I don't see anything in the raw numbers that suggests otherwise (eg no unvaxxed age groups reach a point where deaths stop dwindling due to a constant dark matter death pool).
A similar off-hand thought is that I see where a problem could exist with "excluded vaxxed more likely to die, some portion of excluded vaxxed deaths not in fact excluded but included without vax status (and here the small N would no longer be trivial because they are landing in a lower py denominator)," but this doesn't lead to easy predictions of what we would see in excluded vax death rates; i.e. should they be higher (because dying more) or lower (because actually being included as unvaxxed). Without a clear prediction, the higher excluded vax death rates can't be considered a signal for a problem here. Nor would such a vaxxed death leak necessarily outweigh biases against recording unvaxxed deaths (just because the death rate is high doesn't mean there are zero immortal unvaxxed in PHDA/ONS).
Hopefully I can reply in a less back of napkin fashion soon.
Thanks Brian.
Your response remains very focused on the denominator which is not the issue I am really concerned about here. It's mistakes in the numerator that are causing the problem. Misallocation of a fraction of deaths would not result in a "constant dark matter death pool" because it will always be in proportion to the overall deaths.
I'll let you mull a bit more.
Nice. Once again you provide an important balancing contribution to the debate. Rigorous peer review and scrutiny are what good (citizen) science demands. I'll keep reading both sides of "our side" and theirs. Thanks, Brian!
Thank you for the kind words!
Yup. Healthy User Bias explains it. Fact is, there are far too many non-covid excess deaths two years into the vaccination program. While the ONS study compares deaths to unvaxxed, It would also be useful to compare death rate of vaxxed to the pre-COVID baseline. While this also would suffer from a HUB, a higher rate in the vaccinated compared to baseline would be a conservative indicator the vaccinated death rate is elevated.
Then of course, still to be answered is the reason “Long Covid “ or the Vax , or a synergistic combination
This is why I think examining the relationship between "catch-up effect" and the pivots in vaxxed ACM are illuminating. So, yes, probably an effect of post-breakthrough virus harms, either sudden cardiac death from the virus, or just under-diagnosed acute pneumonia / ARDS i.e. Covid-19.