95 Comments

What's your take on Fenton and Neil's look at this? https://wherearethenumbers.substack.com/p/postmodern-science-delivers-immortality

Expand full comment

If one wants to critique the methodology of the ONS tables by saying, "Here's what seems like a good way to track mortality that ends up producing a surprising and hard-to-understand result, which makes drawing conclusions difficult," that's entirely correct and fair. But that's not the same as finding a problem with the methodology per se. The ONS data is word-for-word what is being asked for by the "show us the data" lobby.

So once again I would just urge people to understand that the only substantiated or substantiate-able complaint here is with the outcome of the ONS tables. They are surprising and hard to understand, which makes drawing conclusions difficult.

Expand full comment

Thank you for taking time to reply to me, all the best.

Expand full comment

Brian, have you had a chance to check out this critique of the ONS data? https://drclarecraig.substack.com/p/deaths-among-the-ghost-population?utm_campaign=post_embed

Expand full comment

Addressed below, as you have noted

Expand full comment

I think our friendly cat therefore has it better stated than Igor and Kirsch. The feline boriquagato, doesn't claim fraud or math errors, which after your explanation neither seem present, but does correctly point out that the data doesn't seem representative (or as you say 'biased') as it deviates too much from overal population. Boriquagato itself likely correctly identifies a too low younger population and too few (young) immigrants, which I think will map fairly well with your suggestion of health biased ('too many older/sick white folks in unvaxed ONS vs general UK unvaxed population')

But nevertheless, the conclusion that the data is deeply flawed hence does seem defensible, as ONS itself claims to be representative!

Now, I agree just as during high school, getting the math answer right with wrong math, doesn't award points, but I hence do think ONS data is flawed, as it depicts wrong conclusions, because of this bias/cohort issue.

(And then there are other issues pointed out. Especially the new data based on newer census uses a different subset than the previous. This has the effect of the vaxed % rising. I likely think this is correct as in that is what the new census showed, but provides a discontinuity vs the older data that many people likely miss. And more worrisome the large efficacy against death jump from vaxed, but not boosted towards boosted in e.g. age 50-59. That indicates a strong cohort issue/bias in itself, as we know this group in itself is not really dying in large numbers from omicron.)

Expand full comment

The two recent posts are a bit of motte and bailey. One is still using imaginary denominators to create imaginary higher rates ("slam dunk"-ing). The other is hewing to how we can't "trust" the ONS data. I don't have any problem with the second message, but questions about the messenger who is still trying to have it both ways. It remains indiscernible whether egm understands why population denominators do not apply to the ONS data, or has realized it, but elected not to publish any sort of acknowledgement or correction.

Meanwhile if you go to the ONS site the first thing you see is a correction notice (mislabeled moths and sex)! https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/datasets/deathsbyvaccinationstatusengland

It was unreasonable to expect any changes in the unvaxxed biases after the update, i.e. there was never any real reason for the ONS to redesign their method in spite of all the weird signals it throws. And the update does show a signal that the vaxxed are dribing excess deaths, you just have to throw out the spurious comparison to the biased buckets (unvaxxed or stragglers) and compare boosted to prior months. It's not a very exciting result, that's all.

Expand full comment

"increasingly devotes posts to demonstrating that excess death statistics do not reflect post-vaccine deaths"

That would only be circumstantial evidence anyway, right? I.e. epidemiological.

However, I feel it might be reasonable to observe "Gee whiz, it doesn't look like those magic vaccines are saving bajillions of lives as promised."

Expand full comment

Right

Expand full comment

"Normal output should resume next week"

You're not the only substacker who tries to stick to a schedule. Speaking for myself, I would not hold you guys to a schedule like a newspaper. I prefer to get content when there's content worth presenting.

Expand full comment

The problem that always happens is I go into trips warmed up, and imagine that when I sit back down at my desk I am going to be able to jolt right back to a post per day. Doesn't happen that way... :(

Expand full comment

https://drclarecraig.substack.com/p/deaths-among-the-ghost-population

Have you seen this analysis from a U.K. doctor?

Expand full comment

It's good work. It models a signal for likely biases in the ONS data.

That said,

1 there's nothing apparently off-base with the ONS vaxxed mortality rates - the difference between teal and purple is single digits in every age group (it only even shows up in the youngest because their y axis is the shortest, otherwise it would be invisible).

2 the assumption that NIMS can provide a correct denominator for the unvaccinated remains unfounded. All they have is names on a list, with "list - vaxxed = unvaxxed." The list probably includes more actual "ghosts" (people who can neither get vaxxed nor die, because they are not still in the system, so they function as immortal unvaxxed) than the ONS set.

Expand full comment

Weird. Before I asked the very same question if you two minutes ago I did a search of this page for her name and it came back blank. I must not have scrolled far enough to load this post. So I have my answer

Expand full comment

This is why I really wish substack had a function for doing a search of all the comments on a particular post.

Expand full comment

👍

Expand full comment

Thanks Brian for your comments. Here is another way to look at the data that I find compelling. Take Table 1, all-cause mortality subset, and plot line graphs of age-standardized mortality rate over time for each of the vax subgroups. We first see that the "Ever vaccinated" curve consistently falls below the "Unvaccinated" curve, presumably demonstrating vaccine efficacy for this cohort over the entire time range. However, every one of the vax subgroup curves shoot immediately higher than the unvaccinated and remain there.

This appears to be a clear Simpson's paradox case, and it is the vax subgroup curves that are more relevant than "Ever vaccinated", which aggregates away the real story. We can also do breakdowns by age from subsequent tables, and the same phenomenon occurs.

Let me also weigh in in favor of analyses made by Igor, Josh, Clare, and Norman. The vitally important issue is that we want to make causal inferences from this cohort to the general population, and for this key problem the estimation of denominators that adjust for inherent biases in the cohort is reasonable and warranted.

Expand full comment

That gets to last year's version of ONS Ball. The whole reason for creating an any vax set is to show that these other signals are confounded / biased in different ways, two ways namely:

Older people get injected first. So if you are not using by-age numbers, you get crazy mortality jumps at first, even age-standardized (and I don't mess with age-standardized numbers myself, only raw rates by age).

Stragglers are weird. Once any age group has qualified for the second injection or once the elderly for the third, only people who don't want to get an injection despite getting the previous ones remain in the X dose ≥ 21 days buckets. Who are these people? Why are they dying more? Probably they had a health downturn that led them to stop getting injected; this could even include adverse reactions to the vax. But the important point is that the "real X dose ≥ 21 days mortality rate" is the X dose ≥ 21 days + X+1 dose buckets combined. Most people are simply in the next bucket.

If that all sounds implausible, remember that once the stragglers have added a handful of extra deaths there's no real way for them to lower their rate again because there's so few people left to add person-years back to the denominator. The same way you're screwed if you get three low ratings on your airbnb, you'll never get enough 5s to go back to superhost. So you only need a handful of deathbed-biased stragglers to ruin these buckets.

Expand full comment

[Edited for clarity:]

Your point about the denominators is well taken. And there is no real issue with using the ONS/Census figures **if** the unvaccinated who accounted for were equally likely to die as the unvaccinated who are not. But if the included unvaccinated were more likely to die than the missing unvaccinated, then the results will be biased in favor of making the unvaccinated appear to die at a higher rate. This is the core of the issue, and there is good evidence that the ONS/Census sample *is* biased in just this way:

Here is Clare Craig's post from a just earlier today: https://drclarecraig.substack.com/p/deaths-among-the-ghost-population

"The more recent data seems to have bias such that deaths in the unvaccinated are more likely to be included in the ONS sample whereas deaths in the vaccinated have the opposite bias and are more likely to be excluded from the ONS sample."

And another paper by Fenton et al. from November also showing a bias in the exclusions from the previous data: https://www.researchgate.net/publication/365202828_What_the_ONS_Mortality_Covid-19_Surveillance_Data_can_tell_us_about_Vaccine_Safety_and_Efficacy

The UK statistics regulator agreed with the key conclusions in that study:

https://wherearethenumbers.substack.com/p/uk-statistics-regulator-agrees-with

Expand full comment

I am sure there is a bias for the unvaxxed, and have discussed the same. But as far as I can tell there is no way to distinguish whether a comparison to NIMS-based mortality, as Craig does, is measuring the bias or just reflecting the wrongness of the NIMS-based unvaxxed denominator. And once again bias / confounding wouldn't actually disappear if (accurate) whole population were being tracked instead of a subset. But imo it would be a bit better. The problem however is that you cannot increase accuracy without a subset. The NIMS / UKHSA denominator is likely full of actual "ghosts," i.e. people who can no longer possibly be recorded as vaxxed or dead and so function as immortal unvaxxed.

Expand full comment

Sarah Caul says here very clearly that people who died who are in the Census but not in NIMS (nor in the extract of not in NIMS), they are classified as unvaccinated.

https://twitter.com/SarahCaul_ONS/status/1628077387513573395

Expand full comment

Holy smokes!!!

Expand full comment

I am very sleepy today but great stuff from Dr Clare!

Expand full comment

Right, and elsewhere says that the just-vaxxed are in the extract. So, either this design doesn't function, or it keeps the just-vaxxed labeled vaxxed. Or I'm misreading the whole thing, in which case I would again cite the lack of a spring 2021 2nd dose deaths dump in the highly sensitive 1D≥21 Days buckets as suggesting there's no real problem here.

Expand full comment

Okay, this post was really good! - and may finally have cleared things up for me =)

Question:

If the ONS spreadsheet is limited to persons in PHDA dataset then it will also not include all deaths ie. if someone in England dies, who is NOT in the PHDA dataset, their death is not included, correct? So there should be a discrepancy between actual total deaths and total deaths in the ONS spreadsheet?

I'm wondering if the percentage of actual total deaths that is not in the PHDA might give us some insights into the make up of the remaining population not included in the PHDA. Exaggerated example: if 5% of total actual deaths were in the non-PHDA population but the PHDA accounts for 90% of the total population, that would be very telling.

Of course, one limiting factor and an aspect which I find is not mentioned enough is that, as far as I know, (and this would be very surprising to some of my fellow mainland European citizens) England really doesn't know how big its population is. For example they have no obligation to register with local authorities when someone changes address. Fine, they have a census and a General Practice Extraction Service (GPES), and tax numbers, and newly everyone needs a share code to prove eligibility to work. But there is a, perhaps larger than expected, undocumented subsection of the population. Again the health of this group will vary from young and healthy to old and infirm, though tending to the former as it is much easier to be undocumented if young and healthy.

Expand full comment

Right, the total deaths for England+Wales are higher in https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/datasets/monthlyfiguresondeathsregisteredbyareaofusualresidence/2022 than in the ASMR sheets, e.g. 50.723k vs 18.45k (table 3) for November 2022 . Of that 18.45k, only .579 are labeled unvaxxed. So there's no sign of dumping extra deaths into the unvaxxed bucket.

On the flip side, for everyone who is living off the grid, there will be people "on the grid" who aren't really there. And these show up in the UKHSA mailing list of vax-eligible, even if they have moved, or died off-record, etc. So you don't want them in the denominator. This is why a more limited subset is better to track, even if it is biased.

Expand full comment

Okay and what percentage of the official total population is in the PHDA daset?

Expand full comment

Again using table 3 November 2022, 1.806065/(30/365.25) = 21.99 million, about 37% population, basically same % as deaths. *edit: or it might be closer to 80%, which is what I myself pegged it at before. *edit 2: OK, I used males only lol. It's (.293+1.512+.279+1.694)(365.25/30) = 46 million

Expand full comment

Thanks! (am on smartphone so can't really check myself)

Will dig around tonight, some more.

Expand full comment

Also note my edit. I dug some more and my previous value (80%) is reflected in other publications, so maybe my table 3 rush-calculation is wrong

Expand full comment

Seems Clare Craig was thinking along similar lines to me

https://drclarecraig.substack.com/p/deaths-among-the-ghost-population/comments

Expand full comment

"...demonstrating that excess death statistics do not reflect post-vaccine deaths. It is necessary to point this fact out, not because it supports any particular conclusion, but because it is true. There is no use pretending that it is not true; illusions will only get one so far in sustaining any belief."

But there is more than one way to skin a cat. Even if I agree entirely with your statistical points, it's a safe bet that excess death statistics *do* reflect post-vaccine deaths simply because we are having this discussion at all ie. the data is so bad, so confusing, that it can only be deliberate. Because no'one in charge wants to see the elephant in the room. If the raw data seen by our ant overlords showed how awesome the vaccine was, it would be plastered in full detail all over billboards.

As John Dee says today:

"So what we need to do next is consider age standardisation, there being a number of ways we can go about this. The simplest method is not to standardise at all but to look at what is happening within age groups. The ONS, in their wisdom decided to provide counts for the 18 – 39, 40 – 49, 50 -59, 60 -69, 70 -79 and +80 year groupings. This is a start but the 18 – 39y group is rather coarse and we don’t know anything about deaths in children despite the vaccine being authorised for use on nippers as young as 5 years. A slight oversight there methinks, and it has got to the sorry state that I wonder what they’re hiding on behalf of the State."

I've been tracking the performance of systems in the real world (combined human and mechanical systems) my whole working life. I can see instinctively, even if I can't "prove" it, by the mechanism of observation and data collection whether the truth of the real effect is desired or not.

Expand full comment

The PHDA explicitly excludes under-10s. Above that, there have been danger signals in the ONS data for the youngest groups for over a year. They haven't been hiding anything here. No one finds it exciting, that's all https://unglossed.substack.com/p/into-the-weeds-uk-deaths-data

Expand full comment

At this point I don't even care about my bruised priors. I'm just happy to see the Fisher-Price basketball hoop meme make its triumphant return.

Expand full comment

I hope Caul tweets it as fan art lol

Expand full comment

It’s always amusing to see your decidedly-not-hot-takes rile folks up. Get with the program, Brian, making bold and (by their nature) unprovable claims that match our existing view of reality is what the internet is for!

To be a bit less snarky, I do know that people like Igor are making their claims in good faith and there’s reasons to be suspicious about any “official” data source. But, uh, Brian is also saying that. The difference is he isn’t trying to put a number on the unknown/unknowable. He’s just trying to write the least sexy articles of all time!

Edit: I think I’m actually underselling Brian’s arguments here a touch, but it’s sort of because the dialogue surrounding this topic is a bit like a game of whack a mole that wavers between “not representative of the entire population” (which it never claims to be) and “deliberate massaging of the numbers” which Brian basically debunks by referring to the earlier versions of the spreadsheet (or at least eliminates the most obvious mechanism for doing so, ie, just making up unvaxxed numbers as the results started to pour in). The point about everyone being in the unvaccinated bucket to start out is not one that can be easily handwaved away by those critical of the ONS data. There’s a conversation to be had about bias and even fraudulent data collecting, but it’s all speculative. There’s no smoking gun here, just logical deductions which everyone has to weigh for themselves.

Expand full comment

I like to think of Brian as a kind of devils advocate, trying to keep the skeptical community honest. Otherwise we fall into the same dogmatism that we accuse the other side of.

Expand full comment

Thank you for appreciating the importance of the pre-transition bucket status, it means a lot to me

Expand full comment

We need a downvote button...

Expand full comment

Seems more like you need a point to make

Expand full comment

4 out of 5 dentists recommend Crest.

Expand full comment

But do they even lift?

Expand full comment

"But do they even lift?" Bro.

Expand full comment

Surely a caveat still standing is that the PHDA (sample) must be representative of the population at large? If it is not, then no reliable inferences can be drawn from the data.

Expand full comment

Since the population at large is no more "randomized" than the ONS dataset, it isn't clear that the failure to reflect the former adds bias (as opposed to decreases, or makes no change).

Expand full comment