By Zachary Stieber
Contributing Writer
Studies estimating the effectiveness of the flu vaccine have a significant flaw, according to a new paper.
The studies are biased because of how they handle the initial days after vaccination, according to the paper by Dr. Eyal Shahar, professor emeritus of epidemiology at the University of Arizona’s Mel and Enid Zuckerman College of Public Health.
Some of the studies classify people as unvaccinated if they became ill or had a clinical encounter within 13 days of vaccination. In others, patients were not counted if they showed symptoms or visited a health facility in the 13 days following vaccination.
The studies examined patients who went to a health facility with respiratory symptoms and were tested for influenza to estimate vaccine effectiveness.
The most recent one, published by the Centers for Disease Control and Prevention, estimated the vaccine was about 40% effective among children and 22% to 34% effective among adults. The corresponding author of that paper, who works for the CDC, said in an automated message that he would be out of the office through Tuesday.
The flaw can be corrected by including effectiveness estimates for the initial days following vaccination, Shahar said in his paper, which was released as a preprint after four journals declined to publish it.
The CDC did not respond by publication time to a request for comment.
Two outside researchers who read the paper said they agree with its conclusions.
“Excluding vaccinated who are infected <14 days after infection or moving them to unvaccinated creates immortal time bias for the vaccinated because they by definition can’t have an infection during this time,” Dr. Tracy Beth Hoeg, former acting director of the Food and Drug Administration’s Center for Drug Evaluation and Research, said in an emailed statement.
“On top of this there is collider bias: They are conditioning the study on those included in the vaccinated group surviving 14 days without infection. When you take out those who are, for whatever reason, most susceptible to infection, you are left with those who are more resistant to infection in the vaccinated group for the actual study/data analysis.”
Including infections that happened shortly after vaccination would fix the issues, but test-negative studies suffer from other problems, Hoeg said. Effectiveness data should be based on randomized studies with very few exceptions, she said.
Dr. Harvey Risch, professor emeritus of epidemiology in Yale School of Public Health’s Department of Epidemiology and Public Health, said the exclusion of the two weeks after vaccination is based on the reasoning that it takes about that time for a vaccine to generate immunity.
“The real problem is deeper. All such studies have two potential outcomes: benefit and harm,” Risch said via email.
“After two weeks, preventing infection is a benefit. However, getting infected during the first two weeks is a harm. Suppose hypothetically that a vaccinated individual had 100-times the risk of getting infected during the first two weeks, after which that went away and the vaccine benefit started. It would be obvious that the vaccine provided both harm and benefit. To know about vaccine effectiveness, would one only address benefit? Of course not—the large initial harm would also have to be weighed. In studies of potential harm, the harm events start immediately, not after two weeks as for benefit. If vaccines cause harm, accounting for that risk starts immediately upon vaccination.
“So the bottom line here is that public health has generally not recognized infection as both an infection to be prevented as a vaccine benefit, as well as a harm occurring before vaccines have been able to generate the beneficial components of their immune responses.”
