How the Stanford vaccine algorithm caused a great deal of controversy and left frontline workers at the end of the line

To choose who would be first in line for a coronavirus vaccine, UCSF Medical Center in San Francisco brought together a group of smart people this fall to study the guidelines and prioritize employees who had the most contact with patients, including caretakers. Kaiser Permanente did almost the same at 21 medical centers.

Stanford Medicine officials also studied the guidelines – then created a mathematical algorithm that prioritized people who, on paper, were at high risk for COVID-19, as older employees, but not the health professionals who had the most contact with patients, such as the medical care intensivist and nurses.

It did not take long for Stanford medical residents and colleagues on the front lines of the crisis to realize that they were being asked to risk their lives while an injection that would protect them was given to others. Doctors drew national attention when they made a furious protest on Friday and confronted the leaders of Stanford Medicine not only about the sloppy implementation, but about what they described as a secret approach. The leaders apologized quickly, promising to change their approach and make the vaccine distribution process more transparent.

The attention then turned to the algorithm that Stanford created to guide his decision.

The private hospital refused to discuss the matter. But the methodology that made Stanford an outlier on how he prioritized the vaccine was confused by MIT scholars and belittled by math geeks on Twitter.

“What happens when you apply the age priority scheme … to a population selected based on their theoretical * exposure * level?” tweeted Whitney Robinson, an epidemiologist at the University of North Carolina School of Public Health. “You get ridiculous results, like a wealthy 68-year-old Stanford Health Care radiologist and executives working from home in spacious homes being prioritized before 29-year-old residents with two roommates who work in the (emergency department) every day . “

An MIT Technology Review article on Stanford’s kerfuffle points to a 2019 study in the journal Science revealing that algorithms commonly used by hospitals to guide healthcare decisions tend to favor white patients over black patients. In one case, where patients’ health expenditures are used as an indicator of how sick they are, the algorithm mistakenly concludes that black patients, who tend to spend less, are healthier than white patients alike sick. The study authors estimate that, if the algorithm were corrected, the proportion of black patients receiving the care they need would increase by 29 percentage points, from 17.7% to 46.5%.

“Asking the algorithm to answer the wrong question results in all kinds of problems – racial prejudice in our study and the (highlighted) questions at Stanford,” said study author Ziad Obermeyer, a physician and associate professor of policy and management in UC Berkeley.

Algorithms can assist in decision making by assigning points to different variables. A graph of the Stanford algorithm, published in the MIT Technology Review, shows that employees were awarded points based on age (65 and older and 25 and younger), California Department of Public Health guidelines, and job characteristics. Jobs and departments received points based on the prevalence of COVID-19, positive percentage for COVID-19 and percentage of collected coronavirus tests.

The Chronicle shared Stanford’s algorithm with Obermeyer, who said that part of the problem appeared to be too much weight in the wrong variable: people at risk of falling ill (older employees), rather than people at risk of infection (ICU workers or emergency room patients.)

“When resources are scarce, any prioritization will always be open for debate,” said Obermeyer. “But the objective of the algorithms must be to make this prioritization fair, transparent and defensible. No one should hide behind ‘an error in the algorithm’, as if the algorithm had a mind of its own, when they literally designed the algorithm. “

UCSF took a different approach to its implementation, bringing together doctors, administrators and “stock experts and ethics experts” to make the decision, said Adrienne Green, the hospital’s medical director. The group first assessed the guidelines that recommend prioritizing employees at greatest risk: those working in COVID units and in emergency rooms. Vaccinating them first can also protect the vulnerable people that employees come into contact with frequently.

Dr. Jennifer Reid, a surgeon, is one of those employees. She’s been nervous for nine months. When patients need emergency care, surgery cannot wait until the results of the coronavirus test return, so Reid dresses in an N95 mask, goggles – which works – and hopes she won’t get sick. It is a great concern, mainly because the mother, in chemotherapy, lives with her. Her husband, a medical assistant at another hospital, is also at risk. He has slept in a hotel room more than once for the safety of everyone.

On December 13, Reid received an alert saying that she had been selected by lot to make one of the first appointments. Three days later, when the serum entered his arm, “I immediately felt relieved,” she said. “I think it was just a weight off my shoulders.” His second dose comes in January.

Dr. Jennifer Reid, a UCSF surgical intensive care and acute care colleague, received her first dose of the coronavirus vaccine on December 16, 2020. She will need a second dose in a few weeks.

UCSF also considered employees of communities with high infection rates, such as the Mission District, Green said. This predominantly Latin area includes many UCSF workers, including caretakers, who also work in the emergency room and in high-risk settings.

“We decided that everyone who worked in those areas would be in the same priority vaccination group,” said Green.

Of the first 896 employees vaccinated with the Pfizer vaccine at UCSF, just over half were nurses and medical residents. Technicians, physiotherapists and other health professionals represented around 19% of the total, and the teaching staff (PS and ICU doctors) were 17.5%. The remaining 8% included drivers, janitors and others.

Reina Lopez, a UCSF housekeeper who uses heavy protection to work, was the second person vaccinated at UCSF on December 16. “I felt all the emotions – happy, excited, scared – but most of all relieved that we were finally moving forward to return to normal,” she said.

Lopez and Reid said they appreciated the implementation of UCSF vaccination. But not all employees initially agreed. Brittany Howze, nurse at the COVID unit in Monte. The Zion campus complained that the process was not going fast enough.

So Howze was surprised last week when he learned that his turn would be Monday this week, and she would be one of the first to be vaccinated.

It almost didn’t happen. The serum triggered severe allergic reactions in some people, so when asked if she had a history of such reactions, Howze hesitated. Then she said no, keeping to herself the serious reactions she had after eating pistachios and cashews: “I was afraid I couldn’t do it,” she later told a reporter. “I have literally dreamed of getting the vaccine at night. And then wake up to remember that it is not true. I am so happy that this day has finally come. “

On Tuesday, UCSF received 7,900 doses of the second FDA-approved vaccine by Moderna, and hopes to inoculate 11,800 employees and doctors in training with a higher risk of infection, the hospital said.

Dr. Stephen Parodi of Kaiser Permanente, an infectious disease specialist who leads the COVID-19 response from the hospital system in northern California and across the country, said Kaiser never considered using an algorithm.

“We felt that having the appropriate clinical experience and operational leadership in the room was really important, and that it cannot be delegated to anyone other than our specialists,” said Parodi, adding that these experts used “a fairness lens” to ensure fairness. .

Parodi said he was not vaccinated “because I don’t work in the intensive care unit, for example. We try to follow that approach. “

Stanford changed his approach on the very night of the protest.

“Stanford has become an MMA,” said Andrea Henkel, one of the doctors who participated in the protest. “It became a first-come, first-served situation until the doses ran out on Friday,” said Henkel, a resident who waited in line for 90 minutes before receiving her injection in the hospital lobby.

In fact, Stanford Medicine has been on a vaccination wave since Friday, inoculating more than 2,000 health workers over the weekend, said spokeswoman Julie Greicius.

A new Vaccine Governance Committee oversees the revised plan that “prioritizes healthcare professionals in our clinical settings”, from residents and nurses to housewives, she said.

Sharif Vakili, another protesting doctor, said he was pleased with the hospital’s response.

“Of course, they are working hard to correct this error,” he said. “And I think they really feel awful about what happened.”

Nanette Asimov is a writer for the San Francisco Chronicle. Email: [email protected] Twitter: @NanetteAsimov