BuzzFeed News; Getty Images
The journalists at BuzzFeed News are proud to bring you trustworthy and relevant reporting about the coronavirus. To help keep this news free, become a member and sign up for our newsletter, Outbreak Today.
Over the last week, two overlapping teams of scientists in California released the first results of big antibody surveys to determine how many people have already been infected with the coronavirus. Their estimates were jaw-dropping.
In Silicon Valley, the true number of coronavirus infections could be 50 to 85 times higher than the number of reported ones. And in Los Angeles County, there might be 28 to 55 times more people infected than the official count.
The numbers, covered in the national press and shared widely on social media, suggested that far more people than previously realized have hidden infections. If that many people have already gotten sick, it also changes the calculation about how frequently the virus can lead to death. In the US, death rates of confirmed cases are over 5%, a high number driven in part by a lack of diagnostic testing.
But the new numbers out of Northern California suggest the virus may kill a much smaller portion of the wider pool of diagnosed and undiagnosed cases, in this case around 0.12% to 0.2%. That would be closer to the death rate for the flu, which is about 0.1%.
Right-wing and libertarian sites immediately seized on the findings, arguing that the economic shutdown has not been worth the public health gains.
Most experts agree there are far more coronavirus infections in the world than are being counted. But almost as instantly as the California numbers were released, critics called out what they saw as significant problems with, or at least big questions about, how the scientists had arrived at them. Chief among their concerns was the accuracy of the test underpinning both studies, and whether the scientists had fully accounted for the number of false positives it might generate.
I think the authors of the above-linked paper owe us all an apology, Andrew Gelman, a statistics and political science professor at Columbia University, wrote on his blog last weekend in reference to the study out of Santa Clara County, home to tech giants like Apple and Google. He added, I think they need to apologize because these were avoidable screw-ups. Theyre the kind of screw-ups that happen if you want to leap out with an exciting finding and you dont look too carefully at what you might have done wrong.
The two antibody surveys, led by researchers at Stanford University and the University of Southern California, are the largest conducted in the US to date. Scientists worldwide are counting on widespread use of these blood-based tests, also known as serological tests, to eventually answer important questions about the pandemic, from who might be immune to reinfection to exactly how widespread the disease is. Such studies are underway around the world, from Germany to Italy to New York.
These are extremely valuable studies, and when theyre done right, theyre going to tell us really important things, Marm Kilpatrick, an infectious disease researcher at the University of California at Santa Cruz, told BuzzFeed News. I just think if theyre not done in careful ways, they can mislead us about whats actually happening.
Kilpatrick worries that the results of these two studies could in turn erode public trust in the need for lockdowns. If thats based on faulty information, that would be terrible, he said.
Here are some of the biggest criticisms about the studies.
The pandemic has kicked academic publishing into warp speed, and scientists are uploading discoveries to the internet every day, bypassing the normal checks of peer review in favor of quickly sharing information. Even so, both research teams who share a member, Neeraj Sood of USC have moved at a pace thats raised some eyebrows in the scientific community. They floated the possibility of scores of uncounted infections to the media before presenting data to back it up, leading some observers to question whether they had rushed to prove a preconceived theory.
On March 17 in Stat, before the antibody surveys had started, Stanford professor John Ioannidis bemoaned the lack of reliable data about the virus, a fiasco that creates tremendous uncertainty about the risk of dying from Covid-19. The next week, in a Wall Street Journal op-ed titled Is the Coronavirus as Deadly as They Say?, two other Stanford faculty argued that projections of the death toll could plausibly be orders of magnitude too high.
Last Friday, a team led by those three researchers uploaded a preliminary draft, or a preprint, about their Santa Clara County study. By early April, there were 956 confirmed cases there. But based on their serological study of 3,300 people, the researchers concluded that the actual number of infections was between 48,000 and 81,000.
On Medrxiv, the preprint server where their results were posted, readers have left 300 comments and counting.
Asked to comment for this story, Jay Bhattacharya, a Stanford professor of medicine and the papers senior author, acknowledged by email Monday night that his team had received a vast number of comments and suggestions on our working paper. They are planning to soon release a revised version incorporating many of the suggestions, with a new appendix addressing many of the most important criticisms we have heard, he wrote.
This is exactly the way peer review should work in science, he added.
And on Monday afternoon, the Los Angeles results were shared in a press conference staged by health officials from Los Angeles County and Sood, vice dean for research at USCs Price School of Public Policy and co-leader of the study there. In early April, the county had reported nearly 8,000 cases. But according to the new serological study of 863 people, the researchers estimated the true number of infections was between 221,000 and 442,000.
Those figures, according to an accompanying press release, suggest that the fatality rate is much lower than thought.
In an unexpected twist later that night, Sood said he then learned that a draft of his paper, which had not been released as part of the press conference, had mysteriously been posted to RedState.com, a right-wing blog. The site took it down upon his request though not before a few scientists found it.
In an interview Tuesday afternoon, Sood said he had no idea how the report wound up online without his permission. Its just upsetting to me that it was done, because I really tried to make sure that something like this doesnt happen, he told BuzzFeed News. (BuzzFeed News has a cached copy but is not discussing it here.)
Sood said he had no choice but to release the numbers under county rules, because anything the public health department is involved with must be disclosed to the countys leaders. But we clearly couched those results as these are preliminary findings,' he said.
Sood said he plans to eventually post a paper online, but only once it has been peer-reviewed and approved for publication.
I don't want crowd peer review or whatever you want to call it, he said. Its just too burdensome and Id rather have a more formal peer review process.
Still, skipping the traditional step of data sharing didnt go over well with some scientists.
You cant report the conclusions without providing scientific evidence or you shouldnt, said Natalie Dean, a University of Florida biostatistician.
A Stanford medical student administers a coronavirus antibody study in Mountain View, California, on April 3.
One of scientists biggest concerns is that the researchers were overly confident in their tests false-positive rate and failed to account for the likely possibility that it could be lower or higher a potential difference that would dramatically affect the studies conclusions.
Tests like these look for antibodies formed by the immune system in response to a past infection, and differ from the nasal- and throat-swab diagnostic tests that spot current infections. Antibodies are usually an indicator of immunity against infectious diseases, but since this virus has only existed for about four months, scientists dont yet know how long such protection might last.
Nevertheless, antibody tests have been touted as key to identifying who might be safe from reinfection and could help reopen the economy. To increase their availability, the FDA is letting them be sold without checking the accuracy rates advertised by their manufacturers. As a result, only four have emergency authorization from the agency and more than 120 others have varying and unverified degrees of accuracy.
Both California studies used tests from Premier Biotech, a Minnesota-based company. These tests were used because they were donated and their accuracy claims were independently verified at Stanford, Sood said in an interview last week.
Before being deployed in Northern California, Premiers test kit was run against a total of 401 samples known to be coronavirus-negative: 371 in the manufacturers testing, 30 in Stanfords testing. Across the two sets of results, Premiers test reported that 399 of the 401 were negative.
The researchers interpreted this to mean that it most likely had a false-positive rate of 0.5%, according to the report. At the same time, it could also range somewhere between 0.1% and 1.7%, according to the researchers confidence interval, a statistical term that accounts for a range of possible errors.
That matters because the Santa Clara study found antibodies in 50 of the 3,330 participants, or 1.5%. Since the tests false-positive rate could be as high as 1.7%, it is possible that many of the so-called positives were not, in fact, positive.
Literally every single one could be a false positive, Kilpatrick said. No one thinks all of them were, but the problem is we cant actually exclude the possibility.
That possibility is even harder to rule out in situations when the number of actual infections is low. If only a minority of Santa Clara County residents are infected, the test would have a higher likelihood of turning up false positives.
In their analysis, the researchers adjusted for this range of rates while calculating their infection estimates. But given the small number of samples used to validate the test, coupled with the fact that the test is almost as new as the virus, critics say its possible that the true false-positive rates could be even higher than presented. The test also generates a large percentage of false negatives, 20%, with a possible range of up to 28%, according to the combined validation efforts.
Theres more uncertainty than theyve accounted for, Dean said.
The wide range in estimates for infections in Santa Clara County in early April from 48,000 to 81,000 infections reflects the difference in accuracy rates calculated for the test across the two times it was validated. Using the manufacturers rates to correct for the total, 2.5% of the county was infected. Using Stanfords, about 4.2% were.
As for the Southern California study, there isnt yet full data to analyze. But researchers there found antibodies in about 35 out of 836 people, or 4.1% of those tested.
So far, serology tests across the world have produced a wide variety of estimates of the number of true coronavirus infections, with those from the California studies on the lower end. At Wuhans Zhongnan Hospital, 2.4% of its 3,600 employees were found to have antibodies. Tests on 500 residents of a German town turned up antibodies in 14% of them. And a study near Boston found that 32% of 200 people had been previously infected.
On Monday, WHO Director-General Tedros Adhanom Ghebreyesus maintained that the prevalence was low, not more than 2 to 3 percent.
Even slight increases or decreases in the number of positives matter because, in such a small sample, they could make a big difference in the estimated infections across the population.
Sood said that he and the Stanford team had done their best to adjust for the tests false positives and false negatives, while acknowledging that they were taking a second look at their confidence intervals. As new data comes in about these tests, we will update these results, he said.
None of this means that testing shouldnt have been done, or that the researchers shouldnt have published their data.
The error, observers say, was in not being more upfront about how little the numbers could be trusted.
The fact that they made mistakes in their data analysis does not mean that their substantive conclusions are wrong (or that theyre right), Gelman said by email. It just means that there is more uncertainty than was conveyed in the reports and public statements.
Another aspect of the Santa Clara County study that has been flagged as a major problem is how it found participants: Facebook ads.
Spreading the word about tests through social media, the researchers said, helped the study get off the ground quickly and allowed organizers to target people by zip code and demographic characteristics like sex, race, and age. Then they had people drive through three testing sites.
One potential downside of this approach, though, is that since testing is so scarce in the US, the mention of a test may have drawn disproportionate numbers of people whod had COVID-19 symptoms but werent able to get tested. That could have inflated the number of positive results. Its unclear by how much: The researchers said they collected data about symptoms but didnt describe how many of the positive testers had symptoms or what the symptoms were.
This recruiting resulted in a group that was markedly different from Santa Clara Countys overall population in a couple ways: Certain zip codes and white women were overrepresented; Latino and Asian people were underrepresented. Given they were Facebook users, the test likely didnt include people without internet access.
When calculating the estimated infections, the researchers accounted for these differences as well as the tests accuracy rates in order to try to make their results representative of the county. They didnt adjust for age, though, even though some of the age groups were also not representative: 5% of participants were over 64, compared with 13% in the county. Sood said that they did not have enough participants across age groups to adjust for age.
All these decisions, among others, influenced the final estimate of infections. When the demographic and geographic differences were adjusted for, the percentage of positive results across the population, 1.5%, nearly doubled.
Kilpatrick believes the researchers did themselves a disservice by not recruiting a more representative group from the get-go. If the group that did the Stanford study had asked any of the scientists who do these studies all the time, Were thinking of recruiting on Facebook, wed say, Dont do it, he said.
An ideal way to recruit, Kilpatrick said, would be to use a county database of addresses and send letters to a subset of random addresses, making sure that any one neighborhood isnt overrepresented. Of course, he acknowledged, theres always the chance that lots of people wouldnt respond anyway.
Other serology studies have taken their own approaches to finding participants. In the Boston suburb of Chelsea, researchers pinpricked the fingers of random passersby in Bellingham Square. Starting this week, New York is testing more than 3,000 people in supermarkets across the state.
For the Los Angeles County study, Sood said he and his team went a different way: They enlisted a market research firm with a proprietary database of thousands of emails and phone numbers of county residents. They invited a random subset to participate in a study about COVID, but didnt say it was about testing.
The team set about recruiting people to fulfill quotas for race/ethnicity, age, and so on, based on the countys demographics. Once a subgroups quota was met, they stopped enrolling people. To make sure they were reaching underrepresented groups, the market research firm made follow-up calls to people in those categories.
Participants were then invited to drive through six testing sites on a recent weekend. Staff also went to some respondents homes to do testing there.
Even though Sood says they went to great lengths to make the group representative of the population, he acknowledged that there is no perfect recruitment strategy.
Of the Santa Clara County study, he said the team had done their best with limited resources. We still thought it was worthwhile doing it even though we fully recognize our methods were not anywhere close to perfect, he said. We still thought it would provide useful information and it would add to the debate about whats going on.
Few people would turn down the chance to find out whether theyve had the coronavirus. But Dean questioned whether, from a public health messaging standpoint, it is helpful to fixate on these infection estimates when they are so preliminary.
No matter how many people may or may not be infected numbers that scientists wont be able to pin down for a long time the real numbers that matter right now, in terms of conveying the threat of the disease, are those of the bodies ending up in ICU beds and funeral homes.
Either way, were ending up with a lot of people being hospitalized and dying, she said. Everyone needs to keep that part in mind.
Go here to see the original:
- What You Need to Know About the New SAG-AFTRA Agreement ... - Casting Networks - November 24th, 2023 [November 24th, 2023]
- Premiership Rugby Highlights Show | Be on TV - ITVX - November 24th, 2023 [November 24th, 2023]
- Elon Musks Neuralink wants to merge your brain with AI at what ... - Vox.com - October 20th, 2023 [October 20th, 2023]
- The World's Job During the War on Hamas: Save the Space For Peace - TIME - October 20th, 2023 [October 20th, 2023]
- Brooklyn Beckham hits back at haters who mock his online cooking videos - Yahoo News UK - October 20th, 2023 [October 20th, 2023]
- Secret to immortality could be just a decade away as scientists predict when well learn to live forever... - The US Sun - August 18th, 2023 [August 18th, 2023]
- Roblox Introduces Heads and Bodies to the UGC Program - Game Is Hard - August 18th, 2023 [August 18th, 2023]
- Bigger Th*t Than Cardi B: Amidst Conor McGregor's Friend's Feud ... - The Sportsrush - August 18th, 2023 [August 18th, 2023]
- Sofia Richie Says Royal Weddings Inspired Her Nuptials - Brides - August 18th, 2023 [August 18th, 2023]
- How Many Times a Day to Post to BeReal a Day - Alphr - August 18th, 2023 [August 18th, 2023]
- Usage of dark web for uploading sensitive defence information is a ... - LawBeat - August 18th, 2023 [August 18th, 2023]
- What Should You Look for in your WiFi Plan? - My Everyday Tech - August 18th, 2023 [August 18th, 2023]
- The Pros and Cons of Using No-Code Websites as a Blogger - MUO - MakeUseOf - August 18th, 2023 [August 18th, 2023]
- The Future Is Now: Wrestling with Ethics, Policy and Brain-Computer ... - NC State News - May 2nd, 2023 [May 2nd, 2023]
- How to Share Images Online Easily With Pasteboard - Dignited - May 2nd, 2023 [May 2nd, 2023]
- The Acid Test, April 2023 - bandcamp.com - May 2nd, 2023 [May 2nd, 2023]
- Top 9 Generative AI Applications and Tools - eWeek - May 2nd, 2023 [May 2nd, 2023]
- Top 50 Most Viewed U.S. YouTube Channels Week Of 04/30/2023 - Tubefilter - May 2nd, 2023 [May 2nd, 2023]
- PPC Automation: When & How To Use It - Search Engine Journal - May 2nd, 2023 [May 2nd, 2023]
- Image SEO: Optimizing images for search engines Yoast - December 16th, 2022 [December 16th, 2022]
- All That Is In The Heavens [Part 14]: Bionic Basim - MuslimMatters - October 19th, 2022 [October 19th, 2022]
- Mill Creek Leverages QR Codes To Help People Store Their Loved One's Messages as Forever Cherishable Memories - Digital Journal - October 19th, 2022 [October 19th, 2022]
- In their own words: County Commissioner candidates Randy George and Steve Skinner on why each should be considered for the open District 3... - October 19th, 2022 [October 19th, 2022]
- Get ready now for the 2022 Great Southern BioBlitz! - Cosmos - October 19th, 2022 [October 19th, 2022]
- ILUMINATIVE Is Reviving the Modern Genre Of Hip-Hop/Rock & Nu-Metal with the Latest Album, 'Endangered' - Daily Music Roll - October 19th, 2022 [October 19th, 2022]
- Enter the Cambridge Independents Photography Awards 2022 - and win great prizes - Cambridge Independent - October 19th, 2022 [October 19th, 2022]
- Coros Vertix 2 review the battery life of this smartwatch is insane - CyclingWeekly - October 19th, 2022 [October 19th, 2022]
- WTF Are POAPs? The Intriguing Power of Proof of Attendance Protocol - nft now - October 19th, 2022 [October 19th, 2022]
- Former NRL star Finch fighting to avoid jail amid fears hell be targeted by hardened criminals - Fox Sports - October 19th, 2022 [October 19th, 2022]
- Muni Long's long and winding road to R&B stardom - Los Angeles Times - October 19th, 2022 [October 19th, 2022]
- She-Hulk Recap: Episode 9 | Jen Breaks the MCU - United By Pop - October 19th, 2022 [October 19th, 2022]
- People with dark personality type desire enhancing their brains with future tech, new study reveals... - The US Sun - October 2nd, 2022 [October 2nd, 2022]
- Emma Chamberlain has opened up the doors of her new home and I havent been able to think of anything else since - image.ie - October 2nd, 2022 [October 2nd, 2022]
- 'Don't believe everything you read': The story of The Onion - Bennington Banner - October 2nd, 2022 [October 2nd, 2022]
- DJ Ace - Peace of Mind Vol 39 (Saxophone Edition Slow Jam Mix ... - September 29th, 2022 [September 29th, 2022]
- Longtermism Movement Misses the Importance of War - Scientific American - September 29th, 2022 [September 29th, 2022]
- Ten warning signs that your phone is spying on you and tips on removing spyware - Scottish Daily Express - September 29th, 2022 [September 29th, 2022]
- JNU Admission 2022: Registration starts for UG admission on the basis of CUET in JNU, fill your form from this link - News Day Express - September 29th, 2022 [September 29th, 2022]
- RTI reply: 42 cars in CM Bhagwant Mann convoy, opposition says 'badlav or waste of money' - The Indian Express - September 29th, 2022 [September 29th, 2022]
- 10 best One Direction rock covers, from the 1975 to Wolf Alice - Alternative Press - September 29th, 2022 [September 29th, 2022]
- How to use GPX maps and trackback on the Galaxy Watch 5 Pro Route tile - Android Central - September 29th, 2022 [September 29th, 2022]
- CHITCHAT OVER COFFEE: Give the Heart the space it needs - Manila Bulletin - September 29th, 2022 [September 29th, 2022]
- How technology will bridge the gap for the next million - Times of India - September 20th, 2022 [September 20th, 2022]
- now allows you to upload your photos and edit them - Gearrice - September 20th, 2022 [September 20th, 2022]
- 2023 Honda CR-V First Drive Review: The Once And Future King? - Motor1 - September 20th, 2022 [September 20th, 2022]
- Pantheon Review: 2022's Wildest Tech Thriller Is a Cartoon | Time - TIME - September 7th, 2022 [September 7th, 2022]
- Worth Watching: Lord of the Rings Prequel, Uploading Gets Serious in Pantheon, NASCAR Race for the Championship - Channel3000.com - WISC-TV3 - September 7th, 2022 [September 7th, 2022]
- The super-rich preppers planning to save themselves from the apocalypse - The Guardian - September 7th, 2022 [September 7th, 2022]
- 8 Anime Outros Too Good to Skip - Geek Girl Authority - September 7th, 2022 [September 7th, 2022]
- From social media to Pollywood, singer Noor Chahal says originality is her secret to success - The Indian Express - September 7th, 2022 [September 7th, 2022]
- Behind the music - Ratios - RTE.ie - September 7th, 2022 [September 7th, 2022]
- According To Believers, 7% Of Us Will Find Code 0010110 And Exit The Matrix On August 27 - YourTango - August 2nd, 2022 [August 2nd, 2022]
- Elon Musk confirms he already uploaded his brain to a cloud, he spoke to himself - Marca - August 2nd, 2022 [August 2nd, 2022]
- Whatever Happened to the Transhumanists? - Gizmodo Australia - August 2nd, 2022 [August 2nd, 2022]
- TREASURES OF THE TRI-STATE 2022 - 97.3 the Rock - Radio Dubuque - August 2nd, 2022 [August 2nd, 2022]
- India versus West Indies second T20 highlights: Fancode yesterday match 2nd T20 highlights where to watch - The Sportsrush - August 2nd, 2022 [August 2nd, 2022]
- Edinburgh internet troll guilty of sending vile online messages to politicians - Edinburgh Live - August 2nd, 2022 [August 2nd, 2022]
- The PhotoStick Reviews (PhotoStick For PC) - Does This Device Really Worth Buying? Must Read This Before You Buy! - Outlook India - August 2nd, 2022 [August 2nd, 2022]
- How Close Are We to Uploading Your Mind? - How-To Geek - July 23rd, 2022 [July 23rd, 2022]
- Ranveer Allahbadia: Of collaborations and making Reels - Hindustan Times - July 23rd, 2022 [July 23rd, 2022]
- Jasmine TXO: The key to cracking TikTok - Ragtrader - July 23rd, 2022 [July 23rd, 2022]
- You can also earn big money sitting at home from YouTube or Instagram, know how? (How to Earn Money From YouTube And Instagram?) - News Day Express - July 23rd, 2022 [July 23rd, 2022]
- What the POTS shutdown will mean for faxing in the healthcare industry - Healthcare Dive - July 23rd, 2022 [July 23rd, 2022]
- Back Issues: Connecting with the past via historical objects - Stuff - July 23rd, 2022 [July 23rd, 2022]
- Hidden MrBeast Video Sees the Popular Millionaire Put Over $50,000 on the Line for His Fans - EssentiallySports - July 17th, 2022 [July 17th, 2022]
- Some of the internet's most homoerotic content is on The History Channel's YouTube? - Queerty - July 17th, 2022 [July 17th, 2022]
- A Twitter test will remind you to add alt text to images - TechCrunch - July 17th, 2022 [July 17th, 2022]
- Beyonc's 7th Studio Album Renaissance is About Release on The 29th of July - Daily Music Roll - July 17th, 2022 [July 17th, 2022]
- Repairing an old thing is a small act of defiance against disposable culture - The Irish Times - July 17th, 2022 [July 17th, 2022]
- Warzone players slam the dumbest Perk in the history of CoD - CharlieINTEL.com - July 17th, 2022 [July 17th, 2022]
- Tatsuki Fujimotos Just Listen To This Song Is A Lesson In Love And Buddhism - DualShockers - July 17th, 2022 [July 17th, 2022]
- Required Documents - School and Student Services by Community Brands - July 3rd, 2022 [July 3rd, 2022]
- Dr. Strange: Can the Multiverse Really Work as a Plot Device? - Walter Bradley Center for Natural and Artificial Intelligence - July 3rd, 2022 [July 3rd, 2022]
- Sophia Mirzas daughters say living with father happily - The News International - July 3rd, 2022 [July 3rd, 2022]
- Warning for MILLIONS of iPhone users before going on holiday this year... - The US Sun - July 3rd, 2022 [July 3rd, 2022]
- Interview: The Vamps Celebrate 10 Years At The Top! - CelebMix - July 3rd, 2022 [July 3rd, 2022]
- The powerful lesson I learned watching my father die - Sydney Morning Herald - July 3rd, 2022 [July 3rd, 2022]
- How to Sign Up for Amazon Primes Free Trialand Save Big Ahead of Prime Day in 2022 - Yahoo Life - July 3rd, 2022 [July 3rd, 2022]
- Firoz Hasan of Free Motion stands by flood affected Sylhet and Sunamganj - The Daily Star - June 29th, 2022 [June 29th, 2022]
- Almost Quote the Movie Serena Williams Fangirls Over Iron Man as She Dons Similar Pair of Glasses Ahead of Her Wimbledon Championships 2022 Campaign... - June 29th, 2022 [June 29th, 2022]