Health products powered by artificial intelligence are streaming into our lives, from virtual doctor apps to wearable sensors and drugstore chatbots.
IBM boasted that its AI could outthink cancer. Others say computer systems that read X-rays will make radiologists obsolete. AI can help doctors interpret MRIs of the heart, CT scans of the head and photographs of the back of the eye, and could potentially take over many mundane medical chores, freeing doctors to spend more time talking to patients, said Dr. Eric Topol, a cardiologist and executive vice president of Scripps Research in La Jolla.
Theres nothing that Ive seen in my 30-plus years studying medicine that could be as impactful and transformative as AI, Topol said. Even the Food and Drug Administration which has approved more than 40 AI products in the last five years says the potential of digital health is nothing short of revolutionary.
Yet many health industry experts fear AI-based products wont be able to match the hype. Some doctors and consumer advocates fear that the tech industry, which lives by the mantra fail fast and fix it later, is putting patients at risk and that regulators arent doing enough to keep consumers safe.
Early experiments in AI provide a reason for caution, said Mildred Cho, a professor of pediatrics at Stanfords Center for Biomedical Ethics.
Systems developed in one hospital often flop when deployed in a different facility, Cho said. Software used in the care of millions of Americans has been shown to discriminate against minorities. And AI systems sometimes learn to make predictions based on factors that have less to do with disease than the brand of MRI machine used, the time a blood test is taken or whether a patient was visited by a chaplain.
In one case, AI software incorrectly concluded that people with pneumonia were less likely to die if they had asthma an error that could have led doctors to deprive asthma patients of the extra care they need.
Its only a matter of time before something like this leads to a serious health problem, said Dr. Steven Nissen, chairman of cardiology at the Cleveland Clinic.
Medical AI, which pulled in $1.6 billion in venture capital funding in the third quarter alone, is nearly at the peak of inflated expectations, concluded a July report from research company Gartner. As the reality gets tested, there will likely be a rough slide into the trough of disillusionment.
That reality check could come in the form of disappointing results when AI products are ushered into the real world. Even Topol, the author of Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again, acknowledges that many AI products are little more than hot air.
Experts such as Dr. Bob Kocher, a partner at the venture capital firm Venrock, are blunter. Most AI products have little evidence to support them, Kocher said. Some risks wont become apparent until an AI system has been used by large numbers of patients. Were going to keep discovering a whole bunch of risks and unintended consequences of using AI on medical data, Kocher said.
None of the AI products sold in the U.S. have been tested in randomized clinical trials, the strongest source of medical evidence, Topol said. The first and only randomized trial of an AI system which found that colonoscopy with computer-aided diagnosis found more small polyps than standard colonoscopy was published online in October.
Few tech start-ups publish their research in peer-reviewed journals, which allow other scientists to scrutinize their work, according to a January article in the European Journal of Clinical Investigation. Such stealth research described only in press releases or promotional events often overstates a companys accomplishments.
And although software developers may boast about the accuracy of their AI devices, experts note that AI models are mostly tested on computers, not in hospitals or other medical facilities. Using unproven software may make patients into unwitting guinea pigs, said Dr. Ron Li, medical informatics director for AI clinical integration at Stanford Health Care.
AI systems that learn to recognize patterns in data are often described as black boxes because even their developers dont know how they reached their conclusions. Given that AI is so new and many of its risks unknown the field needs careful oversight, said Pilar Ossorio, a professor of law and bioethics at the University of Wisconsin-Madison.
Yet the majority of AI devices dont require FDA approval. None of the companies that I have invested in are covered by the FDA regulations, Kocher said.
Legislation passed by Congress in 2016 and championed by the tech industry exempts many types of medical software from federal review, including certain fitness apps, electronic health records and tools that help doctors make medical decisions.
Theres been little research on whether the 320,000 medical apps now in use actually improve health, according to a report on AI published Dec. 17 by the National Academy of Medicine.
The FDA has long focused its attention on devices that pose the greatest threat to patients. And consumer advocates acknowledge that some devices such as ones that help people count their daily steps need less scrutiny than ones that diagnose or treat disease.
Some software developers dont bother to apply for FDA clearance or authorization, even when legally required, according to a 2018 study in Annals of Internal Medicine.
Industry analysts say that AI developers have little interest in conducting expensive and time-consuming trials. Its not the main concern of these firms to submit themselves to rigorous evaluation that would be published in a peer-reviewed journal, said Joachim Roski, a principal at Booz Allen Hamilton, a technology consulting firm, and coauthor of the National Academys report. Thats not how the U.S. economy works.
But Oren Etzioni, chief executive at the Allen Institute for AI in Seattle, said AI developers have a financial incentive to make sure their medical products are safe.
If failing fast means a whole bunch of people will die, I dont think we want to fail fast, Etzioni said. Nobody is going to be happy, including investors, if people die or are severely hurt.
The FDA has come under fire in recent years for allowing the sale of dangerous medical devices, which have been linked by the International Consortium of Investigative Journalists to 80,000 deaths and 1.7 million injuries over the last decade.
Many of these devices were cleared for use through a controversial process called the 510(k) pathway, which allows companies to market moderate-risk products with no clinical testing as long as theyre deemed similar to existing devices.
In 2011, a committee of the National Academy of Medicine concluded the 510(k) process is so fundamentally flawed that the FDA should throw it out and start over.
Instead, the FDA is using the process to greenlight AI devices.
Of the 14 AI products authorized by the FDA in 2017 and 2018, 11 were cleared through the 510(k) process, according to a November article in JAMA. None of these appear to have had new clinical testing, the study said.
The FDA cleared an AI device designed to help diagnose liver and lung cancer in 2018 based on its similarity to imaging software approved 20 years earlier. That software had itself been cleared because it was deemed substantially equivalent to products marketed before 1976.
AI products cleared by the FDA today are largely locked, so that their calculations and results will not change after they enter the market, said Bakul Patel, director for digital health at the FDAs Center for Devices and Radiological Health. The FDA has not yet authorized unlocked AI devices, whose results could vary from month to month in ways that developers cannot predict.
To deal with the flood of AI products, the FDA is testing a radically different approach to digital device regulation, focusing on evaluating companies, not products.
The FDAs pilot pre-certification program, launched in 2017, is designed to reduce the time and cost of market entry for software developers, imposing the least burdensome system possible. FDA officials say they want to keep pace with AI software developers, who update their products much more frequently than makers of traditional devices, such as X-ray machines.
Scott Gottlieb said in 2017 while he was FDA commissioner that government regulators need to make sure its approach to innovative products is efficient and that it fosters, not impedes, innovation.
Under the plan, the FDA would pre-certify companies that demonstrate a culture of quality and organizational excellence, which would allow them to provide less upfront data about devices.
Pre-certified companies could then release devices with a streamlined review or no FDA review at all. Once products are on the market, companies will be responsible for monitoring their own products safety and reporting back to the FDA.
High-risk products, such as software used in pacemakers, will still get a comprehensive FDA evaluation.
But research shows that even low- and moderate-risk devices have been recalled due to serious risks to patients, said Diana Zuckerman, president of the National Center for Health Research. Johnson & Johnson, for example, has recalled hip implants and surgical mesh.
Some AI devices are more carefully tested than others. An AI-powered screening tool for diabetic eye disease was studied in 900 patients at 10 primary care offices before being approved in 2018. The manufacturer, IDx Technologies, worked with the FDA for eight years to get the test, sold as IDx-DR, right, said Dr. Michael Abramoff, the companys founder and executive chairman.
IDx-DR is the first autonomous AI product one that can make a screening decision without a doctor. The company is now installing it in primary care clinics and grocery stores, where it can be operated by employees with a high school diploma.
Yet some AI-based innovations intended to improve care have had the opposite effect.
A Canadian company, for example, developed AI software to predict a persons risk of Alzheimers based on their speech. Predictions were more accurate for some patients than others. Difficulty finding the right word may be due to unfamiliarity with English, rather than to cognitive impairment, said coauthor Frank Rudzicz, an associate professor of computer science at the University of Toronto.
Doctors at New Yorks Mount Sinai Hospital hoped AI could help them use chest X-rays to predict which patients were at high risk of pneumonia. Although the system made accurate predictions from X-rays shot at Mount Sinai, the technology flopped when tested on images taken at other hospitals. Eventually, researchers realized the computer had merely learned to tell the difference between that hospitals portable chest X-rays taken at a patients bedside with those taken in the radiology department. Doctors tend to use portable chest X-rays for patients too sick to leave their room, so its not surprising that these patients had a greater risk of lung infection.
DeepMind, a company owned by Google, has created an AI-based mobile app that can predict which hospitalized patients will develop acute kidney failure up to 48 hours in advance. A blog post on the DeepMind website described the system, used at a London hospital, as a game changer. But the AI system also produced two false alarms for every correct result, according to a July study in Nature. That may explain why patients kidney function didnt improve, said Dr. Saurabh Jha, associate professor of radiology at the Hospital of the University of Pennsylvania. Any benefit from early detection of serious kidney problems may have been diluted by a high rate of overdiagnosis, in which the AI system flagged borderline kidney issues that didnt need treatment, Jha said.
Google had no comment in response to Jhas conclusions.
This story was written for Kaiser Health News, an editorially independent publication of the Kaiser Family Foundation.
- artificial intelligence | Definition, Examples, and ... - March 28th, 2020
- What is Artificial Intelligence? How Does AI Work? | Built In - March 28th, 2020
- Benefits & Risks of Artificial Intelligence - Future of ... - March 28th, 2020
- Return On Artificial Intelligence: The Challenge And The Opportunity - Forbes - March 28th, 2020
- VA Looking to Expand Usage of Artificial Intelligence Data - GovernmentCIO Media - March 28th, 2020
- Global Artificial Intelligence in Healthcare Market - Premium Insight, Competitive News Feed Analysis, Company Usability Profiles, Market Sizing &... - March 28th, 2020
- The Global Artificial Intelligence in Aviation Market is expected to grow from USD 214.36 Million in 2018 to USD 1,824.46 Million by the end of 2025... - March 28th, 2020
- Is artificial intelligence the answer to disease prevention? - The Burn-In - March 28th, 2020
- Why transparency is key to promoting trust in artificial intelligence - IT PRO - March 28th, 2020
- Coronavirus: Spain to use artificial intelligence to automate testing - ComputerWeekly.com - March 28th, 2020
- Bridging the gaps: joining human and artificial intelligence | Technology - Business Chief Canada - March 28th, 2020
- DIAGNOS Will Utilize its Artificial Intelligence Medical Platform FLAIRE in Response to the US White House - Call to Action to Analyse and Transform... - March 26th, 2020
- LiveMD Global Telehealth Platform launches artificial intelligence tracking and triaging tools to help combat COVID-19 (CoronaVirus) Pandemic -... - March 26th, 2020
- New Research from Newark Reveals Strong Adoption of Artificial Intelligence within the Internet of Things Ecosystem - Embedded Computing Design - March 26th, 2020
- AI vs COVID-19: Here are the AI tools and services fighting coronavirus - AI News - March 26th, 2020
- Stanford virtual conference to focus on COVID19 and artificial intelligence | Stanford News - Stanford University News - March 26th, 2020
- Artificial Intelligence in the energy sector: opportunities and challenges - WhaTech - March 26th, 2020
- 31 Companies, Products and People making Artificial Intelligence a Reality in 2020 - PR Web - March 26th, 2020
- KT zu Guttenberg, Artificial Intelligence and You - theTrumpet.com - March 26th, 2020
- BrainChip and Socionext Provide a New Low-Power Artificial Intelligence Platform for AI Edge Applications - Design and Reuse - March 26th, 2020
- Artificial Intelligence Chipsets Market report reviews overview with demographic data and industry growth trends by 2025 - WhaTech Technology and... - March 26th, 2020
- Richmond-based Blue Heron Capital invests in company that uses artificial intelligence to improve lung disease monitoring - Richmond.com - March 26th, 2020
- Artificial Intelligence is Becoming the Future of Investment Platforms - EnterpriseTalk - March 19th, 2020
- Is Your Company Using Artificial Intelligence To Transform An Industry? Nominations For The Forbes 2020 AI 50 List Are Now Open - Forbes - March 19th, 2020
- On the Role of Artificial Intelligence in Genomics to Enhance Precisio | PGPM - Dove Medical Press - March 19th, 2020
- Artificial intelligence myths: Reality check - Livemint - March 19th, 2020
- Battery Researchers Look to Artificial Intelligence to Slash Recharging Times - Greentech Media News - March 19th, 2020
- The Army Will Soon Be Able to Command Robot Tanks With Artificial Intelligence - The National Interest - March 19th, 2020
- Canon Medical's 3T MR System Receives FDA Clearance for Artificial Intelligence-Based Image Reconstruction Technology - BioSpace - March 19th, 2020
- Coronavirus: How Artificial Intelligence, Data Science And Technology Is Used To Fight The Pandemic - Forbes - March 19th, 2020
- Artificial intelligence: The new power dynamic of today - Daily Sabah - March 19th, 2020
- Rethinking Financial Services with Artificial Intelligence Tools - The Financial Brand - March 19th, 2020
- Artificial intelligence recruited to find clues about Covid-19 - The Star Online - March 19th, 2020
- IIT-M to reskill women in artificial intelligence - Campus Varta - March 19th, 2020
- San Diego-Based Company takes Digital Marketing to the next Level by Launching the First Artificial Intelligence Marketing Agency in the United States... - March 19th, 2020
- An Unexpected Ally in the War With Bacteria - The Atlantic - March 19th, 2020
- Compliance For A Digital World: BSA/AML The New ABC's: Artificial Intelligence, Blockchain And How Each Complements The Other - JD Supra - March 19th, 2020
- The next step in digital transformation: is Artificial Intelligence production-ready for green sand foundries? - Foundry-Planet.com - March 19th, 2020
- Insights into the North America Artificial Intelligence in Fashion Market to 2027 - Drivers, Restraints, Opportunities and Trends -... - March 19th, 2020
- Artificial Intelligence by CWI and Amsterdam UMC proposes the best radiation treatment plans in clinical practice for the first time - Centrum... - March 19th, 2020
- H2O.ai Named to the 2020 CB Insights AI 100 List of Most Innovative Artificial Intelligence Startups - Yahoo Finance - March 4th, 2020
- 'There's No Story That Stays Stable for Too Long.' How Artists Are Using Artificial Intelligence to Confront Modern Anxieties - TIME - March 4th, 2020
- Minister declares creation of artificial intelligence centres in Poland - The First News - March 4th, 2020
- Dont forget to consider GDPR when using artificial intelligence in the workplace - ComputerWeekly.com - March 4th, 2020
- Iktos and SRI International Announce Collaboration to Combine Artificial Intelligence and Novel Automated Discovery Platform for Accelerated... - March 4th, 2020
- Why we need to adapt existing EU laws to Artificial Intelligence - European Public Health Alliance - March 4th, 2020
- How a Portland nonprofit is using artificial intelligence to help save whales, giraffes, zebras - Seattle Times - March 4th, 2020
- Artificial Intelligence Infused with Big Data Creating a Tech-driven World - EnterpriseTalk - March 4th, 2020
- Gaming with Artificial Intelligence Technology in 2020 - ReadWrite - March 4th, 2020
- Global Artificial Intelligence-as-a-Service (AIaaS) Market 2020-2024 | Growing Adoption of Cloud Based Solutions to Boost Market Growth | Technavio -... - March 4th, 2020
- WorldMarkets Continues With the Success of Its Trading Artificial Intelligence - Live Bitcoin News - March 4th, 2020
- Why Artificial Intelligence Will Never Beat the Stock Market - Traders Magazine - March 4th, 2020
- Why Neuro-Symbolic Artificial Intelligence Is The AI Of The Future - Digital Trends - January 5th, 2020
- Welcome to the roaring 2020s, the artificial intelligence decade - GreenBiz - January 5th, 2020
- Top five projections in Artificial Intelligence for 2020 - Economic Times - January 5th, 2020
- A reality check on artificial intelligence: Can it match the hype? - PhillyVoice.com - January 5th, 2020
- Illinois regulates artificial intelligence like HireVues used to analyze online job Interviews - Vox.com - January 5th, 2020
- How This Cofounder Created An Artificial Intelligence Styling Company To Help Consumers Shop - Forbes - January 5th, 2020
- The U.S. Patent and Trademark Office Takes on Artificial Intelligence - JD Supra - January 5th, 2020
- Baidu looks to work with Indian institutions on AI - BusinessLine - January 5th, 2020
- Top Movies Of 2019 That Depicted Artificial Intelligence (AI) - Analytics India Magazine - January 5th, 2020
- Shocking ways AI technology will revolutionise every day industries in YOUR lifetime - Express.co.uk - January 5th, 2020
- Artificial intelligence takes scam to a whole new level - The Jackson Sun - January 5th, 2020
- Global Industrial Artificial Intelligence Market 2019 Research by Business Analysis, Growth Strategy and Industry Development to 2024 - Food &... - January 5th, 2020
- IIT Hyderabad to collaborate with Telangana government on artificial intelligence - India Today - January 5th, 2020
- THE AI IN TRANSPORTATION REPORT: How automakers can use artificial intelligence to cut costs, open new revenue - Business Insider India - January 5th, 2020
- Revisiting the rise of A.I.: How far has artificial intelligence come since 2010? - Digital Trends - December 30th, 2019
- Artificial intelligence is helping us talk to animals (yes, really) - Wired.co.uk - December 30th, 2019
- Artificial Intelligence Identifies Previously Unknown Features Associated with Cancer Recurrence - Imaging Technology News - December 30th, 2019
- AI IN BANKING: Artificial intelligence could be a near $450 billion opportunity for banks - here are the strat - Business Insider India - December 30th, 2019
- Artificial Intelligence Is Rushing Into Patient Care - And Could Raise Risks - Scientific American - December 30th, 2019
- Quantum leap: Why we first need to focus on the ethical challenges of artificial intelligence - Economic Times - December 30th, 2019
- In 2020, lets stop AI ethics-washing and actually do something - MIT Technology Review - December 30th, 2019
- The Power Of Purpose: How We Counter Hate Used Artificial Intelligence To Battle Hate Speech Online - Forbes - December 30th, 2019
- The skills needed to land the hottest tech job of 2020 - Business Insider Nordic - December 30th, 2019
- In the 2020s, human-level A.I. will arrive, and finally ace the Turing test - Inverse - December 30th, 2019
- Samsung to announce its Neon artificial intelligence project at CES 2020 - Firstpost - December 30th, 2019
- How Artificial Intelligence Is Totally Changing Everything - HowStuffWorks - December 26th, 2019
- Artificial intelligence jobs on the rise, along with everything else AI - ZDNet - December 26th, 2019
- Why Cognitive Technology May Be A Better Term Than Artificial Intelligence - Forbes - December 26th, 2019