Dr. Tanuj Gupta, vice president at Cerner Intelligence, is an expert in healthcare artificial intelligence and machine learning. Part of his job is explaining, from his expert point of view, what he considers misconceptions with AI, especially misconceptions in healthcare.
In this interview with Healthcare IT News, Gupta discusses what he says are popular misconceptions with gender and racial bias in algorithms, AI replacing clinicians, and the regulation of AI in healthcare.
Q. In general terms, why do you think there are misconceptions about AI in healthcare, and why do they persist?
A. I've given more than 100 presentations on AI and ML in the past year. There's no doubt these technologies are hot topics in healthcare that usher in great hope for the advancement of our industry.
While they have the potential to transform patient care, quality and outcomes, there also are concerns about the negative impact this technology could have on human interaction, as well as the burden they could place on clinicians and health systems.
Q. Should we be concerned about gender and racial bias in ML algorithms?
A. Traditionally, healthcare providers consider a patient's unique situation when making decisions, along with information sources, such as their clinical training and experiences, as well as published medical research.
Now, with ML, we can be more efficient and improve our ability to examine large amounts of data, flag potential problems and suggest next steps for treatment. While this technology is promising, there are some risks. Although AI and ML are just tools, they have many points of entry that are vulnerable to bias, from inception to end use.
As ML learns and adapts, it's vulnerable to potentially biased input and patterns. Existing prejudices especially if they're unknown and data that reflects societal or historical inequities can result in bias being baked into the data that's used to train an algorithm or ML model to predict outcomes. If not identified and mitigated, clinical decision-making based on bias could negatively impact patient care and outcomes. When bias is introduced into an algorithm, certain groups can be targeted unintentionally.
Gender and racial biases have been identified in commercial facial-recognition systems, which are known to falsely identify Black and Asian faces 10 to 100 times more than Caucasian faces, and have more difficulty identifying women than men. Bias is also seen in natural language processing that identifies topic, opinion and emotion.
If the systems in which our AI and ML tools are developed or implemented are biased, then their resulting health outcomes can be biased, which can perpetuate health disparities. While breaking down systemic bias can be challenging, it's important that we do all we can to identify and correct it in all its manifestations. This is the only way we can optimize AI and ML in healthcare and ensure the highest quality of patient experience.
Q. Could AI replace clinicians?
A. The short answer is no. AI and ML will not replace clinician judgement. Providers will always have to be involved in the decision-making process, because we hold them accountable for patient care and outcomes.
We already have some successful guardrails in other areas of healthcare that we'll likely evolve to for AI and ML. For example, one parallel is verbal orders. If a doctor gives a nurse a verbal order for a medication, the nurse repeats it back to them before entering it in the chart, and the doctor must sign off on it. If that medication ends up causing harm to the patient, the doctor can't say the nurse is at fault.
Additionally, any standing protocol orders that a hospital wants to institute must be approved by a committee of physicians who then have a regular review period to ensure the protocols are still safe and effective. That way, if the nurse executes a protocol order and there's a patient-safety issue, that medical committee is responsible and accountable not the nurse.
The same thing is going to be there with AI and ML algorithms. There won't be an algorithm that arbitrarily runs on a tool or machine, treating a patient without doctor oversight.
If we throw a bunch of algorithms into the electronic health record that say, "treat the patient this way" or "diagnose him with this," we'll have to hold the clinician and possibly the algorithm maker if it becomes regulated by the U.S. Food and Drug Administration accountable for the outcomes. I can't imagine a situation where that would change.
Clinicians can use,and are using, AI and ML to improve care and maybe make healthcare even more human than it is today. AI and ML could also allow physicians to enhance the quality of time spent with patients.
Bottom line, I think we as the healthcare industry should embrace AI and ML technology. It won't replace us; it will just become a new and effective toolset to use with our patients. And using this technology responsibly means always staying on top of any potential patient safety risks.
Q. What should we know about the regulation of AI in healthcare?
A. AI introduces some important concerns around data ownership, safety and security. Without a standard for how to handle these issues, there's the potential to cause harm, either to the healthcare system or to the individual patient.
For these reasons, important regulations should be expected. The pharmaceutical, clinical treatment and medical device industries provide a precedent for how to protect data rights, privacy, and security, and drive innovation in an AI-empowered healthcare system.
Let's start with data rights. When people use an at-home DNA testing kit, they likely gave broad consent for your data to be used for research purposes, as defined by the U.S. Department of Health and Human Services in a 2017 guidance document.
While that guidance establishes rules for giving consent, it also creates the process for withdrawing consent. Handling consent in an AI-empowered healthcare system may be a challenge, but there's precedent for thinking through this issue to both protect rights and drive innovation.
With regard to patient safety concerns, the Food and Drug Administration has published two documents to address the issue: Draft Guidance on Clinical Decision Support Software and Draft Guidance on Software as a Medical Device. The first guidance sets a framework for determining if an ML algorithm is a medical device.
Once you've determined your ML algorithm is in fact a device, the second guidance provides "good machine learning practices." Similar FDA regulations on diagnostics and therapeutics have kept us safe from harm without getting in the way of innovation. We should expect the same outcome for AI and ML in healthcare.
Finally, let's look at data security and privacy. The industry wants to protect data privacy while unlocking more value in healthcare. For example, HHS has long relied on the Health Insurance Portability and Accountability Act, which was signed into law in 1996.
While HIPAA is designed to safeguard protected health information, growing innovation in healthcare particularly regarding privacy led to HHS' recently issued proposed rule to prevent information blocking and encourage healthcare innovation.
It's safe to conclude that AI and ML in healthcare will be regulated. But that doesn't mean these tools won't be useful. In fact, we should expect the continued growth of AI applications for healthcare as more uses and benefits of the technology surface.
Twitter:@SiwickiHealthITEmail the writer:bsiwicki@himss.orgHealthcare IT News is a HIMSS Media publication.
Read the rest here:
Cerner AI expert discusses important 'misconceptions' about the technology - Healthcare IT News
- Classic reasoning systems like Loom and PowerLoom vs. more modern systems based on probalistic networks - November 8th, 2009 [November 8th, 2009]
- Using Amazon's cloud service for computationally expensive calculations - November 8th, 2009 [November 8th, 2009]
- Software environments for working on AI projects - November 8th, 2009 [November 8th, 2009]
- New version of my NLP toolkit - November 8th, 2009 [November 8th, 2009]
- Semantic Web: through the back door with HTML and CSS - November 8th, 2009 [November 8th, 2009]
- Java FastTag part of speech tagger is now released under the LGPL - November 8th, 2009 [November 8th, 2009]
- Defining AI and Knowledge Engineering - November 8th, 2009 [November 8th, 2009]
- Great Overview of Knowledge Representation - November 8th, 2009 [November 8th, 2009]
- Something like Google page rank for semantic web URIs - November 8th, 2009 [November 8th, 2009]
- My experiences writing AI software for vehicle control in games and virtual reality systems - November 8th, 2009 [November 8th, 2009]
- The URL for this blog has changed - November 8th, 2009 [November 8th, 2009]
- I have a new page on Knowledge Management - November 8th, 2009 [November 8th, 2009]
- N-GRAM analysis using Ruby - November 8th, 2009 [November 8th, 2009]
- Good video: Knowledge Representation and the Semantic Web - November 8th, 2009 [November 8th, 2009]
- Using the PowerLoom reasoning system with JRuby - November 8th, 2009 [November 8th, 2009]
- Machines Like Us - November 8th, 2009 [November 8th, 2009]
- RapidMiner machine learning, data mining, and visualization tool - November 8th, 2009 [November 8th, 2009]
- texai.org - November 8th, 2009 [November 8th, 2009]
- NLTK: The Natural Language Toolkit - November 8th, 2009 [November 8th, 2009]
- My OpenCalais Ruby client library - November 8th, 2009 [November 8th, 2009]
- Ruby API for accessing Freebase/Metaweb structured data - November 8th, 2009 [November 8th, 2009]
- Protégé OWL Ontology Editor - November 8th, 2009 [November 8th, 2009]
- New version of Numenta software is available - November 8th, 2009 [November 8th, 2009]
- Very nice: Elsevier IJCAI AI Journal articles now available for free as PDFs - November 8th, 2009 [November 8th, 2009]
- Verison 2.0 of OpenCyc is available - November 8th, 2009 [November 8th, 2009]
- What’s Your Biggest Question about Artificial Intelligence? [Article] - November 8th, 2009 [November 8th, 2009]
- Minimax Search [Knowledge] - November 8th, 2009 [November 8th, 2009]
- Decision Tree [Knowledge] - November 8th, 2009 [November 8th, 2009]
- More AI Content & Format Preference Poll [Article] - November 8th, 2009 [November 8th, 2009]
- New Planners Solve Rescue Missions [News] - November 8th, 2009 [November 8th, 2009]
- Neural Network Learns to Bluff at Poker [News] - November 8th, 2009 [November 8th, 2009]
- Pushing the Limits of Game AI Technology [News] - November 8th, 2009 [November 8th, 2009]
- Mining Data for the Netflix Prize [News] - November 8th, 2009 [November 8th, 2009]
- Interview with Peter Denning on the Principles of Computing [News] - November 8th, 2009 [November 8th, 2009]
- Decision Making for Medical Support [News] - November 8th, 2009 [November 8th, 2009]
- Neural Network Creates Music CD [News] - November 8th, 2009 [November 8th, 2009]
- jKilavuz - a guide in the polygon soup [News] - November 8th, 2009 [November 8th, 2009]
- Artificial General Intelligence: Now Is the Time [News] - November 8th, 2009 [November 8th, 2009]
- Apply AI 2007 Roundtable Report [News] - November 8th, 2009 [November 8th, 2009]
- What Would You do With 80 Cores? [News] - November 8th, 2009 [November 8th, 2009]
- Software Finds Learning Language Child's Play [News] - November 8th, 2009 [November 8th, 2009]
- Artificial Intelligence in Games [Article] - November 8th, 2009 [November 8th, 2009]
- Artificial Intelligence Resources - November 8th, 2009 [November 8th, 2009]
- Alan Turing: Mathematical Biologist? - April 25th, 2012 [April 25th, 2012]
- BBC Horizon: The Hunt for AI ( Artificial Intelligence ) - Video - April 30th, 2012 [April 30th, 2012]
- Can computers have true artificial intelligence" Masonic handshake" 3rd-April-2012 - Video - April 30th, 2012 [April 30th, 2012]
- Kevin B. Korb - Interview - Artificial Intelligence and the Singularity p3 - Video - April 30th, 2012 [April 30th, 2012]
- Artificial Intelligence - 6 Month Anniversary - Video - April 30th, 2012 [April 30th, 2012]
- Science Breakthroughs - April 30th, 2012 [April 30th, 2012]
- Hitman: Blood Money - Part 49 - Stupid Artificial Intelligence! - Video - April 30th, 2012 [April 30th, 2012]
- Research Members Turned Off By HAARP Artificial Intelligence - Video - April 30th, 2012 [April 30th, 2012]
- Artificial Intelligence Lecture No. 5 - Video - April 30th, 2012 [April 30th, 2012]
- The Artificial Intelligence Laboratory, 2012 - Video - April 30th, 2012 [April 30th, 2012]
- Charlie Rose - Artificial Intelligence - Video - April 30th, 2012 [April 30th, 2012]
- Expert on artificial intelligence to speak at EPIIC Nights dinner - May 4th, 2012 [May 4th, 2012]
- Filipino software engineers complete and best thousands on Stanford’s Artificial Intelligence Course - May 4th, 2012 [May 4th, 2012]
- Vodafone xone™ Hackathon Challenges Developers and Entrepreneurs to Build a New Generation of Artificial Intelligence ... - May 4th, 2012 [May 4th, 2012]
- Rocket Fuel Packages Up CPG Booster - May 4th, 2012 [May 4th, 2012]
- 2 Filipinos finishes among top in Stanford’s Artificial Intelligence course - May 5th, 2012 [May 5th, 2012]
- Why Your Brain Isn't A Computer - May 5th, 2012 [May 5th, 2012]
- 2 Pinoy software engineers complete Stanford's AI course - May 7th, 2012 [May 7th, 2012]
- Percipio Media, LLC Proudly Accepts Partnership With MIT's Prestigious Computer Science And Artificial Intelligence ... - May 10th, 2012 [May 10th, 2012]
- Google Driverless Car Ok'd by Nevada - May 10th, 2012 [May 10th, 2012]
- Moving Beyond the Marketing Funnel: Rocket Fuel and Forrester Research Announce Free Webinar - May 10th, 2012 [May 10th, 2012]
- Rocket Fuel Wins 2012 San Francisco Business Times Tech & Innovation Award - May 13th, 2012 [May 13th, 2012]
- Internet Week 2012: Rocket Fuel to Speak at OMMA RTB - May 16th, 2012 [May 16th, 2012]
- How to Get the Most Out of Your Facebook Ads -- Rocket Fuel's VP of Products, Eshwar Belani, to Lead MarketingProfs ... - May 16th, 2012 [May 16th, 2012]
- The Digital Disruptor To Banking Has Just Gone International - May 16th, 2012 [May 16th, 2012]
- Moving Beyond the Marketing Funnel: Rocket Fuel Announce Free Webinar Featuring an Independent Research Firm - May 23rd, 2012 [May 23rd, 2012]
- MASA Showcases Latest Version of MASA SWORD for Homeland Security Markets - May 23rd, 2012 [May 23rd, 2012]
- Bluesky Launches Drones for Aerial Surveying - May 23rd, 2012 [May 23rd, 2012]
- Artificial Intelligence: What happened to the hunt for thinking machines? - May 25th, 2012 [May 25th, 2012]
- Bubble Robots Move Using Lasers [VIDEO] - May 25th, 2012 [May 25th, 2012]
- UHV assistant professors receive $10,000 summer research grants - May 27th, 2012 [May 27th, 2012]
- Artificial intelligence: science fiction or simply science? - May 28th, 2012 [May 28th, 2012]
- Exetel taps artificial intelligence - May 29th, 2012 [May 29th, 2012]
- Software offers brain on the rain - May 29th, 2012 [May 29th, 2012]
- New Dean of Science has high hopes for his faculty - May 30th, 2012 [May 30th, 2012]
- Cognitive Code Announces "Silvia For Android" App - May 31st, 2012 [May 31st, 2012]
- A Rat is Smarter Than Google - June 5th, 2012 [June 5th, 2012]