Artificial intelligence is a highly complex technology that, once implemented, requires ongoing oversight to make sure it is doing what is expected of it and ensure it is operating at optimal levels.
Healthcare provider organizations using AI technologies also need to make sure theyre getting the biggest bang for their buck. In other words, they need to optimize the AI so that the technologies are meeting the specific needs of their organizations.
We spoke with six artificial intelligence experts, each with extensive experience in healthcare deployments, who offered comprehensive advice on how CIOs and other health IT workers can optimize their AI systems and approaches to best work for their provider organizations.
Optimizing AI depends on the understanding of what AI is capable of and applying it to the right problem, said Joe Petro, chief technology officer at Nuance Communications, a vendor of AI technology for medical image interpretations.
There is a lot of hype out there, and, unfortunately, the claims are somewhat ridiculous, he said. To optimize AI, we all need to understand: the problem we are trying to solve; how AI can solve the problem; can an existing capability be augmented with AI; and, when AI is not helpful.
Joe Petro, Nuance Communications
For example, is traceability important? AI has a well-known black box limitation every fact or piece of evidence that contributed to a decision or conclusion made by the neural net is not always known.
It is sometimes impossible to trace back through the bread crumb trail leading to the conclusion made by the neural net, Petro explained. Therefore, if traceability is a requirement of the solution, you may need to retreat to a more traditional computational methodology, which is not always a bad thing.
Also, is the problem well-behaved and well-conditioned for AI? For example, he said, are there clear patterns in the solution to the problem that repeat, do not vary widely, and are essentially deterministic.
For example, if you give the problem to a series of experts, will they all arrive at the same answer, he posed. If humans are given the same inputs and disagree on the answer, then AI may not be able to make sense of the data, and the neural nets may deliver results that do not agree with the opinions of certain experts. Rest assured that AI will find a pattern the question is whether or not the pattern is repeatable and consistent.
So in todays world of AI, the problems being solved by AI, especially in healthcare, are deliberately narrowly defined, thereby increasing the accuracy and applicability of AI, Petro explained. Choosing the right problem to solve and narrowing the scope of that problem is key in delivering a great outcome, he advised.
Furthermore, training data needs to be readily available at the volume necessary to create dependable AI models that produce consistently verified results, he added. Unfortunately, sometimes there is no data available in the form that is required to train the neural nets. For example, in some cases, AI requires marked-up and annotated data. This kind of markup is sometimes not available.
When a radiologist reads an image, they may or may not indicate exactly where in the image the diagnosis was made. No data markup makes training sometimes impossible. When a CDI specialist or care coordinator reads through an entire case, they most likely will not indicate every piece of evidence that prompted a query back to a physician.
Again, no data markup makes training sometimes impossible, Petro stated. Therefore, someone needs to go back over the data and potentially add the markup and annotations to train the initial models. Markup is not always necessary, but we need to realize that the data we need is not always available and may need to be expensively curated. The fact is that data is essentially the new software. Without the right data, AI cannot produce the wanted results.
Ken Kleinberg, practice lead, innovative technologies, at consulting firm Point-of-Care Partners, cautioned that AI is being promoted as being able to solve just about any problem that involves a decision.
Many applications that were formerly addressed with proven rules-based or statistical approaches now are routinely being identified as AI targets, he explained. Given the many extra considerations for AI involving model selection, training, validation, the expertise required, etc., this may be overkill. In addition to ROI concerns, using AI may expose organizations to problems unique or common to AI that simpler or alternative approaches are less susceptible to.
Ken Kleinberg, Point-of-Care Partners
Even basic machine learning approaches that may do little more than try out a bunch of different statistical models require some degree of expertise to use, he added.
Considerations of which applications to pick for AI include how many possible variables are in play, known complexities and dependencies, data variability, historical knowledge, availability of content experts, transparency of decision requirements, liability concerns, and how often the system might need to be retrained and tested, Kleinberg advised.
Experience of the model builders and sophistication and investment with an AI platform also should be considered, but AI should not be an expensive hammer looking for a simple nail.
For example, it may be that only a handful of known variables are key to decision making to decide whether an intervention with a patient suffering from a specific condition is needed if the patient has these specific triggers, they are going to be brought in.
Why attempt to train a system on what is already known? he said. Sure, if the goal is to discover unknown nuances or dependencies, or deal with rare conditions, AI could be used with a broader set of variables. For most organizations, they will be safer to go with basic rules-based models where every aspect of the decision can be reviewed and modified as new knowledge is accumulated especially if there are a manageable number of rules, up to a few hundred. That could be a better initial step than going directly to an AI model.
In order to get the most out of an AI investment and optimize the technology for a specific healthcare provider organization, bring in members from across the organization not just the IT team or clinical leadership, said Sanjeev Kumar, vice president of artificial intelligence and data engineering at HMS (Healthcare Management Systems).
Its important to invest time to understand in detailed nuance the full workflow from patient scheduling to check-in to clinical workflow to discharge and billing, he said.
Sanjeev Kumar, HMS
Each member of the team will be able to communicate how AI technology will impact the patient experience from their perspective and how these new, rich insights will impact the everyday office workflow, Kumar said. Without this insight at the beginning of the implementation, you risk investing a significant amount of money in technology that is not used by the staff, that negatively impacts the patient experience or, worst of all, gives inappropriate insights.
Collectively, incorporating staff early on may cause additional investment in manpower, but will result in an output that can be used effectively throughout the organization, he added.
On another technology optimization front, healthcare provider organizations have to be very careful with their data.
Data is precious, and healthcare data is at the outer extreme of sensitive information, said Petro of Nuance Communications. In the process of optimizing AI technology, we need to make sure the AI vendor is a trusted partner that acts as a caretaker of the PHI. We have all heard the horror stories in the press about the misuse of data. This is unacceptable.
Partnering with AI vendors that are legitimate custodians of the data and only use the data within the limitations and constraints of the contract and HIPAA guidelines is a table stakes-governing dynamic, he added.
Make sure to ask the hard questions, he advised. Ask about the use of the data, what is the PHI data flow, how does it move, where does it come to rest, who has access to it, what is it used for, and how long does the vendor keep it. Healthcare AI companies need to be experts in the area of data usage and the limitations around data usage. If a vendor wobbles in these areas, move on.
Another very important consideration for providers optimizing AI technology is the amount of variability in the processes and data they are working with, said Michael Neff, vice president of professional services at Recondo Technology.
For example, clinical AI models created for a population of patients with similar ethnic backgrounds and a small range of ages is most likely simpler than the same model created for an ethnically diverse population, he explained. In the latter population, there will probably be a lot more edge cases, which will either require more training data or will need to be excluded from the model.
Michael Neff, Recondo Technology
If the decision is made to exclude those cases, or if a model is built from a more cohesive data set, it will be very important to limit the use of the AI model to the cases where its predictions are valid, he continued.
The same argument, he said, holds for business variability: A model trained with data sent from a specific payer may not be valid for other payers that a provider works with.
When using AI approaches and especially natural language processing it is key to provide an audit trail to justify recommendations and findings, advised Dr. Elizabeth Marshall, associate director of clinical analytics at Linguamatics IQVIA.
Any insights or features taken from clinical notes and used in AI algorithms need to be easily traced to the exact place in the document they came from, she cautioned. This enables clinical staff to validate the findings and build confidence in AI.
Dr. Elizabeth Marshall, Linguamatics IQVIA
For example, when ensuring a hospital is receiving the right repayment for chronic condition comorbidities such as hepatitis (chronic viral B and C) and HIV/AIDS. It is not only important to capture the data but also to ensure one is able to link the data back to the patients EHR encounter where the information was obtained, she said.
Further, its critical to consider how any insights will be made actionable and incorporated into clinical workflow; having a series of AI algorithms with no way to actually improve patient care is not impactful, Marshall said. For example, clinicians may want to improve the identification of patients who might be missed in a busy emergency department. Time is of the essence, and manually re- reviewing every radiology report, to look for missed opportunities of follow-up, wastes precious time.
Instead, they could use natural language processing to review unstructured sections for critical findings within the reports such as identifying patients with incidental pulmonary nodules, she advised.
When high-risk patients are identified, its critical to have a process in place for appropriate follow-up, she said. To actually improve care, the results need to be flagged in a risk register for follow-up by care coordinators after the patients are no longer in immediate danger.
On the AI spectrum, full replication of human thought is sometimes referred to as strong or full AI. This does not exist yet, certainly not in medicine.
In healthcare, we primarily are focused on narrow or weak AI, which could be described as the use of software or algorithms to complete specific problem-solving or reasoning tasks, at various levels of complexity, said Dr. Ruben Amarasingham, president and CEO of Pieces Technologies. These include specific focal tasks like reading a chest X-ray, interpreting the stage of a skin wound, reading a doctors note and understanding the concerns.
Dr. Ruben Amarasingham, Pieces Technologies
One optimization best practice is to understand how the proposed AI technology is being inserted into the workflow, whether the insertion truly decreases friction in the workflow from the perspective of the stakeholder (provider, patient and family) and effectively measuring and evaluating that value-add immediately after go-live, he said.
If the AI is not reducing stress and complexity of the workflow, it is either not working, not optimized or not worth it, he added.
AI models built by third parties may well serve the local needs of an organization at least as a starting point, but that could be a risk and be an opportunity for optimization, said Kleinberg of Point-of-Care Partners.
As the number of prebuilt models proliferate for example, sepsis, length-of-stay prediction, no-shows it becomes more important to understand the quality of the training and test sets used and attempt to recognize what assumptions and biases the prebuilt model may contain, he said. There has been a ton of recent research on how easily AI particularly deep learning models can be fooled, for example, not paying enough attention to whats in the background of an image. Are the models validated by any independent parties?
Consider an application that recommends the most appropriate type of medical therapy management program for a patient with a goal of increasing medication adherence, Kleinberg advised. To what degree might the test set have been chosen for individuals affected by certain environmental factors (warm versus cold climate), fitness levels, ethnic/economic background, number of medications taken, etc., and how does that compare to the local population to be analyzed, he added.
Retraining and testing the model with a more tuned-to-local demographic data set will be a key practice to achieve more optimized results, he advised.
Amarasingham of Pieces Technologies offered another AI optimization best practice: Health systems should set up governance systems for their AI technology.
I am excited to see the development of AI committees at health systems similar to the development of evidence-based medicine, data governance or clinical order set committees in the recent past, he said. These groups should be involved with evaluating the performance of their AI systems in the healthcare workplace and not rely solely on vendors to oversee their products or services.
They also could be tasked with developing the AI roadmap for their institution over time and as new AI technologies emerge, he added. These committees could be a mix of clinicians, administrators, informaticists and information system team members, he suggested.
Implementing any artificial intelligence technology can require a little more investment than originally anticipated, but if a healthcare organization starts small and plans properly, it will see true returns on that capital and manpower, advised Kumar of HMS.
All healthcare organizations provider, payer and employer will attest that AI has the ability to help transform healthcare operations, he stated. However, AI by itself is not a silver bullet to revolutionizing the system it requires people, process and technology planning, workflow transformation, and time to make sure that it is successful.
This means that to correctly optimize the technology, one needs to go slowly and make sure that one considers all factors that will impact the output from the data going in to how the insights are reported back to providers for action, he said.
In order to ensure that you get the most out of your investment, he concluded, know that you will need to invest more and take longer to see the results.
Twitter:@SiwickiHealthITEmail the writer:bill.siwicki@himssmedia.comHealthcare IT News is a HIMSS Media publication.
View original post here:
Tech Optimization: Getting the most out of AI - Healthcare IT News
- Chinese national arrested and charged with stealing AI trade secrets from Google - NPR - March 8th, 2024 [March 8th, 2024]
- President Biden Calls for Ban on AI Voice Impersonations During State of the Union - Variety - March 8th, 2024 [March 8th, 2024]
- Revolutionize Your Business with AWS Generative AI Competency Partners | Amazon Web Services - AWS Blog - March 8th, 2024 [March 8th, 2024]
- Broadcom Expects AI Demand to Help Offset Weakness Elsewhere - Yahoo Finance - March 8th, 2024 [March 8th, 2024]
- Micron Hits Record High With Analysts Calling It an 'Under-Appreciated AI Beneficiary' - Investopedia - March 8th, 2024 [March 8th, 2024]
- The Adams administration quietly hired its first AI czar. Who is he? - City & State New York - March 8th, 2024 [March 8th, 2024]
- AI likely to increase energy use and accelerate climate misinformation report - The Guardian - March 8th, 2024 [March 8th, 2024]
- This Artificial Intelligence (AI) Stock Could Double, and It Is Way Cheaper Than Nvidia - Yahoo Finance - March 8th, 2024 [March 8th, 2024]
- Fake images made to show Trump with Black supporters highlight concerns around AI and elections - The Associated Press - March 8th, 2024 [March 8th, 2024]
- Artificial intelligence and illusions of understanding in scientific research - Nature.com - March 8th, 2024 [March 8th, 2024]
- Analysis | House AI task force leaders take long view on regulating the tools - The Washington Post - March 8th, 2024 [March 8th, 2024]
- Don't Give Your Business Data to AI Companies - Dark Reading - March 8th, 2024 [March 8th, 2024]
- NIST, the lab at the center of Bidens AI safety push, is decaying - The Washington Post - March 8th, 2024 [March 8th, 2024]
- Essay | AI is Coming! Tips for Staying Calm and Carrying On - The Wall Street Journal - March 8th, 2024 [March 8th, 2024]
- AI can be easily used to make fake election photos - report - BBC.com - March 8th, 2024 [March 8th, 2024]
- 5 Artificial Intelligence (AI) Stocks That Could Make You a Millionaire - Yahoo Finance - March 8th, 2024 [March 8th, 2024]
- AI could be an extraordinary force for good. So why do our politicians still not have a plan? - The Guardian - March 8th, 2024 [March 8th, 2024]
- Mapping Disease Trajectories from Birth to Death with AI - Neuroscience News - March 8th, 2024 [March 8th, 2024]
- India plans 10,000-GPU sovereign AI supercomputer - The Register - March 8th, 2024 [March 8th, 2024]
- SAP enhances Datasphere and SAC for AI-driven transformation - CIO - March 8th, 2024 [March 8th, 2024]
- Jim Cramer names companies and sectors poised to rally on the AI wave - CNBC - March 8th, 2024 [March 8th, 2024]
- The job applicants shut out by AI: The interviewer sounded like Siri - The Guardian - March 8th, 2024 [March 8th, 2024]
- Microsoft confirms Surface and Windows AI event for March 21st - The Verge - March 8th, 2024 [March 8th, 2024]
- Adobes new Express app brings Firefly AI tools to iOS and Android - The Verge - March 8th, 2024 [March 8th, 2024]
- A Google AI Watched 30,000 Hours of Video GamesNow It Makes Its Own - Singularity Hub - March 8th, 2024 [March 8th, 2024]
- Palantir CEO Karp on TITAN, AI Warfare Technology - Bloomberg - March 8th, 2024 [March 8th, 2024]
- Elliptic Curve Murmurations Found With AI Take Flight - Quanta Magazine - March 8th, 2024 [March 8th, 2024]
- 5 AI Stocks to Buy in March 2024, According to Analysts - TipRanks.com - TipRanks - March 8th, 2024 [March 8th, 2024]
- Wix's new AI chatbot builds websites in seconds based on prompts - The Verge - March 8th, 2024 [March 8th, 2024]
- Amid record high energy demand, America is running out of electricity - The Washington Post - March 8th, 2024 [March 8th, 2024]
- AI Crypto Tokens in 5 Minutes: What to Know and Where to Start - Inc. - February 26th, 2024 [February 26th, 2024]
- 'The Worlds I See' by AI visionary Fei-Fei Li '99 selected as Princeton Pre-read - Princeton University - February 26th, 2024 [February 26th, 2024]
- AI is having a 1995 moment, analyst says - Business Insider - February 26th, 2024 [February 26th, 2024]
- Vatican research group's book outlines AI's 'brave new world' - National Catholic Reporter - February 26th, 2024 [February 26th, 2024]
- Honor's Magic 6 Pro launches internationally with AI-powered eye tracking on the way - The Verge - February 26th, 2024 [February 26th, 2024]
- Google explains Gemini's embarrassing AI pictures of diverse Nazis - The Verge - February 26th, 2024 [February 26th, 2024]
- Google cut a deal with Reddit for AI training data - The Verge - February 26th, 2024 [February 26th, 2024]
- What's the point of Elon Musk's AI company? - The Verge - February 26th, 2024 [February 26th, 2024]
- AI agents like Rabbit aim to book your vacation and order your Uber - NPR - February 26th, 2024 [February 26th, 2024]
- Announcing Microsofts open automation framework to red team generative AI Systems - Microsoft - February 26th, 2024 [February 26th, 2024]
- After Nvidia's latest blowout, here are 20 AI stocks expected to rise as much as 44% - Yahoo Finance - February 26th, 2024 [February 26th, 2024]
- 1 Exceptional AI Chip Stock Investors Need to Know About in 2024 - The Motley Fool - February 26th, 2024 [February 26th, 2024]
- Nvidia briefly hits $2 trillion valuation as AI frenzy grips Wall Street - Reuters - February 26th, 2024 [February 26th, 2024]
- AI Chatbots Can Guess Your Personal Information From What You ... - WIRED - October 18th, 2023 [October 18th, 2023]
- Harvard IT Launches Pilot of AI Sandbox to Enable Walled-Off Use ... - Harvard Crimson - October 18th, 2023 [October 18th, 2023]
- Advancing policing through AI: Insights from the global law ... - Police News - October 18th, 2023 [October 18th, 2023]
- Hochul announces new SUNY, IBM investments in AI - Olean Times Herald - October 18th, 2023 [October 18th, 2023]
- Nvidia's banking on TensorRT to expand its generative AI dominance - The Verge - October 18th, 2023 [October 18th, 2023]
- AI expands from MRFs to vehicles - Plastics Recycling Update - October 18th, 2023 [October 18th, 2023]
- AI Reads Ancient Scroll Charred by Mount Vesuvius in Tech First - Scientific American - October 18th, 2023 [October 18th, 2023]
- A DEEPer (squared) dive into AI Harvard Gazette - Harvard Gazette - October 18th, 2023 [October 18th, 2023]
- Florida bar weighs whether lawyers using AI need client consent - Reuters - October 18th, 2023 [October 18th, 2023]
- Cognizant and Vianai Systems Announce Strategic Partnership to ... - PR Newswire - October 18th, 2023 [October 18th, 2023]
- How AI could speed up scientific discoveries, from proteins to ... - NPR - October 18th, 2023 [October 18th, 2023]
- AI challenge to deliver better healthcare | Western Australian ... - Government of Western Australia - October 18th, 2023 [October 18th, 2023]
- Henry Kissinger: The Path to AI Arms Control - Foreign Affairs Magazine - October 18th, 2023 [October 18th, 2023]
- Stability AI releases StableStudio in latest push for open-source AI - The Verge - May 18th, 2023 [May 18th, 2023]
- Google CEO Sundar Pichai Predicts That This Profession Will Be ... - The Motley Fool - May 18th, 2023 [May 18th, 2023]
- Frances privacy watchdog eyes protection against data scraping in AI action plan - TechCrunch - May 18th, 2023 [May 18th, 2023]
- Investing in Hippocratic AI - Andreessen Horowitz - May 18th, 2023 [May 18th, 2023]
- As Alphabet flexes its AI prowess, there's a 'new elephant in the room' for Google - MarketWatch - May 18th, 2023 [May 18th, 2023]
- The Boring Future of Generative AI | WIRED - WIRED - May 18th, 2023 [May 18th, 2023]
- OpenAI readies new open-source AI model, The Information reports - Reuters.com - May 18th, 2023 [May 18th, 2023]
- What every CEO should know about generative AI - McKinsey - May 18th, 2023 [May 18th, 2023]
- AI creates images of the 'perfect' man and woman - Sky News - May 18th, 2023 [May 18th, 2023]
- Audit AI search tools now, before they skew research - Nature.com - May 18th, 2023 [May 18th, 2023]
- 3 Reasons C3.ai Stock Could Be Your Golden Ticket to the AI ... - InvestorPlace - May 18th, 2023 [May 18th, 2023]
- Zoom makes a big bet on AI with investment in Anthropic - VentureBeat - May 18th, 2023 [May 18th, 2023]
- AI voice phone scams are on the rise. Here's how to avoid them - USA TODAY - May 18th, 2023 [May 18th, 2023]
- Amazon is building an AI-powered conversational experience for ... - The Verge - May 18th, 2023 [May 18th, 2023]
- AI speculators need to 'differentiate between actual spending and investment' and hype: Strategist - Yahoo Finance - May 18th, 2023 [May 18th, 2023]
- AI Can Be Both Accurate and Transparent - HBR.org Daily - May 18th, 2023 [May 18th, 2023]
- You're Probably Underestimating AI Chatbots | WIRED - WIRED - May 18th, 2023 [May 18th, 2023]
- AI presents political peril for 2024 with threat to mislead voters - The Associated Press - May 18th, 2023 [May 18th, 2023]
- We need AI to help us face the challenges of the future - The Guardian - May 18th, 2023 [May 18th, 2023]
- End Of Googles Dominance? Stock Gets Rare Analyst Downgrade Over AI Fears - Forbes - May 18th, 2023 [May 18th, 2023]
- Watch 44 million atoms simulated using AI and a supercomputer - New Scientist - May 18th, 2023 [May 18th, 2023]
- AI Is The New Electricity: Bank Of America Picks 20 Stocks To Cash In On ChatGPT Hype - Forbes - March 2nd, 2023 [March 2nd, 2023]
- Tech Giants Are Barreling Headfirst Into an AI Arms Race - February 20th, 2023 [February 20th, 2023]
- Bing's AI Is Threatening Users. That's No Laughing Matter - TIME - February 20th, 2023 [February 20th, 2023]