While CIM can speed up multiplication operations, it comes with added risk and complexity.
Compute-in-memory (CIM) is not necessarily an Artificial Intelligence (AI) solution; rather, it is a memory management solution. CIM could bring advantages to AI processing by speeding up the multiplication operation at the heart of AI model execution. However, for that to be successful, an AI processing system would need to be explicitly architected to use CIM. The change would entail a shift from all-digital design workflows into a mixed-signal approach which would require deep design expertise and specialized semiconductor fabrication processes.
Compute-in-memory eliminates weight coefficient buffers and streamlines the primitive multiply operations, striving for increased AI inference throughput. However, it does not perform neural network processing by itself. Other functions like input data streaming, sequencing, accumulation buffering, activation buffering, and layer organization may become more vital factors in overall performance as model hardware mapping unfolds and complexity increases more robust NPUs (Neural Processing Units) incorporate all those functions.
Fundamentally, compute-in-memory embeds a multiplier unit in a memory unit. A conventional digital multiplier takes two operands as digital words and produces a digital result, handling signing and scaling. Compute-in-memory uses a different approach, storing a weight coefficient as analog values in a specially designed transistor cell sub-array with rows and columns. The incoming digital data words enter the rows of the array, triggering analog voltage multiplies, then analog current summations occur along columns. An analog-to-digital converter creates the final digital word outputs from the summed analog values.
An individual memory cell can be straightforward in theory, such as these candidates:
Still, operating these cells presents mixed-signal challenges and a technology gap that is not closing anytime soon. So, why the intense interest in compute-in-memory for AI inference chips?
First, it can be fast. This is because analog multiplication happens quickly as part of the memory read cycle, transparent to the rest of the surrounding digital logic. It can also be lower power since fewer transistors switch at high frequencies. But there are some limitations from a system viewpoint. Additional steps needed for programming the analog values into the memory cells are a concern. Inaccuracy of the analog voltages, which may change over time, can inject bit errors into results showing up as detection errors or false alarm rates.
Aside from its analog nature, the biggest concern for compute-in-memory may be bit precision and AI training requirements. Researchers seem confident in 4-bit implementations; however, more training cycles must be run for reliable inference at low precision. Raising the precision to 8-bit lowers training demands. It also increases the complexity of the arrays and the analog-to-digital converter for each array, offsetting area and power savings and worsening the chance for bit errors in the presence of system noise.
So is compute-in-memory worthy of consideration? There likely are niche applications where it could speed up AI inference. A more critical question: is the added risk and complexity of compute-in-memory worth the effort? A well-conceived NPU strategy and implementation may nullify any advantage of moving to compute-in-memory. We can contrast the tradeoffs for AI inference in four areas: power/performance/area (PPA), flexibility, quantization, and memory technology.
PPA
Flexibility
Quantization
Memory Technology
The answer to the original question might be that designers should consider CIM only if other, more established AI inference platforms (NPUs) cannot meet their requirements. Since CIM is riskier, costlier, and harder to implement, many should only consider it a last-resort solution.
Expedera explores this topic in much more depth in a recent white paper, which can be found at: https://www.expedera.com/architectural-considerations-for-compute-in-memory-in-ai-inference/
Read this article:
Can Compute-In-Memory Bring New Benefits To Artificial ... - SemiEngineering
- Use of Artificial Intelligence in the Prediction of Chiari Malformation Type 1 Recurrence After Posterior Fossa ... - Cureus - May 23rd, 2024 [May 23rd, 2024]
- Fritz Lang First Depicted Artificial Intelligence on Film in Metropolis (1927), and It Frightened People Even Then - Open Culture - May 23rd, 2024 [May 23rd, 2024]
- Dehumanizing the Wars: Introduction of Artificial Intelligence in the Warfare - Modern Diplomacy - May 23rd, 2024 [May 23rd, 2024]
- Google Parent Alphabet Is Selling Shares of 2 Hypergrowth Stocks and Piling Into a Promising Artificial Intelligence (AI ... - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- Meta AI Head: ChatGPT Will Never Reach Human Intelligence - PYMNTS.com - May 23rd, 2024 [May 23rd, 2024]
- Waiting for Alexa: When Will Amazon Strike Up the AI Conversation? - PYMNTS.com - May 23rd, 2024 [May 23rd, 2024]
- Artificial intelligence town halls? House committee weighs new approach before writing AI rules - Washington Times - May 23rd, 2024 [May 23rd, 2024]
- Inventing AI: Tracing the diffusion of artificial intelligence with U.S. patents - United States Patent and Trademark Office - May 23rd, 2024 [May 23rd, 2024]
- UM6P to Host High-Level Forum on Artificial Intelligence, Development in Africa - Morocco World News - May 23rd, 2024 [May 23rd, 2024]
- DHS warns of threats to election posed by artificial intelligence - ABC News - May 23rd, 2024 [May 23rd, 2024]
- Artificial intelligence, great wealth transfer: Why this financial analyst sees a rosy time ahead for stocks - CNBC - May 23rd, 2024 [May 23rd, 2024]
- As S&P 500 reaches fresh highs, experts say generative artificial intelligence will create new investment opportunities - CNBC - May 23rd, 2024 [May 23rd, 2024]
- Google Parent Alphabet Is Selling Shares of 2 Hypergrowth Stocks and Piling Into a Promising Artificial Intelligence (AI ... - MSN - May 23rd, 2024 [May 23rd, 2024]
- The Best Artificial Intelligence (AI) ETF to Invest $1,000 in Right Now - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- How To Land A Job In Artificial Intelligence (AI) - Forbes - May 23rd, 2024 [May 23rd, 2024]
- Madhumita Murgia On The Good, Bad And Ugly Of Artificial Intelligence In Our Daily Lives - Forbes India - May 23rd, 2024 [May 23rd, 2024]
- Researchers use artificial intelligence to classify brain tumors - Voice of America - VOA News - May 23rd, 2024 [May 23rd, 2024]
- BBVA steps up its plans in artificial intelligence by signing an agreement with OpenAI - BBVA - May 23rd, 2024 [May 23rd, 2024]
- 2 Artificial Intelligence (AI) Stocks You Can Buy and Hold for the Next Decade - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- Study including WVU and Marshall analyzes cyber threats to Artificial Intelligence systems - West Virginia MetroNews - May 23rd, 2024 [May 23rd, 2024]
- Stock-Split Watch: 5 Artificial Intelligence (AI) Stocks That Look Ready to Split - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- 2 Spectacular ETFs That Can Help You Capitalize on the Artificial Intelligence (AI) Boom - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- NASA pledging responsible use of artificial intelligence | National | khq.com - KHQ Right Now - May 23rd, 2024 [May 23rd, 2024]
- 1 Stock Down 11% to Buy Before Artificial Intelligence (AI) Could Supercharge Its Growth and Send It Soaring - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- Alphabet Goes All In on Artificial Intelligence (AI). Is It Time to Go All In on the Stock? - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- A Once-in-a-Generation Investment Opportunity: Here's 1 Artificial Intelligence (AI) Stock to Buy in May and Hold Forever - Yahoo Finance - May 23rd, 2024 [May 23rd, 2024]
- 2 Artificial Intelligence (AI) Stocks That Offer Dividends and Long-Term Returns - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- Forget Nvidia: Billionaires Are Selling It and Buying These 2 Supercharged Artificial Intelligence (AI) Growth Stocks ... - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- 1 Ridiculously Undervalued Artificial Intelligence (AI) Stock to Buy Hand Over Fist - Yahoo Finance - May 23rd, 2024 [May 23rd, 2024]
- Missed Out on Nvidia's Ginormous Gains? Here's an Artificial Intelligence (AI) Stock You Can Buy Right Now. - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- Researchers create innovative verification techniques to increase security in artificial intelligence and image processing - European Research Council - April 29th, 2024 [April 29th, 2024]
- Who in Europe is investing the most in artificial intelligence? - Euronews - April 29th, 2024 [April 29th, 2024]
- 4 Artificial Intelligence (AI) Stocks Members of Congress Can't Stop Buying (and Nvidia Isn't 1 of Them!) - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- The Standard Names Porter Orr Second Vice President of Artificial Intelligence Strategy and Development - NRToday.com - April 29th, 2024 [April 29th, 2024]
- Meet Nvidia CEO Jensen Huang, the man behind the $2 trillion company powering today's artificial intelligence - CBS News - April 29th, 2024 [April 29th, 2024]
- This Warren Buffett Dividend King Stock Just Invested $1.1 Billion Into Artificial Intelligence (AI) - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- Joe Rogan Reveals What Could Be 'Game Over for the Human Race' - Newsweek - April 29th, 2024 [April 29th, 2024]
- LegalTech Artificial Intelligence Market to Soar to $6.4 Billion by 2028: Global Long-term Forecast to 2033 - Strategic ... - Yahoo Finance UK - April 29th, 2024 [April 29th, 2024]
- Artificial Intelligence Tokens Stumble Again! Is The AI Hype Over? - Coinpedia Fintech News - April 29th, 2024 [April 29th, 2024]
- CISA unveils guidelines for AI and critical infrastructure - FedScoop - April 29th, 2024 [April 29th, 2024]
- IBM's Webinar: Cybersecurity In The Era Of Artificial Intelligence Keynotes - AiThority - April 29th, 2024 [April 29th, 2024]
- Being and becoming a good doctor in the age of artificial intelligence - Firstpost - April 29th, 2024 [April 29th, 2024]
- Pope to take part in G7 summit in June to talk Artificial Intelligence - Crux Now - April 29th, 2024 [April 29th, 2024]
- Pope Francis to attend G7 summit to speak on artificial intelligence - Catholic News Agency - April 29th, 2024 [April 29th, 2024]
- 1 Unstoppable Artificial Intelligence (AI) Stock to Buy and Hold Forever - Yahoo Finance - April 29th, 2024 [April 29th, 2024]
- Pope to bring his call for ethical artificial intelligence to G7 summit in June in southern Italy - The Associated Press - April 29th, 2024 [April 29th, 2024]
- Elon Musk Called Tesla an Artificial Intelligence (AI) Robotics Company. Does That Make It a Buy? - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- Emulating neurodegeneration and aging in artificial intelligence systems - Tech Xplore - April 29th, 2024 [April 29th, 2024]
- Pope Francis to participate in G7 session on AI - Vatican News - English - April 29th, 2024 [April 29th, 2024]
- A Once-in-a-Generation Investment Opportunity: 1 Bill Ackman Artificial Intelligence (AI) Stock to Buy Hand Over Fist ... - Yahoo Finance - April 29th, 2024 [April 29th, 2024]
- Tesla Stock Investors Cheer Progress on Its Artificial Intelligence (AI) Future - Yahoo Finance - April 29th, 2024 [April 29th, 2024]
- In Race to Build A.I., Tech Plans a Big Plumbing Upgrade - The New York Times - April 29th, 2024 [April 29th, 2024]
- A Baltimore-area teacher is accused of using AI to make his boss appear racist - NPR - April 29th, 2024 [April 29th, 2024]
- 3 Top Artificial Intelligence (AI) Stocks That Billionaires Jim Simons, Ray Dalio, and Israel Englander Are Buying - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- The iPad Pro can give us a big surprise with its artificial intelligence chips. - Softonic EN - April 29th, 2024 [April 29th, 2024]
- This Artificial Intelligence (AI) Stock Could Soar 70%, According to Wall Street. Time to Buy? - Yahoo Finance - April 29th, 2024 [April 29th, 2024]
- Billionaire Bill Ackman Owns 8 Stocks -- and This Hypergrowth Artificial Intelligence (AI) Stock Isn't One of Them - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- Artificial Intelligence Has Come for Our...Beauty Pageants? - Glamour - April 29th, 2024 [April 29th, 2024]
- 3 Stocks to Grab Now to Ride the Artificial Intelligence Chip Boom to Riches - InvestorPlace - April 29th, 2024 [April 29th, 2024]
- Is ASML's Big Sell-off a Warning Sign to Artificial Intelligence (AI) Investors? - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- 3 Artificial Intelligence (AI) Stocks to Buy With $1,150 and Hold for Decades - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- This combines the power of artificial intelligence with human insights - finews.com - March 22nd, 2024 [March 22nd, 2024]
- Cohere Targets $5 Billion Valuation for ChatGPT Rival - PYMNTS.com - March 22nd, 2024 [March 22nd, 2024]
- IMF: Artificial Intelligence (AI) Will Transform 40% of Jobs. Can Investors Capitalize? - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- Cathie Wood Is Selling These 2 Artificial Intelligence (AI) Stocks. Should You? - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- iShares Robotics and Artificial Intelligence Multisector ETF (NYSEARCA:IRBO) Shares Acquired by Creative Financial ... - Defense World - March 22nd, 2024 [March 22nd, 2024]
- Inaugural Plenary Meeting of States Endorsing the Political Declaration on Responsible Military Use of Artificial ... - Department of State - March 22nd, 2024 [March 22nd, 2024]
- AMD Fell Today -- Is This a Chance to Buy the Artificial Intelligence (AI) Stock? - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- Artificial Intelligence (AI) Stocks Are Red-Hot, but Here's 1 to Avoid (for Now) - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- A Once-in-a-Generation Investment Opportunity: 1 Artificial Intelligence (AI) Growth Stock to Buy and Hold Forever - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- 2 Artificial Intelligence (AI) Stocks That Could Go Parabolic - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- Artificial intelligence will radically improve health care, but only if managed carefully - The Hill - March 22nd, 2024 [March 22nd, 2024]
- Tennessee Makes A.I. an Outlaw to Protect Its Country Music and More - The New York Times - March 22nd, 2024 [March 22nd, 2024]
- When Using Artificial Intelligence In Pharma R&D, Start With Identifying Problem To Solve - HBW Insight - March 22nd, 2024 [March 22nd, 2024]
- Why Super Micro Computer, Advanced Micro Devices, and Other Artificial Intelligence (AI) Stocks Tumbled on Tuesday - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- 3 Billionaires Are Selling Artificial Intelligence (AI) Stock Nvidia and Buying These 10 AI Stocks Instead - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- Nvidia Just Bought 5 Artificial Intelligence (AI) Stocks. These 2 Stand Out the Most. - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- UN adopts first global artificial intelligence resolution - ARMENPRESS - March 22nd, 2024 [March 22nd, 2024]
- This New Artificial Intelligence (AI) Chip Is a Massive Game Changer for Nvidia Stock - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- Nvidia Led the First Phase of Artificial Intelligence (AI), but These 2 Growth Stocks Will Lead the Next Phases ... - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]