While CIM can speed up multiplication operations, it comes with added risk and complexity.
Compute-in-memory (CIM) is not necessarily an Artificial Intelligence (AI) solution; rather, it is a memory management solution. CIM could bring advantages to AI processing by speeding up the multiplication operation at the heart of AI model execution. However, for that to be successful, an AI processing system would need to be explicitly architected to use CIM. The change would entail a shift from all-digital design workflows into a mixed-signal approach which would require deep design expertise and specialized semiconductor fabrication processes.
Compute-in-memory eliminates weight coefficient buffers and streamlines the primitive multiply operations, striving for increased AI inference throughput. However, it does not perform neural network processing by itself. Other functions like input data streaming, sequencing, accumulation buffering, activation buffering, and layer organization may become more vital factors in overall performance as model hardware mapping unfolds and complexity increases more robust NPUs (Neural Processing Units) incorporate all those functions.
Fundamentally, compute-in-memory embeds a multiplier unit in a memory unit. A conventional digital multiplier takes two operands as digital words and produces a digital result, handling signing and scaling. Compute-in-memory uses a different approach, storing a weight coefficient as analog values in a specially designed transistor cell sub-array with rows and columns. The incoming digital data words enter the rows of the array, triggering analog voltage multiplies, then analog current summations occur along columns. An analog-to-digital converter creates the final digital word outputs from the summed analog values.
An individual memory cell can be straightforward in theory, such as these candidates:
Still, operating these cells presents mixed-signal challenges and a technology gap that is not closing anytime soon. So, why the intense interest in compute-in-memory for AI inference chips?
First, it can be fast. This is because analog multiplication happens quickly as part of the memory read cycle, transparent to the rest of the surrounding digital logic. It can also be lower power since fewer transistors switch at high frequencies. But there are some limitations from a system viewpoint. Additional steps needed for programming the analog values into the memory cells are a concern. Inaccuracy of the analog voltages, which may change over time, can inject bit errors into results showing up as detection errors or false alarm rates.
Aside from its analog nature, the biggest concern for compute-in-memory may be bit precision and AI training requirements. Researchers seem confident in 4-bit implementations; however, more training cycles must be run for reliable inference at low precision. Raising the precision to 8-bit lowers training demands. It also increases the complexity of the arrays and the analog-to-digital converter for each array, offsetting area and power savings and worsening the chance for bit errors in the presence of system noise.
So is compute-in-memory worthy of consideration? There likely are niche applications where it could speed up AI inference. A more critical question: is the added risk and complexity of compute-in-memory worth the effort? A well-conceived NPU strategy and implementation may nullify any advantage of moving to compute-in-memory. We can contrast the tradeoffs for AI inference in four areas: power/performance/area (PPA), flexibility, quantization, and memory technology.
PPA
Flexibility
Quantization
Memory Technology
The answer to the original question might be that designers should consider CIM only if other, more established AI inference platforms (NPUs) cannot meet their requirements. Since CIM is riskier, costlier, and harder to implement, many should only consider it a last-resort solution.
Expedera explores this topic in much more depth in a recent white paper, which can be found at: https://www.expedera.com/architectural-considerations-for-compute-in-memory-in-ai-inference/
Read this article:
Can Compute-In-Memory Bring New Benefits To Artificial ... - SemiEngineering
- Artificial Intelligence And Jobs: No Signs Of Worker Fear And Loathing - Forbes - July 3rd, 2024 [July 3rd, 2024]
- Houston area man is accused of using artificial intelligence to create child pornography - KENS5.com - July 3rd, 2024 [July 3rd, 2024]
- China-led resolution on artificial intelligence passes in United Nations - South China Morning Post - July 3rd, 2024 [July 3rd, 2024]
- Voyager Space and Palantir Announce Strategic Partnership Leveraging Artificial Intelligence to Drive Innovation in ... - PR Newswire - July 3rd, 2024 [July 3rd, 2024]
- Embracing the Future: Understanding Artificial Intelligence and Its Expanding Role - Montgomery County Newspaper - July 3rd, 2024 [July 3rd, 2024]
- Here Is My Top Artificial Intelligence (AI) Stock to Buy Right Now - Yahoo Finance - July 3rd, 2024 [July 3rd, 2024]
- MicroStrategy boosts HyperIntelligence with artificial intelligence - CIO - July 3rd, 2024 [July 3rd, 2024]
- Moschino uses artificial intelligence to train its retail teams - Luxus Plus - July 3rd, 2024 [July 3rd, 2024]
- Another Artificial Intelligence (AI) Stock Split Is Coming. Could Broadcom Be the Next Nvidia? - The Motley Fool - July 3rd, 2024 [July 3rd, 2024]
- 1 Top Artificial Intelligence (AI) Stock to Buy Before It Soars 70%, According to Loop Capital - Yahoo Finance - July 3rd, 2024 [July 3rd, 2024]
- Wall Street Thinks These High-Flying Artificial Intelligence (AI) Stocks Are Headed Lower (Hint: Nvidia Isn't 1 of Them) - The Motley Fool - July 3rd, 2024 [July 3rd, 2024]
- SCSU Team Accepted Into Prestigious Artificial Intelligence Prog. - WJON News - July 3rd, 2024 [July 3rd, 2024]
- His Brother Said Artificial Intelligence Would Eliminate His Side Hustle And Then Got Laid Off, So He Responded That ... - Twisted Sifter - July 3rd, 2024 [July 3rd, 2024]
- Forget Sora, Runway is the AI video maker coming to blow your mind - TechRadar - July 3rd, 2024 [July 3rd, 2024]
- This Is My Top Artificial Intelligence (AI) ETF to Buy Right Now - 24/7 Wall St. - July 3rd, 2024 [July 3rd, 2024]
- Investing in Artificial Intelligence (AI) Stocks Can Be Risky, but Here's a Fantastic Way to Do It - The Motley Fool - July 3rd, 2024 [July 3rd, 2024]
- Integrating Artificial Intelligence and Machine Learning in the Marine Corps - War On The Rocks - July 3rd, 2024 [July 3rd, 2024]
- For artificial intelligence, is long-term in the near future? - Aspen Daily News - July 3rd, 2024 [July 3rd, 2024]
- Thailand's homegrown AI project aims to resist U.S. tech giants - Nikkei Asia - July 3rd, 2024 [July 3rd, 2024]
- Bill Maher Talks Artificial Intelligence, Bidens Brain, And Guys Who Need Game - Deadline - July 3rd, 2024 [July 3rd, 2024]
- Artificial Intelligence (AI) Stocks Fueled Market Gains in the First Half. Is the Best Yet to Come? - The Motley Fool - July 3rd, 2024 [July 3rd, 2024]
- Lil Wayne Reacts to 1950s Version of A Milli Made With Artificial Intelligence - Complex - July 3rd, 2024 [July 3rd, 2024]
- Princeton Professor Who 'Picks Apart Artificial Intelligence Hype' Honored by Mozilla - TAPinto.net - July 3rd, 2024 [July 3rd, 2024]
- AI advertising may repel as well as attract - Marketplace - July 3rd, 2024 [July 3rd, 2024]
- Opinion: As artificial intelligence rises, data-centre costs spiral. Quantum is the solution - The Globe and Mail - July 3rd, 2024 [July 3rd, 2024]
- Looking for a Home-Run Artificial Intelligence (AI) Investment? Consider This AI Stock That Nvidia Is Investing In. - Yahoo Finance - July 3rd, 2024 [July 3rd, 2024]
- Are Elon Musk's concerns of AI-surged unemployment a real concern? - The Jerusalem Post - July 3rd, 2024 [July 3rd, 2024]
- 3 Magnificent Vanguard ETFs to Buy That Are Loaded With Top Artificial Intelligence (AI) Stocks - Yahoo Finance - July 3rd, 2024 [July 3rd, 2024]
- AI's Impact on B2B Marketing Spotlights Value of Automation - PYMNTS.com - July 3rd, 2024 [July 3rd, 2024]
- How Many Chips Could You Buy With $74 Billion? - PYMNTS.com - July 3rd, 2024 [July 3rd, 2024]
- Use of Artificial Intelligence in the Prediction of Chiari Malformation Type 1 Recurrence After Posterior Fossa ... - Cureus - May 23rd, 2024 [May 23rd, 2024]
- Fritz Lang First Depicted Artificial Intelligence on Film in Metropolis (1927), and It Frightened People Even Then - Open Culture - May 23rd, 2024 [May 23rd, 2024]
- Dehumanizing the Wars: Introduction of Artificial Intelligence in the Warfare - Modern Diplomacy - May 23rd, 2024 [May 23rd, 2024]
- Google Parent Alphabet Is Selling Shares of 2 Hypergrowth Stocks and Piling Into a Promising Artificial Intelligence (AI ... - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- Meta AI Head: ChatGPT Will Never Reach Human Intelligence - PYMNTS.com - May 23rd, 2024 [May 23rd, 2024]
- Waiting for Alexa: When Will Amazon Strike Up the AI Conversation? - PYMNTS.com - May 23rd, 2024 [May 23rd, 2024]
- Artificial intelligence town halls? House committee weighs new approach before writing AI rules - Washington Times - May 23rd, 2024 [May 23rd, 2024]
- Inventing AI: Tracing the diffusion of artificial intelligence with U.S. patents - United States Patent and Trademark Office - May 23rd, 2024 [May 23rd, 2024]
- UM6P to Host High-Level Forum on Artificial Intelligence, Development in Africa - Morocco World News - May 23rd, 2024 [May 23rd, 2024]
- DHS warns of threats to election posed by artificial intelligence - ABC News - May 23rd, 2024 [May 23rd, 2024]
- Artificial intelligence, great wealth transfer: Why this financial analyst sees a rosy time ahead for stocks - CNBC - May 23rd, 2024 [May 23rd, 2024]
- As S&P 500 reaches fresh highs, experts say generative artificial intelligence will create new investment opportunities - CNBC - May 23rd, 2024 [May 23rd, 2024]
- Google Parent Alphabet Is Selling Shares of 2 Hypergrowth Stocks and Piling Into a Promising Artificial Intelligence (AI ... - MSN - May 23rd, 2024 [May 23rd, 2024]
- The Best Artificial Intelligence (AI) ETF to Invest $1,000 in Right Now - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- How To Land A Job In Artificial Intelligence (AI) - Forbes - May 23rd, 2024 [May 23rd, 2024]
- Madhumita Murgia On The Good, Bad And Ugly Of Artificial Intelligence In Our Daily Lives - Forbes India - May 23rd, 2024 [May 23rd, 2024]
- Researchers use artificial intelligence to classify brain tumors - Voice of America - VOA News - May 23rd, 2024 [May 23rd, 2024]
- BBVA steps up its plans in artificial intelligence by signing an agreement with OpenAI - BBVA - May 23rd, 2024 [May 23rd, 2024]
- 2 Artificial Intelligence (AI) Stocks You Can Buy and Hold for the Next Decade - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- Study including WVU and Marshall analyzes cyber threats to Artificial Intelligence systems - West Virginia MetroNews - May 23rd, 2024 [May 23rd, 2024]
- Stock-Split Watch: 5 Artificial Intelligence (AI) Stocks That Look Ready to Split - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- 2 Spectacular ETFs That Can Help You Capitalize on the Artificial Intelligence (AI) Boom - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- NASA pledging responsible use of artificial intelligence | National | khq.com - KHQ Right Now - May 23rd, 2024 [May 23rd, 2024]
- 1 Stock Down 11% to Buy Before Artificial Intelligence (AI) Could Supercharge Its Growth and Send It Soaring - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- Alphabet Goes All In on Artificial Intelligence (AI). Is It Time to Go All In on the Stock? - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- A Once-in-a-Generation Investment Opportunity: Here's 1 Artificial Intelligence (AI) Stock to Buy in May and Hold Forever - Yahoo Finance - May 23rd, 2024 [May 23rd, 2024]
- 2 Artificial Intelligence (AI) Stocks That Offer Dividends and Long-Term Returns - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- Forget Nvidia: Billionaires Are Selling It and Buying These 2 Supercharged Artificial Intelligence (AI) Growth Stocks ... - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- 1 Ridiculously Undervalued Artificial Intelligence (AI) Stock to Buy Hand Over Fist - Yahoo Finance - May 23rd, 2024 [May 23rd, 2024]
- Missed Out on Nvidia's Ginormous Gains? Here's an Artificial Intelligence (AI) Stock You Can Buy Right Now. - The Motley Fool - May 23rd, 2024 [May 23rd, 2024]
- Researchers create innovative verification techniques to increase security in artificial intelligence and image processing - European Research Council - April 29th, 2024 [April 29th, 2024]
- Who in Europe is investing the most in artificial intelligence? - Euronews - April 29th, 2024 [April 29th, 2024]
- 4 Artificial Intelligence (AI) Stocks Members of Congress Can't Stop Buying (and Nvidia Isn't 1 of Them!) - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- The Standard Names Porter Orr Second Vice President of Artificial Intelligence Strategy and Development - NRToday.com - April 29th, 2024 [April 29th, 2024]
- Meet Nvidia CEO Jensen Huang, the man behind the $2 trillion company powering today's artificial intelligence - CBS News - April 29th, 2024 [April 29th, 2024]
- This Warren Buffett Dividend King Stock Just Invested $1.1 Billion Into Artificial Intelligence (AI) - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- Joe Rogan Reveals What Could Be 'Game Over for the Human Race' - Newsweek - April 29th, 2024 [April 29th, 2024]
- LegalTech Artificial Intelligence Market to Soar to $6.4 Billion by 2028: Global Long-term Forecast to 2033 - Strategic ... - Yahoo Finance UK - April 29th, 2024 [April 29th, 2024]
- Artificial Intelligence Tokens Stumble Again! Is The AI Hype Over? - Coinpedia Fintech News - April 29th, 2024 [April 29th, 2024]
- CISA unveils guidelines for AI and critical infrastructure - FedScoop - April 29th, 2024 [April 29th, 2024]
- IBM's Webinar: Cybersecurity In The Era Of Artificial Intelligence Keynotes - AiThority - April 29th, 2024 [April 29th, 2024]
- Being and becoming a good doctor in the age of artificial intelligence - Firstpost - April 29th, 2024 [April 29th, 2024]
- Pope to take part in G7 summit in June to talk Artificial Intelligence - Crux Now - April 29th, 2024 [April 29th, 2024]
- Pope Francis to attend G7 summit to speak on artificial intelligence - Catholic News Agency - April 29th, 2024 [April 29th, 2024]
- 1 Unstoppable Artificial Intelligence (AI) Stock to Buy and Hold Forever - Yahoo Finance - April 29th, 2024 [April 29th, 2024]
- Pope to bring his call for ethical artificial intelligence to G7 summit in June in southern Italy - The Associated Press - April 29th, 2024 [April 29th, 2024]
- Elon Musk Called Tesla an Artificial Intelligence (AI) Robotics Company. Does That Make It a Buy? - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- Emulating neurodegeneration and aging in artificial intelligence systems - Tech Xplore - April 29th, 2024 [April 29th, 2024]
- Pope Francis to participate in G7 session on AI - Vatican News - English - April 29th, 2024 [April 29th, 2024]
- A Once-in-a-Generation Investment Opportunity: 1 Bill Ackman Artificial Intelligence (AI) Stock to Buy Hand Over Fist ... - Yahoo Finance - April 29th, 2024 [April 29th, 2024]