Micron has a habit of building interesting research prototypes that offer a vague hope of commercialization for the sheer purpose of learning how to make its own memory and storage subsystem approaches more tuned to next generation applications.
We saw this a few years ago with the Automata processor, which was a neuromorphic inspired bit of hardware that focused on large-scale pattern recognition. That project has since folded internally and moved into a privately funded effort from a startup aiming to make it market ready, which is to say that it has all but disappeared from view since that was a couple of years ago.
There is more here for anyone interested in the Automata architecture, but for those curious about why Micron wants to get into the accelerator business with one-off silicon projects like that or its newly announced deep learning accelerator (DLA) for inference, its far less about commercial success than it is learning how to tune memory and storage systems for AI on custom accelerators. In fact, the market viability of such a chip would be a delightful bonus since the real value is getting a firsthand understanding of what deep learning applications need out of memory and storage subsystems.
This deep learning accelerator might be counted among those on the market (and thats a list too long to keep these days) but we do not expect the company to make a concentrated push to go after a large share. This is for the same reasons we dont expect much to emerge into IBMs product line from its research divisions. They are all efforts to build better mainstream products. If there is commercial gain, great, but it is not the wellspring of motivation.
Nonetheless, it is worth taking a quick look at what Micron has done with its inference accelerator since it could set the tone for what we may see in other products functionally, especially for inference at the edge.
Last year, Micron bought a small FPGA-based startup that spun out of Purdue University called FWDNXT (as in Forward Next). It also acquired FPGA startup, Pico Computing, in 2015 and has since been hard at work looking for where reprogrammable devices will fit for future applications and what to bake into memory to make those perform better and more efficiently.
The FWDNXT technology is at the heart of Microns new FPGA based deep learning accelerator, which gets some added internal expertise from Micron via the Pico assets. The architecture is similar to what weve seen in the market over the last few years for AI. A sea of multiply/accumulate units geared toward matrix vector multiply and the ability to do some of the key non-linear transfer functions. Micron took the FWDNXT platform against some tough problems and worked to do things like build tensor primitives inside the memory (so instead of floating point based scatter gather they could go fetch a matrix sitting in a buffer versus going over memory) They have also used the platform to build a software framework that is hands-off from an FPGA programming perspective (just specifying the neural network).
Micron wants to target energy efficiency by going to the heart of the problemdata movement with the performance goal of better memory bandwidth. All of this creates an accelerator that can be useful, but Micron was better able to see how to create future memory by working with FWDNXT to get the device ready.
It became obvious that if we are tasked with building optimized memory and storage we need to come up with what is optimal rather than just throwing in a bag of chips and hoping it works, explains Steve Pawlowski, VP of Advanced Technology at Micron. We are learning about what need to do in our memory and storage to make them a fit for the kinds of hard problems in neural networks we see ahead, especially at the edge.
Pawlowski is one of the leads behind some of Microns most notable efforts in creating specialized or novel architectures like Automata. He previously led architectural research initiatives at Intel where part of his job was to look how prototype chips were solving emerging problems in interesting ways and if those architectures held promise or competitive value. In the process, he developed an eye for building out new programs at Micron that took a research concept and tested its viability and role in using or improving memory devices.
By not having observability into the various networks on the compute side we could only guess if the things we were building into memory would be useful, Pawlowski says. The only way we could get real observability into how neural networks area executing was to have the entire pipeline so we could go in and instrument every piece of it. This is how we end up making better memory.
He adds that they build this base of knowledge by looking at some of the most complex problems and architecting from there, including with a cancer center that is doing disease detection at scale. Here accuracy is the biggest challenge. Theyve also been working with a very large high-energy physics entity (venture a guess) where the drivers are performance and latency. By taking a view of solving problems with different optimization points (accuracy versus raw performance) Micron is hoping to strike a balance that can inform next generation memory.
During these research and productization experiments, Micron does get a forward look at what future memory might need for a rapidly evolving set of workloads like AI.
The funny thing is, what theyre learning is the inherent value of what they already built as a commercial product several years agosomething that had great potential but strong competition. That would be hybrid memory cube (HMC) which has since been folded as a product as Micron focuses on what is next for that concept of memory stacked on top of logic.
As Micron looks at AI workloads the potential for this exact thing, which exists in plenty of devices now as rival HBM, has even more potential, even for inference. It might sound heavy-handed for an energy-efficiency-focused set of workloads, but more demands from the inference side will mean greater compute requirements. Doing all of that in a stacked memory device at the edge might seem like an expensive stretch, but Pawlowski says this is what he sees in his crystal ball.
There may be a renaissance of memory stacked on logic doing neural networks at the edge. The need for higher memory bandwidth will matter more in the years ahead. There will also be a need to reduce memory interconnect power too, Pawloski says, adding, I believe there will come a day when an architecture that is in the HMC style will be the right thing. By then it might not just be a memory device, it could be an accelerator. There will be other capabilities that come along there as well, including better ECC, for instance.
Its hard to tell where the research ends and the commercial potential begins with some of Microns research efforts for new chips or accelerators. If indeed these flow into the next instantiation of HMC, whatever that might be, this is interesting backstory. But when it comes to innovating in memory in a meaningful way that captures what AI chips need now the market might move on before Micron has a chance to intercept it with who knows what analog and other inference devices at the fore.
Featuring highlights, analysis, and stories from the week directly from us to your inbox with nothing in between.
Subscribe now
Read the original:
Why Micron is Getting into the AI Accelerator Business - The Next Platform
- Classic reasoning systems like Loom and PowerLoom vs. more modern systems based on probalistic networks - November 8th, 2009 [November 8th, 2009]
- Using Amazon's cloud service for computationally expensive calculations - November 8th, 2009 [November 8th, 2009]
- Software environments for working on AI projects - November 8th, 2009 [November 8th, 2009]
- New version of my NLP toolkit - November 8th, 2009 [November 8th, 2009]
- Semantic Web: through the back door with HTML and CSS - November 8th, 2009 [November 8th, 2009]
- Java FastTag part of speech tagger is now released under the LGPL - November 8th, 2009 [November 8th, 2009]
- Defining AI and Knowledge Engineering - November 8th, 2009 [November 8th, 2009]
- Great Overview of Knowledge Representation - November 8th, 2009 [November 8th, 2009]
- Something like Google page rank for semantic web URIs - November 8th, 2009 [November 8th, 2009]
- My experiences writing AI software for vehicle control in games and virtual reality systems - November 8th, 2009 [November 8th, 2009]
- The URL for this blog has changed - November 8th, 2009 [November 8th, 2009]
- I have a new page on Knowledge Management - November 8th, 2009 [November 8th, 2009]
- N-GRAM analysis using Ruby - November 8th, 2009 [November 8th, 2009]
- Good video: Knowledge Representation and the Semantic Web - November 8th, 2009 [November 8th, 2009]
- Using the PowerLoom reasoning system with JRuby - November 8th, 2009 [November 8th, 2009]
- Machines Like Us - November 8th, 2009 [November 8th, 2009]
- RapidMiner machine learning, data mining, and visualization tool - November 8th, 2009 [November 8th, 2009]
- texai.org - November 8th, 2009 [November 8th, 2009]
- NLTK: The Natural Language Toolkit - November 8th, 2009 [November 8th, 2009]
- My OpenCalais Ruby client library - November 8th, 2009 [November 8th, 2009]
- Ruby API for accessing Freebase/Metaweb structured data - November 8th, 2009 [November 8th, 2009]
- Protégé OWL Ontology Editor - November 8th, 2009 [November 8th, 2009]
- New version of Numenta software is available - November 8th, 2009 [November 8th, 2009]
- Very nice: Elsevier IJCAI AI Journal articles now available for free as PDFs - November 8th, 2009 [November 8th, 2009]
- Verison 2.0 of OpenCyc is available - November 8th, 2009 [November 8th, 2009]
- What’s Your Biggest Question about Artificial Intelligence? [Article] - November 8th, 2009 [November 8th, 2009]
- Minimax Search [Knowledge] - November 8th, 2009 [November 8th, 2009]
- Decision Tree [Knowledge] - November 8th, 2009 [November 8th, 2009]
- More AI Content & Format Preference Poll [Article] - November 8th, 2009 [November 8th, 2009]
- New Planners Solve Rescue Missions [News] - November 8th, 2009 [November 8th, 2009]
- Neural Network Learns to Bluff at Poker [News] - November 8th, 2009 [November 8th, 2009]
- Pushing the Limits of Game AI Technology [News] - November 8th, 2009 [November 8th, 2009]
- Mining Data for the Netflix Prize [News] - November 8th, 2009 [November 8th, 2009]
- Interview with Peter Denning on the Principles of Computing [News] - November 8th, 2009 [November 8th, 2009]
- Decision Making for Medical Support [News] - November 8th, 2009 [November 8th, 2009]
- Neural Network Creates Music CD [News] - November 8th, 2009 [November 8th, 2009]
- jKilavuz - a guide in the polygon soup [News] - November 8th, 2009 [November 8th, 2009]
- Artificial General Intelligence: Now Is the Time [News] - November 8th, 2009 [November 8th, 2009]
- Apply AI 2007 Roundtable Report [News] - November 8th, 2009 [November 8th, 2009]
- What Would You do With 80 Cores? [News] - November 8th, 2009 [November 8th, 2009]
- Software Finds Learning Language Child's Play [News] - November 8th, 2009 [November 8th, 2009]
- Artificial Intelligence in Games [Article] - November 8th, 2009 [November 8th, 2009]
- Artificial Intelligence Resources - November 8th, 2009 [November 8th, 2009]
- Alan Turing: Mathematical Biologist? - April 25th, 2012 [April 25th, 2012]
- BBC Horizon: The Hunt for AI ( Artificial Intelligence ) - Video - April 30th, 2012 [April 30th, 2012]
- Can computers have true artificial intelligence" Masonic handshake" 3rd-April-2012 - Video - April 30th, 2012 [April 30th, 2012]
- Kevin B. Korb - Interview - Artificial Intelligence and the Singularity p3 - Video - April 30th, 2012 [April 30th, 2012]
- Artificial Intelligence - 6 Month Anniversary - Video - April 30th, 2012 [April 30th, 2012]
- Science Breakthroughs - April 30th, 2012 [April 30th, 2012]
- Hitman: Blood Money - Part 49 - Stupid Artificial Intelligence! - Video - April 30th, 2012 [April 30th, 2012]
- Research Members Turned Off By HAARP Artificial Intelligence - Video - April 30th, 2012 [April 30th, 2012]
- Artificial Intelligence Lecture No. 5 - Video - April 30th, 2012 [April 30th, 2012]
- The Artificial Intelligence Laboratory, 2012 - Video - April 30th, 2012 [April 30th, 2012]
- Charlie Rose - Artificial Intelligence - Video - April 30th, 2012 [April 30th, 2012]
- Expert on artificial intelligence to speak at EPIIC Nights dinner - May 4th, 2012 [May 4th, 2012]
- Filipino software engineers complete and best thousands on Stanford’s Artificial Intelligence Course - May 4th, 2012 [May 4th, 2012]
- Vodafone xone™ Hackathon Challenges Developers and Entrepreneurs to Build a New Generation of Artificial Intelligence ... - May 4th, 2012 [May 4th, 2012]
- Rocket Fuel Packages Up CPG Booster - May 4th, 2012 [May 4th, 2012]
- 2 Filipinos finishes among top in Stanford’s Artificial Intelligence course - May 5th, 2012 [May 5th, 2012]
- Why Your Brain Isn't A Computer - May 5th, 2012 [May 5th, 2012]
- 2 Pinoy software engineers complete Stanford's AI course - May 7th, 2012 [May 7th, 2012]
- Percipio Media, LLC Proudly Accepts Partnership With MIT's Prestigious Computer Science And Artificial Intelligence ... - May 10th, 2012 [May 10th, 2012]
- Google Driverless Car Ok'd by Nevada - May 10th, 2012 [May 10th, 2012]
- Moving Beyond the Marketing Funnel: Rocket Fuel and Forrester Research Announce Free Webinar - May 10th, 2012 [May 10th, 2012]
- Rocket Fuel Wins 2012 San Francisco Business Times Tech & Innovation Award - May 13th, 2012 [May 13th, 2012]
- Internet Week 2012: Rocket Fuel to Speak at OMMA RTB - May 16th, 2012 [May 16th, 2012]
- How to Get the Most Out of Your Facebook Ads -- Rocket Fuel's VP of Products, Eshwar Belani, to Lead MarketingProfs ... - May 16th, 2012 [May 16th, 2012]
- The Digital Disruptor To Banking Has Just Gone International - May 16th, 2012 [May 16th, 2012]
- Moving Beyond the Marketing Funnel: Rocket Fuel Announce Free Webinar Featuring an Independent Research Firm - May 23rd, 2012 [May 23rd, 2012]
- MASA Showcases Latest Version of MASA SWORD for Homeland Security Markets - May 23rd, 2012 [May 23rd, 2012]
- Bluesky Launches Drones for Aerial Surveying - May 23rd, 2012 [May 23rd, 2012]
- Artificial Intelligence: What happened to the hunt for thinking machines? - May 25th, 2012 [May 25th, 2012]
- Bubble Robots Move Using Lasers [VIDEO] - May 25th, 2012 [May 25th, 2012]
- UHV assistant professors receive $10,000 summer research grants - May 27th, 2012 [May 27th, 2012]
- Artificial intelligence: science fiction or simply science? - May 28th, 2012 [May 28th, 2012]
- Exetel taps artificial intelligence - May 29th, 2012 [May 29th, 2012]
- Software offers brain on the rain - May 29th, 2012 [May 29th, 2012]
- New Dean of Science has high hopes for his faculty - May 30th, 2012 [May 30th, 2012]
- Cognitive Code Announces "Silvia For Android" App - May 31st, 2012 [May 31st, 2012]
- A Rat is Smarter Than Google - June 5th, 2012 [June 5th, 2012]