Catherine Dong Contributor
Catherine Dong is a summer associate at Bloomberg Beta and will be working at Facebook as a machine learning engineer.
Major tech companies have actively reoriented themselves around AI and machine learning: Google is now AI-first, Uber has ML running through its veins and internal AI research labs keep popping up.
Theyre pouring resources and attention into convincing the world that the machine intelligence revolution is arriving now. They tout deep learning, in particular, as the breakthrough driving this transformation and powering new self-driving cars, virtual assistants and more.
Despite this hype around the state of the art, the state of the practice is less futuristic.
Software engineers and data scientists working with machine learning still use many of the same algorithms and engineering tools they did years ago.
That is, traditional machine learning models not deep neural networks are powering most AI applications. Engineers still use traditional software engineering tools for machine learning engineering, and they dont work: The pipelines that take data to model to result end up built out of scattered, incompatible pieces. There is change coming, as big tech companies smooth out this process by building new machine learning-specific platforms with end-to-end functionality.
Large tech companies have recently started to use their own centralized platforms for machine learning engineering, which more cleanly tie together the previously scattered workflows of data scientists and engineers.
Machine learning engineering happens in three stages data processing, model building and deployment and monitoring. In the middle we have the meat of the pipeline, the model, which is the machine learning algorithm that learns to predict given input data.
That model is where deep learning would live. Deep learning is a subcategory of machine learning algorithms that use multi-layered neural networks to learn complex relationships between inputs and outputs. The more layers in the neural network, the more complexity it can capture.
Traditional statistical machine learning algorithms (i.e. ones that do not use deep neural nets) have a more limited capacity to capture information about training data. But these more basic machine learning algorithms work well enough for many applications, making the additional complexity of deep learning models often superfluous. So we still see software engineers using these traditional models extensively in machine learning engineering even in the midst of this deep learning craze.
But the bread of the sandwich process that holds everything together is what happens before and after training the machine learning model.
The first stage involves cleaning and formatting vast amounts of data to be fed into the model. The last stage involves careful deployment and monitoring of the model. We found that most of the engineering time in AI is not actually spent on building machine learning models its spent preparing and monitoring those models.
Despite the focus on deep learning at the big tech company AI research labs, most applications of machine learning at these same companies do not rely on neural networks and instead use traditional machine learning models. The most common models include linear/logistic regression, random forests and boosted decision trees. These are the models behind, among other services tech companies use, friend suggestions, ad targeting, user interest prediction, supply/demand simulation and search result ranking.
And some of the tools engineers use to train these models are similarly well-worn. One of the most commonly used machine learning libraries is scikit-learn, which was released a decade ago (although Googles TensorFlow is on the rise).
There are good reasons to use simpler models over deep learning. Deep neural networks are hard to train. They require more time and computational power (they usually require different hardware, specifically GPUs). Getting deep learning to work is hard it still requires extensive manual fiddling, involving a combination of intuition and trial and error.
With traditional machine learning models, the time engineers spend on model training and tuning is relatively short usually just a few hours. Ultimately, if the accuracy improvements that deep learning can achieve are modest, the need for scalability and development speed outweighs their value.
So when it comes to training a machine learning model, traditional methods work well. But the same does not apply to the infrastructure that holds together the machine learning pipeline. Using the same old software engineering tools for machine learning engineering creates greater potential for errors.
The first stage in the machine learning pipeline data collection and processing illustrates this. While big companies certainly have big data, data scientists or engineers must clean the data to make it useful verify and consolidate duplicates from different sources, normalize metrics, design and prove features.
At most companies, engineers do this using a combination SQL or Hive queries and Python scripts to aggregate and format up to several million data points from one or more data sources. This often takes several days of frustrating manual labor. Some of this is likely repetitive work, because the process at many companies is decentralized data scientists or engineers often manipulate data with local scripts or Jupyter Notebooks.
Furthermore, the large scale of big tech companies compounds errors, making careful deployment and monitoring of models in production imperative. As one engineer described it, At large companies, machine learning is 80 percent infrastructure.
However, traditional unit tests the backbone of traditional software testing dont really work with machine learning models, because the correct output of machine learning models isnt known beforehand. After all, the purpose of machine learning is for the model to learn to make predictions from data without the need for an engineer to specifically code any rules. So instead of unit tests, engineers take a less structured approach: They manually monitor dashboards and program alerts for new models.
And shifts in real-world data may make trained models less accurate, so engineers re-train production models on fresh data on a daily to monthly basis, depending on the application. But a lack of machine learning-specific support in the existing engineering infrastructure can create a disconnect between models in development and models in production normal code is updated much less frequently.
Many engineers still rely on rudimentary methods of deploying models to production, like saving a serialized version of the trained model or model weights to a file. Engineers sometimes need to rebuild model prototypes and parts of the data pipeline in a different language or framework, so they work on production infrastructure. Any incompatibility from any stage of the machine learning development process from data processing to training to deployment to production infrastructure can introduce error.
To address these issues, a few big companies, with the resources to build custom tooling, have invested time and engineering effort into creating their own machine learning-specific tools. Their goal is to have a seamless, end-to-end machine learning platform that is fully compatible with the companys engineering infrastructure.
Facebooks FBLearner Flow and Ubers Michelangelo are internal machine learning platforms that do just that. They allow engineers to construct training and validation data sets with an intuitive user interface, decreasing time spent on this stage from days to hours. Then, engineers can train models with (more or less) the click of a button. Finally, they can monitor and directly update production models with ease.
Services like Azure Machine Learning and Amazon Machine Learning are publicly available alternatives that provide similar end-to-end platform functionality but only integrate with other Amazon or Microsoft services for the data storage and deployment components of the pipeline.
Despite all the emphasis big tech companies have placed on enhancing their products with machine learning, at most companies there are still major challenges and inefficiencies in the process. They still use traditional machine learning models instead of more-advanced deep learning, and still depend on a traditional infrastructure of tools poorly suited to machine learning.
Fortunately, with the current focus on AI at these companies, they are investing in specialized tools to make machine learning work better. With these internal tools, or potentially with third-party machine learning platforms that are able to integrate tightly into their existing infrastructures, organizations can realize the potential of AI.
A special thank you to Irving Hsu, David Eng, Gideon Mann and the Bloomberg Beta team for their insights.
Continue reading here:
The evolution of machine learning - TechCrunch
- History of Evolution | Internet Encyclopedia of Philosophy - December 9th, 2016 [December 9th, 2016]
- Evolution - Bulbapedia, the community-driven Pokmon encyclopedia - December 12th, 2016 [December 12th, 2016]
- What is Evolution - explanation and definitions - December 21st, 2016 [December 21st, 2016]
- Evolution (2001 film) - Wikipedia - January 28th, 2017 [January 28th, 2017]
- EvolutionM.net - Mitsubishi Lancer Evolution | Reviews, News ... - February 1st, 2017 [February 1st, 2017]
- YMCA evolution continues at lake - Gaston Gazette - February 7th, 2017 [February 7th, 2017]
- Ivanka Trump's Beauty Evolution, From 1998 to Today Watch - Us Weekly - February 7th, 2017 [February 7th, 2017]
- Lumpy, hairy, toe-like fossil could reveal the evolution of molluscs - The Guardian - February 7th, 2017 [February 7th, 2017]
- How Evolution Alters Biological Invasions - ScienceBlog.com (blog) - February 7th, 2017 [February 7th, 2017]
- Cultural evolution and the mutilation of women - The Economist - February 7th, 2017 [February 7th, 2017]
- Late-night hosts on the evolution of Trump: 'Dickish to dictatorish' - The Guardian - February 7th, 2017 [February 7th, 2017]
- Gold's Gym Regina rebrands to become Evolution Fitness - Regina Leader-Post - February 7th, 2017 [February 7th, 2017]
- Incremental Versus Radical Innovation: A Response to Josh Swamidass on Evolution and Cancer - Discovery Institute - February 7th, 2017 [February 7th, 2017]
- Blockchain: Investment (R)Evolution For Developing Markets - Forbes - February 7th, 2017 [February 7th, 2017]
- See the Evolution of the Famed Porsche 911 in 7 Photos - WIRED - February 7th, 2017 [February 7th, 2017]
- Exhibition charts 500 years of evolution of robots - Phys.Org - February 7th, 2017 [February 7th, 2017]
- How evolution turned ordinary plants into ravenous meat-eaters - Wired.co.uk - February 7th, 2017 [February 7th, 2017]
- Are Evolution Fresh Drinks 'Poison'? - snopes.com - February 7th, 2017 [February 7th, 2017]
- Non-Chromosomal DNA Drives Tumor Evolution - The Scientist - February 8th, 2017 [February 8th, 2017]
- Chimpanzee feet allow scientists a new grasp on human foot evolution - Phys.Org - February 8th, 2017 [February 8th, 2017]
- 'Goldilocks' genes that tell the tale of human evolution hold clues to variety of diseases - Science Daily - February 8th, 2017 [February 8th, 2017]
- Pac-Man is Coming to 'The Sandbox Evolution' Next Week - Touch Arcade - February 9th, 2017 [February 9th, 2017]
- Chimpanzee feet allow scientists a new grasp on human foot ... - Science Daily - February 9th, 2017 [February 9th, 2017]
- Bacteria sleep, then rapidly evolve, to survive antibiotic treatments - Phys.Org - February 9th, 2017 [February 9th, 2017]
- Orangutan squeaks reveal language evolution, says study - BBC ... - BBC News - February 9th, 2017 [February 9th, 2017]
- Evolution gives rhyme its reason - Aurora News Register - February 9th, 2017 [February 9th, 2017]
- Deeper origin of gill evolution suggests 'active lifestyle' link in early vertebrates - Science Daily - February 9th, 2017 [February 9th, 2017]
- From Tara Palmer-Tomkinson to Cara Delevingne: the evolution of the It girl - The Guardian - February 10th, 2017 [February 10th, 2017]
- Banned TED Talk: Rupert Sheldrake The Science Delusion - Collective Evolution - February 10th, 2017 [February 10th, 2017]
- VOTD: Watch the Evolution of Keanu Reeves' Acting Career - /FILM - February 10th, 2017 [February 10th, 2017]
- Pokmon Go Eevee evolution: How to evolve Eevee into Vaporeon, Jolteon and Flareon with new names - Eurogamer.net - February 10th, 2017 [February 10th, 2017]
- Horse evolution bucks evolutionary theory - Science News - February 10th, 2017 [February 10th, 2017]
- Samsung's Chromebook Pro highlights the category's continued evolution - TechCrunch - February 10th, 2017 [February 10th, 2017]
- Scientists solve fish evolution mystery - Phys.Org - February 10th, 2017 [February 10th, 2017]
- Wildfire evolution forces Forest Service into new thinking - The Daily Progress - February 11th, 2017 [February 11th, 2017]
- How the horse can help us answer one of evolution's biggest questions - Raw Story - February 11th, 2017 [February 11th, 2017]
- A primer on Darwin Day: Some religious groups embrace 'Theistic evolution' - LancasterOnline - February 11th, 2017 [February 11th, 2017]
- Apple: Evolution of in-car audio tech moving at 'speed of sound ... - Times of India - February 11th, 2017 [February 11th, 2017]
- Mariska Hargitay's Evolution from '80s Glam to Law & Order: Special Victims Unit - TVOvermind - February 12th, 2017 [February 12th, 2017]
- Evolution of baseball from power to speed has left SBs behind ... - Chicago Sun-Times - February 12th, 2017 [February 12th, 2017]
- More order with less judgment: An optimal theory of the evolution of cooperation - Science Daily - February 12th, 2017 [February 12th, 2017]
- J. Albert C. Uy speaks on evolution, biodiversity in bellied flycatcher population - The College Reporter - February 12th, 2017 [February 12th, 2017]
- See the Evolution of Movie Magic With Every Oscar Winner for Visual Effects in History - Gizmodo - February 12th, 2017 [February 12th, 2017]
- Numerology: Here's What Your Name Says About You - Collective Evolution - February 13th, 2017 [February 13th, 2017]
- The Evolution of Valentine's Day - Inside Science News Service - February 13th, 2017 [February 13th, 2017]
- Why evolution may be tech billionaires' biggest enemy - The Week Magazine - February 13th, 2017 [February 13th, 2017]
- Community Viewpoint: Evolution, like gravity, is much more than theory it is a fact - Kdminer - February 13th, 2017 [February 13th, 2017]
- How the horse can help us answer one of evolution's biggest questions - Phys.Org - February 13th, 2017 [February 13th, 2017]
- How evolution alters biological invasions - Science Daily - February 13th, 2017 [February 13th, 2017]
- Cockeyed squid shines light on deep sea evolution - Christian Science Monitor - February 13th, 2017 [February 13th, 2017]
- Eye Evolution: A Closer Look - Discovery Institute - February 14th, 2017 [February 14th, 2017]
- Evolution always wins: University of Idaho video game uses mutating aliens to teach science concepts - The Spokesman-Review - February 14th, 2017 [February 14th, 2017]
- Geneticists track the evolution of parenting - Phys.Org - February 14th, 2017 [February 14th, 2017]
- How this cockeyed squid shines a light on deep sea evolution - Christian Science Monitor - February 14th, 2017 [February 14th, 2017]
- 4 Possible Roadmaps For macOS and iOS Evolution - The Mac Observer (blog) - February 15th, 2017 [February 15th, 2017]
- The Evolution of the Energy Capital of the World - Texas Monthly - February 15th, 2017 [February 15th, 2017]
- Humons presents an atypical dance evolution - Detroit Metro Times - February 15th, 2017 [February 15th, 2017]
- Pokemon Go Adds 80 Generation 2 Pokemon, New Evolution Items This Week - IGN - February 15th, 2017 [February 15th, 2017]
- Fossil discovery rewrites understanding of reproductive evolution ... - Science Daily - February 15th, 2017 [February 15th, 2017]
- 'X-Men: Evolution' Is the Gateway Drug of Comic Book Shows - Geek - February 16th, 2017 [February 16th, 2017]
- A cultural catch: Evolution of wooden halibut hooks carved by native ... - Science Daily - February 16th, 2017 [February 16th, 2017]
- Bremerton's Fitness Evolution now Planet Fitness - Kitsap Sun (blog) - February 16th, 2017 [February 16th, 2017]
- Eye Evolution: The Waiting Is the Hardest Part - Discovery Institute - February 16th, 2017 [February 16th, 2017]
- Evolution Of The Yeezy: 2009-2017 - HotNewHipHop - February 16th, 2017 [February 16th, 2017]
- Prebiotic evolution: Hairpins help each other out - Science Daily - February 16th, 2017 [February 16th, 2017]
- This 'Live Birth' Fossil Could Change Humanity's Understanding Of Evolution - Daily Caller - February 16th, 2017 [February 16th, 2017]
- Mysterious Ancient Stonehenge-Like Circles Found in Amazon Rainforest - Collective Evolution - February 16th, 2017 [February 16th, 2017]
- 'Pokemon Go': How to Evolve Poliwhirl Into Politoed - Heavy.com - February 17th, 2017 [February 17th, 2017]
- 'Pokemon Go': How to Evolve Slowpoke Into Slowbro or Slowking - Heavy.com - February 17th, 2017 [February 17th, 2017]
- 'Pokemon Go': How to Evolve Gloom Into Bellossom - Heavy.com - February 17th, 2017 [February 17th, 2017]
- Pokmon Go Dragon Scale - how to evolve Seadra into Kingdra and how to get the Dragon Scale - Eurogamer.net - February 17th, 2017 [February 17th, 2017]
- Pokmon Go Eevee evolution: How to evolve Eevee into Umbreon, Espeon, Vaporeon, Jolteon and Flareon with new ... - Eurogamer.net - February 17th, 2017 [February 17th, 2017]
- University of Pittsburgh guest speaker discloses evolution findings - UTA The Shorthorn - February 17th, 2017 [February 17th, 2017]
- 'Pokemon Go' Special Items: Drop Rates for Evolution Items & Berries at Pokestops - Heavy.com - February 17th, 2017 [February 17th, 2017]
- How Vedic Philosophy Influenced Nikola Tesla's Idea of 'Free Energy' - Collective Evolution - February 18th, 2017 [February 18th, 2017]
- Migration to America took long enough for evolution to happen on the way - Ars Technica - February 18th, 2017 [February 18th, 2017]
- How To Choose Your Eevee Evolution In 'Pokmon GO:' Umbreon And Espeon Edition - Forbes - February 18th, 2017 [February 18th, 2017]
- Evolution Items - IGN - February 18th, 2017 [February 18th, 2017]
- Congo River fish evolution shaped by intense rapids: Genomic study ... - Science Daily - February 18th, 2017 [February 18th, 2017]
- Pokmon Go - How to evolve, use Special Items, when to evolve or Power Up your Pokmon - Eurogamer.net - February 18th, 2017 [February 18th, 2017]