When OpenAI first published a paper on their new language generation AI, GPT-3, the hype was slow to build. The paper indicated GPT-3, the biggest natural language AI model yet, was advanced, but it only had a few written examples of its output. Then OpenAI gave select access to a beta version of GPT-3 to see what developers would do with it, and minds were blown.
Developers playing with GPT-3 have taken to Twitter with examples of its capabilities: short stories, press releases, articles about itself, a search engine. Perhaps most surprising was the discovery GPT-3 can write simple computer code. When web developer, Sharif Shameem, modified it to spit out HTML instead of natural language, the program generated code for webpage layouts from prompts like a button that looks like a watermelon.
I used to say that AI research seemed to have an odd blind spot towards automation of programming work, and I suspected a subconscious self-preservation bias, tweeted John Carmack, legendary computer game developer and consulting CTO at Oculus VR. The recent, almost accidental, discovery that GPT-3 can sort of write code does generate a slight shiver.
While the discovery of GPT-3s coding skills may have been somewhat serendipitous, there is, in fact, a whole field dedicated to the development of machine learning algorithms that can code. The research has been making progress, and a new algorithm just recently took another step.
The algorithm, called machine inferred code similarity (MISIM), is the brainchild of researchers from Intel, Georgia Institute of Technology, University of Pennsylvania, and MIT. Trained on the huge amount of code already publicly available on the web, MISIM can figure out what a program is supposed to do. Then, after finding other similar programs and comparing it to them, MISIM can offer ways to make the program faster or more efficient.
It isnt the first machine learning algorithm to make recommendations or compare similarity, but according to the researchers in a new preprint paper on MISIM, it was up to 40 times more accurate at the task when it went head to head with several of its most advanced competitors.
Near term, the AI could be a useful sidekick for todays programmers. Further out, the field could open programming to anyone who can describe what they want to create in everyday language or bring machines that write and maintain their own code.
The pursuit of computers that can code is almost as old as modern computer science itself. While there have been advances in programming automation, the recent explosion in machine learning is accelerating progress in a field called machine programming.
In a 2018 paper on the field, a group of Intel and MIT researchers wrote, The general goal of machine programming is to remove the burden of writing correct and efficient code from a human programmer and to instead place it on a machine.
Researchers are pursuing systems that can automate the steps required to transform a persons intentthat is, what they want a piece of software to dointo a working program. Theyre also aiming to automate the maintenance of software over time, like, for instance, finding and fixing bugs, keeping programs compatible, or updating code to keep up with hardware upgrades.
Thats easier said than done, of course. Writing software is as much art as it is science. It takes a lot of experience and creativity to translate human intent into the language of machines.
But as GPT-3 shows, language is actually a skill machine learning is rapidly mastering, and programming languages are not so different from English, Chinese, or Swahili. Which is why GPT-3 picking up a few coding skills as a byproduct of its natural language training is notable.
While algorithmic advances in machine learning, like GPT-3, are key to machine programmings success, theyd be useless without good training data. Luckily, theres a huge amount of publicly available code on sites like GitHubreplete with revision histories and notesand code snippets and comment threads on sites like Stack Overflow. Even the internet at large, with accessible webpages and code, is an abundant source of learning material for AI to gobble up.
In theory, just as GPT-3 ingests millions of example articles to learn how to write, machine programming AIs could consume millions of programs and learn to code. But how to make this work in practice is an open question. Which is where MISIM comes in.
MISIM advances machine programming a step by being able to accurately identify what a snippet of code is supposed to do. Once its classified the code, it compares it to millions of other snippets in its database, surfaces those that are most similar, and suggests improvements to the code snippet based on those other examples.
Because MISIM classifies the codes purpose at a high level, it can find code snippets that do the same thing but are written differentlytheres more than one way to solve the same problemand even snippets in other programming languages. Simplistically, this is a bit like someone reading a New Yorker article, identifying its topic, and then finding all the other articles on that topicwhether theyre in Der Spiegel or Xinhua.
Another benefit of working at that higher level of classification is the program doesnt need the code to be compiled. That is, it doesnt have to translate it into the machine code thats executed by the computer. Since MISIM doesnt require a compiler, it can analyze code snippets as theyre being written and offer similar bits of code that could be faster or more efficient. Its a little like an email autocomplete feature finishing your sentences.
Intel plans to offer MISIM to internal developers for just this purpose. The hope is itll prove a useful sidekick, making the code-writing process faster, easier, and more effective. But theres potentially more it can do. Translation between computer languages, for example, could also be a valuable application. It could perhaps help coders update government software written in archaic languages to something more modern.
But Justin Gottschlich, director of machine programming at Intel, has an even grander vision: the full democratization of coding.
Combine MISIM (or something like it) with natural language AI, and future programmers could simply write down what they want a piece of software to do, and the computer whips up the code. That would open programming to anyone with a decent command of their native language and a desire to make something cool.
As Gottschlich told MIT Technology Review, I would like to see 8 billion people create software in whatever way is most natural for them.
Image credit: Markus Spiske /Unsplash
More:
This AI Could Bring Us Computers That Can Write Their Own Software - Singularity Hub
- Downloads - Singularity Viewer - December 25th, 2016 [December 25th, 2016]
- What is Singularity (the)? - Definition from WhatIs.com - January 5th, 2017 [January 5th, 2017]
- When Electronic Witnesses Are Everywhere, No Secret's Safe - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- Report: AMD Ryzen Performance in Ashes of the Singularity Benchmark - PC Perspective - February 7th, 2017 [February 7th, 2017]
- Do you believe in the Singularity? - Patheos (blog) - February 7th, 2017 [February 7th, 2017]
- Editorial Note From the Singularity Hub Team - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- Discover the Most Advanced Industrial Technologies at Exponential Manufacturing - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- 10th Letter looks at nature in the time of the Singularity - Creative Loafing Atlanta - February 7th, 2017 [February 7th, 2017]
- Donald Trump Is the Singularity - Bloomberg View - Bloomberg.com - Bloomberg - February 7th, 2017 [February 7th, 2017]
- Wearable Devices Can Actually Tell When You're About to Get Sick - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- AMD 8-core Ryzen benchmark show up on Ashes Of The Singularity ... - VR-Zone - February 8th, 2017 [February 8th, 2017]
- Robot Cars Can Teach Themselves How to Drive in Virtual Worlds - Singularity Hub - February 8th, 2017 [February 8th, 2017]
- Physicists Unveil Blueprint for a Quantum Computer the Size of a Soccer Field - Singularity Hub - February 10th, 2017 [February 10th, 2017]
- How Robots Helped Create 100000 Jobs at Amazon - Singularity Hub - February 10th, 2017 [February 10th, 2017]
- Ready to Change the World? Apply Now for Singularity University's 2017 Global Solutions Program - Singularity Hub - February 10th, 2017 [February 10th, 2017]
- Singularity Containers for Science, Reproducibility, and HPC - Linux.com (blog) - February 10th, 2017 [February 10th, 2017]
- Families Finally Hear From Completely Paralyzed Patients Via New Mind-Reading Device - Singularity Hub - February 12th, 2017 [February 12th, 2017]
- artificial intelligence: the fear of a technological singularity ... - ETtech.com - February 13th, 2017 [February 13th, 2017]
- Holograms Aren't The Stuff of Science Fiction Anymore - Singularity Hub - February 15th, 2017 [February 15th, 2017]
- How the World Has Changed From 1917 to 2017 - Singularity Hub - February 16th, 2017 [February 16th, 2017]
- Preparing for the Singularity - Inverse - February 16th, 2017 [February 16th, 2017]
- Our Health Data Can Save Lives, But We Have to Be Willing to Share - Singularity Hub - February 17th, 2017 [February 17th, 2017]
- Ashes of the Singularity merges with standalone expansion Escalation, no upgrade fee - PCGamesN - February 17th, 2017 [February 17th, 2017]
- Just Stand Inside this Room and it Will Wirelessly Charge Your Phone - Singularity Hub - February 18th, 2017 [February 18th, 2017]
- AMD bundles Ashes of the Singularity with FX processors ahead of Ryzen's launch - PCWorld - February 18th, 2017 [February 18th, 2017]
- Ashes of the Singularity: Escalation being merged with the original game - PC Invasion (blog) - February 18th, 2017 [February 18th, 2017]
- Singularity - GameSpot - February 20th, 2017 [February 20th, 2017]
- The roots of technological singularity can be traced backed to the Stone Age - Wired.co.uk - February 20th, 2017 [February 20th, 2017]
- Jide's new OS is like an Android version of Windows 10's Continuum - The Verge - February 22nd, 2017 [February 22nd, 2017]
- Jide Announces Remix Singularity: The Continuum Alternative for Android - XDA Developers (blog) - February 22nd, 2017 [February 22nd, 2017]
- New Tech Makes Brain Implants Safer and Super Precise - Singularity Hub - February 23rd, 2017 [February 23rd, 2017]
- One Android company wants to use smartphones to make PCs truly dead - BGR - February 23rd, 2017 [February 23rd, 2017]
- Remix tries its hand at the mobile-desktop hybrid OS with Singularity - Android Police - February 23rd, 2017 [February 23rd, 2017]
- Financial Leaders: Make Your Mark on the Future at Exponential Finance - Singularity Hub - February 23rd, 2017 [February 23rd, 2017]
- Remix Singularity is Jide's Android answer to Windows Continuum - SlashGear - February 24th, 2017 [February 24th, 2017]
- AMD Radeon RX 580 Ashes of the Singularity Benchmarks Leaked 4K, Ryzen Combo, CrossFire and More! - Wccftech - February 24th, 2017 [February 24th, 2017]
- Damon Wayans Jr. Joins FX Sci-Fi Comedy Singularity - Den of Geek US - February 24th, 2017 [February 24th, 2017]
- After Man? From Singularity to Specificity - Peace Research Institute Oslo (PRIO) (press release) (blog) - February 24th, 2017 [February 24th, 2017]
- Why the Potential of Augmented Reality Is Greater Than You Think - Singularity Hub - February 24th, 2017 [February 24th, 2017]
- Damon Wayans Jr In Evan Goldberg & Seth Rogen AI comedy - /FILM - February 24th, 2017 [February 24th, 2017]
- Ashes of Singularity: Escalation Gets an Update - CGMagazine - February 24th, 2017 [February 24th, 2017]
- Google Updates: Scuba, Singularity, SMS and suing - The INQUIRER - February 24th, 2017 [February 24th, 2017]
- Singularity Art Show Tonight In San Francisco! - February 25th, 2017 [February 25th, 2017]
- Stardock celebrate v2.1 of Ashes of the Singularity: Escalation with a ... - PCGamesN - February 27th, 2017 [February 27th, 2017]
- Video: AI Is Getting Smarter, Says Singularity University's Neil ... - Wall Street Journal (subscription) (blog) - February 28th, 2017 [February 28th, 2017]
- This Neural Probe Is So Thin, The Brain Doesn't Know It's There - Singularity Hub - March 1st, 2017 [March 1st, 2017]
- Citizen Science Means Anyone Could Discover Planet NineEven You - Singularity Hub - March 1st, 2017 [March 1st, 2017]
- Singularity University establishes new organisation in Denmark - Ministry of Foreign Affairs of Denmark - March 1st, 2017 [March 1st, 2017]
- Singularity University opening organisation in Denmark - The Copenhagen Post - Danish news in english - March 1st, 2017 [March 1st, 2017]
- Does Zapping Your Brain Actually Help You Learn Faster? - Singularity Hub - March 2nd, 2017 [March 2nd, 2017]
- What You Need to Know About Elon Musk's Plan to Fly People to the Moon - Singularity Hub - March 2nd, 2017 [March 2nd, 2017]
- Singularity: Explain It to Me Like I'm 5-Years-Old - Futurism - Futurism - March 3rd, 2017 [March 3rd, 2017]
- Singularity for PC Reviews - Metacritic - March 4th, 2017 [March 4th, 2017]
- Singularity (mathematics) - Wikipedia - March 4th, 2017 [March 4th, 2017]
- See How This House Was 3D Printed in Just 24 Hours - Singularity Hub - March 6th, 2017 [March 6th, 2017]
- NYC's Metrograph theater is running a sci-fi film series featuring Blade Runner, Ex Machina, and Metropolis - The Verge - March 8th, 2017 [March 8th, 2017]
- 3 Exciting Biotech Trends to Watch Closely in 2017 - Singularity Hub - March 9th, 2017 [March 9th, 2017]
- New Burger Robot Will Take Command of the Grill in 50 Fast Food Restaurants - Singularity Hub - March 9th, 2017 [March 9th, 2017]
- Are These Giant Neurons the Seat Of Consciousness in the Brain? - Singularity Hub - March 10th, 2017 [March 10th, 2017]
- How Fully Synthetic Complex Life Just Got a Lot Closer - Singularity Hub - March 12th, 2017 [March 12th, 2017]
- Singularity University launches inaugural Canada Summit | BetaKit - BetaKit - April 8th, 2017 [April 8th, 2017]
- Singularity - Everything2.com - April 8th, 2017 [April 8th, 2017]
- Singularity (Game) - Giant Bomb - April 8th, 2017 [April 8th, 2017]
- Ashes of the Singularity: Escalation on Steam - April 8th, 2017 [April 8th, 2017]
- Approaching the World of Collaboration Singularity - CommsTrader - June 6th, 2017 [June 6th, 2017]
- Berkeley Lab's Open-Source Spinoff Serves Science | Berkeley Lab - Lawrence Berkeley National Laboratory - June 7th, 2017 [June 7th, 2017]
- Beyond Politics: Innovating for a Sustainable Future - Singularity Hub - June 7th, 2017 [June 7th, 2017]
- Tune Into the Future of Fintech at Exponential Finance This Week - Singularity Hub - June 7th, 2017 [June 7th, 2017]
- Experts Weigh in on AI and the Singularity - Futurism - June 7th, 2017 [June 7th, 2017]
- Singularity | Mass Effect Wiki | Fandom powered by Wikia - June 7th, 2017 [June 7th, 2017]
- Quantum Computers Will Analyze Every Financial Model at Once - Singularity Hub - June 8th, 2017 [June 8th, 2017]
- Deloitte and Singularity University Extend Their Relationship To ... - PR Newswire (press release) - June 8th, 2017 [June 8th, 2017]
- Ashes of the Singularity: Escalation 2.3 update adds a new campaign today - PC Gamer - June 8th, 2017 [June 8th, 2017]
- Singularity and Docker | Singularity - June 8th, 2017 [June 8th, 2017]
- Ashes of the Singularity gets a new fully-voiced campaign - PCGamesN - June 9th, 2017 [June 9th, 2017]
- At Exponential Finance, the Singularity University Explores Visionary Applications of Blockchains - Crypto Insider (press release) (blog) - June 9th, 2017 [June 9th, 2017]
- Get It While It's Hot: Why Fintech Is a Goldmine for Investors - Singularity Hub - June 10th, 2017 [June 10th, 2017]
- Forget Police Sketches: Researchers Perfectly Reconstruct Faces by Reading Brainwaves - Singularity Hub - June 14th, 2017 [June 14th, 2017]
- Singularity Summit comes to SA | IT-Online - IT-Online - June 16th, 2017 [June 16th, 2017]
- These 7 Disruptive Technologies Could Be Worth Trillions of Dollars - Singularity Hub - June 17th, 2017 [June 17th, 2017]