Daily Archives: March 29, 2017

It doesnt take a tremendous amount of training to begin a job as a cashier at McDonalds. Even on their first day, most new cashiers are good enough. And they improve as they serve more customers. Although a new cashier may be slower and make more mistakes than their experienced peers, society generally accepts that they will learn from experience.

We dont often think of it, but the same is true ofcommercial airline pilots. We take comfort that airline transport pilot certification is regulated by the U.S. Department of Transportations Federal Aviation Administration and requires minimum experience of 1,500 hours of flight time, 500 hours of cross-country flight time, 100 hours of night flight time, and 75 hours of instrument operations time. Butwe also know that pilots continue to improve from on-the-job experience.

On January 15, 2009, when US Airways Flight 1549 was struck by a flock of Canada geese, shutting down all engine power, Captain Chelsey Sully Sullenberger miraculously landed his plane in the Hudson River, saving the lives of all 155 passengers. Most reporters attributed his performance to experience. He had recorded 19,663 total flight hours, including 4,765 flying an A320. Sully himself reflected: One way of looking at this might be that for 42 years, Ive been making small, regular deposits in this bank of experience, education, and training. And on January 15, the balance was sufficient so that I could make a very large withdrawal. Sully, and all his passengers, benefited from the thousands of peoplehed flown before.

How it will impact business, industry, and society.

The difference between cashiers and pilots in what constitutes good enough is based on tolerance for error. Obviously, our tolerance is much lower for pilots. This is reflected in the amount of in-house training we require them to accumulate prior to servingtheir first customers, even though they continue to learn from on-the-job experience. We have different definitions for good enough when it comes to how much training humans requirein different jobs.

The same is true ofmachines that learn.

Artificial intelligence (AI) applications are based on generating predictions. Unlike traditionally programmed computer algorithms, designed to take data and follow a specified path to produce an outcome, machine learning, the most common approach to AI these days, involves algorithms evolving through various learning processes. Amachine is given data, including outcomes, it finds associations, and then, based on those associations, it takes new data ithas never seen before and predicts an outcome.

This means that intelligent machines need to be trained, just aspilots and cashiers do. Companies design systems to train new employees until they aregood enough and then deploy them into service, knowing that they will improve as they learn from experience doing their job. While this seems obvious, determining what constitutes good enough is an important decision. In the case of machine intelligence, it can be a major strategic decision regarding timing: when to shift from in-house training to on-the-job learning.

There is no ready-made answer as to what constitutes good enough for machine intelligence. Instead, there are trade-offs. Success with machine intelligence will require taking these trade-offs seriously and approaching them strategically.

The first question firms must ask is what tolerance they and their customers have for error. We have high tolerance for error with some intelligent machines and a low tolerance for others. For example, Googles Inbox application reads your email, uses AI to predict how you will want to respond, and generates three short responses for the user to choose from. Many users report enjoying using the application even when it has a 70% failure rate (i.e., the AI-generated response is only useful 30% of the time). The reason for this high tolerance for error is that the benefit of reduced composing and typing outweighs the cost of wasted screen real estate when the predicted short response is wrong.

In contrast, we have low tolerance for error in the realm of autonomous driving. The first generation of autonomous vehicles, largely pioneered by Google, was trained using specialist human drivers who took a limited set of vehicles and drove them hundreds of thousands of kilometers. It was like a parent taking a teenager on supervised driving experiences before letting them drive on their own.

The human specialist drivers provide a safe training environment, but are also extremely limited. The machine only learns about a small number of situations. It may take many millions of miles in varying environments and situations before someone has learned how to deal with the rare incidents that are more likely to lead to accidents. For autonomous vehicles, real roads arenasty and unforgiving precisely because nasty or unforgiving human-causedsituations can occur on them.

The second question to ask, then, is how important it is to capture user data in the wild. Understanding that training might take a prohibitively long time, Tesla rolled out autonomous vehicle capabilities toall itsrecent models. These capabilities included a set of sensors that collect environmental data as well as driving data that isuploaded to Teslas machine learning servers. In a very short period of time, Tesla can obtain training data just by observing how the drivers of its cars drive. The more Tesla vehicles there are on the roads, the more Teslas machines can learn.

However, in addition to passively collecting data as humans drive their Teslas, the company needs autonomous driving data to understand how itsautonomous systems are operating. For that, it needs to have cars drive autonomously so that it can assess performance, but also assess when a human driver, required to be there and paying attention, chooses to intervene. Teslas ultimate goal is not to produce a copilot, or a teenager who drives under supervision, but a fully autonomous vehicle. That requiresgetting to the point where real people feel comfortable in a self-driving car.

Herein lies a tricky trade-off. In order to get better, Tesla needs its machines to learnin real situations. But putting its current cars in real situations means giving customers a relatively young and inexperienced driver although perhaps as good as or better than many younghuman drivers. Still, this is far riskier than beta testing, for example, whetherSiri or Alexa understoodwhat you said, or whether Google Inbox correctly predicts your response to an email. In the case of Siri, Alexa, or Google Inbox, itmeans a lower-quality user experience. In the case of autonomous vehicles, it means putting lives at risk.

As Backchannel documented in a recent article, that experience can be scary. Cars can exit freeways without notice, or put on the brakes when mistaking an underpass for an obstruction. Nervous drivers may opt not to use the autonomous features, and, in the process, may hinderTeslas ability to learn. Furthermore, even if the company can persuade some people to become beta testers, are those the people it wants? After all, a beta tester for autonomous driving may be someone with a taste for more risk than the average driver. In that case, who is the company training their machines to be like?

Machines learn faster with more data, and more data is generated when machines are deployed in the wild. However, bad things can happen in the wild and harm the company brand. Putting products in the wild earlier accelerates learning but risks harming the brand (and perhaps the customer!); putting products in the wild later slows learning but allows for more time to improve the product in-house and protect the brand (and, again, perhaps the customer).

For some products, like Google Inbox, the answer to the trade-off seems clear because the cost of poor performance is low and the benefits from learning from customer usage are high. It makes sense to deploy this type of product in the wild early. For other products, like cars, the answer is less clear.As more companies seek to take advantage of machine learning, this is a trade-off more and more will have to make.

Read the original:

The Trade-Off Every AI Company Will Face - Harvard Business Review

Let's give this points in the Academic Sense of Humour stakes for 2017: the wryly-named Data-mining Textual Responses to Uncover Misconception Patterns, or D.TRUMP, looks to automate the process of working out just how confused someone might be, from how they answer open-response questions.

The problem three boffins from Rice University and Princeton are trying to answer arises because of the rise of large-scale online learning.

At a smaller scale for example, in a lecture theatre or tutorial gathering it's relatively easy for a capable instructor to work out from a student's question what part of a topic they're finding hard to grasp.

That scales badly: in a MOOC (massive open online course), the tutor-to-student ratio could be thousands to one or more, but there's an upside, since the scale of the student body is also a rich source of data.

D.TRUMP seeks to mine that student data for evidence of clue deficit, with the authors of this paper writing: The scale of this data presents a great opportunity to revolutionise education by using machine learning algorithms to automatically deliver personalised analytics and feedback to students and instructors in order to improve the quality of teaching and learning.

To achieve that, D.TRUMP transforms answers into low-dimensional textual vectors using tools like Word2Vec and the like; and the authors' work, which is a statistical model that jointly models both the transformed response textual feature vectors and expert expert labels on whether a response exhibits one or more misconceptions.

The researchers tested their work against 386 students' answers to a total of 1,668 questions in the AP Biology high-school level classes at OpenStax Tutor, giving them a total of 60,000 labelled responses.

It's probably helpful at this point to identify just how fine the line can be between correct and an almost-correct misconception. From the paper:

Question 1: People who breed domesticated animals try to avoid inbreeding even though most domesticated animals are indiscriminate. Evaluate why this is a good practice. Correct Response: A breeder would not allow close relatives to mate, because inbreeding can bring together deleterious recessive mutations that can cause abnormalities and susceptibility to disease. Student Response 1: Inbreeding can cause a rise in unfavorable or detrimental traits such as genes that cause individuals to be prone to disease or have unfavourable mutations. Student Response 2: Interbreeding can lead to harmful mutations.

For those (like the author) who didn't study biology: Inbreeding leads to harmful mutations is a lay understanding of genetics. To be marked correct, the student needs to identify the mechanism, that inbreeding can bring together recessive mutations from mother and father.

Having developed D.TRUMP to the level that it can spot that kind of misconception, the system provides another bit of help to the educator: it can identify groups of students who share a misconception. This could indicate whether the students arrived in a course with a clue deficit, or that the courseware isn't getting its message across.

Read more:

Boffins give 'D.TRUMP' an AI injection - The Register

Although it dates as far back as the 1950s, Artificial Intelligence (AI) is the hottest thing in technology today.

An overarching term used to describe a set of technologies such as text-to-speech, natural language processing (NLP) and computer vision, AI essentially enables computers to do things normally done by people.

Machine learning, the most prominent subset of AI, is about recognizing patterns in data and computer learning from them like a human. These algorithms draw inferences without being explicitly programmed to do so. The idea is the more data you collect, the smarter the machine becomes.

At consumer level, AI use cases include chatbots, Amazons Alexa and Apples Siri, while enterprise efforts see AI software aim to cure diseases and optimize enterprise performance, such as improving customer experience or fraud detection.

There is plenty to back-up the hype; A Narrative Science survey found that 38 percent of enterprises are already using AI, growing to 62 percent by 2018, with Forrester Research predicting a 300 percent year-on-year increase in AI investment this year. AI is clearly here to stay.

Unsurprisingly given the constant evolution of criminals and malware, InfoSec also wants a piece of the AI pie.

With its ability to learn patterns of behavior by sifting through huge datasets, AI could help CISOs by finding those known unknown security threats, automating SOC response and improving attack remediation. In short, with skilled personnel hard to come by, AI fills some (but not all) of the gap.

Experts have called for the need of a smart, autonomous security system, and American cryptographer Bruce Schneier believes that AI could offer the answer.

It is hyped, because security is nothing but hype, but it is good stuff, said the CTO of Resilient Systems.

Were a long way off AI from making humans redundant in cybersecurity, but theres more interest in [using AI for] human augmentation, which is making people smarter. You still need people defending you. Good systems use people and technology together.

Martin Ford, futurist and author of Rise of the Robots, says both white and black hats are already leveraging these technologies, such deep learning neural networks.

It's already being used on both the black and white hat sides, Ford told CSO. There is a concern that criminals are in some cases ahead of the game and routinely utilize bots and automated attacks. These will rapidly get more sophisticated.

...AI will be increasingly critical in detecting threats and defending systems. Unfortunately, a lot of organizations still depend on a manual process -- this will have to change if systems are going to remain secure in the future.

Some CISOs, though, are preparing to do just that.

It is a game changer, Intertek CISO Dane Warren said. Through enhanced automation, orchestration, robotics, and intelligent agents, the industry will see greater advancement in both the offensive and defensive capabilities.

Warren adds that improvements could include responding quicker to security events, better data analysis and using statistical models to better predict or anticipate behaviors.

Andy Rose, CISO at NATS, also sees the benefits: Security has always had a need for smart processes to apply themselves to vast amounts of disparate data to find trends and anomalies whether that is identifying and stopping spam mail, or finding a data exfiltration channel.

People struggle with the sheer volume of data so AI is the perfect solution for accelerating and automating security issue detection.

Security providers have always tried to evolve with the ever-changing threat landscape and AI is no different.

However, with technology naturally outpacing vendor transformation, start-ups have quickly emerged with novel AI-infused solutions for improving SOC efficiency, quantifying risks and optimizing network traffic anomaly detection.

Relative newcomers Tanium, Cylance and - to lesser extent - LogRhythm have jumped into this space, but its start-ups like Darktrace, Harvest.AI, PatternEx (coming out of MIT), and StatusToday that have caught the eye of the industry. Another relative unknown, SparkCognition, unveiled what it called the first AI-powered cognitive AV system at BlackHat 2016.

The tech giants are now playing with AI in security too; Google is working on AI-based system which replaces traditional CAPTCHA forms and its researchers have taught AI to create its own encryption. IBM launched Watson for Cyber Security earlier this month, while in January Amazon acquired Harvest.AI, which uses algorithms to identify important documents and IP of a business, and then user behavior analytics with data loss prevention techniques to protect them from attack.

Some describe these products as first-gen AI security solutions, primarily focused on sifting through data, hunting for threats, and facilitating human-led remediation. In the future, AI could automate 24x7 SOCs, enabling workers to focus on business continuity and critical support issues.

I see AI initially as an intelligent assistant able to deal with many inputs and access expert level analytics and processes, agrees Rose, adding AI will support security professionals in higher level analysis and decisions.

Ignacio Arnaldo is chief data scientist at PatternEx, which offers an AI detection system that automates tasks in SecOps, such as the ability to detect APTs from network, applications and endpoint logs. He says that AI offers CISOs a new level of automation.

CISOs are well aware of the problems - they struggle to hire talent, and there are more devices and data that need to be analyzed. CISOs acknowledge the need for tools that will increase the efficiency of their SOCs. AI holds the promise but CISOs have not yet seen an AI platform that clearly/proves to increase human efficiency.

More and more CISOs fully understand that the global skills shortage, and the successful large-scale attacks against high maturity organizations like Dropbox, NSA/CIA, and JPMorgan are all connected, says Darktrace CTO Dave Palmer, whose firm provides machine learning technology to thousands of companies across 60 countries worldwide.

No matter how well funded a security team is, it cant buy its way to high security using traditional approaches that have been demonstrably failing and that dont stand a chance of working in the anticipated digital complexity of our economy in 10 years time.

But for all of this, some think were jumping the gun. AI, after all, seems a luxury item in an era in which many firms still dont do regular patch management.

At this years RSA conference, crypto experts mulled how AI is applicable in security, with some questioning how to train the machine and what the humans role is. Machine reliability and oversight were also mentioned, while others suggested its odd to see AI championed given security is often felled by low-level basics.

I completely agree, says Rose. Security professionals need to continually reassess the basics patching, culture, SDLP etc. otherwise AI is just a solution that will tell you about the multitude of breaches you couldnt, and didnt, prevent.

Schneier sees it slightly differently. He believes security can be advanced and yet still fail at the basics, while he poignantly notes AI should only be for those who have got the security posture and processes in place, and are ready to leverage the machine data.

Ethics, he says, is only an issue for full automation, and hes unconcerned about such tools being utilized by black hats or surveillance agencies.

I think this is all a huge threat, says Ford, disagreeing. I would rank it as one of the top dangers associated with AI in the near to medium term. There is a lot of focus on "super-intelligent machines taking over"...but this lies pretty far in the future. The main concern now is what bad people will do when they have access to AI.

Warren agrees there are obstacles for CISOs to overcome. It is forward thinking, and many organizations still flounder with the basics.

He adds that with these AI benefits will come challenges, such as the costly rewriting of apps and the possibility of introducing new threats. ...Advancements in technology introduce new threat vectors.

A balance is required, or the environment will advance to a point where the industry simply cannot keep pace.

AI and security is not necessarily a perfect match. As Vectra CISO Gunter Ollmann blogged about recently, buzzwords have made it appear that security automation is the same as AI security - meaning theres a danger of CISOs buying solutions they dont need, while there are further concerns over AI ethics, quality control and management.

Arnaldo critically points out that AI security is no panacea either. Some attacks are very difficult to catch: there are a wide range of attacks at a given organization, over various ranges of time, and across many different data sources.

Second, the attacks are constantly changing...Therefore; the biggest challenge is training the AI.

If this points to some AI solutions being ill-equipped, Palmer adds further weight to the claim.

Most of the machine learning inventions that have been touted arent really doing any learning on the job within the customers environment. Instead, they have models trained on malware samples in a vendors cloud and are downloaded to customer businesses like anti-virus signatures. This isnt particularly progressive in terms of customer security and remains fundamentally backward looking.

So, how soon can we see it in security?

A way off, notes Rose. Remember that the majority of IPS systems are still in IDS mode because firms lack the confidence to rely on intelligent systems to make automated choices and unsupervised changes to their core infrastructure. They are worried that, in acting without context, the control will damage the service and thats a real threat.

But the need is imperative: If we don't succeed in using AI to improve security, then we will have big problems because the bad guys will definitely be using it, says Ford.

I absolutely believe increased automation and ease of use are the only ways in which we are going to improve security, and AI will be a huge part of that, says Palmer.

Read this article:

AI will transform information security, but it won't happen overnight - CSO Online