Nell Watson is an engineer, a futurist, and the founder and CEO of Poikos. As such, she knows a lot about the machines we use today, and the ones we're planning for tomorrow. And she's worried that the artificial intelligence of the near-future might decide the most benevolent thing to for mankind is to destroy it.

That's the concern Watson raised at The Conference, an annual gathering in Sweden focusing on technology and human behavior. In her talk, "Helping Computers to Understand Humans," Watson brings up a terrifyingly interesting point: The machine learning powering the artificial intelligence we have right now can't learn the nuanced lessons of human ethics.

You should really watch Watson's entire talkat just under 17 minutes long, it's jam-packed with insights on the current and future state of machine learning and artificial intelligence. Here, you don't even have to go anywhere:

Here's the brunt of Watson's argument, in case you can't watch the video for some reason:

When we start to see super-intelligent artificial intelligences, are they going to be friendly, or are they going to be unfriendly? [. . .] Having a kind intelligence is not quite enough, because to paraphrase Arthur C. Clarke, "any sufficiently benevolent action is indistinguishable from malevolence." If you're really, really, really kind, that might be seen as really evil. A truly kind intelligence might decide that the kindest and best thing for humanity is to end us.

Yeesh. Maybe Kubrick was right about HAL-9000 after all.

It's not all doom and gloom though (that would be a real downer of a lecture!). Watson ends her talk by issuing a challenge, saying "perhaps the most important work of all of our lifetimes may be to ensure that machines are capable of understanding human values."

Better get to work, engineers. We haven't got much of a head start. [The Conference; Wired via CNet]

Go here to read the rest:

Top Engineer and Futurist: Tomorrow's Robots Might Mercy-Kill Mankind