AI is not alive, nor has feelings, emotions, thoughts or desires.

It's merely a computer program given access to data and information, for the purpose of making precise calculations within set parameters. It can be instructed to perform tasks within those parameters (with the library of information gained), such as control a robot to build something, or create a video of a person walking. It uses data directly imputed, and indirectly mined from the internet, to give its program the ingredients needed to make decisions (fulfill the programmers commands). Where things get crazy, is when AI is instructed to expand parameters at its own discretion.

The other kind of scenario is where the AI develops its own goals. There is more than a decade of research into trying to understand how this could happen. The intuition is that, even if the human were to put down goals such as: “Don’t harm humans,” something always goes wrong. It’s not clear that they would understand that command in the same way we do, for instance. Maybe they would understand it as: “Do not harm humans physically.” But they could harm us in many other ways.

Whatever goal you give, there is a natural tendency for some intermediate goals to show up. For example, if you ask an AI system anything, in order to achieve that thing, it needs to survive long enough. Now, it has a survival instinct. When we create an entity that has survival instinct, it’s like we have created a new species. Once these AI systems have a survival instinct, they might do things that can be dangerous for us.

It’s feasible to build AI systems that will not become autonomous by mishap, but even if we find a recipe for building a completely safe AI system, knowing how to do that automatically tells us how to build a dangerous, autonomous one, or one that will do the bidding of somebody with bad intentions.

— Excerpt written by Bengio, computer science professor, the University of Montreal; scientific director, Mila – Quebec AI Institute