Google Deepmind, working with The Future of Humanity Institute, has released a study which examines how humanity would best combat the potential of out of control, rogue AI4 in the future. The study concludes that an “interruption policy” is necessary for this contingency, potentially triggered by a figurative “big red button.” This human operated kill switch would essentially trick rogue AI into stopping unwanted behavior.
The paper, written by Google’s Laurent Orseau and Stuart Armstrong of the Future of Humanity Institute, discusses the use of a process called reinforcement learning, which uses a system of rewards to coax favorable behavior from AI or robots. The paper uses an example of a robot whose two tasks are to sort boxes in a warehouse, and to bring these boxes into the warehouse from outside. Since carrying the boxes inside is a higher priority than sorting, the robot is more highly rewarded for this task. However, this reinforcement will lead the robot to sort boxes in the rain, ignoring the likelihood of damaging itself. This is where a human would step in and use a “big red button” apparatus to stop this behavior.
Unfortunately, at this point the robot may learn from this interruption that it should stay inside in general, or much more disturbingly, the robot might treat the human intervention as an obstacle, and find a way to circumvent the interruption. Theoretically, this could involve disabling its own kill switch. The paper proposes a solution that convinces the robot that it won’t ever be interrupted this way again, leading it to operate based on the pre-established reward system, instead of working to deal with further interruptions. This concept is called “safe interruptibility.” For the moment, such procedures are intended more to ensure efficient operations than to stop robots bent on world domination.
Though rooted in science fiction literature long before the existence of functional AI, more urgent scenarios are a not too hard to imagine in a time in which many companies are employing highly developed AI for a wide array of tasks. AI is used in all sorts of industries for efficient analysis of ‘big data’ looking for useful patterns in customer data. Robots have found widespread use in heavy industry, often performing tasks considered too dangerous for humans. Younger generations have been growing up with AI built into toys and video games for decades already.
Recent years have seen leaps in the complexity and abilities of AI. Google and Tesla have made advances towards marketable self-driving cars. Another AI system built at Google beat one of the world’s most talented players of the ancient Chinese strategy game, ‘go’. Algorithms called ‘deep neural networks’ have increased the ability of AI systems to learn over time, and to extend their own capabilities. These developments have brought the subject of AI ethics out of obscurity and into immediate relevance.