From the media release for a recent paper:
The idea of artificial intelligence overthrowing humankind has been talked about for many decades, and scientists have just delivered their verdict on whether we’d be able to control a high-level computer super-intelligence. The answer? Almost definitely not.
The catch is that controlling a super-intelligence far beyond human comprehension would require a simulation of that super-intelligence which we can analyse. But if we’re unable to comprehend it, it’s impossible to create such a simulation.David Nield, “Calculations Show It’ll Be Impossible to Control a Super-Intelligent AI” at Science Alert The open access research study is here.
First, the idea that machines can design smarter machines should be treated with skepticism: maybe we are looking at a Robogeddon that can’t happen and a crowd of experts who need it to.
Second, despite inflated claims, as Eric Holloway has pointed out, “Recent AI gains are mostly due to improvements in computational power and access to data. The basic techniques used to formulate and train AI models have remained more or less the same since the 1970s. For example, the well-publicized Deep Learning approach to AI relies on a training algorithm known as backpropagation, which originated in the field of control theory in the 1960s and was then applied to neural networks in the 1970s. The convolutional neural network, the key component of Deep Learning networks, was invented in the 1980s. So, as we see, the basic techniques have remained unchanged for many decades. ” – Eric Holloway
The HAL 9000 is often compared with the SKYNET system in the Terminator movie franchise as a rogue computer that has gained consciousness and become self-aware. HAL did appear to go rogue but, it can be argued, it is not self-aware. It is true that, at one point, HAL declares itself to be conscious. But coding a computer to say “I am conscious” is not difficult. One of the astronauts had worked extensively with HAL. When asked whether HAL had a mind or not, he responded that he didn’t know.
Marks compares the question to “arguing whether Spider Man can beat up the Green Lantern.”
Um, not to bother.
But now, here’s more HAL: