It’s not clear that the many incentives to make your systems powerful and use them online will suddenly change once systems become powerful enough to be dangerous. His warning comes on the heels of last week’s alarming Intergovernmental Panel on Climate Change (IPCC) report warning that we only have 12 years to make changes massive enough to keep global warming to moderate levels. Since some people aren’t convinced that AI is dangerous, they’re not holding the organizations working on it to high enough standards of accountability and caution. There are only a few people who work full time on AI forecasting. To that end, Kenny said that companies must be able to explain what went into an algorithm’s decision-making process, and citizens must be made to understand how AI technologies work. Hebb’s model of neurons interacting with one another set a groundwork for how AIs and machine learning algorithms work under nodes, or artificial neurons used by computers to communicate data. Bostrom makes the case in Superintelligence that AI systems could rapidly develop unexpected capabilities – for example, an AI system that is as good as a human at inventing new machine-learning algorithms and automating the process of machine-learning work could quickly become much better than a human.
While AI safety experts agree on many features of the safety problem, they’re still making the case to research teams in their own field, and they disagree on some of the details. Nick Bostrom, at Oxford, made the case in his 2014 book Superintelligence that a badly designed AI system will be impossible to correct once deployed: “once unfriendly superintelligence exists, it would prevent us from replacing it or changing its preferences. Hawking’s book is ultimately a verdict on humanity’s future. Bostrom’s Future of Humanity Institute has published a research agenda for AI governance: the study of “devising global norms, policies, and institutions to best ensure the beneficial development and use of advanced AI.” It has published research on the risk of malicious uses of AI, on the context of China’s AI strategy, and on artificial intelligence and international security. From electric cars to Mars colonies, he’s made his name by insisting that the future can get here faster.
But dig deeper and there’s something else here too, a faith that human wisdom and innovation will thwart our own destruction, even when we seem hellbent on bringing it about. That’s not to say there’s an expert consensus here – far from it. There’s substantial disagreement on how badly it could go, and on how likely it is to go badly. There’s no technology relation or product relation between what STAR Labs does and Samsung. Develop dynamic AI research and development evaluation mechanisms, focus on AI design, product and system complexity, risk, uncertainty, interpretability, potential economic impact, and other issues. Gong, he says, translates the information into “higher-order insights.” First, it transcribes customer emails, phone and video calls, then it employs machine learning to analyze everything from when a customer is ready to be pitched for a product refresh to which deals are at risk of being lost. Its customer service platform combines the best of human and machine intelligence, enabling businesses to live up to and exceed rising consumer expectations. But when it comes to artificial intelligence, he sounds very different. If we’re not careful, it very well may be the last thing.
It sometimes seems like we’re facing dangers from all angles in the 21st century. “If we’re successful, we believe this will be one of the most important and widely beneficial scientific advances ever made,” writes the introduction to Alphabet’s DeepMind. He researches risks to humanity, both in the abstract – asking questions like why we seem to be alone in the universe – and in concrete terms, analyzing the technological advances on the table and whether they endanger us. Not every organization with a major AI department has a safety team at all, and some of them have safety teams focused only on algorithmic fairness and not on the risks from advanced systems. And I have very high regard for Larry Page and Demis Hassabis, but I do think that there’s value to some independent oversight. These digital replicas of real-world assets, processes or systems, with a two-way link to sensors in the physical world, will help make sense of and create insights and value from vast quantities of data in increasingly sophisticated ways.