It was 2012: Project Debater was born
Seven years after Watson's performance, able to amaze by beating the flesh and blood champions of the game Jeopardy!, IBM tries again to amaze by presenting its new creature: IBM Project Debater.
After the brilliant success of 2011 it was time for Big Blue to set up a new venture. “Everyone out of thousands of researchers … received the same email asking them what the next big AI challenge should be for IBM Research,” recalls Noam Slonim, a member of the IBM research center in Haifa (Israel ). There were no precise specifications, nevertheless the new project should have satisfied certain characteristics, be above all «… scientifically interesting and stimulating and should have some commercial value. Something big, something that would have made a difference." It was Slonim himself who proposed pursuing the goal of creating and training an artificial intelligence that was able to debate meaningfully with a human counterpart. At first, just an ordinary suggestion which, finally, a year later managed to win over the other competing proposals and became the new challenge on which IBM Research had decided to focus.
The result of six years of work by an IBM team coordinated by the Haifa laboratory in Israel, the project aims to develop an AI capable of fully interacting with humans. But, "What is Project Debater?" The question is asked by Big Blue itself, which also provides the answer: «A system that brings artificial intelligence into the art of debate». In essence, as the company is careful to point out, «Project Debater is the first artificial intelligence system capable of discussing complex topics with humans».
The facts
The Watson West in San Francisco last June was the site of two unusual debates between human experts and the latest wonder churned out by IBM. The chosen comparison scheme assigned a total of ten minutes to the speakers, according to this order: four minutes to present one's own thesis, four minutes to reply and refute that of the opponent, two minutes for a conclusive statement.
The subject of the discussion of the first contradictory was: "Should we subsidize space exploration?". Against Noa Ovadia, Israeli debate champion in 2016, in favor of the machine.
The Project Debater argued his thesis arguing that supporting space exploration helps advance science, opens the mind of the new generations and becomes a source of inspiration for them: «… it inspires our kids to pursue training and careers in science, technology and mathematics". He even added a notation that nods to US grandeur: "… owning a space exploration program is a fundamental element of being a great power." Did he try to captivate the audience present?
According to Noa Ovadia, however, the funds could be destined for other research areas, here on earth. The replica of the machine avoided a head-on collision and focused on the fact that the repercussions of investments in space exploration, the desirable technological and economic benefits that would derive from them, would be greater than the costs incurred: «It is rather easy to say that there are things most important places to invest money in, and I don't dispute that. No one is claiming this is the only item on our shopping list. But this is not the point. Since subsidizing space exploration would definitely benefit society, I think it's something the government should pursue."
The second debate pitted IBM's AI against Dan Zafir, another Israeli expert, on the topic: "Should we increase the use of telemedicine?". In both discussions, the Debater showed off his "preparation" and the vastness of the sources he draws on, with quotes ranging from "... a sheikh from the United Arab Emirates, to the number of jobs that space exploration would have produced – according to a German Minister of Economic Affairs.'
At the conclusion of his reply, the automaton relied on a joke: «I am an authentic supporter of the power of technology … as it should be». Whether he did it to mask the difficulty of responding to the remarks made by his interlocutor, as suggested by Dieter Bohn, or to try to ingratiate himself with the audience is difficult to establish. However, the irony of the machine was not involuntary, nor due to chance or some mistake, but is the result of the work of IBM technicians. On several occasions, in fact, the AI has ventured into jokes or jokes, sometimes successful, sometimes less so.
The verdict
It is important to note that the Debater, «… apart from a brief introduction», was not previously instructed, nor did he follow any previously prepared script. Ed Arvind Krishna, Head of Research at IBM, is keen to underline this: «For the first demonstrations of this new technology, we made a choice from a list of selected topics to ensure a meaningful discussion. However Project Debater was never trained on the topics». The subject of the debate was in fact chosen from a list made up of about a hundred topics, for which IBM believes that AI, based on the data stored in its memory, is able to measure itself against a human. Noam Slonim, «…estimated that technology could sustain a “meaningful” debate on those 100 topics 40% of the time».
The amount of information on which the Project Debater was able to rely is vast. It includes several hundred million documents of the most disparate types: from Wikipedia pages to articles and essays, as well as audio and video material, «… a collection of 300 million articles, news and academic works, previously indexed for the purposes of a quick search". Indexed and rigorously vetted by IBM engineers.
At the conclusion of the two comparisons, an audience of journalists and analysts was called upon to decree a winner on the basis of two fundamental categories: the presentation of the arguments and the enrichment in terms of knowledge brought about by the contenders. In summary, the improvised "jury" recognized that the IBM creature has shown to possess relevant elements of knowledge on the issues raised, although the flesh-and-blood opponents have been superior in exposition. While in the first debate the victory went to the human being, in the second the AI was considered more persuasive. If Noa Ovadia managed to get two spectators on her side, nine people admitted to having changed their opinion about the use of telemedicine thanks to the arguments provided by the Project Debater. Although, as noted by the Guardian, «… there were many IBM staff members in the room and they may have been rooting for their own creation», this time we have not witnessed a crushing triumph of AI, but we can speak of a tie. A tie that is even close to us humans.
And a second exhibition, held in July in Israel in front of an audience of journalists, proved it. The IBM machine pitted against two Israeli debate experts, Yaar Bach and Hayah Goldlist Eichler, appeared less brilliant. The issues addressed were respectively: mass surveillance and genetic engineering. As in the previous event organized in San Francisco, the IBM automaton was able to overcome its opponents in the flesh in terms of information and knowledge. However, again, he fell short of the man in putting forward his arguments and, as it were, in the emphasis of the exposition. In conclusion, «the system and Bach drew, however in the second round the victory went to Goldlist Eichler, who managed to change the mind of more people in the audience, despite scoring lower in knowledge of the subject» .
the day after
“Think about it for a moment. An AI system engaged with a human expert in debate, listened to her argument and responded convincingly with its own, unscripted reasoning in order to persuade the audience to consider its position on a controversial topic. Arvind Krishna suddenly projects us into another dimension which, until now, was a hunting ground for science fiction films or documentaries about a plausible, more or less future, future. D'emblée, even with all the uncertainties and shortcomings shown by the machine, such a future has partially become a reality. In fact, how many, techno enthusiasts excluded, would have aimed or only believed possible such a performance of artificial intelligence in the short term? Despite the limitations highlighted, the display of the AI leaves in fact amazed. A "remarkable technology", was the comment of Chris Reed, a professor at the University of Dundee.
Ovadia herself was impressed: "I'm surprised... Technology is truly disconcerting in terms of how many properly human cognitive capacities it is able to perform simultaneously". The Israeli champion had the opportunity to compete with the IBM automaton for a few months, for the purpose of finalizing the system: «At first I was amazed, both by the simple language, but also by the construction of the argument ... The ability to listen and then provide a meaningful response to what I said.
Precisely the peculiar "qualities" indicated by Ovadia are among the main characteristics that make the Project Debater, a "quite special technology ... a significant milestone in the development of an Artificial Intelligence technology", as stated by Daniel Melka, CEO and Country manager of IBM Israel. In essence, the originality of the Debater is based on three innovative skills, «… three crucial skills, each of which opens up new horizons in the field of AI».
1) First of all writing and exposition of a speech based on the available data. "The first venture was the creation of a computer capable of generating an opinion - similar to a newspaper editorial - built on data and expressing it in complete sentences".
2) Secondly, listening to the reply of one's interlocutor, in order to recognize and extrapolate the most important concepts. “The second capability allows the robot to hear and understand its opponents, allowing it to identify key statements hidden in long, continuous spoken language.”
3) Finally, the creation of a single knowledge graph, relating to the doubts expressed by man, in order to allow "ethical" reasoning. "The third feat was to give the computer the ability to express human dilemmas succinctly and with principled arguments."
However, the Debater's performance was not without flaws. In addition to having repeated the same concept several times using different words, at one point, during the debate concerning space exploration, he argued that this «… is more important than [having] good roads or better schools or a high school health care". An argument not really shareable …
Apart from some linguistic uncertainties, in some cases we had the impression that the construction of the sentence was not entirely "natural": sometimes the initial segment of an utterance was not followed by a second part that would appear fluid; sometimes then, anecdotes and quotations have seemed almost casual. At one point, in the middle of a sentence, a “voiceover” insertion revealed that the machine had tapped into a video transcript.
Nonetheless, IBM's new “mirabilia” held its own. As Chris Reed points out, we have witnessed a performance that denotes progress that is anything but trivial: «This is really a significant step forward … I think what struck me was the combination of Artificial Intelligence techniques. Going into something like a debate is not a make-it-or-break thing. … You have to be able to solve a lot of problems and then bring all of those problems together into an engineered solution.”
Conclusions
The machine has shown a lack of empathy, has revealed the current limits of research in the field of artificial intelligence and has fatally provided further material to the discussion concerning the future and the function of AI. Furthermore, despite the many proclamations which place the accent applauding an increasingly close collaboration between man and machine, with AI, we are told, playing an eminently ancillary role, the ghost of competition is reappearing overbearingly. What part will man play in all of this, what function will he play? Will he be able to maintain his "utility", will he preserve his own space of existence, a peculiar uniqueness within a perimeter that appears smaller every day, or will he become a marginal element, even redundant?
Finally, on closer inspection, Project Debater also reveals its own dark side …
IBM envisions a future advisor role for Project Debater. “We believe there is huge positive potential in artificial intelligence that is able to understand us humans,” says Krishna. A counselor free from prejudice, free from emotions and therefore immune to mood and environmental influences, able to provide an objective view of a given topic. “This can raise the level of evidence-based decision-making,” notes Chris Reed.
A tool even able to offer at the same time the pros and cons inherent in a decision. "Over time, and in business environments compatible with this application, we will move more and more towards the use of this AI system to facilitate reasoning, problem solving, on issues that have not yet been resolved", he is keen to reiterate IBM.
Of course, it is strongly emphasized on several occasions that the role of the machine will not be that of making decisions (replacing man), but rather contribute to the discussion, «… act as another voice at the table». The function of the Project Debater, according to Big Blue, "will be that of 'facilitator' in making thoughts, opinions, points of view and reflections circulate better in order to arrive at an effective and efficient synthesis ... a 'Thinker' at the service of flesh and blood thinkers bone".
A stimulating, even reassuring prospect. But let's imagine, for a moment, another dimension. A dystopian future, distinct and distant from the "magnificent and progressive fortunes" that the narration of the unstoppable technological progress promises us. Let's imagine a world saturated with data, crowded with news, with confusion gripping information. An information populated and plagued by fake news …
And if the qualities of tools such as the Project Debater were employed rather than to bring order to the news and sort out the chaos of information, actually to increase this disorder, as well as orient opinions in one direction rather than another? What if, instead of identifying and revealing fake news, it is used to make new ones, flood the network, and not only, with false or distorted news, replace the true with the probable? It would be possible? It is presumable that it is and without even great effort, especially in the world of social media, where communication and news need not necessarily be particularly refined or articulated.
Such a tool could become a sort of “à la carte” Thinker. Worse, a Think thank, as there are many, but automated, capable of making suggestions and "arrangements" in a continuous stream, of influencing, directing, persuading. An instrument capable of assembling facts and elements with sartorial precision, of joining and disjoining, moreover, at a pace unimaginable for a human being. A reality far from us, excessively dystopian? Maybe …
