There is no fire alarm for AI

Why do we need a fire alarm?



You might decide that a fire alarm should give you important evidence of a fire, which allows you to change your plans and leave the building.



In a classic experiment conducted by Latane and Darley in 1968, eight groups of three students were asked to fill out a questionnaire, sitting in a room that soon after the start of the experiment began to fill with smoke. Five of the eight groups did not respond and did not complain about the smoke, even when it became so dense that they began to cough. Subsequent actions showed that a single student complained about smoke in 75% of cases. A student in the company of two other people who were asked not to do anything, reacted to smoke only in 19% of cases. This and other experiments have shown that pluralistic ignorance is actually occurring. We do not want to seem alarmist and be afraid of an emergency, so we try to look calm, watching the others behave out of the corner of the eye - but they, of course, also try to look calm.



I read a large number of reports about the reproduction of this experiment and its variations, and the results were blatant. It seems to me that this result was not destined to die in a crisis of reproduction, and I did not hear about the problems with its reproduction.



Fire alarm creates a well-known knowledge of the presence of a fire; after that, it will be socially acceptable to react to what is happening. When the alarm goes off, you know that everyone else also knows that there is a fire, and you know that you will not be dishonored if you follow the exit.



Fire alarm does not tell us that there is definitely a fire. I can’t remember a single incident in my life when I left the building due to a fire alarm, and there really was a fire. In fact, the fire alarm is no longer talking about the presence of a fire, but about the presence of smoke coming from under the door.



But the fire alarm says that now from a social point of view, it is acceptable to react to a fire. She promises that no one is ashamed of us because we calmly headed for the exit from the building.



It seems to me that this is one of those cases when people do not really understand what they believe in, as in the case when someone loudly asserts about the victory of the team of their city in the competition, but immediately calms down if you offer him money to win. They do not share a consciously pleasant exciting feeling from cries for victory and a feeling of real expectation of winning the team.



I think that when people look at the smoke coming from under the door, they consider that their uncertainty about the existence of a fire arises from the fact that they do not assign a sufficiently high probability to this event, and that they hesitate because they are afraid to spend time and effort. But in this case, they misinterpret their feelings. In this case, they would experience the same vibrations at the sound of a fire alarm, or even fluctuate even more - the fire alarm does not correlate with a fire, but with smoke from under the door. In fact, hesitations arise from worries about what other people think differently, and not because there may not be a fire. Reluctance to act grows from unwillingness to seem like a fool, and not from unwillingness to waste energy. Therefore, one student in a room in 75% of cases takes some action, and therefore people have no problem responding to the weaker evidence provided by the fire alarm.



* * *



From time to time we are offered to postpone reaction to the problems associated with general-purpose artificial intelligence (ION), since it is believed that we are too far from its appearance, and now it is simply impossible to do something productive with it.



Some people still believe that today there are specific things that we can already do: Soares and Fallenstein (2014/2017); Amodei, Olah, Steinhardt, Christiano, Schulman, and Mané (2016); or Taylor, Yudkowsky, LaVictoire, and Critch (2016).)



Even if these works did not exist, or if you, as an AI researcher, read them, and decided that this is nonsense, and you would like to work on this problem, but you would not know, somehow your next wise step was would sit down and spend a couple of hours, sincerely trying to come up with possible solutions. It is desirable without self-sabotage, thinking that you still can’t think of anything - in principle, it’s much more convenient to decide that you can’t do anything, because you have things that you are more interested in doing. Anyway.



So if the idea of ​​ION seems too far away, and you think that it gives you the right to believe that you can’t do anything productive about organizing its work, then the following alternative strategy follows: sit and wait for future some unknown event so far tells us that the ION is approaching, and then we will already understand that we can begin to work on organizing the ION.



It seems to me that this is the wrong approach for a variety of reasons. And here are some of them.



The first. As Stewart Russell said, if you received radio signals from space, spotted a spaceship through a telescope and know that aliens will land in thirty years, you will still start thinking about it today.



You will not say: "Well, it will be in thirty years, and let him." You certainly will not say: “We can’t do anything until they come closer.” You will surely spend a couple of hours, or at least five minutes, brainstorm ideas, thinking about whether you can do something now.



If you say that aliens will arrive only in thirty years, and therefore today you have nothing to do ... In more effective times, someone would ask you to provide a schedule of actions - when and what to do, and long before arrival. And if you would not have such a schedule, then the questioner would understand that you are not acting according to the working table of timely responses, but simply prokrastieniruete. From this, he would have made the correct conclusion that you probably did not think well enough about what you can do today.



Using the expressions of Brian Kaplan [ American economist and blogger / approx. trans. ], everyone who calmly believes that “now nothing can be done to prepare,” is set up incorrectly; they should be much more worried that they cannot come up with any way to somehow prepare. Or maybe ask someone else if he has any ideas on this? Anyway.



The second. History shows that ordinary people, and even scientists who are not part of a key group of advanced researchers, and even scientists who are included there, often believe that key technological discoveries will not occur many decades later, although in reality they are five years from now.



In 1901, two years before he took part in building the first aircraft heavier than air, Wilbur Wright told his brother that even fifty years would pass before active flights.



In 1939, three years before he personally observed the first nuclear chain reaction in a handful of uranium blocks, Enrico Fermi expressed a 90% certainty that uranium could not be used to maintain the fission chain reaction. It seems to me that a year after that, that is, two years before the denouement, Fermi also said that if it is possible to receive decay energy, then it should be another fifty years.



Well, if you are not the Wright brothers and not Enrico Fermi, then you will be even more surprised. Most of the world has learned that atomic weapons already exist and work only after reading the headlines about Hiroshima. Four years later, after the flight of the Wright Brothers flyer, respected intellectuals still claimed that the flight of vehicles was heavier than air was impossible, because then knowledge spread more slowly [ the first flight on an airplane is Clement Ader’s flight made in 1890, 13 years before the Wright brothers - so these intellectuals are 17 years late / approx. trans. ].



Can we say today, in hindsight, that then there were some events that signified the approach of flights heavier than air or nuclear energy? Of course, but if you go back to the past, read the newspapers of that time, and find out what people were saying about it then, it becomes clear that they did not know that these events were some kind of symbolic, or they very much doubted their significance. . Others, I think, played the role of enthusiastic futurists and proclaimed the imminent onset of great change, while others - the role of skeptical scientists who tried to stifle children's enthusiasm. And if among all this hype there would have been a predictor who said “decades” when it really took decades, and “5 years” when it really took five years - well, good searches for you in this noise. Most likely, these predictors said “maybe tomorrow, and maybe after decades” in both cases - when big changes came tomorrow, and when they came after decades.



One of the main reasons that we are strong in hindsight and believe that the past was more predictable than anyone could actually predict at one time, is that we know in hindsight what we need to notice and focus only on one thought related to exactly what each of the available evidence means. But if you look at what people were saying at the time, you will see that they don’t even know what will happen in three months, since they don’t know which events were significant.



You can say that “until ION is 50 years old,” and this may turn out to be true. In the same way, people said that an active flight would take place not earlier than in a few decades, and it took place in a few decades. The problem is, all the evidence looks the same to you, if you really live in this story and not read about it afterwards.



This does not mean that whenever someone says “after fifty years,” this event occurs after a couple of years. This means that a confident prediction about the remote onset of the event corresponds to the knowledge about the technology, which does not change until you get very close to the cutting edge of development. It's akin to “I don't know how to do it,” and sometimes you say that an event will occur in fifty years, and sometimes that in a couple of years, and sometimes you say it, while the Wright brothers flyer flies somewhere where you can not see it.



Third. Progress moves advanced knowledge, not average.



If Fermi and Wright could not have foreseen events in three years, imagine how hard it was to do the rest.



If you are not at the forefront of knowledge in the implementation of some achievement, and you are brought up to date on what is destined to become a leading project, you cannot predict an irreversible breakthrough based on your knowledge. Unless you can brilliantly see the perspective in some way that hunters and gatherers didn’t need, and you don’t know very well that other people may have technologies and ideas that you have no idea about. If you do not consciously compensate for your ideas, taking into account the lessons of history, then you will clearly be inclined to think that decades have passed before this event. Three months before the construction of the first Chicago woodpile, Fermi no longer believed that obtaining nuclear energy was impossible or he had to wait a decade before — he was already aware of everything and saw how it could be done. But any person who was not up to date with events probably thought that there were still fifty years left to this event, at that moment when the wood pile was hissing on the squash court at the University of Chicago.



People do not automatically take into account the fact that the moment of the onset of a major breakthrough depends on advanced knowledge in this area. This range includes people who know the most and have the best ideas; so the knowledge of all other people remains average, and the average knowledge is not enough to understand when a breakthrough can occur. I think that they do not reflect on this at all, and simply make an assessment based on their own sense of the complexity of the project. And if they think somehow more responsibly, do real work on correcting their prejudices, I have never met such reasoning anywhere.



To know that there are still dozens of years before the appearance of the ION, we need to understand the ION well enough to know which parts of the puzzle we still lack, and how difficult it is to get them; we probably will not have such knowledge until the puzzle is complete. It also means that for people outside the advanced circle, the puzzle will seem less complete than it looks in this circle.



Again, this does not mean that what people say “fifty years old” is a definite sign that something is happening on the squash court right now. They talked about "fifty years" and sixty years ago. This means that all those who believe that technological events can actually be predicted in advance by people who are not dedicated to reporting on the progress of leading projects and who do not share the best ideas about how to achieve the goal and how much effort it will take to learn the lessons. stories. Historical books contribute to erroneous ideas, carefully laying out the path of progress and all those visible signs of achievement, the importance of which we only understand now. Sometimes it is possible to predict the consequences of a large breakthrough immediately after it has occurred, but it is very rarely possible to make an accurate prediction about the time of the occurrence of such a breakthrough on periods longer than one to two years. And if you are one of those rare people who can predict such events, if such people exist at all, then no one knows that you need to listen to you, and not enthusiastic futurists or skeptical scientists.



Fourth. In the future, there will be other tools with which you can easily do things that are difficult today, or you can hardly do things that are impossible today.



How do we know that ION will appear only in a few decades? Popular articles for authorship of research laboratories usually give three main reasons:



A) The author does not know how to create an ion with the help of modern technology. The author does not know where to start.



B) The author believes that making impressive things available to modern AI is terribly difficult, he had to spend a lot of time on the farm from the GPU, adjusting all the hyper parameters. He believes that people underestimate how difficult it is to achieve such indicators today, and panic prematurely, because he believes that anyone can just run Tensorflow and make a mobile.



C) The author spends a lot of time interacting with the AI ​​systems, and therefore is personally able to assess how stupid they are and how they lack common sense.



We have already considered some aspects of the argument A. Let us turn to the argument B.



Suppose I say: “Now a single student who has attended a computer science course can do everything in a week that N + years ago the entire research community could do with neural networks”. How big is the number N?



I was tweeted to this question by several people with unknown achievements, but the most popular answer was five - and it seems to me correct based on my own acquaintance with machine learning. Of course, this number cannot be universal, since reality is never so simple. If you could do something in 2012, then today you can do it quite simply with the help of modern GPUs, Tensorflow, Xavier initialization, batch normalization, truncated linear transformation, adaptive moment estimation (Adam), RMSprop, or simply stochastic gradient descent with inertia. So modern technology has become better. Of course, there are things that are not available to us today with the help of these simple methods, they require a lot of work - but in 2012 these things were impossible at all.



In machine learning, everything is arranged in such a way that if you can do something in principle, then in just a few years you can do it very easily with the help of improved future tools. From this point of view, argument B “you do not understand how difficult it is to do what we do,” is simply illogical.



Statement B sounds similar to Rutherford’s statement in 1933, when he called the generation of energy from the splitting of atoms "a pipe dream." If you were a nuclear physicist in 1933, you had to split the atoms by hand, bombarding them with other particles, and that was tedious work. If someone started talking about getting energy from atoms, you might feel that you are underestimated, that people find your work simple.



But, of course, this sensation will always be present for engineers engaged in AI on serious advanced projects. , ( , , ; Google FB). , . .



, , , ; . , , ; . , , , , , . , , ; , , , , . , , , . , - .



Fifth Okay, let's be very frank. I do not think that most of the reasoning about how long to wait until IONI (or how close it is) comes from the use of modeling the future progress in machine learning. I don’t think it’s a bad model; I think the point is the absence of such models.



I was once at a conference where I spoke full of famous luminaries in the AI ​​field, and most of the luminaries nodded and agreed with each other in the sense that it was still far away from the IION - with the exception of two well-known AI luminaries who were sitting quietly and letting others speak.



: «, , . . , , ».



There was silence.



, , , . , « » . , « , – 50-60% — , 80-90%, ».



. 80%, «» , , , , . , , , 44 , 70%.



But that's not the point. , , , , . , , , , , , , . , , , , , , arXiv , , , arXiv, .



In addition, Demis Hassabis was also at the conference [ co-founder of DeepMind, the company that created the AlphaGo computer system, which won the professional go player. trans.], so they knew that if they called something impossible not enough, Demis would make DeepMind take up the task and solve it.



The style of my question was very different from talking at a conference, and required a switch of mental context: the luminaries gathered there had to ask their coarse, barely formed intuitive models of progress in machine learning, and decide which particular achievements, if any, their region’s development model were uniquely sweeps aside in two years. Instead of, let's say, socially acceptable verbal outpourings, designed to muffle all this damn hype associated with ION and deserve predictable applause from the audience.



: , . , , arXiv, , . , , , .



, . , , , - .



2017 352 , ICML [ ] NIPS 2015 [ ]. , « » ( , « , ») 50% 121 . : « , -», , 50% 44 .



, , , , .



* * *



, , , - .



, - ; , , ; , , , .



– - , . , , [ – . trans. ] , ( ), , , .



: - 1974 , , - , , 1974 . , , , «» , . , , , . 1997 Deep Blue , - , - . , Deep Blue, , « », , « ».



AlphaGo - , , 10 ? , « ».



, AlphaGo, , , , - , . , , AlphaGo , – , Deep Blue. , « GPU, », , , , .



, , , , , . . Like you. , , , , – . , - , , , 10, 5 2 . , , , .



, , . , :



A) They will personally understand how to create an ION based on the available tools. They always argue that now it is not so in order to rein in the fools, who believe that the IION is not long.



B) When their personal work ceases to seem to them too difficult. This, from their point of view, is not understood by any plebeians who consider that the ION is approaching, because they never had to sleep until two o'clock in the morning, trying to stabilize the generative-adversary network .



) , , ; , , , ; , , ; , , , .



. Point.



, , , , , , .



* * *



, , AlphaGo , , , , , , .



, , , , , . , , , . , , - , . , , , . , , ; « », ; , .



- arXiv, , , . , , , , 15 , , . , , , , , 10 . , , , , , .



, 15 , , , . , , ( - ) , . , , , , , , , . , , , ( , 2017 , , - tf.unsupervised module Tensorflow 5.3.1.).



, , , , – , - , , -. , , , , , , .



, . , , , – . « arXiv», , . , . « , », . , .



, , , , . , , , , , , . , , , , . , , 10 , 5, 2 . - , , , , , - , .



, , , , . - , - . , , , , , () , () .



* * *



, - , , , .



, , , , , . , , , , , , , . , .



, , . - : «, , , , ». , , , , . , , , -.



, , , .



And today, when all this happens at the same time, one can predict that the same system with its stimuli will not give the correct results, having received an indefinite signal that perhaps aliens will land in five years. The law of constant failures asserts that if the existing authorities make mistakes, believing that it makes sense to transfer the discussion from existential risk to more real, from their point of view, problems like security of robots, then we should expect that in the future they will say all sorts of nonsense.



, , «». . , , , , ) , , ) , « », ) , , « ». - , , , , - – , , .



, , , . , , «», . , , – , « , , ».



, , , (MIRI), , , , , , , " ", « ». , ; , , . , «», , . , , «» , , .



, - -. , , ( ) , (, - , ). , , , , .



, , , , , . , , , , . , - , , , . «» .



- : « , , $5 . , . „“.



, - „“, , . , , , . , , , Facebook. , , - .



- , - , . , „“, , , , . , AlphaGo, . , , , AlphaGo - . ! , , .



, , . , 50% N , – , , , N 50%, 50% N , , P N 50%. , , , P, , , N , .



And almost certainly, you will still experience this uncertainty, regardless of how much the appearance of the ION approached. No matter how long the ION appears, any signs will almost certainly not lead to the generally accepted public knowledge that the IION has a 50% chance of appearing in N years, therefore there will be no agreement that it is entirely acceptable to react to it. pursuing a policy of P.



And if all this becomes public knowledge, then P is unlikely to be ignored by the intervention, and regulation of AI will be ignored by the problem; by the time you have been waiting too long, and it will be too late.



It is more likely that there will be no public knowledge, therefore there will always be some uncertainty about the necessary actions.



, , . , - . , .



, , . , , – . , „“, - - .