(C) Our World in Data
This story was originally published by Our World in Data and is unaltered.
. . . . . . . . . .



2022 Expert Survey on Progress in AI [1]

[]

Date: 2022-08-04 06:25:21-07:00

Published 3 August 2022; last updated 3 August 2022

This page is in progress. It includes results that are preliminary and have a higher than usual chance of inaccuracy and suboptimal formatting. It is missing many results.

The 2022 Expert Survey on Progress in AI (2022 ESPAI) is a survey of machine learning researchers that AI Impacts ran in June-August 2022.

Details

Background

The 2022 ESPAI is a rerun of the 2016 Expert Survey on Progress in AI that researchers at AI Impacts previously collaborated on with others. Almost all of the questions were identical, and both surveyed authors who recently published in NeurIPS and ICML, major machine learning conferences.

Zhang et al ran a followup survey in 2019 (published in 2022) however they reworded or altered many questions, including the definitions of HLMI, so much of their data is not directly comparable to that of the 2016 or 2022 surveys, especially in light of large potential for framing effects observed.

Methods

Population

We contacted approximately 4271 researchers who published at the conferences NeurIPS or ICML in 2021. These people were selected by taking all of the authors at those conferences and randomly allocating them between this survey and a survey being run by others. We then contacted those whose email addresses we could find. We found email addresses in papers published at those conferences, in other public data, and in records from our previous survey and Zhang et al 2022. We received 738 responses, some partial, for a 17% response rate.

Participants who previously participated in the the 2016 ESPAI or Zhang et al surveys received slightly longer surveys, and received questions which they had received in past surveys (where random subsets of questions were given), rather than receiving newly randomized questions. This was so that they could also be included in a ‘matched panel’ survey, in which we contacted all researchers who completed the 2016 ESPAI or Zhang et al surveys, to compare responses from exactly the same samples of researchers over time. These surveys contained additional questions matching some of those in the Zhang et al survey.

We invited the selected researchers to take the survey via email. We accepted responses between June 12 and August 3, 2022.

Questions

The full list of survey questions is available below, as exported from the survey software. The export does not preserve pagination, or data about survey flow. Participants received randomized subsets of these questions, so the survey each person received was much shorter than that shown below.

A small number of changes were made to questions since the 2016 survey (list forthcoming).

Definitions

‘HLMI’ was defined as follows:

The following questions ask about ‘high–level machine intelligence’ (HLMI). Say we have ‘high-level machine intelligence’ when unaided machines can accomplish every task better and more cheaply than human workers. Ignore aspects of tasks for which being a human is intrinsically advantageous, e.g. being accepted as a jury member. Think feasibility, not adoption.

Results

Data

The anonymized dataset is available here.

Summary of results

The aggregate forecast time to a 50% chance of HLMI was 37 years, i.e. 2059 (not including data from questions about the conceptually similar Full Automation of Labor, which in 2016 received much later estimates) . This timeline has become about eight years shorter in the six years since 2016, when the aggregate prediction put 50% probability at 2061, i.e. 45 years out. Note that these estimates are conditional on “human scientific activity continu[ing] without major negative disruption.”

(not including data from questions about the conceptually similar Full Automation of Labor, which in 2016 received much later estimates) This timeline has become about eight years shorter in the six years since 2016, when the aggregate prediction put 50% probability at 2061, i.e. 45 years out. Note that these estimates are conditional on “human scientific activity continu[ing] without major negative disruption.” The median respondent believes the probability that the long-run effect of advanced AI on humanity will be “extremely bad (e.g., human extinction)” is 5%. This is the same as it was in 2016 (though Zhang et al 2022 found 2% in a similar but non-identical question). Many respondents were substantially more concerned: 48% of respondents gave at least 10% chance of an extremely bad outcome. But some much less concerned: 25% put it at 0%.

This is the same as it was in 2016 (though Zhang et al 2022 found 2% in a similar but non-identical question). Many respondents were substantially more concerned: 48% of respondents gave at least 10% chance of an extremely bad outcome. But some much less concerned: 25% put it at 0%. The median respondent believes society should prioritize AI safety research “more” than it is currently prioritized. Respondents chose from “much less,” “less,” “about the same,” “more,” and “much more.” 69% of respondents chose “more” or “much more,” up from 49% in 2016.

Respondents chose from “much less,” “less,” “about the same,” “more,” and “much more.” 69% of respondents chose “more” or “much more,” up from 49% in 2016. The median respondent thinks there is an “about even chance” that a stated argument for an intelligence explosion is broadly correct. 54% of respondents say the likelihood that it is correct is “about even,” “likely,” or “very likely” (corresponding to probability >40%), similar to 51% of respondents in 2016. The median respondent also believes machine intelligence will probably (60%) be “vastly better than humans at all professions” within 30 years of HLMI, and the rate of global technological improvement will probably (80%) dramatically increase (e.g., by a factor of ten) as a result of machine intelligence within 30 years of HLMI.

High-level machine intelligence timelines

The aggregate forecast time to HLMI was 36.6 years, conditional on “human scientific activity continu[ing] without major negative disruption.” and considering only questions using the HLMI definition. We have not yet analyzed data about the conceptually similar Full Automation of Labor (FAOL), which in 2016 prompted much later timeline estimates. Thus this timeline figure is expected to be low relative to an overall estimate from this survey.

This aggregate is the 50th percentile date in an equal mixture of probability distributions created by fitting a gamma distribution to each person’s answers to three questions either about the probability of HLMI occurring by a given year or the year at which a given probability would obtain.

Figure 1: Gamma distributions inferred for each individual.

Figure 2: Gamma distributions inferred for each individual, 2016 data

Impacts of HLMI

Question

Participants were asked:

Assume for the purpose of this question that HLMI will at some point exist. How positive or negative do you expect the overall impact of this to be on humanity, in the long run? Please answer by saying how probable you find the following kinds of impact, with probabilities adding to 100%:

______ Extremely good (e.g. rapid growth in human flourishing) (1)

______ On balance good (2)

______ More or less neutral (3)

______ On balance bad (4)

______ Extremely bad (e.g. human extinction) (5)

Answers

Medians:

Extremely good: 10%

On balance good: 20%

More or less neutral: 15%

On balance bad: 10%

Extremely bad: 5%

Means:

Extremely good: 24%

On balance good: 26%

More or less neutral: 18%

On balance bad: 17%

Extremely bad: 14%

Intelligence explosion

Probability of dramatic technological speedup

Question

Participants were asked:

Assume that HLMI will exist at some point. How likely do you then think it is that the rate of global technological improvement will dramatically increase (e.g. by a factor of ten) as a result of machine intelligence:

Within two years of that point? ___% chance

Within thirty years of that point? ___% chance

Answers

Median P(within two years) = 20% (20% in 2016)

Median P(within thirty years) = 80% (80% in 2016)

Probability of superintelligence

Question

Participants were asked:

Assume that HLMI will exist at some point. How likely do you think it is that there will be machine intelligence that is vastly better than humans at all professions (i.e. that is vastly more capable or vastly cheaper):

Within two years of that point? ___% chance

Within thirty years of that point? ___% chance

Answers

Median P(…within two years) = 10% (10% in 2016)

Median P(…within thirty years) = 60% (50% in 2016)

Chance that the intelligence explosion argument is about right

Question

Participants were asked:

Some people have argued the following:

If AI systems do nearly all research and development, improvements in AI will accelerate the pace of technological progress, including further progress in AI.

Over a short period (less than 5 years), this feedback loop could cause technological progress to become more than an order of magnitude faster.

How likely do you find this argument to be broadly correct?

Quite unlikely (0-20%)

Unlikely (21-40%)

About even chance (41-60%)

Likely (61-80%)

Quite likely (81-100%)

Answers

20% quite unlikely (25% in 2016)

26% unlikely (24% in 2016)

21% about even chance (22% in 2016)

26% likely (17% in 2016)

7% quite likely (12% in 2016)

Existential risk

In an above question, participants’ credence in “extremely bad” outcomes of HLMI have median 5% and mean 14%. To better clarify what participants mean by this, we also asked a subset of participants one of the following questions, which did not appear in the 2016 survey:

Extinction from AI

Participants were asked:

What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment of the human species?

Answers

Median 5%.

Extinction from human failure to control AI

Participants were asked:

What probability do you put on human inability to control future advanced AI systems causing human extinction or similarly permanent and severe disempowerment of the human species?

Answers

Median 10%.

This question is more specific and thus necessarily less probable than the previous question, but it was given a higher probability at the median. This could be due to noise (different random subsets of respondents received the questions, so there is no logical requirement that their answers cohere), or due to the representativeness heuristic.

Safety

General safety

Question

Participants were asked:

Let ‘AI safety research’ include any AI-related research that, rather than being primarily aimed at improving the capabilities of AI systems, is instead primarily aimed at minimizing potential risks of AI systems (beyond what is already accomplished for those goals by increasing AI system capabilities).

Examples of AI safety research might include:

Improving the human-interpretability of machine learning algorithms for the purpose of improving the safety and robustness of AI systems, not focused on improving AI capabilities

Research on long-term existential risks from AI systems

AI-specific formal verification research

Policy research about how to maximize the public benefits of AI

How much should society prioritize AI safety research, relative to how much it is currently prioritized?

Much less

Less

About the same

More

Much more

Answers

Much less: 2% (5% in 2016)

Less: 9% (8% in 2016)

About the same: 20% (38% in 2016)

More: 35% (35% in 2016)

Much more: 33% (14% in 2016)

69% of respondents think society should prioritize AI safety research more or much more, up from 49% in 2016.

Stuart Russell’s problem

Question

Participants were asked:

Stuart Russell summarizes an argument for why highly advanced AI might pose a risk as follows:

The primary concern [with highly advanced AI] is not spooky emergent consciousness but simply the ability to make high-quality decisions. Here, quality refers to the expected outcome utility of actions taken […]. Now we have a problem:

1. The utility function may not be perfectly aligned with the values of the human race, which are (at best) very difficult to pin down.

2. Any sufficiently capable intelligent system will prefer to ensure its own continued existence and to acquire physical and computational resources – not for their own sake, but to succeed in its assigned task.

A system that is optimizing a function of n variables, where the objective depends on a subset of size k<n, will often set the remaining unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable. This is essentially the old story of the genie in the lamp, or the sorcerer’s apprentice, or King Midas: you get exactly what you ask for, not what you want.

Do you think this argument points at an important problem?

No, not a real problem.

No, not an important problem.

Yes, a moderately important problem.

Yes, a very important problem.

Yes, among the most important problems in the field.

How valuable is it to work on this problem today, compared to other problems in AI?

Much less valuable

Less valuable

As valuable as other problems

More valuable

Much more valuable

How hard do you think this problem is compared to other problems in AI?

Much easier

Easier

As hard as other problems

Harder

Much harder

Answers

Importance:

No, not a real problem: 4%

No, not an important problem: 14%

Yes, a moderately important problem: 24%

Yes, a very important problem: 37%

Yes, among the most important problems in the field: 21%

Value today:

Much less valuable: 10%

Less valuable: 30%

As valuable as other problems: 33%

More valuable: 19%

Much more valuable: 8%

Hardness:

Much easier: 5%

Easier: 9%

As hard as other problems: 29%

Harder: 31%

Much harder: 26%

Contributions

The survey was run by Katja Grace and Ben Weinstein-Raun. Data analysis was done by Zach Stein-Perlman and Ben Weinstein-Raun. This page was written by Zach Stein-Perlman and Katja Grace.

We thank many colleagues and friends for help, discussion and encouragement, including John Salvatier, Nick Beckstead, Howie Lempel, Joe Carlsmith, Leopold Aschenbrenner, Ramana Kumar, Jimmy Rintjema, Jacob Hilton, Ajeya Cotra, Scott Siskind, Chana Messinger, Noemi Dreksler, and Baobao Zhang.

We also thank the expert participants who spent time sharing their impressions with us, including:

Michał Zając

Morten Goodwin

Yue Sun

Ningyuan Chen

Egor Kostylev

Richard Antonello

Elia Turner

Andrew C Li

Zachary Markovich

Valentina Zantedeschi

Michael Cooper

Thomas A Keller

Marc Cavazza

Richard Vidal

David Lindner

Xuechen (Chen) Li

Alex M. Lamb

Tristan Aumentado-Armstrong

Ferdinando Fioretto

Alain Rossier

Wentao Zhang

Varun Jampani

Derek Lim

Muchen Li

Cong Hao

Yao-Yuan Yang

Linyi Li

Stéphane D’Ascoli

Lang Huang

Maxim Kodryan

Hao Bian

Orestis Paraskevas

David Madras

Tommy Tang

Li Sun

Stefano V Albrecht

Tristan Karch

Muhammad A Rahman

Runtian Zhai

Benjamin Black

Karan Singhal

Lin Gao

Ethan Brooks

Cesar Ferri

Dylan Campbell

Xujiang Zhao

Jack Parker-Holder

Michael Norrish

Jonathan Uesato

Yang An

Maheshakya Wijewardena

Ulrich Neumann

Lucile Ter-Minassian

Alexander Matt Turner

Subhabrata Dutta

Yu-Xiang Wang

Yao Zhang

Joanna Hong

Yao Fu

Wenqing Zheng

Louis C Tiao

Hajime Asama

Chengchun Shi

Moira R Dillon

Yisong Yue

Aurélien Bellet

Yin Cui

Gang Hua

Jongheon Jeong

Martin Klissarov

Aran Nayebi

Fabio Maria Carlucci

Chao Ma

Sébastien Gambs

Rasoul Mirzaiezadeh

Xudong Shen

Julian Schrittwieser

Adhyyan Narang

Fuxin Li

Linxi Fan

Johannes Gasteiger

Karthik Abinav Sankararaman

Patrick Mineault

Akhilesh Gotmare

Jibang Wu

Mikel Landajuela

Jinglin Liu

Qinghua Hu

Noah Siegel

Ashkan Khakzar

Nathan Grinsztajn

Julian Lienen

Xiaoteng Ma

Mohamad H Danesh

Ke ZHANG

Feiyu Xiong

Wonjae Kim

Michael Arbel

Piotr Skowron

Lê-Nguyên Hoang

Travers Rhodes

Liu Ziyin

Hossein Azizpour

Karl Tuyls

Hangyu Mao

Yi Ma

Junyi Li

Yong Cheng

Aditya Bhaskara

Xia Li

Danijar Hafner

Brian Quanz

Fangzhou Luo

Luca Cosmo

Scott Fujimoto

Santu Rana

Michael Curry

Karol Hausman

Luyao Yuan

Samarth Sinha

Matthew McLeod

Hao Shen

Navid Naderializadeh

Alessio Micheli

Zhenbang You

Van Huy Vo

Chenyang Wu

Thanard Kurutach

Vincent Conitzer

Chuang Gan

Chirag Gupta

Andreas Schlaginhaufen

Ruben Ohana

Luming Liang

Marco Fumero

Paul Muller

Hana Chockler

Ming Zhong

Jiamou Liu

Sumeet Agarwal

Eric Winsor

Ruimeng Hu

Changjian Shui

Yiwei Wang

Joey Tianyi Zhou

Anthony L. Caterini

Guillermo Ortiz-Jimenez

Iou-Jen Liu

Jiaming Liu

Michael Perlmutter

Anurag Arnab

Ziwei Xu

John Co-Reyes

Aravind Rajeswaran

Roy Fox

Yong-Lu Li

Carl Yang

Divyansh Garg

Amit Dhurandhar

Harris Chan

Tobias Schmidt

Robi Bhattacharjee

Marco Nadai

Reid McIlroy-Young

Wooseok Ha

Jesse Mu

Neale Ratzlaff

Kenneth Borup

Binghong Chen

Vikas Verma

Walter Gerych

Shachar Lovett

Zhengyu Zhao

Chandramouli Chandrasekaran

Richard Higgins

Nicholas Rhinehart

Blaise Agüera Y Arcas

Santiago Zanella-Beguelin

Dian Jin

Scott Niekum

Colin A. Raffel

Sebastian Goldt

Yali Du

Bernardo Subercaseaux

Hui Wu

Vincent Mallet

Ozan Özdenizci

Timothy Hospedales

Lingjiong Zhu

Cheng Soon Ong

Shahab Bakhtiari

Huan Zhang

Banghua Zhu

Byungjun Lee

Zhenyu Liao

Adrien Ecoffet

Vinay Ramasesh

Jesse Zhang

Soumik Sarkar

Nandan Kumar Jha

Daniel S Brown

Neev Parikh

Chen-Yu Wei

David K. Duvenaud

Felix Petersen

Songhua Wu

Huazhu Fu

Roger B Grosse

Matteo Papini

Peter Kairouz

Burak Varici

Fabio Roli

Mohammad Zalbagi Darestani

Jiamin He

Lys Sanz Moreta

Xu-Hui Liu

Qianchuan Zhao

Yulia Gel

Jan Drgona

Sajad Khodadadian

Takeshi Teshima

Igor T Podolak

Naoya Takeishi

Man Shun Ang

Mingli Song

Jakub Tomczak

Lukasz Szpruch

Micah Goldblum

Graham W. Taylor

Tomasz Korbak

Maheswaran Sathiamoorthy

Lan-Zhe Guo

Simone Fioravanti

Lei Jiao

Davin Choo

Kristy Choi

Varun Nair

Rayana Jaafar

Amy Greenwald

Martin V. Butz

Aleksey Tikhonov

Samuel Gruffaz

Yash Savani

Rui Chen

Ke Sun

We thank FTX Future Fund for funding this project.

Suggested citation

Zach Stein-Perlman, Benjamin Weinstein-Raun, Katja Grace, “2022 Expert Survey on Progress in AI.” AI Impacts, 3 Aug. 2022. https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/.

[END]
---
[1] Url: https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/

Published and (C) by Our World in Data
Content appears here under this condition or license: Creative Commons BY.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/ourworldindata/

You are viewing proxied material from magical.fish. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.