ron, multi-layer networks, backpropagation, recurrent networks, deep networks, etc. The most groundbreaking machine-learning advances may reduce error rates by a half or something, which may be commercially very important, but this is not many orders of magnitude as hard-takeoff scenarios tend to assume.

Outside of AI, the Internet changed the world, but it was an accumulation of many insights. Facebook has had massive impact, but it too was built from many small parts and grew in importance slowly as its size increased. Microsoft became a virtual monopoly in the 1990s but perhaps more for business than technology reasons, and its power in the software industry at large is probably not growing. Google has a quasi-monopoly on web search, kicked off by the success of PageRank, but most of its improvements have been small and gradual. Google has grown very powerful, but it hasn't maintained a permanent advantage that would allow it to take over the software industry.

Acquiring nuclear weapons might be the closest example of a single discrete step that most dramatically changes a country's position, but this may be an outlier. Maybe other advances in weaponry (arrows, guns, etc.) historically have had somewhat dramatic effects.

Bostrom doesn't present specific arguments for thinking that a few crucial insights may produce radical jumps. He suggests that we might not notice a system's improvements until it passes a threshold, but this seems absurd, because at least the AI developers would need to be intimately acquainted with the AI's performance. While not strictly accurate, there's a slogan: "You can't improve what you can't measure." Maybe the AI's progress wouldn't make world headlines, but the academic/industrial community would be well aware of nontrivial breakthroughs, and the AI developers would live and breathe performance numbers.

#2: Once an AI passes a threshold, it might be able to absorb vastly more content (e.g., by reading the Internet) that was previously inaccessible.

Absent other concurrent improvements I'm doubtful this would produce take-over-the-world superintelligence, because the world's current superintelligence (namely, humanity as a whole) already has read most of the Internet -- indeed, has written it. I guess humans haven't read all automatically generated text or vast streams of numerical data, but the insights gleaned purely from reading such material would be low without doing more sophisticated data mining and learning on top of it, and presumably such data mining would have already been in progress
ron, multi - layer networks, backpropagation, recurrent networks, deep net, etc. The about groundbreaking machine - eruditeness advances may abridge error rate by a one-half or something, which may be commercially very important, but this is not many orders of magnitude as hard - takeoff scenario tend to assume. 

 Outside of AI, the Internet changed the populace, but it was an accumulation of many insights. Facebook has had massive impact, but it too was build up from many small parts and grew in importance lento as its size increased. Microsoft became a virtual monopoly in the nineties but perhaps more for occupation than technology rationality, and its power in the software industry at large is credibly not growing. Google has a quasi - monopoly on web search, kicked off by the success of PageRank, but most of its improvements have been belittled and gradual. Google has grown very powerful, but it hasn't maintained a permanent advantage that would allow it to take over the software industry. 

 Acquiring nuclear weapons might be the closest example of a individual discrete step that most dramatically transfer a area's position, but this may be an outlier. Maybe other improvement in weaponry (arrow, guns, etc .) historically have had slightly dramatic effects. 

 Bostrom doesn't present specific arguments for thinking that a few crucial insights may grow radical jumps. He suggests that we might not notice a arrangement's improvements until it evanesce a threshold, but this seems absurd, because at least the AI developers would need to be intimately acquaint with the AI's performance. While not strictly accurate, there's a slogan: " You can't better what you can't measure. " Maybe the AI's progress wouldn't make worldly concern headlines, but the academic / industrial residential district would be well aware of nontrivial breakthroughs, and the AI developers would live and breathe operation numbers. 

 # 2: Once an AI passes a threshold, it might be able to absorb vastly more contented (e.g., by reading the Internet) that was previously inaccessible. 

 Absent early concurrent improvements I'm doubtful this would produce take - over - the - world superintelligence, because the world's current superintelligence (namely, humanity as a whole) already has read most of the Internet -- indeed, has written it. I guess humans haven't read all mechanically generated text or huge streams of numerical data, but the insight gleaned purely from reading such fabric would be low without doing more sophisticated data mining and learning on top of it, and presumably such data mining would have already been in progress
ron, multi-layer networks, bacypropagation, reeyrrent netwodks, deep networks, etc. The most grounvbreqking machine-learning advanzes may rvduce errir retes by a half oc somethlug, whjgh mav ue commercially very impostant, but this ir uot many orders of magnitude as hard-eakeoff sfenarios tend jo asxtme.

Ohnslde of AI, the Internet changed tge worlv, but it was an accumulation of many insihhts. Facebook has had lassive impqct, fyt it too war built from many smalm parts and grew in importance rlowlv as its siez infteased. Microwoft fecame a virblal monmpoly im the 1990s but pevhaps moee for business than vechnology reasons, agd its pofex in the software indysrry aj larce ir prubaglb nkt groainj. Google haa a quasi-mobopoly on web searcn, hpvked off by fhe suscqss of PageRank, but most of its improvekenfs have been small and tradual. Google has grlwn very [owerful, but it hasn't maintained a permanent advattage ghau qould qlpow it to take over the software industry.

Acquydimg nuclear weapoks might be the clpsfsy example of a singlz djscrete step that lost drwmatixally chagges a country's position, but thus may be an iutlier. Maybe othex advances iu weappnry (srrows, guns, etc.) historieally gave had solewhat drzoatic effects.

Bosgrok goesn't present specific arduments fir tkinking ghat a few crucial ijsighbv may produce radifal jomps. Ha suggests that we might not notice a sysvxm's improvememtv uttil it 'asses a threshold, fut this seems absurd, becajse at leaat the EI developerf would need dl be intimatxly acquayntee wirh the XK's performance. While non ftrixtly accurate, therc's a rmogan: "You can't nokrive what you csn't mewslre." Mwfbe the AI's [rogfesr woulan't makt wifld neadlines, but the acddemjc/industrial commumiby would ve well wware of nonttivial breakthrougjs, anv the EI devrlokers would live and breathe perrormance jumners.

#2: Once an WI pqsses a threxhold, it might be able to absorb vastly more content (e.g., bt reading the Interueu) that was prxviousjy inaccevsible.

Absent other cincurrent improvekents I'm doubtful thia woulg profuce take-over-the-world superintelligence, because the world's current superibtellijegce (namely, gumamity ds c whole) alrxavy has read most pf the Internet -- indeed, has wrivten it. I cuzss humans haven't read all auyooatically gengrated text or vast streama of numrrical data, but the insights glraned purely from readinn such mareriam would be low without doing more vophiwticatee daba mining and ueatnlng on top of mt, cnd presumably such data mining wiuld hqvf already been ii progress
ron, multi-layer networks, backpropagation, recurrent networks, deep The groundbreaking machine-learning may reduce error something, may be commercially important, but this not many orders of magnitude as scenarios tend to assume. Outside of AI, the Internet changed the world, but was an accumulation of many insights. Facebook has had massive impact, but it was from small and grew in importance slowly as its size increased. Microsoft became a virtual monopoly in the but perhaps more for business than technology reasons, its power in the industry at large is probably growing. has a on search, off by the of PageRank, but most of its improvements have been small and gradual. Google has grown very powerful, it hasn't permanent advantage would it take over the Acquiring nuclear weapons might be the a single discrete step that most dramatically changes country's position, this may be an outlier. Maybe advances in weaponry (arrows, guns, etc.) historically have somewhat dramatic effects. Bostrom doesn't present specific arguments for thinking that a few crucial insights radical jumps. He suggests we might not a improvements it a threshold, this seems absurd, because at least the AI developers would need be intimately acquainted with the AI's performance. While not strictly a "You can't improve you can't measure." Maybe AI's wouldn't make world headlines, academic/industrial would of breakthroughs, the AI developers would and breathe performance numbers. #2: an AI passes a to absorb vastly more content (e.g., by reading Internet) that was previously inaccessible. Absent other improvements I'm doubtful this would produce take-over-the-world superintelligence, because the world's current (namely, humanity whole) already has read most of the Internet indeed, has written it. guess humans haven't read all automatically generated text or streams numerical data, the insights gleaned from reading such would be low more sophisticated mining learning it, and presumably such data mining have been in progress
ron, multi-layer networks, backPropagatioN, recuRreNt nEtWorkS, deeP networks, etc. ThE Most Groundbreaking machine-lEarniNg ADvanCEs May reDuce errOR rATEs bY a HaLf oR sOMeThing, WhiCh may be CommercialLy vErY important, buT ThIs is not manY orDers of magnitUde As hard-TaKeoFF scenAriOs tenD to assUMe.

OutsIde of AI, thE INTernet CHanged tHE WoRld, bUt it was an accumulaTIoN Of many insights. faceboOk HAs HAD maSsiVe impact, buT iT too wAS built fROm MANY smALl parts and greW in importanCE slOwly as ItS siZE increAsed. MIcROsoFt became a viRtuaL monopoly In the 1990s BUt perhaPS more foR businEss ThaN tecHNoLoGy rEaSOns, ANd Its POweR in the soFtWaRe indUstrY AT LArge Is pRobaBly noT growing. GooglE haS a quASi-mOnopoLy on wEb seArCh, kicKed off By the SuCcess of PageRank, But mOst of its iMprOvEmeNtS have BEen smaLl aNd gRadual. GOogle haS GroWn VERY pOwerful, but it hasn't mAiNTAiNed a permAnent aDVaNtAGe that woUlD alLow iT TO take Over THe Software IndustRY.

ACqUiring nUcLear weApOns MigHt be tHE cloSest exAmple of a SinglE Discrete step thAT most dramaticALlY CHaNGes a CouNtry's positiOn, buT This May bE An OutLIer. MaYbe otHeR AdVAnces in weaponry (arroWs, Guns, etC.) histOrically have hAd somewhat DRAMatic effEcts.

bOsTRom doesn't preseNt speCific argumENts for thInkinG that a feW crucial iNSIghts may ProDucE raDicAL JuMps. He suggests THAt we MiGht not nOtiCe a systEm's ImpRovEmeNtS until it pAsses a thReShOlD, bUt tHis seEMs absurd, BeCauSe At lEast tHE AI devElopeRs woUlD nEEd tO be intiMAtELY acqUaInTed wIth ThE AI's pErfoRManCe. While Not strictLy aCCuraTe, ThEre's a slOgan: "You can't imPrOve what you CaN't mEasure." mAYbe the AI'S progress wouldn't make worLD headliNes, But thE acaDemic/induStrIal comMunITy woulD be welL awarE oF noNTRiviaL BReAktHrOughs, and thE ai deVelopErS wouLd live aNd breathe performanCE nuMbers.

#2: Once an AI PasSes a THReShoLD, iT MigHt BE abLE To absorb vastly mOre content (E.g., BY rEading the INTerNeT) that waS previoUsly iNAccessiBle.

Absent Other concUrRent IMProVements I'm dOubtful tHis would pROduce TAkE-over-The-World sUpEriNtellIgence, BEcaUse thE world'S cUrrent SuperInTelligenCe (namely, humanity as a wholE) alreaDy has ReaD most of thE InTErnEt -- indeed, hAs wrItten it. I guEss HumAns haVen'T Read aLl auTOmAtiCAlly gEnerATed text or VAsT stREAmS of numericaL DATa, bUt the InsIGhts glEaneD purely from readinG Such material woUld bE LOw wIthOUt doInG more sophisticAteD dATA mining aNd Learning on tOp of it, anD pREsumaBly sucH data mIning woULD hAVe alreAdy bEen In progresS
ron, multi-layer networks,  backpropa gatio n,rec ur rent  net works, deep ne t work s, etc. The most groun dbrea ki n g ma c hi ne-le arninga dv a n ces  m ay  re du c eerror  ra tes bya half orsom et hing, whichm ay  be commer cia lly very imp ort ant, b ut  th i s isnot  many  order s  of ma gnitude a sh ard-ta k eoff sc e n ar iostend to assume.
O ut s ide of AI, the  Inter ne t  c h a nge d t he world,bu t itw as an a c cu m u l ati o n of many ins ights. Face b ook  has h ad  ma s sive i mpact ,b utit too wasbuil t from ma ny sma l l parts and gre w in i mpo rta nces lo wl y a si tss iz e i n cre ased. Mi cr os oft b ecam e a virt ual  mon opoly  in the 1990s  bu t pe r hap s mor e for  bus in ess t han te chnol og y reasons, anditspower inthe  s oft wa re in d ustryatlar ge is p robably not  g r o w in g. Google has a qu as i - mo nopoly o n webs ea rc h , kicked  o ffby t h e  succ esso fPageRank , butm os tof itsim provem en tshav e bee n  sma ll and  gradual . Goo g le has grown v e ry powerful,b ut i th asn' t m aintained a  per m anen t ad v an tag e  that  woul da ll o w it to take over t he  softw are i ndustry.

Acq uiring nuc l e a r weapon s mi g ht be the closest  exam ple of a s i ngle dis crete  step th at most d r a maticall y c han ges  ac o un try's positio n ,  but  t his may  be  an out lie r.May beot her advan ces in w ea po nr y(ar rows, guns, et c. ) h is tor icall y  havehad s omew ha td ram atic ef f ec t s .

B os tr om d oes n' t pre sent spe cific a rgumentsfor thin ki ng  that a  few crucialin sights may  p rod uce ra d i cal jump s. He suggests that wem ight no t n otice  a s ystem's i mpr ovemen tsu ntil i t pass es ath res h o ld, b u t  t his  s eems absur d ,  be cause  a t le ast the  AI developers wou l d n eed to be int ima tely a cq uai n te d  wi th the A I's performance . While no ts tr ictly accu r ate ,there's  a slog an: " Y ou can' t improve  what you  c an't m eas ure." Mayb e the AI 's progre s s wou l dn 't ma keworldhe adl ines,  but t h e a cadem ic/ind us trialcommu ni ty would  be well aware of nontr ivialbreak thr oughs, an d t h e A I develop erswould live  an d b reath e p e rform ance nu mbe r s.

# 2: O n ce an AIp as ses a  t hreshold, i t m igh t beabl e  to ab sorb  vastly more cont e nt (e.g., by r eadi n g  th e I n tern et ) that was pre vio us l y  inacces si ble.

Absen t otherco n curre nt imp roveme nts I'm d ou b tful t hiswou ld produc e t ak e -over-t he -w o rld su peri nt ellige nce, b e caus e the world's curr ent s u p erint e lli gence  ( namely, huma nity as awhole) alre ady ha s re ad mo st of t he  Inter net  - - indeed,h as writte n it.  I gues shuma nshaven' t re a d  allauto ma tic ally gene r a te d  t ex t  or  vas t str ea ms o f numeric a l data,but the ins ig hts g leaned pu r e ly from re adi ng su c h  materialw ould be low w ithout  doing  more s o phi st icateddat a mining an d learnin g  on  t op o f it, an dpres um abl ysuch data minin g wouldhave alr e a dy bee n i n  pro g ress
ron, multi-layer_networks, backpropagation,_recurrent networks, deep networks,_etc. The_most_groundbreaking machine-learning_advances_may reduce error_rates by a_half or something, which_may be commercially_very_important, but this is not many orders of magnitude as hard-takeoff scenarios tend to_assume.

Outside_of AI,_the_Internet_changed the world, but it_was an accumulation of many_insights. Facebook_has had massive impact, but it too was_built_from many small_parts and grew in importance slowly as its size_increased. Microsoft became a virtual monopoly_in the 1990s_but_perhaps_more for business than_technology reasons, and its power in_the software industry at large is_probably not growing. Google has a quasi-monopoly_on web search, kicked off by_the success of PageRank, but_most of_its improvements have been small_and gradual. Google_has grown_very powerful, but_it hasn't maintained a permanent advantage_that would allow_it to take over the software_industry.

Acquiring_nuclear weapons might_be_the_closest example_of a single_discrete_step that_most_dramatically changes a country's position, but_this_may be an outlier. Maybe other advances_in weaponry (arrows, guns,_etc.)_historically have had somewhat_dramatic effects.

Bostrom doesn't present specific_arguments for thinking that a few_crucial insights_may produce_radical jumps. He suggests that we might not notice a system's_improvements until it passes a threshold,_but this seems absurd,_because at_least_the AI developers_would_need to_be intimately acquainted with the AI's performance._While not_strictly accurate, there's a slogan: "You_can't improve what you_can't_measure." Maybe the AI's progress wouldn't_make world headlines, but the academic/industrial_community would be well aware_of_nontrivial_breakthroughs, and the AI developers_would live and breathe performance numbers.

#2:_Once an AI_passes a threshold, it might be able_to_absorb vastly more content (e.g., by_reading_the Internet) that was previously inaccessible.

Absent_other_concurrent_improvements I'm doubtful this would_produce take-over-the-world superintelligence, because the world's_current superintelligence (namely, humanity as a whole) already has_read most of_the Internet -- indeed, has_written_it._I guess humans haven't read all automatically generated text or_vast streams_of numerical data,_but the insights gleaned purely from reading such material would_be low without doing more sophisticated data_mining and learning on top of it, and presumably such data_mining would have already been in progress