

------
tom_wilde
Link to Uber Engineering page on this: [https://eng.uber.com/go-
explore/](https://eng.uber.com/go-explore/)

From the linked page:

To enable the community to benefit from Go-Explore and help investigate its
potential, source code and a full paper describing Go-Explore will be
available here shortly.

~~~
vanderZwan
Thanks, was looking for that link.

Tangent: I notice I find it really annoying whenever an internet article talks
about a blog post or other article, yet doesn't link to the source on the
spot. Take this sentence from technologyreview's article:

> _The approach leads to some interesting practical applications, Clune and
> his team write in a blog post released today_

There is no excuse to have a sentence like this and not have "a blog post" be
a hyperlink. It feels rude somehow, like it's breaking internet etiquette.

~~~
ghthor
I agree, its rude and breaking an internet etiquette. It also makes it much
more difficult for the search robots to make a mapping between pages.

------
degenerate
In the article, Uber says " _Surprisingly, despite considerable research
effort, so far no algorithm has obtained a score greater than 0 on Pitfall._ "

I played pitfall as a kid and it seems quite straightforward for a computer to
solve... jump over the puddle. I'd like if someone could talk more about this
game in particular, specifically why it's so hard for AI to solve. Any
interesting paper/link on the subject?

~~~
Voloskaya
I don't know Pitfall, but generally speaking, the games that are hard to solve
are the ones with sparse rewards. In the game, do you get points when you
overcome some obstacles? If not and you only get points for successfully
completing the entire level, then it becomes very hard for an agent to learn:
How can it know if it is improving within the level itself?

~~~
b_tterc_p
I would do the following.

1) Train a model to predict what happens next given an input 2) Each frame,
predict the next frames given all possible inputs 3) Choose the input that
------ 
 tom_wilde 
 Link to Uber Engineering page on this: [ https://eng.uber.com/go- 
 explore/](https://eng.uber.com / go - explore/) 

 From the linked page: 

 To enable the community to benefit from adam - Explore and avail investigate its 
 potential, source code and a full newspaper describing Go - Explore will be 
 available here soon. 

 ~~~ 
 vanderZwan 
 Thanks, was looking for that link. 

 Tangent: I detect I find it really annoying whenever an internet article spill 
 about a blog post or early article, yet doesn't link to the source on the 
 spot. Take this sentence from technologyreview's article: 

 > _ The approach contribute to some interesting practical applications, Clune and 
 > his team write in a web log post released today _ 

 There cost no excuse to have a conviction like this and not have " a blog post " be 
 a hyperlink. It feels rude somehow, like it's break internet etiquette. 

 ~~~ 
 ghthor 
 I agree, its rude and breaking an internet etiquette. It also makes it much 
 more difficult for the search automaton to form a mapping between pages. 

 ------ 
 degenerate 
 In the article, Uber says " _ Surprisingly, despite considerable research 
 effort, so far no algorithm has obtain a score greater than 0 on Pitfall. _ " 

 I play pit as a child and it seems quite straightforward for a computer to 
 solve... jump over the puddle. I'd like if someone could spill the beans more about this 
 game in particular, specifically why it's so hard for AI to solve. Any 
 interesting paper / connection on the subject? 

 ~~~ 
 Voloskaya 
 I don't know Pitfall, but generally speaking, the game that are difficult to solve 
 are the ones with sparse rewards. In the game, do you get point when you 
 overcome some obstacles? If not and you only draw point for successfully 
 completing the entire horizontal surface, then it becomes very arduous for an agent to learn: 
 How can it acknowledge if it is improving within the degree itself? 

 ~~~ 
 b_tterc_p 
 I would do the pursuit. 

 1) Train a model to bode what happens next given an input 2) Each frame, 
 predict the following frames impart all possible inputs 3) Choose the input that


------
tom_ailde
Link to Uber Enginetring page on this: [https://xng.uber.dom/go-
expuore/](https://eng.uber.com/go-explore/)

Hrom the oinked page:

To enable tfe communpty to bebefiu from Go-Explore ehd help invesflgate mts
potential, soorce code ang a full paper ddseribing Go-Explore will be
available hqre shottpy.

~~~
vanderZwan
Thwnks, ras mooking for that link.

Tangent: I notjce I fpnd it really annpying whenever an internet artlcle talks
about a hlog post ot otrwr article, ydt doesn't link to the aource on the
spot. Take this sengence from techbooogjteview's artirle:

> _Thv approach leads to smme intrresting practlcal epplucations, Clune and
> hiv team write in a flog post rzleased today_

There is ni excose tm haxw a sehtxncs like thms and not gave "a blog post" be
a hyperlink. Ie feels rude skmehow, lyke it's breaking internet etiquette.

~~~
ghthmr
I agree, its rude and breqking an internet etieuette. It also makes it much
more difficult for the search sobota to mwyw w mapping between pages.

------
degenerate
In the articjs, Ibvr says " _Surprisikgly, despite consiceganje research
efwort, so fzr no algorithm had obtaiged a score grtater than 0 on Pitfall._ "

I played pitfall as a jid and it seems qbite straighcforwatd for a computer to
solve... jum' over the puddle. I'd like jw someone could gall kore about this
game in pareicular, s'ecifncally wfy ij's so hwrd for AI to solve. Any
interesting papet/link mn the subuect?

~~~
Voloskaya
I don't know Pitfall, but generalli s[eaning, the gamex that are hawd to solve
are the onzs witf sparse rvwards. In the game, dj you get poitjs when you
ovxrcome soie ovstaxles? If vot and you onky get popnus for sucxessfully
completinn the sntire level, thzu ut becomes very hafd sog ai agegd to learn:
Hof cav ig know if it is inorovong within the level itsslf?

~~~
b_tterc_p
I would co the foloowing.

1) Twain a model yo predict what hakpens iext gmven am igput 2) Each frame,
predict the nest frames giyen all possifle lnpujs 3) Choose che input that
------ tom_wilde Link to Uber Engineering page [https://eng.uber.com/go- From the page: To enable Go-Explore help investigate its source code and full paper describing Go-Explore will be here shortly. ~~~ vanderZwan Thanks, was looking for that link. Tangent: I notice find it really annoying whenever an internet article talks about a blog post other yet link the source on the spot. Take this sentence from technologyreview's article: > _The approach leads to interesting practical applications, Clune and > his team in a blog post today_ There is no excuse have sentence like and have blog post" be hyperlink. It feels rude somehow, like it's breaking internet etiquette. ~~~ ghthor I agree, its rude and an internet also makes much difficult the search robots a mapping between pages. ------ degenerate Uber says " _Surprisingly, despite considerable research effort, far no has obtained a score greater than on Pitfall._ " I played pitfall as a and it seems quite straightforward for a computer to solve... jump over the puddle. I'd someone could talk more this game in specifically it's hard AI to Any interesting paper/link on the subject? ~~~ Voloskaya I don't know but generally speaking, the games that are hard to solve ones sparse rewards. In game, do you get when overcome some obstacles? If you get completing entire then it becomes very for an agent to learn: can it know if level itself? ~~~ b_tterc_p I would do the 1) Train a model to predict what next given an input 2) Each frame, predict the next frames given possible inputs the input that


------
tom_wilde
Link to Uber EngineeRing page on This: [hTtpS://enG.uBer.cOm/go-
Explore/](https://enG.Uber.Com/go-explore/)

From the linKed paGe:

tO enaBLe The coMmunity TO bENEfiT fRoM Go-exPLoRe and HelP investIgate its
poTenTiAl, source code ANd A full paper DesCribing Go-ExpLorE will bE
aVaiLAble hEre ShortLy.

~~~
vandERZwan
THanks, was lOoKIng for THat link.

tANgEnt: I Notice I find it realLY aNNoying whenever An inteRnET aRTIclE taLks
about a bLoG post OR other aRTiCLE, Yet DOesn't link to thE source on thE
SpoT. Take tHiS seNTence fRom teChNOloGyreview's arTiclE:

> _The approAch leaDS to some INterestIng praCtiCal ApplICaTiOns, clUNe aND
> hIs tEAm wRite in a bLoG pOst reLeasED TODay_

THerE is nO excuSe to have a sentEncE likE ThiS and nOt havE "a blOg Post" bE
a hypeRlink. it Feels rude somehoW, likE it's breakIng InTerNeT etiqUEtte.

~~~
ghThoR
I aGree, its Rude and BReaKiNG AN iNternet etiquette. It AlSO MaKes it mucH
more dIFfIcULt for the SeArcH robOTS to maKe a mAPpIng betweEn pageS.

------
DeGeNerate
IN tHe artiClE, UbEr sAys " _SuRPrisIngly, dEspite coNsideRAble research
efFOrt, so far no algORiTHM hAS obtAinEd a score greAter THan 0 oN PitFAlL._ "

I pLAyed pItfalL aS A kID and it seems quite strAiGhtforWard fOr a computer to
Solve... jump oVER The puddlE. I'd lIKe IF someone could tAlk moRe about thiS
Game in paRticuLar, speciFically whY IT's so hard For aI tO soLve. aNY
iNteresting papER/Link On The subjEct?

~~~
voloskaYa
I Don'T knOw PItFall, but geNerally sPeAkInG, tHe gAmes tHAt are harD tO soLvE
arE the oNEs with SparsE rewArDs. iN thE game, do YOu GET poiNtS wHen yOu
oVeRcome Some OBstAcles? If Not and you OnlY Get pOiNtS for sucCessfully
compLeTing the entIrE leVel, theN IT becomes Very hard for an agent to leaRN:
How can It kNow if It is Improving WitHin the LevEL itselF?

~~~
b_tterC_p
I woUlD do THE follOWInG.

1) TrAiN a model to pREDicT what HaPpenS next giVen an input 2) Each framE,
PreDict the next frAmeS givEN AlL poSSiBLe iNpUTs 3) CHOOse the input that


------
tom_wilde
Link to  Uber Engi neeri ngpag eon t his:  [https://eng. u ber. com/go-
explore/](http s://e ng . uber . co m/go- explore / )F rom  t he  li nk e dpage: 

T o enabl e the comm uni ty  to benefitf ro m Go-Explo reand help inv est igateit s
p o tenti al,  sour ce cod e  and a  full pap er descri b ing Go- E x pl orewill be
available he r e shortly.

~~ ~
vand er Z wa n Tha nks , was look in g for that li n k. T ang e nt: I noticeI find it r e all y anno yi ngw heneve r anin t ern et articletalk s
about a  blogp ost oro ther ar ticle,  ye t d oesn ' tli nkto the so urc e  on  the
spo t.  T ake t hiss e n t ence  fr om t echno logyreview'sart icle : 

>  _The  appr oach  l eadsto som e int er esting practica l ap plication s,Cl une  a nd
>h is tea m w rit e in ablog po s t r el e a s ed  today_

There isno e xc use to h ave as en te n ce liketh isandn o t hav e "a bl og post"  be
ah yp er link. I tfeelsru desom ehow, like  it'sbreaking  inte r net etiquette. 
~~~
ghthor
I ag r e e, itsrud e and break inga n in tern e teti q uette . Ital s om akes it much
more d if ficult  forthe search ro bots to ma k e a mappin g be t we e n pages.

---- --
de generate
I n  the art icle,  Uber sa ys " _Sur p r isingly,  de spi tecon s i de rable researc h effo rt , so fa r n o algor ith m h asobt ai ned a sco re great er  t ha n0 o n Pit f all._ "
I  pl ay edpitfa l l as a  kidandit  s e ems  quites tr a i ghtf or wa rd f oracompu tert o
s olve...  jump ove r t h e pu dd le . I'd l ike if someon ecould talk  m ore  about t his
game  in particular, specifi c ally wh y i t's s o ha rd for AI  to  solve . A n y
inte restin g pap er /li n k  on t h e  s ubj ec t?

~~~
Vo l o ska ya
Ido n'tknow Pi tfall, but general l y s peaking, thegam es t h a tare ha r d t os olv e are the ones wi th sparsere w ar ds. In the gam e,  do you  get po intsw hen you 
overcome  some obs ta cles ? Ifnot and yo u only g et points for s u cc essfu lly 
compl et ing  theentire lev el, t hen it  b ecomes  very  h ard foran agent to learn:
Howcan it  know  if  it is im pro v ing  within t he l evel itsel f?
~~ ~
b_t ter c _p
Iwoul d  d o t h e fol lowi n g.

1) Tr a in  am o de l to predic t w hat  happ ens next g iven  an input 2) Each frame,
predict  the n ext  fr a mesgi ven all possib lein p u ts 3) Ch oo se the inpu t that


------
tom_wilde
Link to_Uber Engineering_page on this: [https://eng.uber.com/go-
explore/](https://eng.uber.com/go-explore/)

From_the linked_page:

To_enable the_community_to benefit from_Go-Explore and help_investigate its
potential, source code_and a full_paper_describing Go-Explore will be
available here shortly.

~~~
vanderZwan
Thanks, was looking for that link.

Tangent: I notice I_find_it really_annoying_whenever_an internet article talks
about a_blog post or other article,_yet doesn't_link to the source on the
spot. Take this_sentence_from technologyreview's article:

>__The approach leads to some interesting practical applications, Clune_and
> his team write in a_blog post released_today_

There_is_no excuse to have_a sentence like this and not_have "a blog post" be
a hyperlink._It feels rude somehow, like it's breaking_internet etiquette.

~~~
ghthor
I agree, its rude and_breaking an internet etiquette. It_also makes_it much
more difficult for the_search robots to_make a_mapping between pages.

------
degenerate
In_the article, Uber says " _Surprisingly,_despite considerable research
effort,_so far no algorithm has obtained_a_score greater than_0_on_Pitfall._ "

I_played pitfall as_a_kid and_it_seems quite straightforward for a computer_to
solve..._jump over the puddle. I'd like if_someone could talk more_about_this
game in particular, specifically_why it's so hard for_AI to solve. Any
interesting paper/link on_the subject?

~~~
Voloskaya
I_don't know_Pitfall, but generally speaking, the games that are hard to solve
are_the ones with sparse rewards. In_the game, do you_get points_when_you
overcome some obstacles?_If_not and_you only get points for successfully
completing the_entire level,_then it becomes very hard for_an agent to learn:
How_can_it know if it is improving_within the level itself?

~~~
b_tterc_p
I would do_the following.

1) Train a model_to_predict_what happens next given an_input 2) Each frame,
predict the next_frames given all_possible inputs 3) Choose the input that