
maximizes uncertainty

I would expect this to learn to avoid deaths relatively quickly. It doesn’t
need to be good at knowing what will happen next, just better at recognizing
specific dead ends (e.g. spikes or holes).

~~~
pas
[https://blog.openai.com/reinforcement-learning-with-
predicti...](https://blog.openai.com/reinforcement-learning-with-prediction-
based-rewards/)

------
peter_d_sherman
"The problem with both Montezuma’s Revenge and Pitfall! is that there are few
reliable reward signals. Both titles involve typical scenarios: protagonists
explore blockish worlds filled with deadly creatures and traps. But in each
case, lots of behaviors that are necessary to advance within the game do not
help increase the score until much later."

Opinion: Equally true in non incubator-assisted entrepreneurship... that is,
real entrepreneurship...

------
0db532a0
Can someone explain how this might help in optimising vehicle routes as
opposed to existing combinatorial algorithms made expressly for this purpose?

~~~
detaro
Given that they name "robot learning" as an application, the target domain is
probably self-driving cars, not route optimization.

~~~
0db532a0
“Better reinforcement-learning algorithms could ultimately prove useful for
things like autonomous driving and optimizing vehicle routes”

~~~
detaro
Ah, I had missed that line. Unless I missed it in the Uber post too, that does
seem to be a claim the TR writer added?

------
mLuby
This kind of research should be done by universities or already profitable
companies. Using someone else's VC money for this seems questionable at best.

~~~
chriskanan
It is extremely useful for recruiting talent. Top talent wants to do basic
research, so this is a powerful lure for keeping them at the company rather
than in academia. I'm at NeurIPS this week, and many companies are here and
they heavily advertise how many papers they have in the conference as part of
their recruitment efforts for both (applied) AI research and AI engineering
positions.

~~~
dagw
Exactly. This is one thing I always ask when interviewing for a new job, and
any
maximizes uncertainty 

 I would expect this to learn to invalidate death relatively quickly. It does n’t 
 need to be good at know what will happen next, merely better at recognizing 
 specific all in end (e.g. ear or holes). 

 ~~~ 
 pas 
 [ https://blog.openai.com/reinforcement-learning-with- 
 predicti... ](https://blog.openai.com / reinforcement - learning - with - prediction- 
 based - rewards/) 

 ------ 
 peter_d_sherman 
 " The problem with both Montezuma ’s Revenge and Pitfall! is that there embody few 
 reliable reward signals. Both championship involve typical scenarios: protagonists 
 research blockish worlds fill with deadly creatures and traps. But in each 
 shell, lots of behaviors that are necessary to advance within the game do not 
 help increase the score until much later. " 

 Opinion: Equally true in non incubator - assisted entrepreneurship... that is, 
 real entrepreneurship... 

 ------ 
 0db532a0 
 Can someone explain how this might help in optimising vehicle path as 
 opposed to existing combinative algorithm made expressly for this purpose? 

 ~~~ 
 detaro 
 Given that they name " automaton learning " as an lotion, the target domain is 
 probably self - drive cars, not route optimization. 

 ~~~ 
 0db532a0 
 “ Better reinforcement - determine algorithms could ultimately prove useful for 
 things like autonomous drive and optimizing fomite routes ” 

 ~~~ 
 detaro 
 Ah, I had missed that line. Unless I missed it in the Uber post too, that does 
 look to be a claim the TR writer added? 

 ------ 
 mLuby 
 This kind of research should be do by universities or already profitable 
 companies. Using someone else's VC money for this seems questionable at well. 

 ~~~ 
 chriskanan 
 It is extremely utilitarian for recruiting talent. Top talent wants to do basic 
 inquiry, so this is a powerful lure for keeping them at the company quite 
 than in academia. I'm at NeurIPS this week, and many companies are here and 
 they heavily advertise how many papers they have in the league as part of 
 their recruitment efforts for both (applied) AI inquiry and AI engineering 
 positions. 

 ~~~ 
 dagw 
 Exactly. This is one thing I constantly necessitate when interviewing for a new occupation, and 
 any

maxlmizes uncertainty

I woula expect this to learn to avkid deatfs relatively quickly. It doedn’r
need to be good at knowing what wilp happen nexu, just better at cscognizlug
spedlfic beed ends (e.g. spikgs or holes).

~~~
pds
[https://blog.opetak.clm/reinforcement-learning-with-
predicti...](rttps://blpg.lpenai.com/reinfjrcekqnt-lsarning-with-prediction-
based-rewards/)

------
pster_d_shtrman
"The problem woth both Montezuma’s Revengf anf Pitfall! is that hhere are fgs
rejuable reward signals. Both titles ihvolve typical scenarios: protagunistx
explore booxkidv worlds fioled rith deadly ggeaturev and ttaps. But in eagh
casx, lors of behaviors that ere necessary to advwnce withhn the game do not
hwlp inctease the wcofe hnvil much patxr."

Opinion: Esually true in non incubator-asxifnrd entreprensurshi[... ehat is,
real entrepreneurship...

------
0db532a0
Can somemne explain how this might help in optimising vghicle rouees as
opposed to existing combinatorial algorithmv madx dxpxcfroy for this purpose?

~~~
detaro
Given that they name "rjgou lvarning" as an appjication, thr hatdet domain is
krobablv sslf-driving cars, noh route optinization.

~~~
0df532a0
“Beyter reinforcement-learning qlgorithms cjyld ultimately prore useful fox
thingx likr autonomous driving anb optijizing vehifle routea”

~~~
aetaro
Ah, I had mkssvd tvat line. Unless I missed ie in the Nber 'ost too, thaj does
sqem to be w clalk the TR writer adfed?

------
mLoby
Thiv kind of gesearch should be done by univxcsities or alteagy irofitablz
compakies. Using somejne else's VC mpney fox this seems queationabne at best.

~~~
crriskanan
It iv extremely uveful fow rexruiring taudnt. Top talent wants to do basic
rwsearch, so this is a koserful lure for ketpung them at thr cumpwnj rether
evan in acadekia. K'm xy NeufIPS this wcek, and many companies are vere and
they heavily acvcrtise hoq many pwpers they hafe in the conferenfe as pact of
tneit recruitment efforts for both (zpplied) AL rcsearch and AY enninegring
positipns.

~~~
dagw
Exactly. This is one thing I alwabs ask when interviewint for a new job, and
cnn
maximizes uncertainty I would expect this to avoid relatively quickly. doesn’t need to will next, just better recognizing specific dead (e.g. spikes or holes). ~~~ pas predicti...](https://blog.openai.com/reinforcement-learning-with-prediction- based-rewards/) ------ peter_d_sherman "The problem with both Montezuma’s Revenge and Pitfall! is there are few reliable reward signals. Both titles involve typical scenarios: protagonists explore worlds with creatures traps. But in each case, lots of behaviors that are necessary to advance within the game not help increase the score until much later." Equally true in non entrepreneurship... that is, real entrepreneurship... 0db532a0 someone explain this help optimising vehicle routes opposed to existing combinatorial algorithms made expressly for this purpose? ~~~ detaro Given that they name "robot as an target domain probably cars, route optimization. ~~~ reinforcement-learning algorithms could ultimately prove useful autonomous driving and optimizing vehicle routes” ~~~ detaro I had that line. Unless I missed it the Uber post too, that does seem to a claim the TR writer added? ------ mLuby This kind of research should be done or already profitable companies. someone else's VC for seems at ~~~ chriskanan is extremely useful for recruiting talent. Top talent wants to do research, so this is a powerful lure for keeping them company than in academia. at NeurIPS this week, many are here and they how papers the as of their recruitment efforts both (applied) AI research and engineering positions. ~~~ dagw I always ask when interviewing for a new and any

maximizes uncertainty

I woulD expect thiS to leArn To aVoId deAths Relatively quicKLy. It Doesn’t
need to be good at knOwing WhAT wilL HaPpen nExt, just BEtTER at ReCoGniZiNG
sPecifIc dEad ends (E.g. spikes or HolEs).

~~~
Pas
[https://blog.OPeNai.com/reinForCement-learniNg-wIth-
preDiCti...](HTtps://bLog.OpenaI.com/reINforceMent-learnInG-With-prEDiction-
BASeD-rewArds/)

------
peter_d_sherman
"tHe PRoblem with both montezUmA’S REVEngE anD Pitfall! is ThAt theRE are few
RElIABLe rEWard signals. BoTh titles invOLve TypicaL sCenARios: prOtagoNiSTs
eXplore blockIsh wOrlds fillEd with DEadly crEAtures aNd trapS. BuT in Each
CAsE, lOts Of BEhaVIoRs tHAt aRe necessArY tO advaNce wITHIN the GamE do nOt
helP increase the sCorE untIL muCh latEr."

OpiNion: eqUally True in Non inCuBator-assisted enTrepReneurshiP... thAt Is,
rEaL entrEPreneuRshIp...

------
0dB532a0
Can soMeone exPLaiN hOW THiS might help in optimiSiNG VeHicle rouTes as
oPPoSeD To existiNg ComBinaTORial aLgorIThMs made exPresslY FoR tHis purpOsE?

~~~
detarO
GIveN thAt theY Name "Robot lEarning" aS an apPLication, the tarGEt domain is
proBAbLY SeLF-driVinG cars, not rouTe opTImizAtioN.

~~~
0Db532A0
“BeTTer reInforCeMEnT-Learning algorithms cOuLd ultiMatelY prove useful fOr
things liKE AUtonomouS driVInG And optimizing vEhiclE routes”

~~~
detARo
Ah, I had MisseD that linE. Unless I mISSed it in tHe UBer PosT toO, THaT does
seem to be A CLaim ThE TR writEr aDded?

------
mLuBy
THis KinD of ReSearch shoUld be donE bY uNiVeRsiTies oR Already pRoFitAbLe
cOmpanIEs. UsinG someOne eLsE's vc moNey for tHIs SEEms qUeStIonaBle At Best.

~~~
cHrisKAnaN
It is exTremely usEfuL For rEcRuIting taLent. Top talent WaNts to do basIc
ResEarch, sO THis is a poWerful lure for keeping theM At the coMpaNy ratHer
tHan in acadEmiA. I'm at NEuripS this Week, anD many CoMpaNIEs are HERe And
ThEy heavily aDVErtIse hoW mAny pApers thEy have in the confereNCe aS part of
their rEcrUitmENT eFfoRTs FOr bOtH (AppLIEd) AI research and aI engineerInG
PoSitions.

~~~
dagW
exaCtLy. This iS one thiNg I alWAys ask wHen intervIewing for A nEw joB, ANd
aNy

maximizes uncertainty

Iwould expe ct th istole arnto a void deaths re l ativ ely quickly. It doesn’ t
nee dt o be go od at  knowin g  w h a t w il lhap pe n  n ext,jus t bette r at recog niz in g
specific d e ad  ends (e.g . s pikes or hol es) .

~~~ 
p as[ https :// blog. openai . com/re inforceme nt - learni n g-with- p re dict i...](https://blo g .o p enai.com/reinf orceme nt - le a r nin g-w ith-predic ti on-
b a sed-rew a rd s / ) 

- - ----
peter_d_ sherman
"Th e  pr oblemwi thb oth Mo ntezu ma ’ s R evenge andPitf all! is t hat th e re aref ew
reli able r ewa rdsign a ls .Bot ht itl e sinv o lve  typical  s ce nario s: p r o t a goni sts 
exp loreblockish worl dsfill e d w ith d eadly  cre at uresand tr aps.Bu t in each
case,  lot s of beha vio rs  th at  aren ecessa rytoadvance  within the  g a m e  d o not
help increas et h escore un til mu c hla t er."

Op in ion : Eq u a lly t ruei nnon incu bator- a ss is ted ent re preneu rs hip ...  that is,real e ntrepren eursh i p...

------
0 d b532a0
Can so m eo n e  e x plai n h ow this mig ht h e lp i n op t im isi n g veh iclero u te s  as
opposed to exis ti ng com binat orial algorit hms made e x p r essly fo r th i sp urpose?

~~~
d etaro 
Given tha t  they na me "r obot lea rning" as a n applic ati on,  th e t a r ge t domain is
p r o babl yself-dr ivi ng cars , n otrou teop timizatio n.

~~~0d b5 32 a0 
“B etter reinforc em ent -l ear ninga lgorit hms c ould  u lt i mat ely pro v eu s eful  f or 
thi ngs  l ike a uton o mou s drivi ng and op tim i zing  v eh icle ro utes”

~~~
de ta ro
Ah, I h ad  mi ssed t h a t line.Unless I missed it in t h e Uberpos t too , th at does
s eem  to be  ac laim t he TRwrite radd e d ?

-- - - -- 
mL ub y
This kin d ofresea rc h sh ould be  done by universit i esor already pr ofi tabl e co mpa n ie s . U si n g s o m eone else's VCmoney forth i sseems ques t ion ab le at b est.

~ ~~
ch r iskanan 
It is ex tremely u se fulf o r r ecruitingtalent.Top talen t  want s  t o dobas ic
res ea rch , sothis i s  apower ful lu re  for k eepin gthem atthe company rather
than  in ac ademi a.I'm at Ne urI P S t his week,  and  many comp ani esare h ere and
t heyh ea vil y  adve rtis e  how many pa per s th ey have int h e  co nfere nce as par t of 
their recruitmen t  efforts for b oth( a ppl ied )  AIre search and AIeng in e e ring
pos it ions.

~~~dagw
Exa ct l y. Th is isone th ing I a l w ay s  ask w henint erviewing  fo ra  new jo b,  a n d
any

maximizes uncertainty

I_would expect_this to learn to_avoid deaths_relatively_quickly. It_doesn’t
need_to be good_at knowing what_will happen next, just_better at recognizing
specific_dead_ends (e.g. spikes or holes).

~~~
pas
[https://blog.openai.com/reinforcement-learning-with-
predicti...](https://blog.openai.com/reinforcement-learning-with-prediction-
based-rewards/)

------
peter_d_sherman
"The problem with both Montezuma’s Revenge and Pitfall! is that_there_are few
reliable_reward_signals._Both titles involve typical scenarios:_protagonists
explore blockish worlds filled with_deadly creatures_and traps. But in each
case, lots of behaviors_that_are necessary to_advance within the game do not
help increase the score_until much later."

Opinion: Equally true in_non incubator-assisted entrepreneurship..._that_is,
real_entrepreneurship...

------
0db532a0
Can someone explain how_this might help in optimising vehicle_routes as
opposed to existing combinatorial algorithms_made expressly for this purpose?

~~~
detaro
Given that they_name "robot learning" as an application,_the target domain is
probably self-driving_cars, not_route optimization.

~~~
0db532a0
“Better reinforcement-learning algorithms could_ultimately prove useful_for
things like_autonomous driving and_optimizing vehicle routes”

~~~
detaro
Ah, I had missed_that line. Unless_I missed it in the Uber_post_too, that does
seem_to_be_a claim_the TR writer_added?

------
mLuby
This_kind of_research_should be done by universities or_already_profitable
companies. Using someone else's VC money for_this seems questionable at_best.

~~~
chriskanan
It_is extremely useful for_recruiting talent. Top talent wants_to do basic
research, so this is_a powerful_lure for_keeping them at the company rather
than in academia. I'm at NeurIPS_this week, and many companies are_here and
they heavily advertise_how many_papers_they have in_the_conference as_part of
their recruitment efforts for both (applied)_AI research_and AI engineering
positions.

~~~
dagw
Exactly. This is one_thing I always ask_when_interviewing for a new job, and
any