 simple: if a group of units is moving across the map and one unit trails behind, AlphaStar will send the group back to collect the straggler. There is safety in numbers, and this is a habit human players pick up. Some behaviours are advanced: AlphaStar expends extra resources to create more workers than can be put to use mining. This might seem wasteful, but the backup workers allow AlphaStar to maintain the maximum mining rate after its base is attacked and workers are killed. According to one analysis I’ve seen, the extra productivity following an attack more than outweighs the cost of the backup workers. It seems like AlphaStar may have discovered a new, better strategy that until now has escaped even professional human players.

Even where AlphaStar has an unfair advantage against human players, what it has learned is still impressive. It deftly moves units in and out of the line of fire to avoid attacks while dealing damage. The fact that it knows that it should do this, and it knows how to do it is remarkable. To me, this feels analogous to some of the robotics challenges that engineers are trying to solve today. A self-driving car needs to keep a safe distance away from other road users, while still being aggressive enough to push around a car that’s parallel parking, or a truck that’s unloading in the middle of the street. That’s the same sort of task AlphaStar has learned: keep away when it’s dangerous, press ahead when it’s safe.

AlphaStar learned to play in two stages that used two different machine learning techniques.

Stage 1: Imitation learning

Blizzard, the company that makes StarCraft, has released millions of anonymized human-played games for use in AI research. This allowed DeepMind to use a technique called supervised imitation learning (one of a family of techniques that fall under the umbrella of imitation learning). Supervised imitation learning attempts to predict actions from states. In StarCraft, a state would be everything that AlphaStar knows or perceives about the current match at any given moment. An action would be anything it does in the game. After observing a huge number of state-action pairs in those millions of games played by humans, it tries to predict what a human would do and does that.

Supervised imitation learning is just the application of supervised learning, by far the most popular machine learning technique, to the problem of imitating human behaviour. A typical application of supervised
simple: if a group of units is moving across the map and one unit of measurement trail behind, AlphaStar will send the group back to collect the straggler. There is base hit in numbers, and this is a habit human player pluck up. Some behaviours are advance: AlphaStar use extra resource to create more workers than can be place to use mining. This might seem wasteful, but the backup workers allow AlphaStar to observe the maximum mining rate after its base is assail and workers are killed. According to one analysis I ’ve see, the extra productivity following an attack more than outweighs the monetary value of the backup workers. It seems like AlphaStar may have discovered a new, better strategy that until now has escaped even professional human players. 

 Even where AlphaStar has an unfair advantage against human players, what it has learned is still impressive. It deftly moves units in and out of the line of ardor to debar attacks while deal price. The fact that it knows that it should do this, and it knows how to serve it is remarkable. To me, this feels analogous to some of the robotics challenges that engineers are try to solve today. A self - driving car needs to keep a safe distance off from other road users, while still being aggressive enough to push around a cable car that ’s parallel parking, or a truck that ’s unload in the center of the street. That ’s the like sort of task AlphaStar has learned: keep away when it ’s dangerous, press ahead when it ’s safe. 

 AlphaStar learned to toy in two degree that used two different car learning techniques. 

 Stage 1: Imitation learning 

 Blizzard, the caller that makes StarCraft, has release millions of anonymized homo - played games for habit in AI research. This allowed DeepMind to practice a technique called supervised imitation learning (one of a family of techniques that fall under the umbrella of imitation learning). Supervised imitation learning attempts to bode actions from states. In StarCraft, a state of matter would be everything that AlphaStar knows or perceives about the current match at any given moment. An action would be anything it does in the game. After observe a huge number of state - action pairs in those millions of games play by world, it judge to predict what a human would cause and does that. 

 Supervised caricature learning is just the application of supervised learning, by far the most popular machine learning technique, to the problem of imitate human behaviour. A typical application of supervised
 silple: if a group of units is moving across the kap ans one unkt trails behind, AlphaStar wmll wend uke group back to coluect the dtragglee. Thtre is safety in ihmbers, and thjd is e habit human pkayers picn up. Some behaeiuuxs are advanced: AlphaStar expends exera respugces to create mort wjrkeds than can be put to use mining. Tgis migit seem wastefuk, but the backup workers aplow AlphaStar to mainhain the mazimui mining rate after its base is attzcked and workers are killed. Aczordiug to one abaoyslv I’ve seen, vhe exnra productivlny follmwing am attack more bhan mutqeighs the cost of thx backup workers. It feems lika ClphaStar may have diwcivereg a tew, cwttdr atcatsgy thwt nntil now hzs escaped wven professional himwb players.

Even where AjphaStar has an unfair advantage againsu humzn players, what it has oearned is still imprgssive. It qeftly moves units in and out of the line of fire to atokd cbbackr wjile dealing damage. The fact that it knows thwf ot should do thif, and it knpwd njw to do it ir remaxiagle. To me, this feeps analjgous to some jf tne robotics challenges that engineers age teying to solve todcy. A self-driring csr nerds to keep a safe distcnce asay from otjer road hrers, while still bepng dggressive enough to push wround a rar tkat’s parxllek parkyng, or a tguck bvat’s unloading in hhe mnddle of the stgeet. That’s the same sort of task AlphaStar hax neagned: keep away when it’s danderous, press akead wheu it’s rafe.

AlphaSnar learnxd to play ig two stages djat used two differegt mqchibe learvkng techniques.

Xtage 1: Imptction leaening

Blizzard, the gompavg that makes StcxCtqft, has releasrd oiljilnx jx anonymized humxn-pusyed eames for uwd in AI research. This alnowes DeepMind to use s bechnique called fupervised imotation learning (oje of a hamily of techniques that fall under ths umbrellw on imitation lqarnlng). Fupervised imitation learning attempts to predict actions from stajes. In StarCraft, a stctt would be evxrythigg that AnphaStar knows or peeceives about the current match at any given momejt. An action would be anything it does in the game. After observing a huge numbec jf state-actjon lairs iu tkose mijliois of games played by humans, it tries to predict xhat a humdn would do and does that.

Superfired imitation learning is just the apllicatiom of supervised learning, by far the most popular machinc learnint tecgnique, to vhe problem of imitaying hbman bwhavioue. A bypical applicxtipn of supergisev
simple: if a group of units is the and one trails behind, AlphaStar to the straggler. There safety in numbers, this is a habit human players up. Some behaviours are advanced: AlphaStar expends extra resources to create more workers can be put to use mining. This might seem wasteful, but the backup allow to the mining rate after its base is attacked and workers are killed. According to one analysis I’ve the extra productivity following an attack more than the cost of the workers. It seems like AlphaStar have a new, strategy until has escaped even human players. Even where AlphaStar has an unfair advantage against human players, what it has learned is impressive. It units in out the of fire to while dealing damage. The fact that it should do this, and it knows how do it remarkable. To me, this feels analogous some of the robotics challenges that engineers are to solve today. A self-driving car needs to keep a safe distance away from other while still being aggressive to push around car parallel or truck that’s in the middle of the street. That’s the same sort of AlphaStar has learned: keep away when it’s dangerous, press ahead safe. learned to play two stages that used different learning techniques. Stage 1: Blizzard, company has millions anonymized human-played games for in AI research. This allowed to use a technique of a family of techniques that fall under umbrella of imitation learning). Supervised imitation learning to predict actions from states. In StarCraft, a state would be everything AlphaStar knows about the current match at any given moment. action would be anything does in the game. After observing a huge number state-action in those of games played humans, it tries predict what a do and that. imitation the application of supervised learning, by the popular machine learning technique, to problem human behaviour. A typical application of supervised
 simple: if a group of units is moVing across The maP anD onE uNit tRailS behind, AlphaStAR wilL send the group back to colLect tHe STragGLeR. TherE is safeTY iN NUmbErS, aNd tHiS Is A habiT huMan playErs pick up. SOme BeHaviours are aDVaNced: AlphaSTar Expends extra ResOurces To CreATe morE woRkers Than caN Be put tO use mininG. THIs mighT Seem wasTEFuL, but The backup workers aLLoW alphaStar to maiNtain tHe MAxIMUm mIniNg rate afteR iTs basE Is attacKEd AND WorKErs are killed. ACcording to oNE anAlysis i’vE seEN, the exTra prOdUCtiVity followiNg an Attack morE than oUTweighs THe cost oF the baCkuP woRkerS. it SeEms LiKE AlPHastaR May Have discOvErEd a neW, betTER STratEgy That Until Now has escaped EveN proFEssIonal Human PlayErS.

Even Where ALphaSTaR has an unfair advAntaGe against HumAn PlaYeRs, whaT It has lEarNed Is still ImpressIVe. IT dEFTLy Moves units in and out Of THE lIne of firE to avoID aTtACks while DeAliNg daMAGe. The Fact THaT it knows That it SHoUlD do this, AnD it knoWs How To dO it is REmarKable. TO me, this fEels aNAlogous to some oF The robotics chALlENGeS That EngIneers are trYing TO solVe toDAy. a seLF-drivIng caR nEEdS To keep a safe distance AwAy from Other Road users, whilE still beinG AGGressive EnouGH tO Push around a car That’s Parallel paRKing, or a tRuck tHat’s unloAding in thE MIddle of tHe sTreEt. THat’S THe Same sort of tasK aLphastAr has leArnEd: keep aWay WheN it’S daNgErous, presS ahead whEn It’S sAfE.

AlPhaStAR learned To PlaY iN twO stagES that uSed twO difFeReNT maChine leARnING tecHnIqUes.

STagE 1: IMitatIon lEArnIng

BlizZard, the coMpaNY thaT mAkEs StarCRaft, has releasEd Millions of AnOnyMized hUMAn-played Games for use in AI research. tHis alloWed deepMInd tO use a techNiqUe callEd sUPervisEd imitAtion LeArnING (one oF A FaMilY oF techniqueS THat Fall uNdEr thE umbrelLa of imitation learnINg). SUpervised imitAtiOn leARNiNg aTTeMPts To PRedICT actions from staTes. In StarCRaFT, a State would BE evErYthing tHat AlphAStar KNows or pErceives aBout the cuRrEnt mATCh aT any given mOment. An aCtion woulD Be anyTHiNg it dOes In the gAmE. AfTer obServinG A huGe numBer of sTaTe-actiOn paiRs In those mIllions of games played by hUmans, iT trieS to Predict whAt a HUmaN would do aNd doEs that.

SupeRviSed ImitaTioN LearnIng iS JuSt tHE applIcatIOn of superVIsEd lEARnIng, by far the MOST poPular MacHIne leaRninG technique, to the prOBlem of imitatinG humAN BehAviOUr. A tYpIcal applicatioN of SuPERvised
 simple: if a group of uni ts is movi ng ac ros s t he  map  and  one unit trai l s be hind, AlphaStar will s end t he grou p  b ack t o colle c tt h e s tr ag gle r. Th ere i s s afety i n numbers,  an dthis is a ha b it  human pla yer s pick up. S ome  behav io urs are a dva nced:  Alpha S tar ex pends ext ra resour c es to c r e at e mo re workers than c a nb e put to use m ining.  T h is m igh t s eem wastef ul , but the bac k up w o rke r s allow Alpha Star to mai n tai n thema xim u m mini ng ra te aft er its base  isattackedand wo r kers ar e  killed . Acco rdi ngto o n ean aly si s  I’ v esee n , t he extra  p ro ducti vity f o l lowi ngan a ttack  more than ou twe ighs the  cost  of t he b ac kup w orkers . Itse ems like AlphaS tarmay havedis co ver ed  a ne w , bett erstr ategy t hat unt i l n ow h a sescaped even profe ss i o na l humanplayer s .
E v en where  A lph aSta r has a n un f ai r advant age ag a in st  humanpl ayers,  w hat  it  hasl earn ed isstill im press i ve. It deftlym oves units in an d ou t  ofthe  line of fi re t o  avo id a t ta cks while  deal in g  d a mage. The fact that  i t know s tha t it should d o this, an d i t knowshowt od o it is remark able.  To me, th i s feelsanalo gous tosome of t h e  robotic s c hal len ges t ha t engineers a r e  try in g to so lve  today.  Asel f-d riv in g car nee ds to ke ep  a  s af e d istan c e away f ro m o th erroadu sers,while  sti ll  b e ing  aggres s iv e enou gh  t o pu shar ounda ca r  th at’s pa rallel pa rki n g, o ratruck t hat’s unloadi ng  in the mi dd leof the s treet. T hat’s the same sort oft ask Alp haS tar h as l earned: k eep  awaywhe n  it’sdanger ous,pr ess a headw h en  it ’s  safe.

Al p h aSt ar le ar nedto play  in two stages tha t  us ed two differ ent  mac h i ne  le a rn i ngte c hni q u es.

Stage 1: I mitation l ea r ni ng

Blizza r d,th e compa ny that  make s  StarCr aft, hasreleasedmi llio n s  of  anonymize d human- played ga m es fo r  u se in  AI  resea rc h.Thisallowe d  De epMin d to u se  a tec hniqu ecalled s upervised imitation lea rning(oneofa familyoft ech niques th at f all underthe  um brell a o f  imit atio n  l ear n ing).  Sup e rvised im i ta tio n le arning atte m p t s t o pre dic t  actio ns f rom states. In St a rCraft, a stat e wo u l d b e e v eryt hi ng that AlphaS tar  k n o ws or pe rc eives about  the cur re n t mat ch atany gi ven mom e n t. An act ionwou ld be any thi ng it does  i nt he gam e. A ft er obs erving a hu g e  number of state -acti o n  pair s  in  thos emillion s  ofgames play ed by human s, ittrie s topredict  w hat ahum an  would doa nd does t hat.
Superv is ed i mit ationlear n i ng is  jus tthe  applicat i o no fsu p erv ised  lear ni ng,by far th e  most po pul a r machi ne  le a r ning t e ch n i que, to th e p roble m of imitati n g hu m an behav iour.A typi cal app l ica ti on of s upe r v ised
 simple:_if a_group of units is_moving across_the_map and_one_unit trails behind,_AlphaStar will send_the group back to_collect the straggler._There_is safety in numbers, and this is a habit human players pick up. Some_behaviours_are advanced:_AlphaStar_expends_extra resources to create more_workers than can be put_to use_mining. This might seem wasteful, but the backup_workers_allow AlphaStar to_maintain the maximum mining rate after its base is_attacked and workers are killed. According_to one analysis_I’ve_seen,_the extra productivity following_an attack more than outweighs the_cost of the backup workers. It_seems like AlphaStar may have discovered a_new, better strategy that until now_has escaped even professional human_players.

Even where_AlphaStar has an unfair advantage_against human players,_what it_has learned is_still impressive. It deftly moves units_in and out_of the line of fire to_avoid_attacks while dealing_damage._The_fact that_it knows that_it_should do_this,_and it knows how to do_it_is remarkable. To me, this feels analogous_to some of the_robotics_challenges that engineers are_trying to solve today. A_self-driving car needs to keep a_safe distance_away from_other road users, while still being aggressive enough to push around_a car that’s parallel parking, or_a truck that’s unloading_in the_middle_of the street._That’s_the same_sort of task AlphaStar has learned: keep_away when_it’s dangerous, press ahead when it’s_safe.

AlphaStar learned to play_in_two stages that used two different_machine learning techniques.

Stage 1: Imitation learning

Blizzard,_the company that makes StarCraft,_has_released_millions of anonymized human-played games_for use in AI research. This_allowed DeepMind to_use a technique called supervised imitation learning_(one_of a family of techniques that_fall_under the umbrella of imitation learning)._Supervised_imitation_learning attempts to predict actions_from states. In StarCraft, a state_would be everything that AlphaStar knows or perceives about_the current match_at any given moment. An_action_would_be anything it does in the game. After observing a_huge number_of state-action pairs_in those millions of games played by humans, it tries_to predict what a human would do_and does that.

Supervised imitation learning is just the application of supervised_learning, by far the most popular machine learning_technique, to the problem of imitating human_behaviour. A typical application_of_supervised