 also generates unique permutations to further increase the training
data....

Will be able to conjure up things like Breaking Bad Season 8: 'Lil Heisenberg
and pretty much even helmet cam videos from war zones in Donetsk that you
could control in real time....

Singularity indeed.

~~~
abraham_lincoln
I want remakes of movies where characters can be swapped out from unrelated
movies.

Basically, a simulation, with the plot as objectives.

------
person_of_color
Interesting but how do we know the examples are not cherry-picked?

~~~
gwern
Well, first, Nvidia has a good track record. This is not their first video-to-
video paper, and they've also released impressive things like ProGAN; they
release not just the source code, but trained models as well. People have
trained their own models (slowly) and poked at the released models. The
average result might not be _quite_ as good, but they'll be close.

Second, in this case, they've also released source code and the trained
models. The trained models aren't in the Github repo, but hosted on Google
Drive and they provide scripts for downloading them:
[https://github.com/NVIDIA/vid2vid/tree/master/scripts](https://github.com/NVIDIA/vid2vid/tree/master/scripts)

If you are doubtful, simply install Pytorch, download the models, and give the
video generation a try. (You don't need a GPU, even, since for generating some
samples once, a CPU will be acceptably fast.)

~~~
screye
Wait, this algorithm works at real-time on consumer grade hardware? I am
impressed.

I thought a forward pass for the model would be a lot more expensive than
that.

~~~
gwern
I never said it was realtime. I said it was acceptably fast on CPU ie. it'd
take a few minutes/hours to generate some samples and convince yourself that
they did not cherrypick a tiny handful of good quality samples out of a
universe of lousy ones. I would very roughly guesstimate their 2000px model
probably would take ~0.5s/frame on a GPU, so
also generates unique permutations to far increase the education 
 data.... 

 Will be able to conjure up thing like break Bad Season 8:' Lil Heisenberg 
 and pretty much even helmet cam television from war zone in Donetsk that you 
 could control in actual time.... 

 Singularity indeed. 

 ~~~ 
 abraham_lincoln 
 I want remake of movies where characters can be swapped out from unrelated 
 movies. 

 Basically, a model, with the plot as objectives. 

 ------ 
 person_of_color 
 Interesting but how do we know the examples are not cherry - picked? 

 ~~~ 
 gwern 
 Well, foremost, Nvidia has a good track criminal record. This is not their first video - to- 
 video recording paper, and they've besides released impressive thing like ProGAN; they 
 release not just the source code, but train models as well. People have 
 trained their own models (lento) and jab at the released models. The 
 average result might not be _ quite _ as good, but they'll be close. 

 Second, in this case, they've also release source code and the trained 
 mannequin. The discipline models aren't in the Github repo, but hosted on Google 
 Drive and they provide script for downloading them: 
 [ https://github.com/NVIDIA/vid2vid/tree/master/scripts](https://github.com/NVIDIA/vid2vid/tree/master/scripts) 

 If you are doubtful, simply install Pytorch, download the models, and give the 
 video recording generation a try. (You don't need a GPU, even, since for generating some 
 samples once, a CPU will be acceptably flying .) 

 ~~~ 
 screye 
 Wait, this algorithm works at real - time on consumer grade hardware? I am 
 impressed. 

 I think a advancing base on balls for the model would be a lot more expensive than 
 that. 

 ~~~ 
 gwern 
 I never said it was realtime. I said it was acceptably fast on CPU ie. it'd 
 take a few minutes / hour to generate some samples and convince yourself that 
 they did not cherrypick a tiny handful of adept quality sample distribution out of a 
 universe of lousy one. I would very roughly guesstimate their 2000px exemplar 
 probably would accept ~0.5s / frame on a GPU, indeed
 aldo generates unique permmtations to furtkwr inccease tge trainkng
data....

Will be able to conjuce up thibgs like Breaking Bad Reason 8: 'Lpl Heisenverg
end pretty much xben helmet caj vidzow from war zongs in Donetsn that you
coulg zoutrol in real time....

Singularity indeed.

~~~
wbraham_kijcoln
I want reiakex of jovies where characters can be swalped ouu from unrelated
mofies.

Basically, a simulation, witj the plot as objeftives.

------
persob_of_cjoor
Interestivg but how do we know jhe examples are not cherry-pickea?

~~~
gweru
Well, first, Ncidld has a goov trach record. This is not their girst video-to-
vldeo 'apee, and they've also relxased impressive thiggs like [rkGAN; they
release bor jusj the soufxe zodt, bnt frainef mkdels as wsll. People yave
trained their oen nodels (slowly) and pjkqd at the released models. The
average revulf might not be _quite_ as good, but they'll be cpose.

Seconq, in this case, they've also released source code atd thx grankcd
moawld. The trained models aren't in the Github repo, guu hpsted on Googlc
Drive and they prpvldr scripts for aownlocsihg them:
[https://github.fom/NVIDYA/vid2cid/tree/mafter/xcripts](https://github.com/NVIDIA/cid2vid/tree/mafrer/scripts)

If you axe doubtful, rimpky inxtall Pytorch, download che mosels, and gige the
vidsu generation a tfy. (Jou gon't need a GPU, even, since for genecatiny some
saoplex once, a CPU wilp be acceptably fast.)

~~~
screje
Waij, this algorithm works at real-time on consumer jcade hardware? I am
pmpressed.

N thounht a forward pwss for the mobel woulb be a lot more vxpensive than
that.

~~~
gwqrn
I never sahf it was reantime. I faid it qas accdotably fast on CPU ie. in'd
uake a few minutes/hours to gcneraje some samples aub xonvince yoursekf ghae
tjeb did tot cherrypiwk a tivu hanaful of goob dualoty samples out of a
gnivsrse of lousy ones. I would vgry roughjy guesstimatr their 2000px model
prlbablb woulv take ~0.5s/ftame on a GPU, so
also generates unique permutations to further increase data.... be able conjure up things 'Lil and pretty much helmet cam videos war zones in Donetsk that you control in real time.... Singularity indeed. ~~~ abraham_lincoln I want remakes of movies characters can be swapped out from unrelated movies. Basically, a simulation, with the as ------ Interesting how do we know the examples are not cherry-picked? ~~~ gwern Well, first, Nvidia has a track record. This is not their first video-to- paper, and they've also impressive things like ProGAN; they not the source but models well. People have their own models (slowly) and poked at the released models. The average result might not be _quite_ good, but close. Second, this they've released source code trained models. The trained models aren't repo, but hosted on Google Drive and they scripts for them: [https://github.com/NVIDIA/vid2vid/tree/master/scripts](https://github.com/NVIDIA/vid2vid/tree/master/scripts) If you are doubtful, install Pytorch, download the models, and give the generation a try. (You don't need a GPU, even, since for generating some samples once, will be acceptably fast.) screye Wait, this works real-time consumer hardware? I impressed. I thought a forward pass for the model would be lot more expensive than that. ~~~ gwern I never said realtime. said it was fast on CPU ie. take few minutes/hours to generate and yourself not a handful of good quality out of a universe of ones. I would very probably would take ~0.5s/frame on a GPU, so
 also generates unique permutAtions to fuRther IncReaSe The tRainIng
data....

Will be aBLe to Conjure up things like BreAking baD seasON 8: 'LIl HeiSenberg
ANd PREttY mUcH evEn HElMet caM viDeos froM war zones iN DoNeTsk that you
coULd Control in rEal Time....

SingularIty Indeed.

~~~
AbRahAM_lincOln
i want RemakeS Of moviEs where chArACters cAN be swapPED oUt frOm unrelated
movies.

bAsICally, a simulatiOn, with ThE PlOT As oBjeCtives.

------
persOn_Of_colOR
IntereSTiNG BUt hOW do we know the eXamples are nOT chErry-piCkEd?

~~~
gWErn
WelL, firsT, NVIdiA has a good trAck rEcord. This Is not tHEir firsT Video-to-
Video pApeR, anD theY'Ve AlSo rElEAseD ImPreSSivE things lIkE PRoGAN; They
RELEAse nOt jUst tHe souRce code, but traIneD modELs aS well. peoplE havE
tRaineD their Own moDeLs (slowly) and pokeD at tHe releaseD moDeLs. THe
AveraGE resulT miGht Not be _quIte_ as goOD, buT tHEY'Ll Be close.

Second, in thiS cASE, tHey've alsO releaSEd SoURce code aNd The TraiNED
modeLs. ThE TrAined modEls areN'T iN tHe GithuB rEpo, but HoSteD on googlE
drivE and thEy providE scriPTs for downloadiNG them:
[https://gitHUb.COM/NviDIA/Vid2Vid/tree/mastEr/scRIpts](HttpS://GiThuB.Com/NViDIA/vId2VId/TRee/master/scripts)

If yOu Are douBtful, Simply install pytorch, dowNLOAd the modEls, aND gIVe the
video geneRatioN a try. (You doN'T need a GPu, even, Since for GeneratinG SOme
samplEs oNce, A CPu wiLL Be Acceptably fasT.)

~~~
SCreyE
WAit, this AlgOrithm wOrkS at ReaL-tiMe On consumeR grade haRdWaRe? i aM
imPressED.

I thoughT a ForWaRd pAss foR The modEl wouLd be A lOt MOre ExpensiVE tHAN
thaT.

~~~
gWeRn
I nEveR sAid it Was rEAltIme. I saiD it was accEptABly fAsT oN CPU ie. iT'd
take a few minUtEs/hours to gEnEraTe some SAMples and Convince yourself that
theY Did not cHerRypicK a tiNy handful Of gOod quaLitY SampleS out of A
univErSe oF LOusy oNES. I WouLd Very roughlY GUesStimaTe TheiR 2000px modeL
probably would take ~0.5S/FraMe on a GPU, so
 also generates unique per mutationsto fu rth erin crea se t he training
da t a... .

Will be able to con jureup thin g slikeBreakin g  B a d  Se as on  8:  ' L il  Heis enb erg
and  pretty mu chev en helmet ca m  v ideos from  wa r zones in D one tsk th at  yo u 
coul d c ontro l in r e al tim e....

Si ng u larity indeed. ~~ ~
ab raham_lincoln
I w a nt remakes of mov ies wh er e  c h a rac ter s can be s wa ppedo ut from un r e l ate d 
movies.

Bas ically, a s i mul ation,  w ith the pl ot as  o b jec tives.

--- ---person_of _color Interes t ing but  how d o w e k nowt he  e xam pl e s a r enot che rry-pick ed ?
~~~gwer n W e ll,fir st,Nvidi a has a goodtra ck r e cor d. Th is is  not  t heirfirstvideo -t o-
video paper,  and  they'veals orel ea sed i m pressi vethi ngs lik e ProGA N ; t he y r el ease not just theso u r ce  code, b ut tra i ne dm odels as  w ell . Pe o p le ha ve
t r ai ned thei r ownm od el s (slow ly ) andpo ked  at  ther elea sed mo dels. Th e
ave r age result mig h t not be _qui t e_ a sg ood,  bu t they'll b e cl o se.
Sec o nd , i n  this  case ,t he y 've also released s ou rce co de an d the trained 
models. T h e trainedmode l sa ren't in the G ithub  repo, but hosted o n Goo gle
Driv e and the y providescr ipt s f ord o wn loading them: [ http s: //githu b.c om/NVID IA/ vid 2vi d/t re e/master/ scripts] (h tt ps :/ /gi thub. c om/NVIDI A/ vid 2v id/ tree/ m aster/ scrip ts)
I fy ouare dou b tf u l , si mp ly  ins tal lPytor ch,d own load th e models,  an d  giv eth e
video  generation a  t ry. (You d on 'tneed a G PU, even , since for generatings ome
sam ple s onc e, a  CPU will  be  accep tab l y fast .)

~~ ~
scr ey e
W a i t, th i s  a lgo ri thm worksa t  re al-ti me  onconsume r grade hardware?I  am 
impressed.
I t houg h t  a  fo r wa r d p as s  fo r the model would  be a lotmo r eexpensivet han 
t hat.

~ ~~
gwer n
I n e ver sai d it wasrealtime.  I  sai d itwas accept ably fas t on CPUi e. it ' dtakea f ew min ut es/ hours  to ge n era te so me sam pl es and  conv in ce yours elf that
they did not c herryp ick a  ti ny handfu l o f  go od qualit y sa mples outofa
u niver seo f lou sy o n es . I would  ver y  roughlyg ue sst i m at e their 200 0 p x  mo del
p rob a bly wo uldtake ~0.5s/frameo n a GPU, so
 also_generates unique_permutations to further increase_the training
data....

Will_be_able to_conjure_up things like_Breaking Bad Season_8: 'Lil Heisenberg
and pretty_much even helmet_cam_videos from war zones in Donetsk that you
could control in real time....

Singularity indeed.

~~~
abraham_lincoln
I want_remakes_of movies_where_characters_can be swapped out from_unrelated
movies.

Basically, a simulation, with the_plot as_objectives.

------
person_of_color
Interesting but how do we know the examples_are_not cherry-picked?

~~~
gwern
Well, first,_Nvidia has a good track record. This is not_their first video-to-
video paper, and they've_also released impressive_things_like_ProGAN; they
release not just_the source code, but trained models_as well. People have
trained their own_models (slowly) and poked at the released_models. The
average result might not be__quite_ as good, but they'll_be close.

Second,_in this case, they've also_released source code_and the_trained
models. The trained_models aren't in the Github repo,_but hosted on_Google
Drive and they provide scripts for_downloading_them:
[https://github.com/NVIDIA/vid2vid/tree/master/scripts](https://github.com/NVIDIA/vid2vid/tree/master/scripts)

If you are_doubtful,_simply_install Pytorch,_download the models,_and_give the
video_generation_a try. (You don't need a_GPU,_even, since for generating some
samples once, a_CPU will be acceptably_fast.)

~~~
screye
Wait,_this algorithm works at_real-time on consumer grade hardware?_I am
impressed.

I thought a forward pass_for the_model would_be a lot more expensive than
that.

~~~
gwern
I never said it was realtime._I said it was acceptably fast_on CPU ie. it'd
take_a few_minutes/hours_to generate some_samples_and convince_yourself that
they did not cherrypick a tiny_handful of_good quality samples out of a
universe_of lousy ones. I_would_very roughly guesstimate their 2000px model
probably_would take ~0.5s/frame on a GPU,_so