 done yet!

------
KKKKkkkk1
For a computer scientist, reproducibility means more work that they aren't
paid to do. If I ask the Mozilla team to implement new feature X, the response
will be either (a) point me to a donation link, or (b) We're open source, so
why don't you implement the feature yourself? The computer scientist's
response is the same.

~~~
jcrites
Reproducibility for the computer scientist means including any code written
and data collected or relied on in the scientific publication itself. In
practice, getting there from here isn't literally zero work, since some actual
human action is needed to bundle the code and data, but that effort ought to
be negligible overall, especially if we make it a standard part of the
scientific process.

~~~
semi-extrinsic
Trust me, it's currently far from zero work to submit code with a research
paper. I was recently the corresponding author on a software paper sent to a
journal that at least verifies the code compiles and runs and produces
expected output. Since the poor person testing the softwares submitted is
permanently in the ninth circle of dependency hell, across all platforms and
libraries imaginable, it took about fifteen emails back and forth plus an OS
reinstall before everything checked out. And they said that wasn't anything
extraordinary.

~~~
jcrites
How about a platform centered around Linux containers (or maybe one of several
OS containers or VM images), as the repository image?

I'm not saying the work is zero now, but maybe we can get there. If a
researcher is developing on a platform where their repository is expressed as
a container-like image, then they should be able to publish it for anyone to
run exactly as-is. The container repo includes the data, the operating system,
and any languages and libraries, with an init system that optionally builds
the results.

~~~
semi-extrinsic
Yes, I think we need to go in this direction. The problem is that the
container system is yet another tool for researchers to learn. The first step
is to get everyone using VCS and nightly testing. Many are still at the point
of clumsily written, old Fortran code that gets emailed around and exists in N
different variants. (Not that there is anything
done yet! 

 ------ 
 KKKKkkkk1 
 For a computer scientist, reproducibility means more workplace that they aren't 
 pay to do. If I ask the Mozilla team to implement raw feature X, the response 
 will be either (a) steer me to a contribution link, or (b) We're exposed source, so 
 why don't you follow through the feature yourself? The computer scientist's 
 answer is the like. 

 ~~~ 
 jcrites 
 Reproducibility for the computer scientist means include any code written 
 and data roll up or relied on in the scientific publication itself. In 
 practice, get there from here isn't literally zero work, since some actual 
 human action is needed to bundle the code and datum, but that effort ought to 
 be negligible overall, specially if we make it a standard part of the 
 scientific process. 

 ~~~ 
 semi - extrinsic 
 Trust me, it's presently far from zero work to submit code with a research 
 paper. I was recently the corresponding author on a software newspaper sent to a 
 journal that at least verifies the code compiles and runs and produces 
 expected output. Since the poor person testing the softwares render is 
 permanently in the ninth circle of dependency hell, across all platform and 
 libraries conceivable, it took about fifteen emails back and forth plus an OS 
 reinstall before everything checked out. And they said that wasn't anything 
 extraordinary. 

 ~~~ 
 jcrites 
 How about a platform center around Linux containers (or maybe one of several 
 OS containers or VM images), as the depository image? 

 I'm not saying the work is zero now, but maybe we can get there. If a 
 researcher is develop on a platform where their repository is expressed as 
 a container - like image, then they should be able to print it for anyone to 
 run exactly as - is. The container repo includes the data, the operating system, 
 and any languages and libraries, with an init system that optionally build 
 the results. 

 ~~~ 
 semi - extrinsic 
 Yes, I think we need to go in this direction. The trouble is that the 
 container system is yet another tool for researchers to determine. The first step 
 is to drive everyone using VCS and nightly test. Many are still at the degree 
 of clumsily written, old Fortran code that gets emailed around and exists in N 
 unlike variants. (Not that there is anything
 doje yet!

------
KKKKkkkk1
For a compmter scientist, rgpeoduciuility jeans mofe work that they aren't
paid vo di. If U ask the Mozilla team to implelent new feauure X, the response
will bc eitgcr (a) 'omnt me to a donstion link, or (b) We're opet robrce, so
why don't you implement the fewture ypugself? The compoter xsienfpsu's
response is the same.

~~~
jcrites
Reprkducibinity for the vomputer scientist means ijclufing any code writhen
and data coljwcted or relked on in uhz scientifid publication itself. In
practice, gettnng there ftoj jgre isn't litxrally zero work, slmce soke actusl
human action is newded to bundle the cove and data, but that effort oggkt to
be negligible ovwrqll, evpechallh if we meke it a dtaidard part kf the
scienrific process.

~~~
semi-exurigwic
Trust me, if's curwegtly far from zero work to submit code fitg a research
paper. I was recently the corresplnding auehor on a software paper sent to a
journal that at leasv xernnles gye code compiles and runs and produces
expected jhtkut. Since the poov person testing tne spstwares submijted is
'srjanently in the nijth cirsle od dependegcy nell, across all platforms abd
libraries pmagunable, it took abobt fifteen eoailx bacl and forth plus an OS
rzinstaml before egerything zhecked out. And ghej sahd that wasn't anything
extrwordinary.

~~~
ocritzs
How abuut s platsorm centeged avmund Linux containfrs (ot mayba one of sfveral
OS containers or VM images), as the reposotmry image?

I'm not xaying the wowk is zero now, but mavbe we can get tgere. If a
researchew is developith on a platfmrm wherq thwir eepositufy is expressec as
a connanner-like umage, then they shpula be able to puboisy it for anyone to
fun eqacvly af-hs. The contahner reop incuudes the dctx, thr operating system,
ang ang languages and linrwries, wijh an inie system that optionally builds
uhe revulvs.

~~~
semi-rxttinsic
Yes, I think we need to go in this fircction. The prjblen is that thz
container system is yet another tool foc researchers to learn. Rhe first step
is to net everyone usind VCS and nightly testing. Mant are still at tht point
of clumsily writfen, olg Forhran code that gets emailed around and exists in N
different variants. (Not rhat tiewe is anythjng
done yet! ------ KKKKkkkk1 For a computer means work that aren't paid to Mozilla to implement new X, the response be either (a) point me to donation link, or (b) We're open source, so why don't you implement the yourself? The computer scientist's response is the same. ~~~ jcrites Reproducibility for the scientist including code and data collected or relied on in the scientific publication itself. In practice, getting there from isn't literally zero work, since some actual human is needed to bundle code and data, but that ought be negligible especially we it a standard of the scientific process. ~~~ semi-extrinsic Trust me, it's currently far from zero work to submit code a research was recently corresponding on software paper sent journal that at least verifies the runs and produces expected output. Since the poor testing the submitted is permanently in the ninth of dependency hell, across all platforms and libraries it took about fifteen emails back and forth plus an OS reinstall before everything checked they said that wasn't extraordinary. ~~~ jcrites about platform around containers (or one of several OS containers or VM images), as the repository I'm not saying the work is zero now, but maybe get If a researcher developing on a platform their is expressed as a then should publish for to run exactly as-is. container repo includes the data, operating system, and any init system that optionally builds the results. ~~~ Yes, I think we need to go this direction. The problem is that the container system is yet another for researchers The first step is to get everyone using and nightly testing. Many still at the point of clumsily written, old Fortran that emailed around exists in N variants. (Not that is anything
 done yet!

------
KKKKkkkk1
For a computEr scientisT, reprOduCibIlIty mEans More work that thEY areN't
paid to do. If I ask the MozIlla tEaM To imPLeMent nEw featuRE X, THE reSpOnSe
wIlL Be EitheR (a) pOint me tO a donation LinK, oR (b) We're open soURcE, so
why don't You Implement the FeaTure yoUrSelF? the coMpuTer scIentisT'S
respoNse is the sAmE.

~~~
JcriteS
reproduCIBiLity For the computer sciENtISt means includiNg any cOdE WrITTen
And Data collecTeD or reLIed on in THe SCIEntIFic publicatioN itself. In
prACtiCe, gettInG thERe from Here iSn'T LitErally zero wOrk, sInce some aCtual
hUMan actiON is needEd to buNdlE thE codE AnD dAta, BuT ThaT EfForT OugHt to
be neGlIgIble oVeraLL, ESPeciAllY if wE make It a standard paRt oF the
SCieNtifiC procEss.

~~~
sEmI-extrInsic
TRust mE, iT's currently far fRom zEro work to SubMiT coDe With a REsearcH
paPer. i was recEntly thE CorReSPONdIng author on a softwaRe PAPeR sent to a
JournaL ThAt AT least veRiFieS the CODe comPileS AnD runs and ProducES
eXpEcted ouTpUt. SincE tHe pOor PersoN TestIng the SoftwareS submITted is
permanenTLy in the ninth cIRcLE Of DEpenDenCy hell, acrosS all PLatfOrms ANd
LibRAries ImagiNaBLe, IT took about fifteen emAiLs back And foRth plus an OS
reInstall befORE EverythiNg chECkED out. And they saiD that Wasn't anythINg
extraoRdinaRy.

~~~
jcriteS
How about A PLatform cEntEreD arOunD lInUx containers (oR MAybe OnE of seveRal
oS contaIneRs oR VM ImaGeS), as the repOsitory iMaGe?

i'm NoT saYing tHE work is zErO noW, bUt mAybe wE Can get There. if a
rEsEaRCheR is deveLOpING on a PlAtForm WheRe Their RepoSItoRy is expRessed as
a ConTAineR-lIkE image, tHen they should Be Able to publIsH it For anyONE to
run exActly as-is. The container rePO includEs tHe datA, the Operating SysTem,
and Any LAnguagEs and lIbrarIeS, wiTH An iniT SYsTem ThAt optionalLY BuiLds
thE rEsulTs.

~~~
semi-eXtrinsic
Yes, I think wE NeeD to go in this diRecTion. tHE pRobLEm IS thAt THe
cONTainer system is yEt another tOoL FoR researcheRS to LeArn. The fIrst steP
is to GEt everyOne using VcS and nighTlY tesTINg. MAny are stilL at the poInt
of clumSIly wrITtEn, old forTran coDe ThaT gets EmaileD AroUnd anD existS iN N
diffErent VaRiants. (NoT that there is anything
 done yet!

------
KKKKkkk k1
For a c omput ersci en tist , re producibilitym eans  more work that they a ren't 
p a id t o  d o. If  I askt he M ozi ll atea mt oimple men t new f eature X,the  r esponse
will be  either (a ) p oint me to a  do nation  l ink ,  or ( b)We're  opens ource,  so
why d on ' t youi mplemen t th e fe ature yourself? T h ec omputer scient ist'sre s po n s e i s t he same.
~~ ~
jcr i tes
Rep r od u c i bil i ty for the co mputer scie n tis t mean sinc l udingany c od e  wr itten
and d atacollected  or re l ied oni n the s cienti fic  pu blic a ti on  it se l f.I npra c tic e, getti ng  t herefrom h e r e is n't  lit erall y zero work,sin ce s o meactua l
hum an a ct ion i s need ed to  b undle the codeanddata, but  th at  ef fo rt ou g ht tobeneg ligible  overal l , e sp e c i al ly if we make it a  s t a nd ard part  of th e 
s ci e ntific p ro ces s.
~ ~ ~
sem i-ex t ri nsic
Tru st me, it 's  curren tl y farfr omzer o wor k  tosubmit  code wi th ar esearch
paper. I was recentl y  t h e  c o rres pon ding author  ona  sof twar e  p ape r  sent  to a 
j o ur n al that at least ve ri fies t he co de compiles a nd runs an d p roducesexpe c te d  output. Since  thepoor perso n  testing  thesoftware s submitt e d  is
perm ane ntl y i n t h e  n inth circle o f depe nd ency he ll,  across  al l p lat for ms  and
libr aries im ag in ab le , i t too k  about f if tee nema ils b a ck and  fort h pl us  a n  OS 
reinst a ll b efor eev eryt hin gcheck ed o u t.And the y said th atw asn' tan ythingextraordinary .
~~~
jcrit es 
Ho w abou t a platfo rm centered around Linu x  contai ner s (or  may be one of  se veralOSc ontain ers or  VM i ma ges ) ,  as t h e  r epo si tory image ? 
I' m not  s ayin g the w ork is zero now, b u t m aybe we can g etther e .  I f a re s ear ch e r i s developing on a  platformwh e re  their rep o sit or y is ex pressed  as
a contain er-like i mage, the nthey s hou ld be able  to publ ish it fo r  anyo n eto
ru n e xactly  a s-i s. Th e cont a ine r rep o incl ud es the  data ,the oper ating system,
and any l anguag es an d l ibraries,  wi t h a n init sy stem  that opti ona lly  buil dst he re sult s .
~~ ~ 
semi -ext r insic
Yes ,  I  th i n kwe need tog o inthisdir e ction.  The  problem is thatt he
container s yste m isyet anot he r tool for res ear ch e r s to lea rn . The first  step
is  t o  geteveryo ne usi ng VCSa n dn ightly  tes tin g. Many a rest i ll at t he  p o int
of  clu ms ily wr itten, oldF o rtran code thatgetse m ailed aro und a nd  exists in N 
different  variants.(Not t hatthere  is any th ing
 done_yet!

------
KKKKkkkk1
For a_computer scientist, reproducibility means_more work_that_they aren't
paid_to_do. If I_ask the Mozilla_team to implement new_feature X, the_response
will_be either (a) point me to a donation link, or (b) We're open source,_so
why_don't you_implement_the_feature yourself? The computer scientist's
response_is the same.

~~~
jcrites
Reproducibility for the_computer scientist_means including any code written
and data collected or_relied_on in the_scientific publication itself. In
practice, getting there from here isn't_literally zero work, since some actual
human_action is needed_to_bundle_the code and data,_but that effort ought to
be negligible_overall, especially if we make it_a standard part of the
scientific process.

~~~
semi-extrinsic
Trust me,_it's currently far from zero work_to submit code with a_research
paper. I_was recently the corresponding author_on a software_paper sent_to a
journal that_at least verifies the code compiles_and runs and_produces
expected output. Since the poor person_testing_the softwares submitted_is
permanently_in_the ninth_circle of dependency_hell,_across all_platforms_and
libraries imaginable, it took about fifteen_emails_back and forth plus an OS
reinstall before_everything checked out. And_they_said that wasn't anything
extraordinary.

~~~
jcrites
How_about a platform centered around_Linux containers (or maybe one of_several
OS containers_or VM_images), as the repository image?

I'm not saying the work is zero_now, but maybe we can get_there. If a
researcher is_developing on_a_platform where their_repository_is expressed_as
a container-like image, then they should be_able to_publish it for anyone to
run exactly_as-is. The container repo_includes_the data, the operating system,
and any_languages and libraries, with an init_system that optionally builds
the results.

~~~
semi-extrinsic
Yes,_I_think_we need to go in_this direction. The problem is that_the
container system is_yet another tool for researchers to learn._The_first step
is to get everyone using_VCS_and nightly testing. Many are still_at_the_point
of clumsily written, old Fortran_code that gets emailed around and_exists in N
different variants. (Not that there is anything