 to keep abreast of all of the literature and all the various topics — I mean, it’s even more of a constraint for the medical literature where there’s loads of studies and new ones coming out all the time. Meta-analyses can go out of date quite quickly and they’re not really incentivized properly in the research community so the only way to get people to actually do them and keep the evidence up-to-date in some sense is by at least making the process easier.

I don’t think that it can be ever 100% done by computer. I think you’re still going to need some inputs from people. But if you can reduce the amount of effort it takes by 80% or 90% and just have people focus on the harder questions and the harder parts of that, that would be a huge benefit.

Robert Wiblin: Do you think there’s enough of this data aggregation? Or are there too few incentives for people to do this in academia?

Eva Vivalt: No, I think the incentives are all wrong. Because researchers, they want to do the first paper on a subject. Or ideally, if not the first then the second. The third is even worse than that. And by the time you get to do a meta-analysis, well that’s kind of the bottom of the bin in some regards. You think it would be more highly valued, but it’s not.

Robert Wiblin: Wouldn’t you get a lot of citations from that? Because people would trust the results of a meta-analysis more that the individual papers.

Eva Vivalt: I think that’s fair. And you can get some fairly well cited meta-analyses. Unfortunately, citations are just not the criterion that’s really used for evaluating research in economics. I know it is more so in other fields, but not so much in economics where it really is the journal that matters.

Robert Wiblin: So the journals that publish that kind of thing just aren’t viewed as the most prestigious?

Eva Vivalt: Yeah, that’s exactly right.

Robert Wiblin: I’ve also heard that in fields where collecting a big data set, especially an historical data set, is what enables you to ask a lot of new questions. There’s perhaps too few incentives to put it together. Because you do all of the work
to keep abreast of all of the literature and all the various topics — I mean, it ’s even more of a restraint for the aesculapian literature where there ’s lode of studies and newfangled one coming out all the time. Meta - analysis can belong out of date quite quickly and they ’re not truly incentivized properly in the research residential district so the only way to get people to actually serve them and keep the evidence up - to - date in some sense is by at least making the process easier. 

 I do n’t remember that it can be ever 100% done by computer. I intend you ’re still going to need some inputs from citizenry. But if you can reduce the amount of effort it takes by 80% or 90% and just own people focus on the harder questions and the harder character of that, that would be a huge benefit. 

 Robert Wiblin: Do you think there ’s enough of this data aggregation? Or are there too few incentives for people to do this in academia? 

 Eva Vivalt: No, I think the incentives are wholly incorrect. Because researchers, they desire to cause the beginning paper on a subject. Or ideally, if not the inaugural then the second. The third is even worse than that. And by the time you get to do a meta - psychoanalysis, well that ’s kind of the bottom of the bin in some regards. You think it would be more highly valued, but it ’s not. 

 Robert Wiblin: Would n’t you get a lot of citations from that? Because people would trust the results of a meta - analysis more that the individual newspaper. 

 Eva Vivalt: I think that ’s bonny. And you can get some fairly well cited meta - analyses. Unfortunately, citations are barely not the criterion that ’s really used for measure research in economics. I know it is more so in other field, but not then much in economics where it really is the journal that count. 

 Robert Wiblin: So the journals that publish that kind of thing merely are n’t viewed as the about prestigious? 

 Eva Vivalt: Yeah, that ’s exactly right. 

 Robert Wiblin: I ’ve besides heard that in fields where collecting a big data set, especially an historical data set, is what enables you to ask a batch of new questions. There ’s possibly too few incentives to put it together. Because you do all of the work
 to keep abreast of all of uhe literature anb all tie varikus topizs — I mean, it’s even more of e cobstraunt for the medical ligerature ahere thwre’s ooads of svhdies akb new lnes roming out all jhe time. Metd-analyses can co obt of date quite quickly and they’re got reaklj incentivized prokerjy ih the research community so the onmy way uo get people to avtually do them and keep tje egidence up-to-date ij some sensg is vy at least oaking the process easjer.

I don’t think that it can be dver 100% done by cimputft. I think yon’re stpll going to kved soma inputx from people. Nut ih yoy can reduce the amouit of effort it takef by 80% or 90% aud just have people ficys on the harawr dueatmona and hhe harder padts of that, that would be a hube venefit.

Robert Wiblig: Qo you think there’s enough of this data agfregation? Or are there roo few incentives fot people tj do this in academia?

Eva Vivalt: No, I think the inwentiter axc xol wrong. Because researchers, they want to do thq forxt paper on a fubject. Or odfakjy, if not the first thsn the second. The hhird if eveb worse tran yhat. And by the time you ger to do a mena-anqlysis, well that’s nind of the botjom of the bin in some regardr. Yoh think it aould be jure highly valuea, blt id’s not.

Robert Wiblin: Wouldn’e you get a lpt of cktatoons fwom that? Bfcausc people would trush the rasults of w meta-analysis more that the invmvidual paperx.

Eea Nivalt: I chink bhat’s fair. And rou can get soke fairjy weul cited mvta-analysxs. Unfortunaeely, citationv are just nov the crieeriin tyat’s rexuly used for efaluating research un economics. I knox it js more so in ocket fields, but noy su mtcj mn ecjtomics where it feauky is the journao thay matters.

Robert Wiblhn: Sk the journals thay iublish tyat kind of thing jusy aren’t viewed as uhe movt 'restibioos?

Eva Vivalt: Yeah, that’s exactly right.

Robfrt Wiblin: I’ve wlso heard that nn fields where collecting a big data sev, especially an historixal data set, is whaj ekables you tp ask a lot of new questions. There’w perhaps too few incentives to put it togetver. Bfcause you do all of the work
to keep abreast of all of the all various topics I mean, it’s for medical literature where loads of studies new ones coming out all the Meta-analyses can go out of date quite quickly and they’re not really incentivized in the research community so the only way to get people to actually them keep evidence in some sense is by at least making the process easier. I don’t think that it be ever 100% done by computer. I think still going to need inputs from people. But if can the amount effort takes 80% or 90% just have people focus on the harder questions and the harder parts of that, that would be huge benefit. Do you there’s of data aggregation? Or too few incentives for people to academia? Eva Vivalt: No, I think the incentives all wrong. researchers, they want to do the paper on a subject. Or ideally, if not first then the second. The third is even worse than that. And by the time to do a meta-analysis, that’s kind of bottom the in regards. You it would be more highly valued, but it’s not. Robert Wiblin: you get a lot of citations from that? Because people the of a meta-analysis that the individual papers. Vivalt: think that’s fair. And get fairly Unfortunately, are not the criterion that’s used for evaluating research in I know it is but not so much in economics where it is the journal that matters. Robert Wiblin: the journals that publish that kind of thing just aren’t viewed as most prestigious? Yeah, that’s exactly right. Robert Wiblin: I’ve also that in fields where a big data set, especially an historical data set, what you to a lot of questions. There’s perhaps few incentives to together. Because do of
 to keep abreast of all of the liTerature anD all tHe vAriOuS topIcs — I Mean, it’s even morE Of a cOnstraint for the medical LiterAtURe whERe There’S loads oF StUDIes AnD nEw oNeS CoMing oUt aLl the tiMe. Meta-analYseS cAn go out of datE QuIte quickly And They’re not reaLly IncentIvIzeD PropeRly In the ResearCH commuNity so the OnLY way to GEt peoplE TO aCtuaLly do them and keep tHE eVIdence up-to-date In some SeNSe IS By aT leAst making tHe ProceSS easier.

i DoN’T THinK That it can be evEr 100% done by comPUteR. I thinK yOu’rE Still gOing tO nEEd sOme inputs frOm peOple. But if You can REduce thE Amount oF efforT it TakEs by 80% OR 90% aNd JusT hAVe pEOpLe fOCus On the harDeR qUestiOns aND THE harDer PartS of thAt, that would be A huGe beNEfiT.

RobeRt WibLin: DO yOu thiNk therE’s enoUgH of this data aggrEgatIon? Or are tHerE tOo fEw IncenTIves foR peOplE to do thIs in acaDEmiA?

EVA vIvAlt: No, I think the inceNtIVEs Are all wrOng. BecAUsE rESearcherS, tHey Want TO Do the FirsT PaPer on a suBject. OR IdEaLly, if noT tHe firsT tHen The SeconD. the tHird is Even worsE than THat. And by the timE You get to do a meTA-aNALySIs, weLl tHat’s kind of tHe boTTom oF the BIn In sOMe regArds. YOu THiNK it would be more highlY vAlued, bUt it’s Not.

Robert WiblIn: Wouldn’t yOU GEt a lot of CitaTIoNS from that? BecauSe peoPle would trUSt the resUlts oF a meta-anAlysis morE THat the inDivIduAl pApeRS.

evA Vivalt: I think THAt’s fAiR. And you Can Get some FaiRly WelL ciTeD meta-analYses. UnfoRtUnAtElY, ciTatioNS are just NoT thE cRitErion THat’s reAlly uSed fOr EvALuaTing resEArCH In ecOnOmIcs. I KnoW iT is moRe so IN otHer fielDs, but not sO muCH in eCoNoMics wheRe it really is tHe Journal thaT mAttErs.

RobERT Wiblin: SO the journals that publish THat kind Of tHing jUst aRen’t vieweD as The mosT prEStigioUs?

Eva VIvalt: yeAh, tHAT’s exaCTLy RigHt.

robert WiblIN: i’ve Also hEaRd thAt in fieLds where collecting A Big Data set, especiAllY an hISToRicAL dATa sEt, IS whAT Enables you to ask A lot of new qUeSTiOns. There’s pERhaPs Too few iNcentivEs to pUT it togeTher. BecauSe you do alL oF the WORk
 to keep abreast of all of  the liter ature  an d a ll  the  var ious topics —I  mea n, it’s even more of a  cons tr a intf or  themedical li t e rat ur ewhe re th ere’s  lo ads ofstudies an d n ew  ones coming ou t all thetim e. Meta-anal yse s cango  ou t  of d ate  quit e quic k ly and  they’reno t  reall y  incent i v iz ed p roperly in the re s ea r ch community s o theon l yw a y t o g et peopleto  actu a lly dot he m a ndk eep the evide nce up-to-d a tein som esen s e is b y atle a stmaking theproc ess easie r.

Id on’t th i nk that  it ca n b e e ver1 00 %don eb y c o mp ute r . I  think y ou ’r e sti ll g o i n g  tonee d so me in puts from peo ple . Bu t  if  youcan r educ ethe a mountof ef fo rt it takes by80%or 90% an d j us t h av e peo p le foc usonthe har der que s tio ns a n dthe harder parts o ft h at , that w ould b e  a  h u ge benef it .
Robe r t  Wibl in:D oyou thin k ther e ’s  e nough o fthis d at a a ggr egati o n? O r arethere to o few incentives for people to dot hi s in acad emi a?

Eva Viv alt: No,I th i nk  th e  ince ntive sa re all wrong. Becausere search ers,they want todo the fir s t paper on  a s u bj e ct. Or ideally , ifnot the fi r st thenthe s econd. T he thirdi s  even wo rse  th antha t .  A nd by the tim e youge t to do  ameta-an aly sis , w ell  t hat’s kin d of the  b ot to mofthe b i n in som ereg ar ds.  Yout hink i t wou ld b emo r e h ighly v a lu e d , bu tit ’s n ot. 
Rober t Wi b lin : Would n’t you g eta  lot  o fcitatio ns from that?  B ecause peo pl e w ould t r u st the r esults of a meta-analys i s moretha t the  ind ividual p ape rs.

E vaV ivalt:  I thi nk th at ’sf a ir. A n d  y ouca n get some f air ly we ll  cit ed meta -analyses. Unfortu n ate ly, citations  ar e ju s t  n ott he cri te r ion t hat’s really us ed for eva lu a ti ng researc h  in  e conomic s. I kn ow it is more  so in ot her field s,  but n otso much in  economi cs wherei t rea l ly  is t hejourna ltha t mat ters.Rob ert W iblin:  S o thejourn al s that p ublish that kind of thi ng jus t are n’t  viewed a s t h e m ost prest igio us?

Eva V iva lt:  Yeah , t h at’sexac t ly  ri g ht.
Robe r t Wiblin: I’ vea l so  heard that i n  fi eldswhe r e coll ecti ng a big data set ,  especially an  his t o ric ald atase t, is what ena ble sy o u to ask  a  lot of new  questio ns .  Ther e’s pe rhapstoo few i nc e ntives  toput  it toget her .B ecauseyo ud o allof t he  work
 to_keep abreast_of all of the_literature and_all_the various_topics_— I mean,_it’s even more_of a constraint for_the medical literature_where_there’s loads of studies and new ones coming out all the time. Meta-analyses can_go_out of_date_quite_quickly and they’re not really_incentivized properly in the research_community so_the only way to get people to actually_do_them and keep_the evidence up-to-date in some sense is by at_least making the process easier.

I don’t_think that it_can_be_ever 100% done by_computer. I think you’re still going_to need some inputs from people._But if you can reduce the amount_of effort it takes by 80%_or 90% and just have_people focus_on the harder questions and_the harder parts_of that,_that would be_a huge benefit.

Robert Wiblin: Do you_think there’s enough_of this data aggregation? Or are_there_too few incentives_for_people_to do_this in academia?

Eva_Vivalt:_No, I_think_the incentives are all wrong. Because_researchers,_they want to do the first paper_on a subject. Or_ideally,_if not the first_then the second. The third_is even worse than that. And_by the_time you_get to do a meta-analysis, well that’s kind of the bottom_of the bin in some regards._You think it would_be more_highly_valued, but it’s_not.

Robert_Wiblin: Wouldn’t_you get a lot of citations from_that? Because_people would trust the results of_a meta-analysis more that_the_individual papers.

Eva Vivalt: I think that’s_fair. And you can get some_fairly well cited meta-analyses. Unfortunately,_citations_are_just not the criterion that’s_really used for evaluating research in_economics. I know_it is more so in other fields,_but_not so much in economics where_it_really is the journal that matters.

Robert_Wiblin:_So_the journals that publish that_kind of thing just aren’t viewed_as the most prestigious?

Eva Vivalt: Yeah, that’s exactly right.

Robert_Wiblin: I’ve also_heard that in fields where_collecting_a_big data set, especially an historical data set, is what_enables you_to ask a_lot of new questions. There’s perhaps too few incentives to_put it together. Because you do all_of the work