in reply to: TurnTrout's comment on: Best rea­sons for pes­simism about im­pact of im­pact mea­sures?
Could GPT2 make a good weird sun twit­ter? Prob­a­bly not, but it could at least be a good in­spirobot.
It’s trained on the whole cor­pus of LW com­ments and replies that got suffi­ciently high karma; naively I wouldn’t ex­pect a day to make much of a dent in the train­ing data. But there’s an in­ter­est­ing fact about train­ing to match dis­tri­bu­tions, which is that most mea­sures of dis­tri­bu­tional over­lap (like the KL di­ver­gence) are asym­met­ric; how similar the cor­pus is to model out­puts is differ­ent from how similar model out­puts are to the cor­pus. Ge­offrey Irv­ing is in­ter­ested in meth­ods to use su­per­vised learn­ing to do dis­tri­bu­tional match­ing the other di­rec­tion, and it might be the case that com­ment karma is a good way to do it; my guess is that you’re bet­ter off com­par­ing out­puts it gen­er­ates on the same prompt head-to-head and pick­ing which one is more ‘nor­mal,’ and train­ing a dis­crim­i­na­tor to at­tempt to mimic the hu­man nor­mal­ity judg­ment.
My clos­est an­swer would be some­thing like “in my ver­sion of utopia,” al­though maybe that’s too strong?
I think this im­plies way too much en­dorse­ment. I of­ten find my­self edit­ing a doc­u­ment and think­ing “in Amer­i­can English, the comma goes in­side the quo­ta­tion marks,” even though “in pro­gram­ming, the pe­riod goes out­side the quo­ta­tion marks”.
When some­one has an in­com­plete moral wor­ld­view (or one based on eas­ily dis­prov­able as­ser­tions), there’s a way in which the truth isn’t “safe” if safety is mea­sured by some­thing like ‘re­versibil­ity’ or ‘abil­ity to con­tinue be­ing the way they were.’ It is also of­ten the case that one can’t make a sin­gle small change, and then move on; if, say, you man­age to con­vince a Chris­tian that God isn’t real (or some other thing that will pre­dictably cause the whole ed­ifice of their wor­ld­view to come crash­ing down even­tu­ally), then the de­fault thing to hap­pen is for them to be lost and alone.
I think I have this skill, but I don’t know that I could write this guide. Partly this is be­cause there are lots of fea­tures about me that make this eas­ier, which are hard (or too ex­pen­sive) to copy. For ex­am­ple, Michael once sug­gested part of my emo­tional re­la­tion­ship to lots of this came from be­ing gay, and thus not hav­ing to par­ti­ci­pate in a par­tic­u­lar va­ri­ety of com­pe­ti­tion and sig­nal­ling that was con­strain­ing oth­ers; that seemed like it wasn’t the pri­mary fac­tor, but was prob­a­bly a sig­nifi­cant one.
Another thing that’s quite difficult here is that many of the claims are about val­ues, or things up­stream of val­ues; how can Draco Malfoy learn the truth about blood purism in a ‘safe’ way?
He ap­pears to have had novel ideas in his tech­ni­cal spe­cialty, but his pub­lic writ­ings are mostly about old ideas that have in­suffi­cient pub­lic defense. There, nov­elty isn’t a virtue (while cor­rect­ness is).
My sense is that his wor­ld­view was ‘very sane’ in the cyn­i­cal HPMOR!Quir­rell sense (and he was one of the ma­jor in­spira­tions for Quir­rell, so that’s not sur­pris­ing), and that he was ex­tremely open about it in per­son in a way that was sur­pris­ing and ex­cit­ing.
in reply to: anna_macdonald's comment on: When does in­tro­spec­tion avoid the pit­falls of ru­mi­na­tion?
The tech­niques you men­tion may in­clude fo­cus­ing on causes and con­se­quences, but they are very solu­tion-ori­ented.
Fo­cus­ing, which is an in­tro­spec­tive tech­nique, is ex­plic­itly not fo­cused on solu­tions; it’s fo­cused on figur­ing out what the ac­tual prob­lem is (which gen­er­ally is more about listen­ing to the com­plaint than it is about think­ing about the en­vi­ron­ment or how things could be solved). This then helps some­one find a solu­tion, but they’re likely not do­ing that with Fo­cus­ing.
Sorry for the lack of links on this; it’s stuff that’s cov­ered in the Se­quences, but where ex­actly I’m not sure. I’ll gladly strong up­vote a re­ply to this com­ment link­ing to rele­vant ma­te­rial on these points.
I don’t think it’s in sin­gle posts? Like, there’s the Robin Han­son post The Fal­lacy Fal­lacy, or JGWeiss­man’s Catchy Fal­lacy Name Fal­lacy, but those are mostly about “here are spe­cific is­sues with fo­cus­ing on fal­la­cies” as op­posed to “and also here’s how Bayesian episte­mol­ogy works in­stead.” If I were to point to a sin­gle post, it might be Science Isn’t Strict Enough, which of course is about sci­ence in­stead of about logic, and is do­ing the “this is how this stan­dard di­verges from what seems to be the right stan­dard” ar­gu­ment but in the op­po­site di­rec­tion, sort of.
on: Is vot­ing the­ory im­por­tant? An at­tempt to check my bias.
1. What seems im­por­tant to Vaniver.
I ex­pect that vot­ing sys­tems mostly won’t mat­ter for AI out­comes. It seems like the pri­mary ques­tion is whether or not the AI sys­tem we make does any­thing like what we like/​en­dorse (i.e. whether or not ex­is­ten­tial ac­ci­dents hap­pen), and the sec­ondary ques­tion is whether or not teams co­or­di­nated to form a coal­i­tion to build such a safe sys­tem (or oth­er­wise pre­vented the cre­ation of un­safe sys­tems). Vot­ing seems mostly use­ful for ag­gre­gat­ing prefer­ences over scarce joint de­ci­sions in a band­width-sen­si­tive way (“where should the group go to lunch?” as op­posed to “what do you per­son­ally want to eat?“, or “which of these four can­di­dates should be pres­i­dent?” as op­posed to “what are your com­plete views on poli­tics?“), and the coal­i­tion-build­ing prob­lem will likely look more like ne­go­ti­a­tion (see this pa­per by Critch as an ex­am­ple of the sort of thing that seems use­ful to me in that space) and the prefer­ence-satis­fac­tion solu­tion in the glo­ri­ous tran­shu­man fu­ture will likely look more like tel­ling Alexa how you want your per­sonal en­vi­ron­ment to be and not hav­ing to worry much about scarcity or joint de­ci­sion-mak­ing.
It’s pos­si­ble that gov­ern­ment policy will be im­por­tant, and the health of pub­lic dis­course will be im­por­tant, but it seems quite un­likely to me that elec­tion re­forms will have the de­sired effects in time.
2. Whether it’s the core prob­lem of dis­course, or will be suffi­cient to over­come mod­ern challenges.
Figur­ing out what’s up­stream of wors­en­ing dis­course and push­ing on that (or seek­ing to cre­ate more good dis­course, or so on) is prob­a­bly more effec­tive is bet­ter pub­lic con­ver­sa­tions are ac­tu­ally the goal; and even if this effort helps, if it can’t help enough, it may be bet­ter to write off the thing that it would help.
3. Whether or not it’s im­por­tant if it seems im­por­tant to Vaniver.
It seems good for Alice to not pay the costs of op­ti­miz­ing, and just do the lo­cal im­prove­ments, es­pe­cially if the al­ter­na­tive is that Alice doesn’t make any im­prove­ments. Beatrice will do more im­por­tant work, but is ‘pay­ing twice’ for it, and in situ­a­tions where the bugs are roughly equally im­por­tant this means Beatrice is per­haps less effec­tive than some­one less re­flec­tive. I think that peo­ple who are nat­u­rally in­ter­ested in this sort of max­i­miz­ing al­tru­ism should do it, and peo­ple who aren’t (and want to just be Alice in­stead) should be Alice with­out wor­ry­ing about it too much (or try­ing to con­vince them­selves that, no, they are do­ing the max­i­miz­ing al­tru­ism thing).
in reply to: Raemon's comment on: Is vot­ing the­ory im­por­tant? An at­tempt to check my bias.
It’s a straight­for­ward ap­pli­ca­tion of the Coper­ni­can prin­ci­ple. Of course, that is not always the best ap­proach.