问:
Create loop to subset data by month and year
UPDATE: I have added the dput() input at the bottom of the post.

I have a large dataset of tweets that I would like to subset by month and year.

data_cleaning$date <- as.Date(data_cleaning$created_at, tryFormats = c("%Y-%m-%d", "%Y/%m/%d"), optional = FALSE)

I used the line of code above to format the date variable in the dataframe below.

```<code>id        text                                                                date
1389598   Honored to receive the United Way Award                             2012-05-22
1586410   Joining @TamronHall at 2:15pm today to discuss my bipartisan bill   2013-12-18
138058    Leahy leads 58 senators in lttr to #SecDef Hage                     2013-12-21
1482508   On one anniversary of #Sandy                                        2013-10-29
526459    #SocSec and #Medicare are absolutely vital                          2012-02-02
687900    Check out #MadeInWi BomBoard of Whitewater                          2014-04-14
826551    Congratulations to all of the hard-working @UTSA students           2013-12-21
1409462   Great to see so many proud Colombians                               2012-07-21
1807754   It is our duty to look after our veterans.                          2012-05-28
138057    Leahy presses preserv. of #TotalArmy capability                     2013-12-21
</code>```

I know how to manually create a subset by month with the following code:

```<code>data_cleaning <-  data_cleaning %>% filter(date >= "2012-01-1")
data_cleaning <-  data_cleaning %>% filter(date <= "2012-02-1")
</code>```

But I have 15 years worth of tweets that I would like to create a loop to subset this data by month and year, such that I have separate files for each. For example, ideally, I'd like to have files called "tweets_Jan2012", tweets_Feb2012, tweets_Mar2012", and so on, for each month that appears in the dataset.

Any help would be greatly appreciated.

```<code>Show in New WindowClear OutputExpand/Collapse Output
structure(list(author_id = c("242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999"), text = c("anyon feel unsaf home time alon confidenti help avail even re social distanc https t co jkmayado",
"actual think rue day decid tri intimid major backfir bravo michael https t co sidonllehr",
"congratul michaelemann first rate scientist fearless amp resolut remorseless attack fossil fuel forc https t co uyleddecm",
"today last day rhode island can sign health insur healthsourceri s special enrol period covid look option getcov https t co gkdyasc",
"april th https t co gskujfum", "translat hey captur court fair squar s want re go threaten point https t co ngtcofm",
"btw s coincid winner partisan decis big republican donor interest wsjopinion page alway flack",
"s hard defend dark money prowl around court inde s indefens defens pretend s happen amp just ignor fact overview under problem read https t co bkxvoyyndi",
"appreci shoutout whitehous effect s control suprem court look instead dark money fedsoc select justic dark money judicialnetwork s role confirm justic amp case win record republican donor https t co cxfqfzf",
"imposs burger https t co rtbbfqjbap", "question politico playbook ask present answer reconven senat https t co esnzuwqj",
"tomorrow last day healthsourceri s covid special enrol period uninsur rhode island can sign https t co kyuvaj",
"signal anyth anyon hello interest fund mitch mcconnel amp parti fund dark money machin surround amp stock suprem court obedi court will unpopular thing can t make elect republican https t co zjpfhvbua",
"markpatinkin man roll amaz https t co lbqag", "think corpor greedi check million plus dollar incom individu extract averag million care coronavirus bill took hospit got https t co kfuidhx",
"prioriti protocol senat oper safe push judg big donor fund judici influenc machin republican parti",
"trump nomin walker second power court nation walker s qualif near press appear defend buddi kavanaugh amp track record zealotri support republican parti polit goal like kill aca https t co nnmkvwgc",
"mcconnel delay covid relief bill fli kentucki brett kavanaugh celebr yet anoth unqualifi partisan trump judg justin walker https t co ovquouv",
"small bank still boil mad custom shoulder asid big bank second ppp round power superior autom sba fix happen right amp fair",
"ad ppp money meet need small busi particular small bank amp credit union got push asid big bank now big bank jam system autom applic frustrat small bank sbagov need fix",
"truth law got way even republican appoint justic couldn t stomach deliv big win gun manufactur isn t https t co ngtcofm",
"nra huge republican donor interest didn t get want suprem court right wing donor interest machin kick high gear get justic back line wsjopinion take lead cours https t co goiuipzcb",
"want abl safe reopen key test https t co ytmjalmvd", "coronavirus warn pdb repeat warn convey presid s daili brief alarm appear fail regist presid routin skip read pdb https t co cfsbemtn",
"great graphic nytim worth look well done https t co yqevgdh",
"imagin british paid peopl follow paul rever say british aren t come big hoax fake news can go back bed https t co mvqrxfi",
"climat scientist paul rever climat chang ring bell loud increas urgenc sinc s https t co eqjtsbknf",
"mood markwarn tuna melt d recommend tri rhode island s delici seafood instead mayo microwav requir https t co euojwft",
"thank break true mitch mcconnel special billion dollar special tax provis tuck last covid relief bill benefit millionair amp billionair https t co vimhica",
"ri great colleg univers deserv support thank brownunivers presid paxson https t co ubqmurtgfi",
"meanwhil right wing effort exact bring case front group support front group file amicus brief fund manipul crew lot turn case https t co lkhgypp",
"crack end republican justic gripe court s docket manipul https t co vcaaokwd",
"true one never know crew strong oppos seema verma unilater hurt rhode island hospit one cmsgov hhsgov abl defend https t co aavdutava",
"look like sorri episod will end decent note https t co yxgqkl",
"west warwick teacher make meet hero section https t co edhdkxoc",
"definit first time histori feder health agenc issu formal warn listen presid s offici https t co jbezonef",
"good night light adapt coronavirus start way say good night headlight emerg vehicl light hospit children get flashlight flash good night back s beauti thing even without coronavirus https t co mcggau",
"https t co btrocxmbau", "just anoth public servic whack job creepi dark money land oy https t co sonieqd",
"first need conquer enemi within corrupt anti scienc anti govern ideolog ordinarili kept aliv fring crank now foment massiv industri resent scientif check public oversight interfer money make https t co xlhcpuetp",
"heartbreak will back https t co qtggaeopvt", "mitch mcconnel say prioriti covid allow covid legisl bear mind scam flood billion wealthi investor donor make million dollar year covid connect none https t co oeerchqhf",
"davidcicillin alexthomasdc mzanona sarahnferri heatherscop seungminkim jenhab elistokol markzbarabak mffisher roigfranzia timcarman olgamassov maurajudki laurahayesdc scottagilmor megetz jeffreynpark lisabono afreedma annmmaloney lottieanddoof kfleisher bjeanclement jbonn patijinich jkfrel ericmgarcia emilyaheil mrdanzak juliajfish aaronsfish edmet joelachenbach davidvondrehl abcarianlat chrismegerian davidlaut raineytim ec schneider theodoricmey jdawsey noahbierman ddiamond rachanadixit name perfect movi nomin peopl godfath casablanca blaze saddl butch cassidi sundanc kid darkest hour jimlangevin peterneronha neildsteinberg dawneuer jamesdiossa",
"republican snuck billion millionairesgiveaway pandem relief bill even though noth fight covid replloyddoggett amp introduc legisl repeal shame tax break amp ensur aid goe american need https t co csyldjqgn",
"friend talk treat coronavirus lysol inject lung worri https t co ymjlzmrtj",
"friend exhibit kind behavior worri https t co puoicxw", "pelosi even make mask look cool win mcname getti imag https t co obbxzvujg",
"good night earth https t co dlcuyunow", "pope franci see natur tragedi earth s respons maltreat ad sin earth neighbor end creator time clean act https t co lrkynccunk",
"markpatinkin found moment chronicl coronavirus hero anoth amaz piec hope get pulitz https t co kmihfuzuu",
"serious mitch mcconnel want get state declar bankruptci d better get economist fed treasuri tell look like economi https t co pvhfqoymc",
"republican gave million dollar tax break millionair vital covid relief packag s time regular american get check",
"hell republican think jam high incom big donor link covid giveaway follow money https t co xqjpstjnxp",
"ag threaten governor fight keep citizen safe covid doj serious go sue state tri protect citizen https t co ifzmlkzyd",
"earth hero earth day thank billmckibben https t co kdvpav",
"ps corpor america overal noth congress fossil fuel mischief continu big compani thing want climat effort trade associ good net result corpor america oppos congress climat action time fix",
"corrupt america s govern near evil pollut planet ocean must stop",
"can make earth day promis reclaim healthi earth reclaim healthi polit expos throw crook dark money fake scienc phoni front group apparatus restor democraci work",
"fossil fuel industri live subsidi measur intern monetari fund north billion per year u s give industri massiv incent corrupt polit citizen unit robert five gave tool",
"bipartisanship climat stop dead lost decad fossil fuel fake scienc dark money phoni front group built whole apparatus web denial lie cheat bulli industri scale work",
"januari infam citizen unit decis present unlimit polit spend inde unlimit anonym polit spend fossil fuel industri instant use new power corrupt polit",
"earth day s good rememb lot good bipartisan climat legisl senat republican presidenti nomine solid climat platform",
"serious carbon price stop warm heat amp acidif co", "rememb earth day ocean day can start pass marin plastic bill sos ocean data bill blue globe amp real dollar ocean coastal fund nocsf https t co jryavxcb",
"re serious solut model tool can help get us safe climat republican even meet won t solv climat without break corrupt grip fossil fuel parti spoiler carbon price work https t co kfhuttiav",
"happi earthday free globe fossil fuel corrupt https t co rsshjcf",
"trump administr promis march th near month ago million coronavirus test conduct u s s april st haven t reach million test https t co eoioxyj",
"today pledg continu fight antisemit amp educ everyon holocaust never happen holocaustremembranceday",
"get serious renew energi can creat million good stabl job american current work oil amp gas can transit",
"let s clear s less fossil fuel corpor pillar economi fossil fuel donor central pillar republican fundrais henc parti s creepi crawl obedi fossil fuel interest https t co ddovmoii",
"senschum just now https t co dlnvvpem", "thank jimlangevin davidcicillin hous stand firm back get way improv posit",
"sba hospit test s unfortun mitch insist jam us negoti get big victori insist bipartisanship worth wait",
"faint heart panick misl mitch mcconnel tri jam us sba fund prepar much better bill",
"throw taxpay dollar behind compani expos economi massiv crash risk realli bad idea s m team brianschatz call federalreserv consid climat risk emerg lend program https t co kojqmrl",
"still clear direct trump administr money state money educ remain money hospit etc seem like long damn time sinc pass bill s holdup",
"top straight b giveaway million dollar plus earner republican snuck care s problem need fix money need flow actual small busi worker https t co zhyvmjco",
"proud polari mep ritin other organ help crisi https t co fwpmrhsap run group busi look purchas suppli mask includ product ri s world class textil compani https t co ktrasoevo",
"perhap newspap check rhode island s hospit wisdom cut loos negoti go sba https t co nwkjtxjta",
"republican pretend isn t true preserv flow fossil fuel industri money depend https t co xutocwmwv",
"happen climat disast take place time coronavirus will learn hear warn https t co gfoxydnra",
"oh come re put chuck schumer categori mitch mcconnel serious https t co fwirvmjjw",
"crash number mani marin speci rais price survivor high hunt feroci marin protect essenti stop spiral https t co badbghw",
"still come learn anyth listen warn https t co bzegjn", "happen happen partisan decis suprem court mani mani advanc big republican donor interest https t co ghwhhwewlw",
"health care worker plead person protect equip keep amp patient safe need presid talk less amp getuspp https t co uptrrbt",
"happen trump administr catastroph fail feder coordin leadership everi minut governor health care provid spend tumult wast time better spent problem https t co ikswhfnug",
"net meter help reduc wast fight climat chang keep cost consum ferc act swift reject shadi front group petit make net meter harder https t co uadxpkivi",
"hah somebodi els agre corrupt anti govern anti scienc patholog virus bodi polit thank maxboot https t co gmtqmob",
"earli nation data point troubl racial dispar need research help rais alarm amp protect vulner communiti",
"proud join senwarren amp timkain legisl ensur hhsgov report vital racial amp demograph data test treatment amp fatal rate covid https t co miqseophbd",
"anyon care give odd mention climat chang advoc carbon price claim support sincer check bet flunk https t co qobnzxj",
"markpatinkin fabul job sing unsung hero hero https t co sxygmdreba",
"consensus price carbon emiss pollut free unfair hinder success climat crisi https t co euqvswdygv",
"hope panel will good faith bipartisan effort inform scienc best practic",
"accept presid trump s invit serv bipartisan panel charg determin guidelin safe restart american economi amid covid pandem https t co wiqygnahzh",
"confront covid amp reviv economi congress pass bill send resourc direct rier can find answer question cash payment health protect small busi relief program amp much websit https t co sjjunovzw https t co odhcokgut",
"white guy rifl trump sign woman chief execut lock new materi guy https t co mbxsdofp",
"excel piec immens talent janemayerny dark money fame mitch mcconnel polit dark money oper explain destroy anyth senat stand door corrupt goal https t co eypezjsi",
"didn t mean us https t co qlxzojkzv"), date = structure(c(18382,
18382, 18382, 18382, 18382, 18382, 18382, 18382, 18382, 18381,
18381, 18381, 18381, 18381, 18381, 18380, 18380, 18380, 18380,
18380, 18380, 18380, 18380, 18380, 18380, 18379, 18379, 18379,
18379, 18379, 18379, 18379, 18379, 18378, 18378, 18378, 18378,
18377, 18377, 18377, 18377, 18376, 18376, 18376, 18376, 18376,
18376, 18376, 18375, 18375, 18375, 18375, 18375, 18375, 18374,
18374, 18374, 18374, 18374, 18374, 18374, 18374, 18374, 18374,
18374, 18374, 18373, 18373, 18373, 18373, 18373, 18373, 18373,
18373, 18373, 18372, 18372, 18372, 18372, 18372, 18371, 18371,
18371, 18370, 18370, 18370, 18369, 18369, 18369, 18369, 18369,
18369, 18368, 18368, 18368, 18368, 18368, 18368, 18368, 18368
), class = "Date")), row.names = c(NA, -100L), class = c("data.table",
"data.frame"), .internal.selfref = <pointer: 0x7fac0a810ee0>)
</code>```

答:
```<code># set as data.table
setDT(data_cleaning)

# create year month column
data_cleaning[, year_month := substr(date, 1, 7)]

# split and put into list
split(data_cleaning, data_cleaning$year_month)
</code>```

each element of this list will correspond to each group of the year_month column. Please try uploading a reproducible set off data with dput(data_cleaning) for your question.

Body:
UPDATE: I have added the dput() input at the bottom of the post.

I have a large dataset of tweets that I would like to subset by month and year.

data_cleaning$date <- as.Date(data_cleaning$created_at, tryFormats = c("%Y-%m-%d", "%Y/%m/%d"), optional = FALSE)

I used the line of code above to format the date variable in the dataframe below.

id        text                                                                date
1389598   Honored to receive the United Way Award                             2012-05-22
1586410   Joining @TamronHall at 2:15pm today to discuss my bipartisan bill   2013-12-18
138058    Leahy leads 58 senators in lttr to #SecDef Hage                     2013-12-21
1482508   On one anniversary of #Sandy                                        2013-10-29
526459    #SocSec and #Medicare are absolutely vital                          2012-02-02
687900    Check out #MadeInWi BomBoard of Whitewater                          2014-04-14
826551    Congratulations to all of the hard-working @UTSA students           2013-12-21
1409462   Great to see so many proud Colombians                               2012-07-21
1807754   It is our duty to look after our veterans.                          2012-05-28
138057    Leahy presses preserv. of #TotalArmy capability                     2013-12-21

I know how to manually create a subset by month with the following code:

data_cleaning <-  data_cleaning %>% filter(date >= "2012-01-1")
data_cleaning <-  data_cleaning %>% filter(date <= "2012-02-1")

But I have 15 years worth of tweets that I would like to create a loop to subset this data by month and year, such that I have separate files for each. For example, ideally, I'd like to have files called "tweets_Jan2012", tweets_Feb2012, tweets_Mar2012", and so on, for each month that appears in the dataset.

Any help would be greatly appreciated.

Show in New WindowClear OutputExpand/Collapse Output
structure(list(author_id = c("242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999", "242555999", "242555999", "242555999",
"242555999", "242555999"), text = c("anyon feel unsaf home time alon confidenti help avail even re social distanc https t co jkmayado",
"actual think rue day decid tri intimid major backfir bravo michael https t co sidonllehr",
"congratul michaelemann first rate scientist fearless amp resolut remorseless attack fossil fuel forc https t co uyleddecm",
"today last day rhode island can sign health insur healthsourceri s special enrol period covid look option getcov https t co gkdyasc",
"april th https t co gskujfum", "translat hey captur court fair squar s want re go threaten point https t co ngtcofm",
"btw s coincid winner partisan decis big republican donor interest wsjopinion page alway flack",
"s hard defend dark money prowl around court inde s indefens defens pretend s happen amp just ignor fact overview under problem read https t co bkxvoyyndi",
"appreci shoutout whitehous effect s control suprem court look instead dark money fedsoc select justic dark money judicialnetwork s role confirm justic amp case win record republican donor https t co cxfqfzf",
"imposs burger https t co rtbbfqjbap", "question politico playbook ask present answer reconven senat https t co esnzuwqj",
"tomorrow last day healthsourceri s covid special enrol period uninsur rhode island can sign https t co kyuvaj",
"signal anyth anyon hello interest fund mitch mcconnel amp parti fund dark money machin surround amp stock suprem court obedi court will unpopular thing can t make elect republican https t co zjpfhvbua",
"markpatinkin man roll amaz https t co lbqag", "think corpor greedi check million plus dollar incom individu extract averag million care coronavirus bill took hospit got https t co kfuidhx",
"prioriti protocol senat oper safe push judg big donor fund judici influenc machin republican parti",
"trump nomin walker second power court nation walker s qualif near press appear defend buddi kavanaugh amp track record zealotri support republican parti polit goal like kill aca https t co nnmkvwgc",
"mcconnel delay covid relief bill fli kentucki brett kavanaugh celebr yet anoth unqualifi partisan trump judg justin walker https t co ovquouv",
"small bank still boil mad custom shoulder asid big bank second ppp round power superior autom sba fix happen right amp fair",
"ad ppp money meet need small busi particular small bank amp credit union got push asid big bank now big bank jam system autom applic frustrat small bank sbagov need fix",
"truth law got way even republican appoint justic couldn t stomach deliv big win gun manufactur isn t https t co ngtcofm",
"nra huge republican donor interest didn t get want suprem court right wing donor interest machin kick high gear get justic back line wsjopinion take lead cours https t co goiuipzcb",
"want abl safe reopen key test https t co ytmjalmvd", "coronavirus warn pdb repeat warn convey presid s daili brief alarm appear fail regist presid routin skip read pdb https t co cfsbemtn",
"great graphic nytim worth look well done https t co yqevgdh",
"imagin british paid peopl follow paul rever say british aren t come big hoax fake news can go back bed https t co mvqrxfi",
"climat scientist paul rever climat chang ring bell loud increas urgenc sinc s https t co eqjtsbknf",
"mood markwarn tuna melt d recommend tri rhode island s delici seafood instead mayo microwav requir https t co euojwft",
"thank break true mitch mcconnel special billion dollar special tax provis tuck last covid relief bill benefit millionair amp billionair https t co vimhica",
"ri great colleg univers deserv support thank brownunivers presid paxson https t co ubqmurtgfi",
"meanwhil right wing effort exact bring case front group support front group file amicus brief fund manipul crew lot turn case https t co lkhgypp",
"crack end republican justic gripe court s docket manipul https t co vcaaokwd",
"true one never know crew strong oppos seema verma unilater hurt rhode island hospit one cmsgov hhsgov abl defend https t co aavdutava",
"look like sorri episod will end decent note https t co yxgqkl",
"west warwick teacher make meet hero section https t co edhdkxoc",
"definit first time histori feder health agenc issu formal warn listen presid s offici https t co jbezonef",
"good night light adapt coronavirus start way say good night headlight emerg vehicl light hospit children get flashlight flash good night back s beauti thing even without coronavirus https t co mcggau",
"https t co btrocxmbau", "just anoth public servic whack job creepi dark money land oy https t co sonieqd",
"first need conquer enemi within corrupt anti scienc anti govern ideolog ordinarili kept aliv fring crank now foment massiv industri resent scientif check public oversight interfer money make https t co xlhcpuetp",
"heartbreak will back https t co qtggaeopvt", "mitch mcconnel say prioriti covid allow covid legisl bear mind scam flood billion wealthi investor donor make million dollar year covid connect none https t co oeerchqhf",
"davidcicillin alexthomasdc mzanona sarahnferri heatherscop seungminkim jenhab elistokol markzbarabak mffisher roigfranzia timcarman olgamassov maurajudki laurahayesdc scottagilmor megetz jeffreynpark lisabono afreedma annmmaloney lottieanddoof kfleisher bjeanclement jbonn patijinich jkfrel ericmgarcia emilyaheil mrdanzak juliajfish aaronsfish edmet joelachenbach davidvondrehl abcarianlat chrismegerian davidlaut raineytim ec schneider theodoricmey jdawsey noahbierman ddiamond rachanadixit name perfect movi nomin peopl godfath casablanca blaze saddl butch cassidi sundanc kid darkest hour jimlangevin peterneronha neildsteinberg dawneuer jamesdiossa",
"republican snuck billion millionairesgiveaway pandem relief bill even though noth fight covid replloyddoggett amp introduc legisl repeal shame tax break amp ensur aid goe american need https t co csyldjqgn",
"friend talk treat coronavirus lysol inject lung worri https t co ymjlzmrtj",
"friend exhibit kind behavior worri https t co puoicxw", "pelosi even make mask look cool win mcname getti imag https t co obbxzvujg",
"good night earth https t co dlcuyunow", "pope franci see natur tragedi earth s respons maltreat ad sin earth neighbor end creator time clean act https t co lrkynccunk",
"markpatinkin found moment chronicl coronavirus hero anoth amaz piec hope get pulitz https t co kmihfuzuu",
"serious mitch mcconnel want get state declar bankruptci d better get economist fed treasuri tell look like economi https t co pvhfqoymc",
"republican gave million dollar tax break millionair vital covid relief packag s time regular american get check",
"hell republican think jam high incom big donor link covid giveaway follow money https t co xqjpstjnxp",
"ag threaten governor fight keep citizen safe covid doj serious go sue state tri protect citizen https t co ifzmlkzyd",
"earth hero earth day thank billmckibben https t co kdvpav",
"ps corpor america overal noth congress fossil fuel mischief continu big compani thing want climat effort trade associ good net result corpor america oppos congress climat action time fix",
"corrupt america s govern near evil pollut planet ocean must stop",
"can make earth day promis reclaim healthi earth reclaim healthi polit expos throw crook dark money fake scienc phoni front group apparatus restor democraci work",
"fossil fuel industri live subsidi measur intern monetari fund north billion per year u s give industri massiv incent corrupt polit citizen unit robert five gave tool",
"bipartisanship climat stop dead lost decad fossil fuel fake scienc dark money phoni front group built whole apparatus web denial lie cheat bulli industri scale work",
"januari infam citizen unit decis present unlimit polit spend inde unlimit anonym polit spend fossil fuel industri instant use new power corrupt polit",
"earth day s good rememb lot good bipartisan climat legisl senat republican presidenti nomine solid climat platform",
"serious carbon price stop warm heat amp acidif co", "rememb earth day ocean day can start pass marin plastic bill sos ocean data bill blue globe amp real dollar ocean coastal fund nocsf https t co jryavxcb",
"re serious solut model tool can help get us safe climat republican even meet won t solv climat without break corrupt grip fossil fuel parti spoiler carbon price work https t co kfhuttiav",
"happi earthday free globe fossil fuel corrupt https t co rsshjcf",
"trump administr promis march th near month ago million coronavirus test conduct u s s april st haven t reach million test https t co eoioxyj",
"today pledg continu fight antisemit amp educ everyon holocaust never happen holocaustremembranceday",
"get serious renew energi can creat million good stabl job american current work oil amp gas can transit",
"let s clear s less fossil fuel corpor pillar economi fossil fuel donor central pillar republican fundrais henc parti s creepi crawl obedi fossil fuel interest https t co ddovmoii",
"senschum just now https t co dlnvvpem", "thank jimlangevin davidcicillin hous stand firm back get way improv posit",
"sba hospit test s unfortun mitch insist jam us negoti get big victori insist bipartisanship worth wait",
"faint heart panick misl mitch mcconnel tri jam us sba fund prepar much better bill",
"throw taxpay dollar behind compani expos economi massiv crash risk realli bad idea s m team brianschatz call federalreserv consid climat risk emerg lend program https t co kojqmrl",
"still clear direct trump administr money state money educ remain money hospit etc seem like long damn time sinc pass bill s holdup",
"top straight b giveaway million dollar plus earner republican snuck care s problem need fix money need flow actual small busi worker https t co zhyvmjco",
"proud polari mep ritin other organ help crisi https t co fwpmrhsap run group busi look purchas suppli mask includ product ri s world class textil compani https t co ktrasoevo",
"perhap newspap check rhode island s hospit wisdom cut loos negoti go sba https t co nwkjtxjta",
"republican pretend isn t true preserv flow fossil fuel industri money depend https t co xutocwmwv",
"happen climat disast take place time coronavirus will learn hear warn https t co gfoxydnra",
"oh come re put chuck schumer categori mitch mcconnel serious https t co fwirvmjjw",
"crash number mani marin speci rais price survivor high hunt feroci marin protect essenti stop spiral https t co badbghw",
"still come learn anyth listen warn https t co bzegjn", "happen happen partisan decis suprem court mani mani advanc big republican donor interest https t co ghwhhwewlw",
"health care worker plead person protect equip keep amp patient safe need presid talk less amp getuspp https t co uptrrbt",
"happen trump administr catastroph fail feder coordin leadership everi minut governor health care provid spend tumult wast time better spent problem https t co ikswhfnug",
"net meter help reduc wast fight climat chang keep cost consum ferc act swift reject shadi front group petit make net meter harder https t co uadxpkivi",
"hah somebodi els agre corrupt anti govern anti scienc patholog virus bodi polit thank maxboot https t co gmtqmob",
"earli nation data point troubl racial dispar need research help rais alarm amp protect vulner communiti",
"proud join senwarren amp timkain legisl ensur hhsgov report vital racial amp demograph data test treatment amp fatal rate covid https t co miqseophbd",
"anyon care give odd mention climat chang advoc carbon price claim support sincer check bet flunk https t co qobnzxj",
"markpatinkin fabul job sing unsung hero hero https t co sxygmdreba",
"consensus price carbon emiss pollut free unfair hinder success climat crisi https t co euqvswdygv",
"hope panel will good faith bipartisan effort inform scienc best practic",
"accept presid trump s invit serv bipartisan panel charg determin guidelin safe restart american economi amid covid pandem https t co wiqygnahzh",
"confront covid amp reviv economi congress pass bill send resourc direct rier can find answer question cash payment health protect small busi relief program amp much websit https t co sjjunovzw https t co odhcokgut",
"white guy rifl trump sign woman chief execut lock new materi guy https t co mbxsdofp",
"excel piec immens talent janemayerny dark money fame mitch mcconnel polit dark money oper explain destroy anyth senat stand door corrupt goal https t co eypezjsi",
"didn t mean us https t co qlxzojkzv"), date = structure(c(18382,
18382, 18382, 18382, 18382, 18382, 18382, 18382, 18382, 18381,
18381, 18381, 18381, 18381, 18381, 18380, 18380, 18380, 18380,
18380, 18380, 18380, 18380, 18380, 18380, 18379, 18379, 18379,
18379, 18379, 18379, 18379, 18379, 18378, 18378, 18378, 18378,
18377, 18377, 18377, 18377, 18376, 18376, 18376, 18376, 18376,
18376, 18376, 18375, 18375, 18375, 18375, 18375, 18375, 18374,
18374, 18374, 18374, 18374, 18374, 18374, 18374, 18374, 18374,
18374, 18374, 18373, 18373, 18373, 18373, 18373, 18373, 18373,
18373, 18373, 18372, 18372, 18372, 18372, 18372, 18371, 18371,
18371, 18370, 18370, 18370, 18369, 18369, 18369, 18369, 18369,
18369, 18368, 18368, 18368, 18368, 18368, 18368, 18368, 18368
), class = "Date")), row.names = c(NA, -100L), class = c("data.table",
"data.frame"), .internal.selfref = <pointer: 0x7fac0a810ee0>)

Body:
# set as data.table
setDT(data_cleaning)

# create year month column
data_cleaning[, year_month := substr(date, 1, 7)]

# split and put into list
split(data_cleaning, data_cleaning$year_month)

each element of this list will correspond to each group of the year_month column. Please try uploading a reproducible set off data with dput(data_cleaning) for your question.


来源: stackexchange

问:
Regular expression is not working if json response value contain special character . how to extract the 'sid' value from below mention json
Regular expression is not working if json response value contain special character . how to extract the 'sid' value from below mention json as ""sid"?\s*:?\s*"(\w+)"" is not working as json response contain special character on sid.

0{"sid":"RhANkc9V7-psbnzmJAAGS","upgrades":["websocket"],"pingTimeout":20000,"pingInterval":25000}

答:
Your matching group currently scoped to capture ONLY one or more word characters, In order to capture special characters along with word characters you have to tweak your regular expression like below,

```<code>sid":"(.+?)"
</code>```

In above regular expression, . matches any character (except for line terminators) whereas \w captures only word characters i.e. a-z, A-Z, 0-9, _

ContentLicense: CC BY-SA 4.0

Body:
Your matching group currently scoped to capture ONLY one or more word characters, In order to capture special characters along with word characters you have to tweak your regular expression like below,

sid":"(.+?)"

In above regular expression, . matches any character (except for line terminators) whereas \w captures only word characters i.e. a-z, A-Z, 0-9, _


问:
In Preview is there a way to view two pages one below the other
In Preview, in the Two-Pages View (Cmd-3 or View > Two-Pages), is there a way to view two pages one below the other?

This is useful especially when the pages are in landscape mode e.g. slides.

The default two-page view shows pages next to one another.

答: Try right clicking in your pdf and selecting "Two Pages Continuous"</p>

ContentLicense: CC BY-SA 3.0


问:
How to configure VPN connection between 2 Kubernetes clusters
How to configure VPN connection between 2 Kubernetes clusters.

The case is:
- 2 kubernetes clusters running on different sites
- OpenVPN connectivity between 2 clusters
- In both kubernetes clusters are installed openvpn running in separate container.

How to configure kubernetes clusters (vpn, routing, firewall configurations) so, the Nodes and Containers of any of the kubernetes clusters to have connectivity through VPN to nodes and services to the other cluster?

Thank you for the answers !!!

答: Submariner is a very good solution but unfortunately doesn't support IPv6 yet so if your use case has ipv6 or dualstack clusters, then it could be an issue</p>


问:
Axios get request can't convert undefined to object
I am trying to make a get request in my js/svelte application.
The REST-API works perfectly fine in the browser or when testing with postman.

When I click the Go button on the website then the following error ocours in the console from the inspection tool.

Uncaught TypeError: can't convert undefined to object

mergeConfig mergeConfig.js:92
request Axios.js:39
method Axios.js:129
wrap bind.js:9
login Login.svelte:11
listen index.mjs:412
listen_dev index.mjs:1961
mount bundle.js:3413
mount_component index.mjs:1745
update bundle.js:765
update bundle.js:931
update index.mjs:1075
flush index.mjs:1042
promise callback*schedule_update index.mjs:1000
make_dirty index.mjs:1777
ctx index.mjs:1815
unsubscribeLoc bundle.js:1442
subscribe index.mjs:50
instance$3 Router.svelte:493
init index.mjs:1809
Router bundle.js:1583
create_fragment bundle.js:3583
init index.mjs:1824
App bundle.js:3655
app main.js:3
 bundle.js:3675

And this is the code.

```<code>    <script>
    import {replace} from 'svelte-spa-router'
    import {LoginDto} from "../scripts/data_transfer_objects/LoginDto";
    import axios from "axios";

    let loginTemplate = new LoginDto();

    function login(){
        console.log(loginTemplate.password);
        axios.get("http://localhost:5000/login", {
            auth: {
                username: "test",
                password: "1234"
            }
        });
        replace("#/activities");
    }
</script>

<div>
    <input type="password" placeholder="Password" bind:value={loginTemplate.password}>
    <button on:click={login}>Go</button>
</div>
</code>```

Does someone have a idea what the problem is?

Thanks in advance!

EDIT

I tryied it with a even simpler example.

```<code><script>
    import axios from "axios";

    axios.get("localhost:5000/activities");
</script>
</code>```

This does not work either.
I prduces the same error as described above.

答: The problem may come from an outdated dependency on @rollup/plugin-commonjs. Upgrading from 17.0.0 to 21.0.1 solved it for me.

Body:
I am trying to make a get request in my js/svelte application.
The REST-API works perfectly fine in the browser or when testing with postman.

When I click the Go button on the website then the following error ocours in the console from the inspection tool.

Uncaught TypeError: can't convert undefined to object

mergeConfig mergeConfig.js:92
request Axios.js:39
method Axios.js:129
wrap bind.js:9
login Login.svelte:11
listen index.mjs:412
listen_dev index.mjs:1961
mount bundle.js:3413
mount_component index.mjs:1745
update bundle.js:765
update bundle.js:931
update index.mjs:1075
flush index.mjs:1042
promise callback*schedule_update index.mjs:1000
make_dirty index.mjs:1777
ctx index.mjs:1815
unsubscribeLoc bundle.js:1442
subscribe index.mjs:50
instance$3 Router.svelte:493
init index.mjs:1809
Router bundle.js:1583
create_fragment bundle.js:3583
init index.mjs:1824
App bundle.js:3655
app main.js:3
 bundle.js:3675

And this is the code.

    <input type="password" placeholder="Password" bind:value={loginTemplate.password}>
    <button on:click={login}>Go</button>

Does someone have a idea what the problem is?

Thanks in advance!

EDIT

I tryied it with a even simpler example.

This does not work either.
I prduces the same error as described above.


问:
How to change Python inquirer checkbox's select / unselect symbol from X and o to Y and N respectively?
The sample script:

```<code>import os
import sys
from pprint import pprint
import yaml

sys.path.append(os.path.realpath("."))
import inquirer  # noqa

questions = [
    inquirer.Checkbox(
        "interests",
        message="What are you interested in?",
        choices=["Computers", "Books", "Science", "Nature", "Fantasy", "History"],
        default=["Computers", "Books"],
    ),
]

answers = inquirer.prompt(questions)

pprint(answers)
print(yaml.dump(answers))
</code>```

produces:

```<code>[?] What are you interested in?:
   X Computers
   o Books
</code>```

How do I change "X" and "o" to "Y" and "N" respectively?

PS: It's pretty common knowledge that XOXO means "hugs and kisses", so it may not be appropriate in some working environments.

答:
You have to define Your new theme and pass it as a parameter to inquirer.prompt.

Here is modified code changing  "X" to "Y" and "o" to "N":

```<code>import os
import sys
from pprint import pprint

import yaml
from inquirer.themes import Default

sys.path.append(os.path.realpath("."))
import inquirer  # noqa

questions = [
    inquirer.Checkbox(
        "interests",
        message="What are you interested in?",
        choices=["Computers", "Books", "Science", "Nature", "Fantasy", "History"],
        default=["Computers", "Books"],
    ),
]

class WorkplaceFriendlyTheme(Default):
    """Custom theme replacing X with Y and o with N"""

    def __init__(self):
        super().__init__()
        self.Checkbox.selected_icon = "Y"
        self.Checkbox.unselected_icon = "N"

answers = inquirer.prompt(questions, theme=WorkplaceFriendlyTheme())

pprint(answers)
print(yaml.dump(answers))
</code>```

Body:
The sample script:

import os
import sys
from pprint import pprint
import yaml

sys.path.append(os.path.realpath("."))
import inquirer  # noqa

questions = [
    inquirer.Checkbox(
        "interests",
        message="What are you interested in?",
        choices=["Computers", "Books", "Science", "Nature", "Fantasy", "History"],
        default=["Computers", "Books"],
    ),
]

answers = inquirer.prompt(questions)

pprint(answers)
print(yaml.dump(answers))

produces:

[?] What are you interested in?:
   X Computers
   o Books

How do I change "X" and "o" to "Y" and "N" respectively?

PS: It's pretty common knowledge that XOXO means "hugs and kisses", so it may not be appropriate in some working environments.

Body:
You have to define Your new theme and pass it as a parameter to inquirer.prompt.

Here is modified code changing  "X" to "Y" and "o" to "N":

import os
import sys
from pprint import pprint

import yaml
from inquirer.themes import Default

sys.path.append(os.path.realpath("."))
import inquirer  # noqa

questions = [
    inquirer.Checkbox(
        "interests",
        message="What are you interested in?",
        choices=["Computers", "Books", "Science", "Nature", "Fantasy", "History"],
        default=["Computers", "Books"],
    ),
]

class WorkplaceFriendlyTheme(Default):
    """Custom theme replacing X with Y and o with N"""

    def __init__(self):
        super().__init__()
        self.Checkbox.selected_icon = "Y"
        self.Checkbox.unselected_icon = "N"

answers = inquirer.prompt(questions, theme=WorkplaceFriendlyTheme())

pprint(answers)
print(yaml.dump(answers))


问:
Is for(1) always the same as do in perl?
```<code>for(1){
  print 1;
}

do {
  print 1;
}
</code>```

Is it true?

Or is there any special case these two doesn't equal?

答:
One difference is that for(1) sets $_ to the value of 1, as well:

```<code>for(1){
    print $_;  # prints 1
}
</code>```

Also, do returns the value of the last command in the sequence:

```<code>my $x = do { 1 };  # $x = 1
my $y = for(1){ 1 }; # invalid
</code>```

Body:
for(1){
  print 1;
}

do {
  print 1;
}

Is it true?

Or is there any special case these two doesn't equal?

Body:
One difference is that for(1) sets $_ to the value of 1, as well:

for(1){
    print $_;  # prints 1
}

Also, do returns the value of the last command in the sequence:

my $x = do { 1 };  # $x = 1
my $y = for(1){ 1 }; # invalid


问:
Snowflake - replace selective double quotes
I have data in a table with one column (varchar) that contains the following -

```<code>COL_A
-----
"trying to remove "double" quotes"
</code>```

I would like to remove the inner double quotes and keep the outer ones intact. Which function should I be using?

Essentially I should get the below output -

```<code>COL_A
-----
"trying to remove double quotes"
</code>```

答:
depending if you want to handle strings that are not always wrapped in double quotes. doing a replacement on the substring allows for how saving the input how it arrived. If always want one lot of quotes around you string, then Mara's answer works nicely.

```<code>select
    column1
    ,'\"' || replace(column1, '\"') || '\"' as majas_answer
    ,left(column1,1) || replace(substr(column1, 2, length(column1)-2), '"') || right(column1,1) as answer
from values
    ('"trying to remove "double" quotes"'),
    ('no quotes'),
    ('"only outer quotes"'),
    ('"single inner" quote"');
</code>```

COLUMN1
MAJAS_ANSWER
ANSWER

"trying to remove "double" quotes"
"trying to remove double quotes"
"trying to remove double quotes"

no quotes
"no quotes"
no quotes

"only outer quotes"
"only outer quotes"
"only outer quotes"

"single inner" quote"
"single inner quote"
"single inner quote"

Body:
I have data in a table with one column (varchar) that contains the following -

COL_A
-----
"trying to remove "double" quotes"

I would like to remove the inner double quotes and keep the outer ones intact. Which function should I be using?

Essentially I should get the below output -

COL_A
-----
"trying to remove double quotes"

Body:
depending if you want to handle strings that are not always wrapped in double quotes. doing a replacement on the substring allows for how saving the input how it arrived. If always want one lot of quotes around you string, then Mara's answer works nicely.

select
    column1
    ,'\"' || replace(column1, '\"') || '\"' as majas_answer
    ,left(column1,1) || replace(substr(column1, 2, length(column1)-2), '"') || right(column1,1) as answer
from values
    ('"trying to remove "double" quotes"'),
    ('no quotes'),
    ('"only outer quotes"'),
    ('"single inner" quote"');

COLUMN1
MAJAS_ANSWER
ANSWER

"trying to remove "double" quotes"
"trying to remove double quotes"
"trying to remove double quotes"

no quotes
"no quotes"
no quotes

"only outer quotes"
"only outer quotes"
"only outer quotes"

"single inner" quote"
"single inner quote"
"single inner quote"


问:
How can I improve my coin change calculator so that the total is 1.41 instead of 1.4100000000000001?
I am working on a change calculator that accepts five arguments(dollars, quarters, dimes, nickels, pennies) and calculates the total amount of change. When I put change.py in the command line followed by 1 1 1 1 1 I am supposed to get 1.41 as the total but I am getting 1.4100000000000001 instead.When I put in other numbers it works as it should. Can anyone help with this? Thank you!</p>
```<code>enter code here
import sys

def change(dollars,quarters,dimes,nickels,pennies):

dollars = int(float(dollars))
quarters = int(float(quarters))
dimes = int(float(dimes))
nickels = int(float(nickels))
pennies = int(float(pennies))

total = (dollars * 1.00) + (quarters * .25) + (dimes *.10) + (nickels * .05) + (pennies * .01)

return total
round(total)

dollars = sys.argv[1]
quarters = sys.argv[2]
dimes = sys.argv[3]
nickels = sys.argv[4]
pennies = sys.argv[5]

total = change(dollars,quarters,dimes,nickels,pennies)
print('The total value of your change is ',(total))
</code>```

答:
This is called a floating-point error (see this question). A simple solution would be to just round your numbers:</p>
```<code>return round(total, 2)
</code>```

Body:
I am working on a change calculator that accepts five arguments(dollars, quarters, dimes, nickels, pennies) and calculates the total amount of change. When I put change.py in the command line followed by 1 1 1 1 1 I am supposed to get 1.41 as the total but I am getting 1.4100000000000001 instead.When I put in other numbers it works as it should. Can anyone help with this? Thank you!

enter code here
import sys

def change(dollars,quarters,dimes,nickels,pennies):

dollars = int(float(dollars))
quarters = int(float(quarters))
dimes = int(float(dimes))
nickels = int(float(nickels))
pennies = int(float(pennies))

total = (dollars * 1.00) + (quarters * .25) + (dimes *.10) + (nickels * .05) + (pennies * .01)

return total
round(total)

dollars = sys.argv[1]
quarters = sys.argv[2]
dimes = sys.argv[3]
nickels = sys.argv[4]
pennies = sys.argv[5]

total = change(dollars,quarters,dimes,nickels,pennies)
print('The total value of your change is ',(total))

Body:
This is called a floating-point error (see this question). A simple solution would be to just round your numbers:

return round(total, 2)


问:
Does Z3 have troubles with division?
During a project, I encountered a phenomenon I can't explain. It is boiled down in the following minimal example (using Z3Py) which should be unsat but returns a model.

from z3 import *

solver = Solver()

x = Int("x")

# The next proposition is clearly unsatisfiable
solver.add(And(x > 0, (x + 1) * (1 / (x + 1)) != 1))
solver.add(x == 1)

# Still, Z3 provides a "model"
print(solver.check())
print(solver.model())

```

Is there anything I am missing? Is it forbidden to use division in certain situations?

Z3 version: 4.8.14
Python version: 3.9

答:
Note that when you use / over integers, than the result is an integer itself. That is, we use integer division; which in SMTLib is defined to be the Euclidian one. See https://smtlib.cs.uiowa.edu/theories-Ints.shtml for details.

This means that the expression 1 / (1+1) reduces to 0, since division produces an integer. So you end up with 0 != 1, which is true, and thus your problem is satisfiable.

If you want division to produce reals, then you should either start with a real x (i.e., x = Real("x")) in which case your problem will be unsat; or convert the arguments to be over the reals, via an expression of the form ToReal(x); which again will produce unsat for your problem.

So, in the first case, your modification will be:

x = Real('x')
```
<p>casting the problem over reals. Or, if you want to keep <code>x</code> an integer, yet the division to be over the reals, you should use:</p>
<pre class="lang-py prettyprint-override"><code>solver.add(And(x > 0, ToReal(x + 1) * (1 / ToReal(x + 1)) != 1))
</code>```

which does the conversion.

I should note that mixing integers and reals in this way can lead to hard problems in general, and the solver might end up reporting unknown. But that's besides the point for your particular question here.

Title: Does Z3 have troubles with division?

Body:
During a project, I encountered a phenomenon I can't explain. It is boiled down in the following minimal example (using Z3Py) which should be unsat but returns a model.

from z3 import *

solver = Solver()

x = Int("x")

# The next proposition is clearly unsatisfiable
solver.add(And(x > 0, (x + 1) * (1 / (x + 1)) != 1))
solver.add(x == 1)

# Still, Z3 provides a "model"
print(solver.check())
print(solver.model())

Is there anything I am missing? Is it forbidden to use division in certain situations?

Z3 version: 4.8.14
Python version: 3.9

Body:
Note that when you use / over integers, than the result is an integer itself. That is, we use integer division; which in SMTLib is defined to be the Euclidian one. See https://smtlib.cs.uiowa.edu/theories-Ints.shtml for details.

This means that the expression 1 / (1+1) reduces to 0, since division produces an integer. So you end up with 0 != 1, which is true, and thus your problem is satisfiable.

If you want division to produce reals, then you should either start with a real x (i.e., x = Real("x")) in which case your problem will be unsat; or convert the arguments to be over the reals, via an expression of the form ToReal(x); which again will produce unsat for your problem.

So, in the first case, your modification will be:

x = Real('x')

casting the problem over reals. Or, if you want to keep x an integer, yet the division to be over the reals, you should use:

solver.add(And(x > 0, ToReal(x + 1) * (1 / ToReal(x + 1)) != 1))

which does the conversion.

I should note that mixing integers and reals in this way can lead to hard problems in general, and the solver might end up reporting unknown. But that's besides the point for your particular question here.


问:
Laravel Sail after cloning from Git repository
I currently have my own Laravel application running on Docker using Laravel sail on Windows 11 using Ubuntu on WSL2. This works fine and as intended. I've pushed my work onto a Git repository, but how would I be able to pull this onto a new system? The vendor files that come with Laravel sail when you install won't be sent to the repository, so sail will be useless until composer's vendors files are installed.

I'm new to Docker, would this mean I would have to install composer and PHP on Linux (WSL2) and then install the vendor files? Is there any easier method to this, or is this the conventional way?

Thank you for any help.

答:
I've handled this by using the composer docker image to install the dependencies.

Clone the repo and then run the following command from within the root directory.

```<code>docker run --rm --interactive --tty -v $(pwd):/app composer install
</code>```

By mounting your repository into the container, the composer container will write the vendor directory and it will appear in your host.

Body:
I've handled this by using the composer docker image to install the dependencies.

Clone the repo and then run the following command from within the root directory.

docker run --rm --interactive --tty -v $(pwd):/app composer install

By mounting your repository into the container, the composer container will write the vendor directory and it will appear in your host.


问:
Use zipTree as source set for building and static compiling

Background

Project Alice generates Java source code, stores it in sources.jar, then uploads it to a Maven repository. Project Bob pulls sources.jar down and needs to use it when compiling. Bob does not know that Alice exists, only where to find sources.jar.

Versions: JDK 11, Gradle 7.3.1, IntelliJ IDEA 2021.3.1

Problem

Making gradle (and IntelliJ's IDEA) build using source files embedded in a JAR file. To be clear, the JAR file contents resemble:

$ jar -tvf sources.jar
     0 Thu Feb 03 08:38:56 PST 2022 META-INF/
    52 Thu Feb 03 08:38:56 PST 2022 META-INF/MANIFEST.MF
     0 Thu Feb 03 08:38:30 PST 2022 com/
     0 Thu Feb 03 08:38:32 PST 2022 com/domain/
     0 Thu Feb 03 08:38:30 PST 2022 com/domain/package/
   938 Thu Feb 03 08:38:32 PST 2022 com/domain/package/SourceCode.java
```
<p>Solutions that extract the <code>.java</code> files from the <code>.jar</code> file introduce knock-on effects we'd like to avoid, including:</p>
<ul>
<li><strong>Editable.</strong> Extracted source files can be edited in the IDE. We'd like them to be read-only. We could add a task that sets the files read-only, but that feels like solving the wrong problem (added complexity).</li>
<li><strong>Synchronization.</strong> When a newly generated <code>sources.jar</code> is pulled down, we'll have to delete the extraction directory to remove any stale .java files that were preserved. If there was a way to avoid extraction, then the act of pulling down the new <code>sources.jar</code> file would ensure correctness (no added complexity). By unarchiving the .java files, it's possible to enter an inconsistent state:</li>
</ul>
<pre class="lang-sh prettyprint-override"><code>$ jar -tvf sources.jar | grep java$ | wc -l
61
$ find src/gen -name "*java" | wc -l
65
</code>```

If there was a way to treat sources.jar as a source directory without extracting the files, these knock-on effects disappear.

Attempts

A number of approaches have failed.

sourceSets

Changing sourceSets doesn't work:

```<code>sourceSets.main.java.srcDirs += "jar:${projectDir}/sources.jar!/"
</code>```

The error is:

Cannot convert URL 'jar:/home/user/dev/project/sources.jar!/' to a file.

Using a zipTree with sourceSets doesn't work, although the error message is telling:

```<code>sourceSets.main.java.srcDirs += zipTree(file: "${projectDir}/sources.jar")
</code>```

Error:

Cannot convert the provided notation to a File or URI.

The following types/formats are supported:

A URI or URL instance.

This was expected. What was unexpected was that URL instances are allowed, but seemingly not if embedded within a JAR file.

The following allows building Bob, but the IDE is unable to find SourceCode.java:

```<code>sourceSets.main.java.srcDirs += zipTree("${projectDir}/sources.jar").matching {
  include "com"
}
</code>```

build task

Modifying the build task to extract the generated code first partially works:

```<code>task codeGen {
  copy {
    from( zipTree( "${projectDir}/sources.jar" ) )
    into( "${buildDir}/src/gen/java" )
  }

  sourceSets.main.java.srcDirs += ["${buildDir}/src/gen/java"]
}

build { doFirst { codeGen } }
</code>```

The issue is that removing the build directory then prevents static compiles (because IDEA cannot find the generated source files). In any case, we don't want to extract the source files because of all the knock-on problems.

compile task

The following snippet also does not compile:

```<code>tasks.withType(JavaCompile) {
  source = zipTree(file: "${projectDir}/sources.jar")
}
</code>```

Not updating sourceSets means that the IDE cannot discover the source files.

sync task

We could extract the files into the main source directory, instead, such as:

```<code>def syncTask = task sync(type: Sync) {
    from zipTree("${projectDir}/sources.jar")
    into "${projectDir}/src/gen/java"

    preserve {
        include 'com/**'
        exclude 'META-INF/**'
    }
}

sourceSets.main.java.srcDir(syncTask)
</code>```

While that addresses the clean issue, we’re left with the original problems that we’d like to avoid.

Content Root

Setting the Content Root and marking the Source Folder from within IntelliJ IDEA works. The IDE updates .idea/misc.xml to include:

  <component name="FrameworkDetectionExcludesConfiguration">
    <file type="web" url="file://$PROJECT_DIR$" />
    <file type="web" url="jar://$PROJECT_DIR$/project/sources.jar!/" />
  </component>
```

In theory, the idea plugin may have the ability to set this value.

Question

How would you instruct Gradle to reference and build using source files that are stored in an external Java archive file when compiling a project (without extracting the archive) such that IDEA can statically resolve the source files, as well?

答:
Working Solution

I first believed it wasn’t possible to use a JAR file containing uncompiled Java code as additional sources in IntelliJ. After a few tries I could eventually configure it in the UI, though, thanks to the pointer from your “Content Root” section. A bit of fiddling with the IDEA plugin later, I could finally come up with a fully working solution:

plugins {
    id 'java'
    id 'idea'
}

tasks.withType(JavaCompile) {
    source(zipTree("${projectDir}/sources.jar"))
}

idea.module.iml {
    withXml {
        def baseUrl = 'jar://$MODULE_DIR$/sources.jar!/'
        def component = it.asNode().component[0]
        def jarContent = component.appendNode('content', [url: baseUrl])
        jarContent.appendNode('sourceFolder', [
                url: "${baseUrl}com",
                isTestSource: false,
            ])
    }
}
```
<p>Before opening the project in IntelliJ, you’ll have to run <code>./gradlew idea</code> and then open the generated <code>.ipr</code> file with IntelliJ. If IntelliJ says “Gradle build scripts found”, then <em>don’t</em> attempt to load the Gradle project but instead “Skip” this. You now have an IntelliJ project which is generated by Gradle but which is still independent of Gradle – and hence you can safely call <code>./gradlew clean</code> without affecting IntelliJ.</p>
<h2>Other Thoughts</h2>
<p>Here’s another approach of configuring the compiler from Gradle, using the (rarely used) <code>-sourcepath</code> option of <code>javac</code>:</p>
<pre class="lang-groovy prettyprint-override"><code>tasks.withType(JavaCompile) {
    options.sourcepath = files("${projectDir}/sources.jar")
}
</code>```

I personally would still prefer an approach where I don’t have to let Gradle generate an IntelliJ project but instead to let IntelliJ work with Gradle. However, that’ll require extracting the JAR file. A good Gradle solution for this would be the following:

plugins {
    id 'java'
}

def unzipAlice = tasks.register('unzipAlice', Sync) {
    from(zipTree("${projectDir}/sources.jar"))
    into(temporaryDir)
}

sourceSets.main.java.srcDir(unzipAlice)
```

This solution at least solves the “synchronization” problem mentioned in your question: the Sync task makes sure that no updates to sources.jar are ignored (including deletions).

About the “editable” problem, I wonder why you don’t trust your developers here? They could produce all kind of nonsense in the rest of the code anyway. Or even if the JAR isn’t extracted, they could still replace the files in the JAR. With this extraction solution, IntelliJ at least warns if the sources are edited:

Generated source files should not be edited. The changes will be lost when sources are regenerated.

Ok, if you run ./gradlew clean then IntelliJ will indeed not find the sources anymore.  But this can easily be fixed by either calling “Build” → “Build Project” in the UI or by running “./gradlew unzipAlice”. If the “./gradlew clean” issue should really be a dealbreaker for the extracting approach, then you could still consider extracting the sources outside of the build directory …

Body:
Background

Project Alice generates Java source code, stores it in sources.jar, then uploads it to a Maven repository. Project Bob pulls sources.jar down and needs to use it when compiling. Bob does not know that Alice exists, only where to find sources.jar.

Versions: JDK 11, Gradle 7.3.1, IntelliJ IDEA 2021.3.1

Problem

Making gradle (and IntelliJ's IDEA) build using source files embedded in a JAR file. To be clear, the JAR file contents resemble:

$ jar -tvf sources.jar
     0 Thu Feb 03 08:38:56 PST 2022 META-INF/
    52 Thu Feb 03 08:38:56 PST 2022 META-INF/MANIFEST.MF
     0 Thu Feb 03 08:38:30 PST 2022 com/
     0 Thu Feb 03 08:38:32 PST 2022 com/domain/
     0 Thu Feb 03 08:38:30 PST 2022 com/domain/package/
   938 Thu Feb 03 08:38:32 PST 2022 com/domain/package/SourceCode.java

Solutions that extract the .java files from the .jar file introduce knock-on effects we'd like to avoid, including:

Editable. Extracted source files can be edited in the IDE. We'd like them to be read-only. We could add a task that sets the files read-only, but that feels like solving the wrong problem (added complexity).

Synchronization. When a newly generated sources.jar is pulled down, we'll have to delete the extraction directory to remove any stale .java files that were preserved. If there was a way to avoid extraction, then the act of pulling down the new sources.jar file would ensure correctness (no added complexity). By unarchiving the .java files, it's possible to enter an inconsistent state:

$ jar -tvf sources.jar | grep java$ | wc -l
61
$ find src/gen -name "*java" | wc -l
65

If there was a way to treat sources.jar as a source directory without extracting the files, these knock-on effects disappear.

Attempts

A number of approaches have failed.

sourceSets

Changing sourceSets doesn't work:

sourceSets.main.java.srcDirs += "jar:${projectDir}/sources.jar!/"

The error is:

Cannot convert URL 'jar:/home/user/dev/project/sources.jar!/' to a file.

Using a zipTree with sourceSets doesn't work, although the error message is telling:

sourceSets.main.java.srcDirs += zipTree(file: "${projectDir}/sources.jar")

Error:

Cannot convert the provided notation to a File or URI.

The following types/formats are supported:

A URI or URL instance.

This was expected. What was unexpected was that URL instances are allowed, but seemingly not if embedded within a JAR file.

The following allows building Bob, but the IDE is unable to find SourceCode.java:

sourceSets.main.java.srcDirs += zipTree("${projectDir}/sources.jar").matching {
  include "com"
}

build task

Modifying the build task to extract the generated code first partially works:

task codeGen {
  copy {
    from( zipTree( "${projectDir}/sources.jar" ) )
    into( "${buildDir}/src/gen/java" )
  }

  sourceSets.main.java.srcDirs += ["${buildDir}/src/gen/java"]
}

build { doFirst { codeGen } }

The issue is that removing the build directory then prevents static compiles (because IDEA cannot find the generated source files). In any case, we don't want to extract the source files because of all the knock-on problems.

compile task

The following snippet also does not compile:

tasks.withType(JavaCompile) {
  source = zipTree(file: "${projectDir}/sources.jar")
}

Not updating sourceSets means that the IDE cannot discover the source files.

sync task

We could extract the files into the main source directory, instead, such as:

def syncTask = task sync(type: Sync) {
    from zipTree("${projectDir}/sources.jar")
    into "${projectDir}/src/gen/java"

    preserve {
        include 'com/**'
        exclude 'META-INF/**'
    }
}

sourceSets.main.java.srcDir(syncTask)

While that addresses the clean issue, we’re left with the original problems that we’d like to avoid.

Content Root

Setting the Content Root and marking the Source Folder from within IntelliJ IDEA works. The IDE updates .idea/misc.xml to include:

  <component name="FrameworkDetectionExcludesConfiguration">
    <file type="web" url="file://$PROJECT_DIR$" />
    <file type="web" url="jar://$PROJECT_DIR$/project/sources.jar!/" />
  </component>

In theory, the idea plugin may have the ability to set this value.

Question

How would you instruct Gradle to reference and build using source files that are stored in an external Java archive file when compiling a project (without extracting the archive) such that IDEA can statically resolve the source files, as well?

Body:
Working Solution

I first believed it wasn’t possible to use a JAR file containing uncompiled Java code as additional sources in IntelliJ. After a few tries I could eventually configure it in the UI, though, thanks to the pointer from your “Content Root” section. A bit of fiddling with the IDEA plugin later, I could finally come up with a fully working solution:

plugins {
    id 'java'
    id 'idea'
}

tasks.withType(JavaCompile) {
    source(zipTree("${projectDir}/sources.jar"))
}

idea.module.iml {
    withXml {
        def baseUrl = 'jar://$MODULE_DIR$/sources.jar!/'
        def component = it.asNode().component[0]
        def jarContent = component.appendNode('content', [url: baseUrl])
        jarContent.appendNode('sourceFolder', [
                url: "${baseUrl}com",
                isTestSource: false,
            ])
    }
}

Before opening the project in IntelliJ, you’ll have to run ./gradlew idea and then open the generated .ipr file with IntelliJ. If IntelliJ says “Gradle build scripts found”, then don’t attempt to load the Gradle project but instead “Skip” this. You now have an IntelliJ project which is generated by Gradle but which is still independent of Gradle – and hence you can safely call ./gradlew clean without affecting IntelliJ.

Other Thoughts

Here’s another approach of configuring the compiler from Gradle, using the (rarely used) -sourcepath option of javac:

tasks.withType(JavaCompile) {
    options.sourcepath = files("${projectDir}/sources.jar")
}

I personally would still prefer an approach where I don’t have to let Gradle generate an IntelliJ project but instead to let IntelliJ work with Gradle. However, that’ll require extracting the JAR file. A good Gradle solution for this would be the following:

plugins {
    id 'java'
}

def unzipAlice = tasks.register('unzipAlice', Sync) {
    from(zipTree("${projectDir}/sources.jar"))
    into(temporaryDir)
}

sourceSets.main.java.srcDir(unzipAlice)

This solution at least solves the “synchronization” problem mentioned in your question: the Sync task makes sure that no updates to sources.jar are ignored (including deletions).

About the “editable” problem, I wonder why you don’t trust your developers here? They could produce all kind of nonsense in the rest of the code anyway. Or even if the JAR isn’t extracted, they could still replace the files in the JAR. With this extraction solution, IntelliJ at least warns if the sources are edited:

Generated source files should not be edited. The changes will be lost when sources are regenerated.

Ok, if you run ./gradlew clean then IntelliJ will indeed not find the sources anymore.  But this can easily be fixed by either calling “Build” → “Build Project” in the UI or by running “./gradlew unzipAlice”. If the “./gradlew clean” issue should really be a dealbreaker for the extracting approach, then you could still consider extracting the sources outside of the build directory …


问:
How to left merge table_one with table_two based on latest month_year column?
table_one has distinct IDs. I want to left merge table_one on table_two based on latest date in month_year column. I did the following but didn't work.

```<code>import datetime
import pandas as pd

today = datetime.date.today()
first = today.replace(day=1)
lastMonth = first-datetime.timedelta(days=1)
latest_moyr =str(lastMonth.month)+ '_' + str(lastMonth.year)
</code>```
```<code>final_df = pd.merge(left = table_one, right = table_two.loc[table_two['month_year']== latest_moyr], left_on = 'ID', right_on = 'ID', how = 'left')
</code>```

table_one

ID
Yrs

1001
10

1002
2

1003
5

table_two

ID
sum3
month_year

1001
24.50
2_2013

1002
2.05
4_2013

1003
90.36
5_2013

1001
100
8_2013

1002
122
12_2014

1001
245
9_2018

1003
10.50
7_2011

1002
212
4_2018

1005
5.01
3_2014

I want to get like this

ID
Yrs
sum3
month_year

1001
10
245
9_2018

1002
2
212
4_2018

1003
5
90.36
5_2013

答:
You have to keep only last date per ID before merging. In order to do that, convert your month_year column into a proper datetime64 then sort by date and drop duplicates for each ID.

```<code>df3 = df1.merge(
        df2.assign(dt=pd.to_datetime(df2['month_year'], format='%m_%Y'))
           .sort_values('dt').drop_duplicates('ID', keep='last').drop(columns='dt'),
        on='ID', how='left')
</code>```

Output:

```<code>>>> df3
     ID  Yrs    sum3 month_year
0  1001   10  245.00     9_2018
1  1002    2  212.00     4_2018
2  1003    5   90.36     5_2013
</code>```

Body:
table_one has distinct IDs. I want to left merge table_one on table_two based on latest date in month_year column. I did the following but didn't work.

import datetime
import pandas as pd

today = datetime.date.today()
first = today.replace(day=1)
lastMonth = first-datetime.timedelta(days=1)
latest_moyr =str(lastMonth.month)+ '_' + str(lastMonth.year)

final_df = pd.merge(left = table_one, right = table_two.loc[table_two['month_year']== latest_moyr], left_on = 'ID', right_on = 'ID', how = 'left')

table_one

ID
Yrs

1001
10

1002
2

1003
5

table_two

ID
sum3
month_year

1001
24.50
2_2013

1002
2.05
4_2013

1003
90.36
5_2013

1001
100
8_2013

1002
122
12_2014

1001
245
9_2018

1003
10.50
7_2011

1002
212
4_2018

1005
5.01
3_2014

I want to get like this

ID
Yrs
sum3
month_year

1001
10
245
9_2018

1002
2
212
4_2018

1003
5
90.36
5_2013

Body:
You have to keep only last date per ID before merging. In order to do that, convert your month_year column into a proper datetime64 then sort by date and drop duplicates for each ID.

df3 = df1.merge(
        df2.assign(dt=pd.to_datetime(df2['month_year'], format='%m_%Y'))
           .sort_values('dt').drop_duplicates('ID', keep='last').drop(columns='dt'),
        on='ID', how='left')

Output:

>>> df3
     ID  Yrs    sum3 month_year
0  1001   10  245.00     9_2018
1  1002    2  212.00     4_2018
2  1003    5   90.36     5_2013


问:
java lambda expression in sorting. diff between .sort() and Arrays.sort()
Please see the code below, on the fourth line, put the lambda expression in the .sort() method, it works fine.

The fifth line, using Arrays.sort() will report an error.
I would like to know where is the difference between the two. and how to properly use lambda expressions in Arrays.sort()

```<code>    var strs = new ArrayList<String>();
    strs.add("AA");
    strs.add("BB");
    strs.sort((a,b) -> b.compareTo(a)); // OK
    // Arrays.sort(strs, (a, b) ->  b.compareTo(a) ); // CAN NOT COMPILE java: no suitable method found for sort...
</code>```

Related questions:
using Arrays.sort with Lambda

答:
As duncg mentioned just do:</p>
```<code>Object[] array = strs.toArray();
Arrays.sort(array, (a, b) ->  ((String) b).compareTo((String)a) );
</code>```

Body:
Please see the code below, on the fourth line, put the lambda expression in the .sort() method, it works fine.

The fifth line, using Arrays.sort() will report an error.
I would like to know where is the difference between the two. and how to properly use lambda expressions in Arrays.sort()

    var strs = new ArrayList<String>();
    strs.add("AA");
    strs.add("BB");
    strs.sort((a,b) -> b.compareTo(a)); // OK
    // Arrays.sort(strs, (a, b) ->  b.compareTo(a) ); // CAN NOT COMPILE java: no suitable method found for sort...

Related questions:
using Arrays.sort with Lambda

Body:
As duncg mentioned just do:

Object[] array = strs.toArray();
Arrays.sort(array, (a, b) ->  ((String) b).compareTo((String)a) );


问:
Query update last record only and value from other table
I have two tables

oc_stock_hpp

```<code>id product_id stock
2  1234       0
3  5678       0
4  1234       0
6  1234       0
9  5678       0
</code>```

oc_product

```<code>id product_id quantity
1  1234       7
23 5678       9
</code>```

My purpose is to update value in the stock field table oc_stock_hpp where id=6 become 7 and id=9 become 9 as in oc_product table

I have tried

```<code>UPDATE oc_stock_hpp t1
    INNER JOIN oc_product t2
         ON t1.product_id = t2.product_id
SET t1.stock = t2.quantity
WHERE t1.id = (SELECT MAX(id) FROM oc_stock_hpp GROUP BY product_id)
</code>```

and got error Subquery returns more than 1 row

答:
You must make the subquery return one and only one result:

```<code>UPDATE oc_stock_hpp t1
    INNER JOIN oc_product t2
         ON t1.product_id = t2.product_id
SET t1.stock = t2.quantity
WHERE t1.id = (SELECT MAX(id) FROM oc_stock_hpp GROUP BY product_id ORDER BY t1.id DESC LIMIT 1)
</code>```

This is the easiest way for me.

Body:
I have two tables

oc_stock_hpp

id product_id stock
2  1234       0
3  5678       0
4  1234       0
6  1234       0
9  5678       0

oc_product

id product_id quantity
1  1234       7
23 5678       9

My purpose is to update value in the stock field table oc_stock_hpp where id=6 become 7 and id=9 become 9 as in oc_product table

I have tried

UPDATE oc_stock_hpp t1
    INNER JOIN oc_product t2
         ON t1.product_id = t2.product_id
SET t1.stock = t2.quantity
WHERE t1.id = (SELECT MAX(id) FROM oc_stock_hpp GROUP BY product_id)

and got error Subquery returns more than 1 row

Body:
You must make the subquery return one and only one result:

UPDATE oc_stock_hpp t1
    INNER JOIN oc_product t2
         ON t1.product_id = t2.product_id
SET t1.stock = t2.quantity
WHERE t1.id = (SELECT MAX(id) FROM oc_stock_hpp GROUP BY product_id ORDER BY t1.id DESC LIMIT 1)

This is the easiest way for me.


问:
how can I install package from a downloaded zip
I am behind a firewall on MS Windows and setting the proxy in my init file did not work. I can download the zip archive from GitHub, but when the package has many .el files the install-from-file does not work. I tried saving the folder as .tar but it still did not work.

How can I install the downloaded folder? There is a Makefile in the folder, but I have no make or its alternatives on Windows and will not be allowed to install them.

Prior to the migration to GitHub I was able to download a single file from Melpa and that worked fine.

答:
a workaround is to add the package folder to your load-path and then require the package.

This will not take care of any dependencies, so watch in *Warnings* and *Messages* buffers for any problems caused by unmet dependencies.

See the details here.


问:
Create new dataframe by substracting every column with each other, solving poor performance issues
Error:

"DataFrame is highly fragmented.  This is usually the result of
calling frame.insert many times, which has poor performance.
Consider joining all columns at once using pd.concat(axis=1) instead.
To get a de-fragmented frame, use `newframe = frame.copy()"

Code:

```<code>df_DIFFs = pd.DataFrame()

for a, b in itertools.permutations(df.columns,2):
    df_DIFFs[f'{a}-{b}'] = df[a] - df[b]

df_DIFFs
</code>```

I can't imagine how to use pd.concat or any other more efficient solution in this situation.
Thanks.

Body:
Error:

"DataFrame is highly fragmented.  This is usually the result of
calling frame.insert many times, which has poor performance.
Consider joining all columns at once using pd.concat(axis=1) instead.
To get a de-fragmented frame, use `newframe = frame.copy()"

Code:

df_DIFFs = pd.DataFrame()

for a, b in itertools.permutations(df.columns,2):
    df_DIFFs[f'{a}-{b}'] = df[a] - df[b]

df_DIFFs

I can't imagine how to use pd.concat or any other more efficient solution in this situation.
Thanks.

Body:
You can follow the advice by storing column differences in a list and then concatenating:

# our test df
df = pd._testing.makeDataFrame()

# here we will store columns that are differences
diff_cols = []

# iterate
for a, b in itertools.permutations(df.columns,2):
    tmp = (df[a] - df[b]).rename(f'{a}-{b}')
    diff_cols.append(tmp)

# concatenate
df_DIFFs = pd.concat(diff_cols, axis=1)

output
df_DIFFs

output:

                   A-B         A-C         A-D         B-A         B-C         B-D         C-A         C-B         C-D         D-A         D-B         D-C
----------  ----------  ----------  ----------  ----------  ----------  ----------  ----------  ----------  ----------  ----------  ----------  ----------
igc6RyqBmG   1.02425     1.39726    -0.460256   -1.02425     0.373015   -1.48451    -1.39726    -0.373015   -1.85752     0.460256    1.48451     1.85752
MMIfUNYx8i   1.18588     3.64252     0.0921551  -1.18588     2.45664    -1.09373    -3.64252    -2.45664    -3.55037    -0.0921551   1.09373     3.55037
2MhnKJeZ2K   0.86183     1.00485    -1.12921    -0.86183     0.143022   -1.99104    -1.00485    -0.143022   -2.13406     1.12921     1.99104     2.13406
in9wt14lku  -1.40201    -0.0527898  -1.73563     1.40201     1.34922    -0.333613    0.0527898  -1.34922    -1.68284     1.73563     0.333613    1.68284
7S6tkx2WFN   0.779529    1.40291     0.418794   -0.779529    0.623382   -0.360735   -1.40291    -0.623382   -0.984117   -0.418794    0.360735    0.984117
1C2wmIahde  -2.18965     0.161868   -1.36776     2.18965     2.35152     0.821893   -0.161868   -2.35152    -1.52963     1.36776    -0.821893    1.52963
oRqd898jAl   0.658137    1.58394     0.712051   -0.658137    0.925807    0.0539145  -1.58394    -0.925807   -0.871893   -0.712051   -0.0539145   0.871893
...


问:
Deploy GCP Cloud functions to Artifact Registry using Terraform
I am trying to deploy cloud function to artifact registry instead of container registry using Terraform.

I have created an artifact repository in GCP and Using the google-beta provider.
But I am not able to understand where to mention "docker-registry" path(path for artifact registry)

Following in my main tf file's create CF:-
I have added a parameter called docker-repository(this doesn't exist in terraform) based on https://cloud.google.com/functions/docs/building#image_registry_options
But looks like this parameter doesn't exist in terraform and is giving me errors.

```<code>resource "google_cloudfunctions_function" "appConfigService" {
  provider              = google-beta
  name                  = local.function_names.appConfigService
  description           = "helloWorld"
  runtime               = var.cf_node_run_time
  available_memory_mb   = var.cf_memory
  source_archive_bucket = local.deployment_bucket
  source_archive_object = google_storage_bucket_object.appConfigService_archive.name
  entry_point           = "helloWorld"
  service_account_email = var.default_service_account[var.environment]
  trigger_http          = true
  docker-repository     ="<artifact registry path>" //This is wrong

}
</code>```

I am not able to find any documentation on this anywhere.
Please let me know the correct way of deploying Cloud functions to artifact repository using terraform.

答:
At this time, your will need to use Terraform plus Cloud Build to specify the repository to use. You can then use gcloud --docker-repository in a Cloud Build step.

This document explains how to integrate Terraform with Cloud Build.

Managing infrastructure as code with Terraform, Cloud Build, and GitOps

Body:
I am trying to deploy cloud function to artifact registry instead of container registry using Terraform.

I have created an artifact repository in GCP and Using the google-beta provider.
But I am not able to understand where to mention "docker-registry" path(path for artifact registry)

Following in my main tf file's create CF:-
I have added a parameter called docker-repository(this doesn't exist in terraform) based on https://cloud.google.com/functions/docs/building#image_registry_options
But looks like this parameter doesn't exist in terraform and is giving me errors.

resource "google_cloudfunctions_function" "appConfigService" {
  provider              = google-beta
  name                  = local.function_names.appConfigService
  description           = "helloWorld"
  runtime               = var.cf_node_run_time
  available_memory_mb   = var.cf_memory
  source_archive_bucket = local.deployment_bucket
  source_archive_object = google_storage_bucket_object.appConfigService_archive.name
  entry_point           = "helloWorld"
  service_account_email = var.default_service_account[var.environment]
  trigger_http          = true
  docker-repository     ="<artifact registry path>" //This is wrong

}

I am not able to find any documentation on this anywhere.
Please let me know the correct way of deploying Cloud functions to artifact repository using terraform.


问:
Using Firebase and Firestore as complementary backend
I have developed my own backend for user registration and user authentication. All this info is stored in a database in which I have full control. No passwords are saved (I am using jwt) and there are two tables:

One for users

One for privacy settings

I need to have one more table (let's say a posts) that needs to provide real-time features (streams). For this, the simplest solution I can think of is Firestore as it supports real-time and also sync with mobile app.

My question is: as I want to have as much control as possible, especially when it comes to user data (name, lastname etc) is it a good idea to share my app's backend between my own solution and Firestore for one table?

How I am thinking of it working is as follows:

User with id 123 creates a post

The post data is stored in Firestore with creator_id = 123

My front-end fetches the data and also queries my own db to find the name and surname of this person with the id '123' in order to show it to the front-end.

Do you think it is a viable, efficient and scalable solution? Or should I just dump everything in Firestore?

Please note that at this moment, developing my own real-time solution is not possible, so Firestore is the single solution.

答:
Whether the architecture is a good idea for your app, only you can determine. But it's quite common to combine Firestore (or the Realtime Database, the other database in Firebase) with existing backend infrastructure.

Typical things to consider in such a combined setup:

Does Firestore also need to know about your users? For example, if you want to ensure that each user can only read their own data from Firestore, you will need to sign the users in with Firebase too. To learn how to do that, have a look at the Firebase documentation on creating a custom auth provider.

What it main copy of your data, which all other copies are derived from. It sounds like your existing DB is the main copy, but it's good to be explicit about this for every piece of data.


问:
Does C# do tail recursion?

Possible Duplicate:

  Why doesn't .net/C# eliminate tail recursion?

Does C# do tail recusion?

I can't find any documentation telling me if it does or not.

答:
Unfortunately, it does not, at least not yet.

I'm not sure if the standard itself specifies anything about (dis)allowing tail recursion. Regardless, since .Net supports tail recursion, so it would be nice for this to make its way into C#.

If you really need tail recursion in a .Net language, consider F# as an alternative.

Title: Does C# do tail recursion?


问:
Google Sheets JScript Row match and delete if - Take rows from Sheet A and delete if identical to rows in Sheet B, Sheet C, Sheet D... etc
My data import (Sheet A) has all information from the last known events. It only changes them when another event occurs. So if the event is from two days ago it is still in my data import (Sheet A) until that event changes. I need to have my data import (Sheet A) checked against Sheet B, Sheet C, and Sheet D for identical rows. Then delete the row if it already exists.

So far I have:

function testFilter() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheetA = ss.getSheetByName("SheetA");
var sheetB = ss.getSheetByName("SheetB");
var sheetC = ss.getSheetByName("SheetC");
var sheetD = ss.getSheetByName("SheetD");
var sheetE = ss.getSheetByName("SheetE");
var sheetF = ss.getSheetByName("SheetF");

var valuesA = sheetA.getRange('A2:AC400').getValues();
var valuesB = sheetB.getRange('A2:AC400').getValues();
var valuesC = sheetC.getRange('A2:AC400').getValues();
var valuesD = sheetD.getRange('A2:AC400').getValues();
var valuesE = sheetE.getRange('A2:AC400').getValues();
var valuesF = sheetF.getRange('A2:AC400').getValues();

for (var rowa in valuesA)
{
var deleteRowa = false;
  for (var rowb in valuesB)
  {
    if(valuesA[rowa].join() == valuesB[rowb].join())
      deleteRowa = true;
  }
    if(deleteRowa == true){
      sheetA.deleteRow(rowa)
    }
SpreadsheetApp.flush();
}
for (var rowa in valuesA)
{
var deleteRowa = false;
  for (var rowc in valuesC)
  {
    if(valuesA[rowa].join() == valuesC[rowc].join())
      deleteRowa = true;
  }
    if(deleteRowa == true){
      sheetA.deleteRow(rowa)
    }
SpreadsheetApp.flush();
}
for (var rowa in valuesA)
{
var deleteRowa = false;
  for (var rowd in valuesD)
  {
    if(valuesA[rowa].join() == valuesD[rowd].join())
      deleteRowa = true;
  }
    if(deleteRowa == true){
      sheetA.deleteRow(rowa)
    }
SpreadsheetApp.flush();
}
for (var rowa in valuesA)
{
var deleteRowa = false;
  for (var rowe in valuesE)
  {
    if(valuesA[rowa].join() == valuesE[rowe].join())
      deleteRowa = true;
  }
    if(deleteRowa == true){
      sheetA.deleteRow(rowa)
    }
SpreadsheetApp.flush();
}
for (var rowa in valuesA)
{
var deleteRowa = false;
  for (var rowf in valuesF)
  {
    if(valuesA[rowa].join() == valuesF[rowf].join())
      deleteRowa = true;
  }
    if(deleteRowa == true){
      sheetA.deleteRow(rowa)
    }
SpreadsheetApp.flush();
}
}```

This gives an Exception: out of bounds error on the first attempt of filtering. Does the deleterows() function work for picking out specific rows? Or am I going to have to throw it into a  new list, then delete all rows from original sheet, then insert the list?

答:
Delete Matching Rows in SheetA

```<code>function testFilter() {
  const ss = SpreadsheetApp.getActiveSpreadsheet();
  const shts = ["SheetB", "SheetC", "SheetD", "SheetE", "SheetF"];
  const A1rg = "A2:AC400";
  let uA = [];//building a unique flat array of row join
  ss.getSheets().filter(sh => ~shts.indexOf(sh.getName())).forEach(sh => {
    sh.getRange(A1rg).getValues().forEach(r => {
      let j = r.join("");
      if (!~uA.indexOf(j)) {
        uA.push(j);
      }
    });
  });
  let d = 0;
  const shA = ss.getSheetByName("SheetA");
  shA.getRange(A1rg).getValues().forEach((r, i) => {
    if (~uA.indexOf(r.join(""))) {
      shA.deleteRow(i + 2 - d++);
    }
  });
}
</code>```

This version runs quite a bit faster for me

```<code>function testFilter() {
  const ss = SpreadsheetApp.getActiveSpreadsheet();
  const shts = ["Sheet1", "Sheet2", "SheetD", "SheetE", "SheetF"];
  const A1rg = "A2:AC400";
  let uA = [];
  ss.getSheets().filter(sh => ~shts.indexOf(sh.getName())).forEach(sh => {
    sh.getRange(A1rg).getValues().forEach(r => {
      let j = r.join("");
      if (!~uA.indexOf(j)) {
        uA.push(j);
      }
    });
  });
  let d = 0;
  const shA = ss.getSheetByName("Sheet0");
  shA.getRange(2, 1, shA.getLastRow() - 1, shA.getLastColumn()).getValues().forEach((r, i) => {
    if (~uA.indexOf(r.join(""))) {
      shA.deleteRow(i + 2 - d++);
    }
  });
}
</code>```

Body:
My data import (Sheet A) has all information from the last known events. It only changes them when another event occurs. So if the event is from two days ago it is still in my data import (Sheet A) until that event changes. I need to have my data import (Sheet A) checked against Sheet B, Sheet C, and Sheet D for identical rows. Then delete the row if it already exists.

So far I have:

function testFilter() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheetA = ss.getSheetByName("SheetA");
var sheetB = ss.getSheetByName("SheetB");
var sheetC = ss.getSheetByName("SheetC");
var sheetD = ss.getSheetByName("SheetD");
var sheetE = ss.getSheetByName("SheetE");
var sheetF = ss.getSheetByName("SheetF");

var valuesA = sheetA.getRange('A2:AC400').getValues();
var valuesB = sheetB.getRange('A2:AC400').getValues();
var valuesC = sheetC.getRange('A2:AC400').getValues();
var valuesD = sheetD.getRange('A2:AC400').getValues();
var valuesE = sheetE.getRange('A2:AC400').getValues();
var valuesF = sheetF.getRange('A2:AC400').getValues();

for (var rowa in valuesA)
{
var deleteRowa = false;
  for (var rowb in valuesB)
  {
    if(valuesA[rowa].join() == valuesB[rowb].join())
      deleteRowa = true;
  }
    if(deleteRowa == true){
      sheetA.deleteRow(rowa)
    }
SpreadsheetApp.flush();
}
for (var rowa in valuesA)
{
var deleteRowa = false;
  for (var rowc in valuesC)
  {
    if(valuesA[rowa].join() == valuesC[rowc].join())
      deleteRowa = true;
  }
    if(deleteRowa == true){
      sheetA.deleteRow(rowa)
    }
SpreadsheetApp.flush();
}
for (var rowa in valuesA)
{
var deleteRowa = false;
  for (var rowd in valuesD)
  {
    if(valuesA[rowa].join() == valuesD[rowd].join())
      deleteRowa = true;
  }
    if(deleteRowa == true){
      sheetA.deleteRow(rowa)
    }
SpreadsheetApp.flush();
}
for (var rowa in valuesA)
{
var deleteRowa = false;
  for (var rowe in valuesE)
  {
    if(valuesA[rowa].join() == valuesE[rowe].join())
      deleteRowa = true;
  }
    if(deleteRowa == true){
      sheetA.deleteRow(rowa)
    }
SpreadsheetApp.flush();
}
for (var rowa in valuesA)
{
var deleteRowa = false;
  for (var rowf in valuesF)
  {
    if(valuesA[rowa].join() == valuesF[rowf].join())
      deleteRowa = true;
  }
    if(deleteRowa == true){
      sheetA.deleteRow(rowa)
    }
SpreadsheetApp.flush();
}
}

This gives an Exception: out of bounds error on the first attempt of filtering. Does the deleterows() function work for picking out specific rows? Or am I going to have to throw it into a  new list, then delete all rows from original sheet, then insert the list?

Body:
Delete Matching Rows in SheetA

function testFilter() {
  const ss = SpreadsheetApp.getActiveSpreadsheet();
  const shts = ["SheetB", "SheetC", "SheetD", "SheetE", "SheetF"];
  const A1rg = "A2:AC400";
  let uA = [];//building a unique flat array of row join
  ss.getSheets().filter(sh => ~shts.indexOf(sh.getName())).forEach(sh => {
    sh.getRange(A1rg).getValues().forEach(r => {
      let j = r.join("");
      if (!~uA.indexOf(j)) {
        uA.push(j);
      }
    });
  });
  let d = 0;
  const shA = ss.getSheetByName("SheetA");
  shA.getRange(A1rg).getValues().forEach((r, i) => {
    if (~uA.indexOf(r.join(""))) {
      shA.deleteRow(i + 2 - d++);
    }
  });
}

This version runs quite a bit faster for me

function testFilter() {
  const ss = SpreadsheetApp.getActiveSpreadsheet();
  const shts = ["Sheet1", "Sheet2", "SheetD", "SheetE", "SheetF"];
  const A1rg = "A2:AC400";
  let uA = [];
  ss.getSheets().filter(sh => ~shts.indexOf(sh.getName())).forEach(sh => {
    sh.getRange(A1rg).getValues().forEach(r => {
      let j = r.join("");
      if (!~uA.indexOf(j)) {
        uA.push(j);
      }
    });
  });
  let d = 0;
  const shA = ss.getSheetByName("Sheet0");
  shA.getRange(2, 1, shA.getLastRow() - 1, shA.getLastColumn()).getValues().forEach((r, i) => {
    if (~uA.indexOf(r.join(""))) {
      shA.deleteRow(i + 2 - d++);
    }
  });
}


问:
Spring cloud stream - Kafka consumer consuming duplicate messages with StreamListener
With our spring boot app, we notice kafka consumer consuming message twice randomly once in a while only in prod env. We have 6 instances with 6 partitions deployed in PCF.We have caught messages with same offset and partition received twice in same topic which causes duplicates and is a business critical for us.
We haven't noticed this in non production env and it is hard to reproduce in non prod env. We have recently switched to Kafka and we are not able to find out the root issue.

We are using spring-cloud-stream/spring-cloud-stream-binder-kafka- 2.1.2
Here is the Config:

```<code>spring:
  cloud:
    stream:
      default.consumer.concurrency: 1
      default-binder: kafka
      bindings:
        channel:
          destination: topic
          content_type: application/json
          autoCreateTopics: false
          group: group
          consumer:
            maxAttempts: 1
      kafka:
        binder:
          autoCreateTopics: false
          autoAddPartitions: false
          brokers: brokers list
        bindings:
          channel:
            consumer:
              autoCommitOnError: true
              autoCommitOffset: true
              configuration:
                max.poll.interval.ms: 1000000
                max.poll.records: 1
                group.id: group
</code>```

We use @Streamlisteners to consume the messages.

Here is the instance we received duplicate and the error message captured in server logs.

ERROR 46 --- [container-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator
: [Consumer clientId=consumer-3, groupId=group] Offset commit failed
on partition topic-0 at offset 1291358: The coordinator is not aware
of this member. ERROR 46 --- [container-0-C-1]
o.s.kafka.listener.LoggingErrorHandler   : Error while processing:
null OUT org.apache.kafka.clients.consumer.CommitFailedException:
Commit cannot be completed since the group has already rebalanced and
assigned the partitions to another member.  This means that the time
between subsequent calls to poll() was longer than the configured
max.poll.interval.ms,  which typically implies that the poll loop is
spending too much time message processing.   You can address this
either by increasing the session timeout or by reducing the maximum
size of batches returned in poll() with max.poll.records.
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:871)
~[kafka-clients-2.0.1.jar!/:na]

There is no crash and all the instances are healthy at the time of duplicate. Also there is confusion with error log - Error while processing: null since Message was successfully  processed twice. And max.poll.interval.ms: 100000 which is about 16 minutes and it is supposed to be enough time to process any message for the system and session timeout and heartbit config is default. Duplicate is received within 2 seconds in most of the instances.
Any configs that we are missing  ? Any suggestion/help is highly appreciated.

答:
Commit cannot be completed since the group has already rebalanced

A rebalance occurred because your listener took too long; you should adjust max.poll.records and max.poll.interval.ms to make sure you can always handle the records received within the timelimit.

In any case, Kafka does not guarantee exactly once delivery, only at least once delivery. You need to add idempotency to your application and detect/ignore duplicates.

Body:
With our spring boot app, we notice kafka consumer consuming message twice randomly once in a while only in prod env. We have 6 instances with 6 partitions deployed in PCF.We have caught messages with same offset and partition received twice in same topic which causes duplicates and is a business critical for us.
We haven't noticed this in non production env and it is hard to reproduce in non prod env. We have recently switched to Kafka and we are not able to find out the root issue.

We are using spring-cloud-stream/spring-cloud-stream-binder-kafka- 2.1.2
Here is the Config:

spring:
  cloud:
    stream:
      default.consumer.concurrency: 1
      default-binder: kafka
      bindings:
        channel:
          destination: topic
          content_type: application/json
          autoCreateTopics: false
          group: group
          consumer:
            maxAttempts: 1
      kafka:
        binder:
          autoCreateTopics: false
          autoAddPartitions: false
          brokers: brokers list
        bindings:
          channel:
            consumer:
              autoCommitOnError: true
              autoCommitOffset: true
              configuration:
                max.poll.interval.ms: 1000000
                max.poll.records: 1
                group.id: group

We use @Streamlisteners to consume the messages.

Here is the instance we received duplicate and the error message captured in server logs.

ERROR 46 --- [container-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator
: [Consumer clientId=consumer-3, groupId=group] Offset commit failed
on partition topic-0 at offset 1291358: The coordinator is not aware
of this member. ERROR 46 --- [container-0-C-1]
o.s.kafka.listener.LoggingErrorHandler   : Error while processing:
null OUT org.apache.kafka.clients.consumer.CommitFailedException:
Commit cannot be completed since the group has already rebalanced and
assigned the partitions to another member.  This means that the time
between subsequent calls to poll() was longer than the configured
max.poll.interval.ms,  which typically implies that the poll loop is
spending too much time message processing.   You can address this
either by increasing the session timeout or by reducing the maximum
size of batches returned in poll() with max.poll.records.
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:871)
~[kafka-clients-2.0.1.jar!/:na]

There is no crash and all the instances are healthy at the time of duplicate. Also there is confusion with error log - Error while processing: null since Message was successfully  processed twice. And max.poll.interval.ms: 100000 which is about 16 minutes and it is supposed to be enough time to process any message for the system and session timeout and heartbit config is default. Duplicate is received within 2 seconds in most of the instances.
Any configs that we are missing  ? Any suggestion/help is highly appreciated.