#ubuntu-ci-eng 2014-03-01 <bzoltan> hello folks... would anybody be here to reconfigure the silo13? Set an other strange conflict ... <bzoltan> robotfuel: cyphermox, rsalveti ^ <ogra_> bzoltan, heh, good luck with getting any landings on the weekend <bzoltan> ogra_: I am not after landing ... but to get a bloody non conflicting build in the Silo <bzoltan> ogra_: :) that is the fun part ... all MRs were tested, gave green and merged without conflict locally <ogra_> well, there are simply no propoer plans for driving the silos on weekends yet <ogra_> (i brought that up last week, but there is no solution atm) <bzoltan> ogra_: I do not hav high expectations :) <bzoltan> ogra_: btw, would you reconfigure it? If that is what it needs... <asac> cyphermox: want to help a citrain? <asac> emergency landing? <asac> cyphermox: dont worry <asac> we wait till monday (in case you read this) <bzoltan> whoever reconfigured my silo :) thank you a lot ... <bzoltan> does anybody know what that means http://(864)930-0890/job/landing-013-3-merge-clean/13/console <rsalveti> === Building image 215 === <bzoltan> rsalveti: do you know what that error is about? ^^ <sil2100> mandel: hi! <sil2100> mandel: regarding that broken u-d-m - does the branch attached to the bug fix the problem? <sil2100> asac: I'll try to do an emergency landing ASAP <sil2100> popey: ^ <popey> sil2100: no idea. <popey> sil2100: i can test if someone can build me a deb or whatever? <sil2100> popey: I'll test it as well, I'm building it in a silo now <sil2100> popey: would appreciate your help <sil2100> Major fuckup from my side <popey> sil2100: No craighoward@example.net. Lets focus on getting it fixed and tested and not dwell on who broke it. <popey> sil2100: my device is on 212 right now, should I update to 214 and then await a ppa? <sil2100> I guess 214 might be a good idea, as it's broken <popey> kk <popey> doing now <cjwatson> woo, finally all tests passing on my libclick branch <cjwatson> might not be too far away now ... <popey> sil2100: just ping me when you need something, I'm afk but I get pinged from highlights <sil2100> popey: ppa:ci-train-ppa-service/landing-004 <- PPA link! <sil2100> It's built <popey> sil2100: ok <popey> sil2100: you planning on doing ap tests? <sil2100> popey: dogfooding for sure, not sure if we have AP tests for this? <popey> no idea <popey> root@ubuntu-phablet:/# apt-cache policy ubuntu-download-manager <popey> ubuntu-download-manager: Installed: 0.3+14.04.20140224-0ubuntu1 Candidate: 0.3+14.04.20140301-0ubuntu1 <popey> sil2100: testing installing a new click package and using update manager, both work with that ppa installed * popey tests a bit more * sil2100 reboots after an upgrade <sil2100> popey: can you try the update tests from https://wiki.ubuntu.com/Process/Merges/TestPlan/ubuntu-download-manager as well? <popey> sil2100: ok <sil2100> popey: thanks! <sil2100> So far so good on my device as well <popey> sil2100: hm, i cant do 3g test, no 3g data on this sim <sil2100> popey: then WiFi should be enough, I'll do 3G data test otherwise <popey> thanks <sil2100> hmmmm <sil2100> Crap, 3G downloading doesn't seem to work ;/ <sil2100> Let me revert and see if it worked in the past <popey> you sure you have data available on your sim? not run out of allowance? <sil2100> Webbrowser works fine <popey> ok <sil2100> And I'm pinging stuff fine <popey> worth checking â» <sil2100> But the script for downloading just doesn't work ;/ While it works on WiFi <sil2100> I'll retry the test <sil2100> geh <popey> wassup? <sil2100> So, it seems that strangely now switching from 3G to WiFi (and the other way around) breaks the downloading - at least it seems so <popey> sounds like that problem I reported in browser <sil2100> I would say this: I'll fill in a bug but land this anyway <sil2100> ralsina_: ping <popey> sil2100: i have re-triggered bug 1277589 <ubot5> bug 1277589 in Ubuntu system image "Better protection against concurrent access" [Critical,Fix released] https://launchpad.net/bugs/1277589 <popey> (while testing your u-d-m update" <sil2100> Crap <sil2100> So, it's reintroducing a regression, yes? <popey> however <popey> no <popey> i downgraded u-d-m and it still happens <sil2100> Ah, but you downgraded to which version? <popey> i just think that bug isn't fixed - separately from your u-d-m issue <sil2100> The archive has a broken one <popey> "This bug was fixed in the package ubuntu-system-settings - 0.1+14.04.20140218.1-0ubuntu1 <popey> " <sil2100> popey: to which you downgraded? <popey> no, hang on... <popey> I update to your new u-d-m <popey> triggered the above bug, thought "uh-oh" so I downgraded u-d-m back to the one in #212 <sil2100> Oh <popey> to make sure that didnt re-introduce it <popey> and I can trigger the above bug again on #212 stock <sil2100> And it's there, right? 212 seems to have the 'right' version <sil2100> Ok <sil2100> phew. <popey> yes, 212 has the right version, but its still broken <popey> sorry, shouldn't have mentioned it â» <popey> it's clouding the conversation â» <popey> just going to do one more test of your u-d-m upgrading from 212 to 215 <sil2100> :) <sil2100> \o/ <sil2100> If it goes good then let's release <sil2100> \o\ <popey> k * sil2100 keeps his fingers crossed <popey> sil2100: ok, i upgraded over wifi and it worked fine with your u-d-m, and I did the other tests for installing click packages too <sil2100> popey: awesome, thanks for the help man ;) <sil2100> Let's get this published <popey> np <popey> thanks for jumping in to fix <popey> sil2100: its a shame 215 was so recently triggered, will this be in the 3am image? <sil2100> popey: I guess so <sil2100> popey: I don't have the power of triggering image builds, so we'll have to wait for the next tick <popey> sil2100: rsalveti can <sil2100> popey: you can try poking him if he's around ;) But first wait until it hits the archive properly! <sil2100> Sicne it's probably still in -proposed <popey> well right now 215 is still being qa'ed <popey> http://ci.ubuntu.com/smokeng/trusty/touch/ <popey> so would be preferable to leave it some time. and as it's saturday, might be prudent to just wait for the 3am kick, or piggyback on another image that someone else may kick for whatever reason <ogra_> want an image build once it is in ? <popey> unconvinced it's worth it, what do you think ogra_? <ogra_> who built 215 ? <popey> ricardo <popey> 2.5 hours ago <ogra_> ah, yeah, i see it in the backlog <ogra_> yeah, then testing will still take 3-4h for it <ogra_> which means if i built then 216 will clash with 217 testing <ogra_> so lets juts leave it to cron (3am UTC) <ogra_> else we will only get messy test results in the end <popey> +1 <popey> sil2100: want to reply to that mail on the list letting everyone know? <sil2100> popey: to which mail? <sil2100> popey: I already sent out an info that we did that landing and the next image should be ok <sil2100> To the ML <popey> cool * ogra_ sent a followup about the images <popey> i hadn't seen it, sorry * ogra_ goes back to watch snooker :) * popey goes back to portal 2 <popey> *on* *linux* :D <sil2100> :D <sil2100> Ok, I see u-d-m is ready for migration, so I go now <sil2100> See you on Monday! (hopefully) <ogra_> rsalveti, wow, ofono is seriously broken (at least on e.crash from the scripts per test ... some have two) #ubuntu-ci-eng 2014-03-02 <rsalveti> ogra_: seriously? <rsalveti> yeah, quite a few crashes, weird <rsalveti> wonder if it's only broken with phonesim <asac> rsalveti: online :)? <asac> guess same idea <asac> hehe <asac> rsalveti: is the other sim still on the table? <rsalveti> actually, just 2 crashes it seems, maybe related with some python 2 specific code still <rsalveti> as it's not really ofono <rsalveti> dbus.exceptions.DBusException: org.ofono.Error.Failed: Operation failed <rsalveti> dbus.exceptions.DBusException: org.freedesktop.DBus.Error.ServiceUnknown: The name org.ofono was not provided by any .service files <asac> rsalveti: stop working :) <asac> not productive * asac goes off <bzoltan> ogra_: do you know what the hack causes this -> http://(864)930-0890/job/landing-013-3-merge-clean/ ? <rsalveti> 2014-03-02 11:34:58,300 DEBUG Copying configuration from /srv/juju/vol-0000011d/var/lib/jenkins/silos/landing-013/config to /var/lib/jenkins/status/landing-013 <rsalveti> 2014-03-02 11:34:58,300 ERROR Last step didn't finish successfully. You need to either ignore that the previous step didn't finished successfully or ensuring that prepare, build and checked passed. <rsalveti> maybe you need to build it again? <rsalveti> no, same error <rsalveti> not sure why then <bzoltan> rsalveti: thanks for checking it out ... I have no idea what the error means <Laney> bzoltan: err, it means that the silo hasn't been published, right? <Laney> https://launchpad.net/ubuntu/+source/ubuntu-ui-toolkit <bzoltan> Laney: yes <Laney> so you need to do that before you can do the last step <bzoltan> Laney: So I can not merge to the trunk before it gets published? <rsalveti> bzoltan: no, first published, then you merge <bzoltan> rsalveti: OK, thanks <bzoltan> rsalveti: what does it take to get that MP published? <rsalveti> bzoltan: well, you need to publish your silo, once the packages are in the distro, you hit merge & clean, which will get your MR merged <bzoltan> rsalveti: I can not publish the silo as far as I know... or can I? <ogra_> wow, that email client works already pretty well <bzoltan> rsalveti: I am a low life lander of the SDK :) <ogra_> oops, wrong channel :P <Laney> bzoltan: you need a core-dev or lander to publish it <Laney> s/lander/landing team person/ <bzoltan> Laney: I know .. that is why I am here :) <bzoltan> Laney: I know my chances are low today :) <Laney> I could do it, don't know if I'm allowed to though <Laney> might be best to wait <bzoltan> Laney: you know better the rules than me :) <rsalveti> bzoltan: which silo, 13? <bzoltan> rsalveti: yes <rsalveti> bzoltan: did you test and validate the changes already? <bzoltan> rsalveti: yes <rsalveti> bzoltan: publishing <bzoltan> rsalveti: thank you! <bzoltan> rsalveti: it is cool that my boys will have clean trunk tomorrow morning <rsalveti> :-) <Laney> I was scared by the degraded mode thing <Laney> but good that someone else wasn't :P <ogra_> heh <ogra_> 7666 phablet 20 0 355452 88588 44532 R 103.6 4.8 2:42.59 ubuntu-email-client <ogra_> 103% * cjohnston changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: cihelp | CITrain support - US: robru, cyphermox, rsalveti - EU: sil2100, Mirv, didrocks | CITrain support no answer: use mup bot after 30 minutes, but choose right timezone | Landing instructions: http://goo.gl/8H1Du3. Landing in degraded mode (see http://goo.gl/J1EqPW) | Known Issues: - #ubuntu-ci-eng 2014-03-03 <didrocks> bzoltan: rsalveti: hey, looking at the discussion yesterday after seeing the sdk being published <bzoltan> didrocks: yes sir <didrocks> bzoltan: rsalveti: remember that we are in a degraded mode, and the landing team is retesting every landing <didrocks> seems this one wasn't tested on our side, was it? <bzoltan> didrocks: I do not know about your side <didrocks> bzoltan: so, I guess it's a "no", let's hope we won't discover any issue with the sdk today <bzoltan> didrocks: I am on my side :) <didrocks> (tests are still running) <didrocks> bzoltan: always better to double check. I was happy to relax the "no landing rule" to help getting things landing, but at least, we need to follow the rule :/ <bzoltan> didrocks: one thing is for sure ... merge conflicts are just the first fold, the MRs can have conflicting tests ... <bzoltan> didrocks: of course... I would not never ask for bending the rules. <didrocks> yeah, but the landing was done this way though :/ <didrocks> (the one from this week-end) <didrocks> let's cross fingers nothing was unseen and wait for the test results <bzoltan> didrocks: sorry for that, I was not aware of that <didrocks> bzoltan: no worry, I know it's a misunderstanding, please reread the email on the degraded landing on what we try to do :) * bzoltan goes to the archive room to read <xnox> bzoltan: ubuntu-ui-toolkit as in the ubuntu archive, ftbfs in the test-suite.... <xnox> tst_MainView33ec:265c:71fe:e26d:2682:4ec2:aaa1:99abtestLocalStorage fails for me. <Mirv> didrocks: there'd be the first two CI Train packages to be built in landing-006 (Qt 5.2). I made empty MP:s against those. <didrocks> Mirv: ensure with upstream that they are ok with being "locked" (no more landing) until 5.2 is out <Mirv> thostr_: ^ I'd be starting the Qt 5.2 PPA uploads by "locking" libqtdbustest, libqtdbusmock and hud from your packages. the whole list and ordering can be seen at http://pad.ubuntu.com/qt52-dependencies <Laney> does jenkins delete old autopkgtest results? <Laney> It seems to have 20 in its sidebar <Laney> and going to earlier ones 404s <thostr_> Mirv: should be ok as we don't have any urgent chagnes there today <Mirv> thostr_: ok, thanks <didrocks> Mirv: thostr_: don't plan on "today", I fear this is going to take until thursday at best <Mirv> true, that <thostr_> Mirv: still, should be ok * didrocks doesn't want any misunderstanding :) <didrocks> ok, great, thanks thostr_ :) <sil2100> +1 <sil2100> ;) <sil2100> I don't remember any hud fixes in the CI Train since a while anyway <Mirv> of course, the whole list of 80 packages will become locked the further it's gone through <Laney> jibel: hey, could you see my question ^ please? ;-) <jibel> Laney, IIRC it does, let me check <seb128> hum, the recent ofono update seems to be buggy <seb128> e.g https://errors.ubuntu.com/problem/8a965beab09bc5c1bc862dd24455edc0feba5bd8 <sil2100> seb128: we also noticed a lot of ofono crashes on the smoketesting dashboard <jibel> Laney, it keeps the last 20 builds. <didrocks> seb128: yeah, see CI results as well <didrocks> ricardo confirmed it as well <sil2100> seb128: Ricardo mentioned it on the ML as well <seb128> ranked high on the daily reports <seb128> rsalveti, ^ <sil2100> Right <seb128> k <jibel> Laney, do you think it'd be useful to keep more? <sil2100> seb128: they might revert it soon I guess <didrocks> seb128: let's have the meeting, but I smell the revert for now <seb128> just pointed it while reviewing e.u.c <Laney> jibel: I want to find an old failure log to show to upstream <seb128> didrocks, sil2100: thanks <Laney> I don't know how you could guess that though without keeping everything <didrocks> popey: sil2100: coming? <popey> oh is that the time! <didrocks> it is THE time :) <didrocks> :) <sil2100> ! <thostr_> Mirv: didrocks: will actually everything using Qt be blocked? <didrocks> thostr_: yeah <jibel> Laney, it is even purged from the backups. I can increase the history a bit more but not unlimited so not sure that would have helped. <Laney> jibel: Probably not, as it could be some way back for a test that gets triggered a lot <Laney> I guess it's not possible to keep just failures <mandel> sil2100, sorry I was out, the branch is linked to the bug afaik <sil2100> mandel: hi! It's no longer valid, I already released it during the weekend ;) Thanks <mandel> sil2100, great! <xnox> bzoltan: unity8 devs are pinging me about ubuntu-ui-toolkit py3 getting released.... when are you going to land it? <xnox> bzoltan: and why did it not land last time around? were other branches chosen to be landed over this one? <xnox> bzoltan: not having python3 ubuntu-ui-toolkit, is at the moment blocking migrating all other tests off python2. <xnox> and it's the only blocker... <didrocks> sil2100: FYI, pushed a fix to ci train to get into account debian sync package <sil2100> Oh <sil2100> didrocks: I tested history-service and telephony-service according to the test plans <sil2100> It's ok, so I publish it <didrocks> great! <didrocks> ah, seems elopio is around :) <didrocks> sil2100: for you, I guess ^ <sil2100> Poking him right now ;) <sil2100> Although this might have been an auto-reconnect <didrocks> ok, going for a run while I still can, this evening will be busy :) <didrocks> maybe <sil2100> I'll jump out for lunch soon, but I'll leave elopio some messages in case he appears during my feast ;p <davmor2> didrocks: whats the magic line ending that I am looking for in the crash file please? <didrocks> davmor2: you need to retrace it in gdb and look for d8f_ <didrocks> d8f8* <didrocks> davmor2: you know how to get a retrace? (without debug symbols) <davmor2> didrocks: yeah I think I have the page saved <didrocks> davmor2: you don't need to have the debug symbols, as we are interested in the address :) <didrocks> on the phone: <didrocks> cd /tmp/ <didrocks> wget <crashfile> <didrocks> apport-unpack <crashfile> foo <didrocks> gdb <process_name> foo/CoreDump <didrocks> bt, eventually <didrocks> check that you have something like: #0 0x<â¦>d8f8 in ?? () <davmor2> Cannot access memory at address 0xb0b2a35c <davmor2> #0 0x40d0d8f8 in ?? () <davmor2> Cannot access memory at address 0xb0b2a35c <davmor2> #1 0x40d2d7e8 in ?? () <davmor2> I'd say that is a positive then <didrocks> hum <didrocks> yeah <didrocks> :) <didrocks> davmor2: ok, so user can get the crash <didrocks> asac: FYI ^ <didrocks> davmor2: tell us about the frequency of such issues <didrocks> (thanks!) * didrocks really goes for a run now :p <davmor2> didrocks: once as far as I can tell but I'll do a fresh install now and hit it hard and see what shows up then <didrocks> davmor2: can happen in any application, anything <didrocks> so keep a close eye :) <ogra_> didrocks, what is ofono-simd ? <didrocks> ogra_: AFAIK, it's a mock for dialer-app and other call-related tests <ogra_> are you meaning ofono-phonesim ? <didrocks> yeah, mistyped <ogra_> ok <ogra_> seems the ofono testing needs to take that into account ... i can reproduce the crashes locally as soon as i install ofono-phonesim-autostart ... just looking into it with xnox in -touch btw <didrocks> ogra_: ah, nice that you can confirm it! <xnox> didrocks: well phonesim-autostart polls list-modems and enable-modem for up-to 10 times... waiting for org.ofono to be available on the dbus, but crashes generated from list/enable-modems whilst polling are not craighoward@example.net... and phonesim job appears to be started, before it is ready. <xnox> didrocks: i believe this should fix it http://paste.ubuntu.com/7027072/ <didrocks> xnox: should be easy enough to test on a phone running all AP tests I guess ;) but yeah, seems you are right. however, the thing should wait and try to reconnect later on rather, no? <ogra_> didrocks, well, the .crash files are not from testing but from booting <ogra_> i doubt we need to run all APs to verify ... a few will be enough <ogra_> (in fact booting should be enough to see it is gone ... it creates a new one each boot) <xnox> didrocks: i'll talk to pitti, on how to do it properly. <ogra_> xnox, there is also a crash for the dial-number script <ogra_> that might need the same fix <xnox> ogra_: read it, it's unrelated to this. <didrocks> thx <ogra_> ah, k <xnox> ogra_: but, if fake ofono is not reliably up, i wouldn't trust any test results. <ogra_> yeah <xnox> didrocks: cause, at the moment those two scripts are exected in a time-out loop, if they start waiting/polling themself, we'll square the timeout (e.g. 100s instead of 10s) <didrocks> xnox: oh, indeed * ogra_ reboots with the fixes in place <ogra_> hmm <ogra_> root@ubuntu-phablet:/# ls /var/crash/ <ogra_> _usr_share_ofono_scripts_enable-modem.0.crash <ogra_> still :/ <xnox> ogra_: can you read that file? <xnox> ogra_: or paste it to me.... <ogra_> http://paste.ubuntu.com/7027127/ <ogra_> xnox, you match on service unknown <ogra_> oh, wait, my upstart job was changed ... * ogra_ reverts to original and reboots again <didrocks> ok, running now :) <didrocks> ogra_: feel free to revert the revert if you have clear evidence that we don't have any crash <ogra_> well, i'm trying here ... but the crash still occurs <xnox> ogra_: the crash you are showing, is not the crash we've seen in testing. <xnox> ogra_: any my patch specifically resolves service name unknown, not anything else. <xnox> ogra_: can you first, reproduce original crash? <xnox> ogra_: and _no_ it's not _just any_ python traceback crash from enable-modem/list-modems, but specifically <xnox> as per your original ping, org.freedesktop.DBus.Error.ServiceUnknown <ogra_> xnox, but the crash you catch there wont help with enable-modem <ogra_> https://jenkins.qa.ubuntu.com/job/trusty-touch-flo-smoke-daily/18/artifact/clientlogs/calendar_app/_usr_share_ofono_scripts_enable-modem.0.crash/*view*/ <ogra_> enable-modem always had that crash <ogra_> list-modems has the service unknown one <ogra_> i dont get a list-modem crash locally <ogra_> only the enable-modem one <xnox> ogra_: well, i want ps chain, to the enable-modem executor.... it's from phonesim upstart job? <ogra_> yes, must be, it goes away if i disable the autostart job <xnox> ogra_: or somebody who knows how to debug ofono, to figure out why it's failing to power the modem. <ogra_> its powering the fake modem (or tries to) <xnox> ogra_: can you manually give me "strace -o /tmp/foo.log -s 999 with-ofono-phonesim quiet" <xnox> ogra_: when ofono/ofono-phonesim are both stopped and all crashes are cleared. <xnox> ogra_: and any crashes that that strace will generated. <xnox> ogra_: and the console log output.... (well i could extract it from strace log, but getting it separate would be easier) <xnox> ogra_: which device you are testing this on? it works correctly on the desktop... <ogra_> xnox, flo <xnox> ogra_: and why is phonesim used on the phones, i thought we'd test against real one on the phones. <ogra_> heh, even with "quiet" with-ofono-phonesim is very noisy on the console <ogra_> xnox, http://paste.ubuntu.com/7027184/ <ogra_> http://paste.ubuntu.com/7027192/ is the console outut when running the strace <xnox> ogra_: weird, i see. so it stars up on the third try, but whilst polling it generates ofono.Errors.... so ofono is on the dbus, but the modem is not yet. <ogra_> yeah <xnox> ogra_: ok, i'll change to catch the ofono.Error then in the enable-modem script. <ogra_> yeah, sounds legit <xnox> ogra_: but when pitti is back, we need to fix it up properly. <ogra_> right <ogra_> well, it was no issue before the new ofono <xnox> ogra_: as catching errors like this, is not reliable (e.g. if order of execution changes inside ofono, etc.) <ogra_> i ssume we need to fix it on the ofiono side <xnox> ogra_: yeah, and if ofono internals change again, this may manifest itself... again... <ogra_> right <xnox> ogra_: ... or phonesim... pitti would know better. <ogra_> yup <ogra_> or awe :) <xnox> didrocks: thanks for revert.... how am i suppose to fix and test this now, my debdiffs now do not apply =( <ogra_> hmm, just changing the match doesnt help here <xnox> didrocks: and it's not python2/3 specific, so you should not have reverted that portion of patch. <ogra_> well, revert usually means roll back to last known good package <xnox> didrocks: why did you revert all of it? <ogra_> thats the rule <xnox> ogra_: didrocks: you don't know where the problem is comming from, roll back takes you into a new state, which is unknown. <ogra_> (discuss with asac if you want it different with a good reason) <ogra_> roll back is supposed to take you to the last state ... works fine if you roll back all involved packages in a change <popey> didrocks: did you want 216 or 217 dogfooded? <asac> right, you can rollback transactions. so whatever our granularity of transactions delivered into the baseline is, is what can be rolled back <asac> currently best we have for that is packages, but its not perfect for sure, but would require innovation on how we maintain/deliver things into our baseline <ogra_> http://people.canonical.com/~ogra/touch-image-stats/20140301.1.changes <ogra_> thats what caused the issue ... and thats what was rolled back (i hope) <cjwatson> Hm, this suggests to me that before landing libclick, maybe I have to land my prerequisite stack of miscellaneous cleanups first <cjwatson> Otherwise I get the whole thing rolled back if anything at all goes wrong, and my head explodes <sil2100> I might be back a bit later, my animal has additional health problems ;/ <cjwatson> Though I guess prereq cleanups => 354-line diff, libclick => 8000 or so. Maybe it's not worth it <asac> cjwatson: for complicated/non-trivial landings you could/should coordinate with LT to discuss options - including te option to accept temporary bustage. in most cases that shouldnt be the case though. one clean path that exist is to stage everything - including cleanups - in a silo, once touch baseline is not regressed, all goes in. <cjwatson> Yeah, it's CI-trainable (in theory anyway) <asac> cjwatson: LT folks can help on non-trivial landings by supporting with testing and giving you quick feedback etc. <cjwatson> So OK, I'll just do that once I'm ready, which will be probably tomorrow <asac> nice. lets try <asac> cjwatson: if you want a silo to start staging stuff, let us know <asac> also we might want to do your libclick before doing the qt5.2 <cjwatson> Just because Qt5.2 might take a while, you mean? <asac> qt5.2 might make our tests go red, which then will hit super hard to say with confidence that your work isnt regressing something hidden behind it <asac> for qt5.2 we are probably willing to take a temporary hit (even though we prep as much as we can outside) <asac> just because its awful late <cjwatson> Yeah <cjwatson> Not that I'm sure any of the autopilot tests actually hit click, but I don't really want to be implicated unnecessarily :) <asac> right <asac> it becomes too messy <ogra_> well, we do manual smoketesting too :) <ogra_> (which involves click) <asac> also who knows if suddenly stuff starts breaking somewhere <asac> right <asac> we have to learn the best rules of engagement still. e.g. what can go in safely without putting the lander accidentially into a messy friedrill etc. * asac checks latest APs <asac> for qt5.2 silo <ogra_> so what the ofono issue clearly shows is that pre-landign testing needs to be done with ofono-phonesim-autostart installed <asac> Mirv: didrocks: elopio: can we get a fresh AP run? <asac> ogra_: if ofono goes through Train, hyou could easily hook that detail in the testplan <ogra_> right <asac> ogra_: does it? :) <ogra_> well, i'm not even sure if we should have it installed at all <ogra_> it seems to fake a sim wehere we actually have a sim available <ogra_> imho it should only be used on tablets ... mako should use its existing modem <ogra_> which points to another issue ... we cant really differentiate tests between tablet/phone in this case <asac> ogra_: is tablet a subset of phone packageset? <asac> or are there packages for tablet that we dont want on phone? <ogra_> they are the same <davmor2> didrocks: so the weather app crash is the qmlscene crash so if that is the crash from ci train, and it leaves a window open that is white/grey but no process running. So crash and failed test might be due to that. <asac> ogra_: same? <asac> thought we dont have phone stuff <ogra_> asac, we only have one seed <asac> yeah in know :) <ogra_> same goes for the tuch tests ... <ogra_> we only have one autopilot-touch metapackage <asac> i was trying to figure if thats really the best case <ogra_> it isnt i guess <asac> best approach <ogra_> testing dialer-app or messaging app on a tablet without 3G is nonsense <asac> ogra_: android surely has different packageset in repo for tablet, right? <asac> or is that all the same for them as well? <asac> ogra_: yeah, if the tablet would have a 3g modem at least messaging would make sense though, but we dont <asac> so... :) <asac> Mirv: is the silo in a state that we could poke it with the AP test thingy?: <ogra_> asac, might be ... i think there is even a property that tells us if we are tablet or phone ... but currently we do not differentiate at all <ogra_> (which causes issues) <Mirv> asac: no, apparently the Friday's transitional package is not working correctly for upgrades, so the dist-upgrade method AP testing would use fails. qtbase should be updated either to revert the change or tweak it further <Mirv> (the change that added libqt5core5 back as a transitional package) <asac> Mirv: so you say we would temporarily add that transitional package? <asac> Mirv: if that keeps us moving, lets do it i guess. what would be the implications? <asac> the negatives? <Mirv> asac: no, we added it already like you asked, it's just not working 100% correctly <Mirv> so either it should be reverted similar to what should be done eventually anyway, or another upload made with the transitional package working better <asac> hmm. thought we said we dont do it after the call <asac> Mirv: just remove it. we dont want to test the apps without rebvuilding them we said <asac> assuming that the revert would help <asac> Mirv: wonder... what is th eerror we get? <Mirv> asac: no, it was made before the call since you insisted on it being done. then during the call it was decided it's not needed and that it could be reverted later <asac> ah <asac> ok <asac> didnt know the upload lready happened <Mirv> asac: yep, reverting is the easiest <asac> Mirv: will that work? <asac> Mirv: just revert qt? or also respin the stuff on top? <Mirv> asac: sure, then it's the same as qt5-beta2 where upgrades and AP testing work. the error is that libqt5core5a is tried to be installed before libqt5core5 has been replaced by the new empty package <Mirv> asac: just upload ubuntu3 version of qtbase and let it build. it does not affect the rest of the landing. <asac> Mirv: yeah. i see. so there is a conflict missing. sure that problem doesnt exist for non-transitional package case? <asac> sound we need that libqt5core5a conflicts libqt5core5 < VERSION <asac> in any case <Mirv> asac: the conflict was removed in this ubuntu2 upload so that the transitional package could exist, but it should not have been removed completely but only partially <asac> Mirv: you need to conflict < CURRENT <asac> with transitional packages <Mirv> indeed <asac> thats the practice i remember from my time :) <asac> ok go ahead and fix things ;) <asac> Mirv: byt the edgers ppa is still good? <asac> Mirv: can we have another run there? <asac> given that today mako is pretty reasonably green, the results on a fresh run would be more interestin <asac> g <asac> might be more interesting <Mirv> asac: ok! the edgers is good, we need elopio or someone else who has credentials to trigger the job <asac> what is interesting is that the unity8 AP passed today :) <asac> lol <asac> seems we either were super lucky <asac> or we improved the runtime behaviour in favour of hiding those crashes <asac> psivaa: do you know if we retried the jobs with the crsaehs today? <asac> we have very very few qmlscrene crashes i feel. <psivaa> asac: i did run dialer-app tests a couple more times <asac> suspiciously few <asac> psivaa: that had the qmlscene crash? <psivaa> asac: no i dint rerun any qmlscene crash ones <asac> thats very interesting <asac> we have like just one <psivaa> asac: yes, we noticed that in the landing meeting <asac> didrocks: any theory why things are suddenly as good as before :P <asac> rsalveti: ? <ogra_> because we all rock <ogra_> :) <asac> maybe we can stay on 5.0 after all :P <asac> lol <asac> j.k. <ogra_> asac, one issue was a stale mir socket <ogra_> thats cleaned up with one of the recent unity8 uploads <ogra_> and the other was some missing header intialization ... <ogra_> asac, that fixes it with 5.0 but according to rsalveti introduces other bugs that 5.2 fixes * ogra_ tries to remember in which upload that fix landed <asac> Mirv: if you are fine we could start a run with qt5 edgers now <davmor2> ogra_, popey: have either of you done a fresh flash to the latest image? bluetooth is off by default and you don't seem to be able to turn it on <ogra_> unity8 (7.84+14.04.20140228-0ubuntu1) <asac> elopio: what do you think? <ogra_> that had the fix for stale sockets <ogra_> davmor2, what device ? * ogra_ only has flo and manta to test <davmor2> ogra_: mako <ogra_> we got new kernels ... hmm <davmor2> ogra_: let me try on flo <ogra_> i definitely have a BT indicator here <davmor2> ogra_: so flo works from an upgrade let me install it fresh now <ogra_> but i didnt flash from scratch in a week or so <ogra_> there were no changes to bluetooth regarding mako ... <davmor2> ogra_: this was a bootstrap install I wanted a completely fresh install <ogra_> so the only thing i can imagine is the changed kernels <ogra_> davmor2, new kernels landed in 215 iirc <davmor2> flo is flashing now I'll come back to it after lunch <ogra_> davmor2, can you try if it works on 214 and breaks on 215 on mako ? <ogra_> if so, then it must be the new kernel <davmor2> ogra_: not right now but I can latter on <ogra_> k <popey> davmor2: no <bregma> sil2100, good day, I have ci-train platforms 43 and 44 in need of a launch silo if you would be so kind as to find the time <Mirv> asac: I'm fine with the AP run, I just finished a bunch of rebuilds in qt5-beta2 <asac> elopio: ^^ <asac> can you kick? <asac> Mirv: thats in edgers? <didrocks> ogra_: kicking an image, anything that you are waiting for? <ogra_> didrocks, nope <didrocks> ok, /me does :) * ogra_ sets up an alarm for 218 <Mirv> asac: yes, it's the canonical-qt5-edgers/qt5-beta2 <asac> Mirv: do you have powers to start that AP job? <asac> seems elopio is off or something <ogra_> he did a lot on the weekend by the looks of it * ogra_ sees MPs <Mirv> asac: I don't think so, and it needed some parameters to be known for the job <jibel> asac, is there any specific parameter to pass, or just build with the default? <Mirv> asac: elopio is in hangout, so yes he can then start the job :) <asac> oh right. will join in a bit <davmor2> trying to paste copied text into the notes app and I don't get the option to paste, select all appears briefly but it disappears :( <didrocks> davmor2: ok, thanks for looking! <ChrisTownsend> Has anyone noticed that a autopilot-trusty-daily_release job has been stuck for almost 6 days? http://q-jenkins.ubuntu-ci:8080/job/autopilot-trusty-daily_release/1698/ <bregma> I think they've been doing major changes to support phone testing infrastructure, that may be having some affect <bregma> I expect it will cause massive breakage to our stuff, knowing how they do not test their changes against shipping code using it <didrocks> bregma: both assign <bregma> didrocks, thanks -- is the ci-train really busy right now with qt5-related stuff or should I feel free to grab more slots? * bregma doesn't want to be greedy <didrocks> bregma: qt5.2 is one slot only. You can batch things together if you prefer, what else do you have in the pipe? <didrocks> we do have some room if that's the question :) <bregma> didrocks, a couple of low-priority OIF things <bregma> just to get them off my plate and stop eating my karmna <didrocks> bregma: you can get one slot with all the OIF, that should be fine <didrocks> we do 50% on Monday ;) <sil2100> Damned animals, never again I agree to buy an animal! <sil2100> Nothing but fluffy trouble I tell you <didrocks> sil2100: +1 <didrocks> that's exactly the reason why we don't have any <didrocks> (and if you want to move somewhere during holidaysâ¦) <sil2100> Indeed <sil2100> didrocks: anyway, elopio said he'll take a look today on weather-app tests ;) <didrocks> sil2100: great! seems that davmor2 thinks it's the qmlscene crash as well <elopio> didrocks, sil2100: I will take a look, but in a couple of hours. <elopio> is that ok? <didrocks> elopio: sounds good, I'll mention it in my landing email on the phone ML, mind answering there then? <elopio> didrocks: yes, I will. <didrocks> perfect ;) <elopio> we are also looking at a clock alarm error. <didrocks> yeah, that's one of the remaining one <didrocks> (that + camera-app that Ricardo is looking at) <didrocks> hey dbarth <davmor2> didrocks, sil2100: I'm running watch on the /var/crash dir and it popped up total 1 when the weather crash happened which was the qmlscene crash :) I'm still trying to trigger the unity8 crash but it might take a while :D <didrocks> dbarth: not sure if you noted, but Mirv added some note on your html5/cordova landing (seems he couldn't get the tests to pass) <didrocks> davmor2: ok, I guess that's good input for taking a decision when those remaining issudes are fixed. Is the crash a blocker for promotion or not? <didrocks> (like how often does it occur, and so onâ¦) <davmor2> didrocks: weather it happens with fairly consistently and I think maybe the google+ app too but I've not got to installing apps yet <didrocks> davmor2: so, maybe apps that are js greedy <davmor2> didrocks: could be <davmor2> let me install the g+ app and fire up facebook and twitter <davmor2> oh nice webapp container crash on starting facebook <davmor2> ogra_: flo now doesn't have bluetooth working after bootstrap install of 217 <ogra_> davmor2, right, what about something before 215 <ogra_> the new kernels landed there ... <davmor2> ogra_: I'll hit that in a second <ogra_> awesome <ogra_> apw, ^^^did any bluetooth option change in the recent kernels <davmor2> ogra_: was flo supported in 214? I'm assuming so but I think I was on mwc at the time <ogra_> (i would have assumed not ... but i think i better ask :) ) <ogra_> yes, 214 had working BT <davmor2> ogra_: okay I'll flash that as soon as I figure out how with the new tool I'm assuming -b 214 but I'll look <ogra_> davmor2, BT support for flo was added in 210 <ogra_> so 210-214 should be fine <ogra_> 215 then got new kernels and new android ... i suspect the issue is with one of these two <ogra_> (android rebuild is required to pick up the kernels) <davmor2> ogra_: right flashing 214 on flo now <davmor2> ogra_, asac, didrocks, popey: hmmm that's odd how is it when I adb in I can type apt-get install w3m on a normal freshish install and it install it? I thought the system was R/O <ogra_> davmor2, hmm <popey> davmor2: does it actually install? <didrocks> davmor2: on flo? not sure, I don't have one (but that doesn't work for sure on mako) <ogra_> davmor2, ls -l /userdata/.writable_image <ogra_> davmor2, smells lik a bug in the flasher ... it should remove that file <davmor2> didrocks: no on mako <didrocks> ogra_: it does remove it <davmor2> -rw-rw-rw- 1 root root 0 Feb 27 12:35 .writable_image <popey> it doesnt <popey> i updated mine today and it still has one there <didrocks> davmor2: you did upgrade or reflash? <ogra_> yeah, thats what i suspected <popey> i upgraded <didrocks> yeah, it doesn't remove on upgrade <davmor2> didrocks: fresh bootstrap flash <ogra_> phablet-flash removed it ... i guess the re-write (ubuntu-device-flash) doesnt <didrocks> flashing does <ogra_> davmor2, file a bug against ubuntu-device-flash <didrocks> ogra_: no, it's the upgrader rather <didrocks> let me test something <ogra_> didrocks, ? <davmor2> didrocks: using the following "ubuntu-device-flash --channel=devel-proposed --device=mako --bootstrap" <ogra_> why would it have anything to do with the upgrader <didrocks> davmor2: that's really weird, it should remove it under that consideration <ogra_> --bootstrap needs to remove it <didrocks> ogra_: because the flasher just uses the upgrader <ogra_> i guess its simply an oversight on sergios side <rsalveti> ogra_: the only thing that changed: http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-trusty.git;a=commitdiff;h=625dc76723341bc375d1ba277ff8f26def1c3a25;hp=77576d7e4c72425b2ba519e287d31f3c30f84081 <rsalveti> and http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-trusty.git;a=commitdiff;h=c2ecce599b3159d615c52d269de04cc95a83f409;hp=4948db412bc63cc4815bb20cba99cb65a8aacb14 <ogra_> rsalveti, hmm, then it must be the android side <ogra_> at least if davmor2 can confirm that BT actually works somewhere between 210 and 214 <davmor2> ogra_: I have bt on 214 on flo <ogra_> davmor2, thanks <ogra_> rsalveti, http://people.canonical.com/~ogra/touch-image-stats/20140301.1.changes only this and android/kernels changed <davmor2> ogra_: and I have the spinner in bt settings in the settings app too <ogra_> yeah <ogra_> not sure if it actually connects properly :) i only made sure BT suuport in general is there so the test of the indicator doesnt fail :P <rsalveti> ogra_: right, but you can check the diff for android, nothing specific to that <ogra_> weird <rsalveti> the diff is actually minimal, just your recovery patch <ogra_> yeah, that shouldnt have any influence on BT <ogra_> just sets a filesystem flag <rsalveti> ogra_: and what's up with ofono in the end? <rsalveti> ogra_: what is the proposed fix? <ogra_> rsalveti, well, ofono-phonesim causes massive dbus errors with eth new scripts <rsalveti> right, and do we know why exactly? <ogra_> rsalveti, xnox worked out a fix for one of the scripts <ogra_> not really, no <rsalveti> because once you install phonesim, the rild modem is not even active anymore <davmor2> ogra_: https://bugs.launchpad.net/ubuntu/+source/goget-ubuntu-touch/+bug/1287208 <ubot5> Ubuntu bug 1287208 in goget-ubuntu-touch (Ubuntu) "ubuntu-device-flash doesn't remove the .writable_image flag" [Undecided,New] <ogra_> it seems that phonesim-ofono starts before ofono is properly providing the dbus service <ogra_> it runs a startup script that loops over the ofono scripts ... <rsalveti> oh, maybe we're just having a race or something now <davmor2> ogra_: I will now flash 215 and confirm it dies then <ogra_> for one script that easily explains the issue and the try:/except: fix is the right solution <rsalveti> did we already build a new image after reverting ofono? <ogra_> but there is another one where it doesnt help <rsalveti> right <ogra_> rsalveti, didrocks seems to have started one, seems he forgot to notify the channel <rsalveti> alright * ogra_ has an alarm watcher running, will ping once it is done <rsalveti> ogra_: please ping awe then later today and ask him to sync with pitti <rsalveti> I'm mostly off today, focused on the camera issue <ogra_> rsalveti, pitti is on vac. :( <tvoss> rsalveti, ping <didrocks> I should use ogra_'s tag :) <ogra_> and i fear we need him to fix this <rsalveti> tvoss: hey, did you see my email? <ogra_> didrocks, yes, that helps :) <ogra_> === Image 218 DONE === <tvoss> rsalveti, yup, looking at the code right now <rsalveti> tvoss: awesome <davmor2> ogra_: 215 still has bt <davmor2> ogra_: flashing 216 now <ogra_> oh, intresting <ogra_> 216 only has a new download-manager <ogra_> so it must be 217 <ogra_> hmm <ogra_> which doesnt have any relevant changes either <ogra_> unless the UITK can have any influence here which i dont really like to belive <davmor2> ogra_: dies on 216 <ogra_> but 216 only had the download manager changes o_O * ogra_ blames mandel for breakin bluetooth then :P <davmor2> haha <ogra_> http://people.canonical.com/~ogra/touch-image-stats/20140302.changes <ogra_> thats the changelog of 216 <seb128> ogra_, there were some android uploads saturday <seb128> ogra_, would that show on the .changes? <ogra_> seb128, yes, for 214 <seb128> k <ogra_> seb128, no <ogra_> it isnt in the manifest since we dont install it inside the image <mandel> ogra_, wait what? I did nothing! :) <ogra_> :) <davmor2> mandel: in that case it's your fault for not fixing it :P <mandel> davmor2, oh, then sorry :) <ogra_> heh <davmor2> mandel: nice play along :) <ogra_> davmor2, well, whats weird is that i have BT on my upgraded devices <seb128> ogra_, how do you know on what image the new android landed? <ogra_> seb128, checking the promotion time vs the image build time <davmor2> seb128: he has all the change logs? <ogra_> or if i want it detailed i can look at the livefs build log <ogra_> davmor2, android isnt in the changelogs ... thats the issue <davmor2> ogra_: ah right <ogra_> davmor2, so looking at my flo, i'm actually on 216 here <ogra_> with BT working <seb128> ogra_, well, looking at the .changes timestamps and the time android landed in trusty, I would think it landed in http://people.canonical.com/~ogra/touch-image-stats/20140302.changes <ogra_> i think i flashed with 213 or 212 last <davmor2> ogra_: I wonder if it just effects fresh flashing <ogra_> sabshould be http://people.canonical.com/~ogra/touch-image-stats/20140301.1.changes <ogra_> seb128, ^ <ogra_> davmor2, seems so <seb128> ogra_, if you say so <ogra_> well, i think so <seb128> it might, they are less than 1 hour interval <ogra_> sigh ... i'll check the log ... not you made me uncertain :P <seb128> but android would be a better candidate to create bt issues than the download manager <ogra_> http://people.canonical.com/~ubuntu-archive/livefs-build-logs/trusty/ubuntu-touch/20140301.1/livecd-20140301.1-armhf.out has <ogra_> Get:1 http://ftpmaster.internal/ubuntu/ trusty/multiverse android all 20140228-2008-0ubuntu1 [246 MB] <ogra_> so 1.1 was the right one <ogra_> which is 215 <ogra_> seb128, i fully agree though ... but that sadly doesnt help :) <seb128> :-( <ogra_> and the android change itself is a one liner in teh recover upgrader script ... <ogra_> there were new kernels with that android build though ... <ogra_> thats why i was pointing at them earlier <ogra_> but they also only had minor config changes that cant affect BT <seb128> ogra_, http://launchpadlibrarian.net/168024412/android_20140224-0005-0ubuntu2_20140228-2008-0ubuntu1.diff.gz <ogra_> (and the breakage still starts one image later only) <seb128> ogra_, that's not a 1 liner <seb128> well, the other parts are in libhybris though <ogra_> bootable/recovery/system-image-upgrader was the one i referred to (that was mine) ... i wasnt aware of the other changes <ogra_> yeah, and hybris isnt even involved in the BT stack ... thats actually plain android, we only set the android property to start/stop it on our side <ogra_> thats really confusing <ogra_> davmor2, hmm <ogra_> http://ci.ubuntu.com/smokeng/trusty/touch/flo/216:20140302:20140301.1/6923/unity8/ ... the tests show that the Bt indicator was visible there too <ogra_> test_indicator_exists(Bluetooth) has passed <davmor2> ogra_: I'm reflashing 215 to double check <ogra_> (it fails if there is no BT indicator visible) <ogra_> so UTAH disagrees with you somehow about 216 <davmor2> ogra_: how is utah installing the image? <ogra_> flo and mako on 216 both looks fine (manta doesnt have BT and thus fails all the time on this test) <ogra_> davmor2, it flashes using phablet-flash (still i think) <ogra_> with bootstrap option afaik <ogra_> davmor2, psivaa or plars should be able to tell <davmor2> ogra_: that's a difference then <ogra_> well, a minor one <ogra_> shouldnt influence BT <davmor2> ogra_: yay so 215 has no bluetooth on this flash <davmor2> ogra_: I wonder if it is just a race issue that as normal always effects me :D <ogra_> i wonder if you just hit some kind of weird race over there <ogra_> try a few reboots <davmor2> ogra_: adb reboot and I have bluetooth now <psivaa> ogra_: davmor2: "phablet-flash ubuntu-system -b --channel trusty-proposed" is what we use <davmor2> ogra_: looks like a race <davmor2> psivaa: thanks <plars> davmor2, ogra_: utah isn't installing the image per se, in fact utah doesn't even enter into the picture with installation really. There's a wrapper script called provision which calls phablet-flash (soon to use ubuntu-device-flash), phablet-network, phablet-config, phablet-click-test-setup, etc <awe> didrocks, I saw rsalveti's email from overnight. Do you need my help debugging the ofono related problems? Sounds like they were isolated to dialer AP tests only? <davmor2> plars: thanks <ogra_> davmor2, yay <didrocks> awe: hey, yeah, unfortunately, we saw that ofono-phonesimd wasn't uninstalled after the tests run (which is another bug and for doanac` :)). xnox is working on the fix I guess, maybe check with him? <ogra_> awe, xnox and i did much of the debugging already <didrocks> awe: I reverted, so no stress/hurry to get it fixed, just do when you have time <davmor2> ogra_: so I wonder if it is again an issue with ubuntu-device-flash and/or the way it is flashing? <ogra_> awe, i fear we need pitti for a proper fix <awe> ogra_, agreed... I've never actually touched the ofono-phonesim package <awe> ogra_, didrocks, was the ofono package reverted too? <ogra_> awe, ofono-phonesim calls a pretty evil script on autostart that loops over dbus until the scripts start to work ... not sure why the py3 port causes these dbus errors now <ogra_> awe, yes <awe> ok <didrocks> awe: I just reverted ofono, it was the only one changing <ogra_> awe, the ofono source package was reverted <ogra_> which reverts the scripts as well ... coming from the same source <awe> OK, so basically the py3 change sent the ofono-phonesim package into a tailspin <awe> arrggg <ogra_> http://people.canonical.com/~ogra/touch-image-stats/20140303.1.changes <ogra_> thast the changelog of the last image ... all reverted <awe> k <ogra_> not sure where xnox went ... he was pretty angry about the revert <doanac`> didrocks: i think psivaa has a fix for that issue that can be merged now. i approved his MP a few minutes ago <didrocks> doanac`: excellent, something with autoremove and --purge? <didrocks> doanac`: so next image will (not the one running tests) will have thefix? <psivaa> didrocks: yes, i just pushed it but the tests have already started <doanac`> didrocks: that's exactly what psivaa changed <didrocks> great :) <didrocks> thanks guys! <didrocks> awe: rsalveti: at least, without that ofono issue, we wouldn't have found that infra bug, so in some way, was positive :) <awe> didrocks, always the optimist! ;D <davmor2> ogra_: hmmm so any idea where I should file a bug if it is just a race condition? <didrocks> awe: hum, you don't talk to people knowing me well that often then ;) <didrocks> awe: people call me the pessimist rather :p <awe> well maybe they're wrong. ;D <didrocks> ahah, I keep telling "I'm pessimistic, but most of the time, in the end, it's what happen" ;) <ogra_> davmor2, bluetooth-touch <davmor2> ogra_: ta <popey> ogra_: is 218 released? my mako
{'channel': 'ubuntu-ci-eng', 'month': 3}