#launchpad-meeting 2006-08-01 <SteveA> About forms... <SteveA> the next thing to do is to list all the stuff we need to think about <SteveA> we have <SteveA> - an email from mpt <SteveA> - some requirements from the landscape guys <SteveA> - http://furius.ca/atocha/ <SteveA> - zope.formlib <SteveA> - various specs and bugs from launchpad <SteveA> - Gary Poster's recent refactoring of widgets <SteveA> - The concept of whole-form server-side validation via ajax #launchpad-meeting 2006-08-02 <sivang> SteveA: mornin <sivang> SteveA: you probably know about this, ut I'll paste just in case - http://www.zope.org/Members/tseaver/Zelenium/Zelenium-0.8/README.txt <sivang> SteveA: (re our discussions about integrating ajax and pagetests) <SteveA> hi sivan <SteveA> thanks. cool, isn't it? <sivang> SteveA: very much, although I wonder why I can't just "record" the tests through the selenium browser control rather then the instructions there with tcpwatch. I recall from what I played with it, that is has the recording facility from the selenium web page. <sivang> s/web page/web interface/ #launchpad-meeting 2006-08-03 <SteveA> OMG a pony! <ddaa> SteveA: jamesh: lifeless: meeting about to start <ddaa> once mpool finds the way <ddaa> Meeting starts <ddaa> All things bazaar, yadda yadda yadd <ddaa> * roll call <ddaa> * production status <ddaa> Current focus <ddaa> * Smart server <ddaa> * SFTP advertising (jamesh) <ddaa> * vcs-import knits (ddaa) <ddaa> * supermirror branch browser <ddaa> * private branches <ddaa> * dyson <ddaa> * empty hosted branches <ddaa> Usual end <ddaa> * other meeting actions <ddaa> * critical bugs <ddaa> * pending sysadmin tasks <ddaa> * any other business <ddaa> https://launchpad.canonical.com/BazaarMeetingAgenda <ddaa> * Roll call <ddaa> SteveA is on a pony <ddaa> I'm here <SteveA> hi <jamesh> hi <SteveA> what's the time limit on this meeting/ <SteveA> ? <ddaa> Normally until 11:45 <ddaa> lifeless: pingus <ddaa> Let's move forward. <ddaa> Production status <ddaa> https://launchpad.net/products/launchpad-bazaar/+bug/53825 <ddaa> Still out there <ddaa> Talked with lifeless about how to fix it and updated bug description <ddaa> Would be nice for somebody to do the BzrError -> oops thing soon * ddaa looks at jamesh <SteveA> jamesh will be kinda occupied for the next 2 weeks <jamesh> we need to finish off the plan for doing error reporting from scripts <ddaa> Okay, then I'm happy to keep on spamming launchpad-error-reports for the time being <SteveA> jamesh: that can go on the agenda for next week's infra meeting <jamesh> there is a spec on the wiki about some of the work that needs to be done <mpool> ddaa: lifeless is ill, gives his apologies <jamesh> we should probably move on though, given that we aren't discussing details <ddaa> Last time I spoke to him, he was going to a birthday party ;) <ddaa> * sftp hosting advert <SteveA> birthday parties can do that to you <ddaa> jamesh still supposed to blog about team branches <ddaa> I take it nothing will happen there in the near future unless somebody else gets to do it <jamesh> I've written part of the article. Will post it to the launchpad list when I've fleshed it out <ddaa> Cool <ddaa> Looking forward to seeing that this week <ddaa> * vcs-imports knits <ddaa> Still waiting for spec review from lifeless and SteveA <ddaa> * supermirror branch browser <ddaa> spiv said it was focusing on that last week and should give us a delivery estimate soon, but is on leave <ddaa> * private branches <ddaa> I think there was no progress there <ddaa> still waiting for lifeless reply on spec review <ddaa> * dyson <ddaa> last week's action: jamesh to talk to sabdfl to loosen DB constraint, re grass bug. <ddaa> https://launchpad.net/products/launchpad/+bug/53698 <jamesh> I talked with him and added a comment to the bug <jamesh> I also talked with Keybuk about whether there was a particular use case for recursively checking for matching tarballs <ddaa> I think that's quite a useful thing to do <ddaa> but maybe not by default <ddaa> Is that on somebody's business for the next couple of weeks? <jamesh> yep. In the grass case it probably isn't desirable, but it is for the other examples you posted <mpool> what is grass (in this context)? <jamesh> mpool: a GIS package <jamesh> geographical information systems <ddaa> mpool: see the bug, it was just a package with weird release names <mpool> ok <ddaa> that tripped a db integrity check <SteveA> from "Ge" the god of the earth, and "graphical" to do with drawings. <ddaa> So, it looks like nobody will be working on that in the near future. <ddaa> * empty hosted branches <ddaa> there's a bug in bzr about pushing to an existing directory and overwrite contents <ddaa> mpool: are you tracking that? <mpool> ddaa: i'm aware of it <jamesh> is this the case where we get an empty branch dir with no .bzr subdir <ddaa> I had some discussion with john in the bug, but I do not have the reference handp <jamesh> and bzr won't push to it? <ddaa> jamesh: yes <mpool> right <mpool> do you think it's a high priority? <ddaa> Yes, not critical, but pretty important. <ddaa> Since it's very easy to get into a situation where you get a busted bzr branch on the sftp server <ddaa> * Other meeting actions <ddaa> well, this part of the agenda is out of date <ddaa> mpool: you said you planned to do some nice spec braindumping <ddaa> branch viewing, reviews, email subscription, and so on <mpool> ddaa: most of that time has gone into supermirror-smartserver, and the svn roundtripping thing <mpool> i'll upload the smartserver spec <ddaa> Well spent time. <mpool> this is more of a continuing task for me than a particular action <ddaa> ack, will drop the action reminder from the agenda <mpool> we should talk about roundtripping after the formal completion of this meeting, perhaps verbally <ddaa> Time permitting, this meeting is followed by the launchpad weekly meeting <ddaa> * Critical bugs <ddaa> I started work on https://launchpad.net/products/launchpad-bazaar/+bug/37897 <ddaa> Plan to fix by making importd upload and retrieve cscvs source branches on a remote sftp <ddaa> That change will also enable a number of other nice things in the medium and long term <ddaa> And it's needed for correctness because the failure mode described in bug 37897 has changed with importd-bzr-native and I did not think of that in advance <ddaa> I had preimpl call with jamesh about that before meeting <ddaa> Will abuse bzrlib's SFTPTransport <mpool> (ok the empty-directory bug is https://launchpad.net/products/bzr/+bug/30576, have marked it for 0.10) <ddaa> and use that as a paramiko wrapper <ddaa> bzrlib.transport.get_transport is the best thing since sliced bananas methink <jamesh> I wouldn't describe it as an abuse <ddaa> Ok, creative reuse. <ddaa> I may get around doing https://launchpad.net/products/launchpad-bazaar/+bug/51130 soon <ddaa> I'm tired of delegating, and I want some fun too. <ddaa> * Pending sysadmin tasks <ddaa> Anybody? <mpool> sysadmins have been beautiful <mpool> we're getting a dedicated little machine for benchmarks <SteveA> cool <ddaa> Yeah, elmo got a haircut and a shave, he's cute like that. <ddaa> mpool: are we getting a win32 machine for running regression tests? <mpool> that would be nice too but i haven't asked for it yet <mpool> are you interested in setting it up? <mpool> hey, you could use buildbot? :-) <ddaa> Yeah buildbot is a nice tool for that. <ddaa> It's what it was designed for. <ddaa> But I'm entirely unwilling to touch a win32 machine <ddaa> j-a-meinel seems the most familiar with that part of bzr <SteveA> off topic <ddaa> right <SteveA> time is short <ddaa> * Other business <SteveA> and so is joe pesci <ddaa> SteveA: how's our new hire? <SteveA> things are progressing well <SteveA> we're scheduling further interviews with martin and with me <mpool> ddaa: we will probably hire not a bzr community advocate, but rather one person across lp and bzr <ddaa> Makes mucho sense. <mpool> so, readjusting the role and criteria for that <ddaa> but I was specifically interested in the fate of the prospective new launchpad-bazaar guy <ddaa> Anybody has other business to discuss? <SteveA> I want to confirm a couple of things <SteveA> get an idea of whether they have happened, or when they might happen <SteveA> is that okay ddaa? <SteveA> how long is the meeting left to run? <ddaa> Absolutely, I'm done with the agenda. <ddaa> We have as much time as you are confortable with. <ddaa> Nominally, 11 mins left. <SteveA> ok <SteveA> 1. bzr native <SteveA> is it in RF? <ddaa> Nah, the last bit is/was waiting for review by jamesh <SteveA> ok <SteveA> 2. python import. when can we do it? <jamesh> ddaa: I'll try to get to it soon. pending-reviews also lists your david/cscvs/bzr-native branch as not merged <ddaa> I'm not too hot about doing the python import before we have support for svn renames. <SteveA> why? <ddaa> because it's high profile, and we do not want to give the impression that bzr does not do renames <ddaa> and because it's not something we can fix post hoc without breaking compatibility <mpool> ddaa: this is the kind of thing i was getting into in my mail <mpool> about the priorities for various types of work <ddaa> well, we do do it post hoc without breaking compat <SteveA> ok <SteveA> getting python imported is an important goal for me <ddaa> but what we have already converted will have bad quality <mpool> ddaa: is "we do not want to give the impression that bzr does not do renames" really the real reason? <SteveA> so I'd like a bug opened for that, and a description in there of the things we should do first to enable that <mpool> i mean, "bzr does not convert svn renames" is a much less serious shortcoming <ddaa> jamesh: cscvs/bzr-native needs a fix, thanks for reminding me <ddaa> mpool: SteveA: if everybody is happy with running a python import without renames, I can start it today <mpool> well, is everyone happy with it? <SteveA> I don't understand the thing about renames <ddaa> Will mail you two about it. <ddaa> It's not a difficult decision to make, <SteveA> ok <jamesh> SteveA: if a file gets renamed in SVN, when it gets imported, it will look like an add and a delete, which will affect annotate info, etc <SteveA> so that is all that is holding up python <ddaa> I could find plenty of bad reasons for postponing it, but nothing really serious <jamesh> there aren't that many renames, so it probably won't be visible most of the time <SteveA> I don't think python do a lot of renaming on the mainline <SteveA> so just doit <mpool> i agree <ddaa> okay <ddaa> ACTION: ddaa to import python <SteveA> error reporting <SteveA> 3. error reporting <SteveA> (from buildbot) is this a problem at present? <ddaa> yes <ddaa> I just do not know how to get buildbot to tell launchpad about errors <SteveA> what is the 1 sentence summary of the nature of the problem? <SteveA> why is it important? <SteveA> what user or admin workflow does it negatively effect? <ddaa> Because I want to do something that uses: https://sodium.ubuntu.com/~jamesh/pending-reviews/david/launchpad/importd-ng/full-diff <SteveA> um, affect <SteveA> or whichever <ddaa> Overall, the problem is that users do not see errors. <SteveA> ok <SteveA> so, user adds their upstream svn in launchpad <ddaa> And that it's hard for admin to get synthetic info, though jamesh script can help punctually <SteveA> and then nothing happens <SteveA> because there is an error <SteveA> and the user never sees that there is a problem <SteveA> is that so? <ddaa> SteveA: nothing happens because of the requirement of operator intervention <ddaa> When the operator reload roomba, the import gets tested <SteveA> what is roomba? <ddaa> and if it fails, it gets marked as "test failed" and the use does not know what failed <SteveA> please can we rename things to have less obscure names? <ddaa> SteveA: importd autotest system <mpool> I second that <SteveA> let us call it "importd autotest" <ddaa> that was thoroughly explained in the importd documentation I wrote a few months ago <SteveA> yes <mpool> would it not be possible to have buildbot write to a plain old log file, and just look through that? <ddaa> also, when an import that's syncing starts failing <SteveA> and antidisestablishmentarianism is thoroughly explained in Hobbe's tract of 1643 <SteveA> but that doesn't mean it is a good name for something <ddaa> the only way for the user to notice is to see that the branch has gone out of sync <ddaa> Sorry no way to make a one sentence summary of the issues with the import process. <SteveA> ddaa: by "that" do you mean "the thing called roomba"? <SteveA> or do you mean "the problem with error reporting" <SteveA> <ddaa> that was thoroughly explained in the importd documentation I wrote a few months ago <SteveA> beware of using pronouns on irc <ddaa> that = "an import that's syncing starts failing" <SteveA> much possibility of misunderstanding <SteveA> I see <ddaa> an import that's syncing is run by hoover BTW <SteveA> I'm sorry. I thought you were saying: <SteveA> "roomba is documented somewhere, so it is a good enough name" <mpool> can we back up for a sec <mpool> is the problem that "the system won't automatically tell users what's wrong" <mpool> or "it's too hard for david to find out what's wrong?" <ddaa> yes, but not only <mpool> or a bit of both? <ddaa> it's easy for me to find what's wrong usually, at least when the answer is easy <ddaa> the problem is that "the system does tell users that something is wrong" <ddaa> and that "the system relies on ddaa to babysit it so new user entered data turns into an import" <mpool> ok.. <mpool> and would it be enough to just tell the user broadly what happened <ddaa> The hard be for me is to collate failures to get a sense of what are the important problems. <mpool> like a one-sentence description <ddaa> Nope. <mpool> "failed because of an error while converitng", "couldn't contact server", etc <mpool> what kind of thing do you want to tell them? <mpool> actually we're running out of time <mpool> should we continue or defer it? <ddaa> Please ask me on ML <mpool> ok <mpool> steve, anything else <ddaa> that's a complicated issue and I would have to do some background research to give good answers <ddaa> Meeting closed then <ddaa> Need to take a rest break before the launchpad meeting. <mpool> sure #launchpad-meeting 2006-08-07 <spiv> ddaa: meeting time? <ddaa> spiv: yes <mpool> here? <ddaa> SteveA: ping <spiv> BazaarMeetingAgenda says here. <ddaa> lifeless: ping <lifeless> gnip <SteveA> hi <ddaa> jamesh should arrive soon <ddaa> Meeting starts <ddaa> Agenda <ddaa> * roll call <ddaa> * production status <ddaa> * Smart server <ddaa> * SFTP advertising <ddaa> * vcs-import knits <ddaa> * supermirror branch browser <ddaa> * private branches <ddaa> * dyson <ddaa> * empty hosted branches <ddaa> * Python import <ddaa> * critical bugs <ddaa> * pending sysadmin tasks <ddaa> * any other business? <ddaa> ----- <ddaa> * Roll call <ddaa> I'm obviously here <mpool> here <ddaa> and all participants too, I see <ddaa> * Production status <ddaa> Nothing new. <ddaa> https://launchpad.net/products/launchpad-bazaar/+bug/53825 <ddaa> Still spamming launchpad-error-reports. No short term fix expected. <ddaa> What was the output of the launchpad-infra meeting about oops-from-scripts? <lifeless> jamesh: ? <jamesh> lifeless: yeah? <lifeless> ^^ do you know :) <jamesh> ddaa: we haven't discussed it yet <ddaa> Okay, happy to leave it that way, eventually kiko will grow tired of the spam, but we are not there yet. <jamesh> I'll be discussing some oops stuff with matsubara today, but I don't know if we'll get to oops-from-scripts <ddaa> * Smart server <ddaa> mpool, spiv: report? <mpool> no substantial progress this week <ddaa> did not ask last week <mpool> spiv, we should meet this week to pair on it, then i'd like you to do more, if possible <spiv> mpool: tomorrow? <lifeless> works for me <mpool> ddaa: the status is that it's still trying to get all branch/repo tests to pass <mpool> tomorrow is ok with me <ddaa> Okay, I'll ask again next week. <ddaa> * SFTP advertising <ddaa> jamesh posted draft on launchpad@, I commented, and expect a second draft. <ddaa> I would like some feedback from the other guys (mpool, lifeless, spiv) if you have the time. Anybody wants to make a commitment? <jamesh> I'll post a second draft once I've addressed the problems ddaa mentioned <lifeless> no committment, sorry.I've followed the email discussion and it sounds like it will be good <ddaa> mpool: spiv: ? <spiv> I'm happy to read over the second draft. I didn't pay close attention to the first one as I was trying to get through the flood of mail that built up during my week of leave :) <ddaa> * vcs-imports knits <ddaa> Scream if you want that done this month, as I have enough things to keep me busy until leave (17th to 25th this month). <mpool> draft looked reasonable <mpool> ddaa: what are you working on til then? <lifeless> It would be nice to do it this month. Depending. <ddaa> mpool: I'll tell later, it's on the agenda :) <ddaa> Okay, talk again about it at the end of the meeting. <lifeless> I think weave branches are now into bad publicity area. <lifeless> right, later <ddaa> * Supermirror branch browser <ddaa> spiv: what's your delivery estimate? (you promised one a couple of weeks ago). <ddaa> spiv: you look a bit overloaded with bzr stuff... <spiv> ddaa: Yeah, I've been switching to focusing heavily on smart server for that moment, so I don't intend to work on the branch browser until that's done. <ddaa> Okay, if nobody disagrees, I'll drop the branch browser from the agenda until further notice. <ddaa> Apparently, nobody disagrees. <ddaa> * Private branches <ddaa> lifeless to reply to review comments. Unless somebody disagree, I'll drop that from the agenda. <lifeless> fine <ddaa> * Dyson <ddaa> Progress on that appears stalled. Unless somebody disagree, I'll drop that from the agenda. <lifeless> please keep it <lifeless> jamesh: that is currently with you AIUI <ddaa> Okay. Will keep nagging people about Dyson. <lifeless> jamesh: is that your understanding? <SteveA> The branch browser is important to me for dealing with bad "trivial" landings on launchpad development. <ddaa> mh? <SteveA> If spiv can't do it over the next 2 weeks, I'd like to get someone else to do it <jamesh> lifeless: I got the input from Mark and Scott about it <spiv> SteveA: The task of setting it up internally is considerably smaller than the larger task. <lifeless> SteveA: we can deploy a branch browser for launchpad separately <ddaa> SteveA: isn't that something requiring private branches first? <jamesh> lifeless: I'm not exactly sure how we want to proceed <lifeless> SteveA: thats trivial <spiv> ddaa: not involving douglassanchez@example.org. <lifeless> jamesh: ok, after this meeting, lets talk in #launchpad and get you unblocked. <jamesh> lifeless: I probably won't have time to work on it this week either way. <lifeless> SteveA: I'm completley happy for you to get someone else to do the lp browser instance. If you have someone who can do one for the supermirror too, that would be great. <SteveA> if it is trivial, can ew get it done this week? <ddaa> spiv: SteveA: okay but that meeting item was about the launchpad branch browser. Having a non-launchpad one for rocketfuel is an infra-team thing AIUI. <SteveA> ok <SteveA> ddaa: "Dyson" <SteveA> I forget what that is again <lifeless> SteveA: I will get a core instance up and running and email rt for admin foo, after that its up to you <lifeless> SteveA: upstream release tarball scanner <SteveA> I asked last meeting for more descriptive names for "Dyson" and "Roomba" <SteveA> and mpool seconded <ddaa> SteveA: creating product releases automatically off tarballs found on the internet. <spiv> ddaa: Well, having the internal one was a stepping stone to the larger one. <lifeless> yes, dyson was scotts invention. roomba as mine <ddaa> roomba -> soon-eradicated-importd-autotester <lifeless> spiv: overengineering IMO. one sec <SteveA> jamesh has offered to do the launchpad browser <SteveA> that issue is closed now, thanks <ddaa> dyson -> tarball-spider ? <SteveA> importd-autotester adn tarball-spider <SteveA> fine <ddaa> Moving on. <ddaa> * Empty hosted branches <ddaa> It occured to me that fixing https://launchpad.net/products/bzr/+bug/30576 (bzr push to an extant non-branch directory fails horribly) <ddaa> was only half of the fix. The other half would be fixing <ddaa> https://launchpad.net/products/launchpad-bazaar/+bug/51130 (cannot use +admin on a branch I own) <ddaa> and https://launchpad.net/products/launchpad-bazaar/+bug/34540 (cannot delete a branch), <ddaa> I plan to fix 51130 before leave. <ddaa> That said, I'll drop this item from the agenda. <ddaa> * Python import <ddaa> bad news <ddaa> Keeps failing in mid-course with a pysvn._pysvn.ClientError on PROPFIND "Could not resolve hostname `svn.python.org': Host not found (http://svn.python.org)". <ddaa> It's not clear what's the cause of the problem, but I guess fixing it would involve: <ddaa> - either import from svn dump (in importd) <ddaa> - or replacing uses of pysvn by python-subversion in the critical sections (in cscvs), I expect that would give us better control on network usage. <ddaa> I would like to have this sort of change done _after_ removing Arch support from the corresponding component. <ddaa> In other words, we're not going to have it this month unless it's made a top priority. <mpool> i'd be a bit unhappy to have it deferred so long <ddaa> In still other words, our svn import stuff sucks badly. <mpool> i think you should consider it a key test case <ddaa> *sigh* test case fails, cscvs is not robust, old news <ddaa> Though, fixing that in importd could be done without having to deal with Arch <mpool> lifeless: what's your opinion? <lifeless> work on concrete user visible goals <lifeless> if there is interest in python in bzr-svn copy, then work on making it work <ddaa> since that would be part of the "set up source tree for cscvs" code, which is target-independent <lifeless> I think diagnosing the cause is important too <lifeless> python-subversion was a PITA to work with 18 months ago <lifeless> if it has improved, great. <mpool> ddaa: so it always failes with "host not found"? <mpool> how odd <ddaa> at random points in the conversion <mpool> ok, we should take this to mail... <ddaa> not at the start <ddaa> want me to write email about it? <ddaa> okay, will do <ddaa> * Critical bugs <ddaa> No concrete change. <ddaa> - Cannot set branch associated to a product series. https://launchpad.net/products/launchpad-bazaar/+bug/31308 <ddaa> lifeless to write spec <ddaa> - renaming project, product or series breaks vcs imports. https://launchpad.net/products/launchpad-bazaar/+bug/37897 <ddaa> Have design and done preimpl call. My current top priority. Still expecting lifeless to brain dump about uploading/downloading cscvs source trees on remote servers. <ddaa> - cannot use +admin on a branch I own. https://launchpad.net/products/launchpad-bazaar/+bug/51130 <ddaa> I plan to fix that before leave as part of ongoing plan to have fun improving user-visible stuff. <ddaa> mpool: that answers your earlier question about my current coding focus <ddaa> lifeless: please give me that braindump, I value your input on that design. <lifeless> ok <mpool> ddaa: ok <mpool> (can i retitle that? bug 51130) <ddaa> yeah, the title has gone wrong <mpool> done <ddaa> the expected fix is to delete +admin and move its contents to +edit <ddaa> unless I change my mind <lifeless> SteveA: browser is up and running <ddaa> * Pending sysadmin tasks <ddaa> * Any other business? <SteveA> lifeless: cool! URL please <lifeless> on sodium, http://localhost:8088/ <SteveA> so, RT needed to expose externally <lifeless> this is a minimal config, we can add multiple branches, etc in the future. <spiv> lifeless: can't you run it as a cgi out of your homedir? <lifeless> I'd like you to RT it <lifeless> spiv: use case - trivial - done. fancy-pants-high-performance can wait for the supermirror code alterations <mpool> just to check something from before - jamesh will be working on the branch viewer? <ddaa> * vcs-imports knits, should that be set as a top priority for that month? <spiv> lifeless: running as a cgi is hardly a performance booster ;) But ok. <jamesh> was going to work on the one for launchpad's branches (which lifeless seems to have up and running). Not sure about the supermirror one <lifeless> SteveA: I'd like you to RT it please, choose where you would like it to exposed etc. I've put it up immediately because that fulfils your requirements immediately. <SteveA> lifeless: sure. i'll do that. ta <lifeless> np <lifeless> we aim to please <lifeless> its running as the PQM user. at this point if sodium is rebooted it will go away. <lifeless> we can refactor the implementation later to be better as needed. <jamesh> lifeless: so that's just the main rocketfuel branch. I think we want other branches too (I suppose that can be done later) <ddaa> jamesh: looks like the one for rocketfuel was grabbed by lifeless :) <lifeless> well, I did mean it was trivial. <ddaa> Focus, please. <lifeless> SteveA: please file bugs on the launchpad-development-infrastructure product with things you want changed in the browser - branches to show etc. * lifeless winds that up <ddaa> So, SteveA request was granted, take deployment details on after meeting. <lifeless> knits - IMO its a priority <ddaa> Is that more important than fixing 51130? <mpool> converting existing weave branches to knits? <mpool> why is it difficult? <lifeless> theres 500 to do <lifeless> each has to be done in two places <ddaa> some of them stupidly large <ddaa> do not want to do that manually <ddaa> so need to be implemented as transition automation in importd <ddaa> so, not difficult per see, just a significant coding task <ddaa> Waiting for opinion on whether I should drop one of my current tasks for that, or on somebody to provide extra workforce. <lifeless> I'd like to get it done and out of the way <lifeless> mpool: your thoughts? <mpool> well, i can't judge how much work it woul dbe <mpool> i mean, even if there are many of them surely it should be just a couple dozen lines of shell at most? <ddaa> mpool: we do not want shell <mpool> but i would not want someone actually using the weave branches - they will be awfully slow over http <ddaa> we want that in importd because it's a shared resource issue and format upgrade is a recurrent task <lifeless> mpool: unless you shut down the entire system while you do it you need to lock/unlock the branch being converted at the scheduler level <lifeless> mpool: you also need urls from launchpad <ddaa> and doing that sort of thing in shell is a LARGE pita because it needs to be distributed on many systems. <mpool> i'm not actually suggesting shell, just saying that what actually runs is conceptually small <mpool> so i wonder if there is any lazier way to do it? <ddaa> BTW, fixing 51130 would make it easy to temporarily shut down importd-autotest with importd-production so we get maximal horsepower for the conversion. <mpool> e.g. i'd rather shut down imports for 24-48 hours than have david spend a week writing this <ddaa> s/shut down/merge <ddaa> mpool: who do you think would spend 48 setting up, testing, and babysitting the upgrade hack? <lifeless> see I dont understand why its more than 2 lines of python <lifeless> but thats a different discussion. lets have that post meeting <lifeless> mpool: lets talk priorities now, not costs <ddaa> does the spec I wrote look like more than two lines of python? <lifeless> lets talk after the meeting <ddaa> Okay, so issue still open. <lifeless> the discussion right now should be priorities IMO <mpool> ok, so <ddaa> And meeting time is over. <lifeless> mpool: I think eliminating weaves is importnt <mpool> after the meeting, dda, robert, mbp to talk about ddaa's suffering <ddaa> ta, it's not whining, it's good use of programmer's time <ddaa> Meeting over <ddaa> I'm out for a small break. <mpool> back in 10 say? <ddaa> mpool: I'm back <mpool> back <ddaa> https://launchpad.canonical.com/VcsImportsKnitsUpgrade <ddaa> There appears to be a single import in STOPPED mode, and I'm not sure it should not be syncing instead <ddaa> so we can drop the provisions for STOPPED in that spec <mpool> what does that mean? <mpool> i mean, what does "stopped" mean? <ddaa> it means "was once in sync, is published, but is no longer updated" <ddaa> mainly so we do not get distracted by errors of things that we know will fail for reasons we cannot fix (like the repository went away) <ddaa> that comes from there https://launchpad.canonical.com/SourceSourceRefactoring <ddaa> look at the bottom of the page <ddaa> also look for ImportStatus in canonical/lp/dbschema.py <mpool> i see <mpool> and the public server pulls from the internal http server <ddaa> yes <mpool> but if the branch already exists on the public server, how will the upgrade propagate? <mpool> pull does not do an upgrade <mpool> iirc <ddaa> https://launchpad.canonical.com/VcsImportsKnitsUpgrade <lifeless> I will comment, but I have to finish a meeting first <mpool> oh i see, right <mpool> it's not doing a regular pull <ddaa> if an upgrade is needed, importd will 1. pull 2. upgrade locally 3. recursive delete remote branch 4. push before running cscvs <mpool> " <mpool> The branch-puller preserves changes in branch format, so format changes on the vcs-imports internal publication server will be propagated to http://bazaar.launchpad.net." <ddaa> mpool: that issue has been addressed already <mpool> right but how does the branch-puller do that? <ddaa> if the branch puller sees that the source and the target have different formats (in any way), it will delete the target and do a branch from scratch. <mpool> ok <mpool> so the branch will just disappear from b.lp.n until the upgrade is pulled across? <mpool> i guess that's ok <ddaa> yes <ddaa> maybe there's some special provision to reduce the window of loss of service, but it's not really important <spiv> After all, one of the nice things about distributed version control is that temporary outages aren't totally disruptive. <ddaa> so, it might very well be that the required changes to importd are small <ddaa> but I'm not fluent in bzrlib <ddaa> and there tends to be annoying integration issues in importd <ddaa> so it's not a low risk task <ddaa> I'd say medium risk <mpool> i don't think the interruption is a big deal, i just wondered <mpool> one other question: presumably all of this is backed up? <ddaa> There's also the fact that the conversion is expected to be a big deal processing wise <mpool> so if something goes wrong and it starts deleting branches... <mpool> ? <ddaa> mh... <ddaa> dunno, I expect it is backed up... but access to the repositories are quite restricted <ddaa> and it's a bit hard to go _that_ wrong <ddaa> so, big deal processing wisu <ddaa> definitely not something you want to run on a single system <mpool> i think it'd be sensible to check before starting <mpool> the spec looks reasonable to me <ddaa> I do end-to-end tests with the whole system on my workstation before deployinh disruptive code like that <ddaa> There's usually a minor tweak or two needed when deploying things, but there was never a major disaster. <mpool> cool <ddaa> So, you want multiple systems running the stuff <ddaa> that turns a small one-off hack into a relatively large mess <ddaa> and format conversion is not one-off <ddaa> it's going to happen again in the future <mpool> yeah, it makes sense <mpool> so, essentially we are trying to prioritise the upgrade spec against <mpool> - unjamming the python import <mpool> - cleanups of arch code <mpool> what else? <ddaa> - critical fixes I'm working on now <mpool> ok <ddaa> I particular, I think it makes sense to do the upgrade thing after fixing https://launchpad.net/products/launchpad-bazaar/+bug/37897 <mpool> so it looks like the upgrade mechanism is largely independent of other things - e.g. removing arch dependencies won't make it much easier, or vice versa? <ddaa> because that will allow use to handle the load by putting all four importd systems (including the ones currently running autotest) temporarily in importd-production <ddaa> I'm happy to proritise that before arch cleanups. <ddaa> since it can be put into the bzr-specific code that gets the target branch <ddaa> might not be the optimal way to organise the code, but that would avoid messing up the pattern of arch/bzr testing <mpool> ok <mpool> well, let's see what robert says * ddaa realises that putting all importd in production will require some admin operation to set up ssh keys and db access, but that should not be a big deal <ddaa> lifeless can give support to the relevent requests <ddaa> And it's something I'll need eventually, since I want to remove the need for importd-autotest altoghether (we can now to that since we no longer have to deal with arch namespace issues) <lifeless> ok <lifeless> I'm back <lifeless> where are we up tp? <ddaa> need your input or prioritizing knits upgrade <lifeless> ok <lifeless> so I was going to ask <lifeless> isn't the upgrade as simple as invoking bzr upgrade via python *always* on every branch ? <lifeless> invoke it on both the local branch and the remote one, before pulling etc etc <ddaa> hu <ddaa> remote upgrade? <ddaa> it's on LAN, but still... <ddaa> if we put it down to that, it will be indeed just a couple of lines of code <ddaa> since we currently download the branch anew for every sync <ddaa> but really do not like the idea <ddaa> for example, how will "bzr upgrade" behave if the first upgrade is interrupted (process killed, loss of connection), and we try to run a second one? <ddaa> BTW, need to make a bug importd not breaking sftp locks <ddaa> it should be able to do that <lifeless> bzr upgrade takes a backup copy of data. It marks the branch as being in-upgrade <lifeless> so it will need manual recovery- but this is IMO better than the failure mode of the upgrade you proposed <lifeless> where it will have no data remotely, and no data locally (because it will be trying to pull anew) <ddaa> gn? <ddaa> remote data would only be deleted after a successful local upgrade <ddaa> oh right <lifeless> if we download the branch anew each time, my suggestion is just 'invoked bzr upgrade on the remote branch before pulling it' * ddaa wrote that spec before reading about crash-only software <lifeless> which should be trivial to implement, and result in automatic upgrades as we change the defaults <ddaa> What does importd need to do to recover from an interrupted upgrade? <lifeless> well, there are two failure types <lifeless> a) bzr bugs <lifeless> b) network glitch <ddaa> c) kernel upgrade <lifeless> for b), its - rm -rf the .bzr dir, and move .bzr.backup to .bzr <lifeless> c) counts as b) <ddaa> anh other variants of c, like importd upgrade <ddaa> yeah <lifeless> for a), you need to diagnose it and work with the bzr devs to get the upgrade to work <ddaa> how can you automatically detect b? <lifeless> IMO you should treat them all as a), because theres no magic way to tell what caused the failure <ddaa> you realise that "needs manual diagnostic" for import pretty much means "will not be fixed" <lifeless> is jamesh's log analysis stuff rolled out for importd ? <ddaa> I ran it once <lifeless> if so, then this should be marked clearly in that report <ddaa> but not again, because there were just too many failures to deal with <lifeless> and as any bug in this stuff is of critical importance to bzr, I dont agree that needs manual diagnostic means 'will not be fixed' <ddaa> and there was no easy way to classify them <mpool> ok, well <mpool> i'd like to call it a night... <lifeless> jamesh: is there something that could be easily done to make flagging failed upgrades stand out hugely in your log analyser for importd ? <lifeless> mpool: what time for spiv and I tomorrow ? <mpool> presume you're coming here? <mpool> after 10? <lifeless> I presume that too <lifeless> k <lifeless> spiv: ^? <ddaa> lifeless: does bzrlib gives a specific exception when trying to upgrade a branch that failed upgrade already (and is thus marked upgrade-in-progress)? <lifeless> ddaa: yes <ddaa> okay, that should be easy to grep for <lifeless> ddaa: UnknownFormat <mpool> what effect does an ascii del have on a spiv? i hope it doesn't kill him :-) <lifeless> ddaa: or ~ that - check bzrlib/errors.py ;) <lifeless> ddaa: ok, does this sound like a workable plan then ? <ddaa> Sounds workable. <ddaa> Then _when_ ? <lifeless> ok, cool. <lifeless> yesterday :) <ddaa> ASAP, or when we can put all our cpus on it? <lifeless> IMO ASAP. upgrade will chew through over a couple of days <lifeless> no need for massive parallisation <lifeless> it wont block all imports indefinately, because each import will block just long enough to upgrade, then do a pull from CVS <ddaa> yeah <lifeless> ok, gnight then <ddaa> once we get in the tail of the exponential distribution, (the 5 biggest branches), the other imports should be allowed to sync again <ddaa> lifeless: BTW <ddaa> jamesh appears to busy to review my last importd-bzr-native <lifeless> ddaa: not the case <ddaa> okay <lifeless> ddaa: was discussed in the review meeting, see irc logs or lp-reviews <lifeless> gnight <ddaa> nigh <ddaa> night * ddaa -> lunch <spiv> mpool, lifeless: soon after 10 sounds good to me. #launchpad-meeting 2006-08-10 * Starting logfile irclogs/launchpad-meeting.log -ChanServ(douglassanchez@example.org.)- You do not have channel operator access to [#canonical-ops] * #canonical-ops is desynced from zelazny.freenode.net at 09:21am <carlos> hi <SteveA> hi <SteveA> hi <danilos> hi <SteveA> so, stub and I were just talking about the downtime required to run the rosetta updates <danilos> yeah <carlos> SteveA: do we have info about when is launchpad less used ? <SteveA> stub tells me that with the current code, we're talking about many hours' downtime <SteveA> I have a few comments about this that we should consider <SteveA> 1. we don't want to take down all of launchpad for N hours in order to do rosetta updates <carlos> between 2 and 4 hours <SteveA> 2. is it not possible to do the updates incrementally? <SteveA> about 1, <stub> 2-4 hours per run, or 2-4 hours for both runs? <carlos> stub: per run <SteveA> if we need to do lots of rosetta database work, we can take down just rosetta, <carlos> that's on staging <carlos> and it depends on the load of the server, because sometimes is done in 2 hours and other times in 3 hours and a half <SteveA> by changing the authorization adapters to deny all rosetta edits <SteveA> and changing the "permission denied" page to say about rosetta being under maintenance <SteveA> so, that's one way of managing this <SteveA> so, the rest of launchpad (shipit, bugs, etc.) can stay running <danilos> I think that might be the best way out <SteveA> but <carlos> yeah, at least for this initial run <carlos> next time, for edgy + 1 <carlos> will be rosetta and soyuz <SteveA> let's talk about why this needs to be done all in one go <danilos> on the other hand, it will take longer that way, no? <SteveA> why can't we run it a little at a time <SteveA> with pauses in between? <SteveA> for example, one package at a time <stub> I think Steve means one package at a time with commits in between <carlos> hmm <danilos> well, it was designed in such a way that it can be run that way, since it also supports incremental updates; carlos, isn't that so? <carlos> yes, it is <carlos> not per package <carlos> but per table <carlos> doing it per package will require to add some extra code <carlos> the problem of doing it per table <carlos> is that at some point people will start seeing some inconsistent data <carlos> but I don't think they could actually break anything <danilos> well, then the per-package seems easier and safer <carlos> we don't point to edgy translations so the amount of people looking at that data would be really small <carlos> danilos: but also much more slow <SteveA> that's okay <SteveA> it's okay for it to take a week <stub> yup <SteveA> if it means no interruptions in service <danilos> well, but it will lock only a package at a time <carlos> SteveA: but it will only be useful for this two runs we need now <danilos> and each will be locked for like a really small amount of time <SteveA> if one package is locked for 30 minutes, once <SteveA> then that's fine <carlos> edgy + 1 will not need that approach, right? <stub> It will lock the tables it is inserting into as normal, but the locks will be for a much shorter period and probably unnoticible. <SteveA> 3 minutes is even better than 30 <danilos> yeah, I think it will be far less than 30 minutes <SteveA> but this is a much better approach for our service overall <stub> If it takes 30 mins to do the inserts for one package, or even 1 minute, then I don't know if this approach is doable on the live system. <carlos> oo.org would take more than 5 minutes... <carlos> it's a monster... <SteveA> so, we could do the large packages in a maintenance period <SteveA> we need to find out how long an average-sized package will take <carlos> SteveA: hmmm <SteveA> otherwise, look for a smaller granularity of commit still <carlos> I really prefer if we close rosetta <carlos> this change is going to delay edgy translations again <carlos> and I'm already getting lots of complains about it <carlos> the changes we are talking about would take more than a couple of days between code changes, testing on staging, code reviews and landing on production <carlos> + the time to deploy it <carlos> so that would be end of next week in the best case <danilos> well, this is actually true, so it all depends on the timing <SteveA> stu and i agree that we can take rosetta down <SteveA> if you guys can write some security code that <SteveA> has a launchpad.conf option <SteveA> so that when this option is set <SteveA> permission is denied to everything except for launchpad.Admins <carlos> even for the pages that are read only ? <SteveA> and to either <carlos> I mean, the ones that doesn't need to be logged in <SteveA> 1. raise a custom subclass of ForbiddenAttribute or whatever <SteveA> or 2. customize the forbidden oops page (and authorization required, or whatever) <SteveA> we need to do this for every page that allows writing to rosetta <danilos> so, carlos, what do you think about this? <SteveA> we'll test on staging <SteveA> by locking the rosetta tables <carlos> if it's just for teh pages that allow writing <SteveA> and checking that we can't lock up launchpad <SteveA> just for edit pages <carlos> the first part is quite easy <SteveA> I mean, we could do this in the forms instead <SteveA> instead of the security code <carlos> the second part... is the first time I do it so I'm not sure which option is better <SteveA> if there are only a few rosetta forms that allow updating rosetta tables <SteveA> that might be easier <carlos> there should be 4 or 5 different forms <SteveA> raise a RosettaDownForMaintenance error <danilos> well, we also need to block imports, no? <SteveA> yes <SteveA> and do an error page for RosettaDownForMaintenance <carlos> danilos: that's just a matter of disable the cron job <SteveA> we'll be making the tables read only to the launchpad user anyway <SteveA> so the database-level stuff will be okay <SteveA> but we want to give nice errors and not OOPSes <danilos> ok, just taking notes of everything <carlos> ok <danilos> so carlos <danilos> what do you want us to do? rosetta downtime, right? <SteveA> just to check something <SteveA> when we were in london together <SteveA> we talked about updating the stuff into a temporary table <SteveA> and then in a separate transaction, or separate translactions, moving them across to the real tables <SteveA> is that the approach you've taken? <danilos> mostly, I think carlos put the temp table creation into the same transaction (so it doesn't get out of sync) <SteveA> um <SteveA> that kinda obviates the point <carlos> danilos: yes, I would prefer to do the downtime <danilos> temp table is reading only, so it can probably be moved out without too much problems <danilos> carlos: ok, then lets do that as painlessly as possible <danilos> carlos: btw, didn't we get any improvements with temp tables? do we have indexes in there? <stub> So build all the inserts into tables. Then, a few rows at a time, insert some data into the production tables, delete from the temporary store, and commit. <carlos> danilos: indexes helped a bit <carlos> but not too much <carlos> SteveA, stub: I guess that it could make us to lose some translations for the migration (anyone added after the temporary table is created) <stub> carlos: That would have happened anyway, as the migration script would be using a snapshot of the database at the time the migration was started <carlos> stub: well, if rosetta is closed <danilos> stub: it gets a bit more complicated because of references and us updating table-by-table <carlos> no new translations will be added, right? <carlos> also, we should block any import from poimport script for edgy or as soon as the potemplate table is filled, entries from the queue will start to be importing <carlos> giving us conflicts with the temporary table <carlos> What problem do you see If we close Rosetta ? <carlos> I think that's doable and in the future, that feature would be interesting in case of a performance problem like the ones we had in the past so the rest of launchpad is not also affected <danilos> well, we want to go for that anyway <carlos> In that case, I think that closing rosetta two days <carlos> one per migration <carlos> is not a big problem, if we choose the right time when users are not doing many translations <SteveA> stu and i are discussing various things... <carlos> ok <SteveA> ok <SteveA> here's the plan <SteveA> stu will try the update code on a clone of production, on the production server <SteveA> turning off indexes and shit like that <SteveA> we'll get an accurate estimate of the amount of downtime <danilos> ok <SteveA> meanwhile <SteveA> we can assume it will take at least 2 hrs <carlos> SteveA: should I start working on the 'disable Rosetta' feature ? <SteveA> so you guys need to add a switch to launchpad.conf <carlos> danilos: I think I should work on it as I'm just switching tasks and you already have a lot of open things <SteveA> that causes rosetta edit views etc. to raise RosettaDownForMaintenance when the view is viewed <SteveA> as well as edited <SteveA> and an error view for RosettaDownForMaintenance <SteveA> and we need to test this on staging, by stu turning off write access to rosetta tables <carlos> sounds good to me <carlos> stub: the script should be run with '-d ubuntu -r dapper' the first time and the next one with '-d ubuntu -r edgy' <danilos> carlos: sure, be my guest :) <carlos> the first one will migrate translations from breezy to dapper and the next one will copy all translations from dapper to edgy <flacoste> hi SteveA <SteveA> hi salgado <SteveA> hi flacoste <salgado> hey! <flacoste> i think the spec you were looking for was https://launchpad.canonical.com/LaunchpadI18n <flacoste> actually, that we should have looked for * SteveA waits for stub <SteveA> https://launchpad.net/products/launchpad-support-tracker/+spec/localized-support <SteveA> https://launchpad.canonical.com/LaunchpadI18n <SteveA> first, let's talk about the overall point of LocalizeSupportRequests <salgado> I pasted that on #canonical, didn't I? <salgado> okay <SteveA> * <SteveA> Grandma Lucille, who doesn't speak English, was navigating on the web until she reached a flash page, whose content she can't see because the flash plugin is not installed. She'll then click on "Help -> Get Help Online ..." and make a support request in Launchpad. <SteveA> * <SteveA> Malba Tahan, a very nice guy who knows a lot about Ubuntu is willing to share his knowledge by answering some support requests. He goes to the Support facet of Ubuntu and look for open requests. He sees a list of requests written in one of his preferred languages, pick up a randon one in Arabic and answers it. <SteveA> <SteveA> these are use-cases from the spec <SteveA> so, we have requirements around identifying which support requests are written in a particular language <SteveA> and of allowing people to file support requests in their own language <SteveA> is that right, as overall goals? <salgado> right <SteveA> so, the support tracker needs to be able to track various things about what language the request is made in <flacoste> exactly <SteveA> and that's self-contained in the support tracker application <SteveA> I totally trust you guys to do a good job there <flacoste> that was the plan there <SteveA> what I'm concerned about is how this touches other parts of launchpad <salgado> what
{'channel': 'launchpad-meeting', 'month': 8}