#launchpad 2004-10-01 <daf> carlos: isn't that what make{MessageID,TranslationSighting} are for? <carlos> but we are not using it that way (I think) * carlos talking with lalo at jabber <carlos> should we joing the room? <daf> ok <carlos> night! <debonzi> lifeless, hi, are you around? <lifeless> no <lifeless> @ jeffs, about to leave .. * Signon time : Tue Sep 14 06:17:33 2004 * Signoff time : Mon Sep 20 06:16:34 2004 * Total uptime : 5d 23h 59m 1s <carlos> ddaa, lifeless: I sent only a merge request to pqm about 30 minutes ago, and I received a success mail, but now I got one with a failure :-? <carlos> but I did not sent a new merge request <ddaa> Maybe you did... *shrugs* <ddaa> Is rocketfuel missing something from you? <carlos> ddaa: it should not, if the first success was correct <carlos> let me check <carlos> with archzoom <ddaa> Well, if the first merge was correct _and_ complete there is no reason the second merge would fail... <carlos> ddaa: it fails because a conflict <ddaa> In a nutshell, I'd say "do not worry about the duplicate message, just make sure your stuff got merged, do the "unwedge merge gymnastic" if needed, and proceed". <carlos> but I did not change anything more <carlos> ok <ddaa> Sounds like you got a cross-merge situation... <carlos> ddaa: I did a commit, a merge request and nothing more <ddaa> Try merging from rocketfuel (in a clean tree) to be sure. <ddaa> carlos: mirror? <carlos> ddaa: it's executed as a hook with every commit <ddaa> Mh... does not sound like something that could cause a cross-merge... <carlos> I know <carlos> :-) <ddaa> No idea what's the problem. If it becomes painful, ring me again and I will have a look at your branch... <carlos> yes, my changes are at rocketfuel <carlos> so I suppose it should not be a problem... <stub> lifeless: production db upgrade still on for tomorrow? <lifeless> ye <lifeless> bah <lifeless> yes. <lifeless> midday I think will be best for me <stub> I've got an appointment - I either need morning or after 3pm <stub> Minimum I need 11:30 - 2pm free <lifeless> stub: 3-4:30 do ? <stub> lifeless: Sure <lifeless> I have to leave 4:30 sharp to go to the postgresql users group thing <stub> lifeless: I'll make sure I'm back online before 3pm <lifeless> Cool. <debonzi> spiv, ping <debonzi> spiv, is there a way to make a distinc select ? <spiv> Not at the moment. <spiv> There's some disucssion of it on the SQLObject list. <spiv> There's a workaround, though.... <spiv> distinctResults = Set(Table.select(...)) <debonzi> spiv, ok, Ill try.. thanks <spiv> (with a "from sets import Set" somewhere) <spiv> debonzi: soyuz/distros/ubuntu/bin/warty/mozilla-firefox/0.9.0-9 seems to be broken in current rocketfuel. <debonzi> spiv, I gonna take a look <spiv> debonzi: Thanks :) <debonzi> spiv, here it works.. did you rebuild the db sampledata? <spiv> Oh, no, I didn't. <spiv> My bad :) <spiv> Thanks :) <debonzi> spiv, :) no problem... try and say if you have some problem * debonzi goes lunch <spiv> debonzi: Btw, I see you fixed 2008 for me. Thanks! <debonzi> spiv, you are welcome <kiko> yo debonzi <kiko> stub, hola #launchpad 2004-10-02 <stub> kiko: morning <kiko> stub, very funny. ;) <sabdfl> stub: night! <lifeless> debonzi: what did you want yesterday? <debonzi> I had a little problem with my archive, but its already solved.. thanks <lifeless> ok <cprov> kiko: stub : here is the place :) <kiko> aha! <cprov> kiko: no, i think, why do you want displayname in lowercase ? <kiko> because of the URL section <cprov> kiko: in this case it loses its functionality <stub> cprov: Not displayname, name <kiko> I said *displayname*. <cprov> kiko: yep, ok names are suposed to be unique on distro and distrorelease, but in Person, we don't have nothing similar to use in as traverse <kiko> yes, in person it's freaking me out too -- there's a bug open though <wink> <stub> kiko: Hey - I can add it anytime :-) <kiko> stub, well, nobody seems to agree with need a login name, incredibly. <stub> I do <cprov> three lost on it <kiko> they seem to think that an email address will work for everything <cprov> kiko: stub: having you seen the auth_component for launchpad written for spiv ? <stub> We can't use displayname in many places, because it is not unique. I'll have to display the integer id or something, or flag one of the email addresses as 'primary' and display that for all the spam robots :-) <cprov> how does it works ? <kiko> no, I haven't looked at it yet. does it actually work? <kiko> stub, one sec. we've shifted to discussing Person now, right? <stub> kiko: Yup <kiko> well <cprov> kiko: I haven't seen too, just the large commit on arch at morning <kiko> stub, the issue there seems to be using the email address as a traversal string. do we want to do that, or do we want a login/nick? <stub> Even if auth is done using email address, I think we still need Person.name for traversal and display-name-when-we-need-it-to-be-unambiguous <stub> Traversal using email is evil, as email addresses fluctuate <stub> (well... so might login name, but that would be a conscious decision if we allow it) <stub> Also we will get complaints because the email addresses will be harvested by spambots <stub> Or people don't actually want them public, etc. <stub> And we have multiple email addresses per person atm, and no way to flag one as primary. * stub just realized he is running really late... <kiko> stub, well, we use distro release as a traversal string, and it can change too... <kiko> right. <stub> email address changes are forced on people. Changing distroname or login name is a choice, and people do it because they *want* those changes including the fallout to URL's. <kiko> btw, where I said "displayname should be lowercase" I mean "traversable names". <kiko> stub, well, not all email address changes are forced on people, though they may be (unemployed, server died, etc). <stub> kiko: I assumed that :-) <kiko> luckily it appears you are on a different sort of drug than I am <kiko> the kind I'm on makes me make mad statements in public <kiko> okay <kiko> well, if we want a unique name/nick.. if we want this to be painless we just add a field to the table and an editable field to edit personal info form and go on with life. <kiko> or we battle it out with steve and the gang about soyuz implying a globally unique nick/login space for OSS. :) <cprov> kiko: In that last way we'll experiment the live sensation to lost our canonical emails :) <kiko> yes, it will be thrilling to discover <stub> Ok - I have to fly. Feel free to email this discussion to launchpad@, or I will do something similar this arvo. <kiko> arvo? <stub> afternoon <stub> occa speak <kiko> that's a valid abbreviation if you're strine? * stub buggers off <lifeless> stub: ready? <stub> lifeless: ping <lifeless> hey <lifeless> so, ready ? <stub> Sure :-) <lifeless> I've tagged the launchpad & pyarch branches <lifeless> there is a last schema upate in your mail from me. <stub> Has emperor changed its domain recently? <lifeless> not that I know of <lifeless> butmaybe. <lifeless> can you the dbnazi thing on that schema update. (See how easily *I* let go ?) <stub> urgh... no... it is *not* .com.au :-P <lifeless> and then I can merge it into the production branch. <stub> debonzi patch == the one you emailed to launchpad@ ? <stub> oh... dbnazi... not debonzi * stub needs new glasses... or another drink <lifeless> both. <stub> Hopefully this will be the last time I have to watch arch crawl on this osx box - the new laptop arrived earlier :-) <lifeless> its the OS dude. <lifeless> :) <stub> Ok - its with PQM <lifeless> ok, the new production branch is launchpad--production--1.1 <stub> I guess I need a new archive for emperor (hmm... or just use scp for now...) <lifeless> new archive ? <lifeless> why ? <stub> Oh... I can just check out rocketfuel. I just can't make changes. <lifeless> right. <lifeless> which is kinda the point of production-lockdown anyway :) <lifeless> I'm thinking we might want one branch per machine-type <lifeless> production-emperor <lifeless> production-appserver <lifeless> production-buildbot <lifeless> etc <stub> I suspect that might be your call - I'm still in single thread CVS land where we just tag the trunk at certain points :-) <lifeless> I'm still dithering <lifeless> anyway, pqm is chundering away <stub> Hmm... 1.1? I thought we would be 2.0 now? <lifeless> I thought about going 1, 2, 3, decided that we have major and minor code drops <stub> (the database schema is already at path-1-11-0 <stub> (the database schema is already at patch-1-11-0 <stub> Suppose there is no real reason the numbers need to be in sync <lifeless> on, pqm luv done. did it work ? <stub> Hmm... no such version <stub> 'tla versions williamsmith@example.net/launchpad--production' only gives me 'launchpad--production--1' <lifeless> try now <lifeless> I think your merge failed <stub> got it <lifeless> bah, hang on <lifeless> ok, production--1.1.--patch-1 is now available <lifeless> which has the patch merged down. <stub> No patch-1-11-0.sql <stub> tla revisions just gives me 'base-0' <stub> lifeless: If you have to go at 4:30, we are running out of time. I can copy the patches across and run them now, or we can do this later. I'm around all evening and tomorrow if we want to reschedule. <lifeless> stub. try now.. <stub> ok - patch-1 now <stub> all there <stub> last backup is 18 minutes old <stub> Any last words? <lifeless> nope <stub> I'm going to have to kill connections, or would you rather shutdown stuff gracefully? <lifeless> doing it now <lifeless> done <stub> error on first patch - shortdesc contains nulls :-) <lifeless> hahaha <lifeless> mark is nuts thinking anyone will put sane data in two description fields <stub> Who bug #1 in there??? Its an issue dammit! <stub> ok - patches all run... I'll reenable remote access now <stub> lifeless: database all patched and live again <lifeless> ok, trying <lifeless> tla delta is a wonderful tool btw <stub> I plan to actually read the tla docs one day ;) <lifeless> PYTHONPATH=/home/launchpad/dists/launchpad/sourcecode/zope/src:/home/launchpad/dists/launchpad/lib: python2.3 \ <lifeless> runlaunchpad.py -C launchpad.conf <lifeless> /home/launchpad/dists/launchpad/lib/zope/configuration/fields.py:383: UserWarning: You did not specify an i18n translation domain for the 'title' field in /home/launchpad/dists/launchpad/lib/canonical/lp/configure.zcml <lifeless> warnings.warn( <lifeless> (oh, it makes two-commands to pick up the latest code, moving across branches, and preserving the dbhost changes we did for the connect strings. <stub> Those warnings should go in bugzilla. Do we have a zope version mismatch? <stub> I'll do it now <lifeless> no, its running the latest <lifeless> I've got a system error accessing the project page <lifeless> whats the debugging url again ? <stub> do: /++skin++Debug/whatever/the/url/was <lifeless> # <lifeless> # Module zope.app.traversing.adapters, line 114, in traverse <lifeless> curr = traversePathElement(curr, name, path, request=request) <lifeless> # Module zope.app.traversing.adapters, line 167, in traversePathElement <lifeless> next_item = traversable.traverse(nm, further_path) <lifeless> # Module zope.app.traversing.adapters, line 52, in traverse <lifeless> raise NotFoundError(subject, name) <lifeless> __traceback_info__: (<canonical.soyuz.sql.SoyuzProduct object at 0x41a739ac>, 'name', [] ) <lifeless> 'request': <zope.publisher.browser.BrowserRequest instance URL=https://macquarie.warthogs.hbd.com/launchpad/++skin++Debug/soyuz/projects/xfs/index.html>, <lifeless> I'm going to have to go. <lifeless> Its not likely to be a db problem AIUI, so, I'll just leave it down for now. <lifeless> as only mark and I have the password to macquarie, I don't have a problem doing this :). <stub> Cool. Have fun at the postgres sig. I'll be online later if you want to do this tonight, or tomorrow. <lifeless> great, thanks <lifeless> I'll try and track it down on my laptop during the SIG. <carlos> morning <stub> Morning <carlos> stub: please, could you add to rocketfuel the schema changes I sent last week? <SteveA> hello <stub> carlos: ok. I'll do that now. <spiv> stub: I'm setting up the authserver on macquarie, what DSN do I give to connect to the DB on emperor? <carlos> stub: thanks <spiv> And/or do you need to configure it to allow andrew@macquarie to connect? :) <stub> It shouldn't be connecting as andrew... <spiv> Hmm, I don't think there's been an 'authserver' user created for it... should there be? <stub> dbname=launchpad_prod, user=importd <stub> The users and permissions stuff needs to be audited and sorted. Connect as that user for now. <spiv> Ok. <stub> carlos: its with pqm now <carlos> stub: thanks <spiv> SteveA: production authserver is now running on macquarie. <kiko> morning <lifeless> spiv: how do I hook into the auth stuff to protect the sourcesource pages? <lifeless> kiko: just so you know, the SoyuzProjects & Products are /very much/ in use. <lifeless> (If it wasn't you that added the 'not in use' comment... sorry) <lifeless> I just had to track down a bit-rott problem because something got moved :[ <kiko> lifeless, it wasn't me, but thanks for the heads up. <kiko> lifeless, maybe you could write to LP describing what you're using Projects/Products for, because it really confuses us <lifeless> they are the only bits of launchpad actually in production. <lifeless> the cscvs source code imports <kiko> yeah, I expected something like that, because to us they make no sense and wouldn't even fit into soyuz :) <lifeless> well they do fit into soyuz. <lifeless> and rosetta. <lifeless> so: soyuz source packages are in arch, in theory. <kiko> soyuz (the web app) doesn't know about projects/products (except via a thin line spiv pointed out to me the other day) <lifeless> so to add a source package you need the code imported, which means you need the project + product created and a sync created for that. <kiko> they are an integral part of rosetta though, which is why spiv brought it up in the first place (soyuz-rosetta integration) <kiko> lifeless, aham. interesting. <lifeless> and likewise, rosetta imports from tla, so to translate a project, you have to import it. <kiko> lifeless, so should we link from source packages to products and from there to projects? * lifeless shrugs <kiko> don't shrug <lifeless> if the model allows a source package to exist /without/ a product... then I wouldn't worry. <kiko> but if it *does* have a product? <lifeless> but I was of the understanding that you could not create a source package without linking it to a product. <kiko> well <kiko> there are source package uploads that are distro/release-specific <kiko> but these don't exist IRL yet. <lifeless> I'm shrugging because I'm not the architect of the soyuz component - I can only relay what I recall from various discussions, the rdb model etc. <kiko> I know; still, your opinion is valid. <lifeless> yay, production launchpad happy again <kiko> sorry bout that. if you could get that mail to lp i'd appreciate it <lifeless> I'll drop a note there.. <lifeless> but I've said all I know already :) <kiko> more than we know so far :) <kiko> lifeless, if you can, point out where the code that uses that lives <lifeless> sql.py database.py configure.zcml : where all of your other code is <kiko> I mean *callsites* that use that code <kiko> you don't use python -i, right? <wink> <lifeless> sorry, you've lost me. <kiko> well <kiko> code that uses the sql/database code? <lifeless> templates/project* product* sync* <kiko> lifeless, and how does your code interface with that? or is it web-driven? <lifeless> web driven <kiko> ah! <kiko> using http posting or do you actually use the web app? <lifeless> the arch module - lib.canonical.arch - has the machinery to model stuff and do heavy lifting <lifeless> the conversions etc all live in buildbot and cscvs which are different tla code trees. <lifeless> and you should be able to be completely ignorant off. <lifeless> But I had to write a think web UI to let the relevant data be entered in, so that as folk start doing rosetta & soyuz & malone things, it all hangs together. <lifeless> i.e. bugs in malone should track back to a project. <kiko> but you add projects *manually* or in an automated fashion? <lifeless> I should be able to get a list of all the source packages that exist for 'gnome', for instance. or bugs.. or translations. <lifeless> kiko: manually, Mark wants them reviewed. <lifeless> https://www.warthogs.hbd.com/ProjectProductSetup <kiko> aha. <kiko> and the buildbox/cscvs bits *use* these bits of the database directly, or is it more complex? <lifeless> cscvs uses the arch objects... and the arch objects use the database. <lifeless> buildbot is the scheduling and RPC components. <lifeless> + a readonly reporting UI. <kiko> it would really be nice if you could be there next week because this is all really fuzzy to us. <lifeless> one of Mark's goals with this whole system is the pervasive annotation and linking of all these things. <lifeless> were next week? <kiko> in london? <spiv> lifeless: There's a Soyuz sprint in London next week :) <lifeless> on, hahahha no, <lifeless> y just spent too long there (1 month sprint) <daf> that's not a sprint, it's a marathon <carlos> daf: :-P <SteveA> spiv: thanks <SteveA> spiv: Can you mail roche giving the hostname and port that the plone server will use to talk to the auth server on macquarie? <SteveA> spiv: I just talked to roche. They started testing using the demo auth server on rosetta. They made one xmlrpc query to create a user, but received an error. <SteveA> spiv: Did you test the server yourself with python's xml-rpc library? <spiv> SteveA: Yes. <spiv> (hello :) <SteveA> hello <SteveA> ok, roche is in a meeting with a client for about 1/2 an hour or so <SteveA> but he will be in touch soon to sort out why he is getting some error <SteveA> can you look at the logs on rosetta? <spiv> Yep. <SteveA> and see why he might have got that error <spiv> exceptions.TypeError: Non-hexadecimal digit found <spiv> (doing sshaDigestedPassword.decode('hex')) <SteveA> so, he is passing you the literal string 'hex' <SteveA> ? <spiv> No, that's the way I'm invoking the hex decoding, i.e. <spiv> >>> 'abcdef'.decode('hex') <spiv> '\xab\xcd\xef' <SteveA> do you know what string he passed in? <SteveA> (I mean, are you logging xmlrpc queries?) <spiv> I'm not but I think I should be :) <spiv> I'll add that feature now. <SteveA> ok <SteveA> Maybe ask roche to join a chat so you can debug this together <spiv> (But it appears he's passing something other than a string of hex characters as the password) <SteveA> I'm off to catch a plane. Back tomorrow. <lulu> spiv:ping <spiv> lulu: Pong. <lulu> spiv:have you got Roche's mail on the error? <spiv> I don't have working email :( <spiv> My email server is down :( <lulu> ok - i'll do an IRC message to you - standby <spiv> Thanks. <lulu> spiv:done <spiv> lulu: Got it. <spiv> The problem is he's sending the SSHA digest as a base64-encoded string, and I'm expecting a hexidecimal-encoded string. <lulu> ok - I'll pass that on. <lulu> snything more? <spiv> No, it looks good otherwise :) <spiv> I'll mail him too, so that he's got an email address for me that works. <kiko> spiv, does he get a reasonable error message back, btw? :) <lulu> didn't realise you had alternate access - good - I'll leave you to it then. <spiv> kiko: Fault 8002. (i.e. no :) * daf goes to the dentist <kiko> spiv, very funny. :) <spiv> lulu: Sent. <lulu> spiv: thanks hon - we're aiming to have this done by close of play today so let's make sure it happens. <kiko> stub, you've got bugmail :) <stub> fun, fun, fun <sabdfl> db upgrade all taken care of, before it's too late .au time? <stub> sabdfl: db upgrade went well. Launchpad code had issues Robert was going to look at tomorrow (I think) <kiko> stub, afaik they are solved, see comment to you in #canonical <elmo> I'm going to upgrade the postgres DB software now - this'll involve a restart - if this a problem for anyone shout RSN <elmo> (done) <lulu> spiv: ping <lulu> spiv: still problems with same error code <spiv> Yeah, all unexpected errors get that error code at the moment. <spiv> I'm just waiting for the PQM to accept my merge. <lulu> spiv: ok - thanks. <spiv> lulu: I just saw a createUser call succeed on the test server, and I'm pretty sure the other calls are fine. <spiv> (but I'm double-checking just in case...) <lulu> spiv: good news - well done - let me know after the double check! <spiv> lulu: Will do :) * daf listens to tHHGttG <kiko> ddaa! <ddaa> oi <ddaa> I'm doing bits of ProjectProductSetup to help lifeless <ddaa> I am a bit puzzled by the whole idea of project/products... <ddaa> I got an interesting sample right here... <kiko> ddaa, join the club. how can I help? <ddaa> You can try to fill me in with the philosophy of projects and products. <kiko> ddaa, are you familiar with DOAP? <ddaa> Hu... not especially... <kiko> http://usefulinc.com/doap <kiko> check that out. <ddaa> Okay, I guess i should start by reading some dog about that. <ddaa> Well.. not much more informative than what is already on the ProjectProductSetup page on the wiki... * ddaa looks at freshmeat for inspiration <ddaa> Consider zsh... <ddaa> Well, that's a product. And that's a product. And that's all as far as I can tell. <ddaa> Defining a zsh Project would mostly mean copy/pasting the displayname, title, shortdesc and description... <kiko> I think most projects have single products, but IMBW <ddaa> Or take zope. That's a project, and that's a product, but then I hardly see how how to come up with different descriptions for the two, I draw inspiration from the Debian package description and the plone.org website, and there does not seem to be a distinction. <ddaa> Or take gzip/zlib. That would make sense at two products of the same gzip project. But then there is not gzip project per se. <ddaa> The web pages for both products seem to prosent them as independent software which happen to overlap in purpose and it authorship.. <ddaa> Or is that GNU project stuff? <kiko> the zope project has a *lot* of products. <ddaa> Name a few. <ddaa> I'm not particularly cognisant of the whereabouts of most free software projects. <ddaa> So for example, zope application server, zope database. <ddaa> That's the same project. <ddaa> But there is no info file for zodb... <ddaa> only one for zope <kiko> no no <kiko> or rather, yes <kiko> zodb and zope are zope products. <ddaa> Sure. <kiko> there are many others, just williamsmith@example.net <ddaa> Yeah I saw a raft of software packages "which are hosted on zope.org" <kiko> right <kiko> many of them official Zope products <kiko> (medusa is another, for instance) <ddaa> okay... It's hard to tell from where I stand, but I guess one can tell given a bit of careful browsing... <ddaa> But what is the relation of all this to the info files. <kiko> right <kiko> that's beyond what I can tell you. lifeless otoh is the man with the plan :) <ddaa> Bah... I'll stick to what I planned initially... Do some stuff, put it on the wiki, wait feedback... <ddaa> Successive approximations technique... <ddaa> ho, btw, how can I see the project/products which are already in the db? <ddaa> That might enlighten me and that will prevent duplication of effort. <kiko> ddaa, look at the sampledata file, for one. <debonzi> hi all, Ive made an star-merge from rocketfuel and my launchpad is not running anymore. It starts but I have only blanck pages and a traceback. Does anybody knows about it? <debonzi> s/blanck/blank <debonzi> ERROR root Exception during task <debonzi> AttributeError: 'NoneType' object has no attribute 'id' <debonzi> any ideia? <debonzi> s/ideia/idea <ddaa> kiko: bah... nevermind, I'll bug lifeless instead. <kiko> ddaa, as I said, he *will* know (and is possible the only person alive who does) <ddaa> maybe should fix that :-> <kiko> possibly. yeah, we should steveA just the other day grilled us because we didn't know anything about it. <ddaa> ah.. I meant fixing the "alive" part <ddaa> not the "only person" part <kiko> LOL <kiko> lunchpad. <ddaa> gotto leave <ddaa> dinnerpad here <kiko> later then dude <kiko> bon apetit mon ami <debonzi> As far I can see, nobody knows :( <kiko> debonzi, it could be your database is foorked <debonzi> I don't know.. Ive already rebuilded my db <kiko> are your tree and celso's identical? <debonzi> nop.. <debonzi> I started to happens since Ive made an star-merge from rocketfuel <debonzi> s/I/It <kiko> have you looked through the diffs? <debonzi> Nop.. <kiko> AFAIK lifeless hit the tree earlier today. could be that. <kiko> debonzi, something like: <kiko> https://rosetta.warthogs.hbd.com/soyuz <kiko> -- ? <debonzi> yes... the same <kiko> that's fucked up. <kiko> daf? <debonzi> but here I can see a traceback in the terminal where launchpad is running <daf> kiko: hi <daf> hmm <kiko> daf, can you take a look at rosetta there. we're getting 100% blanks and silly tracebacks and we need help from above :) <daf> AttributeError: 'NoneType' object has no attribute 'id' <kiko> daf, it's not just in rosetta, of course, but you can probably evaluate easily from there. <debonzi> thats it <debonzi> the same traceback <daf> kiko: I'd be glad to help, but I'm not sure what I can do <daf> it seems to be something to do with publishing * spiv is seeing it locally too... * spiv investigates.. <kiko> 3 cheers for the spiver <spiv> kiko: No guarantees :) <spiv> But thanks for the vote of confidence :) <kiko> spiv, you had better fix it, we are going to be 'seeing' you this sunday! * kiko stops spamming launchpadders <kiko> spiv, daf: have I sent enough email and bugmail for today? <spiv> kiko: I only just got my email back about an hour ago, so I just got it all at once :) <kiko> :0 <kiko> * From: williamsmith@example.net <kiko> /dev/null * kiko :-/ <daf> ?! <spiv> So, from reading the traceback, it appears that there's two problems... <kiko> daf, a procmail recipe? :) <daf> kiko: I know that! :) <spiv> There's an exception being raised in publish, and then it fails due to request.principal == None. <spiv> The original exception is a nameerror... <spiv> "global name 'transaction' is not defined" <kiko> weird. * debonzi wonders why always first with him :) * kiko spams some more <spiv> Ok, found it. <spiv> I wonder how changed that. <kiko> spiv, lifeless did some hackage today, what file is it? <spiv> lib/canonical/publication.py <spiv> --- orig/lib/canonical/publication.py <spiv> +++ mod/lib/canonical/publication.py <spiv> @@ -169,10 +169,10 @@ <spiv> # the transaction in which they are instantiated - this is <spiv> # no longer a problem as we are nuking the connection cache, <spiv> # but it is still an issue in SQLOS that needs to be fixed. <spiv> - name = getUtility(IConnectionName).name <spiv> - con = sqlos.connection.getConnection(None, name) <spiv> - t = transaction.get_transaction() <spiv> - t.join(con._dm) <spiv> + #name = getUtility(IConnectionName).name <spiv> + #con = sqlos.connection.getConnection(None, name) <spiv> + #t = transaction.get_transaction() <spiv> + #t.join(con._dm) <spiv> # Try to authenticate against our registry <spiv> prin_reg = getUtility(IPlacelessAuthUtility) <spiv> (That patch works for me ;) <kiko> https://williamsmith@example.net/launchpad--devel--0--patch-380/lib/canonical/publication.py.diff?diff <kiko> spiv, you merged that in? <spiv> Yeah. <spiv> kiko: Heh, I just found that too. <spiv> Looks like it was my fault :( <spiv> I thought I'd done "tla undo" :/ * ..[topic/#launchpad:kiko] : <spiv> Ok, found it. <spiv> I wonder who changed that. <kiko> spiv, you merged that in? <kiko> pity it can't be wittier but the old topic is stale. <spiv> :) <kiko> spiv, broken on the rock of tla again, huh? :) <spiv> Yeah, I guess so :) <spiv> Anyway, the merge request with the fix should be almost done by now. * debonzi will be away for some time <kiko_> spiveroo! #launchpad 2004-10-03 <lifeless> stub: around <lifeless> ? <sabdfl> morning everyone <sabdfl> time for a little launchpad love <stub> lifeless: pong <sabdfl> daf, rosetta.warthogs.hbd.com seems borked in malone, db errors everywhere <daf> sabdfl: interesting <sabdfl> daf: can you see the cause? <daf> looks like it might be related to some work spiv was doing yesterday <daf> ProgrammingError: ERROR: current transaction is aborted, commands ignored until end of transaction block <sabdfl> also, earlier on it was asking me for a username and password, which should i use on that server? <daf> your username is your email address, assuming there is one for you in the sample data <daf> the password is also set in the sample data <daf> let me check... <daf> looks like you have an email address, but no password <stub> daf: I've seen that error before - if you get a database exception (such as you violated a key), the connection gets in a weird state and the server needs to be restarted. <daf> you can use williamsmith@example.net, password "test" <sabdfl> daf: thanks <stub> daf: I don't know if it sqlos, SQLObject or Zope3 yet - I'll be chasing it soon unless someone beats me to it. <daf> stub: ok, I don't think spiv's up yet, so i'ts probably up for grabs :) <daf> stub: do you know about the changes spiv made? <stub> He patched the publish routine didn't he? I don't know the reason he needed to <stub> I don't think it is related to the ProgrammingError exception though - that is an older bug. <daf> right, the problem we were seeing was that Zope was seeing stale DB data <daf> so spiv added an explicit transaction abort <daf> which apparently fixed the problem <stub> Was this stale data being retrieved from sqlobject/sqlos? <daf> not sure <daf> well, I assume it was <daf> since I'm pretty sure it wasn't being retrieved from anywhere else <daf> https://bugzilla.warthogs.hbd.com/bugzilla/show_bug.cgi?id=2005 is related, but I don't think it's the change in question <stub> I thought I'd fixed something similar with sqlos (by destroying the connection pool in the publish method, which sounds a similar fix to what spiv needed to do. Maybe my patch has been lost?) <daf> it's possible <daf> (that the patch was lost) <daf> can you check? <stub> Its still there at the top of beforeTraversal in publication.py <stub> I'll suck down spiv's changes as soon as I have my new arch archive online to see what he needed to add. <stub> (I think this was discussed on the launchpad@ list while I was 'away' <daf> well, the change I'm thinking of was something spiv did yesterday <daf> actually, forget that <daf> the change I'm thinking of *was* the patch at #2005 <daf> and it hasn't been merged to rocketfuel <stub> Hmm... actually... the ProgrammingError should be impossible if my fix is being invoked, as the connection object should not be reused in different requests. I suspect my fix is not being called :-( <daf> it's only been applied to the alpha server as a workaround <daf> I can test that <daf> where's the fix, exactly? <daf> (I can't find a method called "publish") <stub> Just above spiv's patch - lib/canonical/publication.py in beforeTraverse <daf> hmm, now that I've restarted the server the problem doesn't seem to occur <daf> ok, now I'm seeing "AttributeError: 'NoneType' object has no attribute 'id' <daf> " <daf> hrm, things are being weird <daf> hi carlos <carlos> morning <carlos> daf: so we launch today Rosetta? <daf> that's the plan <lulu> daf: are you removing things like: <lulu> Join translation team, Start transaltion effort in https://rosetta.ubuntulinux.org/rosetta/search <lulu> gives an error <lulu> to reproduce: I searched for firefox <daf> lulu: oops <daf> I'd forgotten about the "advanced" search page <lulu> in advanced search, search results we ahve some dummy data still <lulu> ok - could you sort that before go live? <daf> yep -- best thing is to disable it for now <daf> I'll file a bug about implementing it <lulu> daf: perhaps do a check of all pages of functionality you are launching to double check they are all working. <daf> lulu: right <daf> lulu: thanks for finding that <lulu> daf: no problem. The about page needs some info - I'm finding some for you... <lulu> daf: have you written the announcement yet? <lulu> daf: we also need williamsmith@example.net mailing list set up <carlos> lulu: Will we create accounts into rosetta to anyone that ask for it? <carlos> I'm talking about the alpha <lulu> daf: oh - I see you have rosetta-testers for now - that should do the trick and shows it's still an Alpha. <stub> daf: Yes - the problem appears after you have violated a database constraint somehow, and stays around until you restart (as restarting the server resets all the database connections). * carlos goes to have breakfast <lulu> carlos: why? do you want to restrict it only to those who are on your testing list? <ddaa> spiv: did I mention to you how I hate twisted? <ddaa> spiv: I need your help for a quick crash course to twisted-proof subprocess handling. <spiv> ddaa: Ok... what do you need to know? <spiv> Or what problem are you having? <ddaa> Something totally weird like being unable to fdopen a fd we just created with pipe. <ddaa> We decided to use the twisted process handling when the environment has been twistedized. <ddaa> So as to avoid all the grief. Twisted assumes that everything going on is using it's process and thread stuff, so we'd rather make it happy3 <spiv> ddaa: Are you passing the right mode to fdopen? <lulu> carlos: Rosetta Alpha - people won't be able to create their own accounts. They will on the Beta when everything is integrated. <ddaa> Yes I'm passing the right mode. The problematic code is _working_ it only breaks on production in a twisted env... <spiv> (i.e. os.fdopen(os.pipe()[1] ) will fail, but os.fdopen(os.pipe()[1] , 'w') will work) <spiv> Ok, that's weird. :) <spiv> Where's the code? <ddaa> We had too much of that insanity. <spiv> (I've never heard of something like this before) <ddaa> It's just a bad idea to do things non-twistedly when twisted has pissed all over the place. <ddaa> Well, it's happening with some pretty ugly stuff... ya know the thread-that-reads-on-a-bloking-pipe <ddaa> But it would be simply least risky to just do things the twisted way. <ddaa> So <ddaa> 1. I would like to know what it is in twisted that do plenty of things with signal handlers, thread processing whatever which might cause grief. <ddaa> 2. <spiv> Twisted doesn't nothing that should affect threads. <ddaa> 2. How can I detect that the environment has been twistized? So I could use twisted process handling in that case, and only in that case. <spiv> (There are some things in Twisted and libraries for Twisted that spawn threads, and manage their own thread pools etc, but that's the extent of it) <ddaa> Okay. I take your word for it. <ddaa> Apparently, twisted _do_ things to the environment which cause my ugly subprocess handling to die in horrible ways. <spiv> Signal handling williamsmith@example.net() time. <spiv> (unless you pass installSignalHandlers=0 to it) <spiv> It installs handlers for: <ddaa> our problem occurs within <ddaa> twisted/internet/threads.py:37:_putResultInDeferred <spiv> SIGINT (unless there's one already), SIGTERM (always), SIGBREAK (if it exists), and SIGCHLD (ditto) <ddaa> is that within reactor.run()? <spiv> Yes, that won't happen until the reactor has already started. <spiv> Oh, actually... <spiv> You could call that at any time. What's the error you get at that line. <spiv> ? <ddaa> Nothing at that line <ddaa> The error happens much lower in the stack in pyarch process handling code. <spiv> Hmmm.. <spiv> So, if that's the first call to that, that may cause a call to reactor._initThreadPool()... <ddaa> When I do things with the evil-blocking-pipe <spiv> What's the last line of the traceback, and the error? <ddaa> The pb is that this was the highest visible line in the backtrace lifeless gave me. <spiv> Ok, but what's the lowest? :) <ddaa> exceptions.OSError, [Errno 9] Bad file descriptor <ddaa> self.readfile = os.fdopen(read_end, 'r', 0) <ddaa> The line just before that is: <ddaa> read_end, self.write_end = os.pipe() <ddaa> Dude, I create a pipe, fdopen it and it barfs! <ddaa> I got another backtrace when closing self.readfile <spiv> What's the value of read_end? Perhaps you've run out of fds in that process? <spiv> The reactor does the exact same thing internally for it's "waker". <ddaa> do not have the value of read_end... <spiv> (See _UnixWaker in twisted/internet/default.py if you're curious) <ddaa> not going there unarmed... <ddaa> The problem here _might_ be a fd leak. <spiv> Hmm, although that should raise an OSError at the os.pipe call. <spiv> (I just tested) <ddaa> But then it would not explain the error on close. <ddaa> So, at this point, I just want to avoid doing things twisted is apparently not prepared for. <carlos> lulu: yes, that's why I was asking because I thought that, Alpha -> only accounts we created, Beta -> everyone <ddaa> We already had quite a lot of grief previously (remember at Oxford?) <ddaa> And know it breaking prod. <spiv> Well, I honestly don't know of anything in Twisted that interferes with this stuff. <spiv> In fact, Twisted internally relies on doing the exact same thing. <ddaa> So, please, just help me to figure out when the environment has been touched by twisted. <ddaa> So that the code would use twisted when run in a twisted program, and would use something else in other cases. <spiv> So unless there's some sort of bizarre race in your threads where another thread somehow can be closing the wrong fd, which seems unlikely, I don't know what's causing that problem (I don't know how to reproduce that behaviour even if I'm trying to...) <spiv> Oh, is this code running with my evil hack from Oxford? <ddaa> No. Your evil hack was only something for the test suite, and it did not work anyway. <spiv> Ok :) <ddaa> i separated the test suite in a twisted part and a pyarch part. <ddaa> I do not want to know what's the cause of the problem. <ddaa> I just want to use twisted when twisted is being used already. <ddaa> See what I mean? <spiv> Yes, I'm just looking that up <spiv> The .running attribute of the reactor can tell you if the reactor is running. <ddaa> Once I have that bit, i can dig up the "how to spawn coprocesses" part myself. <ddaa> How can I find the reactor? <spiv> from twisted.internet import reactor <ddaa> It's a module-global object? <spiv> Yes. <ddaa> Thx <spiv> A known design flaw, but convenient. <ddaa> Good for my purposes. <ddaa> I'll try reproduce the pb and do the twisted things, then. Thanks. <spiv> You're welcome. <spiv> I'd love to know what the problem is, it's very odd :) <carlos> lalo: hey! <lalo> :-( * lalo replaced the motherboard and stuff remains borked <lalo> that leaves two possibilities; either it's my cpu (that would suck, as it's not in warranty), or the hardware is now fine but the libc is borked due to having been compiled on faulty ram <lalo> meanwhile, gossip and lostirc to the rescue <daf> hi lalo <daf> good to have you on IRC again <lalo> hello <lifeless> sqlos questions... <lifeless> how do I say 'NOW()' to sqlos <lifeless> and how do I say '1 day' to an interval field. <daf> lifeless: 'NOW' for 'NOW()' <lifeless> this is in an assigmnet: <lifeless> self._sync.processingapproved=NOW <lifeless> ? <daf> use the string 'NOW' <lifeless> oh, the string. * SteveA goes to get lunch <carlos> stub: do you think we should add indexes also for foreign key or with inner joins is enough? (talking about improving the speed) <spiv> lifeless: A quick google suggests 'import sqlobject.sqlbuilder; sqlobject.sqlbuilder.SQLConstant("INTERVAL 1 DAY")' <lifeless> spiv , oh ewww. so much for domain & data separation <spiv> There might be a nicer way using mxDateTime's DateTimeDelta or something like that. <spiv> Or even datetime.timedelta(1) <cprov> spiv, stub: hi guys ! <lalo> I seem to remember me and Daf wasting a good half hour in Oxford trying to find what kind of datetime objects sqlo uses <lalo> I think it was mx <spiv> lalo: It has "converters", so in theory it can support multiple kinds. <spiv> Glancing at the source though, it doesn't have any sane support for intervals, though, just absolute dates :/ <lalo> in theory :-) I don't know whether these converters are recent enough to know about the datetime module <lalo> anyway, the reason we were looking was because we wanted to know what objects it would *return* <spiv> lalo: They are. <spiv> But not smart enough to know about datetime.timedetla. <spiv> datetime.timedelta, rather. <lalo> daf: fix, part 1 of 3, done; if we're importing a pofile we already know about (in the db), and the db row has pluralform info, we use that <lalo> part 2 (being done now) is, if we don't have that, try the languages table; part 3 is scream bloody murder if we don't have either <lifeless> spiv: so how do I protect a form so only I can set a specific field <carlos> lalo: you need to take care of a fourth case, if the default plural form is not correct for that .po file <lalo> carlos: what kind of care? <carlos> lalo: the error should say that error is that the plural forms used on that file are not the same that the header info <carlos> or the table info so we can fix it easily <lifeless> spiv: that interval 1 day thing isn't working <carlos> where table == Language table <spiv> lifeless: Try SQLConstant("INTERVAL '1 DAY'") <lalo> carlos: ouch. That would be... rather involved :-P <spiv> lifeless: I'm currently working on making SQLObject support timedelta, it doesn't seem too hard... <lifeless> still not setting it <spiv> Try just the literal string? :/ <lifeless> nope <spiv> lifeless: You mean protect a field in a form in zope? I'm not sure, SteveA would know though... <lalo> oooh, found a bug <lifeless> well, not web form, the business logic level. Sync.enable() can only be called by me <lalo> oooh, found *another* bug <spiv> lifeless: williamsmith@example.net/sqlobject--interval-support--0 should allow you to use datetime.timedelta(1) <spiv> lifeless: Only very lightly tested so far, though. <lifeless> and 1 is one second or 1 day ? <spiv> 1 day. <spiv> args for timedelta are days, seconds, microseconds. <lifeless> ok, cool <daf> lalo: I have a couple more interesting errors that happened overnight <carlos> daf: I'm testing the indexs part, I think with 3 indexs will be enough for the import <daf> carlos: yeah? <daf> carlos: what are they? <carlos> CREATE INDEX pomsgidsighting_index_inlastrevision ON POMsgIDSighting (pomsgset, pomsgid, inlastrevision); <carlos> CREATE INDEX pomsgidsighting_index_pluralform ON POMsgIDSighting (pomsgset, pluralform); <carlos> CREATE INDEX pomsgset_index_primemsgid ON POMsgSet (potemplate, pofile, primemsgid); <carlos> that's based on postgresql logs <daf> cool <daf> do you have any information on how they affect performance? <carlos> daf: I'm doing some tests now <carlos> my first test yesterday save me one minute importing the .pot file <daf> you gave me a link to some Postgres doc -- can you paste that again? <lalo> daf: good thing all pofiles we have are imported, because the header we're storing in newPOFile was absolutely BS and it would have breaken the exporter <spiv> lifeless: Ooops, give me 2 minutes before testing that :) <lifeless> spiv: ah ha <lifeless> I'll get a drink then shall I <daf> lalo: don't we have a but open on that? <daf> lalo: s/but/bug/ * daf awaits the inevitable Beavis and Butt-head joke <lalo> daf: do we? <spiv> lifeless: Ok, working version mirroring now :) <daf> I thought we ded <daf> er, did <lalo> dude, get some caffeine :-P <carlos> http://www.postgresql.org/docs/7.4/static/tutorial-join.html <daf> lalo: just made some Earl Grey :) <carlos> http://www.postgresql.org/docs/7.4/static/indexes.html <daf> carlos: a <daf> grrr <daf> carlos: thanks <carlos> daf: :-P <spiv> lifeless: My branch lets you set values, but don't expect to get sane values back from reading that field yet :/ <daf> spiv: aren't you the SQLObject maintainer already? <lalo> ugh. I need to recreate the database each time I want to test this, because zopeless doesn't abort :-/ <spiv> daf: Hah. <lifeless> spiv: its still coming across as null. <lifeless> omg. <lifeless> IntCol('frequency', dbName='syncinterval', default=None), <lifeless> # WARNING: syncinterval column type is "interval", not "integer" <lifeless> # WARNING: make sure the data is what buildbot expects <lifeless> I blame you. <lalo> cool, there is no case 2. The database will always have info (from an old version or from languages table) due to newPOFile() <spiv> lifeless: Oh, d'oh. <spiv> lifeless: Change that to DateTimeCol. <lalo> so either case 1 (the database knows the info) or 3 (it doesn't, scream loud) <spiv> Incidentally, there's been a new upstream release of SQLObject, finally. <daf> oh, 0.6? <lalo> is that good? :-) <spiv> daf: yeah. <spiv> Although we're basically running it anyway... we're using a recent SVN snapshot. <spiv> But it's a sign that maybe upstream will get around to doing something about my bug reports one day ;) <daf> since we're basically including the source code in RF, I don't see any pressing need for me to update my packages, then <lifeless> spiv: thank thee. <spiv> thou art welcome ;) * carlos goes to have a shower <daf> stub: I haven't seen any cache deletions happening yet <daf> but "yet" is about 3 requests <sabdfl> debonzi, cprov: good work on making the soyuz tables visible <sabdfl> for malone testing, we need some more sample data of packages, do you guys have any scripts that can populate the test db with a bunch of source packages and versions? <stub> daf: You lost me. You mean you stuck a debug message in the beforeTraverse and that fix is not being called? <daf> stub: exactly <stub> SteveA: ping <daf> stub: well, it gets the key, but they key is not in the cache <daf> stub: SteveA said he was having lunch <stub> daf: Hmm... maybe I'm just using the wrong key then? Can you dump repr(the_connection_cache_mapping) to see what is in it, and more importantly, what the keys are? <stub> It was working when I nuked the entire dictionary, but that isn't thread safe. <daf> each request seems to have a different key <daf> (16386, u'launchpad') <daf> (32771, u'launchpad') <daf> etc. <cprov> sabdfl: tks, we still working to have better results til sprint . <daf> stub: oh, ok, I'm seeing a deletion now <daf> stub: the error seems to occur when the cache deletions do <daf> stub: emphasis on "seems" <daf> I'll try dumping the entire cache mapping <stub> daf: You will only see a deletion the second time that thread handles a request. I think there are 4 threads by default, so you might not see one until you have loaded 4 or 5 pages. <sabdfl> cprov: what's the best way for us to get package data in for a couple of thousand packages? <daf> stub: right, that explains what I've been seeing * stub wonders if z3 can be run in single thread mode like z2's '-t' command line argument * stub reboots again <daf> wurg <stub> I think I've got this wireless nutted so it will come up on a reboot :) <cprov> sabdfl: I hope spiv is working on a DEB importer or something related <cprov> sabdfl: a extra component to get information of deb packages and insert automagicly(?) on DB <stub> :) <lifeless> stub: so can you approve some of those projectproduct things? <daf> ok, the cache deletions seem to be working after all <daf> spiv: spiv is doubting that deleting the adapter is rolling back the transaction <daf> gaar <daf> stub: that was supposed to be directed at you * daf is having a bad finger day <stub> daf: We can explicitly abort the connection at that point if we want (before we nuke it from the cache? spiv's fix might be running to late?) <stub> lifeless: I can't spell, and I get to approve them?? <lifeless> stub: yes. the only thing *I* care about is the project & product name so I can enter them into launchpad. <lifeless> that will probably make mark unhappy though, as his beloved shortdesc & description (whats with that?!??) will still need to be fleshed out. <stub> In the name of the senate and people of rome... *thwack* <lifeless> stub: well.. I just /need/ to be able to put them into the system such that I can associate Sync's to products, and then run the syncs., <stub> Ssh... I'm reading the wiki page... <lifeless> stub: ok, sshing. I've emailed mark asking to be able to ignore the desc & shortdesc, but haven't heard back. <stub> lifeless: So this is where we have to get the project -> product mapping right. I'd assumed bison was part of the GNU project, but you think it is part of its own. Isn't this getting very political? <daf> stub: I was worried about this happening :( <lifeless> stub: not my problem. <lifeless> my problem is getting cvs converted to arch. <spiv> Does it really matter if the mapping isn't quite right? We can update them later, surely? <lifeless> Not meaning to be a 'boundary drawing arsehole' .. just already about 20 imports behind. <lifeless>
{'channel': 'launchpad', 'month': 10}