#ubuntu-meeting 2004-12-04 <smurfix> Is anybody going to open this meeting / are we waiting for people / ??? * ..[topic/#ubuntu-meeting:smurfix] : Tuesday 23 November 2004 at 16:00 UTC: Community Council meeting -- https://www.ubuntulinux.org/wiki/CommunityCouncilAgenda <pitti> smurfix: I pinged sabdfl * Kamion is here <sabdfl> hi all <sabdfl> nothing in the wiki agenda <sabdfl> can we call for any other business? (Kamion/#ubuntu-meeting) I believe there were some NM candidates to approve (Kamion/#ubuntu-meeting) the note in the agenda is from last week's TB meeting if memory serves correctly <smurfix> Kamion: it serves <haggai> sabdfl asked me last week to ask here about NM <sabdfl> haggai: process is to become a contributor, then a maintainer in universe, then in main <sabdfl> Kamion: is there a list of NM candidates that the TB approved last week? (Kamion/#ubuntu-meeting) I'm just trawling the irclog <smurfix> the TB didn't <smurfix> they were told to ask here first <sabdfl> i think all we did was confirm people who had already been approved at the previous CC meeting (Kamion/#ubuntu-meeting) haggai is the only one I can see there who was referred to the CC for approval <smurfix> s/they/at least one person (Kamion/#ubuntu-meeting) the other people mentioned were already approved last meeting <sabdfl> haggai: do you have a sponsor, someone on the existing maintainer team who can vouch for you? <sabdfl> or can you point us at upstream / other distro maintainership credentials? <haggai> plenty I hope :) (Kamion/#ubuntu-meeting) possibly enrico too, not clear on his current status <sabdfl> enrico's fully approved, both TB and CC <haggai> sabdfl: OOo / Debian maintainer etc <sabdfl> ah <sabdfl> packages in debian? (Kamion/#ubuntu-meeting) sabdfl: haggai is the OOo guy whose mail I referred to you last week <sabdfl> Kamion: chris? (Kamion/#ubuntu-meeting) yes <haggai> sabdfl: that's me * sabdfl blushes furiously <haggai> sorry for the obscure nick <sabdfl> certainly no objections from me, i'd already asked elmo to approve your key for KDE-related uploading <sabdfl> elmo, mako, comments on chris halls? (Kamion/#ubuntu-meeting) Chris' only Debian package in his own name is ayttm, but he's done a great deal of work in teams; I'm happy <elmo> sabdfl: no problems at all with haggai <sabdfl> mako around? (Kamion/#ubuntu-meeting) mako doesn't seem to be up yet <haggai> yeah OOo in a team is kind of a lot more more work than owning a normal package :) <sabdfl> if we have mdz and keybuk, maybe we can get TB go-ahead today too <smurfix> mako's been idle for 10h (Kamion/#ubuntu-meeting) haggai: quite so :-) <sabdfl> ok <sabdfl> i think that's a CC go ahead <sabdfl> don't want to wait a week for TB go ahead <haggai> ok thanks very much :) <sabdfl> haggai, if you can get an email from mdz to go with keybuk's go-ahead then you're sorted (Kamion/#ubuntu-meeting) mdz just showed up, I guess he's trawling backlog (mdz/#ubuntu-meeting) was distracted by the backlog on -devel actually :-) <Keybuk> haggai definitely has my go-ahead :) <sabdfl> elmo: will you act on a go-ahead for haggai from mdz? <sabdfl> i think this can extend to main ;-) <elmo> sure <sabdfl> elmo: do we have an easy way to let people upload to universe / multiverse only? <elmo> sabdfl: not right now, but it's an hour or so to implement <sabdfl> elmo: rock on ;-) * mako waves <elmo> sabdfl: the existing approved maintainers - should they have access to main? <sabdfl> yes <elmo> ok <elmo> (they do right now) <sabdfl> we need to create a faster process for non-main uploaders <sabdfl> any other business? * mako is caught up <sabdfl> kamion, thanks for standing in for mdz (mdz/#ubuntu-meeting) yes, thanks (Kamion/#ubuntu-meeting) np, will get back with progress later today; it was a relatively quiet week TBH <sabdfl> not in this house :-) (Kamion/#ubuntu-meeting) heh <sabdfl> ok <sabdfl> anything in connection with The Mataro Sessions people want to raise? <sabdfl> looking fwd to meeting the local LUG'ers <sabdfl> thanks everyone (mdz/#ubuntu-meeting) has a BOF about community processes already been proposed? <sabdfl> silbs: ^^? <silbs> yes, believe that's on the list <sabdfl> see you all there (Kamion/#ubuntu-meeting) we don't seem to have anything scheduled yet for bringing new Ubuntu maintainers up to speed <silbs> lulu is drafting a schedule now. so other suggestions for bofs would be welcome (Kamion/#ubuntu-meeting) might be worth having some kind of a Q&A session <mako> silbs: i think i had one or two <mako> Kamion: yeah, there is a plan for this <silbs> Kamion: what kind of q&a do you mean? <sabdfl> we haven't implemented some of the ideas that came out of oxford <sabdfl> "ubuntu enhancement proposals" <mako> Kamion: there will be a day for questions and more introductory stuff (mdz/#ubuntu-meeting) yet <sabdfl> be nice to have that up and running for the conf <sabdfl> we could have a bof on each significant proposal <sabdfl> mako: where are we on shipit stats? (Kamion/#ubuntu-meeting) silbs: what mako said; we have a number of non-Canonical Ubuntu maintainers coming to the conference, and I'm sure some of them have questions about process that are best answered in person; the non-Debian guys may also have technical problems we can help with <lulu> kamion: we have a BOF scheduled for this at the conf on the 8th (mdz/#ubuntu-meeting) there is documentation that needs to be written <smurfix> mako: ... and will there be a huge stack of CDs to carry off in Mataro? ;-) <mako> sabdfl: i've been finalizing the dump from early last week.. but when i checked on thursday we had i think 800 orders between sunday and thursday so somewhere in the range of just under 2000 new order orders (average of ~10 cds) each way <mako> sabdfl: week (Kamion/#ubuntu-meeting) lulu: ok, good <sabdfl> ok <sabdfl> so 20,000 / week going out <sabdfl> that's awesome <mako> sabdfl: it will fluctuate depending on press and such <sabdfl> can we wrap up? <sabdfl> elmo, mako, thanks <sabdfl> cheers <mako> yay, easy summary :) * smurfix waves <haggai> thanks folks * Kamion makes that a 19-minute meeting (Kamion/#ubuntu-meeting) awesome * ..[topic/#ubuntu-meeting:mdz] : Tuesday 30 November 2004 at 16:00 UTC: Technical Board meeting -- https://www.ubuntulinux.org/wiki/TechnicalBoardAgenda #ubuntu-meeting 2004-12-11 (pitti/#ubuntu-meeting) so I copied the ascii file to http://people.ubuntu.com/~pitti/langpacks.txt (smurfix_/#ubuntu-meeting) pitti: thx <mdz> morning (pitti/#ubuntu-meeting) Hi mdz <mdz> pitti: your agenda item was masked by a markup error, I didn't see it until I tried to edit the page (pitti/#ubuntu-meeting) mdz: huh? (pitti/#ubuntu-meeting) mdz: it was fine for me <mdz> http://www.ubuntulinux.org/wiki/TechnicalBoardAgenda (smurfix_/#ubuntu-meeting) was OK here too (smurfix_/#ubuntu-meeting) s/here/for me/ <mdz> elmo,fabbione,kamion,keybuk,lamont,mvo_,sabdfl (fabb1one/#ubuntu-meeting) mdz: i am here (fabb1one/#ubuntu-meeting) just a sec that the wartylog is desynced <fabb1one> test <fabb1one> ok much better <Keybuk> quite finished? ;o) <sabdfl> hi all <Keybuk> btw, I'd just like to say, for the record, that Cadbury's are evil <mdz> ok, we seem to have a quorum <pitti> Keybuk: what's Cadbury? <elmo> lol <Keybuk> pitti: they make chocolate <elmo> pitti: chocolate manufacturer <sabdfl> pitti: it's a backup weapon to the jellybean <mdz> the only predetermined agenda item is: language packs * pitti understands :-) <pitti> mdz: that was me, BTW * Keybuk is going off to bomb Bourneville when this is over <Keybuk> (and then feel ill) <Keybuk> :p <mdz> if I'm not mistaken, this is carried over from the last TB meeting :-) <sabdfl> mdz: we had a long discussion during your break <mdz> ...which I have only skimmed <pitti> mdz: yes, I distilled the discussion to http://people.ubuntu.com/~pitti/langpacks.txt and to ubuntu-devel <pitti> however, the discussion on the ml wasn't very active <pitti> so I would like to talk about it here <sabdfl> i have a tech question for pitti <pitti> we don't need to agree about every single detail * pitti listens <sabdfl> if we set the default build process for packages so that english is installed by default by the package <pitti> but we should agree on the general architecture <pitti> sabdfl: this is the default, btw <sabdfl> and had a mechanism so that a local build could install it's own translations for specific locally required languages, overriding the lang pack <sabdfl> would this resolve the conflicts issue? <sabdfl> largely <pitti> sabdfl: you mean if you locally rebuild the debs from the source package? <sabdfl> IOW i imagine someone building the package locally would automatically get (a) english and (b) the languages that their Ubuntu langpacks are installed for <sabdfl> yes <sabdfl> without having to rebuild the langpack <pitti> it should be easy to confine the build to some languages, yes <Keybuk> package would conflict with langpack in that situation, no? <Kamion> isn't it quite important that local builds = our builds by default? <pitti> this essentially means to delete all other languages from /usr/share/locale <mdz> Kamion: yes <fabbione> Kamion: yes <sabdfl> Kamion: they would, in the sense that the deb would include english by default, as would ours <pitti> but this would have similar drawbacks as F5 <Kamion> sabdfl: I mean in all senses, i.e. same list of files <mdz> if we're going to do language packs, the language pack should become the central point for translation activity <Kamion> that's really important for developers testing stuff <pitti> I really thought over this issue <pitti> IMHO putting the translations into proper debs leads to nothing but trouble <sabdfl> agreed <pitti> either our archive size or our mirrors explode <pitti> and we become totally incompatible to the rest of the deb-world <mdz> pitti: what do you mean by proper debs? <mdz> e.g., language packs? <mdz> I think there are certainly benefits <pitti> mdz: putting the po files into a real deb (as in the F2 options) <Kamion> pitti: nod - the approach of grabbing stuff straight to /usr/share/locale/ is looking kind of appealing right now, considering all the alternatives <pitti> mdz: vs. managing the po files (and other stuff) aside from debs <mdz> it allows us to safely update translations without touching the package itself <sabdfl> F2-3: we should expect upwards of 200-500 languages within a year <pitti> I still think that F4 is the sanest variant <smurfix_> pitti: that kills every otion but F4 (and F6 ;-) <pitti> with some ideas from F3-2 <pitti> i. e. download new translations to e. g. /var/cache/locale <mdz> pitti: did you make some estimates for disk space requirements? <sabdfl> Kamion: you mean, taking /usr/share/local/ out of dpkg control, and managing it separately? <pitti> and provide this as an alternative gettext root <Kamion> sabdfl: with respect I find that hard to believe, considering 10+ years of free software translation work that have produced maybe 20 languages with plausible coverage <mdz> 200-500 is quite a lot more than any package we currently ship <pitti> mdz: the user would only download the languages and translations that he wants <sabdfl> Kamion, mdz: these are clear goals of ubuntu/rosetta <pitti> mdz: so disk space is minimal, without any deb overhead <Kamion> sabdfl: I know, I'm just saying I find it hard to believe. <pitti> mdz: and overhead of unwanted translations <Mithrandir> pitti: why would F2 include conflicts? Replaces should be enough. <mdz> sabdfl: yes, I'm saying that we need data to know what that would really look like in terms of resources <sabdfl> fair nuff, just so you understand that's why i'm pushing this so hard now <pitti> Mithrandir: I think it's a bad idea to have a language pack replace a complete application deb <Mithrandir> pitti: Replaces: doesn't mean that. <pitti> sabdfl: I do _not_ propose to change anything in /usr/share/locale, btw <sabdfl> what about this? <Kamion> sabdfl: don't think /usr/share/locale/ should be taken out of dpkg's control; pitti's F4 suggests somewhere in /var <pitti> sabdfl: I only want to have additional translatiosn in /var/cache/locale <sabdfl> could we update the ubuntu gettext to look in *two* locations? and take the newer? <pitti> Kamion: yes <sabdfl> language packas would go in /usr/share/langpack/ <pitti> sabdfl: this should be easy for the majority of packages <mdz> gnome-panel has 70 locales totalling 4.2M <pitti> sabdfl: however, we need to manually fix packages which link gettext statically * Mithrandir thinks /usr/local/share/locale would be the right place, but that's a detail. <Kamion> avoiding any location that is currently used produces the huge benefit that we do not become instantly incompatible with everyone <sabdfl> then the local build could go into a diff location <Kamion> Mithrandir: it's still distro-managed though <smurfix_> pitti: that makes sense anyway <pitti> Mithrandir: I would favor /var; /usr might be r/o <sabdfl> the main thing being that the langpack would not put files anywhere a locally built deb would expect to be able to do <pitti> My idea is to have a language pack which controls the downloading and installation <Mithrandir> pitti: depends on where the langpacks comes from -- if they are .debs, then /usr is r/o => you lose. <mdz> I like pitti's idea of treating the two pools of translations separately <pitti> Mithrandir: as I said, I don't think it's a good idea to put updated translations in deb <mdz> but it highlights a question about the overall process <Mithrandir> pitti: why not? <mdz> is our goal that these translations be passed upstream? <sabdfl> mdz: yes, through rosetta <pitti> Mithrandir: I explained in detail in my list <Mithrandir> pitti: how else are you going to make sure that cruft is removed? <mdz> if so, upstream inherits a lot of these problems <pitti> Mithrandir: it's nothing but trouble <pitti> Mithrandir: this must be handled by the download manager (i. e. the language pack) <mdz> their source tarball grows, their default install size grows for source-based installs and for other packagers as well <Kamion> mdz: upstream don't really, Debian does though <smurfix_> mithrandir: either you have LOTS of debs or your update is a 99% no-op <pitti> Mithrandir: it shouldn't be too hard to sort out cruft in gettext files <Kamion> most upstreams don't seem to care about source tarball size :) <smurfix_> mithrandir: both is not nice <sabdfl> btw: bazaar 1.0 release in progress now <Mithrandir> pitti: I'm talking about cruft as in "handling languages which are no longer on the system". You would be reinventing dpkg. <Kamion> how about /usr/share/locale/extra/ or something? <Mithrandir> Kamion: do people care about installed-size? <pitti> Mithrandir: maybe, but using dpkg is even worse... <Kamion> would be nice to keep within FHS directories <pitti> Mithrandir: deleting a language pack can just remove /var/cache/locale/<lang> <smurfix_> mithrandir: it's a somewhat different problem space <pitti> Mithrandir: s/delete/purge/ <Kamion> or at least get this standardised somehow, since it sounds as if everyone will run into death-by-language-bloat soon <mdz> Mithrandir: if installed-size were not an issue, I think language packs would be a non-issue as well <mdz> well, almost <pitti> mdz: there are also troubles with mirrors and pacakge rebuilds <mdz> there would still be the issue of .deb size <pitti> mdz: installed-size of language packs is the least evil point <Mithrandir> mdz: I'm asking whether it's really an issue or whether we just have a bunch of whiners who can be ignored. (To be harsh) <mdz> Mithrandir: any whiners are hypothetical at this stage <Kamion> Mithrandir: CD size is pretty hard and fast <mdz> we're extrapolating what would happen if the number of translations exploded <smurfix_> kamion: teaching dpkg to basically ignore some directories should be reasonably simple <mdz> smurfix: that would only address installed-size, and not .deb size <Kamion> smurfix_: that approach is pretty fragile and hard to reconfigure later though <Kamion> and, as mdz said <pitti> smurfix: why you want to do that? <sabdfl> easy, Mithrandir <Kamion> smurfix_: if somebody says "please can I have French translations now", you have to reinstall everything <Mithrandir> sabdfl: I didn't intend to stomp on any toes -- I was wondering if we were discussing a real problem or not. :) <mdz> we are <mdz> though we are trying to discuss it before we actually have it <pitti> Can we at least agree to drop the F2 options? <sabdfl> in terms of where i would like ubuntu to be in two years time, which is much more widely translated than rhat, suse or debian, yes this will be a real issue <pitti> i. e. extract translations during build into separate debs? <smurfix_> kamion: assuming that the files can't be downloaded from rosetta-or-whatever <sabdfl> fine by me <mdz> wait <mdz> F2 are the only options which address the problem of .deb size <mdz> oh, also F5 <sabdfl> our build process will have to be different <pitti> mdz: ? <Mithrandir> mdz: F5 means the shipped debs are different from what you get from apt-get -b source $package, though. <sabdfl> in that we will not include the translations in a default .deb build <mdz> Mithrandir: yes <mdz> F5 has big problems in terms of package management <mdz> but it does address the problem of .deb size <Mithrandir> pitti: F3, F4 doesn't shave any size off the shipped debs. <pitti> mdz: the point is that _anything_ using debs created by us makes troble <elmo> F4 could be combined with a stripping <Mithrandir> pitti: which we want to do, in order not to hit the 650MB limit. <mdz> we could do a slight modification of F5 which results in having the same .debs in the archive and on the CD <elmo> F4+strip down to English +whatever we can fit is by far the most sane option IMHO <pitti> Mithrandir: yes, but _we_ have to create debs regularly <mdz> local builds break <pitti> Mithrandir: as opposed to F4, when the _user_ decices when to download what <Kamion> sabdfl: I really think patching the source packages is the only correct way to modify the build process <sabdfl> Kamion: yes, i expect that <pitti> Mithrandir: this simplifies both building and downloading a lot <Mithrandir> Kamion: I agree with you on that. <Kamion> sabdfl: ok, good, alleviates my major concern :) <mdz> sabdfl: based on our experience so far with merges, that would be a prohibitively large workload for our team <elmo> mdz: okay, F4 +strip in binary and source <pitti> Kamion: but I think if we can manage without patching source packages at all, this would be even better <mdz> patching every source package is not an option for us <smurfix_> elmo: why both binary and source? Agreeing on either makes more sense imho <mdz> elmo: hmm <Kamion> mdz: the fact that a source package specifies what goes into its binary packages is an invariant; breaking that just for scheduling reasons is so wrong <elmo> mdz: it's not every source, nly what's on the CD <elmo> we can't not patch what's on the CD if you want local builds to work, AFAICS <Kamion> if we're going to modify the build process then that change must be in Ubuntu <Kamion> (i.e. not buildds) <mdz> elmo: I think I like that very much <elmo> smurfix: eh? the binary have to be modified to get the space down, the source has to be modified, if you want a local rebuild to procue an installable .deb <smurfix_> I'd prefer mangling the binaries when they arrive from the autobuilder. <mdz> (F4 + stripping) <Kamion> smurfix_: so wrong, so wrong <mdz> that avoids the conflict issues with local builds <pitti> Am I right that stripping and F4 are actually completely orthogonal? <Kamion> think I agree with elmo here <pitti> so we can discuss that independently? <Mithrandir> mdz: and then leaving /var/cache/locales non-dpkg-managed? <mdz> pitti: yes <mdz> good plan <mdz> we can decide separately how to best remove translations from .debs, and how to provide the translations separately <smurfix_> elmo: why would a local rebuild create a conflict? <mdz> the former is the difficult bit <Keybuk> stripping how? <mvo_> I think the later is tricky too :) <Kamion> mdz: adding dh_striplocales (say) to debian/rules wouldn't be that bad, would it? <Kamion> if it's a one-liner ... <Keybuk> at buildd-time, or in the source package? <elmo> smurfix: oh, okay, it wouldn't necessarily, but it wouldn create a different .deb, and there's a lot to be said for idempotency of builds <Mithrandir> Kamion: aka rm -rf debian/$packagename/usr/share/locale ? :) <sabdfl> it should be a one-time one-liner <pitti> mdz: stripping binary debs is easy, but stipping source packages? We have to change every orig.tar.gz... <Kamion> pitti: and then various upstream projects will accuse us of forking <pitti> Kamion: how would that help to reduce the source package size? <Kamion> (c.f. openssh) <Keybuk> Kamion: the bit where it would go doesn't tend to change hugely -- though it'd trip on a major change to debian/rules and need manually fixing <Kamion> Keybuk: yeah, but it should be mostly automergeable <Keybuk> 9/10 times <Keybuk> maybe 99/100 too <sabdfl> pitti: source package size is not the issue <pitti> elmo: do you actuallly mean "strip the source package" or "strip during building the debs from the source package"? <Kamion> sabdfl: source CDs are a reality, various people want them <elmo> pitti: latter <pitti> sabdfl: ah, okay. I misunderstood <pitti> elmo: okay, that's easier <mdz> Keybuk: is it feasible to detect the case where we _only_ made this change, and fast-track it? <elmo> kamion: point them at > 650Mb ISOs, what do we care? <sabdfl> Kamion: they can be multi-cd sets <Keybuk> mdz: yah <sabdfl> dvd <Kamion> sabdfl: they're already 3 CDs <Kamion> sabdfl: turning them into 30 seems a bit non-trivial <sabdfl> true :-) <Kamion> (3 CDs for hoary as it stands) <elmo> Kamion: don't you like a challenge?? <Kamion> elmo: depends how fast you want auckland's disk space used up <mdz> last I heard, we weren't hurting for disk space <Keybuk> mdz: wish we were based on darcs rather than arch now though :p could have a "add dh_striplocales before dh_builddeb" change token :p <sabdfl> ok, girls, focus ;-) <Kamion> mdz: we will be if daily CD builds multiply by ten in size <mdz> Kamion: I don't think we quite need _daily_ source CDs <Kamion> mdz: we already have them, 'cos building CDs daily is about the only sane way to make sure they keep working <pitti> before we discuss the details, is there anybody who thinks that strip+separate download is _not_ the solution? <sabdfl> i don't think we need to keep them around though <mdz> Kamion: if it's only as a test, we can throw away the result if the space become sa problem <mdz> what sabdfl said <Kamion> mdz: true <Mithrandir> pitti: I wish they are handled by dpkg, but it seems people disagree with that, so. <mdz> so, considering that we must remove translations from .debs in order to maintain the One True CD, I think we only have two options for the "remove translations from .debs" piece <mdz> (source or binary) <pitti> Mithrandir: if you find a way to sanely handle translations in deb, tell it to us :-) <sabdfl> +1 source <mdz> source is more correct in every way from a pure technical perspective <pitti> Mithrandir: I would prefer proper debs from a cosmetic viewpoint too <pitti> mdz: binary is easier, but source cleaner <Kamion> source => change source package, binary => adjust binary package after build? <mdz> Kamion: correct <Mithrandir> pitti: they are just a bunch of files. Wrap them up in a deb; make a daily deb of all the files for a language if you so desire. <Keybuk> source++ <Kamion> definitely +1 source inasmuch as I don't get a TB vote <Keybuk> Mithrandir: "Dear Mirrors, hello, here's SOME DEB" <pitti> Mithrandir: either the mirrors or the number of debs would explode <mdz> however, source thrusts us into a situation of having modified several times as many packages as we currently do <smurfix_> hmm, if source, how do the stripped-off translations end up where people can download them? <mdz> so it is dependent on a greater level of merge automation <Mithrandir> Keybuk: one per language; the biggest language on my system is french, that is about 8MB, compresses down to about 2.7MB. <Keybuk> mdz: from 233 to "all of them" ? <pitti> mdz: we can modify debhelper to do it automatically <pitti> smurfix: but them into rosetta, I think <mdz> some have claimed that we would only modify the packages on the CD <Keybuk> Mithrandir: 200 languages, every day, half a gigabyte to each mirror each day <mdz> which I think would be about 3x as many packages as we currently modify <Mithrandir> Keybuk: only build if they have changed that day, then. <Kamion> I think modifying dh_builddeb or something around that level in Ubuntu might be just about acceptable <Mithrandir> Keybuk: I doubt we'll have changes to each language each day. <Kamion> although it's a bit worrying <Keybuk> Mithrandir: we upload source every day, leading to a language-pack change every day, for every language <mdz> Kamion: except for non-debhelper packages <mdz> Kamion: do you feel the same way about modifying dpkg-deb? ;-) <Kamion> mdz: yeah, pretty trivial number though <Kamion> mdz: I feel deep abhorrence for that idea; layering violation <Mithrandir> Keybuk: I thought translations were to be handled completely in rosetta? <mdz> Kamion: I agree, fwiw <pitti> Mithrandir: but you will have a daily change in at least one package, which would trigger a rebuild of the whole langpack <Kamion> debhelper is allowed to know about details of Debian policy as regards package layout; dpkg-deb is just an archiver * Keybuk tightens the chastity belt on dpkg-deb, just in case mdz gets any more funny ideas :p <Mithrandir> pitti: uhm, why would that be? <Kamion> so s/Debian/Ubuntu/ and it looks barely tolerable <Kamion> the problem is that that would break rebuilds of Debian packages <Kamion> which would be a bad thing <elmo> WRT debhelper, why don't we just declare it build-essential for us, then we can dh_strip_locale to any package even if it doesn't use debhelper? <pitti> Mithrandir: if you have one deb per language, it just is like this <mdz> Kamion: how so? <mdz> I think all source-based solutions share that problem <pitti> Mithrandir: if you have one deb per language per package, the number of debs explodes <Kamion> mdz: no they don't, modifying our source packages doesn't <Kamion> i.e. our source packages say what we want them to do <mdz> Kamion: the source package modification doesn't make it into Debian <mdz> you said rebuilds of Debian packages <pitti> mdz: I wouldn't change dpkg-deb, only debhelper <Kamion> mdz: uh-huh, but people rebuild Debian packages on Ubuntu <Mithrandir> pitti: I'm talking about one deb per language. And if we do get 200 languages, well, then we have half a gig extra data to push. <Kamion> mdz: e.g. backports <elmo> Mithrandir: dude, please read the logs for the last meeting, we went over this TO DEATH <Kamion> if some maintainer script in a Debian package does stuff to /usr/share/locale (and I believe some of them do), there'd be hell to pay <Keybuk> yeah, I think some people would panic if I uploaded dpkg with all the translations stripped out <pitti> Mithrandir: you have to push half a gig for _every_ language every day <Mithrandir> elmo: ok <Keybuk> I vote for dh_striplocales <Keybuk> or dh_stripmessages <pitti> Mithrandir: right now my /usr/share/locale is about 200 MB, but that will increase <Mithrandir> pitti: /query? <Kamion> Keybuk++ <mdz> ok <pitti> Mithrandir: ? <mdz> so the broad solution of "modify the source package" seems to be favoured <smurfix_> kamion:not if we modify gettext and put the rosetta translations somewhere else <mdz> within that solution, we have some choices <mdz> - modify debian/rules <mdz> - modify a debhelper tool (and, if debhelper is not used by the package, debian/rules also) <mdz> - others? <pitti> mdz: + modify rules <Kamion> - add a debhelper tool to simplify the modification of debian/rules <fabbione> - something that is absolutely shared by every package? <pitti> mdz: least surprise <mdz> Kamion: let's consider the debian/rules option under the assumption that we'll make the change trivial <mdz> by whatever means <mdz> fabbione: the only thing that falls into that category is dpkg-deb, and that's been declared evil <fabbione> mdz: aren't we? ;) <mdz> modifying debian/rules is going to mean adding a build-dependency as well <mdz> in order to guarantee a consistent result <Keybuk> mdz: or modifying build-essential * Kamion could live without the build-dep for the purposes of expediency ... <mdz> hmm, even to guarantee that it builds <mdz> modifying debhelper lets us forego the build-dep <Kamion> upload debhelper, wait for elmo/lamont to install it in all chroots, upload everything else <mdz> if it's built with a debhelper that doesn't contain the change, they just don't get a stripped package <elmo> modify build-essential .. <Kamion> if we add a new debhelper program then the build will fail if it's missing <Kamion> which is good enough <mdz> hmmm <mdz> and then the user is left scratching their head over why the package doesn't build <fabbione> lamont will kill somebody just the amount of mails he will get :-) <elmo> we need to ensure that if a user does 'apt-get build-dep gnupg', and tries to build ubuntu's source, it'll work <Kamion> mdz: hoary'll ship with this modified debhelper presumably <elmo> the only way to guarantee that is to modify our build-essential <mdz> we'd break building Ubuntu debs on Debian, unless our debhelper change goes upstream <Kamion> we could even ask joeyh to include the tool in mainstream debhelper <elmo> we could (haha) || true it, or enclose it in a if [ -e ... ] type test <mdz> I think he'd accept the dh_striplocales variety <Kamion> mdz: we've already broken building a lot of Ubuntu .debs on Debian mind you <pitti> Kamion: essentially it would just throw away any po files, right? <mdz> perhaps not the dh_builddeb variety <Kamion> mdz: plenty of them require Ubuntu-specific build-deps <mdz> Kamion: really? <Kamion> I think dh_striplocales is more correct anyway <Kamion> mdz: sure, take GNOME <mdz> I thought there were quite few of those <Kamion> mdz: or our d-i <mdz> d-i, sure <mdz> Keybuk: didn't you have some numbers on how many packages don't use debhelper in warty/main? <Kamion> might be worth ignoring all packages without translations <haggai> I think you should try to find a solution that is upstreamable so that co-operative Debian maintainers can add the tool call to their source to reduce the debian->ubuntu diff <mdz> 745/1022 I it looks like <mdz> Kamion: I think the plan is to translate those, too :-) <Keybuk> mdz: yeah, it came down about 30 don't <sabdfl> we have a hard limit at this stage on those which don't use gettext at all <Kamion> mdz: how about the ones without strings? :) <mdz> 745/1022 have a build-dep on debhelper <elmo> wow, that's less than I expected <Keybuk> mdz: build-dep-indep too <sabdfl> ok, i think we have an emergin consensus <mdz> grep-dctrl -FBuild-Depends,Build-Depends-Indep -nsPackage debhelper <mdz> 745 <Keybuk> mdz: grep-dctrl in hoary is broken <mdz> oh, sweet <Keybuk> grep-dctrl -s Package -FBuild-Depends,Build-Depends-Indep <Keybuk> debhelper /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_hoary_main_source_Sources <Keybuk> | wc -l <Keybuk> 952 <Keybuk> Actually it catches over 90%, and of the remaining 98 packages only 28 <Keybuk> contain translations anyway (using the "does it depend on gettext or <Keybuk> po-debconf" test). <pitti> sabdfl: I think so too, since the guys are already talking about specific details <Keybuk> -- <pitti> sabdfl: but I think the general strategy works and is the sanest one <mdz> given some magic which saves us from the crushing merge workload, the preferred solution for stripping locales is a dh_striplocales one <pitti> agreed <sabdfl> ok, next! <mdz> can we really take that for granted at this stage? <mdz> language packs are a hoary goal <mdz> this level of divergence almost requires that we can put most of these merges on full automatic mode <mdz> including the upload <sabdfl> mdz: let's evaluate that after the first week of Mataro Sessions <sabdfl> we'll have a handle on hct at that point <Keybuk> hct doesn't help any more than mom <Keybuk> a little less, in fact, because it requires tla/baz to play nice <elmo> it's worth noting, that we're not running out of space yet either, so they may be a goal but they're not a 100% we can't ship without this <sabdfl> yes <mdz> elmo: however, it is a high priority item on Mark's hoary list <Keybuk> mom works by breaking apart, mutating and driving patch itself <mdz> which is fairly close to the same semantics <Keybuk> we don't get that power with arch <Kamion> (we have 55MB free on the powerpc CD, which is tightest) <mdz> Kamion: and we need to get a ppc64 kernel image onto it :-/ <mdz> Keybuk: yet :-P <Kamion> mdz: should be OK ... <mdz> that'll eat about 15M <mdz> guestimating from the powerpc one <Kamion> mdz: depends which of powerpc/power3/power4 we need <Kamion> (can we bounty benh to fix the MMU crap that means we need those three? :-/) <elmo> can't we pay/beg/bribe/hold hostage someone to get rid of the stupid sub-arch split... doh <mdz> ok, so regarding stripping locales, we're fairly settled on dh_striplocales, on the assumption that Keybuk can provide the level of support we need for the merges <pitti> sabdfl: is there actually a "next" item on the list? <mdz> Kamion: let's talk about that a bit later <mdz> Kamion: doesn't sound unreasonable at first hearing <mdz> pitti: yes <mdz> the second half is the approach for providing the translations in separate .debs <mdz> it sounds like we were moving in the following direction there: <mdz> - buildds collect the translations via dh_striplocales and drop them into a pool <mdz> - some automated process runs periodically and builds .debs from the pool <elmo> debs are crack for this <mdz> - the .debs contain translations in some directory which is not /usr/share/locale <mdz> oh? was that decided at the last meeting? <elmo> I dunno, I'm just saying what I think <mdz> ok, s/\.debs/language packs/ for now <pitti> mdz: My idea is to have language-support-XX manage the download, install, and housekeeping <pitti> mdz: but not stuffing the po files into a deb itself <mvo_> I think the problems reamins the same whatever format we use <pitti> a reasonable compromise would be to build a deb on the fly on the user's machine <mdz> pitti: I think that introduces much more complexity than we want <pitti> mdz: but if we push debs, we have all these problems <mvo_> if we don't use deb we need to have package manager like functionality in a language-pack-get tool <pitti> mdz: assembling debs on the fly on user's hosts seems much more sane then <Mithrandir> pitti: I'm not too comfortable with that -- l-s-xx needs to be able to acquire, unpack, record what's installed and uninstall the files later. <mdz> pitti: I think that has the same problems <pitti> mvo_: but only downloading, copying and cleaning <mdz> there is a lot of value in having a language pack blob which actually contains the translations <pitti> Mithrandir: why record? <mdz> which can be placed on the CD, passed around by the user, etc. <Mithrandir> pitti: it needs to know what it has installed so it knows what it should update. <pitti> mdz: but the whole point of all this is to avoid debs in the first place <pitti> at least debs we provide on our mirrors <Mithrandir> elmo: you think .deb files are insane because of the mirror load? <mdz> it is? <Mithrandir> elmo: or anything else too? <elmo> Mithrandir: Packages files are useless for them and too large <elmo> and yes, I don't desperately want them in the pool proper <pitti> Mithrandir: not only because of the mirror load, also because of the download redundancy <pitti> The user should be able to download only the translations for packages he actually uses <mdz> elmo: don't desperately want, or desperately don't want? <haggai> could new functionality for registering conffiles etc with dpkg (Keybuk?) be used to register new translations with the dpkg database? <mdz> pitti: what it sounds like you're describing is a .deb which can't be installed without access to <someplace>.ubuntu.com <pitti> mdz: ? <mvo_> when will he download them? will he need to do a apt-get upgrade and a lang-pack upgrade? <mdz> pitti: <pitti> mdz: My idea is to have language-support-XX manage the download, install, and housekeeping <pitti> mdz: l-s-XX would download the translations, create a deb and isntall it <mdz> pitti: correct <Mithrandir> mvo_: think install from cd, for instance. <pitti> mdz: you need access to Rosetta, right <elmo> mdz: depends what format the debs are, one per source or one per lang? <mdz> that means that installing l-s-XX would require that the target system be able to talk to <someplace>.ubuntu.com <elmo> mdz: the latter can be over my dead or P45-ed body <pitti> mdz: but you need that anyway to get _any_ update <mdz> which, in a wide variety of environments, just won't be true <Keybuk> haggai: doesn't work, conffiles still need a replaces:/--force-overwrite <pitti> mdz: building the deb can happen on the client <smurfix_> mithrandir: so the files are on CD instead of being downloaded <elmo> mdz: the former.. I'm pretty unhappy about too, but I'd need to look at proper stats to be sure <mdz> pitti: that isn't true, users have lots of ways of moving around .deb files <mdz> pitti: the user needs to be able to insert a CD and obtain a language pack that way <pitti> mdz: they can build the deb on a place with net access <haggai> Keybuk: I didn't mean that. I meant, the ability to register a new file with the language-support deb <pitti> mdz: they can as well just copy the po tree to their computer (with the help of l-s-xx) <mdz> pitti: this defeats the purpose of providing a plug-and-play language pack <mdz> what is the problem that we would hope to solve with such a solution? <pitti> mdz: I still don't understand the problem <mdz> large downloads for people tracking the development branch? <pitti> mdz: where the deb is built, or who does it makes no difference <mdz> pitti: the problem is that it's essentially an installer .deb, and installer .debs are inherently problematic <pitti> mdz: we can put it onto a CD as well <sabdfl> we can accept a certain lag during the freeze <Mithrandir> mdz: large downloads for people who are running stable but want updated translations, I think. <sabdfl> so, say, langpacks are updated weekly rather than daily <mdz> sabdfl: right <mvo_> sabdfl: sounds good too me <sabdfl> i think elmo is right that there are benefits to avoiding the pool altogether for this <mdz> and we can provide a separate repository with daily updated language packs for those who really want them <pitti> guys, it is just insane to publish translation debs in regular intervals <smurfix_> why do we need .debs anyway? <sabdfl> as is mithrandir in that it's a small step towards reinventing dpkg :-) <mdz> if the problem is that the .deb is too large, we break it up <Mithrandir> .. you know, dpkg started out as a perl script. We could reinvent it in python. <sabdfl> but... fundamentally, a langmgr on an ubuntu desktop needs to keep track of the languages that desktop needs, and sync in the data <pitti> let the user choose which translations he want, at which intervals and give him an easy way to update his system with a "pull" strategy * Mithrandir hides. <sabdfl> it might even use rsync! <Kamion> Mithrandir: actually it was a shell script before that ... <Kamion> (IIRC) <mdz> sabdfl: why does it need to do that? <Keybuk> you could always use the deb + delta idea ... <pitti> mdz: what do you mean by installer deb? <mdz> sabdfl: the user selects their languages, and from that point forward it's a simple upgrade <mvo_> Keybuk: +1 <mdz> pitti: a .deb which doesn't contain the data that it is meant to provide, but instead downloads it over the network (or requires that the user do so) <pitti> mdz: with per-package debs or per-language? <mdz> Keybuk: not for hoary we can't <sabdfl> mdz: you mean, if we have an ubuntu-xx langpack in the pool? elmo hates that idea, and he's bigger than me, and i happen to agree with him <pitti> mdz: you can also install readymade translation debs <mdz> sabdfl: I'm going to go out on a limb and ask him to justify his hatred :-P <mdz> sabdfl: you, too, if you agree <pitti> mdz: it should be _possible_ to assemble them on the fly, not required <sabdfl> the langpack debs will drive the mirrors crazy <sabdfl> we can publish the translations as an rsyncable tree <Mithrandir> sabdfl: if they're only updated when they need to, or weekly or something as well? <mdz> as I said <mdz> if the problem is that the .debs are too big <mdz> then we break them up <elmo> mdz: it's utter crack. if you change one translation in one source package, this n hundred Mb unrsyncable (and not normally rsynce danyway) .deb changes <sabdfl> then dpkg/apt need to scale to more debs than they should have to <Mithrandir> mdz: then you get a huge amount of them. <Kamion> mdz: doesn't help, lots of them will generally change at once <pitti> ARGH <mdz> depends on how you break them up <pitti> this is the very same discussion we had last time <Kamion> mdz: either you have a source message which changes and lots of translations change, or you have a common translation of a frequently occurring string that changes <Kamion> mdz: you can't win either way <elmo> pitti: yes, I know. let's petition sabdfl to not let mdz go on holiday and miss TB meetings ;-) <pitti> pushing debs to mirrors is crack <sabdfl> elmo: only if you promise not to go on holiday too ;-) <smurfix_> I agree with sabdfl. Just use rsync and ignore all files belonging to packages you don't have installed. Problem solved. <pitti> smurfix: when using /var/cache/locale, we don't even have the possibility of hitting pacakge conflicts <sabdfl> smurfix_ it's easier than that, since the desktop install generally installs a solid well-defined set of packages <sabdfl> we only do this for those <mdz> I think you guys are optimising for the development branch case <mdz> when we need to optimise for the stable release case <sabdfl> yes, understood <mdz> these problems are non-issues for the stable release <pitti> mdz: you cannot force users to download huge amounts of debs for small translation changes <Mithrandir> sabdfl: you'd need to ship something to rsync off on the cd, then. That is a lot more work than shipping one more deb. <pitti> mdz: s/cannot/it wouldn't be the best solution/ <sabdfl> but i /really/ dislike the idea of switching something this big on only at release time, we discussed that last time <elmo> mdz: dude, this is not "optimization", this is "having a mirror network vs. not" <smurfix_> pitti: exactly. local packages use /usr/share. So check there too, and pick the newer one. <elmo> mdz: if we implemented this with even the number of languages/translations we have today, we'd have zero mirrors within a month <elmo> hell, _I_ wouldn't mirror us over the GB LAN :-P <sabdfl> elmo: i love how you tell the truth :-p <smurfix_> mithrandir: so park the rsync-master file tree onto the cd ..? <mdz> elmo: ok, I understand. let's try to change our approach to adapt to that, rather than throwing it away <Mithrandir> smurfix_: have you touched debian-cd? <Kamion> smurfix_: er ... uh-huh. <mdz> I'll say again that this is strictly a development branch issue <Mithrandir> smurfix_: doing that would be very icky and interesting. <Mithrandir> smurfix_: I do not think Kamion would like you afterwards. :P <pitti> mdz: so we need a system that is both able to install premade language debs, prepare these and also update directly from the network <mdz> and I do not think that we should sacrifice having the feature look the way it should in the final release <Mithrandir> mdz: I'm worried that elmo is so much against it, though.. <smurfix_> kamion: On the language-pack CD of course. Maybe packaged inside a cpio tree if too many small files are too icky. <Kamion> smurfix_: I think all this is highly premature <Mithrandir> smurfix_: that sounds like you're reinventing rpm rather than dpkg, then. <mdz> Mithrandir: the issues that elmo raises apply only to the development branch, and I think the use case is different enough that we may need to solve the problem in two slightly different ways <Mithrandir> mdz: mirrors mirror all, though <Kamion> smurfix_: language pack CD structures aren't something we've even started to think about yet, and I'm not convinced that they're something we ought to be generating centrally anyway <pitti> mdz: why, if stable users should be able to download their daily translation updates as well, it affects stable too <mdz> we are not going to force a huge delta increase on mirrors <Kamion> better to give people an easy way to build an Ubuntu CD for their language <elmo> mdz: dude, it's not just the development branch. what about a security update that changes a translation? <mdz> the sky is NOT falling <mdz> elmo: they don't <elmo> bullshit <elmo> you've uploaded new upstream versions before - it's rare, but it happens <Kamion> mdz: also security issues exist in translations; consider format string errors <mdz> if we have a clusterfuck like that, we put the changed translations into the security update .deb <smurfix_> Well, I'm brainstorming on "how could this work" rather than "how do we make .debs to make it work", so ... <elmo> mdz: and what Replaces: the langpack? <Kamion> and if the langpack is a .deb we then run into the non-commutativity of Replaces: <Kamion> (muttermutterdpkgbugsmuttermutter) <mdz> elmo: uses a different path <Mithrandir> versioned conflicts with the broken version of the langpack. (Yes, it's broken, but it works-ish) <elmo> mdz: and how do apps know which to use? <Kamion> Mithrandir: urgh <Kamion> Mithrandir: yay for removing langpacks while upgrading packages <mdz> elmo: they prefer the one provided by the package, if present (generally not) <Mithrandir> Kamion: doesn't apt/dpkg DTRT and upgrade? <smurfix_> ... or the one timestamped newer. <elmo> mdz: that technology exists for locales already? <mdz> elmo: if it doesn't, it seems completely trivial to implement <Kamion> Mithrandir: not if an updated langpack isn't available yet <Mithrandir> Kamion: then we have to make sure it's available. (Yes, I know that's not trivial either) <Keybuk> Mithrandir: dpkg does not upgrade on << conflicts. <Mithrandir> Keybuk: what about apt and dselect? <mdz> Mithrandir: depends <elmo> mdz: *shrug* ok then. so how/when do you propose to transition to the stable model as part of the development process? <mdz> depends on how we approach the development-branch model <elmo> I still think you're trying to fit a square into a round hole, 'cos the round hole is familiar and looks pretty, but that's probably just me <pitti> err, what is our "stable model"? and why it should be any different from the development one? <mdz> pitti: put yourself in end-user mode for a moment and consider the problem <mdz> "I want my Ubuntu to speak <language>" <pitti> mdz: stable versions do not have any fewer updates than unstable ones <elmo> oh, hang on <elmo> mdz this doesn't work <mdz> what's "this"? <elmo> mdz: one of the reasons for all this crack, is sabdfl's desire to have translation updates out of bound <elmo> i.e. in stable, still get translation updates <pitti> mdz: apt-get install l-s-<language> with a net connection works <elmo> so there is no stable model <pitti> mdz: if you don't have a net connection, make it easy to copy the files from another host which has <pitti> mdz: but that is orthogonal to the particular solution anyway <Mithrandir> elmo: push updated translation-debs to warty-updates once a week? <mdz> pitti: "you must have Internet access in order to do that, otherwise use this manual process"? <sabdfl> mdz: what elmo said. but not about the pretty familiar round hole <mdz> that answer won't fly for a lot of deployments <pitti> mdz: how do you want to download stuff without net? <pitti> mdz: it doesn't matter if you download debs or po file tarballs <pitti> mdz: "you" == the language pack which manages the stuff <mdz> sabdfl: what's the basis for updating stable with new translations, as opposed to doing the translations as part of the freeze process, as GNOME does? <elmo> Mithrandir: that has the "mirror admins lynch you" problem all over again <Mithrandir> elmo: yes, that might be a slight problem. <sabdfl> mdz: reality <mdz> pitti: there is a big difference between a packaging system which downloads packages, and a package which downloads .po files <Mithrandir> pitti: we end up having to reinvent dpkg to grab the translations off $MEDIA <pitti> mdz: we can include the current language snapshots on the CD too if we want <mdz> pitti: you open yourself to a lot of site-specific problems <mdz> outbound access policy, proxy authentication, ... <pitti> mdz: but we can use debs to hold the actual updates <mdz> pitti: and have the package copy them from the CD when it's installed? <mdz> then suddenly you need to implement this in apt <pitti> mdz: the point is not to avoid debs, but to avoid building and pushing them to our servers regularly <pitti> mdz: as I said, you can assemble the debs either on the fly or use premade ones <Mithrandir> would having
{'channel': 'ubuntu-meeting', 'month': 12}