#ubuntu-ensemble 2011-07-01 <_mup_> Bug #804120 was filed: add-relation between add-unit instances gives instances out of range <Ensemble:New> < https://launchpad.net/bugs/804120 > <_mup_> Bug #803855 was filed: peer hooks are not invoked <Ensemble:Confirmed> < https://launchpad.net/bugs/803855 > <hazmat> SpamapS, negronjl my first comment on the bug shows the output of the peer hook.. <negronjl> hazmat: checking the bug now... <hazmat> negronjl, relations aren't added between units but between services <hazmat> a peer relation is self-referential.. i gave the example in the bug of adding it for the tomcat service <negronjl> hazmat: my last post on the bug shows the error that ensemble returns when I try to add-relation between services <hazmat> negronjl, it shows adding a relation between units which isn't valid <hazmat> oh.. nm <hazmat> negronjl, i see now <SpamapS> hazmat: it looks like theres a relation with no services listed in zk <hazmat> negronjl, but that command does add the relation and the hooks do fire <hazmat> negronjl, i've succesfully run debug-hooks against the peer relation hooks in your formula, and watched them execute after i got gustavo's first email re <hazmat> negronjl, that error in status is a bug though <negronjl> hazmat: I really don't understand your position on this. I just showed you the command + output. What am I doing wrong here? <negronjl> hazmat: Are you running the latest version from PPA or something newer ? <hazmat> negronjl, i'm saying the relation is added by the ensemble add-relation tomcat6 tomcat6.. and that the hooks do fire <hazmat> negronjl, i agree the status command appears to have issues displaying peer relations <hazmat> negronjl, i also agree that peer relations should probably be auto added by deploy <hazmat> but my position is that peer hooks do work now <SpamapS> [zk: localhost:2181(CONNECTED) 7] get /relations/relation-0000000000/peer <SpamapS> {name: cluster, role: peer} <hazmat> SpamapS, what do you expect to see there? <hazmat> SpamapS, ls on that same node <negronjl> hazmat: I'll try again with the debug-hooks running but, the output of ensemble status is still a bug even if I see hooks running...I'll get back to you in a minute so I can try again <hazmat> negronjl, agreed re status <SpamapS> hazmat: two units <hazmat> SpamapS, as expected <SpamapS> hazmat: this code sees no "rel_services" in status... <hazmat> SpamapS, i think we've established there's a bug in status <SpamapS> rel_service = rel_services[0] <hazmat> ill fix the status bug now <SpamapS> # filter out 'self' <SpamapS> rel_services = [rsn for rsn in rel_services if rsn.service_name != <SpamapS> service.service_name] <hazmat> yeah.. its unfortunate that the status doesn't have any peer tests <hazmat> SpamapS, that code is from status i assume <SpamapS> yes, line 220 <SpamapS> removing it fixes it <SpamapS> But probably will cause bad output on non peer relations <hazmat> negronjl, did that work? <negronjl> hazmat: still spinning up. one more minute <hazmat> sounds like its been an exciting week, wish i could have been sprinting with you guys * hazmat looks at status <hazmat> SpamapS, thanks for the pointer <SpamapS> hazmat: its been a little nuts yeah <negronjl> hazmat: just checked it and it works but, status is busted <hazmat> negronjl, cool, i'm working on status right now, it will be in the review queue in an hr or two, but it probably won't get enough reviews to go in b4 next week <hazmat> the PWD for hooks to be the formula directory has been in review for a week as well.. <hazmat> negronjl, you could probably bug bcsaller and jimbaker to do a review and we can get it pushed tomorrow if you need to demo it <_mup_> Bug #804135 was filed: peer relations should be automatic <Ensemble:New> < https://launchpad.net/bugs/804135 > <hazmat> i'll see if i can get to that one as well <hazmat> tonight <negronjl> hazmat: I will. Thanks for the help hazmat <hazmat> negronjl, np... sorry if i seemed a little short... its just that i saw it working, i didn't understand what the issue was <negronjl> hazmat: no worries <hazmat> negronjl, SpamapS btw if you guys are doing a demo, its a little bit of a cheat, but in the bzr history, there's an auto-deploy rev, that will auto deploy dependent formulas and add required relations automatically when deploying <hazmat> very magical ;-) <SpamapS> hazmat: heh.. demo time is in about 7 hours .. I'm just going to demo my LXC branch. ;) <SpamapS> if I wake up in time <hazmat> SpamapS, nice <hazmat> SpamapS, get some sleep.. its late there i assume <SpamapS> hazmat: hacking on interfacing w/ cobbler. ;) <SpamapS> and yeah its about time to hit it <hazmat> SpamapS, via xml-rpc of cobbler? <SpamapS> hazmat: yep <SpamapS> lp:~clint-fewbar/ensemble/orchestra-provider <SpamapS> VERY VERY raw <hazmat> SpamapS, nice <_mup_> Bug #804199 was filed: Ensemble needs a way to pin a formula to a specific package version <Ensemble:New> < https://launchpad.net/bugs/804199 > <_mup_> Bug #804202 was filed: Ensemble needs to enforce configuration sanity <Ensemble:New> < https://launchpad.net/bugs/804202 > <_mup_> Bug #804203 was filed: Ensemble needs to communicate securely with Zookeeper <Ensemble:New> < https://launchpad.net/bugs/804203 > <m_3> http://paste.ubuntu.com/636296/ <m_3> negronjl: http://paste.ubuntu.com/636296/ <m_3> negronjl: http://paste.ubuntu.com/636296/ <m_3> sorry... irssi retard <_mup_> Bug #804226 was filed: Relation Get should show all elements <Ensemble:New> < https://launchpad.net/bugs/804226 > <_mup_> ensemble/expose-provision-machines r269 committed by rodriguezcrystal@example.org <_mup_> Watch service unit assignment on machines <_mup_> ensemble/expose-provision-machines r270 committed by rodriguezcrystal@example.org <_mup_> More guards <_mup_> ensemble/expose-provision-machines r271 committed by rodriguezcrystal@example.org <_mup_> Refactored tests <hazmat> zookeeper multi-node functionality lands <_mup_> Bug #804284 was filed: REST API for managing ensemble environments, aka expose cli as ensemble daemon <Ensemble:New> < https://launchpad.net/bugs/804284 > <_mup_> Bug #804327 was filed: Implement formula include <Ensemble:New> < https://launchpad.net/bugs/804327 > <_mup_> ensemble/expose-provision-machines r272 committed by rodriguezcrystal@example.org <_mup_> Robust testing without sleep on working with machines <hazmat> argh.. includes <jimbaker> hazmat, yeah, we need stacks :) <bcsaller> hazmat: bare metal deployments changes the timeline on the multi availability zone stuff. I am writing up the conversation I just had, we need to chat about this soon <hazmat> jimbaker, indeed, noted the same on the bug comment <jimbaker> bcsaller, so they want this to work in a high-latency env? <hazmat> bcsaller, not sure i see the relation between the two.. multi-availability zone is basically rack level awarness in physical terms.. a --location param seems to encompass both <hazmat> jimbaker, az are low latency xc communication <hazmat> 3-10ms, avg 5ms afaicr <bcsaller> hazmat: deploy openstack to create more than one Avail Zone <bcsaller> then use ensemble to manage services in that zone <bcsaller> we have new expectations at many levels of that deployment story <hazmat> bcsaller, cool, looking forward to the write up <bcsaller> hazmat: I am writing up a review of the AMI branch now, keep having to service interrupts though <hazmat> bcsaller, no worries, i'm dealing with the same (low throughput, kaleb hanging on my arm literally atm) <bcsaller> hi kaleb <_mup_> ensemble/status-w-peers r264 committed by rodriguezcrystal@example.org <_mup_> sattus works with peer relations <bcsaller> I'll review that today as well ^^ <hazmat> bcsaller, cool, thanks, i had some problems on some of the render tests, but they seem to predate the branch.. not sure, maybe missing i'm missing graphiz locally the error expressed oddly as unable to obtain zk handle <bcsaller> hazmat: if you have an old stack trace or can reproduce point me at it and I can take a look <bcsaller> I might not see it otherwise :-/ <hazmat> bcsaller, https://pastebin.canonical.com/49276/ <bcsaller> hazmat: and thats intermittent or not? <hazmat> bcsaller, constant <bcsaller> ok, looks like its not getting a handle to ZK in the dummy provider, that might indicate that something in setUp is failing <hazmat> bcsaller, i get it on trunk as well <bcsaller> I can take a look <bcsaller> doh! <hazmat> when just running the status tests <hazmat> bcsaller, hmm.. might be my local setup.. something seems strange <hazmat> lots of tests fail for me on trunk.. randomly.. i might have a bg zk <bcsaller> yeah, I don't get that, just ran the branch w/o issue <bcsaller> hazmat: thats what it sounds like to me <hazmat> bcsaller, cool <kirkland> hazmat: yo! <kirkland> hazmat: how goes :-) <hazmat> kirkland, wasup? any good irish explorations ? <hazmat> kirkland, pretty good.. hanging at the beach w/ my son, watching wimbledon, fixing bugs, but wishing i was in dublin <kirkland> hazmat: heading to scotland next week for some hiking and whisky tasting <kirkland> hazmat: nice <hazmat> kirkland, awesome <kirkland> hazmat: bcsaller says you've got the bug about ensemble requiring its own images <kirkland> hazmat: we're blocking a bit on that right now <hazmat> kirkland, yeah.. i fixed it earlier this week, its in review, ensemble will be able to use any cloud-init enabled ubuntu image <kirkland> hazmat: as we're developing formulas for cloudfoundry that require a) oneiric and b) much larger instances <kirkland> hazmat: rock! <kirkland> hazmat: can you point me to that bug/merge? <hazmat> kirkland, alternatively you can use the ./bin/ensemble-make-image script to make a 64 bit image for use with the environments.yaml config 'default-instance-type' <hazmat> and setting the image id <hazmat> kirkland, https://bugs.launchpad.net/ensemble/+bug/791501 <_mup_> Bug #791501: Ensemble should use standard amis and the ppa <Ensemble:In Progress by hazmat> < https://launchpad.net/bugs/791501 > <kirkland> hazmat: sweet, thanks <hazmat> kirkland, cutting out the use of bzr with the ppa, makes the initial install time about the same as a prebaked image (minus the bootstrap node which needs to install java) <hazmat> twisted takes a long time to install as well oddly. <kirkland> hazmat: nice <hazmat> kirkland, any other blocker bugs for you guys? <hazmat> kirkland, i'm working on auto adding relations for peers that troubled negronjl yesterday <kirkland> hazmat: hmm, well, we've filed a handful this week <hazmat> kirkland, well most of them seemed like wish list items <kirkland> hazmat: lynxman and negronjl have been filing them; i've been marking them "confirmed" and adding additional notes <kirkland> hazmat: i think our highest is the generic images/bigger instances <kirkland> hazmat: let me check the others .... <hazmat> the templating one is a wish list, the parallel ssh exec one is going to need some more thought <hazmat> security stuff i'm going to be making a big push on for most of july <kirkland> hazmat: good, security stuff will be essential before we can actually start pushing this to real customers <hazmat> kirkland, i updated that bug with a link to the security spec that forms the initial design <kirkland> hazmat: the templating one, yeah, wishlist; something we really need/want, so that we can define tunables and just put our selections in the config <kirkland> hazmat: absolutely necessary for formula reuse <hazmat> kirkland, absolutely.. the last branch for config/tunables is in review <hazmat> its had a rough time through review, but it should land hopefully next week <kirkland> hazmat: cool, that's more important to me than the templating itself <kirkland> hazmat: which we can handle as formula writers <kirkland> hazmat: but we need help from ensemble on the config side <hazmat> kirkland, sounds good <hazmat> kirkland, how so? <hazmat> beyond offering the configuration options on a service <kirkland> hazmat: heh, just that -- offering the config options and getting them through to the install hook <hazmat> kirkland, cool <hazmat> off to grab some lunch, bbiab * hazmat tries to tap the packaging experience in the room <_mup_> ensemble/expose-provision-machines r273 committed by rodriguezcrystal@example.org <_mup_> Test to verify that machine applies firewall policy after unit has been assigned #ubuntu-ensemble 2011-07-02 <_mup_> Bug #804892 was filed: Traceback when accessing relation data in relation-broken <Ensemble:New> < https://launchpad.net/bugs/804892 > #ubuntu-ensemble 2011-07-03 <m_3> 1 #ubuntu-ensemble 2011-07-04 <niemeyer> Hello! <hazmat> niemeyer, hallo! <hazmat> niemeyer, i think most of the state side people are watching fireworks today <hazmat> er. us <niemeyer> hazmat: Oh? <hazmat> niemeyer, 4th of july holiday, us independence day <niemeyer> hazmat: Ahhh, superb <niemeyer> hazmat: Congrats then! :) <hazmat> niemeyer, thanks, dc is a mad house already with security.. i dropped kaleb (my son) off at camp this morning, it was a ghost town besides road closures <niemeyer> hazmat: Crazy.. I'd expect that would be _the_ place for the celebrations <hazmat> niemeyer, i'll be hacking today.. how was your trip back? <niemeyer> hazmat: It was very good.. it was a new experience for me as well to go straight to a conference on arrival <hazmat> niemeyer, it will be this evening, crazy amount of fireworks in the area <hazmat> niemeyer, cool, was that FISL? <niemeyer> hazmat: Yeah <hazmat> niemeyer, cool, i've always heard good things about that conference <niemeyer> hazmat: It was pretty nice indeed.. was also very happy to have a talk on Ensemble that people understood, for a change. :-) <hazmat> niemeyer, indeed, being able to see and touch it, makes a big difference <hazmat> niemeyer, i've just been in bug-fixing and support mode the last few days, i'm going to setup deploy to auto add peer relations and take a look at the relation-broken bug <hazmat> that's on my plate for today, along with trying to push some security stuff <hazmat> niemeyer, what's the next sprint date? <niemeyer> hazmat: Monday (!) <hazmat> niemeyer, ??! <hazmat> niemeyer, i'm mia till july 23rd <niemeyer> hazmat: (The list of things sounds awesome) <hazmat> niemeyer, the review queue definitely needs some love... i think i've got reviews in all the items that i haven't put in <niemeyer> hazmat: I'll check the queue once I manage to handle some of the key conversations going on <hazmat> niemeyer, we might need to revisit the 2 reviews per branch.. it sounds like we're going to need more rapid turnaround.. and 50% team buy in per branch might be excessive atm <hazmat> niemeyer, sounds goodf <niemeyer> hazmat: Re. the sprint, it'll be about bare metal deployments <niemeyer> hazmat: In Miami <hazmat> niemeyer, i'm not going to be able to make any sprints till july 23rd while kaleb is in town. <niemeyer> hazmat: I'll attend, and will try to have William there <hazmat> niemeyer, cool re william <niemeyer> hazmat: No worries.. if you could be somewhat hooked in as you were last week, that'd already be awesome <hazmat> niemeyer, sounds good... when's the next next sprint ;-) <hazmat> is that austin in august? <niemeyer> hazmat: Austin, early next month <hazmat> cool <niemeyer> hazmat: Yep * hazmat grabs an espresso <niemeyer> Lunch time <hazmat> <_mup_> Bug #805554 was filed: Ensemble from ppa does not work with nova. <Ensemble:New> < https://launchpad.net/bugs/805554 > <kim0> Is it ok to file a bug to break the one service-unit per instance assumption? <niemeyer> Restarting due to kernel change.. <niemeyer> kim0: It certainly is, assuming there's nothing there yet <_mup_> Bug #805585 was filed: Break the one service-unit per instance assumption <Ensemble:New> < https://launchpad.net/bugs/805585 > <franciscosouza> niemeyer: ping <hazmat> franciscosouza anything i could help you with? <franciscosouza> last saturday I went to a conference where niemeyer presented an ensemble demo video, do you know where I can get/watch this video? * hazmat looks around <hazmat> franciscosouza, http://www.youtube.com/watch?v=AMHcy63wRL0 and http://www.youtube.com/watch?v=qxMhKbDSbOw <hazmat> are the two extant videos that i know of <franciscosouza> thank you, that's what I was looking for :) <_mup_> Bug #791042 was filed: *-relation-broken has no way to identify which remote service is being broken <Ensemble:New> < https://launchpad.net/bugs/791042 > <daker> any formula expert here ? <daker> or an ensemble expert :) <daker> my formula has state: install_error, how can i know what's going on with it, debug-log doesn't show any thing useful :/ <hazmat> daker, hmm.. yeah. there's an outstanding issue with debug log and install hooks (the log isn't active till after the debug hook) <hazmat> er after the install hook <hazmat> daker, so there are two options, using debug-hooks right after deploying to debug the install hook, or checking the on disk log of the unit agent to see what happened with the failed installed hook <daker> hazmat, the logs are located in ? <hazmat> daker, to check the on disk file.. ensemble ssh service_unit_name && cd /var/lib/ensemble/units/service_unit_name/ && cat formula.log (i think) <daker> ah ok <niemeyer> hazmat: It sounds like we should rethink this in the near term <niemeyer> Something like a rolling log, at least, might be nice <niemeyer> Maybe even better would be to have easier access to the on-disk log #ubuntu-ensemble 2011-07-05 <kim0> Morning <hazmat> sidnei, nice logo ;-) <kim0> :D <alienpulse> :) <kim0> alienpulse: so you were interested in writing an ensemble formula :) <kim0> welcome <kim0> alienpulse: we just had a private chat .. but let's continue talking over here <alienpulse> kim0, okay no prob. <kim0> so basically the best way to get started is to start reading the couple of tutoirals <kim0> https://ensemble.ubuntu.com/docs/user-tutorial.html <alienpulse> ill <kim0> https://ensemble.ubuntu.com/docs/write-formula.html <kim0> with getting started <kim0> maybe checkout the 2 vids we have to get a feel <kim0> then pick something you'd like to ensemblize <kim0> and start hacking <alienpulse> :) <alienpulse> thanks a lot .. <alienpulse> ill work on that ASAP ! <kim0> alienpulse: in every step along the way .. if you have questions ask here in this channel and ping me if you dont get an answer .. I'll always help <kim0> alienpulse: AWESOME :) <kim0> \o/ <kim0> woohoo <alienpulse> kim0, Cool :) <alienpulse> \m/ <kim0> alienpulse: once you decide which package you'd like to do .. let me know .. rock n roll <alienpulse> kim0, okay ill do .. i think im gonna go for Drupal <kim0> alienpulse: nice choice .. I've written the writing a formula tutorial on drupal .. so you can start from there <kim0> alienpulse: I didn't really link the formula to principia .. so you can do it <alienpulse> kim0, ohh okay <kim0> also, I used drush to keep things too simple .. but you might want to do it differently <kim0> Have fun <alienpulse> emm <alienpulse> ok <kim0> drupal is a good choice to start with :) <kim0> alienpulse: what's your LP username <kim0> alienpulse: I'll create a bug and assign to you so you :) <alienpulse> kim0, PM :) <kim0> alienpulse: just assigned :) <alienpulse> kim0, thanks a lot ... <alienpulse> :) <kim0> alienpulse: https://bugs.launchpad.net/principia/+bug/805955 <_mup_> Bug #805955: Formula needed: Drupal <new-formula> <Principia Ensemble:New for abedeltawil> < https://launchpad.net/bugs/805955 > <kim0> alienpulse: I think the best way, is to assemble the drupal formula from the tutorial, get it working, then start hacking on it to make it a "real" formula :) <kim0> enjoy <alienpulse> kim0, okay man ill do my best ,,, thanks again , <kim0> Awesome .. and remember for any questions .. we're all here for you <kim0> rock on <koolhead17> kim0, am not able to move anywer :( <kim0> koolhead17: don't feel bad :) <kim0> koolhead17: you're the best .. are you hitting a problem with phpmyadmin .. or just not enough time ? <koolhead17> i would say i need to learn bit more about dbconfig-common :) <kim0> the preseed thing ? <kim0> I dont really know much about that :) <koolhead17> hmm. i will try to nail it tonight <koolhead17> kim0, BTW people are coming up with nice Ensemble logo`s E for Ensemble :) <kim0> koolhead17: hehe yeah :) <_mup_> ensemble/auto-peer r264 committed by rodriguezcrystal@example.org <_mup_> ensemble deploy automatically adds peer relations. <_mup_> ensemble/test-api r236 committed by rodriguezcrystal@example.org <_mup_> a new test api for construction of environments. <_mup_> ensemble/auto-peer r265 committed by rodriguezcrystal@example.org <_mup_> riak peer rel test for deploy <niemeyer> Woot <hazmat> niemeyer, its in review now as well <niemeyer> hazmat: Awesome, thanks a lot <hazmat> niemeyer, for the relation-broken cli api.. i'm thinking just return empty values for relation-list, relation-get (except for self which returns valid values), relation-set is unclear, i think it should error <niemeyer> hazmat: It depends a bit on the use case <niemeyer> hazmat: I think Adam had in mind doing actions depending on the remote data, I'd think <niemeyer> hazmat: It's not clear if should handle this case, or advise to store any _needed_ information locally <hazmat> niemeyer, hmm.. there is no guarantee that remote data even exists at that point <niemeyer> hazmat: Yeah, that's the key point to ponder <hazmat> its assymetric cleanup across the relation endpoints <niemeyer> hazmat: The question is whether we should really do this or should allow querying of last available state <hazmat> niemeyer, access to local data is fine, and we could even store it (i'd prefer not to... but it could be valid).. the relation data is effectively dead, even if the relation is resurrected (it will be as a new relation) <hazmat> s/store/allow writes to local data <niemeyer> hazmat: Nah, that'd be weird.. it's a clean up really <hazmat> niemeyer, agreed <niemeyer> hazmat: We should talk to Adam to see what he's trying to do <bcsaller> niemeyer: he wanted to do things like remove grants in the local db when the remote side goes away <niemeyer> bcsaller: Morning! <niemeyer> bcsaller: bcsaller: Yeah, that's what I got from the bug description too <bcsaller> morning <niemeyer> bcsaller: The question is what is he trying to fetch from remote side <niemeyer> bcsaller: Why is relation-get relevant in this context? <niemeyer> Maybe all he needs is a relation id <bcsaller> yeah, I think he had to know which relationship was broken so he could map the data <bcsaller> I suspect thats correct <hazmat> niemeyer, hmm. clint put in a bug last month about wanting to get the service name of the remote side for cases like this <niemeyer> In which case we'd be fine with the already filed bugs <hazmat> so he could cleanup local resources for the remote side <niemeyer> hazmat: Yeah, I've linked the relevant bugs on Adam's bug too <bcsaller> niemeyer: we didn't talk about it at the rally but we should still talk about the bug where relations should get ids <niemeyer> hazmat: None of the other bugs require access to the data, though <hazmat> niemeyer, its unclear if the relation-id would always contain the relevant data <bcsaller> I wasn't clear on the use case in the sense that I didn't know which tools would take those ids to do anything useful <niemeyer> bcsaller: Yeah, we have to implement this <hazmat> niemeyer, ie. if the remote side specifically requested a particular principal/db name etc <niemeyer> bcsaller: There are a few use cases described in the bug <niemeyer> hazmat: Well, that'd be weird either way <niemeyer> hazmat: That kind of cleanup should happen locally <niemeyer> hazmat: Rather than trusting the remote data (which may have change) <niemeyer> d <hazmat> true, but its a communication protocol, i can definitely see cases where local cleanup might want remote state, although i think it could be argued that state should be copied local if its needed for cleanup <niemeyer> hazmat: Right.. it's a bit strange in that the relation was broken, and one wants to talk to the remote besides that. <niemeyer> Either way, let's look at the use case <hazmat> hmm... i think i need to add a generic file manifest to formulas to be able to do clean upgrades ( remove obsolete files) <niemeyer> hazmat: Interesting idea <niemeyer> hazmat: We probably don't need an actual file, though <niemeyer> hazmat: The content of the formula bundle is its own manifest <hazmat> niemeyer, yeah.. its computed on demand * niemeyer => lunch ftw <daker> hi can someone pls check this http://paste.ubuntu.com/638462/ * kim0 looks <kim0> oh more licensing :) * hazmat looks as well <hazmat> daker, unless your working at canonical and doing this for work, i'm pretty sure you need the canonical copyright statement, ie. is it an original work, on a not for hire basis <hazmat> er.. you don't need <daker> hazmat, you mean the first section ? <hazmat> daker, any of the sections that reference canonical copyright would be dependent on this being done as work for hire, contribution to ensemble itself would require copyright assignment per canonical policy, but formulas are effectively data inputs to ensemble, and are licensed/copyrighted by the author as an independent work. If the author is performing the work as a result of a fiduciary agreement, the terms of such agreement come into play. for mo <hazmat> st common situations this results in the work being considered a work for hire, with copyright assignment (and thus licensing choice) to the hiring entity. If you do work at canonical, i'm actually unclear what our formula licensing policy is.. ensemble itself is AGPL. <daker> hazmat, no, i don't do work for canonical <hazmat> daker, than your free to claim copyright solely for yourself if its an independent work, and license as you see fit, the canonical copyright statements are nesc in that case. <hazmat> UGH.. sorry are not nesc in that csae <hazmat> kim0, we should get this stuff in a FAQ and vetted <daker> hazmat, good now http://paste.ubuntu.com/638471/ ? <hazmat> daker, looks good <kim0> hazmat: thanks .. taking note and acting <_mup_> ensemble/formula-upgrade-removes-obsolete r264 committed by rodriguezcrystal@example.org <_mup_> dynamically computed manifests for formulas <hazmat> kim0, also added links to your screencasts to the wiki front page.. i don't see an easy way to embed them directly without a moin plugin. <_mup_> Bug #806037 was filed: FAQ needs updating for formulas licensing <Ensemble:Confirmed for kim0> < https://launchpad.net/bugs/806037 > <kim0> hazmat: Yeah doing that one too <hazmat> kim0, cool, i just finished adding them their near the bottom <kim0> ah ok <hazmat> it would be nice to have them embedded closer to the top <hazmat> but i think that requires some ops support, perhaps niemeyer would know more re the site setup and options for moin plugins <kim0> yes .. I looked at that before <kim0> it needs a moin plugin indeed <kim0> I think we're getting a new site that's more than a wiki <kim0> so that's probably enough for now <kim0> I saw people working on a new website during the past sprint <hazmat> hmm.. i think i'm making this way more complicated than needed (clean formula upgrades), i could just remove the whole formula dir instead of just the stale entries, and extract the new <kim0> hazmat: does that look good http://paste.ubuntu.com/638479/plain/ <daker> btw you can buy the cloud portal from me :D <kim0> Mostly punctuating your wording :) <kim0> daker: lol :) <kim0> hehee <hazmat> kim0, looks fine to me, but it would be nice to have this vetted by the org <kim0> hmm ok <serge_> jamespage: hey, trying out your new jenkins formulas. i've probably forgotten how to do it right, but i'm not seeing any plugins available? <serge_> (wanted to test with git) <kim0> hmm I tried robbie, but he seems to be OoO till next week .. was wondering who can approve this formula licensing question .. any thoughts are welcome <m_3> kim0: clint and dustin had some examples <serge_> (biab) <kim0> guess I'll just push the faq entry in the review queue for now .. then once vetted can be merged <kim0> m_3: thanks man <m_3> kim0: clint's updated mysql copyright... http://pastebin.com/X5EnhCxt <kim0> SpamapS: If you have a sample formula license for an independant contributor .. let me know so I'd link to it <kim0> m_3: problem is that it's copyright Canonical <kim0> m_3: which shouldn't be the case for most people <kim0> at least that's my current understanding <kim0> I'm basically trying to get approval over http://paste.ubuntu.com/638479/plain/ .. <m_3> kim0: yeah, that was my understanding... dunno who can approve <niemeyer> hazmat: There's a major revamp of the website front page coming <niemeyer> kim0 knows about it too <kim0> Yeah I told him <hazmat> niemeyer, nice <jamespage> serge_: are you just getting an empty list? <serge_> jamespage: yeah <serge_> (just commented in the bug) <jamespage> serge_: its probably unrelated <jamespage> sometimes the plugin manager can be a bit flakey <serge_> ok <serge_> so, i could add a relation, destroy the relation, and i can destroy the jenkins-slave service. <serge_> BUt what is the ensemble command to just remove one machine from a servcie? <jamespage> serge_: remove-unit <serge_> jamespage: d'oh! i was looking at the ensemble help output list, and somehow never saw that one <serge_> thanks! <serge_> is there an easy to set up distcc plugin? so i can fire up 3 slaves and get fast kernel compiles? <serge_> (that's a jenkins q, not ensemble :) <jamespage> serge_: nope - you would have to setup distcc sep and then call from jenkins <jamespage> SpamapS: around? <jamespage> serge_: BTW I added a bit of intelligence into the slave setup - it now calculates the number of executors based on the number of cores the system has <m_3> jamespage: SpamapS is out this week <jamespage> m_3: bah <jamespage> not sure why that did not show on my calendar <jamespage> oh - it did I just missed it :-) <m_3> jamespage: dunno... he was gonna have spotty coverage.. in-laws in central europe <jamespage> its no problem really <jamespage> I've been working on upgrading the zookeeper package in Debian/Ubuntu and wanted to catchup <jamespage> serge_: I can't reproduce your issue - I can see plugins just fine. <serge_> jamespage: excellent <serge_> jamespage: my new instance has plugins. yay! (wonddr what happened before) <hazmat> jamespage, awesome re zk, it sounded like the debian version was going to need a new maintainer (the old one bowing out), the latest release of zk is packaged in zk, its a pretty minimal diff to the packaging of the older version in debian <jamespage> serge_: hmm - sometimes it take a while to scan the update site <jamespage> hazmat: well I'm the new maintainer for my sins :-) have the package prepped and have done a basic test via PPA <kim0> jimbaker: hazmat bcsaller niemeyer SpamapS m_3 .. Could you guys please send me your take on Ensemble happenings of past week. Any decisions at the sprint? any new bits, formulas or bugs that you hacked, please send me asap so I can tell the world about it :) <kim0> This also goes to anyone in this channel .. let me know cool things you hacked <hazmat> jamespage, right on.. now i know who to bug ;-) * jamespage is looking forward to being bugged already :-) <jamespage> serge_: hey - one other thing - the python-jenkins package is in the new queue for oneiric <m_3> kim0: will do <kim0> m_3: Thanks man :) <jamespage> I've stuck it in a PPA for natty; but I guess once we can use oneiric AMI's we should update the formula to check * kim0 poinst to the dev team :) <kim0> points even <serge_> jamespage: cool. any roadblocks right now to its inclusion? <jamespage> serge_: I don't think so (fingers crossed :-)) <_mup_> ensemble/formula-upgrade-removes-obsolete r265 committed by rodriguezcrystal@example.org <_mup_> stale formula files are removed on formula upgrade. <m_3> hazmat: about removing stale formulas... <m_3> hazmat: what does that do to a rolling migrations? <hazmat> m_3, its more about removing stale dirents in a formula when upgrading <hazmat> m_3, nothing <m_3> s/migrations/migration/ <m_3> ah, ok... misunderstood then <hazmat> m_3, for rolling upgrades, i'm thinking its something that's more formula implemented with ensemble providing a cli api for something like leader election (which is also useful for other contexts) <hazmat> although that has implications in that implies potentially long running hooks. <m_3> hazmat: just was imagining a situation where there are multiple versions deployed at the same time <m_3> hazmat: yeah, we're still just hitting the surface of the tools (like leader-elect) needed by formulas <hazmat> m_3, indeed that is possible with the current upgrade scheme, each of the units is upgrade independently although concurrently in practice, we stop hook execution during upgrades atm <hazmat> m_3, yeah.. i wrote up some docs on more advanced upgrade scenarios, which aren't covered by the simple formula upgrade that's in place right now <m_3> hazmat: thanks man... still learning and need to test these scenarios <hazmat> https://ensemble.ubuntu.com/docs/upgrades.html <m_3> hazmat: cool thanks! <hazmat> m_3, the current upgrade mechanism is mainly about supporting the formula development/iteration story <m_3> hazmat: right <hazmat> at least that was the primary use case i was targeting, simple service upgrades are possible with it, but more complex ones will definitely find it lacking without additional support <hazmat> i think effectively we'll need to support perhaps some different policies, like rolling migration, or upgrade one with the rest stopped, etc <m_3> hazmat: yeah, I started thinking about this in your 'Service Upgrades' section <m_3> hazmat: with a rails app that pulled from source too... so even more complex <niemeyer> kim0: Maybe put it online somewhere so people can review what you have at the moment <niemeyer> kim0: I think you were present in most of the key decisions we've made <m_3> hazmat: agree about policies... until then, we'll try to produce example scenarios <hazmat> kim0, i've mostly been doing bug fixes, but all my work from last week is still in the review queue, the highlights are more robust handling of transient network issues, ability to use standard ubuntu images (really any cloud-init enabled ubuntu/deb image), hooks executing inside of formula dirs, peer relations automatically added on deploy of a service, fixes for ensemble status with peer relations.. that's all i recall atm <hazmat> m_3, yeah.. even a simple wordpress service with a db migration runs into this <negronjl> hazmat: most of the issues mentioned on your last post affect me. Any idea when will the changes go through ? <hazmat> negronjl, as soon as they get reviewed <negronjl> hazmat: lol... I guess the question is when will they be reviewed <hazmat> negronjl, i <hazmat> negronjl, btw end of the week at latest i'd assume, they would have gotten reviewed last week except for the sprint/rally i think <zul> what does this error mean? "2011-07-05 13:44:13,333 ERROR Connection was refused by other side: 111: Connection refused." <negronjl> zul: ensemble is not ready for you yet :) <hazmat> zul, typically it means ssh isn't active on yet on the remote side even though the machine is up <zul> negronjl: well duh :) <hazmat> kim0, oh and i wrote a security specification for how we'll be approaching security wrt to zk <m_3> zul: wait longer <zul> hazmat: so wait and try agian? <niemeyer> negronjl: Today <niemeyer> negronjl: I mean, they'll be reviewed today, at least once <negronjl> niemeyer: thx...that's blocking a few things on my side. <hazmat> zul, yeah.. which command where you running? i think we try to auto-retry most of the time <zul> hazmat: ensemble bootstrap <niemeyer> zul: This is a known issue, actually.. it should really be waiting by itself <hazmat> hmm... that's very odd <hazmat> bootstrap doesn't talk to any of the ensemble machines.. just the machine provider <kim0> hazmat: thanks for the list .. that's awesome :) <hazmat> zul, is this against openstack? <zul> hazmat: yep but there is somethign wrong with my install though euca-describe-images is taking a long time <hazmat> ic <zul> i think i found it <hazmat> niemeyer, re populating hostname automatically for relations, doing so effectively assumes its static, and if that's the case we might as well populate it directly to the machine state, so the unit agents can avoid having to query the machine provider <niemeyer> hazmat: Why does it assume it's static? <hazmat> niemeyer, hmm.. good question.. we don't really capture any event for the unit to correct it upon change <niemeyer> hazmat: If the hostname in a relation change, things should just work normally.. relation-changed hooks firing, etc <hazmat> niemeyer, the question is how do we know it changed to update the relation data <niemeyer> hazmat: That's a separate problem <hazmat> its effectively an internal detail of the machine provider <hazmat> afaik most of them give out statics for the lifetime of the instance <hazmat> subject to reuse upon termination <niemeyer> hazmat: It's not a detail of the machine provider.. it's a detail of the machine <niemeyer> hazmat: and the fact IaaS'es don't change it right now doesn't really affect the design <niemeyer> hazmat: It just feels like a pretty convenient way to get such information <niemeyer> hazmat: and one that allows for it to change as well, without any effort <hazmat> niemeyer, agreed, we can propogate it nicely upon change if know about the change, but it sounds like your suggesting we cron to check on the machine.. else we don't have a way of detecting and propogating the change <niemeyer> hazmat: I'm suggesting a design that allows for us to change it.. I'm not suggesting we address the change right now <hazmat> or cron internal to the agent <hazmat> niemeyer, sounds good, so unit agents directly pulling ip address from the machine and populating hostname for relation data when instantiating the unit in the relation <niemeyer> hazmat: +1 <niemeyer> Man.. I almost feel bad to say that now.. damn Google. <_mup_> Bug #806098 was filed: fact repository for Ensemble <Ensemble:New> < https://launchpad.net/bugs/806098 > <hazmat> is there a query syntax for launchpad bugs.. stuff like AND OR etc? <hazmat> out to run some errands bbiab <niemeyer> hazmat: Don't think so <niemeyer> hazmat: But there are some advance options <niemeyer> hazmat: See ya <niemeyer> Got a broken rodriguezcrystal@example.org.. the joys of assembling a house together. <kim0> broken pipe .. for a moment I was thinking SIGPIPE :) <kim0> it's an actual pipe <niemeyer> kim0: Hehe, yeah <niemeyer> kim0: A real SIGPIPE <kim0> hehe :) <niemeyer> hazmat: ping <niemeyer> hazmat_: ping <kim0> Here's our weekly report .. edits are most welcome http://pad.ubuntu.com/ensemble-week-5th-July <hazmat_> niemeyer: pong <niemeyer> hazmat_: Hey <niemeyer> hazmat_: Was reviewing the session-event-handling conversation again <kim0> I'll publish in an hour .. make sure your edits go there <hazmat_> unity seems to have issues for me with multiple screens, i'm currently in a VT on my laptop working on my screen session from my desktop.. sadness <hazmat_> niemeyer: awesome <niemeyer> hazmat_: Regarding your questioning on [4], what's your take on it? <niemeyer> hazmat_: I read your answer on it, but it's not clear what you have in mind to sort out the fact there's background logic that won't complete <niemeyer> kim0: Cool, thanks <kim0> great <niemeyer> kim0: The LXC stuff should probably not be there.. <kim0> too early ? <niemeyer> kim0: At least not with that wording <hazmat_> niemeyer: so there are two options, leave it as is.. or on fatal errors (session expired) propogate to both <niemeyer> kim0: People will want to try that, and it's not integrated yet <niemeyer> kim0: You can mention the experiment was successful and all <hazmat_> both being the connection level handler and the extant deferred <kim0> ok, I'll change to "work has started on ..." <hazmat_> niemeyer: the thing is we don't really expect the extant watchers to do anything except die, which makes for some additional unhandled error logging, or storm of such when those events happen <niemeyer> hazmat_: This is the problem.. <niemeyer> hazmat_: The logic simply _stops_. There's no chance for error handling or anything <niemeyer> hazmat_: Like, oh, if that doesn't work I'll log a warning, or.. if that doesn't work, remove the temporary directory, or anything really <hazmat_> the connection level handlers allow us to centralize the handling, which i see as a good thing, watch level error handling is going to be more error prone imo and its going to be subject to stampede effects <niemeyer> hazmat_: I can accept that as an ugly hack we're putting in place knowing it's a terrible way to program. <niemeyer> hazmat_: and with a known way out <hazmat_> niemeyer: how about i put an option to allow for either <hazmat_> ie. fatal session events propogated to watchers <hazmat_> those deferreds are effectively dead in that case.. the problem is that its tricky, the fatal error might come in response to a synchronous call not tied to a session event <niemeyer> hazmat_: But it's important to make it very clear that killing _any_ action is simply not feasible.. it's like crashing, except the application continues running <hazmat_> er. async call that is <_mup_> Bug #806184 was filed: peer interface relationship data not propagating <Ensemble:New> < https://launchpad.net/bugs/806184 > <hazmat_> hmm <negronjl> Disregard bug: 806184 <negronjl> I marked it invalid. <niemeyer> negronjl: Cool <niemeyer> I've been talking to hazmat_ for something like two hours already.. we'll write down some notes when we're done <_mup_> Bug #806241 was filed: It should be possible to collocate service units <Ensemble:New> < https://launchpad.net/bugs/806241 > <adam_g> is there any way of having a service not start after the install hook? other than having a noop install hook? <hazmat_> adam_g: ensemble is just executing the named hooks, what you do in them is up to you <hazmat_> adam_g: some packages automatically start the service/daemon after install a pkg <hazmat_> adam_g: you can switch them to be manually started with an upstart override and set to manual.. or just stop them after install (if its the package) <hazmat_> probably a few other ways #ubuntu-ensemble 2011-07-06 <adam_g> hazmat_: right. i was thinking more in terms of deferring executiion of start until some configuration has been completed <niemeyer> adam_g: Yeah, that'd be fine <niemeyer> adam_g: I mean, deferring start of the service to another hook <hazmat_> adam_g: definitely.. i do something similiar in the mongodb-config server, it only starts when there is a relation to a mongos server (else its just a regular mongodb server) <_mup_> txzookeeper/trunk r41 committed by rodriguezcrystal@example.org <_mup_> merge session-event-handling. Much improved session event and connection error handling with zk cluster tests. <niemeyer> It's past bed time here.. night all <koolhead17> kim0, <kim0> koolhead17: Morning <koolhead17> kim0, i don`t want to see some executive in Tie/Coat surfing cloud in the logo :D <kim0> koolhead17: lol :) <kim0> it's too early to choose hehe <koolhead17> cool, then :D <kim0> yeah * kim0 checks Inbox <koolhead17> hi et_ <et_> koolhead17: holla <koolhead17> notthing much <kim0> Hi everyone, just letting you know we're having the Ubuntu Cloud Days irc event on the 25th/26th. Everyone is invited to add a session at https://wiki.ubuntu.com/UbuntuCloudDays/Timetable Please add your session as soon as you can, if unsure about the title, just write TBD. Ping me for any details, thanks <niemeyer> Morning all! <hazmat> g'morning <niemeyer> hazmat: Hey man, how're things there <hazmat> niemeyer, pretty good, just got back from a long drive <m_3> morning gang <jcastro> kim0: do I need the PPA for oneiric? In oneiric itself python-txzookeeper is uninstallable <kim0> jcastro: r u on oneiric again ? <kim0> I was contemplating an upgrade :) <kim0> jcastro: yeah you generally want the ppa <kim0> jcastro: to get the daily builds <jcastro> ok <kim0> m_3: o/ <niemeyer> jcastro! <niemeyer> jcastro: Rock the world <niemeyer> fwereade: Welcome, officially! :) <fwereade> niemeyer: thanks! <niemeyer> Folks, William is joining the Ubuntu Server team to help developing Ensemble _today_! * fwereade waves shyly <fwereade> still figuring a lot of things out, but looking forward to the sprint <niemeyer> fwereade: and here comes your first code review <niemeyer> fwereade: https://code.launchpad.net/~hazmat/ensemble/auto-peer/+merge/66913 <niemeyer> fwereade: Can you please have a look at the change and see what you think of it? <fwereade> niemeyer: gladly <niemeyer> fwereade: Then just approve/needs fixing/comment <niemeyer> fwereade: The change is almost trivial <niemeyer> fwereade: But I'm sure you'll have to step back a bit and understand the context <niemeyer> fwereade: Check out the bug (linked from the merge request), and the documentation for more details about what was wrong <fwereade> niemeyer: yep, it'll probably take me a little while <fwereade> niemeyer: I'll hassle you if anything seems unduly confusing <niemeyer> fwereade: Absolutely <koolhead17> hello fwereade :) <niemeyer> fwereade: I'm stepping out in a bit for lunch, but I'll be around soon, and others are around to give a hand too <koolhead17> hi niemeyer kim0 <niemeyer> koolhead17: Yo! <kim0> koolhead17: hey :) <koolhead17> niemeyer, finish up your lunch. <fwereade> koolhead17: hello :) <kim0> fwereade: wow welcome :) <niemeyer> koolhead17: I have to start it first! ;-) <fwereade> kim0: thanks :) <koolhead17> kim0, i have changed content of my pycon talk, it will mainly focus on cloud-init and ensemble now :) <niemeyer> koolhead17: Wow, that's _awesome_ <koolhead17> niemeyer, :D <kim0> koolhead17: that sounds good :) If that'll make you work on your first formula LOL <koolhead17> kim0, yes sir!! <kim0> don't sir me man :) <koolhead17> kim0, :) <m_3> fwereade: welcome! <koolhead17> hi m_3 <fwereade> m_3: thanks :) <m_3> fwereade: just started a couple of weeks ago... feel free to ping me and I'll share the little I know! <m_3> fwereade: utlemming's new on the server team too <fwereade> m_3: awesome, I'll be taking you up on that shortly I think :) <hazmat> fwereade greetings and welcome <fwereade> hazmat: and thanks also :) <fwereade> hazmat: am I right in thinking you'll be in Miami next week? <niemeyer> bcsaller: I'm assuming your review on ~hazmat/ensemble/status-w-peers was an Approve <bcsaller> niemeyer: yes, sorry <niemeyer> bcsaller: Hey! <hazmat> fwereade, sadly not <bcsaller> hi :) <niemeyer> bcsaller: Wasn't sure if you were already around <niemeyer> bcsaller: Morning :) <niemeyer> bcsaller: Cool <bcsaller> barely <fwereade> hazmat: ah, shame :( thought I saw you on one of the related pages <bcsaller> trying to get some coffee down <hazmat> fwereade, there's always a next time, hope your enjoy your first sprint, i'll be participating remotely <fwereade> hazmat: cool, I look forward to it * niemeyer => lunch <_mup_> ensemble/status-w-peers r265 committed by rodriguezcrystal@example.org <_mup_> adjust comments per review. <jcastro> default-instance-type: accepts what as a setting in environments.yaml? I am assuming the instance API name right? t1.micro, etc. <_mup_> ensemble/trunk r265 committed by rodriguezcrystal@example.org <_mup_> merge status-w-peer [r=bcsaller,niemeyer][f=804120] <_mup_> Fixes a bug with ensemble status subcommand when displaying <_mup_> peer relations. <fwereade> ensemble in particular I guess ;) <hazmat> jcastro, indeed just that re instance-type.. m1.large, t1.micro.. eg. ec2 api symbols for instance-type <hazmat> s/eg/ie <jcastro> thanks, I'll add a note in the docs <hazmat> a question for formula authors in the room, which directory would you want your hooks to run in.. The formula hooks directory, the formula directory, or the root of the unit directory (the formula for the unit lives at '/formula' under the unit root)? <hazmat> jcastro, m_3, jimbaker SpamapS, fwereade, serge_ , kim0 ^ <fwereade> what else lives in the unit root at the moment? <fwereade> the formula directory feels natural to me but I am basically 100% ignorant <hazmat> fwereade, with lxc, the unit root will be the fs root "/" of the container <fwereade> ah ok <hazmat> at the moment, there's just the service unit log file and the formula directory in the root <fwereade> ok, the hooks directory feels wrong because... well, it's the formula's interface, and if we run in that dir we encourage people to clutter it up with other stuff <fwereade> I don't think I have a context to judge the distinction between running in the formula dir and the unit root <hazmat> fwereade, agreed, authors can include arbitrary library or configuration stuff in the formula, its probably nicer to address that from the root, then including a '..' everywhere or cluttering up the hooks directory <hazmat> er... s/root/ i mean the formula directory <fwereade> yep * hazmat makes it so * niemeyer puts aside his bricklayer cap and goes back to software development <m_3> hazmat: we've been starting from `pwd` <m_3> hazmat: i.e., not assuming it's anywhere in particular <m_3> hazmat: doesn't really matter with anything I've seen so far... pick one <hazmat> m_3, pwd isn't reliable atm <hazmat> it will be shortly <m_3> hazmat: great, we're starting to do more library sorta stuff <m_3> templates in separate files and such <adam_g> hazmat: id personally prefer it run in formula hooks directory. in the formulas ive written, im usually keeping some file that contains common stuff that i source in, and currently need to do some extra steps to get the location of that file <hazmat> adam_g, i think the notion is that what's in the hooks directory is the public interface of the formula, its fine to put that stuff in anywhere in the formula, you could put in the root of the formula, and still continue to source 'common.env' with the hook executing in the formula directory <hazmat> but perhaps executing it in the hooks directory is the most obvious/intuitive understanding of where it will be executed <niemeyer> adam_g: With the new approach it will be predictable as well <niemeyer> adam_g: You'll just have to source hooks/... instead of ./... <m_3> consistent good <kim0> my first impression was to be under hooks/ with my templates ..etc .. but sourcing hooks/* is fine as well <niemeyer> kim0: That's actually one of the reasons we're leaning towards the formula dir as cwd <niemeyer> kim0: templates should be under templates/, rather than hooks/ <kim0> indeed cleaner +1 <hazmat> but also probably a faq <hazmat> kim0, niemeyer how's this for an faq entry? http://paste.ubuntu.com/638989/ <niemeyer> hazmat: That's very nie <niemeyer> nice <kim0> hazmat: great indeed .. just a question do we have other "resources" other than templates <kim0> when you mentioned templates should be under templates/ .. I didn't know formulas should have a templates/ directory ?! <hazmat> kim0, libraries is the other one that comes to mind, but potentially also binaries <kim0> wonder if formulas should have a directory structures, where authors know where to put stuff <hazmat> kim0, they don't per spec have one, but formula authors are atm free to organize any resources needed by their hooks how they like <kim0> ah got it <hazmat> we might develop more best-practices with time as the formula collection goes, and have those enforced for an official formula distribution <_mup_> ensemble/debug-with-formula-dir r262 committed by
{'channel': 'ubuntu-ensemble', 'month': 7}