#ubuntu-cloud 2011-03-01 <HugoKuo> SC and CC terminated by init ~"~ <HugoKuo> ther fatal error is FATAL [SystemClock:SystemClockTimer] java.lang.OutOfMemoryError: Java heap space <HugoKuo> should I install bridge on CC+SC ? <HugoKuo> or just install it on NCs ? <TeTeT> HugoKuo: depends on your plans. If you only have one CC, bridge on the NC is good enough. If you have multiple CC's, they can utilize a bridge as well <kim0> Edulix: hey <kim0> Edulix: Can you scribe your name here https://wiki.ubuntu.com/UbuntuCloudDays/Timetable <kim0> for the hadoop session <kim0> that would be lovely :) <kim0> hallyn: what about a lxc session in the Cloud Days .. everyone I know of can't wait to learn more about it <kim0> :) <hallyn> kim0: yeah, I think so <hallyn> woudl then prolly want to do a screencast <hallyn> kim0: I'll just write it down as 'linux containers' now, may change the title if I think of a good angle <kim0> hallyn: awesome! :) <kim0> I don't even know what Daviey is working on, to suggest a topic :) <hallyn> man, ubuntu wiki does not like me the last few weeks <kim0> ew, can't edit ? <jamespage> I'm having trouble running cloud-init scripts on natty images in ec2 <jamespage> http://paste.ubuntu.com/574012/ <hallyn> no it lets me edit, but always sends me to an error page saying it couldn't. But, when i resend, it says it already committed <jamespage> any ideas? looks like something todo with the source archive.... <kim0> hallyn: same here .. it always saves the data <kim0> it's not just you .. the wiki hates everyone :) <hallyn> must be the WIKI TIKI (scooby doo) <kim0> hehee <pmatulis> my instance is not booting properly. it's having trouble contacting the meta-data service. troubleshooting ideas? <kim0> hallyn: you mentioned spice builds in ur ppa, but you didn't have time to test it ? <hallyn> kim0: right, they're up. I"ll get to testing them soon. you can certainly try. ppa:serge-hallyn/spice <kim0> hallyn: I got the virtio drivers from http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/bin/ <kim0> I wanna make a couple of blog posts about that <kim0> but it would be nice to have those come from your ppa as well <kim0> is it ok to have that iso and vfd packaged as well ? <kim0> or is that against the way we package things somehow <kim0> hallyn: is fedora it's a package 'yum install virtio-win' <kim0> s/is/in/ <hallyn> these are the windows virtio drivers? <kim0> yep <kim0> to have the boot partition over virtio disks <hallyn> you've looked at the license? totally ok to redistribute? <kim0> um no I haven't <kim0> :s <hallyn> i don't mind putting them in my ppa if they are unencumbered <kim0> hmm <hallyn> did you have to do anything fancy with them? is it less than 3 steps in your blog entry to use them from http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/bin/ itself? <hallyn> we might want to seek guidance from Daviey here :) <hallyn> maybe an empty package which just wgets those files from fedoraproject.org? <kim0> hallyn: nothing fancy .. you just boot windows installer, feed it the vfd driver, rescan, installs <kim0> then pass the iso, install the virtio-{disk,net} <kim0> hallyn: after that, I install spice et voila .. sweetness <hallyn> kim0: then maybe the simplest way (for the end-user) is to have them, fromthe windows VM, go to your blog, click on the URLs for those files from fedoraproject, and d/l. <kim0> hallyn: hmm, it seems those drivers are signed by M$, redhat paid them for that, it seems it's not gonna be easy to redistribute that, although I still dont exactly know <kim0> yeah, I guess I'll just put the paths, I don't really care <hallyn> kim0: if we were to package those in my ppa, they'd then still have to copy the files over to the windows partition, right? <kim0> well no <hallyn> oh? <kim0> the iso is mounted <kim0> by kvm <hallyn> oh <hallyn> what's the other file <kim0> vfd .. that's a floppy disk image <kim0> kvm mounts that too <kim0> and passes it to the installer <kim0> so it knows how to work with a virtio disk in installation <hallyn> ok. so we want to give them a 3-line shellscript that wgets the two files, then fires up kvm with those arguments (plus $1 as the disk iamge) <hallyn> kim0: how about you put that script as text in your blog right now, and once it's all finessed I can make a trivial package for it <kim0> hallyn: got it .. I think that's the way to go <hallyn> kim0: for now, lemme know if the packages work for you :) We wnat to pursue MIR asap if so... <kim0> btw, didn't find it in ftp://ftp.redhat.com/redhat/rhel/beta/6/x86_64/os/Packages/ <hallyn> (I'm off to finish some SRU proposals) <kim0> great <kim0> thanks a lot <hallyn> thank you :) <kim0> :) <jamespage> smoser: any chance you can help me out with a natty/ec2 issue (see above)? <smoser> W: Failed to fetch bzip2:/var/lib/apt/lists/partial/us-west-1.ec2.archive.ubuntu.com_ubuntu_dists_natty_universe_source_Sources Hash Sum mismatch <smoser> it is what it says <smoser> :-( <smoser> i've opened an IS ticket. i had only seen it in us-east-1. <jamespage> Seems the same in eu-west-1 as well - have not tested the other zones. <smoser> fyi, right now many mirrors are hosed <jamespage> mah - just when I was getting going with some automated testing <jamespage> is that just natty mirrors? I've not seen issues with maverick or lucid <smoser> i've seen this on maverick and lucid in us-east-1. i'm not sure about 'archive.canonical.com' wrt lucid or maverick. but i can't build an image with debootstrap at the moment from it on natty. <jamespage> nice <smoser> yeah, lucky its not like i need a build for an alpha "like right now" <jamespage> even nicer :-( <smoser> yeah maybe i should look at your testsuite rewrite <jamespage> its coming on OK - have now added support for running and monitoring multiple instances <jamespage> and a yaml based config file for running a big test. <jamespage> just figuring out how to plumb it into jenkins.... <kim0> smoser: ping, can you please fill a session on Thu https://wiki.ubuntu.com/UbuntuCloudDays/Timetable <smoser> jamespage, i just pulled lp:~ubuntu-server-iso-testing-dev/+junk/ubuntu-server-ec2-testing <smoser> kim0, what do you want me to talk about ? <kim0> smoser: whatever you're happy to ? rebundling images ? <smoser> TeTeT, are you planning on talking about the uec-persistency ? <kim0> smoser: I think he mentioned talking about that new excercise thing you guys did <TeTeT> smoser: yes, would love to, just waiting for mgr to approve <smoser> right. i just didn't see his naem there. (that is uec-persistency is what he is calling it) <TeTeT> smoser: or if you want to present it, fine for me! <kim0> there's enough sessions for everyone :) <smoser> no. <smoser> i've got to work on getting that better together and put a blog post up on it though. <TeTeT> ok, good <smoser> TeTeT, just fyi, i've got it working (the kexec-loader stuff) on the Eucalyptus Community Cloud (http://open.eucalyptus.com/CommunityCloud) <smoser> there are smoser-ubuntu-images and smoser-ubuntu-images-testing buckets there. <TeTeT> smoser: nice! <smoser> and, since they limit lifetime of buckets and such, all of it is scripted to re-populate <kim0> awesome :) <kim0> Edulix: ping <Edulix> kim0: hey! I was away, I just come back =) <Edulix> kim0: I'll do the hadoop session :P <kim0> Edulix: awesome :) <kim0> Edulix: can you edit the wiki, or should I do it <Edulix> I'll do it <kim0> Edulix: https://wiki.ubuntu.com/UbuntuCloudDays/Timetable <Edulix> kim0: I have no perms <kim0> Edulix: are you logged in <Edulix> yes <Edulix> ah no, I can edit it <Edulix> =) <kim0> :) <kim0> I always get confused as well :) <Edulix> kim0: Connecting Organizations with Multiple Availability Zones << sounds interesting <Edulix> for agora :P <kim0> Indeed sounds interesting, no idea about it though <kim0> if however is presenting that is around .. ping me :) <Edulix> whoops, Internal Server Error <Edulix> now <kim0> Edulix: refresh and it should be there <kim0> not uncommon :) #ubuntu-cloud 2011-03-02 <crazed> holy f <crazed> part-handler <crazed> there's some serious power here <crazed> i can write my own mime-type handler to handle chef's json attributes <crazed> if you include a part hanlder, does that ruin the pre-existing cloud-init support for various mime types? <crazed> hm having problems with user-data on the ubuntu 10.10 ami <crazed> http://ubuntu-cloud.pastebin.com/LwgRX710 <crazed> doesn't seem to set the date or anything really <crazed> cloud-init.log http://ubuntu-cloud.pastebin.com/n9rRgww5 <smoser> crazed, how did you make the multipart file ? <smoser> crazed, i'm wondering if it might not like your dos line feeds. <smoser> crazed, so, i downloaded your pastebin, then manually separated into 4 files (using vim), then <smoser> write-mime-multipart ./part-handler.py:text/part-handler ./node.json:text/chef-attributes ./solo.rb:text/chef-solo ./cloud-config.txt:text/cloud-config > ../user-data.txt.new <smoser> and launched with that user-data.txt.new as user-data-file, and i think it had the affect you wanted. <smoser> resultant user-data at http://paste.ubuntu.com/574292/ <smoser> crazed, but realize that 'output: <smoser> all: "> /tmp/cloud-init.log"' is only in natty <crazed> oh really? <crazed> thanks for the tips <crazed> smoser: i used write-mime-multipart the same way you did there <smoser> crazed, well, it did "work for me" when i tested it. <crazed> that's actually awesome <crazed> what AMI did you use? <crazed> or were you on euca <crazed> http://groups.google.com/group/ec2ubuntu/browse_thread/thread/8eb2f732e03b0334 <crazed> has more details of what i did <smoser> i did it on a private cloud. but used maverick <crazed> hmm <crazed> i'm going to give it another try <crazed> also i did notice that write-mime-multipart wasn't in the bzr repo under tools anymore? <crazed> i wound up running it from a machine that had cloud-init <crazed> 0.5.15-0ubuntu3 is the version of cloud-init i'm running <smoser> its in cloud-utils now <smoser> crazed, <smoser> and fwiw, do not write log to /tmp <smoser> or the tmp cleaner will clean it <crazed> hm alright, i was just seeing if it would work <smoser> ie, you'll start writing there early in boot, and then aprocess that does an 'rm -Rf /tmp/*' (effectively) will remove your file <smoser> that wont work for 10.10 <smoser> timezone, final_message, and output are all 11.04 only. <crazed> ah <crazed> where is the output of the parthandler print messages supposed to go? /var/log/cloud-init.log? <crazed> in 10.10 that is <crazed> hm i've copied your multi part file, still don't get the part-handler to run it appears <crazed> is there a verbose way of initiating the cloud-init run, without relaunching instances? <photerran> smoser: i've been using cloud-config to bootstrap machines in EC2 for the last couple months, and just ran into an issue yesterday for which i'd like to offer some feedback. it seems the security.ubuntu.com repository is overwhelmed with traffic (since at least yesterday morning) and this causes cloudinit to die while waiting to connect (for cloud-config's apt_update or apt_upgrade). Using the EC2 mirror for the <photerran> security repository seems to work just fine, but apt_mirror has no effect on the security repository location. <photerran> besides that one small issue cloudinit & cloud-config are great and I'm excited about the new features coming in natty, thank you! <crazed> they do seem pretty powerful, just wish i could get my part-handler to work :) <kim0> crazed: I had written this on multipart handlers, might be helpful http://foss-boss.blogspot.com/2011/01/advanced-cloud-init-custom-handlers.html <crazed> kim0: that's where i got most of my info from <crazed> very good stuff there <kim0> hehe, ok glad then :) <crazed> i see the part handler getting output to /var/lib/cloud/data/plugin <crazed> but it doesn't seem to actually run <crazed> i see nothing in the logs, so i'm not sure what's goin gon <crazed> OH <crazed> it's in teh console output <crazed> ah my python syntax is off lol * crazed has been using too much ruby lately * kim0 likes snakes not gems <crazed> i like chef for config management <crazed> python is pretty nice though <photerran> kim0: just wanted to add i've also learned a lot from your blog, thank you <crazed> oh wow you're in egypt? <smoser> crazed, i often do 'rm -Rf /var/lib/cloud && reboot' <smoser> if you want to change your user data, then you have to use the 'seed' stuff <crazed> hmm <smoser> https://groups.google.com/group/ec2ubuntu/browse_thread/thread/d4d51238a2afb55b <crazed> oh i did see that thread too <crazed> should have read it :) <crazed> smoser: in 10.10 is it possible to get script output elsewhere besides the console? or is that natty only at this point <crazed> just takes amazon more time to display the output than it does for me to login and check files <smoser> yeah, its annoying. <smoser> in 10.10, any script or program you run, just have it redirect its own output to a file elsewhere rather that writing to stdout <smoser> ie <smoser> #!/bin/sh <crazed> ah i know what you mena, that's what i was doing originally <smoser> exec >/root/my.log 2>&1 <crazed> then i had this idea to separate the node attributes file from the script <crazed> to make things more portable <smoser> photerran, hm.. i don't really knwo what to say about that. <crazed> yay part-handler is now working for me <smoser> yesterday the security mirrors were *really* slammed <photerran> smoser: indeed they were. i was thinking it would be nice to be able to set the security repository location in cloud-config (apt_mirror_security?) or just using the EC2 mirror as is done for the other repository locations... <crazed> smoser: is there an order in which things are ran? part-handler, user-scripts, etc <smoser> apt_mirror_security might be ok, with an explicit warning on it. <smoser> photerran, the reason they don't do that is that they want security fixes out as fast as possible <smoser> and mirrors lag <smoser> crazed, part-hander is "very early" . actually while cloud-init is running and consuming metadata <smoser> user-scripts are "rc-local" like <crazed> awesome <crazed> that should work out how i want then <photerran> imho, using the EC2 mirror would get the updates out faster than this <smoser> part-handlers can block boot entirely <smoser> just fyi. <photerran> maybe all the EC2 machines out there are what is causing the extraordinary congestion on security.ubuntu.com <crazed> smoser: how so? <kim0> photerran: oh thanks! <smoser> photerran, right, but it takes time to propogate to the mirrors. <smoser> its originally published to security, then the mirrors sync from that <crazed> hmm damn it <crazed> missing ppa <photerran> ah i see. do you think there's any chance for getting the updates published to the EC2 mirrors when they're published to security? know where I might go to suggest that to the right people? <smoser> crazed, they can hang block boot, ie by 'sleep 3600' <smoser> but i fyou stack trace in one, it shouldn't screw up anything else in cloud-init <smoser> did it ? <smoser> ie, if cloud-init did not catch the exception, please file a bug <crazed> oh no it didn't screw up anything else <crazed> i just wasn't seeing my syntax errors <crazed> since they were in the console output, i didn't think to check there * kim0 rings a bell, Community meeting starting in 2 hours. All welcome to hangout and chat <crazed> what's that? <kim0> crazed: A chance for everyone to get to know everyone else <kim0> chat about how they're using ubuntu/cloud <crazed> where, here? <kim0> share info, have a few laughs :) <kim0> right here <kim0> in 2 hours <crazed> oh <crazed> i'll be here <kim0> awesome <kim0> photerran: You're welcome to attend as well <kim0> Everyone is of course :) * kim0 jumps back to finishing his virtio blog entry <photerran> tyvm! i intend to. <kim0> awesome :) <crazed> virtio! <crazed> what are you writing about it <kim0> crazed: just getting Windows7 installed over KVM with optimized virtio disk and network drivers <crazed> ah alright <crazed> http://ec2-72-44-58-95.compute-1.amazonaws.com/ <crazed> :) fully configured using cloud-init, chef-solo, and drush <kim0> rock n roll :) <crazed> seriously impressed with this <crazed> now to try and scale it out using cloud formation <mrjazzcat> crazed: would love to hear your thoughts about the tools and process to get this running, if you have the time and inclination to write something down. <crazed> i'll be writing a blog post <crazed> probably tonight <patrickw_> Anyone know of problems installing sun jdk (6u24, 64bit, rpm format) in a chrooted loop device? I've been trying to get this installed but it hangs the loopback and I can't umount... <mrjazzcat> crazed: awesome, I'll look forward to that. Can you point me to your blog so I can look for it? <crazed> mrjazzcat: http://allanfeid.com <mrjazzcat> crazed: thanks! <photerran> patrickw_: are you installing right from the partner repo? * kim0 rings a bell .. Cloud community meeting starting <kim0> :) <kim0> any new faces around <kim0> let's all get to know each other <kim0> now everyone is silent hehe * photerran is new here <kim0> photerran: hey o/ <photerran> nice to meet you all <kim0> let's get to know each other better <kim0> can you introduce yourself please <photerran> well you can call me photerran, i'm a sysadmin and lately have been working on building a hosting platform in EC2 using ubuntu <kim0> building a hosting platform on top of ec2 ? <photerran> indeed <kim0> interesting stuff :) <kim0> photerran: do you offer VMs directly to your customers <photerran> no we provide a web service to ad agencies and i'm working on a new platform to run our custom application <kim0> hehe <kim0> awesome :) <kim0> crazed: howdy o/ <photerran> i'm using cloudinit to bootstrap puppet to manages services & apps, and using glusterfs (on top of EBS and instance stores) to provide shared storage for different parts of the app <photerran> yeah it is awesome <photerran> really beats having to roll my own AMIs :) <kim0> indeed it does hehe <kim0> hmm that stuff sounds interesting <kim0> photerran: have you heard about cloud days <photerran> yeah about two hours ago hehe <kim0> hehe <kim0> ummm <kim0> photerran: what about giving a session <kim0> sharing the knowledge you've gained building that system basically <kim0> it's nothing hard or complicated <photerran> would this be over IRC? <kim0> Yep <kim0> just like this ;) <crazed> hmm <crazed> interesting <kim0> photerran: actually, if you join #ubuntu-classroom <kim0> there's a session going on right now <kim0> just to get a feel <kim0> it's part of Ubuntu Developer Week <kim0> cloud days is very similar to that <kim0> crazed: hey .. what's the interesting part :) <photerran> thanks i'll have a look <crazed> using gluster on ec2 <crazed> how's the performance with that? <kim0> the first thing that came to my mind as well :) <kim0> hehe <photerran> do you want the long or the short answer? lol <crazed> both haha <kim0> LOL <photerran> short answer is "not too bad" <photerran> long answer is... <photerran> it's latency sensitive, so performance increases as file size increases <crazed> hmm <photerran> if your files are 100kB+ then performance should be OK, but I'm using front-end caching in apache & varnish to make it better <crazed> what excatly are you storing on gluster? <crazed> static content or? <photerran> also, the speed seems to be limited by the EC2 LAN, not the instance size, so spreading the gluster volume out over more machines (and using the native gluster client) can get better aggregate throughput even tho single transfer rate may not be all that great <photerran> this is all very preliminary btw <photerran> YMMV of course <photerran> storing web content, HTML, JS, CSS, JPG, etc... the usual stuff <kim0> photerran: wonder if you can aggregate many little files into one bigger file to use the increased throughput <crazed> any reason you're not using s3 for that? <kim0> can't think of a way though <photerran> crazed: i'm a sysadmin not a developer. thats a requirement of the app <kim0> photerran: did you play with ceph as an alternative <photerran> i read about it but it seems too new to bet the farm on <photerran> my conclusion at the time was "maybe when btrfs is ready for production" <smoser> kim0, nice work: http://cloud.ubuntu.com/ami/ <smoser> the ap-northeast-1 images are there. <photerran> also i'm not too excited about having a metadata server... gluster is fully distributed <smoser> you beat alestic.com to it :) <kim0> hehe <kim0> smoser: oh northeast .. that's the Tokyo one * smoser gives erichammond a rasberry <kim0> cool! <crazed> photerran: fair enough, same here actually. we deal with managing applications in the cloud, and typically will steer away from any shared filesystem usage as s3 gives you the added benefit of faster image loads (using asset host) and the ability to use cloudfront as a basic CDN. usually we will help teh devs understand what needs to be done an help migrate <smoser> kim0, yes. all our released images are there (not milestones though) <kim0> awesome :) <smoser> it sure seems to me like you're not caching though <kim0> smoser: you mean performance wise ? <smoser> yeah <smoser> just seems like "processing" <photerran> crazed: yeah we'd like to move in that direction, but we decided to migrate the legacy app first & develop those new features later <kim0> well .. it's a chance to have you look at the code <kim0> momento :) * kim0 prepares to be embarrased * smoser apologizes for interupting a conversation <crazed> lol <photerran> no appology needed <crazed> either way, it's cool to see people doing things with puppet <crazed> which i see is supported in cloudinit <crazed> personally i'm a fan of chef, so i've been working on getting a solid solution using the part-handler parts of cloudinit <crazed> only cool thing i have so far is the ability to launch a full dev environment of drupal 7 with any number of modules installed <photerran> well thats a start <crazed> it is, i have a decent chunk of chef cookbooks which i should be able to tie into relatively easily <crazed> then put that together with cloudformation to bring up larger environments, but i'm not sure how to handle making sure the various pieces exist <crazed> for example, there's a race condition, what if my database isn't up before i try to connect to it <crazed> i suppose i could just add some retrying in there <photerran> i have yet to try cloudformation, but it looks nice. i'm hoping for better documentation before I dive in <kim0> crazed: I'm interested in what you describe as a race condition <kim0> so basically .. chef configs a bunch of machines <kim0> and starts a bunch of services <crazed> yes <kim0> but you have no way to ensure dependencies right <crazed> exactly <crazed> for example in a cloud formation stack, i have an RDS instance and an ec2 instance <kim0> nor pass info between machinse (DB passwords ..etc) <crazed> the ec2 instance is going to try and populate the rds instance database <crazed> but if it doesn't exist, that will fail <crazed> i added basic cloud formation support to fog recently, so i've been playing around with it <crazed> seems cool, but would really like to see some sort of control on how to make sure resources of the stack are available <photerran> hmm the main cloudformation web page suggests there is a way to do just that <photerran> under the "Easy to Use" section <crazed> really? <crazed> hm <crazed> i've only got some basic cloud formation templates <crazed> there's a lot of syntax there <photerran> btw, the cloudformation main page i mentioned is http://aws.amazon.com/cloudformation/ <kim0> haven't checked that out yet .. do you have to write that template yourself <crazed> yes but there's a few of pre-written ones from amazon <photerran> it's brand new, announced on Friday, barely a week old * kim0 tweets the new Tokyo Ubuntu images â http://twitter.com/#!/ubuntucloud .. Thanks smoser <kim0> crazed: photerran if you guys are interested .. follow that twitter account <photerran> I havent yet jumped on the twitter bandwagon, but when i do, i will follow <photerran> until then i just check in on the web every once in a while <crazed> https://s3.amazonaws.com/cloudformation-templates-us-east-1/AutoScaledPHPWebServerWithMultiAZRDS-1.0.0.template <kim0> photerran: are you on the facebook bandwaggon ? hehe https://www.facebook.com/ubuntucloud <photerran> nope * kim0 hugs photerran :) a clean sole <crazed> haha <crazed> still looking for this ability to depend on other resources.. bha <photerran> the whole shoe is clean <photerran> j/k <crazed> https://s3.amazonaws.com/cloudformation-templates-us-east-1/Drupal-NoSSH-1.0.0.template <photerran> crazed: do either of those templates show how to declare an EC2 -> RDS dependency? <crazed> not that i can see <patrickw_> photerran: I'm actually installing into a centos image, thus the rpm <crazed> hm it might be handled by some scripts on the AMI they're using <crazed> because i don't see any user data scripts to install drupal either <crazed> just basic userdata here <photerran> hopefully AWS will publish better docs soon, i suspect there's more to cloudformation than these templates show. you might want to ask the question on the AWS forums in the mean time though <kim0> crazed: man, consider giving a session on chef automation as well .. sounds damn interesting :) <photerran> kim0: how often do you do these community meetings? <kim0> photerran: weekly same time <kim0> it's just a time for everyone to get together and talk <kim0> photerran: and crazed .. it would be great to mark your calendars <kim0> and try to make it every week <kim0> I'm trying to build some momentum around ubuntu/cloud which is for everyone's benefit really <photerran> ok cool. i like it. i've been using ubuntu a long time and am just now starting to get involved with the community. <photerran> until recently i've been using ubuntu personally & centos/rhel professionally, but that's changing as I mentioned earlier * kim0 was a centos junkie for about 5 years ago <kim0> but I've been building some consistent respect for ubuntu server! <kim0> lost of innovative stuff really .. <kim0> and since we're still a small community .. every contribution counts :D if you're thinking about registering a session, "just do it" hehe <photerran> yeah its exciting. my experience so far with ubuntu on EC2 has been very positive until yesterday when the congestion on security.ubuntu.com got in the way of spinning up new instances * kim0 nods <kim0> smoser: any idea why do those mirrors melt .. it seems it has happened a few times already <kim0> can't we balance on a few <smoser> $ host us-east-1.ec2.archive.ubuntu.com <smoser> us-east-1.ec2.archive.ubuntu.com has address 502.619.2016 <smoser> us-east-1.ec2.archive.ubuntu.com has address 502.619.2016 <smoser> us-east-1.ec2.archive.ubuntu.com has address 502.619.2016 <smoser> us-east-1.ec2.archive.ubuntu.com has address 502.619.2016 <smoser> i have a ticket open. they should not (as you say) melt. but yes, they have. <photerran> it was the default location for the security repo (in sources.list) that got me, as i mentioned to smoser earlier <photerran> changing it to the EC2 mirror works great, but that's not easily automated when cloudinit it bootstrapping an instance for the first time <photerran> is bootstrapping* <photerran> fwiw, in my case i'll be removing the apt_update & apt_upgrade from my cloud-config, leaving only the puppet agent stuff in there, and doing the apt_update & apt_upgrade later on in the process with puppet, after puppet has updated the sources.list <smoser> photerran, the easiest way to do it would be to do dns trickery <smoser> you can write security.ubuntu.com into /etc/hosts before apt comes through and updates. do that with a 'boothook' on maverick <smoser> this is not tested. but i think it would work. <photerran> ooh good idea, thank you <smoser> and if you try to intsall a package (puppet), then it will do an update. <photerran> ah yes that would have bitten me soon enough <crazed> kim0: i don't mind doing a session on chef automation, though i'm by no means 100% expert at chef <crazed> there's a lot to know about it <crazed> i use chef-solo, mostly because we don't have a subscription ot the opscode platform and the opensource chef-server doesn't support multiple clients that well <kim0> crazed: a pretty basic session would be more than enough <kim0> crazed: that idea is to introduce the concept and answer questions <crazed> i could handle that <kim0> the time doesn't really allow for anything close to advanced :) <kim0> crazed: awesome! <kim0> do you have a wiki account <crazed> i do <crazed> actually <kim0> crazed: edit https://wiki.ubuntu.com/UbuntuCloudDays/Timetable <crazed> now i have to remember my wiki account password :) <kim0> hehe lol <crazed> oh this is launchpad <crazed> hm i think i know that login <kim0> :) <kim0> the wonders of the human memory <photerran> oh yeah that reminds me, what are the requirements to be a presenter? wiki account... anything else? <kim0> photerran: not even that :) <kim0> I could edit the wiki for you <kim0> crazed: could you email me lauren32@example.com <photerran> oh nice <kim0> photerran: you too please if you'll be giving a session <kim0> this is just so that I can remind you and coordinate <kim0> thanks guys <crazed> kim0: sure what kind of email? <kim0> crazed: just "hi" :) wanna get your email <kim0> photerran: so do you have a wiki account? have you decided on the sesion thing yet ? <kim0> photerran: scalable cloud web app would be great :) <photerran> no i dont have an account, and yeah i'd like to do a session. <kim0> awesome <kim0> photerran: what's the session title * kim0 will edit <photerran> not sure how much detail i can go into, i'll talk it over with my team leader & let you know in my email what I'll cover <photerran> give me an hour or two to get it sorted out <kim0> great .. again, there's usually not much time for lots of details .. it's mostly laying down the basics and answering qquestions ..etc <kim0> so nor worries <kim0> no* <kim0> photerran: thanks for the contribution <photerran> ok great, it's my pleasure <kim0> all mine :) <crazed> now i have to figure out UTC <crazed> woo <kim0> lol <kim0> crazed: time now in UTC is 7:33pm <crazed> date -d "14:00 EST" -u <kim0> your tz skills are l33t :) <kim0> I end up googling hehe <photerran> wolframalpha is good for that too <photerran> http://www.wolframalpha.com/input/?i=time+utc <crazed> kim0: how long are these sessions typically <kim0> crazed: just one hour <crazed> seems all the times on there are during my work hours <kim0> hmm <crazed> woah actually <crazed> i'm 80% sure i'll be in seattle those two days lol <crazed> going to amazon hq <kim0> hmm <kim0> does that mean you may not be able to present ? <kim0> just email me once you know <crazed> possibly not this time, is this monthly/weekly? <kim0> mm .. it's usually of less frequency, although there's zero problem doing this more often if people are willing to share :) <kim0> crazed: let me know once you're sure whether or not you can make it <crazed> kim0: will do <kim0> thanks <photerran> just my luck... signed up for launchpad to get into ubuntu wiki, now ubuntu wiki gives 500 error after the SSO gateway <photerran> i was going to just do this once i logged in, but maybe i should mention it here first anyway <photerran> kim0: the session details link (to Sessions page) on the cloud days page is broken, maybe you meant it to be the Timetable page instead? <semiosis> kim0: I sent you an email about the cloud days session. i'm all cleared but had to change nick (used to be photerran) as I explained in the email <kim0> semiosis: just read your email <kim0> that's awesome <kim0> what you mentioned is totally fine <kim0> semiosis: puppet or gluster .. hmm what about two sessions :) <kim0> semiosis: still unable to edit the wiki ? should I do it <semiosis> let me try again, i was getting 500 errors earlier <kim0> semiosis: that sometimes happen, but the "action" usually completes fine <semiosis> ah yes thats the case <semiosis> now getting 500 errors trying to open the edit page... i'll try again later. in regards to two sessions... i wouldnt want to take up two slots just yet, hopefully someone else will see an opening and want to contribute as well <kim0> semiosis: then I'd take gluster and more generally .. storage best practices for scalable web apps <kim0> semiosis: should I write that title ? <kim0> let me know what to write <semiosis> how about "Scaling legacy web apps in the cloud with GlusterFS" ? <semiosis> btw since we're going in the direction of gluster i should note that I dont use the ubuntu packages, but the latest version from gluster <semiosis> kim0: the wiki is working for me now, i added the topic to the timetable. thanks for your support! #ubuntu-cloud 2011-03-03 <crazed> hmm <crazed> where does write-mime-multipart come from <crazed> don't see it in the cloud-init repo, cloud-utils or cloud-init package <crazed> on 10.04 or 10.10 <crazed> though i do see it on the filesystem of an ec2 image <smoser> on 10.04 and 10.10 it might not be packaged at all <smoser> in 11.04 it is in cloud-utils <smoser> if you see it in the filesystem of the ec2-image, dpkg -S /path/to/file <crazed> ah -s! <crazed> didn't think of that <smoser> it is cloud-init if it is there in 10.10 <smoser> and you probaly dont want to isntall cloud-init on your laptop <smoser> (or other non-cloud instance) <smoser> which is why it moved to cloud-utils <smoser> its stand alone, though, you can jsut grab it from cloud-utils bzr <smoser> i have to go <smoser> later <crazed> oh good call <TeTeT> what would be the best way to give cloud credentials to an instance? <semiosis> TeTeT: credentials for what, exactly? <TeTeT> semiosis: credentials for S3, so the instance can store data there <semiosis> TeTeT: from what i've seen it depends on the tool you're using. for example, euca2ools would read /etc/euca2ools/eucarc, but another tool (or suite) may look elsewhere <TeTeT> semiosis: yeah, but how to transfer the eucarc and cert files to the instance? Using user data? <semiosis> TeTeT: i use puppet but you could very easily do it with cloud-config <semiosis> TeTeT: ...and a little shell script <TeTeT> semiosis: right, I will consider cloud-config. I want to leave out puppet for now <TeTeT> even though it's very useful for configuring instances <semiosis> TeTeT: couple things you may be interested in: storing the file or script in s3 & just pulling it with cloud-config (using a presigned S3 url), and also using IAM to create an access key with limited rights to put on the instance <TeTeT> semiosis: I don't know about presigned s3 urls and IAM, can you clarify or point me to some docs? <TeTeT> semiosis: nevermind, googled http://docs.amazonwebservices.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/GeneratePresignedUrlRequest.html <semiosis> right on <TeTeT> semiosis: I fear both are not available on UEC yet :( <TeTeT> semiosis: but thanks for your help, learned something new! <semiosis> yw <TeTeT> hggdh: hi, a first result from the runs of my scheduler script, bug 728018. Does it look familiar to you? e.g. is this a usual failure or something extraordinary? in the past I thought my script for capturing the ip was bad, but now it's using boto directly <uvirtbot> Launchpad bug 728018 in eucalyptus "10.04 LTS: Failure to start instance due to network address failure" [Undecided,New] https://launchpad.net/bugs/728018 <hggdh> TeTeT, looking <hggdh> TeTeT, I do not remember this error on 10.04 (when I tested it) <hggdh> TeTeT, so I would say you hit a new one... <TeTeT> hggdh: ok, wonder if it's also in 10.10 or 11.04. Unfortunately I cannot upgrade the training cloud, as I lose the ability to reset it then and IS has to intervene <TeTeT> hggdh: i thought it was quite rare, after launching hundreds of instances, but just a few mins ago it failed after 29 instances, so seems to be quite random, though rare <hggdh> TeTeT, I ran a lot of thousands of instances on 10.10, and did not see it. I am not sure we had a similar setup, though <TeTeT> hggdh: hmm, ok <hggdh> TeTeT, but it is something to try now, on 11.04 <TeTeT> hggdh: you've got a testing cloud with 11.04? <hggdh> TeTeT, yes. but euca is right now broken on DHCP <TeTeT> hggdh: oh, ok, so no point for me to upgrade my cloud at home yet and try to test <hggdh> TeTeT, no, not right now <hggdh> TeTeT, I can ping you when euca is workable on 11.04, if you wish <TeTeT> hggdh: yes, would be nice. FYI I've added a basic hadoop workload to the scheduler script, now on to others <hggdh> cool! <TeTeT> hggdh: I also moved the branch on lp to lp:~canonical-pse/+junk/uec-scheduler if you want to lauren32@example.com. It's now python/boto and cloud-init shell scripts <hggdh> TeTeT, pulling it now ;-) <TeTeT> hggdh: he he, don't be afraid of bad code ;) <hggdh> TeTeT, I am not, I write them all the time ;-) <TeTeT> hggdh: I'm just writing down some setup script <TeTeT> hggdh: errr, doc <TeTeT> hggdh: pushed, though it's untested :) Have to run an errand now but will be back later <vadi2> Is there a micro instance available for ec2 asia? <jamespage> smoser: around? <smoser> yes sir <jamespage> hey - time for a bit of help with cloud-init? <smoser> gladly <jamespage> OK; so I've been working on expanding the test cases for ec2 testing. <jamespage> And I've started to look at the User Data tests..... <jamespage> originally I had put a test execution into the cloud-config part; however this gets executed before all of the user data scripts <jamespage> so that did not work great when I wanted to test that they all ran OK. <jamespage> question is: can I imply an order in which the user data scripts/data/parts will be executed (I want to run the tests at the end)? <smoser> where di you put it in the cloud-config part ? <smoser> in runcmd ? <jamespage> yes <smoser> runcmd is run in the same way as user-data scripts <smoser> it gets put into a directory and run with run-parts <smoser> so you can manipulate the ffilename of the other user-scripts to cause it to run last (make them named earlier in order) <jamespage> ah - right - I see - let me try that out <smoser> *or* you could run those scripts as a upstart job that runs after stopped-cloud-init-final.. or something. <smoser> the way i ran tests inside the instance was to ssy after the instance was up <smoser> as , if you're testing user-data, you can't really be sure that cloud-nit is going to run the stuff that you put in there. <jamespage> smoser: thanks for you help - that worked just great! <smoser> read ^ <smoser> i suggest though not running those things via cloud-init <jamespage> That would be another way around it; would pickup a break in cloud-init better. <jamespage> At the moment my test would time out after 5 minutes of waiting for the test results to appear <lotreck> does anyone have experience using a 10gbit ethernet card on ubuntu lucid ? I just want to know if there are drivers on ubuntu for it . I'm looking at the Intel X520 line. <lotreck> I'd hate to purchase new dell gear and some 10 GB nics only to find out that they don't work together. <jeremydei> hm, this is weird, i just noticed that maverick on ec2 doesn't have /proc/xen/* <jeremydei> anyone know what's up with that? im particularly interested in the xen clock stuff .. and how it appears to be mia <jeremydei> smoser, am i crazy or is the xen independent wallclock not in maverick ec2? should be in /proc/sys/xen/independent_wallclock <flaccid> lotreck: this is ubuntu-cloud. you might want to check driver support for the chipset. #ubuntu-cloud 2011-03-04 <smoser> jeremydei, well, maverick use the pv_ops kernels instead of a specific xen kernel <smoser> there could have been fallout and loss of xen specific sutff <jeremydei> smoser, yeah I started to realize that with pvops that xen stuff prolly won't be showing up :) the clock apparently skews pretty badly I've been told .. <jeremydei> im guessing some kernel params need tuning or something .. to keep in sync with the underlying xen clock, def need to research it some more to understand how it should work <HugoKuo> http://forum.eucalyptus.com/forum/can-not-run-instance-theres-only-one-error-log-cloud-errorlog-help-plz#comment-17579 looking for help <HugoKuo> I'm stuck over ther 1 day ago <TeTeT> HugoKuo: I'm trying to take a look, but my internet connection is currently dead slow. Have you shut down all instances and restarted eucalyptus with CLEAN=1 <TeTeT> ? <HugoKuo> not yet <HugoKuo> bcz I never success to run up an instance <HugoKuo> there r no any error in cc.log <HugoKuo> or I can restart CC now! <TeTeT> did you do a packaged install or cd based? What version? 10.04 LTS or 10.10? <HugoKuo> oh man , I use ubuntu UEC before but got some problem <HugoKuo> so I try debian <TeTeT> ah, did you check if the components are registered? <HugoKuo> in this ticket , OS is Debian <HugoKuo> yes , all components is registered!:> <TeTeT> sudo euca_conf --list-walruses --list-clusters --list-scs --list-nodes <TeTeT> shows them all? <HugoKuo> yop <HugoKuo> tup <HugoKuo> yup <TeTeT> What's the output of euca-describe-availability-zones verbose? <HugoKuo> normal <HugoKuo> 0003 / 0004 1 128 5 <HugoKuo> oh but there's something abnormal.. <HugoKuo> you can see free CPU is 3 .... <HugoKuo> but actually , I already terminate the instance <TeTeT> hmm, so is there's an instance running? <HugoKuo> I always need to restart CLC to correct it <HugoKuo> there's no any instance <HugoKuo> for now <TeTeT> is your network bridged on the NC? <HugoKuo> yup <TeTeT> ah, see the log files now on the forum <HugoKuo> VNET_MODE= MANAGE-NOVLAN <HugoKuo> I used to use UEC , but while I upgrade to 2.0.0 there r may probleem , something like outofmemory , it cause CC & SC terminated automatically <TeTeT> MANAGED-NOVLAN <HugoKuo> that's why I turn to debian <HugoKuo> oh yes <TeTeT> hmm, I don't see any obvious errors in your config, sorry I can;t be more helpful <HugoKuo> sorry I meant it <HugoKuo> whatever , Thanks you so much man <TeTeT> no problem <TeTeT> what I usually do is to tail -f on the nc.log when trying to start an instance <TeTeT> if there's only doDescribe stuff, then the CC never contacts the NC <TeTeT> so I then turn to the CC, then the CLC logs <TeTeT> but I'm not really familiar with them <TeTeT> but then, you already did all of that basic debugging stuff <HugoKuo> I guess CLC did not send request to CC <ajith_kgs> I am trying to setup a UEC. I got an image from uec images store and registered it using uec-publish-tarball. When i start the instance the describe instances command says my instance is running but i cannot connect to the instance (ping and ssh not working). I have added permissions using euca-authorize. <ajith_kgs> the console.log corresponding to the running instance on the nc is empty(0 bytes). please advice <ajith_kgs> the console.log corresponding to the running instance on the nc is empty(0 bytes). Cannot ping or ssh to the running instance. Any way of checking if an instance is running proper;y. please advice <ajith_kgs> the console.log corresponding to the running instance on the nc is empty(0 bytes). Cannot ping or ssh to the running instance. Any way of checking if an instance is running properly. please advice <kim0> ajith_kgs: what if you ssh to the nc <kim0> and run <kim0> sudo virsh list <smoser> jeremydei, i'm interested in your feedback, and we can change things in ubuntu to make things "just work" <TeTeT> smoser: thanks for the thorough review of the exercise, I've sent you an update, but no need for you to do anything. I'll poke into the img2disk script idea when I've some idle time <kim0> TeTeT: hey o/ <kim0> Do you know yet if you'll be able to deliver that session in Cloud Days <TeTeT> kim0: no, still haven't got a go from my direct manager * kim0 nods <kim0> TeTeT: let me know once you do <kim0> thanks :) <kim0> TeTeT: I think you wanted to share a screen session. I'm thinking about using ajaxterm to share a screen with the class .. still didn't try that yet <kim0> just wish there was something more straightforward <TeTeT> kim0: ajaxterm? Nice idea, kirkland recommended that screen via amazon ec2 sharing <TeTeT> kim0: I'll bring it up with Martin this afternoon in our 1:1 <kim0> TeTeT: yeah probably both :) <kim0> screenbin + ajaxterm <kim0> cool <RoAkSoAx> smoser: o/ Hey I guess that starting next week I can now help you with whatever you needed since I will now create an ec2 account <smoser> horray. <smoser> wit hthe addition of another regoin (ap-northeast-1) we are now more in need of publishing parrallel <RoAkSoAx> yeah I can imagine <zul> or an assitant ;) <TeTeT> kim0: I've got permission for the EBS based instance on UEC for the Cloud Days. Let's call the session 'UEC Persistency' and allocate a Europe friendly time for me, please :) <kim0> TeTeT: woohoo :) <kim0> TeTeT: edit https://wiki.ubuntu.com/UbuntuCloudDays/Timetable <kim0> Thu 17:00UTC is good I hope ? <TeTeT> kim0: yes, it's fine <kim0> awesome :) * kim0 preparing for a boto session in one hour <kim0> smoser: hey, does boto ec2 control UEC as well ? <TeTeT> kim0: yes, take a look at my uec scheduler code if need be, https://code.launchpad.net/~canonical-pse/+junk/uec-scheduler <TeTeT> kim0: it's easier to understand than the euca2ools code, which use boto too <TeTeT> and gone :) <Daviey-> kim0, BTW - I developed the first iteration of awstrial against UECC <Daviey-> kim0, http://bazaar.launchpad.net/~awstrial-dev/awstrial/trunk/view/head:/awstrial/trial/ec2_helper.py <Daviey-> see the commented section to use boto againt UEC <kim0> Daviey-: Thanks :) * kim0 rings a bell .. Final Session in Ubuntu Cloud Days <kim0> Anyone who wants it .. register at https://wiki.ubuntu.com/UbuntuCloudDays/ <kim0> Daviey-: you're welcome to take it :P <Daviey-> kim0, easier to read rev1 http://bazaar.launchpad.net/~awstrial-dev/awstrial/trunk/revision/1/trial/ec2_helper.py <Daviey-> kim0, Can do... but looks like TeTeT has done more with boto than me <kim0> Daviey-: it doesn't have to be boto <kim0> you get to pick the topic :) <Daviey-> kim0, okay.. got any ideas? <kim0> Daviey-: working on anything cool lately ? <kim0> hehe <Daviey-> nah.. all boring :) <kim0> haha <kim0> thought so <kim0> hmm <Daviey-> kim0, Hmm <Daviey-> kim0, give me the weekend to brain storm <kim0> Got it <kim0> Daviey-: I'll count you in though <Daviey-> kim0, Is awstrial mentioned there? <kim0> nope <Daviey-> kim0, okay.. sounds good * kim0 nods <kim0> let me know when you decide <Daviey-> when is it again? :) <kim0> Mar 24th <kim0> https://wiki.ubuntu.com/UbuntuCloudDays/ <Daviey-> Hmm * Daviey- checks his cal <Daviey-> looks good <kim0> awesome * kim0 joins realLife afk for now <Daviey-> have fun o/ <kim0> Daviey-: you too o/ :) <patrickw> What is the status of cloud-init on centos? Is there an easy way to install cloud-init from yum repos? <Daviey-> patrickw, no idea, but have you seen http://code.google.com/p/amzn-cloudinit? <patrickw> Deviey-, I discovered that yesterday while hacking my own simple stuff to process user-data in my centos vms... but neither that site nor checking out the code was helpful and googling hasn't been fruitful. #ubuntu-cloud 2011-03-06 <Dotted> how do you create a new user on ubuntu 10.10 with a blank password? all my attempts using passwd, editing /etc/passwd and /etc/shadow has "su" still asking for a password even if its blank <flaccid> Dotted: use sudo with the ubuntu user and adduser <Dotted> the user is already created <flaccid> what shell does the user have in /etc/passwd? <Dotted> /bin/bash <flaccid> are you doing su from root? <Dotted> yeah from an init script <Dotted> http://www.minecraftwiki.net/wiki/Server_startup_script#Script <flaccid> check logs to see what the issue is i guess <flaccid> maybe it needs to be a system user or something <flaccid> add -x to the init script to see how it bombs out too <flaccid> someone needs to package that software <Dotted> http://pastie.org/1637811 <flaccid> [ root == ubuntu ] is incorrect bash syntax <flaccid> causes [: 1: root: unexpected operator <flaccid> also it does su to ubuntu user after that and not root user <flaccid> thats intentional but yeah. this is not a good init lauren32@example.com. <Dotted> well ideally it would su to minecraft user, just used ubuntu here for testing <flaccid> if [ "$ME" == "$USERNAME" ] ; then should be if [ "$ME" = "$USERNAME" ] ; then <Dotted> http://pastie.org/1637831 <flaccid> run su - ubuntu -c cd /home/ubuntu/minecraft and also screen -dmS minecraft java -Xmx1024M -Xms1024M -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalPacing -XX:ParallelGCThreads=1 -XX:+AggressiveOpts -jar minecraft_server.jar nogui <flaccid> then check ps <flaccid> at least one of the processes is dying thus why if ps ax | grep -v grep | grep -v -i SCREEN | grep $SERVICE > /dev/null fails <flaccid> though this is all the wrong way to do this stuff but meh.. <Dotted> Password: <Dotted> su: Authentication failure <Dotted> and thats the problem in a nutshell, despite blank password it still asks for it <Dotted> happends on both ubuntu and minecraft user <flaccid> that would be expected. its meant to be run under root. <Dotted> minecraft_server.jar is meant to be run as root? <Dotted> that seems highly unlikely <flaccid> no the init script. as you can see there is a USERNAME variable and it uses su -c to run it under this username <flaccid> you won't be able to su properly without root <flaccid> su != sudo <Dotted> well the init script is already run as root <Dotted> sudo sh -x /etc/init.d/minecraft start
{'channel': 'ubuntu-cloud', 'month': 3}