#ubuntu-cloud 2010-12-01 <Makere> why I get randomly "unable to mount root filesystem" on VM's? <Makere> General error mounting filesystems <flaccid> what VMs are these <flaccid> i'm off <Loof> Quick question, are there any tools to reset an instance (clear cloud-init locks, hostname, etc) and prep for bundling a 'clean' AMI? <Ashtray777> Hey <Ashtray777> What do you fellows think about CompTIA Linux+ <Makere> what's that? :P <Makere> anyways, anyone done ubuntu desktop installation on kvm and bundled into cloud, anything special I need to do? <Makere> stuck on pending ._. <kim0> Makere: I just started an Ubuntu AMI and made it Ubuntu desktop. http://foss-boss.blogspot.com/2010/11/show-off-ubuntu-desktop-on-cloud.html <Makere> thanks <Makere> is it possible to increase the partition inside a VM <Makere> kim0: I run out of space when trying to install ubuntu-desktop <TeTeT> Makere: you want to use a bigger instance type, like c1.medium or somesuch with 10G + \ <TeTeT> Makere: I guess you're on 10.04? Or 10.10? <Makere> 10.0 <Makere> 10 <Makere> 10.10* <Makere> so hard to type lol <Makere> I'm running bigger instance <Makere> but the filesystem partition is always just over 1.5gigs <TeTeT> Makere: he he, so your container should be 10G each, you can check with df -h <TeTeT> Makere: can you use resizefs to increase the size? Never done that actually and it is an interesting problem to encounter <Makere> can't because the filesystem is in use <TeTeT> Makere: hmm, what if you use -f? <TeTeT> Makere: given that you're just testing, aren't you? and an instance is cheap anyway <Makere> well I sorta gave up on <Makere> it <Makere> and now running KVM inside an instance <Makere> :DDD <Makere> sorry my life is destroyed by could <Makere> cloud* <TeTeT> Makere: it still is an interesting problem. Maybe smoser has an idea how to achieve that, to increase the filesystem size of / of the AMI to be big enough for Ubuntu Desktop <smoser> Makere, where are you running this instance ? UEC ? <smoser> Makere, i'm guessing you're urunning on UEC, and that you've uploaded one of the ubuntu images. they've got a 1.4G filesystem by default to fit into the default UEC small of 2G. <smoser> you can resize that up and re-register <smoser> download tarball, then : uec-publish-tarball --resize 6G downloaed-tar-file.tar.gz my-new-bucket amd64 <kim0> pretty simple huh :) <TeTeT> smoser: awesome, great to read that uec-publish-tarball has the magic <kim0> #start Ubuntu Cloud Q+A gathering <kim0> Hi everyone <kim0> If you're here to attend this Q+A meeting, please say Hi <kim0> hmm .. slow day :) <kim0> well .. If anyone has any thoughts or comments that relate to Ubuntu in the cloud .. now is a good time to start a discussion <orangeacme> what gathering is this? <orangeacme> my main thought is: "Ubuntu is perfect for the cloud" ;-) <kim0> orangeacme: hehe <kim0> orangeacme: https://wiki.ubuntu.com/UbuntuCloudMeeting <kim0> orangeacme: this is a weekly gathering for everyone interested in ubuntu and the cloud <kim0> basically to ask questions .. share experiences and connect <kim0> orangeacme: so how are you using ubuntu in the cloud <orangeacme> a few VMs in ec2. all working great as expected <orangeacme> i was going to bring up https://bugs.launchpad.net/ubuntu-on-ec2/+bug/670161 <uvirtbot> Launchpad bug 670161 in ubuntu-on-ec2 "10GB root partition for EBS boot AMIs on EC2" [Medium,Fix committed] <orangeacme> but it looks like that is underway <kim0> orangeacme: Yeah <smoser> orangeacme, expecting those probably by end of next week as released <smoser> remember, you can use an ami from -daily and if it happens to fit your bill, you just have to know that that ami-id might disappear in the future. <smoser> that will have no ill affects on your instances that yo'uve already launched, just stop you from launching them in the future <orangeacme> this is where i thank everyone for their hard work * kim0 hugs orangeacme <kim0> smoser is da man :) <kim0> smoser so we're standardizing on 8G AMIs ? <smoser> yes, per amazon recommendation. <kim0> Awesome <smoser> EBS root will be 8G, instance store will be 10 <kim0> Landscape gets a yummy new look http://blog.landscape.canonical.com/2010/11/30/our-new-look/ <niemeyer> Neat <Makere> smoser: oh neat, you can resize it <smoser> the way ec2 and uec work is that you get a root filesystem of whatever size was uploaded. <smoser> our images are, by default, 1.4G for UEC consumption, so the "small" size can run there (which has 2G disk allocated) <Makere> I thought I'd have to whip out parted and do resizing somehow manually <smoser> if you register a larger root disk (ie, using --resize) it will not run in a m1.small <smoser> Makere, no, its just much easier than that because it is a raw partition image rather than a disk image. <smoser> so you can just truncate the file larger, and then call resize2fs on it <smoser> (which is what uec-resize-image does) <Makere> ya <Makere> neat that there's a command for that <Loof> Are there any tools to automatically 'rebundle' Ubuntu AMIs... Specifically, to remove cloud-init locks, ssh keys, etc prior to bundling? <Loof> I've looked around, but I've not seen anything to actually purge the system files and make it a 'fresh' image <smoser> Loof, no, there isnt anything. <smoser> you should not have to remove anything for cloud-init, as its locks are instance-ids specific <smoser> but... all the same. we are hoping to have something like that for 11.04, but even then, the suggested rebundle mechanism is to download images from http://uec-images.ubuntu.com and mount loopback and operate on them then register <smoser> that is jsut a cleaner approach <smoser> that leaves less likelyhood of your keys or private data being left in the image <ramanK> Hello <ramanK> Is there any plan for PaaS on ubuntu ? <ramanK> something like jboss on RHEL <ramanK> !ping #ubuntu-cloud 2010-12-02 <kim0> Hi everyone <kim0> Got an interesting question on the cloud forums .. http://ubuntuforums.org/showthread.php?t=1635728 <kim0> basically does UEC utilize multiple ethernet channel in any special way .. <kim0> can it have a separate network for "storage" "management" "internet" ...etc <kim0> would bonding them all be a good idea <kim0> Daviey: o/ :) <smoser> kim0, uec does not do anything special with multiple ethernet channels <smoser> Daviey, you can tell me i'm full of it if you'd like, but i'm 98% sure of that <Daviey> kiall: o/ <Daviey> smoser: you are full of it <Daviey> smoser: but this is a feature I haven't heard of, so i painfully, have to agree on this instance :) <smoser> thanks for taking the opportunity to say that <kim0> Daviey: smoser hehee :) <kim0> so just bonding all eths is the way to go huh <smoser> hggdh, ping <smoser> have you done any UEC testing for Natty ? <hggdh> smoser: tried, but complete failure to *install* Natty so far on the test rig <hggdh> smoser: latest one is bug 684304 <uvirtbot> Launchpad bug 684304 in linux "cciss module does not identify resources" [High,New] https://launchpad.net/bugs/684304 <smoser> hggdh, ok. so we should at least try to test the natty images <smoser> you have a way to do that ? <smoser> ive taken down my lucid/maverick cloud <hggdh> smoser: I can run the natty images on the Maverick ISOs/UEC. Is this enough? <smoser> please <smoser> if you want to give me access, i can do it for you <hggdh> smoser: no prob, I am starting a maverick UEC install now <hggdh> smoser: you *should* have access to the test rig <smoser> yeah, i probably do <smoser> just let me know when its up <hggdh> smoser: will do <hggdh> smoser: and then we need to talk about the EC2 tests pass-over <smoser> we sure do ! <smoser> :) * smoser does a jig <leonardopires> hello ubuntu friend'S! <leonardopires> where i can get a list of certified hardware for ubuntu cloud? <hggdh> smoser: I use the uec.images.u.c/natty/current, correct? <smoser> no <smoser> use 20101130 <smoser> thats what is [already] going to be released <hggdh> oh, OK <hggdh> smoser: loading the thingy now <hggdh> smoser: 500 instances (mix of i386/amd64) is OK? <smoser> really you just need to test 1 <smoser> :) <smoser> ideally, running through <smoser> http://iso.qa.ubuntu.com/qatracker/test/4820 <smoser> http://testcases.qa.ubuntu.com/System/UECCloudImages <smoser> for both amd64 and i386 <hggdh> smoser: except for the metadata (which I will do manually right now) I am doing it for 10 different keys/groups <smoser> s/metadata/userdata/ ? <hggdh> userdata, darn it! <hggdh> :-) <hggdh> smoser: I will, normally, run tests on (i386|amd64) for all instances types, for Lucid, Maverick and Natty <hggdh> smoser: done, passed <smoser> woot. <smoser> you regster them in the iso tracker ? <smoser> skaet_, ^ <hggdh> yup <smoser> thanks. <skaet_> hggdh, smoser, Thanks! <hggdh> skaet_: I had forgotten to update the i386 UEC tests, just did it <skaet_> heh., was just double checking and going to come back and ask. :) <skaet_> smoser, you may as well go ahead and start pushing out the working images then now. :) <smoser> oh. we're public <smoser> http://uec-images.ubuntu.com/releases/natty/alpha-1/ <skaet_> good-o, rest will be following. you're first this time. ;) #ubuntu-cloud 2010-12-03 <HugoKuo> is there have ubuntu hardy image for eucalyptus ? <elro> Is it possible to configure a virtual machine to use a physical disk? <elasticdog> does UEC not use the front-end machine's /etc/eucalyptus/eucalyptus.conf file to configure the available IP address pool? <elasticdog> http://pastie.org/1342671 is what I put in there (and the VNET_DNS), but even with restarting everything, available IPs don't seem to change <elasticdog> someone else set up the machines originally, and put in 156.14.161.142-156.14.161.142 during the installation procedure, but I can't find that range referenced anywhere <elasticdog> nobody has experience changing the address pool with UEC post-installation? <kim0> elasticdog: I'm not at all sure <kim0> elasticdog: BUT you may need to do this <kim0> sudo stop eucalyptus CLEAN=1 <kim0> the CLEAN=1 flushes cached network info AFAIK .. which may be what you're looking for <kim0> sudo service eucalyptus-nc stop CLEAN=1 <kim0> too and perhaps use them when starting too <elasticdog> yep, I've done the clean restarts and tried rebooting all of the machines as well, with no luck <TritoLux> Hello there, I have a freshly installed UEC 10.10 cloud based on MANAGED-NOVLAN config and I am having a problem with EBS: if I use iSCSI then everything seems to work fine, but if I reboot the CC-SC machine, then the existing volumes cannot be attached anymore. If I use AoE, then it's even worse, as the new volumes I create can be attached, but they do not even appear within the instance, regardless of VIRTIO_DISK setting. I follo <TritoLux> wed official instructions step by step and I am only using images from the store. Any help would be greatly appreciated. Thank you in advance. <TritoLux> Another weird issue I am having is that if I run the image 10.04 64 bits, installed from the Store, then it never comes up properly the first time. I always have to reboot it after launching it in order to gain access to it. <TritoLux> Did anybody else experience above problems already? Are they known issues? #ubuntu-cloud 2010-12-04 <TritoLux> anybody there? * flaccid yawns <TritoLux> :) <TritoLux> I'd like to report a problem with the 10.04 64bit image at the UEC Store.. in my fresh installation I never managed to run it properly at the first boot. I always have to reboot it in order to run it properly and access it. <flaccid> i think you can submit a bug <TritoLux> ok I'll do it.. I was just wondering if anybody else had the same problem or if it was depending on my setup <TritoLux> I just found out that 9.10 runs fine immediately though, so I don't think it's a setup issue <rampr> is it possible to change the availability-zone after the instance is stopped ? <rampr> with ec2-modify-instance-attribute <liam> does anyone know if it is possible to download a key pair again after it has been created. I cant see anywhere in the console to do this.... #ubuntu-cloud 2010-12-05 <ssj6akshat> What should I name my post? <ssj6akshat> 'Ubuntu Cloud Portal is a one stop shop for all cloud enthusiasts' <ssj6akshat> ? <asac> http://twitter.com/asacasa/status/11397904388333568 <asac> "would be precious if there was a 10GB official @ubuntucloud EBS AMI available for use with e.g. http://aws.amazon.com/free/ #ec2" <asac> maybe i am missing something and its easy to resize the EBS down form 15 to 10? * asac the clueless cloudy <asac> ok got answer on twitter.... thx <kiall> So .. is there anything special that needs to be done to get a re-bundled ubuntu instance (on UEC) to boot? .. Every attempt so far has stalled during boot #ubuntu-cloud 2010-12-06 <smoser> kiall, there should not be, but giving console output (euca-get-console-output | pastebin ) would be helpful. also, what version of "ubuntu instance" <kiall> smoser, 10.10 UEC and 10.10 instance, will grab the console output now .. <liam> i need to connect to my ubuntu ec2 server from my ubuntu machine using filezilla with sftp but it won't connect. How can I specify the key to use in filezilla? <kiall> console output: http://pastebin.com/RHkixNnS <kiall> liam, they have a howto for that :) http://wiki.filezilla-project.org/Howto <liam> kiall: it wants a .ppk..... do you know how I can convert my ec2.pem to a .ppk? <kiall> can puttygen (PPK is Putty's key format) not do that? <kiall> (Its been a while since I've used windows...) <kiall> yea it can .. http://invoice.geeksontheway.com/wiki/index.php/Working_with_EC2#Converting_pem_files_to_ppk_files_for_use_with_putty <kiall> smoser, just noticed the " | pastebin" ... how didnt I know about the pastebinit package before .. :) <kevinw> can anyone shed light on the error: EXPIRED: Instance expired after not being reported for 600000 ms <kevinw> also secondly, if both the node controller and the cloud controller are both on public routable IP's, which interface do i bridge to make the VM accessible with a public routable IP? <TeTeT> kevinw: the cloud controller/front end interface. If you check the iptables rules there, you see a NAT for the public IP for any instance you've setup <kevinw> TeTet: im not doing any nat <TeTeT> kevinw: it's automatically done on the front end if your instances is using a public / private network address <kevinw> ok, ill look into that, my main problem is that the machines wont start, they stay as pending then come up with an error about not being reported in 600000ms <TeTeT> kevinw: you want to check the node controllers log files, e.g. /var/log/eucalyptus/nc.log <TeTeT> do a ''tail -f /var/log/eucalyptus/nc.log | grep -v doDescribe'' when starting an instance and watch for any errors <kevinw> root@node1:/var/log/eucalyptus# tail -f /var/log/eucalyptus/nc.log | grep -v doDescribe <kevinw> [Mon Dec 6 12:33:10 2010][001197][EUCADEBUG ] doStartNetwork() invoked <kevinw> [Mon Dec 6 12:33:10 2010][001197][EUCAINFO ] StartNetwork(): SUCCESS return from vnetStartNetwork 0 <kevinw> [Mon Dec 6 12:33:10 2010][001197][EUCAINFO ] StartNetwork(): done <kevinw> it gets an IP ok but then just sits pending <TeTeT> kevinw: hmm, is the front end's walrus service reachable from the node? What's the output of sudo euca_conf --list-walruses <kevinw> root@node1:/var/log/eucalyptus# sudo euca_conf --list-walruses <kevinw> ERROR: cannot locate eucalyptus database, try logging in through the admin web interface. <kevinw> root@node1:/var/log/eucalyptus# <kevinw> root@cloudcc1:/var/log/eucalyptus# sudo euca_conf --list-walruses <kevinw> registered walruses: <kevinw> walrus +1-905-866-7687 <kevinw> how do i go about registering walruses on the node controller? <kevinw> scrap that, got that working <kevinw> root@cloudcc1:/var/log/eucalyptus# sudo euca_conf --list-walruses <kevinw> registered walruses: <kevinw> walrus +1-905-866-7687 <TeTeT> kevinw: can you ping +1-905-866-7687 from the node controller, is it reachable? <kevinw> yep, <kevinw> pings between the two physical servers are OK <kevinw> the instances stay as pending no matter what I do <TeTeT> kevinw: is there ever a kvm started on your node controller? Is the image copied from walrus to the local storage? <kevinw> how do i check that? <TeTeT> kevinw: check /var/lib/eucalyptus/instances/admin on the node controller <TeTeT> kevinw: are there any directories named 'i-<id>'? <TeTeT> kevinw: and inside there should be a name called 'disk' <kevinw> nope there isnt <TeTeT> hmm <TeTeT> kevinw: is there a cache directory, /var/lib/eucalyptus/instances/eucalyptus/cache with emi- and eki- inside? <kevinw> theres a directory but its empty <kevinw> if it helps im not using the admin account, <TeTeT> kevinw: somethings really strange, can you try as admin to rule out any problem with the user? Not that there should be one <TritoLux> With UEC 10.10, Is there any reason why iscsi volumes get unaccessible after a SC reboot? Everything is working fine but if the server gets restarted then all existing volumes are not accessible anymore. Did anybody managed to get EBS to work fine on 10.10? <kevinw> TeTeT - same nothing in either directory <kevinw> should the walrus be registered on both the cloud controller and the node controller? <TeTeT> kevinw: nope, the front end is enough. <kevinw> the cc? <TeTeT> yes <kevinw> hmm <kevinw> the instances have IP's from the allocated range OK so that shows comms between the servers is OK, <kevinw> i wonder why its not registering <TeTeT> hmm, when you do a euca-describe-images, do you see the uploaded images? Is there something in /var/lib/eucalyptus/bukkits? <TeTeT> somehow the image from walrus fails to be transfered to your node controller <TeTeT> something in, meant something in on the front end <kevinw> root@cloudcc1:/var/lib/eucalyptus/bukkits# ls <kevinw> image-store-1291376044 image-store-1291393809 image-store-1291393871 <kevinw> im using the images from the store <TeTeT> ok, when you copy your eucarc to the node controller, source it there, can you do an euca-bundle-image /etc/hosts; euca-upload-bundle -m /tmp/<manifest> -b etc-hosts? <TeTeT> wonder if the upload works <kevinw> ill give it a try, i notice in the logs that its trying to get to http://+1-905-866-7687:8775/axis2/services/EucalyptusNC but I cant get to that in a browser.. <TeTeT> kevinw: hmm, seems your network setup between front-end and node controller is no good then <kevinw> i agree, but i dont see where, its two servers, with 1 NIC each, plugged into a switch.. <TeTeT> is there a firewall running on either NC or front-end that might block the traffic? <kevinw> only whatever was installed by the setup <smoser> erichammond, awake ? <TeTeT> kevinw: hmm, then there's nothing there. did the upload to walrus eventually work? <smoser> kiall, i would guess that you have something in /etc/fstab that mountall is waiting on <kevinw> im going to scrap and start again, for clarification theres no reason why have the CC and the NC on public IP's with the VM on the same range wont work <smoser> well, erichammond if you do see this, alestic.com gives me: "Gateway Time-out <smoser> The gateway did not receive a timely response from the upstream server or application." when i try to login with my launchpad id (https://launchpad.net/~smoser) <CharlieSu> How do you change limits like this on an Ec2 instance? (* soft nofile 65536) do you have to edit limits.conf and restart the server? <jmgalloway> is it possible to create a vm without exchanging the key? <CharlieSu> How do you change limits like this on an Ec2 instance? (* soft nofile 65536) do you have to edit limits.conf and restart the server? <erichammond> smoser: Just got up, but need to commute for hour, then scheduled lunch. <smoser> fair. i had this bug once before when trying launchpad openid on alestic.com also <smoser> ie, its not new, but i'd like to use my launchpad.net id <erichammond> smoser: I saw a gateway timeout on http://alestic.com last night, but refresh worked. <smoser> tried it twice... no dice, but i can try again <erichammond> smoser: I get it regularly when using dynamic MT (Movable Type) pages. <erichammond> Thought it was a bug in MT, but it might instead be related to a DDOS defense tweak I made a while back for another web site on that server. <erichammond> I'll look into it later today. <smoser> well i consistently get it. <erichammond> hm <smoser> you can try too, just try your launchpad account <erichammond> ah, yes, consistent with MT dynamic stuff. <erichammond> especially if it is requiring a wait on an external resource like launchpad. <erichammond> smoser: I tweaked timeouts on http://alestic.com <erichammond> smoser: Let me know if you still have problems logging in. <asac> odd ... a few hours ago i was able to ping something like www.yahoo.com from my ec2 instance ... doesnt work anymore ... i am trying to stop the instance in web interface ... and that doesnt work either it seems :/ ... it stays in stopping state and i can still ssh into it <asac> should i just run sudo halt ;)? <asac> crazy * asac feels lost... and scared ... not even http://us-east-1.ec2.archive.ubuntu.com is reachable anymore <asac> but i can ssh in !!! #ubuntu-cloud 2010-12-07 <Makere> So I downloaded maverick-server-uec-amd64.tar.gz from http://uec-images.ubuntu.com/maveric/current/ and used uec-publish-tarball with --resize 8G <Makere> It's stuck on pending when I'm trying to launch it <Makere> any tips? <Makere> I don't see what I'm doing wrong <Makere> writing GET/GetDecryptedImage output to /var/lib/eucalyptus/instances//admin/i-51A60A11/kernel is the last thing I see about it in nc.log <Makere> after that it just keeps doDescribeInstances and gives info about it <Makere> I get similar behaviour with my own image <Makere> but the UEC image works without modifications <Makere> after some time I get EXPIRED <kiall> Makere, I havent see that before .. but I presume the instance type your using is allowed 8GB of disk? <kiall> (i have no idea what the defaults are anymore .. but i seem to remember the smaller instance types being about 5GB <Makere> yea <Makere> adjusted the amounts <Makere> also tried launching with larger <kiall> so do you see a kvm process start up? (or the dd process preparing the disk image before that..) <kiall> (or xen) <kiall> wait .. this is #ubuntu-cloud not #eucalyptus .. of course is kvm ;) <Makere> didn't actually check this time, but last time it did this, no kvm process <Makere> I'll check inside an hour <Makere> "hour" <Makere> no kvm processes lol <kevinw> how does one go about de-registering a node controller in euc? <kevinw> let me rephrase that, i see the node controller registered but with 0 available vm's, whats causing this? <Makere> anything in cc/nc/registartion logs? <kevinw> nope <Makere> well I recommend giving up on cloud now while you can <kiall> lol ... <kiall> kevinw, what have you got in /var/log/eucalyptus/euca_test_nc.log ? <kiall> (for total memory and cores) <kevinw> total_memory=1026 <kevinw> nr_cores=2 <kevinw> i see the node as registered: root@cloudcc1:/var/log/eucalyptus# euca_conf --list-nodes <kevinw> registered nodes: <kevinw> 213.xxx.xxx.xx cluster1 <kevinw> root@cloudcc1:/var/log/eucalyptus# <mcella> hi there, we are using ubuntu amis on ec2 <mcella> what's the proper/best way to launch new instances with a custom hostname? <mcella> we need a predicable hostname for our own applications (+ rabbitmq) <kiall> mcella, you can "fudge" the hostname with RabbitMQ .. <kiall> HOSTNAME=test / NODENAME=rabbit@test in /etc/rabbitmq/rabbitmq.conf gives a fudged hostname of "test" to rabbitmq <kiall> (assuming you have test in your hosts file or DNS) <mcella> kiall: that's interesting, but we still required a know hostname for our own apps :-/ <kiall> DNS and an elastic IP will give you that .. <kiall> if you set the hostname on the server to microsoft.com, you dont start getting MS's traffic .. so an elastic (static) ip and a DNS record like server.yourdomain.com pointing at it will work .. <mcella> kiall: yep, but our instances are short lived, we need to start and stop them for demonstration purpose, so we decided to not use elastic ips <mcella> it would cost too much <mcella> we are also exploring route 53 <kiall> then there really isn't much you can do - without an elastic IP, there is no standard way to allow one instance determine the dynamic IP of another .. You'd have to get Dynamic DNS, or pushing the IPs to specific S3 keys or something like that going .. <kiall> but only dynamic DNS will help with a consistent hostname .. <kiall> ah wait .. i read S3 there .. not route 53 their new service <kiall> I'm betting that can help you now that you mention it .. been meaning to read up on it <mcella> kiall: yeah, anyway we don't need intra instances communication <mcella> we just need a way to launch/create an instances and have it accessible at a know location (parametrized by the instance user data probably) <mcella> we have a base ami (built from the ubuntu one) to create new instances <mcella> another option is to set a static know hostname but cloud init at the first run will override it right? is there a way to avoid that thing <mcella> ' <mcella> ? <kiall> by "know location", you mean a hostname like "demo.domain.com" I presume? You need DNS for that - be it static pointing at an elastic IP .. or dynamic pointing at a non-elastic IP .. <kiall> and by accessible, you mean you can point your browser at it? <kiall> (Im assuming a web app here .. but you get the idea) <kiall> yea - cloudinit will override it .. I haven't looked, but im sure you can turn that off .. otherwise, you can just use cloudinit to set it to something after its reset it <mcella> kiall: exactly <mcella> kiall: at what stage runs cloudinit? <mcella> before or after rabbit? <mcella> I assume it runs as early as possibile in the boot process right? <kiall> Its prerry early <kiall> rc.local -ish if i remember right <kiall> either way - adjusting the hostname on the instance is not required, nor will it help with, access the instance from outside amazon, or from other instances ... <kiall> accessing* <mcella> yep, that's true <kiall> It would help with RabbitMQ .. but is an awful hack in my opinion .. Telling RabbitMQ to use a specifc, additional hostname thats been planted in /etc/hosts (or DNS) is way less hacky... <kiall> (... instead of fudging the entire server .. your only fudging RabbitMQ.. aka less hacky ;)) <mcella> :-) <jmgalloway> does anyone know why Im getting this error "bash: .ssh/authorized_keys: Permission denied" when I try to exchange keys from the cluster controller and the cloud controller? <RoAkSoAx> jmgalloway: probably something realted to the permissions of authorized_keys in the CC <RoAkSoAx> i experienced something similar <jmgalloway> I'm doing a clean install and this step is failing <RoAkSoAx> jmgalloway: yeah it is that then <RoAkSoAx> jmgalloway: might wanna take a look to eucalyptu's autherized_keys file, and check that the ownership is set to the eucalyptus user <jmgalloway> ok let me look <RoAkSoAx> jmgalloway: i experienced exactly the same issue <RoAkSoAx> Daviey: ^^ might be of your interest <jmgalloway> it is owned by root <jmgalloway> -rw-r--r-- 1 root root 396 2010-12-06 12:26 authorized_keys <jmgalloway> should I change "sudo -u eucalyptus ssh-copy-id -i ~eucalyptus/.ssh/id_rsa.pub eucalyptus@cc" to "sudo -u root ssh-copy-id -i ~eucalyptus/.ssh/id_rsa.pub eucalyptus@cc" <jmgalloway> or I could just chmod the permissions of the keys file.. #ubuntu-cloud 2010-12-08 <Makere> overnight all instances have lost IP-addresses <flaccid> which IP address? <Makere> public <flaccid> which cloud? <Makere> my own UEC <flaccid> ok <Makere> Not enough addresses left in the network subnet assigned to requested group: default <Makere> Why I get this when trying to launch multiple VM's <Makere> should be enough IP's left <Makere> I have 200 IP's reserved and trying to launch like 30VM's <RoAkSoAx> d <Makere> weird that every fifth instance I launch gets stuck on pending <TeTeT> Makere: maybe one of your node controllers is bad? <Makere> maybe <Makere> found 2 machines with no instances running <Makere> I mean <Makere> pending <Makere> :) <Makere> shutting them down <Makere> should reinstall fix those? <Makere> naw <Makere> they start pending on nodes that worked 100% <kim0> Hola everyone <kim0> Any new community members around <daker> kim0, no Q+A for today ? <kim0> daker: hey o/ <kim0> daker: well it's 15UTC .. but no one was active <kim0> I'm here if you have any questions though :) <daker> ok <zul> stupid natty <hallyn_> zul: ? <zul> hallyn_: am having issues <hallyn_> no i was asking about natty <hallyn_> j/k #ubuntu-cloud 2010-12-09 <Makere> Why the instances lose IP-addresses overnight? <Makere> "Public" IP <Makere> they still have the private one <Makere> I can associate IP-addresses but get no route to host <Makere> when trying to ssh <Makere> I think there's a bug that if you have a Cloud Controller, CC, Walrus and SC in one PC and try to install second machine as CC and SC, it doesn't ask for the Cluster name and assumes the same name as the old one <TeTeT> Makere: it's the auto discovery, if you want to have two clouds in the same LAN, it's basically not possible <TeTeT> Makere: you need to separate the two clouds completely, either by different LANs or VLANs, to the best of my knowledge <kim0> TeTeT: Does that mean UEC always assumes a single cluster installation <TeTeT> kim0: single cluster per LAN, yes, it's due to the auto discovery with avahi <TeTeT> kim0: for example in training we have to had 2 clouds in the same physical LAN, so the admins used a VLAN to separate them <kim0> TeTeT: So for a single cloud, with 2 clusters (2 CC nodes, each with multiple NC nodes). The CC nodes should be connected to CLC via different network segments ? <TeTeT> kim0: hmm, having some doubts now - I was talking about two clouds in a LAN, not two clusters. But it's quite possible that the two clusters setup is broken too <kim0> I think that was the original question <TeTeT> kim0: right, I misread that, apologies Makere <Makere> np <Makere> I have managed to setup a cloud controller + walrus on one PC and 2x CC and SC PC's <Makere> but if the Cloud Controller, Walrus and CC+SC are on the same PC and trying to setup another CC+SC, it doesn't ask for the Cluster name <Makere> and takes the same name <Makere> maybe because the SC somehow overrides that Q on the installation <VinceG> Hi, does anyone have a good ubuntu AMI to use with EC2 with their cluster compute instance? <VinceG> There's no official one yet <flaccid> VinceG: this may help http://support.rightscale.com/12-Guides/Dashboard_Users_Guide/Clouds/AWS_Region/EC2_Placement_Groups/Actions/Launch_a_Cluster_Compute_Instance . search for AMIs with 'HVM' in name <VinceG> thanks, flaccid <flaccid> np <CharlieSu> Is amazon having problems today? #ubuntu-cloud 2010-12-10 <buuo> Hey any one can help a cloud noob with some questions? :P <flaccid> no <flaccid> how long is a piece of string? <zoopster> that wasn't nice flaccid <zoopster> buuo: ask your question <flaccid> :) <mathiaz> smoser: hi! <mathiaz> smoser: where are the new releases of Ubuntu EC2 images usually announced? <mathiaz> smoser: ie which blog/mailing list should I follow? <smoser> ubuntu-cloud and ubunt-server i think <mathiaz> smoser: great - thanks! <smoser> but programattically, you should just check at http://uec-images.ubuntu.com/query <mathiaz> smoser: is amy08@example.org still active? <smoser> no. folded into -cloud <mathiaz> smoser: hm - nope <mathiaz> smoser: can the lucid images from http://uec-images.ubuntu.com/releases/10.04/release/ be used with kvm/libvirt? <smoser> not terribly easily. <smoser> https://code.launchpad.net/~smoser/+junk/boothooks is what i used in that timeframe to do local testing. <asac> smoser: so a script in user-data ... where does that log to? <smoser> asac, console <smoser> ie /dev/console, in ec2/uec that should be seeable in euca-get-console-output <asac> smoser: nice... thx. will try!! <smoser> asac, you can also very easily write it wherever you want <smoser> if its shell, just do <smoser> { <smoser> my script here <smoser> } 2>&1 | tee some-other-file.txt <asac> good idea ... now i would have to figure how to make wiki show my #!/bin/sh if i place it at beginning of line ;) <asac> smoser: https://wiki.linaro.org/Platform/Android/Specs/LinaroAndroidBuildInfrastructure#Experimental%20Node%20Scripts ... is that about the right approach to do something like this? <smoser> asac, yeah, that should work. <smoser> 2 things, though <smoser> a.) "cloud-config" can do similar things (the update an package install) <smoser> b.) see https://help.ubuntu.com/community/CloudInit for multipart information on user data. <smoser> multipart becomes useful if you've got separate sections of code that want to add things to cloud-config or cloud-init . then you can keep them separated in input parts <asac> right <asac> looks interesting <asac> for now its prototyping ... more important for me is to remember the internal ip of the controller so i can copy the results and logs back ;) <asac> i thought about encoding that in user-data ... but guess <asac> i will read through that page and see how i can make things cleaner with cloud-init/config <asac> smoser: user-data is constrained to 16k, right? <smoser> yes, but it can be compressed. <asac> ok ... just wonder how to best ensure that i dont grow more than that ... but seems its better to put everything in bzr like i try now <asac> and just have the update, apt, etc. stuff and then run that <smoser> "runit" in http://bazaar.launchpad.net/%7Eawstrial-dev/awstrial/trunk/annotate/head%3A/awstrial/trial/ec2_helper.py is an example of appending parts <asac> smoser: i know its lame, but is it safe to just copy a script like i have there in the user data field when launching instances on the webUI? <asac> or better use ec2- commands with --user-data-file? <asac> (well better is wrong question ...) <smoser> yes, it should work. <asac> cool ;) <smoser> regarding 'safe' no one makes promises on sensitive data you put in there though. <smoser> asac, please let me know if pasting into that box does not work when running from command line does (for plain text) <smoser> i dont ever launch from that ui <asac> sure ... safe == script is properly run ;) <asac> smoser: i am still not sure how to copy stuff back ... :/ <smoser> copy back ? <smoser> as in phone data home ? <asac> smoser: is hte idea to create a keypair for one run ... copy back to controller and then invalidate keypair? <asac> smoser: yeah <asac> like i want the logs and the image produced come back to my controller <asac> for persistence <smoser> right. <asac> but i dont know what best to do ... and how to ensure that that is not completely insecure <smoser> so it really just depends on how secure you want to be. <asac> like ... putting private id_rsa in user-data ;) <asac> what are the options? <smoser> well, no one makes promises on secure. but realistically, nothing outside your instance is going to amy08@example.org. <asac> yeah ... but what are the options? <smoser> so, you could: <smoser> - create a set of iam credentials, share that with the instance, let hte instance write to s3, then invalidate the credentials <smoser> (you can even make a set of credentials that only work from a single IP address) <smoser> - share a secret in user data or in an snapshot/ebs-volume or s3 to the instance, then have it use that to post back to the controller <smoser> (ie, in a header of http post) <smoser> probably lots of other ways... <asac> yeah <asac> i think i like the idea to http post a notification back ... and then have the controller pull stuff over <asac> rather than a push from the instance <asac> but i dont want to muss the notification because then i guess the shutdown would have to be done by controller as well <asac> and i hate having logic to shut down stuff after a certain amount of time if nothing happened ... though maybe i hav eto do that anyway <asac> i will think a bit more about it ;) <asac> so you say i can create a bucket for each instance and create a special key for just that bucket? <asac> in s3? <asac> hmm. i think there was a constrained of number of buckets <asac> smoser: last thing for today ... in user data web field ... base64 encoded ... should i check that? <asac> or does it mean that i pasted base64 encoded stuff? <asac> (guess the latter) <asac> ok fired off without base64 <smoser> there is a limit to the number of buckets you can have (i think) <smoser> but you may be able to limit to s3 bucket path <smoser> yeah, i would think it means you're pasting base64. <asac> ok ... will check that out... also look into messaging stuff etc. <smoser> i didn't know you coudl do that. <asac> let me see if my job runs ;) <smoser> asac, http://docs.amazonwebservices.com/IAM/latest/UserGuide/UsingWithS3.html <smoser> Example 5: Allow a partner to drop files into a specific portion of the corporate bucket <smoser> so you could have a bucketk "results" <asac> smoser: one question wrt to money ... if i shutdown after 5 minutes do i pay for full hour? <smoser> asac, yeah. <asac> damn ;) <smoser> the ceiling function kinda sucks. <asac> my script failed now i cannot shut it down <asac> guess i will try on micro next time ;) <smoser> there are ways to play faking user data post-first boot <asac> so the user data script is not run in /root/ home directory it seems <smoser> in maverick its fairly easy <asac> cannot find where it did the checkout :/ <smoser> hm... i dont actually know what the path woudl be. <asac> etting up bzrtools (2.2.0-2) ... <asac> Processing triggers for python-central ... <asac> You have not informed bzr of your Launchpad ID, and you must do this to <asac> write to Launchpad or access private data. See "bzr help launchpad-login". <asac> Branched 2 revision(s). <smoser> nothing changes cwd so, whatever upstart would run something at <asac> /var/lib/cloud/data/scripts/part-000: 8: ./ec2-tools/node/setup-android-build-node: not found <smoser> https://groups.google.com/group/ec2ubuntu/browse_thread/thread/d4d51238a2afb55b/bca69f3e89c88d0b?lnk=gst&q=user+data#bca69f3e89c88d0b discusses how you can test user data after initial launch <asac> thx <asac> guess i will use a spot instance next time i try it ;) <smoser> asac, its only $0.10 <asac> let me check <asac> smoser: 0,34 <asac> smoser: problem is i need 64-bit for android <asac> so either micro with slow EBS ... or large :/ <smoser> true. <asac> not really a big problem either ;) ... will just expense it <smoser> i do use spots a lot for playing. the problem with them for development is that they're slower to start <smoser> ie, total time till up and reachable will be probably 8 minutes rather than < 3. <asac> damn <asac> my auto java license accept didnt work :((( <asac> guess i have to set seen as well <asac> yeah <asac> smoser: do spot instances get killed earliest after 1h? e.g. if i pay in 1h rates? <smoser> you always pay in 1 hour in crements. <smoser> i was always under the impression that spot instances would be killed on 1 hour units. but i'm not sure. <smoser> ie, it would seem somewhat unfair for you got get a spot instance, and then them kill you after 10 minutes and charge you for 55 <smoser> er... for 60 <smoser> soren, you around ? <patrickw> Why don't UEC images have devpts in fstab? Is it unnecessary because of the kernel? I'm customizing a Centos image (that will use an Ubuntu kernel) and was wondering about this difference. <asac> smoser: http://paste.ubuntu.com/542014/ ... m2.xlarge with latest instance store 64-bit maverick :/ <asac> doesnt boot :/ <smoser> known issue <asac> smoser: oh ... so just for that instance type? <smoser> hold on. getting bug <asac> can i select a different kernel? <smoser> bug 651370 <uvirtbot> Launchpad bug 651370 in linux "ec2 kernel crash invalid opcode 0000 [#1]" [Medium,Fix committed] https://launchpad.net/bugs/651370 <smoser> you can't specify a different kernel <smoser> asac, hopefully updates next week maybe, with that through -proposed <asac> hmm <asac> i can select kernels when creating an instance ... whats that? <smoser> its what you think. <smoser> sort of <smoser> :) <asac> why cant proposed kernels be made available there? <smoser> you can select a kernel, and prior to maverick, we would have published that kernel to ec2 that you could select <asac> ah <smoser> but with maverick the kernel is "pv-grub" <asac> maybe "proposed" AMIs ? <smoser> and pv-grub loads the kernel from inside the image <asac> oh <asac> ok <smoser> proposed amis woudl be somewhat useful for this sort of thing <asac> yeah <asac> definitly ;) <asac> i would use it!! <smoser> and right now, lucid is building with --proposed <smoser> because i wanted to test some things, but in general, the idea is that the dailies get turned into the rleeases <smoser> bit for bit <smoser> maybe i could start doing a weekly -proposed, which would never get promoted <smoser> but i tihnk in general we have too much selection up there (with the daily builds) and that confuses people. <asac> hmm <asac> selection where? you mean on uec-images? ... maybe <smoser> (ie, you wouldn't get or expect an ISO to have -proposed enabled or have been built from there) <smoser> as in selection, i really mean in number of "ubuntu" images on ec2 <asac> so this bug ... is that juts for my instance type? or just more frequent for large memory instances? <asac> the reporter talksa bout t1.micro <asac> but my previous instances worked well <asac> like m1.large etc. <asac> just m2.xlarge is causing pain now <smoser> its really only the larger memory instances. <asac> damn ;) <asac> those i want to test now :-P <smoser> t1.micro is what the opener (me) opened it with <asac> ah <smoser> i just launch a t1.micro and run 'ubuntu-bug' <smoser> especially if you can't seem to get a instance type that you want to open to start :) <asac> alrighty <smoser> the issue is really with getting a certain processor <asac> so is lucid perfect? <asac> ;) <smoser> lucid has some issues too. <asac> booting big memroy? <smoser> lucid's big issue is https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853 <uvirtbot> Launchpad bug 614853 in linux-ec2 "kernel panic divide error: 0000 [#1] SMP" [Medium,Confirmed] <smoser> there is a patch, but you can look at it (http://launchpadlibrarian.net/58956370/lp614853.patch) its not that pretty <soren> smoser: Yes. <smoser> soren, i'll probaly try to ping you monday <soren> smoser: I'm intrigued :) <smoser> soren, well, it was regarding the ec2 metadata swrevi ce <smoser> andthe fun that that that delivers <smoser> basically, i'm trying to make "full disk images" of the uec images, that ccan then just be booted under kvm. <smoser> in maverick i did this, but i did so via a floppy and bootinga partition image. the floppy loaded the kernel from inside the image but passes some custom kernel parameters that disable the waiting for the ec2 metadata service <soren> smoser: How does that relate to metadata? The block mapping? <smoser> but if i just have one disk image, i'd like for its contents to boot correctly and not wait some stupid long time for the metadata service to come about. <smoser> i was wondering if you had any good ideas for how i could decide when to wait for it. but i can't use kernel parameters, because on uec, the grub.cfg is used for passing kernel params (so i can't just have that disable ec2 wiating) <soren> I remember having this discussion a couple of times. :) <soren> With you, even :) <soren> I think our conclusion was that there's no really good way to tell the difference. <soren> smoser: Ok, let's attack this from a slightly different angle. <smoser> soren, no, ther isn't. <soren> smoser: What if we try to minimise the impact of this wait time? Why do we wait for it to turn up? <smoser> and i said i should have waited till monday as i have to run righ tnow. <soren> Instead of just letting it wait in the background. <soren> smoser: Ah. <soren> :) <smoser> we wait for it to turn up because its possibly important <soren> Alright, Monday it is. <soren> smoser: Use cases? <smoser> really its only most important on first boot <smoser> but the idea is that the user can do things on first boot to an instance that will get done <smoser> such as setting mount points or importing ssh keys... <smoser> some of the stuff might be difficult to do "later" <smoser> anyway, i do have to run. <soren> Sure. <soren> I'd be happy to talk on Monday. <soren> Have fun. #ubuntu-cloud 2010-12-11 <rsvp> so I installed cloud-init via synaptic... there's no man page... so is there a good intro to its functions?? <flaccid> rsvp: on the wiki <smoser`> rsvp, you dont' twant to install it on a non "clodu" machine (ie, outside of uec or ec2) <smoser`> you should uninstall before you reboot <rsvp> smoser, thanks for the clarification re: "download cloud-init only on machines in the cloud (not your desktop)" -- perhaps it could be more explicit on the wiki page and in the apt description -- since otherwise an user could get the impression that it helps generate/merge/prettify/test the config file. <rsvp> smoser, another feature that could be made more explicit: "alternatively place a URL pointing to the config file, instead of the config file." -- I'm not sure what exactly are the conditions on that URL and whether that method could circumvent the file size limitation. That would be a nice feature since machines on the cloud could then consistently initialize from a large file in a version controlled repository. <rsvp> smoser, could you kindly clarify the URL_as_config for cloud-init ?? and thanks so much for your effort and work! <Evet> 2 servers are enough to build a private cloud? #ubuntu-cloud 2010-12-12 <smoser> rsvp, regarding "not desktop" that is bug 635188 . i hope to get that fixed in 11.04. <uvirtbot> Launchpad bug 635188 in cloud-init "installing cloud-init on a non-ec2/UEC server results in a 20 minute boot wait" [Low,Triaged] https://launchpad.net/bugs/635188 <smoser> regarding "URL_as_config" where did you see that ? <rsvp> smoser, here's the URL_as_config reference: "Note, the maximum total size that can be passed to user-data is 16K. If you are above that limit, you can gzip the data or you can move the items to files accessable through a URL and pass the URLs to user-data." http://www.rndguy.ca/2010/11/20/using-cloudinit-to-automate-linux-ec2-setup/ <smoser> ah. ok. <smoser> rsvp, so if you do do something like: <rsvp> so does the URL_as_config "feature (if it really exists -- no time so far this weekend to confirm)" work at the Amazon AWS web console input box ?? <smoser> printf "%s\n%s\n" "#include" "http://your.site.here/some-url.cloud-config" > my.config <smoser> ec2-run-instances --user-data-file my.config <smoser> then cloud-init will read the "#include" and pull the data from the url just as if it had been inline. <smoser> rsvp, it should work there, yes. i've not actually tested, but putting the same there should be same as on config line. <smoser> https://help.ubuntu.com/community/CloudInit explains #include some. <rsvp> yes, that makes sense... maybe if the web input box only contains one URL line, cloud-init could do that my.config thing implicitly in the background. <smoser> it could also at least allow for a single line: #include http://... <rsvp> yes, that would be super convenient! <smoser> if you want to open a bug on that, i'd look into it more. see if i could fit it in backwards compatibly. <rsvp> ok cool <Evet> what are the minimum specs of a vm i can create? #ubuntu-cloud 2010-12-13 <liam> is there an ubuntu ec2 ami image with only 10gb hdd ebs that anyone can link? <rsvp> liam, check http://cloud.ubuntu.com/ami <asac> smoser: please let me know when there are new AMIs out after kernel landing for the high memory instances? <asac> thx!! <flaccid> asac: you have problem with current AMIs? <asac> flaccid: yes ... kernel bug https://bugs.launchpad.net/ubuntu/+source/linux/+bug/651370 <uvirtbot> Launchpad bug 651370 in linux "ec2 kernel crash invalid opcode 0000 [#1]" [Medium,Fix committed] <asac> for high memory instances <flaccid> ah that one * asac would love to see a verification AMI being made for this bug ;) <flaccid> smoser is most likely on the case <asac> yeah ... talked about that with him <asac> eventually it will go to -updates ... and then new AMIs to be made <asac> i hope the proposed doesnt take much longer though :/ <asac> hmm ... i wonder if i need to tweak some sys/proc stuff to get _more_ file caching on high memory <asac> IO is slow so i would like the full build tree to live in mem <asac> btw, are there other cloud services with better IO that are nicely integrated with ubuntu? <asac> i found storm and gogrid, but those seem to be not really well API/tools wise <flaccid> rackspace is available <asac> but rackspace doesnt really have better IO, right? <flaccid> test it out i guess <asac> http://blog.cloudharmony.com/2010/06/disk-io-benchmarking-in-cloud.html <asac> yeah ... <asac> i guess i will wait for high mem and put whole build on tmpfs ;) <Evet> i want whole server to make a single cloud node <Evet> do i still need host os, guest os seperation? <kim0> Evet: The only supported configuration is 2 machines at least for a cloud, although some people have made it work on a single machine (sort of) <Evet> kim0: i have multiple servers <Evet> want to build one node per physical server <kim0> Evet: the way it works is to have one cloud controller node (CLC) .. and all other boxes become Node Controllers (NCs) .. would that work for you ? <Evet> kim0: yes <kim0> Evet: great .. that's the standard setup :) <kim0> Evet: https://help.ubuntu.com/community/UEC/CDInstall <Evet> kim0: so can create 4gb ram node on a 4gb box? <kim0> Evet: 4G VM on 4G physical box .. yeah should be ok I guess (3.5G or so would surely work) otherwise your bare-metal installation would be close to running out of ram <Evet> hmm, max is 3.5 <kim0> Evet: I'm not saying that .. I'm just guessing .. feel free to try it <smoser> asac,
{'channel': 'ubuntu-cloud', 'month': 12}