#ubuntu-ec2 2010-06-05
<boxbeatsy> hi, i apologize for the newb question, but once I start up an EC2 instance, how do I upload my application or dl pkgs from the internet?  Do i need to configure ftp or should i use scp?
<flaccid> flaccid: boxbeatsy: depends what you are trying to install and where from..
#ubuntu-ec2 2010-06-08
<amalgameate> hey guys, is there a standard ubuntu AMI you can recommend to get started with?
<amalgameate> im insatlling http://groups.google.com/group/typhoonae and they give no instructions past use Debian Ubuntu
<amalgameate> it apparently installs everything else fro you
<flaccid> amalgameate: the alestic ones are good
<amalgameate> flaccid: sweet thanks :]
<flaccid> np
<flaccid> alestic.com
<flaccid> well the later ubuntu ones there are canonical
<amalgameate> hi flaccid i chose one of the most recent alestic AMI's and i'm trying to install java, but when i do apt-get java-6-sun isnt available. would you happen to know why or how to get it?
<amalgameate> (turns out i need to insatll python2.5, and java myself)
<flaccid> amalgameate: thats the wrong package name. java install is outlined on the ubuntu help/wiki site
<amalgameate> o i see, thanks!
#ubuntu-ec2 2010-06-09
<flaccid> hey erichammond looks like vmbuilder only supports Ubuntu distro ?
<flaccid> VMBuilder.exception.VMBuilderUserError: No such distro. Available distros: ubuntu
<flaccid> appears to be the case
<erichammond> flaccid: Yes, vmbuilder only supports Ubuntu right now.
<flaccid> erichammond: i guess they probably don't have plans to support Debian?
#ubuntu-ec2 2010-06-16
<flaccid> i'm researching ec2 kernels and mods - what to use for debian particularly
<flaccid> confused on what to use basically
<flaccid> ubuntu lucid i386 linux image 2 6 32 302 ec2 v 2 6 32 302 6 kernel i think will work ok with squeeze
<erichammond> flaccid: The supported Ubuntu kernel images (AKIs) would be the ones which are used by the active public Ubuntu AMIs.
<erichammond> Anything else is potentially subject to removal, which could ruin your day.
<flaccid> well in terms of debian however?
<flaccid> squeeze seems to run well on the lucid aki but needs the linux modules (which i guess i could install from the lucid repos)
<flaccid> erichammond what you think?
<flaccid> http://thecloudmarket.com/image/ami-bb3a6bfe--debian-6-0-squeeze-server#/definition uses older fedora kernel/ramdisk images
<erichammond> I abandoned support for Debian squeeze because of kernel incompatibilities.  I would avoid any Alestic squeeze AMIs.
<flaccid> well i'm finally getting around to the debian support thing
<flaccid> did lenny play nicely enough with the amazon kernels?
<erichammond> I'd recommend building your AMIs from scratch so that you can reliably reproduce the process as time goes on.
<flaccid> erichammond: yep thats what i'm doing now
<flaccid> i have created a servertemplate
<flaccid> erichammond: i've updated http://support.rightscale.com/12-Guides/RightLink/04-Creating_RightScale-enabled_Images_with_RightLink/RightLink_Installer_for_Ubuntu to work with lucid
<flaccid> you can add that to your vmbuilder process which will give the latest rightlink v5 package
<flaccid> so now i'm going back to cover debian
<flaccid> wb
<flaccid> i basically would like to know the link between the kernel and it loading kernel mods from /lib/modules on boot
<flaccid> it appears to be a hardcoded relationship
<flaccid> i'm trying to make the ami independent of the kernel used
<erichammond1> The kernel modules must match the kernel version exactly.
<erichammond> Whenever you run a different kernel, you must have those kernel modules available.
<erichammond> Ubuntu was looking at downloading the kernel modules at run time or adding them to the ARI instead of the AMI.
<erichammond> but I don't think that went anywhere.
<flaccid> yeah its cool to have a switchable kernel but if the mods are not installed then no good
<flaccid> so i guess the ami can only support 'a range of kernels'
<erichammond> Yes, this is the general practice.
<flaccid> so where would i get the kernel modules for http://thecloudmarket.com/image/aki-f70657b2.html?page=6 (aki-f70657b2) ?
<flaccid> this is the default kernel to Debian Squeeze Alestic, root@
<erichammond> Most often it is a single recommended kernel and new AMIs are released when new kernels come out.
<erichammond> flaccid: What region?
<flaccid> us-west atm
<flaccid> i just want to bundle vol with the same kernel and ramdisk as you are and see how it boots
<erichammond> If you have an AMI that uses that kernel, then the kernel modules should be installed on that AMI.
<flaccid> erichammond: thats what i am trying to do
<erichammond> No, I'm saying you could copy the kernel modules from the Alestic AMI.
<flaccid> erichammond: where did they come from originally? i'm debootstrapping
<erichammond> The Alestic Debian AMIs were built with ec2ubuntu-build-ami which would have pointers to the downloadable kernel modules.
<flaccid> must of missed it in http://ec2ubuntu.googlecode.com/svn/trunk/bin/ec2ubuntu-build-ami
<flaccid> http://alestic-downloads.s3.amazonaws.com/ec2-kernel-modules-156.14.161.142-2.fc8xen-ec2-x86_64-20090930.tar.gz
<erichammond> looks right.
<flaccid> doing a re-run now
<erichammond> flaccid: The point, however, was that that old Amazon kernel will no longer work with squeeze.
<flaccid> does it work with lenny?
<flaccid> i'm also creating the bundle on /tmp/image-bundles/foo of the deboostrap root. does not seem like an ebs volume is required for s3 only building
<flaccid> bundled created:
<flaccid> Kernel image	Amazon vmlinuz 2.6.21 2.fc8xen ec2 v1.0.i386.aki
<flaccid> Ramdisk image	Amazon initrd (864)930-0890 2.fc8xen ec2 v1.0.1.i386.ari
<flaccid> boot instance
<flaccid> doesn't seem to like lenny
<erichammond> flaccid: The old lenny AMI I built months ago still runs.  It's possible lenny upgrades no longer work with that old kernel.
<flaccid> erichammond: i got squeeze going with an ubuntu lucid kernel
<erichammond> flaccid: Congrats on getting squeeze to run.  The ec2debian Google group might be interested in your findings.
<smoser> erichammond, "new 32-bit Karmic ebs server based on Alestic's ami-6743ae0e"
<smoser> when will you stop taking credit for Ubuntu's work ;-)
<erichammond> Where are you reading that?
<smoser> i'm amazed at the namespace you've carved out (and i completely realize that you give Ubuntu credit and correct those statements)
<smoser> ec2ubuntu mail
<erichammond> It's amazed me, too.
<erichammond> I did originally recommend that you put "canonical" in the bucket names to make the source clear to folks.
<erichammond> oh, wait.  That AMI *is* one I published.
<erichammond> I use the Ubuntu image built by Canonical, but then, pretty much every Linux image anybody publishes uses files created by somebody else.
<erichammond> :)
<erichammond> The key is that if I die and Amazon cancels my account, that AMI will disappear :-/
<smoser> gah! it is, you're right.
<smoser> your naming is the same as ours too :)
<smoser> so i was confused.
<erichammond> er, your naming is the same as min :)
<erichammond> e
<smoser> anyway
<smoser> sorry
<erichammond> but your original point is true with other examples.
<smoser> thats what you get for doing a helluva good job
<smoser> i'll respond to that thread with: "I would help you, but i have no idea what that crazy 'alestic.com' does with their images"
<smoser> :)
<erichammond> :)
<erichammond> I think you already know this, but the Karmic EBS boot AMIs I published are exact file system image copies from your published Ubuntu images.
<smoser> yeah i know
<erichammond> smoser: Thanks for all the great support you've been providing to the ec2ubuntu group.  If you'd like to take over approving new posters and filtering spam, let me know :)
<flaccid> erichammond: and smoser yeah guys still not very happy with kernels for debian* atm. still pondering on what to use that will work with at least lenny and squeeze without even the smallest issue
#ubuntu-ec2 2010-06-17
<smoser> flaccid, what are your issues with the ubuntu kernels ?
#ubuntu-ec2 2010-06-18
<flaccid> smoser: http://dpaste.org/Se9H/ only a couple of things - Couldn't get a file descriptor referring to the console; error: unexpectedly disconnected from boot status daemon; FATAL: Could not load /lib/modules/2.6.32-305-ec2/modules.dep: No such file or directory (need to work out best way to install these on squeeze/debian);
<flaccid> and Waiting for /dev to be fully populated...udevd-work[258]: kernel-provided name 'evtchn' and NAME= 'xen/evtchn' disagree, please use SYMLINK+= or change the kernel to provide the proper name
<flaccid> and Setting kernel variables ...error: "net.ipv6.bindv6only" is an unknown key
<flaccid> most of the above is kernel related. smoser i just need to address/fix the above and all should be good. if you could also advise what the latest actual kernel is that would be great
<smoser> "actual kernel"
<smoser> all the ubuntu published items conform to https://wiki.ubuntu.com/UEC/Images/NamingConvention
<flaccid> yes
<smoser> (or close to it)
<smoser> for released kernels, they'll have ubuntu-images/<...> in their name (per describe-images)
<smoser> er.. for kernels it will be
<flaccid> yeah so what is the latest kernel id ?
<smoser> well, for us-east-1, and lucid (released) that would be:
<smoser> 099720109477/ubuntu-kernels/ubuntu-lucid-amd64-linux-image-2.6.32-305-ec2-v-2.6.32-305.9-kernel
<smoser> aki-0b4aa462
<flaccid> is there someone that lists these ?
<smoser> i have considered putting kernels in the data at http://uec-images.ubuntu.com/query/
<smoser> but largely, we don't consider kernels aki to be a product as much as a side effect of the image
<smoser> that said, http://uec-images.ubuntu.com/query/lucid/server/released.current.txt
<smoser> will have a list of the latest ami
<smoser> and their assicoated aki
<flaccid> thanks that helps a lot
<smoser> there is 'daily.current.txt' there also
<flaccid> now back to the errors if you don't mind helping?
<smoser> and maverick ...
<smoser> so lots to chose from
<smoser> :)
<smoser> i'vve seen the "Couldn't get a file descriptor" error before, but dont really recall what that was about.
<smoser> fwiw, jjohansen is the kernel team member for this ec2
<smoser> i'm looking at some logs
<smoser> we see the "Could'nt get a file descriptor" on karmic currently
<flaccid> cool thanks
<smoser> and on lucid prior to alpha2
<smoser> err.. prior to alpha3
<flaccid> and would also like your advise on installing/enabling the kernel mods from the ubuntu package in debian and if this is an ok practice etc.
<smoser> where did you get the ramdisk ?
<smoser> or am i mistaken:
<smoser> Begin: Loading essential drivers... ...
<smoser> Done.
<smoser> Begin: Running /scripts/init-premount ...
<flaccid> its ubuntu lucid i386 linux image 2 6 32 301 ec2 v 2 6 32 301 4 ramdisk 20100114 1
<flaccid> ari-3f2d7c7a
<smoser> yeah, so we dont use ramdisk anymore
<flaccid> right so set to default
<smoser> set to default ?
<flaccid> i'm assuming default=none now
<flaccid> i'll relaunch with that
<smoser> yeah, it probably wont hurt or affect it.
<smoser> that has been my experience.  the ramdisk doesn't hurt anything. just doesn't help.
<smoser> regarding installing ubuntu packages, are you asking from licensing perspective ?
<smoser> those modules come from the packages... our ec2 kernels are normal kernle builds and have normal packages.
<smoser> https://launchpad.net/ubuntu/+source/linux-ec2
<smoser> i dont know how well those would install onto a debian
<flaccid> more from an operational point of view. this is to address FATAL: Could not load /lib/modules/2.6.32-305-ec2/modules.dep: No such file or directory
<flaccid> so yes the question is regarding installing them on debian
<flaccid> see because the amazon kernels are not working with debian, i am having to use the ubuntu kernels
<smoser> yeah. so i dont know whether or not the packages there will "just work" (ie install) on debian or not.
<smoser> but almost certainly a dpkg -x linux-image-*.deb would get you the /lib/modules/ tha tyou need
<flaccid> well i would either have to use the lucid repos briefly and install via apt or manually install the package and its deps via dpkg
<flaccid> well the linux-image-* pkgs have deps..
<flaccid> ok well no ramdisk fixed that
<flaccid> zcat: invalid magic - another small one to address
<flaccid> i mean no ramdisk fixed the console descriptor errors
<flaccid> well it still defaulted to ubuntu lucid i386 linux image 2 6 32 301 ec2 v 2 6 32 301 4 ramdisk 20100108 smoser0
<flaccid> that must be tied to the kernel
<flaccid> wb
<flaccid> smoser: if i do a reboot and i have installed these modules, this should remove the FATAL: Could not load /lib/modules/2.6.32-305-ec2/modules.dep: No such file or directory message right?
<smoser> well maybe
<smoser> you may be getting that from the ramdisk
<smoser> at least i'm fairly certain that ramdisks do 'modprobe' which will result in a message like that.
<flaccid> but if the file is there, it should not return No such file or directory right?
<smoser> are you using a ramdisk ?
<flaccid> this kernel appears to default to ubuntu lucid i386 linux image 2 6 32 301 ec2 v 2 6 32 301 4 ramdisk 20100108 smoser0
<smoser> if you are using a ramdisk,and it was built with a differnet kernel then it won't have that /lib/modules/<kernel-ver> and it will try to modprobe
<flaccid> aki-296f3e6c
<smoser> kernels dont default to a ramdisk
<smoser> this is something that sucks.
<flaccid> oh but an image defaults to a ramdisk?
<smoser> if you registered an image with a ramdisk and kernel, then you are not able (that I know of) to boot without a ramdisk or kernel
<smoser> and, actually, it goes back step, since you can give ramdisk and kernel to the bundle command
<flaccid> ok so ramdisk is bogus, no need to build image with a ramdisk value?
<smoser> if you've done that, then there is no way to specify "None'
<smoser> right
<smoser> then, you can (if you wanted) specify --ramdisk <ari-abcdefg>
<smoser> but there is no --ramdisk <none>
<flaccid> sweet then i'll omit ramdisk
<smoser> right. thats a superset of the functionality, but then, if you ever wanted a ramdisk, the user has to specify one, which just isn't very friendly
<flaccid> if ec2-bundle-vol rcvs --kernel --ramdisk --generate-fstab will it simply ignore the no param values
<smoser> just do not pass --ramdisk
<smoser> or --kernel
<smoser> on bundle or on register
<smoser> fwiw, there are scripts 'uec-publish-image' that you could use that, in my opinion are more friendly than ec2-bundle, ec2-upload, ec2-register
<flaccid> well i still want a default kernel
<smoser> http://manpages.ubuntu.com/manpages/lucid/man1/uec-publish-image.1.html
<smoser> right.
<smoser> i *think* (you'd have to test this)
<smoser> that you can bundle without anything
<smoser> and then add --kernel when you register the image
<smoser> which, in this case, would allow you to avoid re-bundling
<flaccid> thats cool i'll test that
<flaccid> hmm it doesn't specify params http://docs.amazonwebservices.com/AmazonEC2/dg/2007-01-03/CLTRG-register.html
<smoser> see the date in that url
<smoser> http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/index.html?ApiReference-cmd-RegisterImage.html
<smoser> is what you want
<flaccid> oh yeah thanks, eek on google!
<flaccid> thanks muchly :)
<flaccid> standby, i'll have some results soon
<flaccid> 1 is building now
<flaccid> the good thing i've noticed so far is that these AMIs i'm creating work fine with the lucid kernel and boot up really fast
<flaccid> hmm that doc doesn't mention -K and -C options yet i am using them currently..
<flaccid> Required option '-K, --private-key KEY' missing (-h for usage)
<flaccid> heh its not in their doc
<flaccid> ah must be in http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/index.html?ApiReference-cmd-RegisterImage.html
<flaccid> err http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/index.html?CLTRG-common-args-api.html
<flaccid> Unable to read instance meta-data for ramdisk-id
<flaccid> Unable to read instance meta-data for product-codes
<flaccid> ^^ from ec2-bundle-vol. what should i do about this ?
<uvirtbot> flaccid: Error: "^" is not a valid command.
<flaccid> smoser: built and registered with no ramdisk, a launch with default assigns ubuntu lucid i386 linux image 2 6 32 301 ec2 v 2 6 32 301 4 ramdisk 20100108 smoser0
<flaccid> the default kernel being assigned is ubuntu-kernels-testing/ubuntu-lucid-i386-linux-image-2.6.32-306-ec2-v-2.6.32-306.11-kernel
<smoser> those errors don't surprise me.
<flaccid> i guess we'll see in console output if it is wanting 306 or 301 modules that will determine if the ramdisk is asking for it or not
<flaccid> yes but i want to get rid of those errors :)
<smoser> no, the "unable to read instance metadat" errors from bundle-vol
<smoser> they dont surprise me
<flaccid> yeah i want to fix those..
<smoser> bundle-vol is a yucky hack. you shouldn't rely on it.
<flaccid> well thats what aws publishes..
<smoser> bundle-vol basically has to "un-boot" a system.
<smoser> which is just simply going to require a bunch of distribution/OS specific HACKs
<smoser> its much cleaner to use debootstrap and bundle-img
<flaccid> well FATAL: Could not load /lib/modules/2.6.32-306-ec2/modules.dep: No such file or directory. so its the kernel that wants it. i'll extra the mods in this build
<flaccid> where is bundle-img ?
<flaccid> i am using debootstrap
<smoser> ec2-ami-tools
<flaccid> oh you mean ec2-bundle-img
<smoser> yeah
<smoser> can you give me output of describe-istances for the image you booted ?
<smoser> i think you booted with a ramdisk.
<smoser> or, you dont 'have those modules in the root filesystem
<smoser> ec2-describe-instances i-abcdefg
<flaccid> for some reason that returns Client.InvalidInstanceID.NotFound: The instance ID 'i-ba897dfe' does not exist
<flaccid> yet its running
<flaccid> i didn't include the modules in that build however
<flaccid> rightscale dashboard is indicating use of ubuntu lucid i386 linux image 2 6 32 301 ec2 v 2 6 32 301 4 ramdisk 20100108 smoser0
<smoser> it maybe that rightscale is buggy
<smoser> if you did not bundle with '--ramdisk'
<smoser> and did not register with '--ramdisk'
<smoser> then you should not have a kernel
<smoser> err... you should not have a ramdisk
<smoser> it is possible that rightscale is being nice, and trying to use that one, in which case, that really sucks.
<smoser> but describe-instances certinaly should be able to give you output
<smoser> what about describe-images of the ami
<flaccid> that returns does not exist as well. no idea why
<flaccid> hmm are these region specific?
<flaccid> i think its only returning us-east when i need us-west
<smoser> yes. all of it is region specific
<smoser> each region is its own "cloud". the only data they share is your account id
<flaccid> nps
<flaccid> this is the image
<flaccid> IMAGE	ami-b5b8e9f0	041819229125/RightImage-Debian-squeeze-20100618033019-i386-v5.4.6	041819229125	available	private		i386	machine	aki-3197c674			instance-store
<flaccid> this is the instance
<flaccid> INSTANCE	i-ba897dfe	ami-e7b6e7a2	ec2-204-236-179-7.us-west-1.compute.amazonaws.com	ip-10-162-143-68.us-west-1.compute.internal	running	CJF: AWS US-West (testing)	0		m1.small	2010-06-18T03:39:43+0000	us-west-1b	aki-c18edf84	ari-8f2978ca		monitoring-disabled	(864)930-0890	(864)930-0890			instance-store	
<flaccid> so i guess either rightscale is allocating ari-8f2978ca or ec2 (likely rightscale)
<flaccid> can you help confirm that there is no such thing as a 'default' ramdisk when registering or launching?
<flaccid> we seem to be using 'default' with a reserved ari then..
<smoser> yep
<smoser> its rightscale is being "nice"
<smoser> well, when you register or bundle and pass '--ramdisk' that gives you a "default ramdisk"
<smoser> and if you do that, you can't ever run an instance without a ramdisk.
<smoser> but, the describe-images info above shows that the image has no ramdisk associated with it.
<smoser> which means either
<smoser> a.) the user must provide one in run-instances
<smoser> b.) there will not be one used
<flaccid> that would of came up in the describe-images but right? i took this out of the build process
<flaccid> yep we are on same level
<smoser> right. it would be shown in describe-images output
<flaccid> i will raise a bug for the rightscale dashboard issue
<smoser> look at any other one, you'll see an ari
<smoser> fwiw, that 'smoser0' is a debug ramdisk :)
<smoser> (with some interesting characteristics, it can process user data)
<flaccid> hehe, looks like we chose your ramdisk image for the default... :)
<smoser> our choice to not have ramdisks has thrown errors much furrther than i thought it would have
<flaccid> there is issues with no ramdisk?
<flaccid> is there any way to remove a debootstrap target dir? i.e. you can't remove /proc and /dev
<smoser> you have to unmount proc and dev first
<smoser> then remove
<smoser> there are no issues with no ramdisk other than tools that expected a ramdisk
<smoser> UEC had fits (Eucalyptus)
<smoser> righscale apparently had fits
<smoser> the expectation of an ari was just hard coded many places
<smoser> oh yeah, other places.
<smoser> landscape
<smoser> and the canonical "image store" (part of UEC)
<flaccid> i see
<flaccid> ok now launching with a new image
<flaccid> smoser: extracting from the .deb for those modules doesn't give the desired files
<smoser> no?
<smoser> you're right.sorry.
<flaccid> i'll cp them across from the worker which is lucid
<smoser> just the wrong package
<smoser> the linux-ec2 is a metapackage
<smoser> that depends on the right linux-image-2.6.32-305-ec2
<flaccid> i'm extracting linux-image-2.6.32-305-ec2_2.6.32-305.9_i386.deb
<flaccid> i tell ya what but it boots so fast
<smoser> http://packages.ubuntu.com/hu/lucid/linux-image-2.6.32-305-ec2
<smoser> yeah, thats what you want
<flaccid> hopefully cp -R /lib/modules/2.6.32-305-ec2 "$DEBOOTSTRAP_TARGET_DIR"/lib/modules/ is good enough..
<smoser> i'd think so
<flaccid> i'm getting 1.2min startups
<flaccid> 14.5mins to create and build an AMI
<flaccid> smoser: only a few things left to address now, http://dpaste.org/5fAp/
<smoser> start up is all over the map
<smoser> what do you think is left ?
<smoser> it looks pretty good
<smoser> what do you mjean by "1.2min startups" ?
<smoser> total time to login/ssh from what
<smoser> flaccid,
<flaccid> i think thats the boot time, not sure how its configured
<flaccid> need to address the ' WARNING: Currently emulating unsupported memory accesses'
<flaccid> there is no /lib/tls on that instance
<flaccid> also Waiting for /dev to be fully populated...udevd-work[86]: kernel-provided name 'evtchn' and NAME= 'xen/evtchn' disagree, please use SYMLINK+= or change the kernel to provide the proper name
<flaccid> and the few under cannot unset powersave mode
<flaccid> and Setting kernel variables ...error: "net.ipv6.bindv6only" is an unknown key
<flaccid> and maybe /build/buildd/linux-ec2-2.6.32/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
<flaccid> also Setting kernel variables ...error: "net.ipv6.bindv6only" is an unknown key
<flaccid> i'll build a lenny now
<flaccid> lenny works but not with rightlink
<smoser> i'm going to bed. good luck.
<flaccid> thanks; cya later
#ubuntu-ec2 2010-06-24
<rberger> Any idea how to get the hostname (as reported by hostname -f) is a FQDN on Eucalyptus / UEC? Right now hostname is just set to a representation of the IP address with no domainname. This breaks a lot of packages.
<erichammond> rberger: You might try asking over on #ubuntu-server or #eucalyptus
<erichammond> rberger: Nothing happens on this channel.
<rberger> erichammond thanks for the tip!
<rberger> I had already craighoward@example.net. No real answer. It may be a UEC thing
#ubuntu-ec2 2010-06-26
<flaccid> guys i'm looking at https://wiki.ubuntu.com/UEC/Images/NamingConvention and i can't actually see a naming conventions for the images themselves..
#ubuntu-ec2 2010-06-28
<flaccid> smoser: i took your advice; now i am doing ec2-bundle-image; just about to test one
<flaccid> erichammond, smoser : i noticed that bundling an image and uploading takes longer, i guess this is because its a full uncompressed img, in this case 10GB, right?
<erichammond> flaccid: bundling and uploading an image takes longer than what?
<flaccid> just saying that ec2-bundle-image takes longer than ec2-bundle-vol
<flaccid> because the image has a physical size
<flaccid> thus why lots of parts compared to the compressed userland on bundle vol
<erichammond> flaccid: Ah.  I use bundle-vol.  It avoids potential security issues with deleted files and lets you start up faster.
<flaccid> yeah, i found the opposite with the security files thing
<erichammond> flaccid: However, I think S3 based AMIs are obsolescent and should only be used for rare cases.  EBS boot should be the default for most folks.
<flaccid> so you think ebs boot even if you are not going to bundle again i.e. true appliance?
<erichammond> flaccid: With EBS boot you snapshot instead of bundling.  You can register the snapshot as a new AMI if you want.
<flaccid> yeah
<flaccid> i'm talking about re-snaphotting in this case for persistence
<flaccid> i think you can ebs vol boot as well
<flaccid> 625seconds to bundle from an image
<flaccid> that is considerably more
<flaccid> 36 parts
<flaccid> upload 72 seconds
<flaccid> good rule of thumb to upload image parts and manifest into a dedicated 'sub-bucket' ?
<flaccid> the bucket can get large with lots of images
<flaccid> e.g. i just did rightscale-rightimages/rightimage_debian_squeeze_i386_server_v5.4.6_20100628.1.manifest.xml
<flaccid> rightscale-rightimages will get very big with child objects
<flaccid> looks like it takes about an extra 3mins for startup with a bundled image
<flaccid> yeah hmm 6 mins to go from pending to booting
<flaccid> i guess its unpacking a 10GB
<erichammond> flaccid: There's a fairly low limit on buckets, so I put bundles into relatively few buckets.  Of course, I recommending using EBS boot instead and with EBS boot, you don't have to worry about buckets.
<flaccid> how does ebs boot work - do you boot directly from an ebs volume that is restored from an ebs snapshot ?
<erichammond> flaccid: Yes. http://aws.amazon.com/about-aws/whats-new/2009/12/03/amazon-ec2-instances-now-can-boot-from-amazon-ebs/
<flaccid> yeah so you just pay a bit more for the ebs usage right?
<erichammond> ye
<erichammond> s
<flaccid> so you make persistent by taking an ebs snapshot and restoring/booting from that?
<erichammond> The root EBS volume is persistent itself.  You can snapshot to improve reliability.
<erichammond> You can also register a snapshot as an AMI to use as the base for new instances.
<flaccid> yeah, the equiv of a rebundle to s3 right?
<erichammond> Depends on your use, but generally.
<flaccid> i assume the snapshot size is irrelevant to the size of your volume
<flaccid> which means the snapshot is only the size of the data in the fs
<erichammond> Just like normal EBS volume snapshots.  Depends on the data onthe disk and the blocks modified since the last snapshot.
<flaccid> yep the delta
<flaccid> i'm just thinking about the process when using a bootstrapped ami mounted to loopback..
<erichammond> You create the original AMI by copying the file system over to an EBS volume and creating a snapshot.
<flaccid> yep i'm just thinking of avoiding the copying
<flaccid> don't think its possible
<flaccid> rsync -avz
<erichammond> If you don't copy, then you have more modified blocks on the EBS volume including any deleted files.
<erichammond> It is possible, but I recommend the rsync.  No need to use -z
<flaccid> ok thanks
<flaccid> basically everything is strapped to a .img
<flaccid> then s3 or ebs are options after that..
<erichammond> You don't need to use loopback.  You can debootstrap to a subdirectory
<erichammond> assuming you're still building Debian.
<flaccid> yeah but my build process uses an image so you can also archive that
<flaccid> and optionally remount and check prior to bundling/regstration
<erichammond> ok
<flaccid> its handy for the possibilities of generic images between ec2 and eucalyptus
<erichammond> flaccid: Yep, that's what Canonical is doing.  I start my EBS boot AMIs by downloading their pre-built images.
<flaccid> sweet as
#ubuntu-ec2 2010-06-29
<boxbeatsy> hi guys, do you all know if there is an EC2 AMI that has mySQL set up for external access?
<flaccid> boxbeatsy: likely not, you could install it however in a running instance via a script and optionally re-bundle the instance into a new AMI. or you could just use rightscale
<boxbeatsy> flaccid: i see, i think i'll just install it myself.  might as well learn.
<boxbeatsy> thanks!
<flaccid> np
<flaccid> your distro will have the required documentation plus the mysql doc for altering the user to hosts
<boxbeatsy> gotcha, thanks much
<smoser> i think that subsequent spot-instance-requests by the same user are penalized
<smoser> to avoid usage of spot-instance-request instead of normal launch
<smoser> it seems that the first time I do one it comes back quickly.
<smoser> soon after if I do another i'll get a *long* delay.
<smoser> even when the price is well over the current rate
<smoser> anyone else seen that sort of thing ?

{'channel': 'ubuntu-ec2', 'month': 6}