Ask HN: How have you left Google's services? - <AUTHOR> Google seems to be more and more of a one-stop shop for me (and many others). There are privacy concerns, as well as concerns about data being available/under your control for the long haul.
What have you done to mitigate this risk? If you've left Google's services, what services have you replace, and how?
I rely heavily on Gmail, Search, Drive & Photos. The main issue I have is making my data highly available and resilient at low/no cost. <comment><USER2> [https://www.privacytools.io/](https://www.privacytools.io/) and [https://prism-break.org/en/](https://prism-break.org/en/) are good places to start. They list most (all?) of the major alternatives for the different categories of apps.

On a personal note, it's definitely possible. I've done it without much loss of convenience, which I think is the main drawback.

> The main issue I have is making my data highly available and resilient at > low/no cost.

I think you'll find that most alternative, privacy-conscious services will cost something, primarily because they're not able to profit off of your personal data like Google and Facebook are. For me, I'm definitely willing to pay for that value.

<comment><USER3> +1 If you are not paying, you are the product.

The insistence that everything has to be free is materially harmful to everyone who works in IT, software development, etc. You have Google on one hand and Richard Stallman oon the other.

For $40 you can get full fat service from Fastmail with your own domain name, which costs another $10 a month or so. That's pretty cheap.

<comment><USER4> If you're paying $10/month for a domain name, you're paying too much!

</comment></comment></comment><comment><USER5> Disclaimer: I haven't actually left Google's services but I have removed my reliance on them. I might not be giving you the answer you want.

Most of my strategy relies on a home server (full disk encrypted), a server I colocate nearby (also full disk encrypted) and backups.

Email: I use Gmail my own domain and regularly backup my email to my home server. In the even that something happens to my Google account, there are only issues for a few hours while DNS updates propagate.

Chat: After getting fed up with Facebook Messenger, I set up a hosted Prosody server on one of my dedicated servers and use it to communicate with my wife using the Conversations XMPP client for Android.

Photos: I auto sync photos from my phone to Google Photos and frequently (through Google Drive) pull them back down to my home server. My home server backs these up to Google Drive (unencrypted), Amazon Cloud Drive (encrypted) and my colocated server.

Drive: The home and colocated servers give me a ton of storage to do whatever I want with. I've never used a proper "sync" client however, so this isn't a good idea for everyone.

Search: DuckDuckGo is pretty good and I use it whenever I don't want to use Google for any reason.

Generally: I'm happy to use Google's services as:

\- They're convenient

\- Google operates in and is regulated in so many jurisdictions that I'm confident it's no less protective of my privacy/data than any other US based company

\- It's not going away any time soon

\- Tools like TakeOut make it incredibly easy to leave

<comment><USER6> How do you backup your email to your home server? Is it just an mbox dump via takeout on a cron schedule? How would you restore it in the case of leaving Google?

<comment><AUTHOR> [http://gmvault.org](http://gmvault.org)

Just used this to backup my inbox the other day. Compresses emails into .eml.gz files. Also has an export option for other mail clients. Love that its a Python package, Installable through pip.

<comment><USER6> Excellent! So you just put this into a cron job and it does incremental backups?

<comment><AUTHOR> I've only used it once. According to the docs, you should be able to do a "quick sync" which only gets the last 7 days.

Set up a cron job for that (after a full initial sync), and should be good to go.

</comment></comment></comment></comment></comment><comment><USER7> I'm using Fastmail instead of Gmail. Smartphone app is not really good but you can use any other smtp/imap client. For android you can synchronize your contacts with "CardDAV-Sync" (carddav is native for iOS).

Fastmail offer contacts and calendars too. Which have a really nice web interface.

I'm still looking for a Drive/Photos alternative. Any advice?

<comment><USER2> > I'm still looking for a Drive/Photos alternative. Any advice?

I haven't found any photo storage services that really sell the privacy- conscious angle. That said, I'd trust [https://www.smugmug.com/](https://www.smugmug.com/) more than Google, Facebook, or anyone else.

For a vanilla, non-photo-specific storage service, there's [https://spideroak.com/solutions/spideroak- one](https://spideroak.com/solutions/spideroak-one).

<comment><USER8> You should try hosting your own photos with GNU Mediagoblin.

<comment><USER2> Looks good as a self-hosted option.

</comment></comment></comment></comment><comment><USER9> Drive & Photos: Dropbox and WD Cloud[1] - similar privacy concerns, WD Cloud works great on local network and I can ssh in to access remotely

Search - DuckDuckGo, it's not quite as good, but it works fine IMO

Gmail - Still use regularly, I may switch to Fastmail

The goal (for me), was/is to distribute who has my data. This will blur the image of who I am, so to speak. My friends and I will also tag each others faces in photos on Facebook - i.e. I tag myself as my wife, my wife tags me as her. That way it'll screw with any automated process.

[1] [https://www.amazon.com/Diskless-Cloud-Network-Attached- Stora...](https://www.amazon.com/Diskless-Cloud-Network-Attached- Storage/dp/B01AWH05KK/ref=sr_1_4?ie=UTF8&qid=1471363383&sr=8-4&keywords=wd+cloud)

</comment><comment><USER10> I was a heavy Google user. My solution cost me some money.

-I bought a U1 rack used off craigslist and have it hosted in a local data center. -I installed ESXi on it.

made a few VMs -one runs pfsense and the rest of the VMs sit behind it -one runs my email, contacts, calendars (mailinabox.email) -one runs my cloud storage and private wiki -a few others running projects I'm working -each VM is fully encrypted at boot up.

<comment><AUTHOR> This sounds like the real deal. So you're no longer using anything Google?

What did this cost approximately - the initial setup and the monthly recurring?

<comment><USER10> I'm using a Dell PowerEdge 1950 with a Intel Xeon 5345. 32GB RAM and 1TB HDD. It cost me about $400 -- which I think was a good deal.

Then my only cost was the datacenter fees. Which runs about $50 per month -- includes 5 IP addresses and unl data.

Like I said though, I use it for multiple things, not just e-mail/contacts/calendar/storage.

EDIT: I believe the server costs a lot less now.

</comment></comment></comment><comment><USER11> I'm a big fan of SmugMug. I wish they'd give a little more attention to their mobile apps, but for $40/year I can store whatever I want on it, and I have a lot of control over how I share my photos.

The company is as susceptible to NSA-type stuff as anyone, but I like that they are a photo company, not a data company that happens to have a photo app.

<comment><USER2> I agree. They don't sell the privacy-conscious angle as a feature like some, but I'd certainly trust them more than Google or Facebook.

</comment></comment><comment><USER12> Well no cost is a bit of a misnomer. Usually, unless you host your own, when you don't pay in dollars you pay in privacy.

That being said, ownCloud is a great open source alternative to Drive (I host mine on digitalocean which is just starting to introduce block storage but amazon is an equally good option and very cost effective), Duck Duck Go is a very good web search engine these days with privacy as a core value.

It really depends how you use Photos. The automatic face detection and searching for people/events/things in your history is just about unparalleled and is unlikely to be surpassed by a more privacy friendly option in the near future. The reason for this is that these algorithms rely heavily on data collection, which is diametrically opposed to privacy. If you are only using it to backup/store photos in the cloud, ownCloud has similar functionality.

I have heard good things about fastmail to replace gmail but I have not used it myself.

I know you didn't mention it but for chat Matrix is a fantastic option with multiple options for open source clients. I use [https://vector.im/beta](https://vector.im/beta) as a client to access matrix.org but you could easily host your own matrix and/or vector server.

</comment><comment><USER13> Nine steps road to privacy and freedom from Google and everyone else:

Step 1: Switch to DuckDuckGo.

Step 2: Create a openmailbox.org account. (Gives you a mailbox with IMAP and POP3 access, but also an ownCloud account which replaces many of Google's services. All free software, no tracking, and it's reliable.)

Step 3: Switch to a good GNU/Linux distribution that cares about your freedom and privacy, like Fedora or Debian, and not Ubuntu.

Step 4: Use Mozilla Firefox if you are using another browser. All the other major browsers are proprietary, browsers that aren't among the major ones almost never support extensions.

Step 5: Install uBlock Origin and configure it to your liking.

Step 6: Install and configure the Evolution mail client with your openmailbox.org account.

Step 7: Use Pidgin for chat. It supports (with plugins) all the chat protocols, including the proprietary ones such as Skype, Facebook Chat, Hangouts.

Step 8: Use duplicity to have encrypted backups of your home directory made automatically to a portable hard drive or a USB drive.

Step 9: Use applications that run on your computer instead of web apps whenever you can.

You now control all of your data, have all the services you had before but running on your own computer and not on some third party's servers, use only free software that you can trust, and are much harder to track online. Congratulations!

<comment><USER14> > _Step 3: Switch to a good GNU /Linux distribution that cares about your > freedom and privacy, like Fedora or Debian, and not Ubuntu._

why not ubuntu?

<comment><USER15> This may help explain it: [https://www.youtube.com/watch?v=DXnfa0H30L4](https://www.youtube.com/watch?v=DXnfa0H30L4)

Which I believe had to do with it's configuration of Unity (Ubuntu's default DE)? (Can't watch the video ATM to confirm.)

Canonical (the people in charge of Ubuntu) have done things that the more zealous advocates of user-first freedoms/privacy are skeptical about or have caused distrust.

<comment><USER14> Ah I remember people complaining about Unity(which had Amazon search enabled by default)

Any recommendation for another flavor that's more user-freedom/privacy focused but still relatively stable?(I read somewhere Debian isn't good for personal use since it's designed for servers, is that true?)

<comment><USER13> Debian is stable and good for personal use. All the major distributions can usually be used on servers as well as for personal use. They are also all stable as far as bugs are concerned (even the riskier rolling release ones), though some distributions like Debian and CentOS use stability in a stronger sense of not pushing any updates to the stable releases except bug fixes and vulnerability fixes.

<comment><USER14> thanks!

</comment></comment></comment></comment></comment></comment><comment><USER16> The only Google services I use are its search and YouTube. I moved away from Gmail, though I have throwaway Gmail accounts.

Email: Private server and domain

Chat: Facebook because few people seem to use anything else, let alone answer their phones or emails. I otherwise hate it.

Photos: Keeping photos is overrated. I keep the few photos I take on a network drive at home and rarely share anything with anyone. If it burns in a fire, I wouldn't care very much. Yet still not worth being in the hands of some company.

Drive: See above. If I need to share something, it's either through Email or Box.

Search: Still mostly Google, but increasingly Bing as I feel like I get better results(I suspect that Google is underhandedly trying to make the web more "friendly"). Just type in some mean or evil phrase into both search engines.

Docs: I rarely have to write docs or spreadsheets. Google's online spreadsheet is pretty good, but for a "document", I end up writing stuff in Markdown using Dillinger.io.

Music: Most of the music and content I enjoy is free on YouTube.

Eventually, I'd like to leave Google all together. Changing search engines is easy, but YouTube and Maps are near impossible to beat at this point.

</comment><comment><USER17> I moved my email to FastMail recently and am very happy with it. I still use Google Search, Calendar (planning to move that to FastMail, too), and Photos.

My Gmail address is now a forward to FastMail, but I'm in the process of changing it everywhere. Trouble is that lots of people have it in their address book, so I'll never really stop getting emails to it.

Dropbox can replace Drive and Photos easily and quite well, I hear.

</comment><comment><USER18> I've switched from Google's search engine to a combination of Bing and DuckDuckGo (usually in browser incognito mode).

Google's results are increasingly full of low quality spam sites, so the switch really wasn't painful at all.

I haven't left GMail yet, but I started using my own domain mail many years ago. As such, that switch will be fantastically simple when I get around to it.

</comment><comment><USER19> I moved my email to Fastmail (I love it!)

OneDrive for photos (I was a heavy Windows Phone user for a long time, it was easy to set up and use)

My contacts are spread across Windows Live and Google (Again, because of the Windows Phone)

I use DuckDuckGo for search (With the "!g" modifier when I think the results could be better)

Dropbox for files without sensitive information, Tarsnap for files with sensitive information.

</comment><comment><USER20> Fastmail users: The feature of gmail that seems unstoppable to me is server side search. Using an imap client and doing client side searches with indexes just isn't viable anymore on multi-GB mailboxes. Both the web and app versions can "instantly" search all my mail, ever. How does Fastmail compare in regards to that?

<comment><USER2> Their web UI is extremely snappy. Definitely on par with Gmail's web UI, if not faster.

I don't have tons of messages in FastMail — so take that into consideration — but their web UI search seems very quick as well.

I haven't used their mobile app, so I can't speak to that.

</comment></comment><comment><USER21> I always use Zoho[0] for email on new websites I start that need a custom domain. It's the only web-based email provider I've found that is free for custom domains and modern looking.

[0] [https://www.zoho.com/](https://www.zoho.com/)

<comment><AUTHOR> I used them too, but then moved all my domains to Google Domains, which has free forwarding addresses - usually enough for most cases.

</comment></comment><comment><USER22> I'm totally in the Google world and the idea of getting out is somewhat appealing. However if I get out, I want to handle it myself. Fastmail seems wonderful, but where can I find a project that provides similar functionality that I can host myself? (And doesn't suck)

<comment><USER2> A few people have mentioned [http://dovecot.org/](http://dovecot.org/) combined with either [https://roundcube.net/](https://roundcube.net/) or [http://www.rainloop.net/](http://www.rainloop.net/).

</comment></comment><comment><USER23> Personally, I've a server at home running Linux. If you don't want to run your server at home, you can rent a server (Scaleway C1 costs ~3.5 euros / month for 50GB SSD, Online.net first servers are at 10 euros / month for 1TB HDD).

To replace Google Drive, I'm using Seafile, which I find is a better alternative than Owncloud (based on Python). Seafile has a mobile + desktop client and can be used over webdav. And the killer features: Seafile handles file versioning and encryption. [https://www.seafile.com/en/home/](https://www.seafile.com/en/home/)

To replace Gmail, I've installed Dovecot + Postfix on my server. I'm using Roundcube as my webmail. There is also Rainloop which is quite popular. [https://roundcube.net/](https://roundcube.net/)

To replace Google Calendars / Contact, I'm using Radicale. [http://radicale.org/](http://radicale.org/) . I've found some caldav/carddav connector on the play store.

To centralize everything in one place, I'm planning a migration to Open Xchange, an open source java software which handle your emails, your calendars (with caldav support), your contacts (with carddav support), your files (no more versioning or encryption however but support webdav) and you can even edit your .docx and .xlsx in place. You have a mobile application. But it lacks some documentation and some features are not open source, like IM or the desktop client. [https://www.open-xchange.com/](https://www.open-xchange.com/)

To have a single account for every services, I've installed a LDAP server (openldap).

To conclude, I try to use as far as possible open and standard protocols (webdav, carddav, caldav, smtp, imap, ldap, etc.) as there is always a software or a library to handle them.

And to replace Google Search, I'm using Qwant, or at least trying to.

<comment><USER24> > To centralize everything in one place, I'm planning a migration to Open > Xchange

Don't know if you know about SOGo, but my e-mail provider uses it and it works quite nicely, it might be another solution to consider.

[https://sogo.nu/](https://sogo.nu/)

<comment><USER23> Thanks, the version 3 looks really promising !

I'm testing Open Xchange for a few weeks now, and I really like it. There are lot of small cool features: remind an email in x minute, meeting organization, OX Guard (a PGP module), the integration with Sieve, etc.

And that's not a secret, such solutions are not easy to deploy. I've spend a certain amount of time writing my ansible script for Open Xchange - that's why I'm not really motivated to try it on my server now.

If I find something in OX which prevents me from switching, SOGO will be the next candidate.

</comment></comment></comment><comment><USER25> I have left all google services apart from search.

email: I run my own server. After the initial "reputation building" for my domain, it now works reliably well.

calendar/files/photos/contacts: nextcloud on my home server. It's not really on par with the google offerings, but it works well enough for me. And when I need more storage I just get a 2TB usb disk off amazon.

play store: f-droid. It has everything I need. I understand that not everyone might be happy with just the apps on f-droid, but I was bored with downloading crappy games and uninstalling them after 2 minutes anyway. f-droid apps are all well done, and I trust them not to invade my privacy.

</comment><comment><USER26> Take a look on [https://www.privacytools.io/](https://www.privacytools.io/) , which lists alternatives in various categories. It doesn't include photos from your list though.

</comment><comment><USER27> Google gives you the option to download all your data stored on their servers, across all Google products [1]. Unfortunately they do not give you an option to _delete_ all your data, but most services have facilities for bulk-deleting [2].

Every few months, I download a backup and then delete everything I can. This is a good tradeoff for me: I get to use all the great Google products, but they don't get to keep a permanent archive of all my data.

One more note: lots of people react to this with, "But what if you desperately need an email from X years/months ago?". Two answers: 1) I've been doing this for a couple years, and I've never needed an old email, and 2) I have the local backup if it's something really important.

[1] [https://takeout.google.com/settings/takeout](https://takeout.google.com/settings/takeout)

[2] [http://www.makeuseof.com/tag/clear-data-google-attempt- regai...](http://www.makeuseof.com/tag/clear-data-google-attempt-regain- privacy/)

EDIT: Formatting

<comment><USER2> This is an interesting/creative idea. I've heard about people doing something similar with Twitter and Facebook. While they want you to keep everything forever, it's actually pretty easy to use them more ephemerally.

</comment></comment><comment><USER28> I've recently been looking at Fastmail and Kolab ([https://kolabnow.com/](https://kolabnow.com/)) to replace Gmail and Google Calendar. Kolab adds limited notebook and file storage functionality, but I haven't found any decent reviews of the service.

Is anyone using Kolab who'd like to comment on how it's working for them?

</comment><comment><USER29> I've left Google mostly for usability reasons. There's no reason to trust any 3rd party solution more than another one, so until I'll have my own hardware in some secure data center in EU (I do not trust US hosting and networks now), I'll choose what's more comfortable and reasonably secure.

Gmail -> moved my custom domains to Yandex Mail (personal), Outlook.com (corporate), because usability of Gmail to me becomes worse and worse.

Storage + documents (never used Google services) -> OneDrive + MS Office 365 paid subscription

Music -> VK.com (finally they are making it legal by signing agreements with labels)

Social Networks -> VK, Facebook (because friends are there)

Messaging -> Facebook, Telegram (because friends are there)

I'm still using Google search, but not 100% of time. Bing is quite good too.

</comment><comment><USER30> Search: [https://www.startpage.com](https://www.startpage.com) (free)

Mail: [https://www.posteo.de](https://www.posteo.de) (12 EUR/yr)

Contacts: posteo does CardDAV (included)

Calendar: posteo does CalDAV (included)

Chat: Signal (though not decentralised, the only service that seems robust, trustworthy and usable by the not-so-technical (=friends)); (free)

Drive: Dropbox w/ partly encrypted content (currently GPG, but should move to cryptomator or similar for convenience/scaling; Spideroak is not open-source despite promising to become that for years now, don't see why to trust them); (free plan)

Youtube: :( - impossible to replace, i guess

Maps: [https://www.openstreetmap.org](https://www.openstreetmap.org) on occasion, mostly still G. Maps :(

also:

Desktop OS: Ubuntu (I don't see significant privacy concerns currently; also, this is the only OS i would recommend to either browse-and-mail users or interested and motivated friends)

Mobile: Still on OEM-Android; would switch to CyanogenMod if i could root safely, see below (in future: CopperheadOS); using it with most G. Apps disabled, NetGuard blocks all but G. Services (required for Play & Signal)

Browser: Firefox w/ uBlock, NoScript, Request Policy, HTTPS Everywhere, Decentraleyes (hasn't been mentioned here?!), Random Agend Spoofer (disable in reverse order if this seems excessive/redundant)

looking for:

Skype: good SIP client and especially server - anyone? :) (also non-SIP alternatives...)

Mobile hw: good hardware (Nexus seems paradoxically logical to either get serious updates or compatibility to projects like Copperhead/Cyanogen; maybe also S. Galaxy?)

Mobile sw: open-source scripts to root above "good hardware"; open-source su apps (phhusson's seems to be the only one outside CyanogenMod? and still in rough beta)

</comment><comment><USER31> One thing that really helped me stop relying on Google for search was when I realised that I could precede any search on DuckDuckGo with `!g` if I didn't find what I wanted on DDG and it would switch to encrypted.google.com

"!g my search term"

<comment><USER32> Prefixing with _!sp_ will redirect your query to Startpage, which is essentially an anonymous proxy for Google.

So Google will still get to log and analyse your search terms, but they will be aggregated with all other users of Startpage - lots of noise.

<comment><USER31> good stuff, thank you

</comment></comment></comment><comment><USER33> To me, there are several classes of privacy concerns: those specific to a particular company, those specific to third party data storage, those specific to the internet as a whole. A good solution for an actual individual person will be based on the weight assigned to each area and the convenience tradeoffs. I mean Richard Stallman's approach works for him it seems, but I'm not ready to unplug from the internet to that degree.

1\. Concerns over the company: Google, Microsoft and Apple are largely fungible in terms of services and variation among their policies and other business practices may make one better than another for a particular person. For _me_ , I place more trust on Microsoft because I give more weight to the alignment between a company's business model and _my_ interests and believe Microsoft provides the best alignment with _my_ interests among the big three. Someone else could easily and reasonably come to a different conclusion.

2\. The cloud in general: I like having important data on a 'disk' I physically control. I'm all for the cloud as backup, but I don't want to be locked in. I suppose I'm willing to trade a little continuous pain for reducing the risk of great pain all at once. Anyway, the cloud as primary storage often seems just like more work which is just like the work of managing local resources again.

3\. The internet in general: Most of what I do is pretend I am beating big data. But I silo logins: HN, Facebook, Linkedin, Gmail, all have their own browser instance to reduce information leakage. The result is not so much strict privacy as preventing my information from being the low hanging fruit. Raising the cost of cross referencing my activity seems to reduce 'personalization' and a bit more 'anonymity' when surfing the web is the sort thing I am interested in.

4\. As my data has accumulated, I've been less concerned over keeping it...I really don't enjoy filing and record keeping and moving data from one store to another. So I'm much more willing to delete old emails and documents...and new emails and documents. In the past, I've had a tendency to over-estimate the value of saving things.

Good luck.

</comment><comment><USER34> I use Dropbox instead of Drive and Photos.

Gmail is hard to replace because it's well integrated with other Google services, in particular Google Apps with its own ecosystem.

Google Search is hard to replace because of the quality. Bing is ok but not close in quality.

<comment><USER2> > I use Dropbox instead of Drive and Photos.

For the privacy-conscious, Dropbox is saddled with the same pitfalls as Google and Facebook, unfortunately.

> Google Search is hard to replace because of the quality. Bing is ok but not > close in quality.

[https://disconnect.me/search](https://disconnect.me/search) used to show Google search results, but it looks like Google has blocked it for the time being.

I personally use [https://duckduckgo.com/](https://duckduckgo.com/). The quality of results is definitely below Google, but I believe that'll only improve as more people switch to them.

<comment><USER7> I agree that Dropbox is nearly same than Google for privacy but it can be a first step to split your data between multiple companies instead of giving all your data to Google.

I know it's totally not perfect, but is it logical?

<comment><USER2> Yeah, I think that's a fair point, and can certainly be a logical step toward increased privacy.

</comment></comment></comment></comment><comment><USER35> I have deleted most of my gmails, now I only have one because is the main email where I have a lot of accounts signed up that I don't remember, when I get an email from those services I change my email immediately.

Now I use [http://posteo.de](http://posteo.de) as my main email, any new contact or new service I use this email. The only con is that I can't use my own domain on this, but I don't really care.

The only thing that keeps me from closing all my google accouns is google maps, for a person that sucks memorizing street names and locations this is very useful for me.

</comment><comment><USER36> I went through the Google service manager:

Bookmarks -> Firefox sync plus manual downloads back ups Photos -> For online albums, Microsoft OneDrive Reader -> Digg Reader Drive -> Dropbox and OneDrive YouTube -> OneDrive and Vimeo

I'm still using Calendar, Gmail, Maps. I could migrate Gmail to Outlook, but I haven't committed to that change. Not sure how I can replace Calendar and Maps.

</comment><comment><USER37> Gmail: My own domain/mailserver, using dovecot and postfix.

photos: Piwigo

search: ddg

music: mpd and ampache

drive: I use svn/git repos that are hosted on my server

</comment><comment><USER38> My Gmail accounts became useless, and everything else killed gtalk (at least for me). Only thing I still use a lot is search, which often lead me to two places: stackoverflow or Wikipedia. I guess it won't take much time until I stop google.

</comment><comment><USER39> I have pretty much moved all my services to cloudron.io. It's not free but you can pick and choose whatever apps you want. They are quiet responsive about packaging any new apps.

</comment><comment><USER40> For email, Yandex Mail is excellent. For search, DuckDuckGo is okay.

<comment><USER41> I'm not sure I like the idea of my email being hosted in Russia.

<comment><USER42> You are protected somewhat by their extra level of ineptitude and reluctance to collaborate with North American / European authorities.

</comment></comment></comment><comment><USER43> I'm locked out of gmail, so i don't use it anymore. It's what happens when they can't track you. "suspicious activity" they call it.

</comment><comment><USER44> Switched everything over to Microsoft, because Google is evil now and Microsoft isn't. Will switch back in a few years when they trade places again.

</comment><comment><USER45> This is probably the 5th posting I've seen in a month or so. I struggle with this problem as well. Looking forward to hearing peoples experiences.

</comment><comment><USER46> People talk about DuckDuckGo as great ... but it returns the same garbage results as Google.

I want results for a search where I can exclude those with a number of cookies/js links that would have been blocked by uBlock (or similar) above some threshold.

</comment><comment><USER47> I use StartPage for search. Openmailbox for email.</comment>


Employment contracts: What are they good for? - <AUTHOR> http://37signals.com/svn/posts/2239-employment-contracts-what-are-they-good-for

<comment><USER2> Some advice I once got about contracts:

If you have to go back to the contract and start telling the other person "The contract says you would do X", then the relationship is already screwed up and it is almost beyond repair.

<comment><USER3> That's a _little_ extreme.

Contracts are good for memorializing simple numeric facts like "you get X shares vested in this way on thus and so date". Much better than depending on the memories of even scrumptiously honest people.

The assignment of IP part is also important, at least when I learned this in the '80s. Makes it crystal clear that the employee is doing work for hire and has no copyright interest in his code or whatever.

As noted elsewhere, this is easy to put in one page and you should strive to do that. "Just the facts, Mam".

</comment></comment><comment><USER4> Most Valley startups use a one-page "letter offer" type of agreement, which works well and which need not set a wrong tone of "we don't trust each other."

In addition, there is always an "inventions and confidentiality" type of agreement that: (1) clarifies the employee's confidentiality obligations; and (2) makes clear that all IP generated by the employee belongs to the company.

In purely technical terms, under California law, you could actually skip both these documents because the default relationship is "at will" and normally can easily be terminated by either party if it does not work out and because the law automatically says that inventions conceived by employees in the course of performing their duties belong to the company and not to the employee.

Why not keep it simple, then?

Of course, any company can choose to do so.

But:

1\. If compensation is not in writing, you can get claims of the "I took reduced pay because I was promised x percent of the company" variety.

2\. If an employer does not take prudent steps to show that it is keeping its trade-secret information confidential, the law can find that it is not really protectable trade-secret information after all, thereby potentially jeopardizing the company's right to protect it from theft by any employee.

3\. Usually the employee is given an option grant and any promise relating to equity should always be hedged by statements like "this is subject to approval of our Board of Directors" and "terms of the grant will be governed by our option plan and standard option agreement" - and followed up by a signing of the formal stock option agreement (among other things, if you don't have such documentation, your ISO classification of incentive options is jeopardized and the employee will miss out on whatever tax benefits such a classification provides).

4\. People do make things up when disagreements arise and it is very prudent to have at least a basic document that specifies exactly what the relationship was in order to avoid problems (perhaps this could be solved by the globally posted employment policies that the articles alludes to).

5\. Finally, some forms of compensation are incentive-based, e.g., commissions, and it is a serious mistake to agree to such arrangements verbally without clarifying such basic issues as "to what does the commission apply," "is there a tail on it if employment terminates," etc.

The above are basically random thoughts but they do suggest that you do not want to scrap routine employment documents unless there is a pretty compelling reason to do so, and I don't think the issue of wanting to avoid an atmosphere of mistrust is by itself necessarily compelling. Indeed, most employees that I know routinely accept the idea that basic documentation and "legalese" is in order and do not interpret this as some sign of mistrust.

I don't think this is an issue of letting lawyers run a company's life. It is just basic prudence and common sense.

<comment><USER5> > the [California] law automatically says that inventions conceived by > employees in the course of performing their duties belong to the company and > not to the employee.

It's been awhile since I researched this, but the law used to be that, absent an invention-assignment agreement, the employee had to be either "hired to invent" or "set to experimenting" for the employer to be the automatic owner.

I had to litigate the latter point one time for a large telecom company. A non-technical employee had invented an improvement in some specialized equipment that the company used, but he hadn't been hired to invent and didn't have an invention-assignment agreement. The company filed a patent application. The in-house patent lawyer went to see the employee to get him to sign a patent assignment - on the sidewalk he ran into the employee, who had just been laid off. The employee sued the company for misappropriating his invention. The company won a summary judgment that the employee had been 'set to experimenting,' but it would have been way simpler and cheaper if the guy had just signed an invention-assignment agreement.

<comment><USER4> Oops, I had oversimplified things in making my statement as I did (I deal with these cases where people routinely are "hired to invent" and so took that for granted in making the statement). Thanks for the clarification.

Of course, this is one more reason (as you point out) why employers should not treat this issue too loosely.</comment></comment></comment>


Security guard robot ends it all by throwing itself into a watery grave - <AUTHOR> https://arstechnica.co.uk/gadgets/2017/07/knightscope-k5-security-bot-drowned/ <comment><USER2> I wonder if insurance of the future will cover robot "suicides"</comment>


'Generous' approach to replication confirms many social science findings - <AUTHOR> http://www.sciencemag.org/news/2018/08/generous-approach-replication-confirms-many-high-profile-social-science-findings <comment><USER2> >And the 62% figure "certainly is consistent with there being a problem" in the field, he says. "It seems funny that there's been a drift in standards to the point where 62% seems very respectable."

I'd sure say so. Statistics in science is just so broken. Lots of inappropriate techniques chosen on tradition, ignorance (or concealment) of multiple comparisons, the base-rate fallacy, etc, etc. It's not just social sciences, but medicine, biology, and just about every field that relies greatly on lower-power studies.

Bayesian techniques are really called for in those situations.

<comment><USER3> I'm not a scientist, nor am I familiar with statistics. I could imagine this being troubling for a scientist or someone familiar with the field. But for a non-scientist it's fairly devastating if you consider the implications.

What it means is that if I trust a study - _any_ study (from my perspective) - I'm essentially flipping a coin. If non-scientist citizens can't rely on what is coming out of the field, then it seems like a massive problem that needs solving before any other.

<comment><USER4> You _shouldn 't_ trust any single study. Researchers are not perfect, reviewers are not perfect, the systems they observe are complex. Even in the "hard" sciences even if the experimental data itself is solid there can be confounders that were overlooked and will only be uncovered by follow-up studies, depending on importance of the result it might take years.

That's why have meta-analysis and systematic reviews and even those aren't exactly bullet-proof.

<comment><USER5> > Even in the "hard" sciences even if the experimental data itself is solid > there can be confounders that were overlooked and will only be uncovered by > follow-up studies, depending on importance of the result it might take > years.

I work in fluid dynamics. In one subfield I work in there is at least one confounder in most studies (Weber number and Reynolds number are confounded, in particular) and an important variable is often omitted (turbulence intensity). Many people don't seem aware that turbulence intensity matters, either, and most who do think it matters don't want to measure or estimate it, so it tends to be ignored. Just because it's hard to measure does not mean it's not important! This is supposed to be "hard" science, but sometimes I feel it's not much better than social science.

How long has this been a problem? I'd say over 50 years, easily!

At a recent conference, one of my papers addressed these issues briefly (I tried to avoid them in a data analysis), but I think it'll take more than that to correct these problems. So I'm planning a new article specifically addressing these concerns. I still expect progress to be slow, but any movement in the right direction would be appreciated...

<comment><USER3> That's really interesting. Seems like good science is always going to be an uphill battle against human nature/complacency.

</comment></comment></comment></comment></comment><comment><USER6> > If experts [in prediction markets] can instinctively spot an irreproducible > finding, "that kind of begs the question of why that doesn't seem to be > happening in peer review," says Fiona Fidler, a philosopher of science at > The University of Melbourne in Australia. But if future studies can identify > and weigh the best predictors of replicability, reviewers might be given a > rubric to help them weed out problematic work before it's published.

That's a troubling suggestion. Results are valuable to the extent that they're both accurate and surprising. To systematically suppress surprising results as a negative predictor of accuracy sounds like a formula for suppressing surprisingly valuable papers.

<comment><USER7> The danger with surprising results is that they often use the surprise part as an excuse for small sample sizes, etc.. Another way to think about this situation is to flip it and say that surprising (and spurious) results are more likely from exploring a small dataset.

Andrew Gelman talks a lot about this issue on his blog:

[https://andrewgelman.com/2014/08/01/scientific-surprise- two-...](https://andrewgelman.com/2014/08/01/scientific-surprise-two-step/)

</comment></comment><comment><USER8> > If an initial replication attempt failed, the researchers added even more > participants.

This is an obvious way to tamper with the results; it's just more of the same kind of p-hacking that bad researchers are so often doing. They are using "we re-do the study with a larger population" as a way to re-roll the dice if the first die roll doesn't come up the way they want. (Note that if the die roll _did_ come up the way they want, they don't re-do the study with a larger population in order to see if the replication fails).

Nobody should be taking this seriously.

<comment><USER9> There are known methods to account for this, and if you adjust the statistics properly to account for the repeat, it isn't p-hacking.

<comment><USER8> Did they? I can't tell from the article!

</comment></comment></comment><comment><USER10> The prediction-market aspect is fascinating; they asked a group of experts to predict which would and wouldn't replicate, and while we don't see the experiment-level results, the overall average was apparently very close.

Is this common in modern replication studies? Do the results of the prediction-market ever get announced/published prior to the replication results themselves?

</comment><comment><USER11> > experts also participated in an online "prediction market," trading shares > that corresponded to studies, which paid out only if the given study was > replicated.

> Both approaches did well at predicting the outcome for individual studies, > and they predicted an overall replication rate very close to the actual > figure of 62%.

I think the real headline here is that we should supplement peer review with replication prediction markets.

<comment><USER12> Is there any information on these prediction markets? Is there a reason only "experts" can participate? This would be an interesting application for ML side projects that are monetarily beneficial while also benefiting society as a whole.

<comment><USER13> The experts aren't necessarily credentialed experts as we would typically think of them. You can participate in forecasting tournaments like Good Judgement Project and if you score high enough you can participate in invite- only events that are in partnership with entities like DARPA. But the people come from all backgrounds- I got all this from the book Superforecasting by Philip Tetlock.

Edit: got the title wrong

</comment></comment></comment><comment><USER14> The facts are spread throughout the article, so this might help:

* Prior replication studies replicated 39% of papers in psychology journals [Ed note: That doesn't mean the other 61% were complete failures; most just didn't produce results as statistically strong as the originals IIRC] and 61% in economics journals.

* This replication study greatly increased the number of participants in some experiments, and for two papers, that changed the replication results from failure to success. Overall 62% of the studies were replicated successfully.

* One researcher "points out that the project repeated only one experiment from each paper, and in his case, it wasn't the strongest or the most important."

<comment><USER15> You're missing arguably the single most important sentence in this article:

* If an initial replication attempt failed, the researchers added even more participants.

<comment><USER16> Which is not unlike doubling-up at blackjack, which will get you thrown out of a casino.

Start with 25 participants, then if you don't get the result you want, add 50 and try again. Carry on doubling until you get the result you want.

Not a statistician here, but as each addition is essentially a new trial, shouldn't they have applied Bonferroni correction to the results?

</comment></comment></comment><comment><USER14> I assume Science knows what it's talking about, so I'm confused on a couple of points. Perhaps some practicing scientists could clear them up for everyone:

For any arbitrary paper, what is your assumption about its accuracy? How much do you rely on it? Can you put a number to it? The null hypothesis that research papers, especially in very difficult fields like social science, are near-infallible seems to be the error, AFAICT. It seems like something people outside science would assume, but I see scientists who are surprised by a 62% replication rate. I'm not surprised or concerned, but maybe I should be.

Why do they call increasing the number of participants, "generous"? Doesn't that increase accuracy, and isn't accuracy the whole point of the replication study? Generosity implies some sort of favor, something above and beyond, while in this case it seems necessary based on the fact that the results changed - it would be failure of the replication study to not increase the number of participants.

<comment><USER17> "For any arbitrary paper, what is your assumption about its accuracy? How much do you rely on it? Can you put a number to it? The null hypothesis that research papers, especially in very difficult fields like social science, are near-infallible seems to be the error, AFAICT. It seems like something people outside science would assume, but I see scientists who are surprised by a 62% replication rate. I'm not surprised or concerned, but maybe I should be."

There is no such thing as "an arbitrary paper". It will depend very much on the study in question - including soft factors like who wrote it, but also the study design, sample size, if I think they approached the statistics appropriately, etc.

"Why do they call increasing the number of participants, "generous"? Doesn't that increase accuracy, and isn't accuracy the whole point of the replication study? Generosity implies some sort of favor, something above and beyond, while in this case it seems necessary based on the fact that the results changed - it would be failure of the replication study to not increase the number of participants."

The reason it might be thought of as generous is the criteria they determine for saying something is replicated is a significant effect in the same direction as the original study (for the record, I _hate_ this criterion). A larger study is more likely to find a significant result if indeed there is one there, so they're giving studies that report an effect a very strong chance of seeing that replicated if indeed it was real.

</comment></comment><comment><USER18> I would appreciate a reader-friendly list of the studies that were successfully replicated, being it the 39% of the 2015 paper or the 62% of this paper.

Knowing that I have no strong reason to trust most of the conclusions is useful. But know which papers I _can_ trust is more useful.

Anyone knows of such a list?</comment>


A Google+ loophole used for a creative campaign
 - <AUTHOR> http://thenextweb.com/industry/2011/07/05/israeli-firm-launches-the-first-ever-creative-campaign-on-google/

<comment><USER2> Nitpick: I'd be even more impressed if they ran their campaign through a spell checker.

</comment><comment><USER3> An invitation probably isn't enough. I've already got two and still cannot sign up.

<comment><USER4> I'm in the same boat.

I actually got my invite from someone who _works_ at Google, but I keep getting the "We've temporarily exceeded our capacity. Please try again soon." message when I try to join.

My friend says keep trying, but I might just wait until they open up to more people.

</comment></comment><comment><USER5> I don't know if this tweet is true, since I'm already on G+. The rest of you can try and report back:

<http://twitter.com/#!/rothgar/statuses/87767198084505600>

<comment><USER6> It's not true.

<comment><USER5> Thanks for trying. That's too bad. That place needs people.

</comment></comment></comment><comment><USER7> So they exposed all of their contacts on Google+? I wonder how the contacts felt.

<comment><USER8> They're all marketing contacts, they're probably less annoyed than average.

</comment></comment><comment><USER9> I don't think its a loophole, It must be intentional.

Second way which is popular now, you can add the gmail ids in your circle and share a comment to the circle. This will send a invite to everyone in the circle :)

<comment><USER5> I've tried that too. No one has reported getting any Invite or email.

<comment><USER10> On your stream page just type a message in the share something box "Hi." for example. Then in the sharing selection box put in their email address. They'll get a message with a link to see what you shared with them and for whatever reason that seems to have a much higher success rate though it can still take several attempts at clicking the link in the email before they get the sign up instead of the we're full message.

Also, if you are sharing with a circle you have to make sure you check the little box during circle selection that says "Also email X people not yet using Google+" or they won't get the email.

About 90% of the people that I've sent invites to are signed up now.

</comment></comment></comment><comment><USER11> You don't need the invitation via email to get into Google+, just tell your friend to invite you (tagging or posting, whatever). Go to <http://plus.google.com>. Log out of your Google Account. Log in back. Voila!

More information: [https://plus.google.com/107083942933833320656/posts/2fLmrDmy...](https://plus.google.com/107083942933833320656/posts/2fLmrDmyAKL)

Still working, up until now...

<comment><USER12> So, that totally looked like it was going to work.

But after I "logged in" to google plus it just redirected me to the same old _"We've temporarily exceeded our capacity. Please try again soon."_ page.

<comment><USER13> yeah didn't work for me either, but i agree, it did seem to look like it was going to work. i was excited for a second.

</comment></comment></comment><comment><USER14> This was how I got invited (out of the blue invite advertising some site) and how I invited over 20 people over a 24 hour period (tagging them in a photo/status update or sharing a status update to a circle/email address). They didn't get any "Exceeded Capacity" messages although the real invite UI still doesn't work. And it was after I heard on HN/Twitter that "the loophole has been closed". Maybe some cliques are more welcome than others.

</comment><comment><USER5> Have they since closed that loophole? I can't see a way to use it.

<comment><USER15> Ya I'd like to know too...anyone tested?

<comment><USER5> I figured out the loophole and tried it. No one has gotten any Invites. Damn. That places needs people.

</comment></comment></comment><comment><USER16> Well you COULD just share a link to the people wo want to join: <https://plus.google.com/up/start/>

<comment><USER16> Don't forget that you need an Google Account to join.

<comment><USER5> Every person I've tried to Invite has GMail. Not getting Invites. Grrrr.

</comment></comment></comment><comment><USER17> (How is TNW getting all these Circle stories?)</comment>


AWS CloudFormation: BeanStalk with custom stacks - <AUTHOR> http://aws.amazon.com/cloudformation/ <comment><USER2> This is cool and all, but its part of an overall trend about aws/ec2 thats starting to get on my nerves. They're spending their time and resources working their way up the stack instead of improving the lower layers of the stack for which you use them in the first place. Right now ~2+ms inter-node latencies, arbitrary 200+ms hangups on EBS, and the general lack of an SSD option make AWS a non-starter for things that need to scale without being architecture-astronaut computer science projects. For many workloads I _have_ to colo instead of use AWS because of the age old "spend 10K on hardware or hire an FTE developer" tradeoff.

<comment><USER3> Designing for fault tolerance and scaling out instead of individual node durability and scaling up is, in most cases, a sounder approach. Since the vast majority of applications map well to this case, it makes sense for Amazon to focus on features that assume/assist scale-out.

It's easier to design for scale-out than one might think; a great resource to get started is <http://highscalability.com>.

Regarding EBS, I haven't seen the hangup issue you've described. Any data?

As for low inter-node latencies, Amazon has an offering that specifically addresses that need: <http://aws.amazon.com/ec2/hpc-applications/>

Overall, I think Amazon is making a lot of inroads in areas with specific hardware demands. They just launched GPU compute options, and I expect we'll even see SSDs soon.

<comment><USER2> Scaling out vs. scaling up is the false dilemma, the answer is both at the same time solving the bottlenecks you have using the time and resources you have. The "vast majority" of applications do not map well to an n-wide map- reduce-esque design, the "vast majority" of applications bottleneck on an authoritative datastore's I/O capacity.

I'm the sysadmin (not the dba) for a website that does 30K http req/s at the edge, and our mysql cluster does about 3000 req/s during mid-day with most queries in the 1 - 5ms range.

Highscalability is a _terrible_ place to get started, its a great place to share notes but if you get started there you will waste insane amounts of time on architecture astronaut nonsense.

<comment><USER4> What do you consider architecture astronaut nonsense?

</comment></comment></comment></comment><comment><USER5> The AWS team never ceases to amaze me. Seriously. They release new, great, useful services at an amazing rate. It's difficult to believe that other cloud services will be compete over time simply because they cannot innovate as fast (I know others will survive and that's a healthy thing, but if I were making a bet today, I'd bet on AWS in the blink of an eye).

<comment><USER6> Same here. They just keep on going and going and going.

</comment></comment><comment><USER7> They even embedded a JSON-based programming language! From the example at [https://s3.amazonaws.com/cloudformation- templates/CloudForma...](https://s3.amazonaws.com/cloudformation- templates/CloudFormationSample_WordPress.template):

"UserData" : { "Fn::Base64" : { "Fn::Join" : [ "\n", [ { "Fn::Join" : [ "=", [ "dbName", { "Ref" : "WordPressDBName" } ] ] }, { "Fn::Join" : [ "=", [ "dbUser", { "Ref" : "WordPressUser" } ] ] }, { "Fn::Join" : [ "=", [ "dbPassword", { "Ref" : "WordPressPwd" } ] ] }, { "Fn::Join" : [ "=", [ "dbHost", { "Fn::GetAtt" : [ "WordPressDB" , "Endpoint.Address" ] } ] ] }, { "Fn::Join" : [ "=", [ "dnsName", { "Fn::GetAtt" : [ "WordPressELB" , "DNSName" ] } ] ] } ] ] } }

Certainly not the nicest syntax, but I guess it gets the job done.

<comment><USER8> Ugh. The first thing I'd do is write a preprocessor so I never have to see anything like that. Why not

       UserData: Base64("dbName=$WordPressDBName\ndbUser=$WordPressUser\n...")

I'll give Amazon the benefit of the doubt and assume they just didn't want to impose a syntax, not that they think humans should be writing JSON like that.

</comment></comment><comment><USER9> Try to understand this monster, taken from:

[https://s3.amazonaws.com/cloudformation- templates/CloudForma...](https://s3.amazonaws.com/cloudformation- templates/CloudFormationSample_WordPress.template)

Amazon guys trying to reinvent JS in JSON:

        "WordPressLaunchConfig" : {
          "Type" : "AWS::AutoScaling::LaunchConfiguration",
          "Properties" : {
            "ImageId" : "ami-16668c7f",
            "SecurityGroups" : [ { "Ref" : "WordPressEC2Security" } ],
            "UserData" : { "Fn::Base64" : { "Fn::Join" : [ "\n", [ { "Fn::Join" : [ "=", [ "dbName", { "Ref" : "WordPressDBName" } ] ] }, { "Fn::Join" : [ "=", [ "dbUser", { "Ref" : "WordPressUser" } ] ] }, { "Fn::Join" : [ "=", [ "dbPassword", { "Ref" : "WordPressPwd" } ] ] }, { "Fn::Join" : [ "=", [ "dbHost", { "Fn::GetAtt" : [ "WordPressDB" , "Endpoint.Address" ] } ] ] }, { "Fn::Join" : [ "=", [ "dnsName", { "Fn::GetAtt" : [ "WordPressELB" , "DNSName" ] } ] ] } ] ] } },
            "InstanceType" : { "Ref" : "InstanceType" }
          }
        },

After some identation cleanup:

      { "Fn::Base64" :
            { "Fn::Join" :
                [ "\n", [
                    { "Fn::Join" : [ "=", [ "dbName", { "Ref" : "WordPressDBName" } ] ] },
                    { "Fn::Join" : [ "=", [ "dbUser", { "Ref" : "WordPressUser" } ] ] },
                    { "Fn::Join" : [ "=", [ "dbPassword", { "Ref" : "WordPressPwd" } ] ] },
                    { "Fn::Join" : [ "=", [ "dbHost", { "Fn::GetAtt" : [ "WordPressDB" , "Endpoint.Address" ] } ] ] },
                    { "Fn::Join" : [ "=", [ "dnsName", { "Fn::GetAtt" : [ "WordPressELB" , "DNSName" ] } ] ] }
                ] ]
            }
        }

DSL like this would be much more readable and maintainable:

        base64(<<<USERDATA
        dbName=$WordPressDBName
        dbUser=$WordPressUser
        dbPassword=$WordPressPwd
        dbHost=${GetAtt("WordPressDB","Endpoint.Address")}
        dnsName=${GetAtt("WordPressELB","DNSName")}
        USERDATA)

And this just a small example, I wander how complex this JSON will be for real-life cases. Instead of reinventing Javascrict in JSON, they should have added some simple shell-like parameter substitution DSL (simple sigils) or allow some sand-boxed sanitized Javascript.

Now I understand why tools like Chef/OpsCode using Ruby for this.

</comment><comment><USER10> Surely we're better off using and developing Fog (and similar tools) so that we're not locked into the Amazon infrastructure?

</comment><comment><AUTHOR> Jeff Barr's post: [http://aws.typepad.com/aws/2011/02/cloudformation-create- you...](http://aws.typepad.com/aws/2011/02/cloudformation-create-your-aws- stack-from-a-recipe.html)

</comment><comment><USER11> So it was good that I didn't spend time learning chef/puppet or going with RightScale till now?

<comment><USER3> Nope, CloudFormation is for infrastructure provisioning. Configuration management is still valuable post-boot.

I've been using CloudFormation in conjunction with Chef - they're complementary tools, really.

<comment><USER11> Can you share some example of how you are using them together. I am currently using Python scripts with boto to bring up the instances and doing all kinds of configuration stuff on them. Just wanted to see if there is any better way of doing this.

<comment><USER3> Basically, you just need to pass a Chef bootstrap script in through the instance's user-data.

Knife, a Chef management and interaction tool, provides several bootstrap scripts you could use as a starting point: [https://github.com/opscode/chef/tree/master/chef/lib/chef/kn...](https://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap)

Let's use the Ubuntu 10.04 gem bootstrap script as an example ([https://github.com/opscode/chef/blob/master/chef/lib/chef/kn...](https://github.com/opscode/chef/blob/master/chef/lib/chef/knife/bootstrap/ubuntu10.04-gems.erb)). The basic steps to use this with Chef would be:

(Note that this assumes you have a Chef server or are using the Opscode Platform. There's also a server-less option called "chef-solo" that you can use)

1\. Create an appropriate first-boot.json to be used by the node(s). This will probably just be a "run_list" with entries like "role[app_server]" (function) and "role[production]" (environment).

2\. Replace line 42 of the bootstrap script with your first-boot.json. Since this bootstrap script is actually a template, make sure to replace all other .erb stuff ("<%= %>") as appropriate.

3\. JSON-encode the bootstrap script (handy tool: [http://artur.ejsmont.org/blog/content/ultimate-web- encoder-d...](http://artur.ejsmont.org/blog/content/ultimate-web-encoder- decoder))

4\. Build your CF template, placing the JSON-encoded script in the "user data" section of your instance/auto-scaling group(s).

5\. Instantiate your CF stack

If you'd like to use Chef for the entire lifecycle of your instances, just include a Chef recipe for running chef-client via cron every, say, 5 minutes. This allows changes to be propagated automatically in a pull style.

Done right, the end result is a complete, Chef-managed environment provisioned from bare metal using just one command.

<comment><USER11> Cool, thanks a lot for sharing. Your post has given me the motivation to start learning Chef. Maybe you can blog about it somewhere to help wanna be cloud programmers like me.

</comment></comment></comment></comment></comment><comment><USER12> Werner's post: [http://www.allthingsdistributed.com/2011/02/aws_cloudformati...](http://www.allthingsdistributed.com/2011/02/aws_cloudformation.html)

</comment><comment><USER13> We are just getting into cloud computing in a big way but the main problem has been spooling new machines up in a clean and repeatable way. Our current thought is to boot up a "raw" AMI and use Puppet to provision it.

CloudFormation might be a better way. Instead of having Puppet maintain the system, why not just update the CF template and spool up a new instance to replace the old, outdated one? I guess it just depends on how much you can stick in a template. If you can specify specific versions of packages (Ruby 1.9.2) it might just work.

<comment><USER14> I'm using Judo (<https://github.com/heroku/judo>) to do 'launch-to-running- services-on-devices'. Great tool for what I'm doing, though it looks like CloudFormation has a whole lot of overlap.

</comment></comment><comment><USER15> Wonder what this means for companies like <http://jumpbox.com/> We used Jumpbox in the in the past to setup some quick examples for testing.

</comment><comment><USER16> RIP Rightscale server templates.

</comment><comment><USER17> This looks wonderfully simple to configure considering what happens under the hood, and the conversation on Convore is already about how long it will be before a Django template is written by the community.

</comment><comment><USER18> So this is like an AWS-specific Puppet implementation? It doesn't seem to go all the way down to installing linux packages for example which would be really nice.

</comment><comment><USER19> This is big. Makes AWS a wholesome offering. Kudos to Amazon!</comment>


How we are providing 15 designers for the price of 1 - <AUTHOR> http://draftss.com/blog/index.php/2018/07/24/15-designers-for-the-price-of-1/ <comment><USER2> Interesting concept. How is it feasible for you guys?

<comment><AUTHOR> We are able to achieve this as we are based in Mumbai, India where the cost of a professional graphic designer is low

<comment><USER2> That's a location advantage which turned out favourable for you guys. How long have you been operational?

<comment><AUTHOR> We had been running a traditional design shop for more than 10 years. We launched draftss.com - design on subscription model around 3 months back. Here is an outlook for our first month: [http://draftss.com/blog/index.php/2018/06/18/zero- to-3325-mr...](http://draftss.com/blog/index.php/2018/06/18/zero-to-3325-mrr- in-30-days/)</comment></comment></comment></comment>


Ask HN: I'm giving away my startup taketake.com for free - who wants it? - <AUTHOR>

I could go in a long explanation of the personal reasons why my co-founder and I have decided to give away our startup, but let's just say in a nutshell that we've burned out.
We're looking for at least two people to give it away to. Preferably a team with both tech and UI/UX experience. We're running on our own PHP framework. You'd need to master nginx, redis, resque, Node, MySQL and MongoDB as well.
You will get the majority stake after a vesting period. You'd be in charge of everything and it would be your startup to do with as you see fit. My co-founder and I would retain a minority equity stake in the startup and have an advisory role.
We thought of shutting down everything, but would rather "give" the startup to another team to take over. We believe in taketake a great deal and have some solid ideas for its business model.
We are looking for a team who are ready to fully dedicate their time to taketake to launch it, raise money and grow it.
We think we got solid branding and a compelling product. We were accepted in mini Seedcamp (we didn't go) and have been invited to pitch at Echelon next month. We have an agreement with a top online retailer to share their users purchases on taketake and also have top bloggers interested to cover our launch. We're at a good place.
If you are interested, please email matt-at-taketake.com <comment><USER2> *assuming this is not an april fools joke

I don't usually login to post, but I just wanted to say, although I do not know your situation at all, if you're putting it up due to burn out, please take a second look and make sure what you're doing is for the best. Startups always have their down cycles, so I would hate for you to give away your company if it could be retained in some way, as burn out / down cycle are problems that can eventually be fixed.

I say this as I was on the brink of shutting my own startup very recently before I found my passion and drive for doing it again (after a burnout that took 8+ months to recover), and I'm very glad I ended up sticking with it.

<comment><USER3> I agree completely. Startups can feel a lot like "swimming" a marathon instead of running one, but what great rewards when the really hard work is done. I'm sure there are many circumstances that take anyone out of the game, but I agree with you. Maybe they could wait a while longer and see if they get through their down cycle??? I wish them good luck whichever way it goes!

</comment></comment><comment><USER4> I just wanted to say that -- wow, this is one beautiful site design. Sorry it didn't work out so far. If you end up giving it away, I hope it finds good hands!

That said, I hope the two of you can take long vacations, find new energy, and that we'd see a "How we got 1000x users" type of post from you in the future months.

</comment><comment><USER5> Clickable link to the site:

<http://taketake.com/>

Looks nice, btw.

<comment><USER6> Loved the clickable link. I'm such a procrastinator.

</comment></comment><comment><USER7> Assuming this is not an April's Fool joke: This is actually a very nice idea. Keep a small share and let other people run it. I hope someone skilled will take over. Please keep us posted how it went. Also I like your "Help us get featured on", idea. You're very clever marketers indeed. :)

<comment><AUTHOR> It's no April's fool joke! Thanks for the kind words :)

</comment></comment><comment><USER8> I like the "Help us get featured on" with all the logos, at first glance it probably looks like you've been on all those, creative!

</comment><comment><USER9> i like the "help us get featured on X"....at a glance it looks like you already were

<comment><USER10> and for a moment I felt betrayed.

</comment></comment><comment><USER11> Still thinking this is a well done attempt to get viral press. You might want to repost tomorrow if it's true.

Aside from that, I hate to be a downer, but I'll be surprised if this takes off. I'm actually wondering if the whole concept is an April fools joke in itself. It's basically relationship status for everything you own. I'll stick to reading amazon product reviews I think.

</comment><comment><USER12> Who did the screenplay and video work for the demo?

<comment><AUTHOR> <http://epipheostudios.com>

<comment><USER13> expensive ? seems it would be to get them to produce a video ? are you able to share the cost ?

<comment><AUTHOR> I'd rather not say here, but it's a bit on the pricy side but totally worth it.

<comment><USER13> yeah - >$5K - hot upwards or cold downwards ? :)

<comment><AUTHOR> $5k is cheap, epipheos aren't cheap.

</comment></comment></comment></comment></comment></comment><comment><USER14> I just want to encourage you guys to go on. It's really a cool site and this is coming from someone who doesn't do much social networking. Given that already quite a number of HN readers are impressed, maybe you can recruit a small bunch of us to help develop, grow the user base, raise money, etc., in return for some upfront equity.

<comment><AUTHOR> That's a cool idea and we will certainly consider it if we can't find a rockstar team to take over. Whoever wants to take over taketake would need to understand well the technology we built it with.

I'm hoping we can find a team to take over. My co-founder and I will certainly be there to help with all the technical stuff and the business model.

</comment></comment><comment><USER15> After watching the video I WANT TO USE THIS PRODUCT please someone take it and develop it.

<comment><USER16> I had that exact same thought! Looked really awesome and a great potential for consumer and creator!

</comment></comment><comment><USER17> Whoa you've gotten to the point of having a Real Team <http://taketake.com/about> ... how can you give it away?... burnout? As others have said here, take a week off. Also, startups can change direction at all times, maybe your initial idea isn't driving you enough. Evolve the idea to get excited about this once again... well I'm just ranting it out, you sure have your reasons. But looking at /about this don't seem like a weekend project to be flipped later at SitePoint or sth. You've built something already! Why not let it live and thrive to see the world.

<comment><AUTHOR> We also want to see taketake thrive and we are doing everything we can to see that happen. My co-founder and I simply can't be leading it anymore. We're burned out and skid off our cash runway. We did take a week(s) off. This is not an easy decision.

By getting another team to take over, it's almost as if we were getting one/two other co-founders to come on board. My co-founder and I will take an advisory role and that's why it's fair that the new team gets a majority stake in the company. We would retain 10-15% of common stock each.

</comment></comment><comment><USER18> Can anyone say guerilla marketing? We want to give part of taketake.com away. You spend a bunch of much finishing things, marketing it and running it. We keep ownership in the upside.

There is no angel or vc that will invest in anything where the original team wasn't confident enough to stick around and see something through unless those ties have completely been severed and a new captain is running the ship. Plus, there is a lot of competition out there already.

</comment><comment><USER19> I have been following TakeTake.com for quite a while now (we're also in this space called "social commerce"), and I still remember your Ask HN thread about whether you should launch after building/refining it for months and your subsequent launch with the explanatory video, which did an amazing job in explaining the service. I would hate to see anyone give up something that looks so promising, but I wish you the best of luck.

</comment><comment><USER20> Out of curiosity, why do you use both Redis and MongoDB?

<comment><AUTHOR> We use Redis and MongoDB for two different purposes.

Redis is used for our messaging queue (resque), sessions and caching. MongoDB is used to store our user data. Redis could be used to store our user data as well but we feel that MongoDB is a lot easier to work with because of its query language.

MongoDB fits better to store our user data and Redis fits better for our messaging queue, sessions and caching. We have yet completed migrating from MySQL to MongoDB which is why I've listed both in my post.

<comment><USER21> whats the reason for the migration from MySql to mongo ?

</comment></comment></comment><comment><USER22> okay this must not be true but at first, it looks like you sent it to HN and other sites so people go online, sign up to browse around(greed for buy) and this will spread the concept, make it famous and then you might decide to keep it with you or give it to your wife or something. I think such silly things. I am sorry.

</comment><comment><USER23> Impressive site design.

On what basis did you decide this wasn't working out? Low virality, low engagement, SEO didn't work out....?

The 'long explanation' of why you got burned out would be a very interesting post.

<comment><AUTHOR> Well, made a lot of mistakes. Discovered the lean startup principles too late. We should have released an MVP earlier and iterate. Instead we were never really happy with what we built and started from scratch too many times. We became really frustrated with not being able to execute up to our expectations. Waited too long and slid off our cash runway.

I will probably write a more thorough post about it later on, but right now we're trying to focus on finding a team to take over and save taketake.

</comment></comment><comment><USER24> That is beautiful site design, and I'm thinking this is an April Fool's day joke!

<comment><AUTHOR> It's not :)

<comment><USER25> Wether this is an april fools joke or not: I think it's a smart move either way. A post like this is ought to get some attention :)

<comment><USER22> Exactly

</comment></comment></comment></comment><comment><USER26> When I go to peoples websites I always ask my self "Would I use this?". At first I was like "nah" but by the end of the video I was "hmm yeah let me sign up!".

It's a shame you're giving this away.

<comment><USER27> I completely agree. As much as I hate videos that play automatically, your intro was inoffensive enough that I listened until it got interesting.

On a side note, why do you have invite only on?

Best of luck whichever way you go with your startup. Maybe you just need a short holiday to sleep on it...

<comment><AUTHOR> The site is not completely ready and we got stuck into that non-sense of making everything perfect before we opened it.

</comment></comment></comment><comment><USER28> If it's money you need (would you stay on if someone offered you an angel round?), consider asking for funding, kickstarter style. We're all very impressed, and we think you should stick with it.

<comment><USER28> And if you're determined to get rid of it, you should not give it away- you should sell a chunk of it off for a substantial amount of money just so the incoming party has the same interest in making it work. In otherwords, who ever you give it to will not have poured sweat over it, and will likely not give it the love it rightly deserves.

</comment></comment><comment><USER29> Very beautiful site indeed! Very clean design and i liked the xxx have requested invitation + the send a tip!

Will you guys be still coming to the Echelon? in Singapore right?

<comment><AUTHOR> Hopefully we can get a team to take over and then possibly be at Echelon in Singapore. We will see how things develop in the next few days :)

<comment><USER29> Good luck! And hope to catch you guys in Singapore :)

</comment></comment></comment><comment><USER30> Just so you guys (or potential buyers) know: <http://www.mywot.com/en/scorecard/taketake.com>

<comment><USER31> How and from what are those scores derived?

I tested this by entering the address of my former blog. It ranked high in "vendor trustworthiness" even though there is no commercial aspect to the site at all. It has a high child safety rating even though some of those posts contain very strong language.

Am I mistaken or is MyWOT useless?

<comment><USER32> Ratings come from users of the WOT browser extensions. Users are able to rate sites based on their personal experience with them. Sites with poor reputation yield warnings from the extension upon later visits.

If no one has rated you, your rating will stay positive, or neutral, until someone does.

<comment><USER31> That's what I expected. Like I said, my test site wasn't "neutral", it was clearly rated positive on vendor reliability and child-friendliness. So what it boils down to is that someone arbitrarily gave taketake.com a very negative rating, not necessarily based on anything real. I'm not saying it may not be justified, but this kind of reputation system is clearly very flaky and prone to manipulation.

</comment></comment></comment></comment><comment><USER33> Since nobody else has said it, I'll take it!

<comment><USER34> Don't you mean taketake it? \ _groan\_

I'd of had it, even just for the domain name. I can understand giving it away but not giving it away to just anyone with no commitment to continue the project.

</comment></comment><comment><USER35> April Fools I reckon, the site is quality ! Love the look of it, hurry up and accept my invite so I can take a propper look.

</comment><comment><USER36> If this isn't an april fools joke why would you post it today? I call shenanigans. I guess we'll see tomorrow though.

<comment><AUTHOR> It didn't cross my mind to be honest. Had much more serious stuff on my mind than April's fool when I posted this. It's not a prank.

</comment></comment><comment><USER37> Is it possible to get a user login to see what functionality has already been established?

<comment><AUTHOR> For people who are serious about it, have the right skillset and contact me by email this will be provided :)

</comment></comment><comment><USER38> I can tell this an April Fools joke, but that's a nicely designed website.

</comment><comment><USER3> I really like your site. Hope all works out for you! Great intro. :)

</comment><comment><USER39> Sorry things haven't worked out...as yet! Don't give up keep pushing.

</comment><comment><USER40> Ummm.... what's the date again people? This looks like way too much work to just "give it away". I was expecting an empty homepage with a single search box. Nice prank though. Me likey.

<comment><AUTHOR> It's 100% real and not a prank, but yeah I guess I should have waited a day before making my post.

</comment></comment><comment><USER41> How long has the site been in invite-only mode?

<comment><AUTHOR> Beginning of January.

<comment><USER25> So have invites been already sent out? Interested in an inside-view...

<comment><AUTHOR> They haven't been sent out yet.

<comment><USER42> So, you don't actually have any users?

</comment></comment></comment></comment></comment><comment><USER43> So, it's angie's list for products... ?

</comment><comment><USER44> Looks pretty, but all this talk about relationships with inanimate objects in the video made me feel like it aimed at owners of real dolls. I have relationships with people (and maybe animals and myself), objects just enable me to have these relationships. Maybe an object oriented website just isn't my thing.

</comment><comment><USER45> No "startup" of any value would ever be given away, regardless of how burnt out its employees or owners might be. This is rediculous. Either this is an April Fools joke, or you never had the dedication to succeed in the first place.

<comment><AUTHOR> We're not giving away 100% of it. We don't want to have an active role anymore and are willing to give away a majority equity stake to a qualified and competent team who would be taking charge of it. It's almost as if we were taking another co-founder on board. But because we would be taking an advisory role, it's just fair that (after vesting) the new people in charge have the majority stake in it.</comment></comment>


Python Lists vs. Tuples - <AUTHOR> http://nedbatchelder.com//blog/201608/lists_vs_tuples.html <comment><USER2> So, I've long since argued the importance of what Ned terms the "cultural difference", or the difference between dynamic homogeneous structures (lists/relational columns), and static heterogeneous structures (tuples/relational rows). There are important performance ramifications regarding the use of one or the other, and conflating the two (i.e. dynamic heterogeneous) is a strong design smell. Some languages "get it", others don't.

Python is _not_ a language that "gets it", despite the article's examples. Python chooses the much-less-useful distinction between "immutable sequence" and "mutable sequence"; the lack of any other distinction is borne out by the standard library (as dozzie points out here [1]). Furthermore, the issue of immutable vs. mutable is _totally orthogonal_ to that of dynamic/static and homogeneous/heterogeneous. There's no reason that "cultural" lists need be mutable, or "cultural" tuples need be immutable. So Python's choice doesn't make sense from this angle either.

For example, OCaml has both immutable lists and immutable tuples; its type system enforces exactly the "cultural difference". Similarly with C, which has both mutable lists (arrays) and mutable tuples (structs), both of which can be rendered immutable (const). Why Python chose lists (and only lists!) to have both immutable and mutable flavors, with otherwise identical semantics, is beyond me.

Erlang used to "get it", but they lost the thread when they came out with heterogeneous syntax support for a homogeneous structure (map). TLA+ totally conflates tuples with lists, but its type system is rich enough to enforce the difference (and as a bonus, allows for the elusive static homogeneous structure).

[1] [https://news.ycombinator.com/item?id=12323849](https://news.ycombinator.com/item?id=12323849)

<comment><USER3> "here's no reason that "cultural" lists need be mutable, or "cultural" tuples need be immutable."

er, yes there is. if a list is not mutable, doing list.append(something) in a loop would re-initialize the list every loop iteration. the same reason you shouldn't add to a string in a loop, as strings are imitable.

to me, an immutable list is not a list.

<comment><USER2> Have you used a functional language? (Lisp, OCaml, Erlang, etc.) Lists are immutable, yet appending items is O(1). That is achievable with even a basic singly-linked list implementation.

<comment><USER4> Lists aren't immutable in Lisp, though. More precisely, Lisp doesn't have lists: it has chains of mutable cons cells.

<comment><USER5> They're not _technically_ immutable, but _culturally_ Lisp programmers are not in the habit of mutating lists unless they know where and for what purpose the list was created. Mutating some list whose provenance you don't know -- maybe you got it by calling some library function that doesn't explicitly promise to return a freshly consed list -- is a bad idea, and experienced Lisp programmers know this instinctively.

So in practice you can often get away with pretending the lists you create are immutable, if that's what you want to do.

<comment><USER4> Culture isn't something I can take into account when reasoning about what a piece of code means. The actual semantics of the programming language in question is all I really have.

<comment><USER5> I doubt that. I think it would take too long to work out the meaning of every piece of code you see from first principles. I think any competent programmer has a large collection of unconscious heuristics they use to understand code. Learning to write code in such a way that the reader's heuristics will not mislead them is, I think, a big part of becoming a software engineer.

I was just trying to describe the state of practice in the Lisp world. I do agree that Lisp benefits from more functional data structures -- that's why I wrote FSet [0]! I would point out, though, that since Lisp lacks any kind of access control, there's nothing besides common sense and convention that prevents FSet client code from mucking about inside the guts of the FSet data structures. It would be a stupid thing to do, and everybody knows that, but the language doesn't prevent it. So the situation is not that different with FSet collections than it is with lists of unknown provenance.

Even in languages that nominally support access control, like C++ and Java, you can usually still get around it with casting or reflection. Most people know that they should play games like that only in very specific circumstances.

[0] [https://github.com/slburson/fset](https://github.com/slburson/fset)

<comment><USER4> > I think it would take too long to work out the meaning of every piece of > code you see from first principles.

Just as in mathematics, once you've proven something, you can reuse the result as often as you want. But "the community always does this" isn't a proof.

> I do agree that Lisp benefits from more functional data structures (...) > there's nothing besides common sense and convention that prevents FSet > client code from mucking about inside the guts of the FSet data structures.

Then Lisp doesn't have functional data structures. That's no surprise, Lisp has no more compound values than Python or Ruby do - that is, none.

> Even in languages that nominally support access control, like C++ and Java, > you can usually still get around it with casting or reflection.

Standard ML actually has type abstraction, and, once you've made something abstract, there's no way clients can inspect it. No downcasts, no reflection, no anything - abstract means abstract.

<comment><USER5> > Then Lisp doesn't have functional data structures. That's no surprise, Lisp > has no more compound values than Python or Ruby do - that is, none.

Ah, the arrogance of youth.

<comment><USER4> It isn't arrogance, it's just the semantics of the programming languages in question. Programming languages being such fundamental tools, you would think every programmer should understand them well, but that is far from being the case. It is tempting to blame programmers, but the real culprit is the subtlety and depth of the topic. Two great resources to get started are [https://www.amazon.com/dp/0262201755/](https://www.amazon.com/dp/0262201755/) and [https://www.amazon.com/dp/0262690764/](https://www.amazon.com/dp/0262690764/)

Anyway. A value is something you can bind to a variable. Unfortunately, you can't bind _the_ list [1,2,3] (or whatever syntax you might prefer) to a variable in either Python or Lisp, because those languages simply don't have lists (or any other kind of compound value).

<comment><USER5> I read back through your recent comments. I notice you get downvoted a lot. Does this bother you?

<comment><USER4> I don't particularly mind.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> Great article!

One more important difference between tuple and list that's worth knowing about: syntax. Ned makes it seem like tuples have the syntax "(x, y)", but they actually have the syntax "x, y".

I find that a lot of Python programmers, even long-time ones, don't realize this important difference. It's what allows one to write code constructions like "x, y = 1, 2" or "return 1, 2".

One needs the parentheses only to "clarify" this syntax in certain cases; for example, as dict keys, the parens are necessary -- "x = {(1, 2): 2}". Although interestingly, for dict lookup, they are not -- "x[1, 2] == 2".

One way I remind myself of this rule is to realize that "x = 1," is the same as "x = (1,)", but "x = (1)" is actually the same as "x = 1". This is quite different from lists, who are syntactically defined by the "[]" brackets, and thus you require them, or a "list()" call, whenever you make lists, making it just that much more heavyweight as a data structure.

<comment><USER7> When I used to write python "x = 1," bit me _way_ more than it should have. And I don't see why you wouldn't want to be explicit about your tuple construction every time..

<comment><USER8> I usually use x = tuple((1,) ) so that non-Python programmers on the team know what they're dealing with.

</comment></comment></comment><comment><USER9> It's much simpler to me. Tuples are used when you need a sequence but don't need to modify the contents, for instance returning multiple objects from a function call. Lists are used when you want to add things to it.

Namedtuples are wonderful! It's like a class with only attributes, except you don't have to define a new class and get slapped across the back of your head by more experienced programmers for creating a useless class. Access objects by index, by name, or by iteration. It's very versatile.

<comment><USER10> If you like namedtuples, you'll love attrs. [https://pypi.python.org/pypi/attrs/16.0.0](https://pypi.python.org/pypi/attrs/16.0.0)

<comment><USER11> I like namedtuples, but attrs looks _so_ wrong to me.

</comment></comment></comment><comment><USER12> > The Cultural Difference is about how lists and tuples are actually used: > lists are used where you have a homogenous sequence of unknown length; > tuples are used where you know the number of elements in advance because the > position of the element is semantically significant.

Oh, if only it were so simple! Python itself doesn't understand what the heck tuples are, in that the built-in functions often treat tuples as simply "immutable lists". See isinstance() and issubclass() functions, which expect what's meant to be a list of classes, but you need to pass a tuple. *args is another such example, as the article mentions.

If what Python calls tuples were the thing with structure instead of order, they wouldn't be iterable.

</comment><comment><USER13> There is an additional technical difference: Tuples are hashable and can be used as keys in a dictionary.

        grid[(1,1)] = True

<comment><USER14> The article mentions this.

Also you can write this as `grid[1, 1] = True`

</comment></comment><comment><USER15> Interesting article. I've always heard that about lists and tuples, but this article made me wonder _how much_ less efficient with storage is a list.

The following trivial example shows that, at least for very small lists and tuples, except for the fixed overhead of 16 bytes for a tuple vs. 72 bytes for a list they're the same [edit: no it doesn't. I wasn't looking at the diff between a 1-element and 2-element tuple]:

Perhaps there's more to it?

      >>> from sys import getsizeof
      >>> list = [94]
      >>> getsizeof(list)
      80
      >>> list = [94, 37]
      >>> getsizeof(list)
      88
      >>> list = [94, 37, 21]
      >>> getsizeof(list)
      96
      >>> list = [94, 37, 21, 19]
      >>> getsizeof(list)
      104
      >>> tuple = (94)
      >>> getsizeof(tuple)
      24
      >>> tuple = (94, 37)
      >>> getsizeof(tuple)
      72
      >>> tuple = (94, 37, 21)
      >>> getsizeof(tuple)
      80
      >>> tuple = (94, 37, 21, 19)
      >>> getsizeof(tuple)
      88

<comment><USER15> Another (admittedly trivial) experiment with speed and storage: Iterating through a 10,000 element list was about 2% slower than iterating through a 10,000 element tuple, and the list was 16 bytes bigger than the tuple:

      #!/usr/bin/python

      from sys import getsizeof
      from random import sample
      from time import time

      l = sample(xrange(10000), 10000)
      t = tuple(l)

      print "The tuple's size is " + str(getsizeof(t)) + " and the list's size is " + str(getsizeof(l))

      s = time()
      z = 0
      for x in l:
        z += x

      print "Iterating through a 10,000 element list took " + str(time() - s) + " seconds."

      s = time()
      z = 0
      for x in t:
        z += x

      print "Iterating through a 10,000 element tuple took " + str(time() - s) + " seconds."

      doug@supermicro:~$ ./x
      The tuple's size is 80056 and the list's size is 80072
      Iterating through a 10,000 element list took 0.00589299201965 seconds.
      Iterating through a 10,000 element tuple took 0.00538778305054 seconds.

      doug@supermicro:~$ ./x
      The tuple's size is 80056 and the list's size is 80072
      Iterating through a 10,000 element list took 0.00553607940674 seconds.
      Iterating through a 10,000 element tuple took 0.00536203384399 seconds.

<comment><USER16> This code seems inconclusive -- my first three runs showed that lists are the same, slower, AND faster! (updated for Python 3.5.2 on Windows)

            C:\Temp>python listtuple.py
    	The tuple's size is 80048 and the list's size is 80064
    	Iterating through a 10,000 element list took 0.0005006790161132812 seconds.
    	Iterating through a 10,000 element tuple took 0.0005006790161132812 seconds.

            C:\Temp>python listtuple.py
    	The tuple's size is 80048 and the list's size is 80064
    	Iterating through a 10,000 element list took 0.0020074844360351562 seconds.
    	Iterating through a 10,000 element tuple took 0.001031637191772461 seconds.

            C:\Temp>python listtuple.py
    	The tuple's size is 80048 and the list's size is 80064
    	Iterating through a 10,000 element list took 0.0 seconds.
    	Iterating through a 10,000 element tuple took 0.001001596450805664 seconds.

<comment><USER15> Interesting! The previous example I posted was Python 2.7.12 on Ubuntu 16.04.01. Here is OSX 10.11.5 w/Python 2.7.10 (also inconclusive with respect to speed, but OSX was half the memory):

      mb:~ doug$ ./listtuple
      The tuple's size is 40028 and the list's size is 40036
      Iterating through a 10,000 element list took 0.00113916397095 seconds.
      Iterating through a 10,000 element tuple took 0.00111603736877 seconds.
      mb:~ doug$ ./listtuple
      The tuple's size is 40028 and the list's size is 40036
      Iterating through a 10,000 element list took 0.00126791000366 seconds.
      Iterating through a 10,000 element tuple took 0.00133585929871 seconds.
      mb:~ doug$ ./listtuple
      The tuple's size is 40028 and the list's size is 40036
      Iterating through a 10,000 element list took 0.00114607810974 seconds.
      Iterating through a 10,000 element tuple took 0.00117492675781 seconds.

</comment></comment></comment></comment><comment><USER17> When I taught Python, the way I explained was that a tuple is basically one value, shrinkwrapped. It's one value so that you can put anywhere that expects a value (such as a return value or a key of a dictionary). Also it's one value so you cannot change it.

The way you look at a statement like (a,b) = (2,3) is that this is a normal assignment, where you imagine there are one value flowing in the equal sign, from right to left. But it happens to contain two values within its wrap.

</comment><comment><USER5> It's really too bad that functional data structures weren't much of a thing yet when Guido started developing Python. The language really cries out for them -- particularly with the "mutable initial value" problem and the 'set'/'frozenset' distinction.

<comment><USER18> Can you elaborate on how either of those, uh, "features" would be helped by functional "persistent" data structures?

In the case of the mutable initial value problem, that's just a choice of semantics (in my opinion, a very bad choice). Without having looked at the implementation, I would assume that if you have no backwards-compatibility concerns, a fix would take a few minutes.

In the case of sets/frozensets, most Python programmers _want_ their sets to be mutable, in a syntax-sugar-y way that's incompatible with using functional persistent data structures.

<comment><USER5> Isn't it obvious? If the initial value is an empty _functional_ list, set, or map (dict), there's no possibility of it being modified. The problem disappears.

> most Python programmers _want_ their sets to be mutable

They're _accustomed_ to mutable sets because those are all they've known. But ask a Clojure programmer how they feel about functional collections, and you'll be told they're fine. Functional collections are actually _easier_ to use because you don't have to worry about unintended aliasing.

And functional behavior is _not strange at all_. No programming language I have ever used (and it's a long list) has used shared, mutable semantics for numbers. Consider, in C:

      int a = 3;
      int b = a;
      b++;

After this code sequence, 'a' is still 3, of course! _Nobody_ with programming experience expects 'a' to be incremented at the same time 'b' is; it's not even something you stop to think about -- it's completely automatic. This is true even though I wrote 'b++' rather than 'b = b + 1' \-- 'b++' could more easily be misread as a mutating operation on a shared integer object, like 's.add(x)'. However, there's no language I'm aware of in which that's what it means. (You can introduce aliasing intentionally in C++ with a reference, but this requires an explicit '&' in the code.)

</comment></comment></comment><comment><USER19> The explanation is alright, but would have really benefited from a brief overview of tuple use in the C API.

<comment><USER20> The C API seems like a completely separate topic.

<comment><USER19> C and Python APIs of CPython are really just difference faces of the same thing. Due to the internal use of tuples some of the Python behavior is governed by related C behavior.

For example:

> Another conflict between the Technical and the Cultural: there are places in > Python itself where a tuple is used when a list makes more sense. When you > define a function with _args, args is passed to you as a tuple, even though > the position of the values isn 't significant, at least as far as Python > knows

This is the C behavior of tuple usage being exposed to the Python side. If one explores the C API it becomes clear that an implementation detail of C is being exposed to Python.

> The Cultural Difference is about how lists and tuples are actually used: > lists are used where you have a homogenous sequence of unknown length; > tuples are used where you know the number of elements in advance because the > position of the element is semantically significant.

Here the Culture has moved away from the Technical roots of Python and now they're at odds. On the C side tuples are _not* just used when the length is known ahead of time - tuples are not always immutable on the C side.

The context is important if you really want to understand.

</comment></comment></comment><comment><USER21> I was asked this in an interview. This is essentially the answer I gave. :)</comment>