Google Spreadsheets and Python - happy-go-lucky https://www.twilio.com/blog/2017/02/an-easy-way-to-read-and-write-to-a-google-spreadsheet-in-python.html <comment><USER2> I _love_ Google Sheets. It is so much more powerful than 99.9% of the users appreciate. Here is how I use it at my day job as head of PM at New Relic:

      - Fetch deals closed and lost hourly from Salesforce
      - Fetch for each of our 14k+ paid accounts usage metrics using our Insights product
      - Pull both items into a nice color-coded business dashboard that is near realtime
      - Send said dashboard out as a PDF to a bunch of stakeholders daily/weekly
      - Save PDF snapshot into Google Drive so I can easily pull up historical reports

If you haven't tried out Google App Script and supercharged your spreadsheets, give it a shot. It's also basically the only way the engineer in me gets to code much these days, so I have fun with it too :)

PS: Google Sheets is definitely much slower for pure spreadsheets than Excel, no question about that. But by using cloud-based JavaScript + custom code, I can often work around that issue by writing functions that process the data quickly and report on what I need. Ultimately I end up getting much more out of it than Excel.

<comment><USER3> I get all the good things Google Apps do. It's wonderful. Yet, until Google has a "contract with our users" that includes a level of customer service and human conflict resolution I would not touch them with a ten foot pole.

Why? Because they could take it all away --and I do mean ALL-- overnight. Email, apps, etc.

I've seen many people run into the Google ban for reasons not more complicated than not being experts and using one of Google's tools in a way that triggers an algorithmic ban. And this doesn't have to have anything to do with fraud or scams.

Had one client, probably ten years ago, who moved all of his 200+ domains to a service Google introduced at the time called "Google for Domains". A domain parking service with ads inserted into the parked domains auto-magically by Google. Prior to that all of his domains were parked at GoDaddy, who used Google to stuff ads into the domains. As soon as the service was offered he figured it was a good idea to remove GoDaddy as the middle-man in that equation.

He transferred all of his domains. There was an approval process. All domains were approved overnight. Done deal. Right?

Wrong! Two days later Google sent him an email informing him that all of his Google accounts were now permanently suspended (aka: closed) with no recourse. Reason given: Unusual click activity on some of the domains parked with Google for Domains.

This guy ran (runs) a multi-million dollar manufacturing business. The very idea that he would sit there clicking on ads to earn a quarter of a cent per click is, well, stupid.

Anyhow, this one event cost him all of his Google tools. His email, calendar, contacts, apps and whatever else he was using at the time. No recourse. Not conversation with a human being. Nothing. Done. Puff. Evaporated.

Since that one event I have been of the opinion that using any of these Google tools for a business is, to put it plainly, stupid and dangerous. You could be betting the farm on a petulant algorithm and no customer service of any kind to protect you from it.

I still have Excel files from decades ago that I can open and manipulate today. Microsoft could go out of business and I could still open these files. And the software has never lacked functionality, either directly, through add- ins or programmability.

The bottom line for me is that if something is business-critical it isn't a good idea to rely on a service that could be yanked overnight (I do mean that literally). If Google had customer service and a human (and humane) process to deal with issues it would be a very different story. I don't want free stuff. We have tens of thousands of dollars invested in all kinds of software. I want stuff I can rely on because my businesses depend on it.

The only way you are going to have to security is if you pay for well supported software that has a team behind it who understand how important it is for a business to have the ability to wake up every morning knowing that the tools they come to rely on for their daily work won't evaporate overnight.

I truly don't care how many shinny new things, candy and chocolate Google throws on the table. Until they prove they understand this one point they represent a business-killing risk nobody should be willing to accept.

BTW, this is a problem with nearly all Internet giants. Facebook, Amazon and others are horrible companies from the standpoint of how they deal with their business customers.

<comment><USER4> In terms of your anecdote about your client's experience - I can't speak to this, as I'm not directly familiar with this "Google for Domains" service and I don't have enough details from your post to comment.

However, I can assure you that Google definitely has a customer support department - I work with them every day =).

Also - it's important to distinguish between the free products we offer - <AUTHOR> the paid products - where you actually give us money. In the latter case, there is nearly always a customer support hotline, and in many cases, a binding SLA agreement.

I work in Google Cloud, on the support organisation side of things - and I can confirm there is definitely phone support (along with chat and email), we do have SLAs on all of our products and you are definitely speaking to a human. It's not particularly hard to reach a person - either go through your admin panel ([https://admin.google.com](https://admin.google.com) \- click the big question mark), or all of our 24/7 support numbers for each country are right there on the website:

[https://support.google.com/a/table/3247295?hl=en](https://support.google.com/a/table/3247295?hl=en)

That's on the business side. On the consumer side, the phone numbers shouldn't be too hard to find either on a search e.g.:

[https://support.google.com/googleplay/answer/7100415?hl=en&r...](https://support.google.com/googleplay/answer/7100415?hl=en&ref_topic=3364260)

My own anecdote - several years ago before I started with Google, I subscribed to Google Play Music - I accidentally un-favourited a track once, and I couldn't find a way to find it again. So on a whim I rang Google Play Music support, and went straight to a person within seconds. She was very friendly, and tried to help me find the history - unfortunately it wasn't there, but the customer service was pretty damn good - a heck of a lot better than all of my experiences with several un-named mobile phone and broadband companies. (Hint: I'm in Australia)

Disclaimer: I work for Google.

<comment><USER3> You might not know the answer to this question. If you don't, it might be interesting to see if you can get to the answer internally.

Let's say I setup a site and have AdSense place ads on it. I don't touch any of the ads. I never click any of them. Yet, for some reason, the ads see a bunch of click activity. The reasons can be many, from sheer randomness to a competitor using it as an attack vector (to deny revenue and more).

I have seen this precise scenario lead to the absolute shutdown of a Google account. And that means every single service, including email and docs. Gone. Done. No recourse. No way to even recover the data.

Has this changed?

Do you have a way for people affected by such algorithmic issues to engage in a conversation with something that does not resemble a totalitarian regime?

The paid vs. free part is immaterial. People are relying on your products and offerings. Free is a strategy for dominance. I can understand free products not having a human in the loop. What I can't understand is the dictatorial/totalitarian relationship with your users, who, free or not, have come to rely on these tools.

<comment><USER4> I believe you're referring her to Adwords clickfraud - is that right? I don't work on that team, so can't comment on their policies (and even if I did, I probably can't comment publicly).

However, my understanding is normally AdWords accounts are suspended for billing/fraud related reasons (e.g. you don't pay your bills) - e.g.:

[https://support.google.com/adwords/answer/2375414?hl=en- AU](https://support.google.com/adwords/answer/2375414?hl=en-AU)

If you AdWords account does get suspended, there's a form to appeal it here:

[https://support.google.com/adwords/contact/pf_suspended](https://support.google.com/adwords/contact/pf_suspended)

I haven't seen it lead immediately to the actual Google login itself being suspended. However, if your the actual Google account was suspended - there is a set procedure you can follow to un-suspend it - there should have been a link in the notification email.

Also - paid or free isn't immaterial - it actually does make a difference.

If it's a free account (E.g. @gmail.com), you can fill in the online forms to appeal the suspension.

If it's a paid account (i.e. part of a GSuite domain - or it's old name, Google for Work), then what often happens are individual accounts are suspended, and you talk to your own domain administrator to get that resolved. There are ways to get a domain suspended (e.g. you don't pay your bills) but they're usually fairly obvious.

(Disclaimer: I work for Google, but the above comments are my own opinions).

<comment><USER3> You are confusing your own products. AdSense. Not AdWords.

The various forms Google makes available for appealing suspensions are pretty much useless.

I urge you not to believe what I am saying here as well as not feeding within your own internal echo chamber. Use your own search product to search the 'net for the thousands of stories of various forms of algorithmic account suspensions, the damage they cause and how frustrating it is to get anywhere with Google.

Based on what I've witnessed personally and what I've read online the idea of relying on Google products for anything business critical terrifies me.

You --Google-- needs to truly care for it's clients and, as I said in a prior post, issue a guarantee. I am NOT talking about a guarantee of uptime for a service, that's irrelevant here.

What I am referring to are guarantees of service longevity as well as protection and recourse from algorithmic account suspensions. A business person needs to have the ability to address and discuss misunderstandings or problems and not have their entire Google-provided infrastructure evaporate overnight with nowhere to go.

To me the matter of free vs. paid is immaterial. Google and companies like Google use free services to gather and monetize audiences. For example, nobody would pay for search. Nobody would pay for a Facebook account. So you use free to bait the hook and capture audiences. This is a technique as old as the Internet, with browsers such as Netscape and earlier being the first land- grab-through-free-products.

If you are going to have a virtual monopoly by resorting to free products you also have to have the responsibility of not causing your user base irreparable damage by pulling those product either by early termination of the services or termination or suspension of account without recourse. At the moment you are large enough to not have to care about this one bit and nobody has challenged you in court for this terrible issue.

Start here and read through a few pages of links:

[https://www.google.com/webhp?sourceid=chrome- instant&ion=1&e...](https://www.google.com/webhp?sourceid=chrome- instant&ion=1&espv=2&ie=UTF-8#q=google+suspended+my+account)

<comment><USER4> You ask for guarantees that an account will never be suspended, or that a service will never be discontinued.

I don't think that's even possible.

For one - abuse is a real thing.

Just ask Amazon about abuse and spam on their AWS platform. There is a reason that you can't just type in any credit card, and spin up 100 EC2 instances, or send 100,000 emails - depending on signals, Amazon seem to manually verify some accounts. But on top of that, they take proactive steps to suspend accounts acting in suspicious ways.

We do much the same.

It is a hard problem, and undoubtedly there are false positive and false negatives. That is regrettable - but I don't think allowing AWS or GCP to become a free haven for spammers or abusers is the answer.

You are correct that we need good procedures to deal with appeals of these things.

Regarding never deprecating a product - Google has started _many_ products, and some of those have been shutdown. I know there's a lot of angst on HackerNews about Google Reader in particular - I myself used it a lot in fact. However, taking off my Google hat - of all companies, they are usually the best about letting you export your data, or migrate it away. When a product gets shutdown, you usually have 6 months, a year etc. to export your data. Even before I started there, they had Google Takeout which basically lets you export your data from any Google product in portable formats e.g. mbox, JSON files etc. Open standards are important to us - e.g. see our work on Kubernetes.

If there's a specific are you think we're failing on - please let me know, and I'll see if there's anything I am able to share, or if there's somebody I can link you up with.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> A friend who works at the Dallas Animal Shelter (a non-profit) asked me to help automate their process of generating reports from data gathered via Google Forms, which they were doing by hand every month... As a developer who is used to high technical investment into complex systems, I had to take a completely different approach to storing their information.

Google Forms + Spreadsheet + Python works perfectly for use cases like this: it is free, scaling isn't a concern, users can view and modify data at any time, and users can create Google Forms at any time. If you are trying to help a non-profit and aren't planning on maintaining their systems long-term, this is a great approach.

Code for the project: [https://github.com/georgeaf99/das-care-contact- forms](https://github.com/georgeaf99/das-care-contact-forms)

<comment><USER6> Really curious in the "low" technical investment answer: where/how does the Python end of this system get run?

<comment><USER7> >which they were doing by hand every month

I would assume they just installed python on whatever machine they were originally using to hand-generate reports and the job of that person now is to just run the python script and wait for it to finish.

<comment><USER8> Google's AppEngine would likely be free or extremely low cost in this situation. Gspread appears to work there fine: [https://github.com/burnash/gspread/issues/39](https://github.com/burnash/gspread/issues/39)

</comment></comment></comment></comment><comment><USER9> [https://gspread.readthedocs.io/en/latest/](https://gspread.readthedocs.io/en/latest/) is great, but it's limited in what kinds of things it can set (formatting, notes, etc.). One other option is to create an endpoint on script.google.com which can access a much richer SpreadsheetApp API: [https://developers.google.com/apps- script/reference/spreadsh...](https://developers.google.com/apps- script/reference/spreadsheet/spreadsheet-app) . You can POST to it using an auth token from the same service account oauth creds (though you need to add drive and drive.scripts to your scopes), and it can run arbitrary JS to translate reads and writes from the JSON payload/response into API calls.

As another note, we realized that far more useful than _using_ Google Spreadsheets as the canonical backing database, was to be able to bidirectionally synchronize it with our primary database. That way, users who wanted to annotate entities in spreadsheet form could do so in GSheets, always working with up-to-date data, and keeping track of "I updated a.x in the spreadsheet, but a.y was updated upstream, so merge the two." Here were the semantics of our integration:

        Returns a list of updates between last_synced_data and sheet.
        Subsequently, if upstream_data is provided, then load it into the sheet,
        adding rows on the end as needed, or merging if there is a match in the merge_key column
        (note that any updates to the live sheet data since the last sync
        override any upstream data, and those live updates are returned without changing the live sheet).

The caller would then be responsible for taking the returned list of updates and cleaning it for the database, as well as maintaining a record of what the state of the last sync was. Essentially we maintain enough information to do a three-way merge. We've since built internal applications that allow real-time spreadsheet-like interactions in a much more domain-specific manner, but it definitely did the job for quite a while.

If there's interest in seeing open-source code for all of this, we could definitely extract from our corporate repo (we're [https://www.belstone.com/](https://www.belstone.com/) ). Let me know!

<comment><USER10> One thing to note is that the new version (v4) of the Sheets API can access the same fancy functionality that Apps Script can. So if gspread moves to that (which e.g. [https://github.com/burnash/gspread/issues/435](https://github.com/burnash/gspread/issues/435) alludes to), some of those limitations will go away.

(And the sync thing is neat!)

<comment><USER11> [https://github.com/nithinmurali/pygsheets](https://github.com/nithinmurali/pygsheets) uses APIv4; someone couldn't wait!

</comment></comment></comment><comment><USER12> The big downside I've found using this a lot in the past is that Google has no SLA for sheets. Sure its Google, so it has high availability but things can quite often fail with no support from Google. So just be careful of building anything mission critical with this. The API can be wonky every once in a while. At one point it even took them like 6 months to fix an issue I had with the Apps Script API. If my business relied on that functionality, I would have been screwed.

The Jupyter Notebook use case sounds great but if you move to something that is more critical use a real data store.

<comment><USER4> Regarding your claim about SLAs - there actually is an SLA for all of the paid Google Cloud products - e.g. the following would apply for GSuite, which covers Google Sheets:

[https://support.google.com/work/answer/6056635?hl=en](https://support.google.com/work/answer/6056635?hl=en) [https://gsuite.google.com/terms/reseller_sla.html](https://gsuite.google.com/terms/reseller_sla.html)

If you mean the _free_ products e.g. Google Sheets with a consumer account - AFAIK, there isn't a contractual SLA per-se, but the uptime is pretty good =).

You can also see a status dashboard with historical status here:

[https://www.google.com/appsstatus#hl=en&v=status](https://www.google.com/appsstatus#hl=en&v=status)

RSS feed link is at the bottom of that page.

(Disclaimer: I work for Google Cloud, on the Drive/Docs/Sheets side of things).

<comment><USER12> But does that cover the API and if devs break something during a release? That's where I see the problem often. New API gets released that breaks something but the broken thing doesn't get fixed quickly.

<comment><USER4> Yes, the SLA does cover the API - these fall under the main product.

The status dashboard ([https://www.google.com/appsstatus#hl=en&v=status](https://www.google.com/appsstatus#hl=en&v=status)) lists what's covered or not covered - table at the top is all covered services (e.g. GMail, Google Drives/Docs/Sheets/Slides, Google+, Google Groups, Hangouts, Calendar etc.)

The table at the bottom lists things that _aren 't_ covered (e.g. Blogger, Google Voice, Google Analytics, Google Realtime API) by the SLA.

I believe you can find the list of covered/non-covered service on the support site as well ([https://support.google.com/](https://support.google.com/)).

And what do you mean by devs break something during the release?

Do you mean if we introduce a bug in the code, and it affects the availability of the API? Yes, that is covered as well.

You mention you've seen this before - I can't promise anything, but if you're hitting any issues here, or roadblocks, and you're a GSuite customer (i.e. you are paying us some kind of money, even if it's $5 a month), I can certainly try looking into it for you - ping me your details and I can reach out.

</comment></comment></comment></comment><comment><USER13> Having run an (internal) app that is deeply integrated with Google Sheets for about a year, I think that Sheets is good and bad. It's _extremely_ powerful for allowing business users to easily get access to and manipulate data manually. Downside is that it really struggles past a few thousand rows when you've got more than 20 or so columns. Definitely good for a quick and dirty way to expose data to internal users, but it essentially can't scale. Engineers need to be ready to move off of it if the system works.

<comment><USER14> If anyone from Google is reading this: please add a database (yes, like a cloud version of MS ACCESS) in the Google Docs/Drive suite.

Has anyone looked at "Fusion Tables?"

A google search reveals that is what it is supposed to be.

<comment><USER15> They already have one its called CloudSQL [https://cloud.google.com/sql/](https://cloud.google.com/sql/)

<comment><USER16> That's not part of the drive/docs suite.

<comment><USER17> Yep, this should be accessible to users. Someone needs to bridge that gap from spreadsheets to real apps. One click to convert your Spreadsheet into a database. Or hell, just silently migrate it when (NUMROWS + NUMCOLUMNS) > BIGNUMBER .

<comment><USER18> Honestly, if clippy suggested to excel users something similar, I'd be extremely happy.

</comment></comment></comment></comment></comment></comment><comment> stephen-mw I never published it, but I actually wrote a service that uses google sheets to manage SSH access on your servers. Users simply submit a google form with their SSH key, and it's automatically pulled by a daemon on each host. It comes complete with admin/sudo access toggling.

[https://github.com/stephen-mw/gdoc_ssh_manager](https://github.com/stephen- mw/gdoc_ssh_manager)

</comment><comment><USER19> I think there's a bit of a gap in the software ecosystem today in that there's no tool that lets semi-technical people like myself create simple applications, e.g. to read and write to databases. Bit like MS Access used to do.

Google Spreadsheets actually goes a long way (I've done some stuff using the IF function etc. in it), but obviously has its limits. So combining it with Python sounds interesting. Not sure if it's what I'm looking for, but I'll check it out.

<comment><USER20> Have you seen AirTable? It's definitely the best modern alternative to Access I've seen - easy to create tables, great web UI and a phenomenal iOS mobile app.

[https://www.airtable.com](https://www.airtable.com)

<comment><USER21> Is there a downloadable version of this, paid or free?

<comment><USER20> No, it's entirely cloud based. You can install their iOS and Android apps but they'll only work if you have an internet connection.

</comment></comment></comment></comment><comment><USER22> Just because you can doesn't mean you should.

You're putting the entire Internet between you and your "database". You'll receive all the latency and general reliability problems that go with doing that.

In the v3 API, the first blank row terminated the column data set. Which meant you couldn't access rows after that blank row, which may have been accidentally inserted by a user. They may have addressed this in v4, not sure.

You have almost no ability to restrict the values the user enters. They can be literally any string and your app has to handle every possibility. Restricting type on a Sheet is not really possible.

What if a user is in the middle of editing the spreadsheet when your app attempts to access it? Since the sheet is constantly saving itself, and the user may not have completed their edits, are you getting incomplete or inaccurate data?

</comment><comment><USER23> It's worth mentioning this service that turns Google Sheets into a RESTful API for you: [https://sheetsu.com/](https://sheetsu.com/)

Also sheetrock, which is JS-specific: [http://chriszarate.github.io/sheetrock/](http://chriszarate.github.io/sheetrock/)

I'm not the creator and have only used these for trivial tinkering.

</comment><comment><USER24> Potentially, putting the python function into a AWS Lambda function (or any other serverless function) + attaching an API gateway, you could make a RESTful endpoint for all the CRUD operators for google sheets. Could be a extremely light weight way of storing data and exposing it through REST :)

</comment><comment><USER25> In a project I worked on the implementation of a very complex form (insurance company) with lots of conditions. Given the number of stakeholders and people involved having a say (legal, product owner, sales people, customer care, copywriter, etc) normal iteration (gathering feedback => implement => deploy => repeat) would simply take too long.

I very successfully used Google Spreadheets as a backend for this. Putting all text and the conditional logic in the spreadsheet, and allowing the form to be built as a web page based on the Google Spreadsheet data for instant preview. This allowed the different members in the project to even work in parallel (thanks to collaboration features) with stakeholders - and implementing the feedback immediately themselves DURING the meeting - to see if the results was what they expected.

I wrote a small blog post about this, unfortunately in Swedish, but hey - there's a video at least :) [http://www.rebelandbird.com/hyperiterativ- prototypning-med-g...](http://www.rebelandbird.com/hyperiterativ-prototypning- med-google-spreadsheets/)

Another powerful way I use personally is to use Google Spreadsheet as a data backend for Jekyll based static websites. Here is a Grunt plugin I did to deal with this [https://github.com/stpe/grunt-gss-to- json](https://github.com/stpe/grunt-gss-to-json) \- example usage; my retro games collection [http://games.stpe.se/](http://games.stpe.se/)

</comment><comment><USER26> Google Forms got a big re-do in the fairly recent past and it's an excellent product. My partner uses it a lot and she calls the results her "apps" becaues they are saved to her homescreen on iOS. She has some pretty sophisticated forms going because there's a "Go to section based on answer" feature that lets you build conditions.

If you haven't checked it out in a while it's worth a shot.

</comment><comment><USER27> I'm going to have to be a contrarian on this one.

On my team, we absolutely loathe Google Spreadsheets with a passion. Not necessarily the product itself. It actually is a truly capable and nice spreadsheet program. However, it encourages some extremely bad uses.

Because of the low friction of creating a spreadsheet versus, say, a database, business users have started using it as a substitute for something that should probably live in a database. For instance, I'm thinking of a particular portion of our sales organization that kept their business hierarchy in a Sheet, ie x person is on y team. Now, when you want to do analysis on the data you now have to use something similar to this library to retrieve the data and return it in a dataframe, which then pipes into the rest of your analysis and you end up with a finished product, be it a dashboard or some job.

What always happens is that you write your analysis, set it up to run regularly, and all of the sudden, three weeks later, you start receiving exception emails on the script. When you investigate it, inevitably some jackass business user upstream altered the "schema" of the document, breaking your downstream analysis.

In addition, once you have hundreds or thousands of these scripts running repeatedly, all reaching out to the Google API and in some cases retrieving 100s of MBs of data, Google very quickly rate limits you and you end up with dozens of scripts that broke.

Sheets is great for analysis and one offs, but you have to push back on your business users who will constantly try to use it as an operations platform because they don't know any better.

\- "I need to see an analysis of xyz data."

\- "Ok, where does the data live?"

\- "Well, all of the facts come from this SQL table, and we pull from this web service, and finally we run it through this lookup table we maintain in Google Sheets."

\- "Nope. Come back to me when you put that data somewhere else. Spreadsheets are not databases."

</comment><comment><USER28> Cloudstitch founder here (YC S15).

We wrap Google Sheets and Excel 365 in an API you can use to GET/PUT data along with a number of other goodies:

       - security policies (e.g., row-based auth)
       - file upload support
       - email triggers upon upload
       - a notion of "frozen" releases versus live dev data
         (released but yet undocumented -- email hello@ for details)

Very happy to provide support for anyone looking to use us as an API into their sheets.

</comment><comment><USER29> I build HasGluten [1], fully backed by a spreadsheet. Code is at [2], unfortunately not very up to date... it was my first react project.

[1] [http://hagluten.com](http://hagluten.com)

[2] [https://github.com/hasgluten/hasgluten](https://github.com/hasgluten/hasgluten)

<comment><USER30> You have a small typo in the link to your site. [1] should be [http://hasgluten.com](http://hasgluten.com)

</comment></comment><comment><USER31> Interesting to choose this over simpler storage, such as sqlite. I suppose you get offsite backups for free.

<comment><USER32> (author here) Main advantage in our usecase is getting the CUD interface for free.

<comment><USER33> For free ? Where would the python code run ?

<comment><USER34> He said just the CUD part (Create Update Delete). The assumption is that your data can be entered manually through the Sheets UI and your code only needs to do the Read part.

<comment><USER31> How would the django/flask admin app compare?

</comment></comment></comment></comment></comment><comment><USER35> Google Sheets fully integrates with Google BigQuery as well! [0]

You can create a BigQuery table that's powered by a Google Sheet, or export data straight to Sheets for quick and dirty data wrangling. Some folks use Google Forms or Google Sheets to update ledgers, SKUs, or what have you, with data flowing straight to BigQuery for analysis against clickstream data, server logs, Google Analytics data, weather, etc.

Love AppScript - so many possibilities for a "serverless" framework. How about ODBC?

[0] [https://cloud.google.com/blog/big-data/2016/05/bigquery- inte...](https://cloud.google.com/blog/big-data/2016/05/bigquery-integrates- with-google-drive)

(worked on BigQuery, work in Google Cloud)

</comment><comment><USER36> Huh. I bet that in the Docs SRE team at Google there's a senior SRE who's having a "I felt a great disturbance in the Force moment." Google Sheets is awesome, people at Google are smart, but they are not capable of magic.

In the case of Sheets, the most reasonable sharding scheme for the database backing it would be based on some collection of spreadsheets (in the extreme case it would be just one spreadsheet, but that probably wouldn't be practical). The important assumption here is that all the traffic pertaining to one spreadsheet will ultimately go to one server. So, if there's too much traffic coming related to _one_ spreadsheet, you cannot really scale horizontally by adding more servers – you have to give it more resources. A single Google Sheet is sometimes used by lots of users at the same time, but they are able to limit it from the frontend (for example by making it degrade gracefully).

If the traffic comes from the API, it's going to be a little bit more tricky – especially that IIRC the current rate limiting for the API uses a daily quota, leaving room for some really nice spikes.

So, as soon as a website built using this hack get somehow popular someone is going to have a really rotten day with a lot of pages.

(Of course, this is pure speculation on my side, I have no idea how Google Sheets is _actually_ built – I would be very curious if there's a smart way of overcoming this sort of issue.)

<comment><USER37> One of the good things to come out of Google Wave was a better understanding of how to scale using (eg) operational transform. Even without that, scaling a mostly read only spreadsheet backend isn't as hard as you are making out.

<comment><USER36> Well, a spreadsheet used as intended is mostly read-only, that's no necessarily true if it's backing a website :)

<comment><USER37> Based on the read-write characteristics of every website I've ever built I'd say "mostly read-only" is a pretty good characterization.

Even things like forums have many, many page loads than new posts.

</comment></comment></comment></comment><comment> rob-olmos Like another commenter said, the built-in Apps Script is pretty powerful as it is that it could probably handle requirements more easily and portably.

For example, I've created a simple Apps Script for a Google Spreadsheets that uses OAuth to sync the list of client accounts from QuickBooks Online allowing us to keep track of more structured notes and segmentation, which QBO is still sorely lacking. The sync allows someone to easily update the spreadsheet with any new clients not in the spreadsheet.

However, if I was going to be doing any in-depth calculations and/or a lot of data then I'd probably go the same route with using Python externally.

Forgot to mention: The only thing I'm wary of, and have seen other comments in other threads about, is using it for anything more crucial like a core Excel/Access app because of the potential future change that'll break things.

</comment><comment><USER38> I went pretty far down the path of trying to build applications with Google Spreadsheets and Apps Scripts (we still have some major business processes running on it). It's definitely made me feel the need for something between Spreadsheets and custom development or Salesforce - something like what I imagine MS Access solved in the earlier days.

Here's what it'd need:

\- Spreadsheet like UI \- Validations and field data types (major liability of spreadsheets) \- Good forms integration \- Custom code like Apps Script capability \- Something around workflow and business logic

I think Airtable gets pretty close but I haven't been willing to switch cause of some limitations but I do like where they're going. I'm considering pulling the trigger for Salesforce but keep hearing that it's really easy to shoot yourself in the foot.

<comment><USER39> Depending on what you're doing, App Maker ([https://developers.google.com/appmaker/](https://developers.google.com/appmaker/)) might be what you're after.

Disclosure: I work on Google Cloud, but not App Maker.

</comment></comment><comment><USER40> Back at my previous company, which was a travel startup, there was a list of curated vacation packages that were created for every weekend. So, the itineraries team used to spend 5 days designing the packages and getting vendors on board, and then they used to fill up a google sheet with the details in it and the devs used to run a script, which used to populate this rows in the database and make any other updates required and it used to show up on the website. The reasons why this worked so well for us:

1\. There were at the most 30-40 rows. Google Spreadsheets works amazing for small table sizes (although not as small as 30-40, I'd assume upto 1500-5000 should be fine as well).

2\. All non-dev teams were EXTREMELY comfortable with Google Spreadsheets. For people with non-engineering backgrounds, a spreadsheet is an amazing, low- barrier entry to structured data which I believe is what made this solution amazing.

3\. The dev team was completely removed as a dependency, and we had staging environments where they would run it first to ensure it was working properly, so a dev could be engaged only if their spreadsheet run wasn't working as expected or due to some other issue. The previous method was a huge email sent out to the devs, who would handcraft it into a JSON, which would then be passed to a script and then written to the database. This required every run to be effectively final.. there were only so many times one could engage a dev.

4\. While we had to engage a dev to execute the script, it actually is very easy to integrate a menu option within the google sheets interface itself (if you are using Google Apps) for your domain, which would say something like 'Deploy to Staging', although we never got around to actually building this.

For startups where product turnaround time is required to be short, this works as an amazing solution as it makes so easy for non-tech guys to input data into the system. While there is no doubt that a fully developed panel for these operations would be the best solution, one doesn't always have the luxury of time.

</comment><comment><USER41> I wrote about it a year ago as well: [https://mionskowski.pl/using-google- apps-scripts-for-a-simpl...](https://mionskowski.pl/using-google-apps-scripts- for-a-simple-backend)

</comment><comment><USER42> I am interested in the opposite use case where Google Spreadsheets/Google Apps Script calls an existing python script... possibly using Google Compute Engine? Anyone have any experience and/or suggestions? [https://developers.google.com/apps- script/articles/appengine...](https://developers.google.com/apps- script/articles/appengine#section4) is the closest thing I have found searching myself but it states that it is no longer maintained.

</comment><comment> adam-p I used Google Sheets as the DB backend for a membership management site for my local neighbourhood organization. So far it has worked great (for few users, but that was part of the spec).

Here's the project, and there's more rationale/investigation in the README: [https://github.com/adam-p/danforth-east](https://github.com/adam-p/danforth- east)

(And... it looks like I need to update from Sheets API v3 to v4.)

</comment><comment><USER43> With google spreadsheet appscript, i've built a dashboard with bootstrap, implemented a database with dopost and doget, downloaded 8000+ webpages for data scraping overnight. You can overcome the 5 minutes limit by using this [http://patt0.blogspot.in/2014/08/continuous-batch-library- up...](http://patt0.blogspot.in/2014/08/continuous-batch-library-update- for.html)

</comment><comment><USER44> I'm using Python and Google Sheets to manage my wedding invitation list. Lob for cards, google contacts for contact data, gsheets is the main db. It works well.

</comment><comment><USER45> Did a NodeJS read only Sheets recently no auth needed. Gets array of objects, way simple.

[https://runkit.com/58a77f7a86c06300158dfcf5/58a77f7a86c06300...](https://runkit.com/58a77f7a86c06300158dfcf5/58a77f7a86c06300158dfcf6) [https://github.com/digplan/gsheet-web](https://github.com/digplan/gsheet-web)

</comment><comment><USER46> I would love if I could use Ruby in Excel to program styling and transform cells, I wonder why I haven't seen anything like that.

<comment><USER47> There's something about Excel, I think there are certain aspects of it (programmability, object model) that I think Microsoft is just never going to improve - not that they lack manpower, or that users don't desperately want improvements, it feels more like a matter of principle.

My best guess is that they don't want it to be _too_ good lest it cannibalize their other product lines; I would say the very exact thing was the downfall of MS Access.

Of course this sounds like a silly conspiracy theory, but if you're very familiar with Excel, and especially the pile of smoldering crap that is VBA, I'd be surprised if you don't see at least a glimmer of truth.

That google now has a modern, capable language that can be used with google sheets, perhaps now MS will actually wake up and begrudgingly give their faithful users something they've been asking for, for literally over a decade.

</comment></comment><comment><USER48> I'm also a big fan of this approach, especially for sharing data responsibilities with non-technical colleagues. If you work in nodejs, the node-google-spreadsheet library is pretty good: [https://www.npmjs.com/package/google- spreadsheet](https://www.npmjs.com/package/google-spreadsheet)

<comment><USER49> thanks.. this is useful.. tried it out and perfectly matches my needs.

</comment></comment><comment><USER50> I have built something similar that works with LibreOffice Calc as the backend instead of Google Sheets:

[https://github.com/robsco-git/spreadsheet_server](https://github.com/robsco- git/spreadsheet_server)

</comment><comment><USER51> I remember seeing people in Second Life talking about using Google Spreadsheets as database using LSL so people didn't have to run external databases. Pretty interesting stuff even though I don't think it's really meant for this use case.

</comment><comment><USER52> For a port of gspread which uses API v4 - supports new features like formats, notes etc. Checkout [https://github.com/nithinmurali/pygsheets](https://github.com/nithinmurali/pygsheets)

<comment><USER53> how long the maintenance of this library will last after gspread moves to v4?

</comment></comment><comment><USER54> Exactly for cases like this I came up with DataText, a simple text format for tables with an editor just like an excel sheet.

[https://github.com/kuyawa/DataText](https://github.com/kuyawa/DataText)

</comment><comment><USER55> We use google spreadsheets and gspead/Python for analysis of our AdWords. The AdWords api is a PITA and so we just have a sceduled export to a google spreadsheet that we can then easily access. Works very well!

<comment><USER39> Have you tried exporting to BigQuery ([https://developers.google.com/adwords/scripts/docs/solutions...](https://developers.google.com/adwords/scripts/docs/solutions/bigquery- exporter)) instead? Depending on your analysis (and analysts!) it's probably easier to write SQL. As a reminder, there's also a generous free-tier for BigQuery!

Disclosure: I work on Google Cloud.

</comment></comment><comment><USER56> Theres one easy to use google spreadsheet api here built on gspread [https://github.com/shkr/GoogleDocsAPI](https://github.com/shkr/GoogleDocsAPI)

</comment><comment><USER57> I have a CMS where the client adjusts their website in Google Spreadsheets.

Easy and simple. Multiple auths and roles and it's cached, so it's not queried every minute ( pressing a link clears the cache)

<comment><USER58> I use something similar, but not quite as involved to let a restaurant update their daily specials on their website.

</comment></comment><comment><USER59> The problem with google sheets is row limits. Excel has them too but it's more. Eventually only manageable way to deal with larger datasets is in SQL and managing CSVs with bash to split.

</comment><comment><USER60> Worth also looking at Airtable here as well, becoming a big fan of it.

</comment><comment><USER61> I did a major integration with Google sheets API recently and found the documentation sorely lacking. Docs and Sheets could be so much better.

</comment><comment><USER62> you could also store the data in a json file an let a static web app consume it

</comment><comment><USER63> This is super cool. I will probably use this in the future when sketching out ideas!

</comment><comment><USER64> Cool but it's a non-starter if you go beyond a few thousand rows.

</comment><comment><USER65> nice

</comment><comment><USER66> Our school district has moved to Google sheets for kids grades and I am blown away how well it fits with this use case.</comment>


Building a Developer Tools Business - <AUTHOR> https://manifold.co/blog/founders-guide-developer-tools-sell-to-engineers <comment><USER5> I find that the key thing to overcome when selling to developers is the extreme reluctance to spend money buying something that they can build themselves. You see,

1.) Developers love to build things.

2.) Developers hate spending money.

3.) Developers undervalue their time.

If your product looks like it would have been fun to build, you'll lose the entire "insufficiently supervised developer" demographic. Those guys will happily spend tens of thousands of dollars of billable hours implementing an in-house version of your thing to avoid the possibility of outgrowing your Free Tier.

I've seen this play out with S3stat customers (which costs $10/month, or three minutes twenty seconds of fully loaded engineer cost), where somebody will spend a week building an in-house version of the service and standing up a server to run it. Nicely done. You'll break even on your investment in 21 years.

I've had moderate success with my latest API product pointing to a "Boss Page" that outlines things like build-vs-buy costs, and why you really would be better off paying us for this thing rather than dedicating an in-house guy to building and maintaining it.

It's a tough one.

<comment><USER6> This simplified view misses some critical points:

\- The product won't do things exactly the way you want. If you build it yourself, you get it the way that you wish it had been built.

\- You're investing in someone else's product. Everything you invest in training and customization is wasted if you have to move to something else. You no longer have to worry about tech support either.

\- You're locking yourself into someone else's product, and you're stuck if suddenly there's a big increase in price, the product shuts down (very common), or changes in some way (even more common).

\- The "developer cost" is tricky. Salespeople like to pretend that you can calculate that using someone's salary. If you have an $80/hour developer spending five hours on something like this, you don't literally write a check for $400. Instead, that developer might have been doing something useful, but might also have been in a meeting or responding to email or just sitting around. One argument for 20% time is that the 20% spent on that work wouldn't have been used productively anyway.

<comment><USER7> > \- The "developer cost" is tricky. Salespeople like to pretend that you can > calculate that using someone's salary. If you have an $80/hour developer > spending five hours on something like this, you don't literally write a > check for $400. Instead, that developer might have been doing something > useful, but might also have been in a meeting or responding to email or just > sitting around. One argument for 20% time is that the 20% spent on that work > wouldn't have been used productively anyway.

This is actually a lower bound on the opportunity cost of having your developer work on purchasable items.

\- If the dev is making $80/hr they should be adding more value to the organization than $80/hr.

\- If 20% of your devs are working on purchasable items you'll need to hire another ~20% to replace them, but then you'll need 20% more managers, QA, etc..

\- Developer productivity isn't linear, if you hire twice as many devs you don't get twice as much code written.

<comment><USER8> > if you hire twice as many devs you don't get twice as much code written.

or if you do, that could be a bad thing ;P

</comment></comment></comment></comment><comment><USER9> I would also recommend dev tools founders (really any founder) read GitLab's company handbook at [https://about.gitlab.com/handbook/](https://about.gitlab.com/handbook/), especially the sales and marketing sections. It's awesome how public they are about how they operate. You'll learn a lot.

We (Sourcegraph) have started our own company handbook ([https://about.sourcegraph.com/handbook](https://about.sourcegraph.com/handbook)), inspired by GitLab. So far it has helped new teammates onboard more quickly and (along with other efforts to diligently document internal practices) helped us work efficiently on a growing team spanning many timezones.

</comment><comment><USER10> Call me cynical, but this article looks like one of those ghost-written SEO articles for self-promotion and improving search result position.

Take a look at DigitalOcean's articles and compare to this one: night and day.

<comment><USER11> Sure looks like it. Part one, two and three are written by three different "Guest authors", who are freelance technical writers.

</comment></comment><comment><USER12> I've noticed that a lot of paid dev tools are just priced too high or that I run the risk of blowing my budget depending on scale. I wish more paid dev tools were aware that I don't actually know my usage up-front, and that I'm scared of potentially ending up with a ridiculous bill if I overlook something.

I totally get that it's on me, but things [like this]([https://hackernoon.com/how-we-spent-30k-usd-in-firebase- in-l...](https://hackernoon.com/how-we-spent-30k-usd-in-firebase-in-less- than-72-hours-307490bd24d)) happen, and I'm way more apprehensive of services that make that a possibility. E.g. I'd love to use Auth0 for everything, but the risk of spending way too much on something I can roll myself is always on my mind.

<comment> jon-wood This is something I'm hitting up against a lot at work currently. Our application is high volume, with a significant proportion of users who don't bring in any recurring revenue, which makes us very cautious about picking up new SaaS services which are billed on a per-user basis.

We went through the same process around Auth0, and eventually built our own solution for handling authentication because even with our current user volume all Auth0 would tell us on pricing was "contact sales", which tends to be the case with every product we look at these days.

<comment><USER13> Auth0 quoted us 25 - 50k a year for 7K MAU, we found this insanely high as we don't handle any sensitive data.

It seems we might use Amazon Cognito, which seems a bit rough / neglected when it comes to aws products but the difference is basically free up to 50K MAU.

Couldn't find a nice middle ground, Okta and the others were all prohibitively expensive as well.

<comment><USER14> If you don't mind sharing, what was it from Auth0's "call us" column for pricing that bumped you up to that tier? Middle one is still high, but under $25k, let alone $50k, for 7k MAU.

</comment></comment></comment></comment><comment><USER15> As someone who first sold to devs and then switched, many years ago, to the largest end user market that we originally served (indirectly), if I did it all again I wouldn't bother with the first part.

It slightly depends where your price point is, but the issue is there are plenty of good devs and plenty of really bad devs. The bad ones pay you the annual maintenance and transform into help vampires. The problem is the issues are technical and complex and it sucks up developer time supporting them.

It happens in the end user market, sure, but it's easier to find non-technical first line support to clear out the noise.

</comment><comment><USER16> This is a promotion of manifold itself. Put it under "Show HN".

</comment><comment><USER17> There also seem to be a 2nd and 3rd part to this blog post.

\- Part 2: [https://manifold.co/blog/founders-guide-developer-tools- mark...](https://manifold.co/blog/founders-guide-developer-tools-marketing)

\- Part 3: [https://manifold.co/blog/founders-guide-developer-tools- go-t...](https://manifold.co/blog/founders-guide-developer-tools-go-to-market)

</comment><comment><USER18> > Docs should be detailed, easy to read, and packed with examples.

But aside from all the examples and tutorials, please don't forget about a proper manual and full API docs. I've seen too many different tools and tech that have spent a lot of money on writing up the happy path, but then completely left me on my own when I encountered some weird error code.

Just follow the structure of the whole API you have, and make sure that each member, each function argument and each possible return value is documented. It's not as glamorous as "one button integration" \- but that one button integration usually turns into nightmare when you want to wire it up to your automatic builds anyway.

</comment><comment><USER19> Developers hate to buy stuffs. thats why they build stuffs. Few weeks your tool is out in market, some outraged developer is going to release a free version.

<comment><USER20> This is very true, that's what always puts me off when "new exciting idea" about developers-related tool or service comes to my mind. Additionally developers themselves are in their own niche so even if at first sight your brand new product seems to be innovative and non-existing on the market then it's very likely that it already exists in some form.

<comment><USER8> SaaS businesses succeed based on the service part, not the software part.

</comment></comment></comment><comment><USER21> Or sell by telling how to sell ... Not bad

</comment><comment><USER22> Here's my opinion without any credentials attached:

* Free tier - <USER3>

* Documentation - <USER3>

* Stack overflow tag - <USER4>

<comment><USER23> I would love to hear your thoughts about adding a free tier for my PDF filling service [1]. What's your first impression about the current pricing?

I experimented with a "pay as you go" plan that starts at $0 and includes some free PDFs, but so far I've earned close $0 from all of these customers. But this might just be because the people who asked about alternative pricing are much more price-sensitive and early-stage, while bigger companies don't think twice about $49/mo.

[1] [http://docspring.com](http://docspring.com)

<comment><USER24> I wasn't able to find the free tier plan on your site. Free tier would mean: 2 PDF templates, 100 API requests/month or something similar. All I could find was a 2 weeks trial. Maybe consider adding a true "pay as you go" tier with a small amount of money/PDF template and, something perfect for startups and don't start with a $49 tier.

<comment><USER23> Sorry yes I meant I don't have a free tier now, and have been thinking about adding one. Thanks for the suggestions! I've also been thinking about adding a "pay as you go" credit system similar to Twilio, but I'm not 100% sure if it would be a good business model.

</comment></comment></comment></comment><comment><USER25> Most importantly, polish the tool.

I'm not going to get my boss to pay for something that is made out of duct tape. If you want to sell something to people who probably know how to build it themselves, you need to impress them with something shiny. While it should work perfectly most of the time, it should also come with helpful error messages when it doesn't like the input. If I come across some cryptic, ungooglable internal error or access violation or assertion where I'd have to contact support to even understand what's wrong, I'd rather delete your tool immediately.

</comment><comment><USER26> A Founders guide to generalising your customers.

«If you want to sell something make features the customer wants to buy and tell them about it»

</comment><comment><USER27> Does the free-tier/trial really worth it? I've read that from business perspective those are a waste of time.

</comment><comment> foreign-inc There are a few rules:

\- Don't be a solo founder, no body is going to fund you. Your idea may be crappy, you can pivot. But solo founder is a big no no, unless you are successful founder with a new venture.

\- Either you or your co-founder is "the" person in your product area or you are able to hire someone who is. Developer tools companies always try to do this to become #1 (at least in twitter).

\- You probably should make a new database or a security tool if you are in the B2B space. Otherwise you will have a very hard time making money.

<comment><USER28> Could you explain why do you think db or security tools are profitable only?

<comment><USER29> The security market is predicted to grow substantially.

This gives space to new entrants.</comment></comment></comment>


Show HN: Harmony of the Spheres – Gravity Simulator - <AUTHOR> https://thehappykoala.github.io/Harmony-of-the-Spheres/ <comment><AUTHOR> I just realized that I'm yet to properly introduce myself to you folks; name's Darrell (the Happy Koala - I love koalas the way other people seem to love cats!). I'm a self taught developer and the project this thread is concerned with is actually the reason, pretty much, I started dabbling with code. If you want more of a background, you can check out an article I wrote over at Medium about the evolution of this project and how it led to me becoming a web developer - [https://email@example.com/how-an- interest-i...](https://email@example.com/how-an-interest-in- physics-led-me-to-a-new-career-in-web-development-f863ea118099) (please bear with any spelling or grammatical mistakes as English is not my first language). I'm still very much of a junior empty stack developer, but I love coding and intend to continue doing so, so hopefully we'll have lots of interesting discussions here on Hacker News in the future! Having said that, have a nice start to the week!

<comment><USER3> (I used to simulate classical mechanics for a living, so...)

It's pretty good except it runs a lot slower than it has to.

The default step size is too small for most cases so you have to wait a long time for the planets to move around.

When I tried increasing the step size with the slider I'd inevitably increase it too much and the planets would fly away.

Personally I always use RK4 and similar algorithms with adaptive step size, mostly because it is a pain to tune the step size by hand

[https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods#Ad...](https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods#Adaptive_Runge%E2%80%93Kutta_methods)

also if planets are passing close to each other you may need a shorter step when you are close and a longer step when you are far away.

RK4 or RK5 with adaptive timesteps will make your demo much better with very little work.

There are methods that use higher order derivatives and other tricks to take large steps, for instance I like this method

[https://github.com/alexrudy/bsint](https://github.com/alexrudy/bsint)

If you were integrating the solar system for millions of years you would want a symplectic integrator.

[https://en.wikipedia.org/wiki/Symplectic_integrator](https://en.wikipedia.org/wiki/Symplectic_integrator)

this might not be more accurate than another integrator but it will preserve phase space area which makes it possible that the long-term dynamics will "look like" the real long-term dynamics. (Otherwise it certainly won't!)

If it were me in cases where there is a central large mass I would integrate in spherical or cylindrical coordinates, probably I would use a predictor- corrector method.

<comment><USER4> Any thoughts on Leapfrog integration? I've got a super basic n-body simulator that I'm using for gravity and for coulomb's law. I also need to advance to adaptive time steps.

[https://landgreen.github.io/physics/notes/electromagnetism/p...](https://landgreen.github.io/physics/notes/electromagnetism/potential/index.html#three- potential-sims)

<comment><AUTHOR> Can't stop myself from staring at those modulating peaks; hypnotizing stuff! Leapfrog integration, as foxes pointed out, is not symplectic if you apply an adaptive time step. I'm not that concerned about the integrator being symplectic as it's not my intention to run simulations that last for millions of years. If it's possible to add comments to a thread here on hacker news (new to this site!) indefinitely, I'll give you a heads up when I've implemented an integrator with an adaptive time step, and in case you get to it before I do, do share :D!

</comment></comment></comment></comment><comment><USER5> Any idea why Juno appears to be pre-programmed to start with a pretty huge radial outward velocity?

<comment><AUTHOR> Hi Pavel!

Every body in the simulation start out with a set of initial position and velocity vectors. For the solar system scenarios, I got these vectors from NASA JPL's Horizon System ([https://ssd.jpl.nasa.gov/horizons.cgi#top](https://ssd.jpl.nasa.gov/horizons.cgi#top)), so the orbit of Juno that you are observing is its actual orbit... Its eccentricity (the extent to which the shape of the orbit departs from a perfect circle) is extremely high; at its closest point, the orbit brings Juno right above the cloud tops of Jupiter, and at its furthest point Juno finds itself beyond Themisto (one of Jupiter's irregular Moons which I should perhaps add to the scenario). Juno was meant to have a less eccentric orbit, but as technical problems were encountered when they were going to fire Juno's thrusters to reduce the eccentricity of the orbit, NASA played it safe and let Juno stay on its eccentric orbit. One of the advantages of this is that Juno only spends a very short amount of time in Jupiter's insanely powerful and to a spacecraft damaging magnetic field, so the technical mishap is probably one of the reasons JunoCam is still operating and giving us beautiful images of Jupiter's swirling clouds! Here's a gif from WikiPedia showing how stretched out Juno's orbit is [https://en.wikipedia.org/wiki/Juno_(spacecraft)#/media/File:...](https://en.wikipedia.org/wiki/Juno_\(spacecraft\)#/media/File:Animation_of_Juno_trajectory_around_Jupiter.gif). Sorry for the long rant, but I find these things to be awesomely fascinating.

<comment><USER5> > _Sorry for the long rant_

You've got nothing to apologize for, that was fascinating! I knew its orbit had some eccentricity, but I didn't know it was _that_ much!

</comment></comment></comment><comment><USER6> I would love if kerbal, simplerockets and the like used this realistic gravity simulation. The piecewise conic approximation is great, but after a few weeks of playing it gets very tiresome and too obviously limited...

<comment><USER7> Have you looked into Orbiter 2016? [http://orbit.medphys.ucl.ac.uk](http://orbit.medphys.ucl.ac.uk) -

I recently learned about it and wanted to try it after playing with KSP, but it seems it's Windows only.

<comment><USER8> Bach's Air is now permanently ingrained into my brain, paired with the soothing rumbling of booster rockets.

</comment></comment></comment><comment><USER9> For long simulations I'd like to increase the timestep, but then RK4 becomes less accurate. For the n body problem, do symplectic integrators work well?

<comment><USER10> Yes, the difference is enormous. Use e.g Verlet integration and it's much more stable. That it's also much simpler and more elegant RK4 is an added bonus.

<comment><AUTHOR> If for whatever reason you have time and would like to make a contribution, I think doing so should be quite easy as I think you'd just have to subclass the Euler class. You can get an idea of how to do this by looking at how I did it with the RK4 subclass ([https://github.com/TheHappyKoala/Harmony-of-the- Spheres/tree...](https://github.com/TheHappyKoala/Harmony-of-the- Spheres/tree/master/src/js/Physics/Integrators)). If not, I'll add Verlet integration myself one of these days, but right now I'm working on introducing a particle system so that I can simulate the rings of Saturn (the particles will feel the gravity of all the masses in the simulation, but they won't exert a gravitational force on the other particles and masses in the simulation; this way you should be able to have fun distorting the rings of Saturn with a celestial object of your choice).

</comment></comment></comment><comment><USER11> This actually works suprisingly well on my phone! Well done, love being able to modify masses mid-scenario and see the whole system disintegrate.

<comment><AUTHOR> Thankie!

</comment></comment><comment><USER12> I've just spent some time on this. The Oumuamua scenario is superb - <USER2> alone is worth every penny.

Thank you.

</comment><comment><USER13> Couldn't try this as I am phone. But I do find gravity simulators oddly satisfying.

<comment><AUTHOR> My CSS skills aren't what they should be, but I'm going to try to make a mobile friendly version of the simulator at some point once I know more about flexboxes, media queries and what have you.</comment></comment>


Show HN: Email keyword alerts from podcasts - <AUTHOR> https://www.listennotes.com/alerts/ <comment><USER2> This looks really cool. You put a lot of effort into it, and it's noticable.</comment>


Structure and Interpretation of Computer Programs (lectures) - <AUTHOR> http://inst.eecs.berkeley.edu/~cs61a/su10/

<comment><USER2> There are some webcasts for these available too.

[http://webcast.berkeley.edu/course_details_new.php?seriesid=...](http://webcast.berkeley.edu/course_details_new.php?seriesid=2011-B-26281&semesterid=2011-B)

Don't know if they run in parallel completely, but could be handy.

</comment><comment><USER3> SICP is such an incredibly clear book, I've never really understood the need for the lectures - but then again, different people have different learning styles. I guess some people prefer to sit through X hours of lectures than to read through Y hours of a book...

<comment><USER4> It would've been funny if the authors had agreed with you about lectures being unnecessary and had never taught the course they wrote the book for!

</comment></comment><comment><USER5> These lectures were geared more toward budding computer scientists, whereas I feel that SICP was geared more towards experienced programmers. Hope they reach that target audience :).</comment>


Why Haskell Is Worth Learning - <AUTHOR> https://spin.atomicobject.com/2013/03/05/why-haskell/ <comment><USER2> It is worth learning, but the chances of using it for more than toy scripts are slim. Unless your workplace has already bought into it, you'll find it a hard sell. There are several other functional languages out there with more vibrant communities that have useful libraries.

One selling point, the terseness/simplicity of code, can be a specious one. Beyond the magic of map/flatmap/etc, the things that allow you to achieve terseness in a general sense often require more cryptic constructs as scaffolding.

One developer friend made a comment that summarizes functional languages well in practice: "They're very useful, but try to avoid code that requires your coworkers to know abstract algebra and category theory in order to contribute to a project."

<comment><USER3> > They're very useful, but try to avoid code that requires your coworkers to > know abstract algebra and category theory in order to contribute to a > project.

Have you ever learned a 'design pattern' in Java, Python, C++, etc? What's the difference between learning this and learning about monoids (for example). Monoids are super easy and actually compose, unlike the majority of design patterns. In my experience, learning OO design patterns was substantially more difficult than learning algebra, but I think this is mainly due to how CS education is carried out in many universities. I feel if people start with mainly FP, then OOP is what seems odd and necessitating learning odd, abstract concepts, except there's the added bonus that all the abstractions are leaky.

Ultimately though, you don't even need to learn any algebra or category theory. It was multiple years after I had already learned and started using Haskell that I even began to become interested in these topics. My interest was mainly piqued by a desire to not duplicate code -- it certainly isn't required to learn algebra to understand Haskell, in the same way it's not required to understand machine assembly to program in C++, or GPU transistor layout to understand OpenGL.

> a general sense often require more cryptic constructs as scaffolding.

My professional experience is mainly as a Haskell dev. What is 'scaffolding'. I don't think I've ever used such a thing.

<comment><USER4> Design patterns are usually taught by describing several example problems, and then introducing the design pattern as something to solve all of them. The value of the abstraction is obvious. Haskell is more often taught as though flexible abstraction is an end in itself, and practical uses for it are somehow distasteful. This teaching style only appeals to people who think mathematics is "elegant" or "beautiful" instead of just a useful tool.

<comment><USER3> Perhaps you're reading the wrong documentation. I've personally never found it that way. Most documentation on hackage is very concrete, and for the ones that aren't (i.e., lens), there are plenty of tutorials.

I've definitely encountered the abstraction-for-the-sake-of-abstraction types, but those are mainly academics, in my experience.

I mean, monads are (for some reason or the other) the killer feature of Haskell abstraction, and their use case is extremely concrete. map, fold, etc are also motivated by very concrete desires.

Even more advanced concepts, like zippers, generics, lenses, ftpe, etc are all motivated by _extremely_ concrete problems.

</comment></comment></comment></comment><comment><USER5> So what's a good set of examples or exercises that will show an experienced coder some new ways of thinking?

<comment> m-j-fox [https://github.com/data61/fp-course](https://github.com/data61/fp-course)

</comment></comment><comment><USER6> The author uses "plays well with C" to mean C source code.

I tried supporting haskell from within postgres as a new language for UDFs. I ran into many problems, but perhaps those were more with GHC than haskell itself.

</comment><comment><USER6> Haskell would be great as a data language. With some good relational libraries it could be an interesting alternative to SQL.

</comment><comment><USER7> > Not including the type signatures, that's just 4 lines of Haskell!

Yeah + years of learning it, but that's not showing up as code. I'm glad he can express that in 4 lines. I'm still not convinced this is a good investment of time. I use functional style in JavaScript, but that's probably the farthest I'm willing to take it at the moment.

> "Do you want to be locked into tricycles because they're easy to learn?"

That would be great ending quote if the next step he suggest wouldn't be a Penny-farthing.</comment>