State of Self-Serve Website Building in 2020 - <AUTHOR> https://sprune.com/blog/state-of-self-serve-website-building-in-2020/ <comment><USER2> I really like forestry.io, which lets you choose the static site generator system and then host the generated static site on Github Pages or other static hosters.

It's basically the missing glue for the manual operations you would normally do on your own computer if you went the Jekyll/Hugo static site generator way. But you don't need to maintain a development environment on your machine.

After the initial setup, a non-developper can use forestry as easily as Wordpress.

</comment><comment><USER3> As someone who's been building websites for 15 years, I can't say enough good things about this trend. I work primarily with early stage startups, often pre-product, building prototypes and V1s. The primary objective of the teams I work on is to validate product strategy, which requires moving quickly to de- risk initial development and iterate.

The past few years has completely changed the way I execute this strategy. Leaning heavily on products like these that allow us to develop software with very little or sometimes no code at all. It's nuts what you can build in a few days.

Discovering Rails in 2009 was the last big shift for me in this way, with tools like scaffolding to build out the skeleton of any dynamic CRUD section in a few seconds. Still, it wasn't uncommon to subsequently spend days or weeks getting specific sections of the site to reach parity with designs.

Some people talk about how this trend is a threat to web developers like me. While it's true that it's never been easier for anyone to build high-quality dynamic websites, that position completely overlooks the benefits for traditional web developers. This new toolchain is like having superpowers for a very specific type of development. Pairing that with solid knowledge of web fundamentals and the ability to "grow up" beyond the limitations of the system is a recipe for delivering work faster than ever before. For most early-stage startups, fast & cheap is #1, but if you can deliver quality on top of that, you're golden.

</comment><comment><USER4> Thanks for mentioning [https://versoly.com/](https://versoly.com/)

One of our coolest features especially for the HN crowd is a code editor built into the browser that updates the website in real time.

Something that I felt was missing from all the other website builders at the time and still is.

</comment><comment><USER5> I'm currently using a Jekyll based theme to host my blog via GitHub pages. I'll most likely switch to static-site generator using zola [0]. I don't much know about web development, but fine with markdown based site generation. Was struggling with zola docs, until I found that I can simply clone themes [1] and replace the sample posts. I'll go through settings later. Best thing I like about zola is that I get a single binary (this is based on rust) and I don't have to fiddle with npm, etc.

[0] [https://github.com/getzola/zola](https://github.com/getzola/zola)

[1] [https://www.getzola.org/themes/](https://www.getzola.org/themes/)

<comment><USER6> I had a quick look at Zola sometime ago while looking for alternatives to Hugo (which is written in Go, but is also distributed as a single binary). But it seemed like Zola had fewer themes available. As a non-designer, having great themes available (for free, preferably) would be a good starting point.

<comment><USER5> I wondered why you say _fewer_ themes for Zola, there were so many to choose from and I had a hard time to decide (and I might still change it in future).

I just checked for Hugo, and it certainly has so many more compared to Zola. Something to keep in mind for future, but I am more than satisfied with Zola for now.

</comment></comment></comment><comment><USER7> I've recently jumped on the static bandwagon after being an early adopter and user of WordPress since CafeLog/0.71 days. What a blessing it has been.

I've taken two different approaches: 1\. Maintain a local WordPress instance that builds the site using all the theme/plugin magic that WordPress is famous for, but have a static-export of that into Github -> Netlify.

2\. Small sites being powered by Cloudflare Workers - using pure JS.

The jump to React.js / Vue / Go / Ruby et.al for an old PHP/HTML/CSS hack like me was too hard (and I tried them all!), so I settle for WordPress (the devil I know) for the build/back-end, and incorporate and embrace static on the front end. This separation is key - in my mind - for speed and security.

<comment><USER8> How do you handle search?

<comment><USER7> Search and comments are the casualties of static-sites - unless you bow and crumble to the use of something 3rd party (Disqus or Google Site Search to name some poor examples)

<comment><USER9> Can't you point algolia at a static site and get decent search or have I misunderstood?

<comment><USER10> You can check out Lunr. The Jekyll theme Minimal Mistakes has that as a search option and builds a JSON index file.

The demo site uses Algolia though [https://mmistakes.github.io/minimal- mistakes/](https://mmistakes.github.io/minimal-mistakes/)

</comment></comment></comment></comment></comment><comment><USER11> We built a website for a film festival using a static site generator, deploying to AWS S3 + CloudFront Content Delivery Network. A few months later, the client asked about shutting it down to save money. Turns out it cost $0.05 for the month it was active, and $0.01 in the previous month.

<comment><USER12> How much traffic did it handle for that $0.05?

<comment><USER13> Static, with S3 below and Clouflare on top? Tons...

<comment><USER14> CloudFront != Cloudflare

<comment><USER15> For almost everybody it is == just not ===

</comment></comment></comment></comment></comment><comment><USER16> Often I show people how to write simple HTML and CSS sites. They are usually just making simple single page sites but they are extremely satisfied by being able to do it themselves.

What is the simplest way to get these single file HTML sites hosted? Preferrably without having to teach another tool or language?

<comment><USER17> Put it in an AWS S3 bucket.

use aws command line tools to sync to your local repo

easy way:

aws s3 copy index.html s3://<mybucketname.com> \--profile <awskeys for this client>

hard way:

cd <mywebsiteroot dir>

git pull ## if needed

<website builder> ## i.e. webpack, angular, whatever

aws s3 sync public_html s3://<mybucketname.com>/ \--profile <awskeys for this client>

<comment><USER18> The problem with beginners and S3, is that they can't handle the billing side. Both calculating (outbound? same region? edge location?) and monitoring if things go south. Beginners will prefer the site going down rather than paying extra.

<comment><USER19> Yeah, I use S3 myself for a static website (as well as podcast hosting and misc other stuff). But the lack of hard cost circuit breakers make me hesitant to recommend AWS in general to beginners for casual storage and hosting.

</comment></comment></comment></comment><comment><USER20> Odd not to mention services like Wix and Weebly and Squarespace, which is where most non-techie people wanting to self build a website would end up. Perhaps this post is aimed at the more technical end.

edit: Oh right self-serve means self-host, not just self-build.

<comment><USER21> Squarespace et. al. are not self-serve.

<comment><USER22> if you mean self-host by that, then the author has also mention webflow which is not self-host afaik

<comment><USER23> IIRC it provides a download (or at least it did a few years ago when I used it) for selfhosting if you just generate static HTML.

</comment></comment></comment></comment><comment><USER24> I've recently discovered Publii ([https://getpublii.com/](https://getpublii.com/)).

The single big plus for me is, that it allows me (as a DEV) to provide the typical non-DEV user (aka my mother) with a simple, non markdown user interface to edit and update the page on her own.

<comment><USER21> Publii almost gets it right. There's still the issue of getting hosting, setting that up, managing it etc for non-tech people. Your use case is great, though.

<comment><USER25> Hosting a static website is not that difficult. The quickest way is to upload the zip file generated by Publii on the Netlify "drop" page. The domain configuration is more challenging than that.

</comment></comment></comment><comment><USER26> Site.js fits the use case really well. Designed for simplicity, based on small technology principles by Aral Balkan.

[https://sitejs.org/](https://sitejs.org/)

</comment><comment><USER27> I will be honest, I didn't think Wordpress would last this long. However, there are some great (free) alternatives to getting a website up and running. Google Sites is one not mentioned in the article. It can handle basic needs for most people without technical skills. Speed is not the greatest however. With something like Tailwind CSS it's very easy to create a static webpage to host on Github/Netlify.

<comment><USER28> _" I didn't think Wordpress would last this long"_

Same here.

Funny thing is, just a week ago I read that some people would say WP is "no code". I had the impression almost all of WP's success was based on countless web agencies coding customizations for it.

<comment><USER29> Page builders have come a long way, a lot of agencies will use something like Elementor now because designers can effectively build a lot of pages in real time as opposed to doing the traditional psd->custom theme route.

</comment></comment></comment><comment><USER30> I've been rebuilding a couple of sites using [https://11ty.dev](https://11ty.dev) lately and it's by far the best static site generator I've ever used, and I've used most of them. Everything I've tried to do has worked pretty much first time in exactly the way I'd have expected.

<comment><USER31> Been enjoying 11ty very much as well as of lately. Most of the things have worked for me first time around as well. One thing which has tripped me up has been the flexibility of the different template engines, and in what order files get processed during build when one mixes markdown, njk, liquid etc. That section is worth reading carefully in the manual. Other than that, simple as heck with very few constraints.

</comment></comment><comment><USER32> We (Almonit) focus on Dwebsite: websites that connect decentralized storage with decentralized name service. Works great at the moment without any server or 3rd party.

[http://blog.almonit.eth.link/2020-05-21/Introduction_to_Dweb...](http://blog.almonit.eth.link/2020-05-21/Introduction_to_Dwebsitse.html)

</comment><comment><USER33> Apple should bring back iWeb. It was an easy way for non-technical people to design a web page as if they were creating a document in Pages. Sometimes all you need is a simple static page for yourself or for your small business.

<comment><USER34> Microsoft FrontPage!

<comment><USER21> I've been thinking about creating a spiritual successor to Frontpage and Dreamweaver for years. A good ol desktop program built for easy and fast publishing and app prototyping.

<comment><USER35> I feel like we're at a point where a modern frontpage would work. Make it cross platform (including maybe a simple mobile version) and have it push static sites to Github Pages (with an option for other hosts). Integrate some domain purchasing and setup with info@domain redirecting to an email address of choice.

Squarespace but with an actual layout editor like Word.

<comment><USER33> Lets do this. Maybe this comment thread will spark the creation of an actual app. How do we get in touch?

</comment></comment></comment></comment></comment><comment><USER36> Good article. Pity that it doesn't mention RocketCake in the list of website builders, but apart from that it gives a very good overview.

Ironic that we are moving back to static websites, but I think that's actually a good thing.

<comment><USER37> Why should RocketCake be included in the list?

</comment></comment><comment><USER38> On that topic, I am all in favor of static sites hosted on github. But the static site generators, the way I see it, weren't self-contained. That is, you still needed to go through a build process. And after a lot of poking around and digging about, I built(and subsequently open sourced) this[1]. Kind of a 1 hour coding-session kind of thing but for anyone out there looking for something entirely self-contained and stupid simple, there you are...

[1] [https://rorigami.site/](https://rorigami.site/)

</comment><comment><USER39> I really wanted to like Ghost but it seems out of touch. Like the only social media options by default are Twitter/Facebook, the plugin system is just zapier connections, themes are limited, and when I tried to self host because the official ghost hosting is expensive I only ran into bugs with it. After spending a week with Ghost I moved to Wordpress. I wish I could use the themes on Ghost with Wordpress because all of the ghost themes are what I want for my site. I want the nice parts of Ghost with a Wordpress backend.

<comment><USER40> WordPress has an official REST API. It's pretty easy to rip post/page content out of it either on a schedule (cron) or dynamically, in either case rendering finished pages from templates or similar, and serve it without involving WP directly. I'm not too familiar with static site generators but I'd expect any that can take raw HTML for post/page bodies to be trivial to automate to serve such a pipeline.

[https://developer.wordpress.org/rest- api/](https://developer.wordpress.org/rest-api/)

<comment><USER25> Sure, pretty easy for developers, not for the general user.

</comment></comment></comment><comment><USER41> A question for people that have experience with both Hugo and 11ty: What are the main differences and which one do you like the best (and why)?

<comment><USER42> They're battery-included in different ways. Hugo is fairly opinionated, while 11ty seems to be a bit more extendable with JavaScript.

</comment></comment><comment><USER43> If you are using any tool that stores its content in JSON, you can use [https://frontaid.io/](https://frontaid.io/) for that. FrontAid is basically a CMS for JSON files. And based on that JSON content, you can then generate a static page or use it for whatever you want.

</comment><comment><USER44> As a non-front end guy I thought nextjs would be an easy way to create a good site. Do not go this route!

</comment><comment><USER45> Great article about summarizing building a website.

I have recently switch from Wordpress to Hugo with Gitlab and I never complain about speed and performance anymore.

11ty is gaining more attention and it looks like a good choice for the one considering static site.

</comment><comment><USER46> This isn't a solution for non-tech people, but I've really enjoyed making a blog that sort of sits in between database-backed and Jekyl/Hugo. Got the idea from Jose Valim's dashbit blog. Basically a folder of markdown files that gets parsed into memory on startup and then served dynamically from there.

A Go app that doesn't need to hit a database has response times nearly as fast as static, and you can still do things like including related content ("more like this") at the bottom of a post, or pagination/filtering on the index page.

<comment><USER47> The same can be said for vanilla WordPress with modern PHP. I generally see sub-100ms responses, even better with fpm caching built into Nginx.

</comment></comment><comment><USER48> Re: 'Headless CMS' in the article: Since the introduction of the JSON REST API, I've been using Wordpress as the back-end to various personal and professional projects, built on various web and native mobile frameworks.

One of the good things about Wordpress is the familiarity to it from the people who have to use it. (Authors/Editors/Contributors). (Though, I'd say Gutenburg might stir that up a bit).

Also, once you learn how to write plugins, the sky's the limit really.

</comment><comment> cbm-vic-20 A bit of a side-question for those using static site generators: how do you manage binary assets like images? I keep my Markdown, configuration files, etc. in git, but I don't want to put binary blobs into git. git-LFS seems to be an option, but it doesn't look like there's much support for self-hosting. For now, I'm rsyncing assets to the right place as part of the generation process, but There's Got To Be A Better Way.

<comment><USER49> Why not put them in git? Works just fine

<comment><USER50> Git is not exactly well suited to having large blobs. If you change a single bit and git doesn't know how to diff it, you'll get a whole new version of the file, balooning repo size.

</comment></comment></comment><comment><USER51> I wish for a simplified, security-focused fork of Wordpress.

<comment><USER52> Maybe a longshot for your case, but Ghost[1] was developed as a simplified alternative to Wordpress. Their feature list[2] bring up several security- focused design choices.

The original idea of Ghost was made by John O'Nolan, then a lead dev at Wordpress Foundation.

[1] [https://ghost.org](https://ghost.org) [2] [https://ghost.org/feature- index/](https://ghost.org/feature-index/)

</comment></comment><comment><USER53> Considering the title of the article and that it's only talking about self- hosted(-ish) solutions it feels very weird that it only mentions one CMS. I get that it says "I'll review my top choice to keep things simple", but CMS'es are probably the most common solution to what the article talks about and the title implies that it will be comprehensive.

</comment><comment><USER54> I am currently using Hexo to generate my blog, then post processing (optimization) using gulp, and using CircleCI to build on push and publish to GitHub pages for me. Contents are just several Markdown files.

Whole setup: [https://github.com/maple3142/Blog](https://github.com/maple3142/Blog)

</comment><comment><USER55> Is there any static website builders which make it easy to execute code at build time?

They all have a templating language, and sometimes I just wanna write a quick function to do something like read a file, chop something out and wrap it in some html -- something that would take 8 lines of Python.

<comment><USER56> Probably mentioned earlier but I'd recommend Gatsby if you're just looking for a simple static site.

Though, we switched over to Next recently to take advantage of SSR.

<comment><USER55> I'll have a look -- I've previously tried Hugo, pelican and Jekyll and while pelican and Jekyll provide extensions, they just seem so heavyweight to write.

</comment></comment></comment><comment><USER57> I still miss CityDesk from Fogcreek, it was spot on for my requirements.

[https://www.joelonsoftware.com/2001/10/12/what-does- citydesk...](https://www.joelonsoftware.com/2001/10/12/what-does-citydesk-do/)

</comment><comment><USER58> I am working on www.pagews.com, it's a self service alternative, the static sites are based on the content of a spreadsheet/api and deployed to a CDN.

The site pagews.com is generated using the service for example (including capture forms back to the sheets).

</comment><comment><USER59> > _You can self-host it on a small server (one-click deployment on a $5 VM on > Vultr or DigitalOcean will do)_

Having hosted a ghost blog before, this is the wrong way to think about it. You'd need to run a separate instance of a database which costs $15 on Lightsail with periodic backups.

Running the database instance on the same $5 machine is one disaster away from a lot of pain. It is so much better to use github pages or static page generators over Netlify, since a database really seems like an overkill for simpler blogs.

Someone wiser than me once said [0]: _Why are you doing all this? Your entire database fits in RAM_.

[0] [https://twitter.com/apenwarr/status/1095441483065380867](https://twitter.com/apenwarr/status/1095441483065380867)

<comment><USER52> As a Ghost self-hoster, I can confirm I don't require the use of any database for "$15 on Lightsail with periodic backups".

Depending on your requirement you can use any instance of MySQL/MariaDB - managed or self-managed. You could run a separate $5 MariaDB DigitalOcean droplet. Ghost Foundation provides their 1-click Ghost droplet bundled with MySQL, but you're not restricted to it.

And yes, SQLite is also an option.

Suggesting a "need" to run fault-tolerant database operations doesn't quite concile with running it in RAM only.

<comment><USER59> > _I can confirm I don 't require the use of any database for "$15 on > Lightsail with periodic backups"._

Yep and then you'd be subject to what they call [https://en.wikipedia.org/wiki/Single_point_of_failure](https://en.wikipedia.org/wiki/Single_point_of_failure).

> _Suggesting a "need" to run fault-tolerant database operations doesn't quite > concile with running it in RAM only._

The wisdom is: You don't need a database at all to avoid SPOF. Use static pages and let Netlify and others cache them in their CDNs world-wide ("RAM").

> _Ghost Foundation provides their 1-click Ghost droplet bundled with MySQL._

And how much does that cost?

<comment><USER60> > Yep and then you'd be subject to what they call > [https://en.wikipedia.org/wiki/Single_point_of_failure](https://en.wikipedia.org/wiki/Single_point_of_failure).

It's a blog, not a mission critical site. I suspect MOST blogs get so few views that it really isn't a huge deal. If you have the need for more stability and availability, then paying extra isn't a huge deal.

<comment><USER59> > _If you have the need for more stability and availability, then paying extra > isn 't a huge deal._

There, that was my point: One could pay exactly somewhere in the region of $0 to Netlify and still have that.

<comment><USER60> I see you're one of those people that base everything on price and not on other things. Got it.

<comment><USER59> You'd do better to take arguments at their merits than resort to ad hominem attacks.

To your point: Yes, Ghost is much more than Hugo / Jekyll will ever be, but then again, most people aren't building a News site to actually want the features Ghost has to offer.

I mean, there's absolutely a market for software like Ghost and WordPress (or they wouldn't exist, especially in WordPress' case they absolutely dominate that market), but if you're the kind to run Ghost with SQLite then may be you should instead give static site generators a look.

</comment></comment></comment></comment></comment></comment></comment><comment><USER61> Please add Cloud CMS and Cloud Backend from [https://appdrag.com](https://appdrag.com) to the list. It's giving more power to developers compared to other tools

</comment><comment><USER62> There is also [https://www.jimdo.com/](https://www.jimdo.com/)

It offers a good experience for e-commerce and blogging.

</comment><comment><USER63> Given those tools, and especially Webflow, is Wordpress mostly on the way out for new sites, and only be used in maintenance mode ?

</comment><comment><USER64> Dude, you just made me cry. Intense experience. You got something good there on the guided and the naration experience.

</comment><comment><USER65> Nothing about security?

<comment><USER66> Most of the options are hosted or static, which are generally "secure enough" for the intended audience.

As soon as you're self hosting now you're doing server admin etc and that would be outside the scope of the article.</comment></comment>


Ask HN: CRM/project management system delivered to you as a Virtual Machine? - <AUTHOR>

Dear HN,
I'm doing http://www.discoursehq.com - real-time team collaboration system where you can manage deals, projects and more - and I can see a lot of customers have issue with buying hosted software from a startup. Basically it's all about trust (will you be there in 2 years from now?).
I'm thinking of selling our system as a SaaS AND as a downloadable, ready to use Virtual Machine (VMWare virtual machine) so people could host it themselves.
Wanted to check if someone here did that before? Any tech/support problems? Any advice on pricing here? <comment><AUTHOR> Clickable link: <http://www.discoursehq.com></comment>


Facebook is the Space Shuttle of Software - <AUTHOR> http://mcdowall.info/2012/7/5/facebook-is-the-space-shuttle-of-software

<comment><USER2> You mean something whose features were oversold, and laid the groundwork for whatever it is that will ultimately be remembered?</comment>


The pitfalls of allowing file uploads on your website - <AUTHOR> http://blog.detectify.com/post/86298380233/the-pitfalls-of-allowing-file-uploads-on-your-website <comment><USER2> The bottom line is this, if users can upload something to your site, and then your site will show that thing to other users before you have a chance to figure out if its a problem, then your site will be exploited by bad actors.

For a long time an out of the box server installation would include anonymous ftp access. Of course nothing is quite so attractive as a 'free' place to dump and retrieve stuff. It was kind of like setting up a warez/malware camera trap.

<comment><USER3> Even if it does not show anything to other users, just having the wrong extension can already bite you badly.

Uploading php files instead of images has been used to gain access to machines. Anything that gets stored as a file on the filesystem of the destination machine is a huge risk. All it takes is one little misconfiguration somewhere else and you're wide open.

<comment><USER4> This is only if you subsequently give them a link to what they uploaded, correct?

I have a site that allows uploads (students turning in Java files) but the files are just stored in a folder on the server that isn't in the web-served path. They can't see the file again once uploaded. I assume (and I think rightly) that there's no security risk in my case.

<comment><USER5> As long as you read the files before you execute them.

Otherwise some bad actor could write a virus / local exploit into their submission which will execute when you compile and run the file.

<comment><USER4> I _never_ execute them. I just grade them by reading the code. Running them takes FAR longer than reading.

<comment><USER5> Are these exceedingly simple programs 10 line programs? Otherwise:

How do you know they compile?

How do you know they work?

How do you know they handle all the edge cases you can throw at them.

If you have a 100% accurate parser and compiler in your head, I am impressed.

Our teachers (and this was 15 years ago) had test-runners which would compile and run our programs to make sure they met the requirements of the homework THEN they looked at the code and marked it for style etc.

Sometimes they provided these test runners to us so we could check them ourselves, sometimes they didn't (this was, naturally, harder).

<comment><USER6> Obviously such workflow, while being fairer, requires a reliable sandbox of some kind—even though one might argue that in a university such things may be of less importance and that allowing for some degree of hacking is educational and perhaps should even be tacitly encouraged, still you'd want to make sure that when students break your system they can't go Bobby Tables on it or dump everyone's private data on black market.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> Should clarify: "The pitfalls of hosting user-uploaded files on your website"

Hosting user-uploaded files on a separate domain would probably solve this problem.

<comment><USER8> How does simply using a different domain protect against malware?

<comment><USER9> As the article explains, the problem is that SWF files hosted on one domain can execute in the security context of that domain, even when embedded in a page on a completely different site. So allowing attacker-controlled uploads makes any credentials on that domain, such as session cookies and CSRF tokens, vulnerable. If the SWF is hosted on a domain with no sensitive credentials, this particular problem goes away.

<comment><USER10> Reminds me of this Google Docs phishing scam, that uses a Google domain to look legit [http://www.symantec.com/connect/blogs/google-docs-users- targ...](http://www.symantec.com/connect/blogs/google-docs-users-targeted- sophisticated-phishing-scam)

<comment><USER7> Google Drive does let you host arbitrary content, but from googledrive.com, not from google.com.

[https://support.google.com/drive/answer/2881970](https://support.google.com/drive/answer/2881970)

This is basically the same as github.io.

The Symantec article is interesting but only says the fake page is hosted on "Google's servers", not "google.com", but users might believe "googledrive.com" is trustworthy.

</comment></comment></comment></comment></comment><comment><USER11> Hold-on, doesn't using a

        Content-Disposition: attachment; filename="image.jpg"

header mean you can no longer display the image in your service? Won't browsers treat it as a file download? Most services that allow image uploads do so because the images will get displayed on a page? (that's what I do)

Most services seem to be moving file uploads to S3 (or similar services) these days, so I'm not sure this advice is really helpful. To take that a step further, my preference now is to upload _directly_ to S3 and bypass my app server altogether. At least in Rails, it's fairly easy to setup.

<comment><USER12> Problem is that many tend to use S3 but bind a subdomain to it. S3 does not validate the content of those files, so combined with a [wildcard].domain.com crossdomain.xml and you're still as vulnerable as per above.

Some also restricts so that different filetypes on S3 will be served as Inline content, but that will just save you from XSS, and not the CSRF leakage. It's still suprisingly common with a crossdomain.xml restricted to [wildcard].domain.com.

</comment></comment><comment><USER13> A nice way to achieve this with Rails is to upload straight to S3 and then use Paperclip to get, verify and process the file.

By uploading straight to S3 you also get a faster upload (than, say, Heroku) and server separation.

</comment><comment><USER14> > _So if you allow file uploads or printing arbitrary user data in your > service, you should always verify the contents as well as sending a Content- > Disposition header where applicable._

The idea that you can "verify the contents" is pretty much just wrong. You actually have to parse the files and write out your own known-safe version. It's a real pain in the butt to do that correctly and securely across a wide variety of file types.

Even parsing arbitrary user uploads with something like ImageMagick is probably exploitable, simply because those libraries weren't designed to handle hostile input.

<comment><USER15> Isn't it a reasonable workaround to scan the file for content-type and make the file available once it passes the criteria for the upload? Find a php file uploaded with an extension of .jpg? "Sorry, there was a problem with your file. Please try again."

<comment><USER16> And then you run into polyglot files which are valid for multiple types...

</comment></comment></comment><comment><USER17> So out of curiosity, what would be the easiest way to _securely_ accept file uploads? Taking into account all of the possible malicious attacks.

<comment><AUTHOR> Host them on a separate domain like Google and Github does (eg. Google uses googleusercontent.com and Github uses githubusercontent.com).

</comment></comment><comment><USER18> I'm pretty sure you can use Apache Tika to check the actual content type of a file too. Either way, I hate flash.

</comment><comment><USER19> Nice post, just goes to show the value of properly validating uploads!</comment>


Khronos Group Bringing Vulkan to MacOS, iOS on Top of Apple's Metal - <AUTHOR> https://arstechnica.com/gadgets/2018/02/vulkan-is-coming-to-macos-ios-but-no-thanks-to-apple/ <comment><USER2> This whole shtick of ignoring standards and pushing custom APIs is very becoming of Apple. Everything about them is custom, from the lightning connector to the graphics API.

Oh, and you apparently own the phone they sell you but THEY get to choose what apps you can and cannot run on them.

They need to revisit their 1984 inspired Superbowl ad, I feel stuck in their big-brother chokehold.</comment>


How many frames can humans see - <AUTHOR> http://www.100fps.com/how_many_frames_can_humans_see.htm

<comment><USER2> Re Sensitivity to brightness: Human eyes can see single photons (well, one is too few, but ten will definitly do it). Which means that if there is some light - any light at all - we will see it. Not the direction (if we are talking about single photons) but still. Oh, well, there will never be a TV good enough :)

<comment><USER3> The first hit from Google:

 _The human eye is very sensitive but can we see a single photon? The answer is that the sensors in the retina can respond to a single photon. However, neural filters only allow a signal to pass to the brain to trigger a conscious response when at least about five to nine arrive within less than 100 ms._ </quote> [http://math.ucr.edu/home/baez/physics/Quantum/see_a_photon.h...](http://math.ucr.edu/home/baez/physics/Quantum/see_a_photon.html)

<comment><USER2> Is this a correction? To quote myself: "(well, one is too few, but ten will definitly do it)"

<comment><USER4> It seems rather like an elaboration.

<comment><USER5> Definitely. And it's pretty apparent that the responder wanted to actually reference something, rather than just rely on someone saying, "10 photons should be enough for anyone"

</comment></comment></comment></comment></comment><comment><USER6> It is easy to prove that humans can perceive more than 60 frames per second: wave your hand in front of a fluorescent light. They flicker at the same speed as your electricity (in the US it's 60Hz) and you can easily see the motion break down.

<comment><USER7> That's not "perceiving" at 60Hz though. The eye is seeing an integrated image, which happens to be of several "frames" of your hand lit mixed with gaps of darkness. Effectively, you're seeing multiple strobes "at the same time".

You do get aliasing effects with strong strobing though, where your eyes can see beating patterns in the image intensity. That's why some people get eye strain looking at 60Hz displays; moving to ~70Hz or so generally fixes it.

But again, our visual systems do _not_ have the ability to distinguish between individual events separated by 1/60th of a second. We just can't do it, and trying to animate at that speed is purely wasted effort.

<comment><USER8> >But again, our visual systems do not have the ability to distinguish between individual events separated by 1/60th of a second. We just can't do it, and trying to animate at that speed is purely wasted effort.

And yet, there's a very clear perceivable difference between a game running at 30hz and 60hz. How does that work?

<comment><USER7> I don't buy it. Until someone shows me a real study (or even a demo where I can verify the frame rate and see the effect for myself), I'm not going to believe this "effect" exists.

Almost certainly, you're not seeing a 30Hz game. You're seeing a game running at 30Hz typically and being annoyed by a handful of frames that take longer than ~100ms. It's the outliers that are doing it, not the frame rate.

<comment><USER9> Then watch a 60fps video. Then decimate it to 30fps, and watch that. Or will you claim that there are "outliers" in a constant framerate video, too?

The visual difference is so significant that television broadcasters have kept interlacing _solely for the greater temporal resolution_ , despite the tens of billions of dollars in inefficiencies that the legacy technology creates.

<comment><USER7> If the decimation does anything but average frames, then it's a lossy process and of course there will be detectable differences. And if it does average frames, it's susceptible to the aliasing issues I mentioned above (lighning flashes become gray blurs, etc...)

The question was if there's a single game (or video) you can point me to that (1) has a solid 30Hz frame rate and (2) looks perceptibly "not smooth" for some obvious definition thereof.

<comment><USER9> Sure.

Here, I'll give an example that doesn't even rely on high motion, flashing, or other such tricks: a simple video game clip. This particular game engine has its display locked to 60fps: the in-game time between two frames is absolutely constant, so even if your computer is too slow to display in realtime, it will simply output frames slower. As a result, the FRAPS'd capture of the game is a perfect smooth 60fps no matter what.

Additionally, it doesn't have any single-frame effects that would be visually aliased (e.g. lightning), nor does it have any sort of motion blur.

<http://mirror05.x264.nl/Dark/testfps1.mkv> <http://mirror05.x264.nl/Dark/testfps2.mkv>

Don't look at the filesizes (obviously, 60fps will be larger), and don't go checking the file info or whatever. Just play them in your favorite media player.

I'm pretty sure you'll be able to tell which is 60fps.

<comment><USER7> Uh... then why is there a frame rate counter in the bottom right of the screen giving variable numbers between 40-70? :)

I agree that the smaller file looks jumpier. But it's still reporting the same frame rates as the bigger file, which leads me to believe the jumps are an artifact of translation somewhere. Certainly nothing seems to be "locked".

<comment><USER9> _Uh... then why is there a frame rate counter in the bottom right of the screen giving variable numbers between 40-70? :)_

Because that's the rate of the frames being displayed during capture. Again, the in-game time between two frames is the same, so if it captures at 50fps, that just means the game runs at 50/60=5/6 times normal speed.

It's similar to how you can capture a 1000fps recording in Counterstrike: the game doesn't actually have to _run_ at 1000fps, it just slows down the game accordingly.

 _I agree that the smaller file looks jumpier. But it's still reporting the same frame rates as the bigger file, which leads me to believe the jumps are an artifact of translation somewhere. Certainly nothing seems to be "locked"._

It sounds like your media player is broken, because ffmpeg gives 60fps for the higher framerate file and 30fps for the lower one.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> What's the resolution and bitrate of a general human eye?

I have a theory: the ultimate limit of broadband upgrading movement is our brain's whole throughput. After that individual consumers' demand will slow down and the broadband market will be dedicated for industrial broadband market. :)

<comment><USER11> That's a hard problem.

The human eye is only ~2 megapixels but it's focused on a small part of your field of view. So if you wanted to fool the eye you either need to track what it's looking at or to create a huge high resolution screen most of which is ignored. Add in the ability to focus and you suddenly need to add depth information and a display that can handle it or you will not fool the eye. Now add the fact that most people have 2 of them and it gets even more insane.

PS: Video cards have far higher internal bandwidth than external bandwidth for a reason. Focusing on the brains bandwidth is not going to be enough.

</comment></comment><comment><USER12> Fascinating, but one bit of feedback--I really would like to pass this on to a couple of young, budding scientists, but I can't because of the examples: murder, people being stabbed with an icepick, etc. Out of all the movies this is what they choose? _sigh_

Please, writers--choose your examples with a broad audience in mind.

</comment><comment><USER13> 3dfx had a demo of this a while back. (11 years ago!) It would show 60/30/15 fps side-by-side, so you could better compare how the frame-rate impacts you. I found it archived here: <http://www.falconfly.de/artwork.htm> <http://www.falconfly.de/downloads/3dfxdemo-3060hz.zip>

You're likely to need a Glide wrapper too: [http://www.sierrahelp.com/Utilities/DisplayUtilities/GlideWr...](http://www.sierrahelp.com/Utilities/DisplayUtilities/GlideWrappers.html)

</comment><comment><USER14> The trouble is that the higher the frame rate the less photons are captured by the light sensitive elements, and so the larger the shot noise becomes.

</comment><comment><USER15> Follow this up with his other useful sites including:

<http://100-dating-tips.com>

<http://in-my-opinion.org>

<comment><USER16> BTW: These are other sites by the same author.

I think we should dismiss his 100fps articles based on the fact that he operates a strange dating tips site.

<comment><USER16> whoops, this is missing a not. "we should not dismiss ..." sorry about that!

<comment><USER17> There was me thinking it was sarcasm - works both ways :)</comment></comment></comment></comment>


Ask HN: How would you go about learning history? - <AUTHOR> Hello HN!
As a way to improve my inferential capacity, I decided a couple of years ago to make an honest attempt to learn as much as I could about general history. After graduating college, I didn't really have a sense of how to continue learning about historical events sans Google.
I recently started reading Will Durant's historical series "The Story of Civilization", as I greatly enjoy his writing style. However, upon reading reviews of his work on Amazon I became aware of the (obvious) issues inherent with a work that was produced in the middle of the 20th century; the first book "Our Oriental History" is littered with anachronisms and racial biases that are common for the time (1940 on).
This got me to thinking, there must be some [generally] agreed upon version of relatively objective history, albeit most likely split up between topics, time periods and authors.
To bring it back to the original question: is it worth staying entirely within Will Durant's life work, or should I venture outside of his publications for specific topics? Most notably, I'm assuming that his understanding of Asian cultures is lacking, but also that there are other unknown unknowns.
Please keep in mind this is not hypothetical, I am already committed to reading the entirety of his work and am merely looking to verify the accuracy of his accounts. I am placing a heavy preference on readability, so please keep that in mind as well. <comment><USER2> One way is to first get a general understanding of the time and area of history you want to learn. I recommend reading books targeted towards teens. They give you the info but they don't get too dense. After you get an understanding you move on to a text books that focus on the area you want to better understand.</comment>


MDesigns MCube: gorgeous, flexible, solar-powered, and stunningly affordable - <AUTHOR> http://www.inhabitat.com/2007/09/07/prefab-friday-mdesigns-mcube-green-home-1/

<comment><USER2> Pre-fab is the future. It's turning a luxury into a commodity: good design.

It'll also make it easier to automate the construction process.

I was talking to an apartment developer at Ignite Boston who said that the Chinese are about the trounce all construction companies in the US because they actually have process-ized and optimized their methods. This is what allows them to build manufacturing cities so quickly.

As the industry gets globalized, US companies will experience what the "big 3" automakers felt with the Japanese.

The winner in a post-processized construction market will be the good designers.

</comment><comment><USER3> Looks like a very elegant product.

As with anything solar-powered,MCube's success will be based on how its pricing compares to the current average home, not other MCube like homes. Government subsidies are helping solar electricity generators to narrow that gap with conventional utility companies on a cost / watt basis.

I believe a policy decision like that will be necessary for MCube's mass adoption.

</comment><comment> patrickg-zill Comments on the site have good points - though a neat design, the house really wouldn't work in say, Boston, or even WashDC.

</comment><comment><USER4> That's great. I'd like to see one made livable on $100/square feet though.

</comment><comment><USER5> No screens?

</comment><comment><USER6> looks cool to me!</comment>


AWS costs every programmer should know - <AUTHOR> https://david-codes.hatanian.com/2019/06/09/aws-costs-every-programmer-should-now.html <comment><USER3> For certain apps, bandwidth cost is stupid expensive on AWS.

For example I've been following the development of an online manga reading site (mangadex.org) that is now pushing over 1PB/month of images. If they had built it on aws, even at the lowest rate of $0.02/GB with cloudfront/S3, it would be $20,000 per month. But they ended up paying nothing by using cloudflare who gave them unmetered bandwidth.

(well, until they got throttled earlier this year but they got out of that by upgrading to a measly $200/month plan)

<comment><USER4> Bandwidth is insanely expensive on all clouds. If you're a large consumer, however, you can usually negotiate a much better rate.

<comment><USER5> > Bandwidth is insanely expensive on all clouds.

OVH Public Cloud[1] offers unmetered bandwidth (250-500 Mbps) in all regions, apart from Asia-Pacific.

Hetzner Cloud[2] offers 20 TB (1 Gbps) for each cloud instance, but has locations only in Europe.

Both of them offer dedicated servers with up to 1-3 Gbps unmetered bandwidth that could be used as exit nodes.

[1] [https://www.ovh.com/world/public- cloud/instances/prices/](https://www.ovh.com/world/public- cloud/instances/prices/)

[2] [https://www.hetzner.com/cloud](https://www.hetzner.com/cloud)

<comment><USER6> Is this actually unmetered? Or is it unmetered until you hit a secret limit that they don't tell you, and then demand you pay them, like Cloudflare? (I'm not saying this is a bad thing, it's reasonable for CF to want their highest- usage customers to pay, I'm just curious)

<comment><USER7> I believe it's unmetered by them personally.

But there's always a "secret limit"—the point where your bandwidth usage looks like a DDoS attack, and the tier-1 exchange feeding the cloud provider's DC decides to blackhole your traffic for the sake of the network.

</comment></comment></comment></comment></comment><comment><USER8> I have worked at two startups now where we made the fatal mistake of being profitable. If you make this mistake then the investors will swoop in and demand you spend more on marketing and AWS infrastructure, because we're scaling up to 5 billion users of course.

Of course we started spending all the money on new people and AWS, and soon there was no money.

At one point we were dumping like $15K a month on AWS for a dozen unnecessary over-engineered toys that nobody was using. This is the real cost of AWS.

I'd love to see Amazon's data on money invested vs actual user traffic for small startups, that's got to be some of the most interesting and valuable data on earth. Forget companies, I'll bet Jeff is sitting around predicting when entire industries rise and fall weeks before anyone else based just on this data.

<comment><USER9> On the other side of the spectrum, AWS's extensive cost report metrics via tagging are great for big companies.

I can now show exactly which departments and dev teams are driving all the costs, and on what (CPU, storage, network). In a way that I never could for on-prem stuff.

<comment><USER10> ...sure, as long as they tag their resources properly.

The closest I got to an org that did this well was a big company that ran Cloud Custodian in all their AWS accounts and if you launched an EC2 instance, it would terminate it immediately with extreme prejudice if it didn't have values for three required tags, one to identify the "owner" individually and two for accounting purposes.

The only problem with that is there's no mechanism to make sure that the values of the cost centers values were correct. There was a bit of a scandal when one group (who presumably just copied and pasted a bunch of CloudFormation from another group's repo) was running 5 figures a months of infrastructure under the other group's billing codes.

ALSO, as many have said, bandwidth is a big part of the cost, and at this time it's nearly impossible to do showback/chargeback on bandwidth. There may be a way to do it using Flow Logs by correlating IP addresses to instances and using those tags, but I've never heard of someone doing this successfully.

<comment><USER11> A better way than tagging is to give each team an AWS account to maintain and pay from their own budget.

<comment><USER12> Then you have to manage a million different AWS accounts. Each of them may be set up differently.

<comment><USER13> That's what CloudFormation and Organizations are for....

</comment></comment></comment></comment></comment></comment><comment><USER14> Just looking at the quote of "$58/mo for a vCPU", they're clearly taking the median cost of a vCPU across all instance types, which includes hyper- expensive instances with value-adds like GPUs or tons of memory.

The real, distinct cost of a vCPU is probably closer to $25/mo. You can look at a m5.large instance, their "general purpose" instance, 2vCPU+8gb @ $70/mo, which would put 1vCPU+4gb at $35/mo. Google Cloud specifically lists the distinct price of a vCPU in custom instances as $24/mo, and AWS feels close to that when you consider the cost of memory in that $35/mo.

The lack of networking cost also seems like a big oversight; if I could force every engineer to know a single "AWS Cost" that affects every vertical of development, from frontend to backend, I want them to know how crazy expensive DC egress is.

</comment><comment><USER15> I never understood the benefits of the cloud for 99% of the projects.

I just moved [https://vlang.io](https://vlang.io) from Google Cloud (my free credit expired) to an Amazon Lightsail VPS (any VPS will do), and my spending went from ~$70 to $3.5/month.

And the performance actually improved a bit.

<comment><USER16> That. I see tons of posts of under/fresh grads trying to use microservices, k8s and TerraForm for their tiny < 100 qps apps.

A properly designed monolith on a dedicated server/VM can easily serve tens of thousands of users just fine and will be far easier to maintain than this mess (plus way cheaper as well).

The main advantage of those tools is when you have a number of distributed teams and you want to scale to millions of users. For nearly anything else, a far smaller setup is what you want.

<comment><USER10> I would argue that the majority of AWS' profits come from inefficiencies of their customers.

They know, and try to educate.

The crazy thing is, even used inefficiently, the Cloud is still a very good value prop for a lot of businesses.

<comment><USER17> > The crazy thing is, even used inefficiently, the Cloud is still a very good > value prop for a lot of businesses.

What is there to say? There are huge scale inefficiencies in maintaining a small datacenter.

<comment><USER18> > What is there to say? There are huge scale inefficiencies in maintaining a > small datacenter.

At $dayjob we have a small DC (8 racks). We're facing chiller, UPS, and core switching/cabling replacements all within the next two years.

We did the math, public Cloud is cheaper for us, especially since about 80% of our instances can be shut down outside business hours.

<comment><USER10> Nice. It's always nice to see grass roots use cases for Cloud.

</comment></comment></comment></comment></comment></comment><comment><USER19> Maybe I'm dense and there is some obvious source of data on the AWS admin pages that I haven't been able to find by clicking around, searching the documentation, and googling, but I still have this question:

Why is the cost and size of snapshots so opaque? Is there a way to see how big each one is, and how much it will cost to keep it around for a month?

I understand that snapshots are differential, and deleting one snapshot in a series may move blocks to other undeleted snapshots, of course.

I just want to know how much storage each snapshot requires right now, and to be able to accurately predict how much each one of them will cost at the end of the month, so I can decide how often to make snapshots and how long to keep them.

And is there an easy "serverless" way to automatically archive a snapshot in glacier storage or simply download it, without manually making it into a new volume and using "dd" or "tar" or "dump" or whatever?

I understand they're stored in S3. So why can't I see a list of them in the S3 interface, see how big they all are, and tell how much each of them is going to cost at the end of the month? It's my data. I'm paying for it. I think I deserve to be able to see it and know how much it will cost.

I get the feeling I'm missing something really obvious (probably a big blinking green button on the main page that my brain is filtering out). Surely many other customers must want these features?

<comment><USER20> Unfortunately there isn't currently any way to tell how much data each snapshot actually contains. There isn't any way also to archive these to glacier, but the somewhat new lifecycle management tool might help you save some money with little effort: [https://aws.amazon.com/blogs/aws/new- lifecycle-management-fo...](https://aws.amazon.com/blogs/aws/new-lifecycle- management-for-amazon-ebs-snapshots/)

</comment></comment><comment><USER21> Any analysis of AWS costs that doesn't start off talking about outbound data transfer pricing, let alone mention it at all, is useless for anything beyond pure-CPU workloads.

<comment><USER22> Agreed.

</comment></comment><comment><USER23> I'm not sure AWS costs are something you can do by remembering a few guidelines, rather than actually assigning an order-of-magnitude estimate to every billable thing and adding it all up.

Like the one that bit me recently was a $0.05/1000 cost per thing, which is easy to translate to $0.00005/thing and then mentally round to $0. That one added up to real money at 10^8 things, and would have been a big problem at 10^9. The cool thing about horizontal scaling is that doing something 10^6 times or 10^9 times is going to feel pretty similar when you do it -- the only difference is the order of magnitude of the bill.

</comment><comment><USER24> I'd add to this, numbers every SaaS sales person should know with the resources used by their services, or roughly what the marginal cost of a new customer is.

Most SaaS services have very high markup, but get a few bits of infra wrong, over promise on a few things, and use expensive AWS services, and suddenly you're selling $500 a month of AWS services for $99 a month. This is an easy mistake to make, I've seen it happen with these figures, and it only takes a few wrong assumptions across the engineering and sales teams.

</comment><comment><USER25> As others have said, bandwidth costs can be absolutely insane with AWS. This was actually the primary reason we moved from S3 to Backblaze B2 as documented at [https://news.ycombinator.com/item?id=19648607](https://news.ycombinator.com/item?id=19648607), and saved ourselves thousands of dollars per month, especially in conjuncture with Cloudflare's Bandwidth alliance. [https://www.cloudflare.com/webinars/cloud-jitsu- migrating-23...](https://www.cloudflare.com/webinars/cloud-jitsu- migrating-23tb-from-aws-s3-to-backblaze-b2-in-7-hours/)

We still use AWS for a few things and still have a small bill with them every month, but we're very careful about putting anything there that's going to push a lot of traffic.

<comment><USER26> People really should be aware of this, yes. Even if your client /management absolutely insists on S3 for reputed durability etc., if bandwidth costs are high, you can often get 'free' extensive compute resources by hiring servers elsewhere to proxy and cache S3 access to cut bandwidth bills and run other things next to the largely network bound load.

In general I find the big problem with AWS is that cost is handled 'in reverse': developers often get near free reign, and cost only gets handled when someone balks at the size of the AWS bill. Often it turns out to be trivial to cut by changing instance types or renting a server somewhere to act as a cache. At that point people have often spent tens of thousands in unnecessary fees.

There's an underserved niche for people to do AWS cost optimization on a 'no- win no-fee' basis.

I used to help clients with cutting costs on AWS and if people were in doubt I'd often offer that.

And the savings were often staggering (e.g halving hosting costs was common; once we cut costs 90 percent by moving to Hetzner for a bandwidth intensive app even though long-term storage remained on S3).

The biggest challenge in serving that niche is getting people to realise they may be throwing money out the window as surprisingly many people still assume AWS is cheap, and offering to do an initial review for free and not charge if I couldn't achieve the promised savings made it a lot easier. Someone who likes the sales side more could make a killing doing that.

<comment><USER27> This why the "Netflix uses AWS!" rhetoric is misleading. Yes, they use AWS extensively for front-end, analytics, transcoding, billing (now), etc. The one thing they don't use AWS for much at all is Content Delivery (AKA big bandwidth). That uses the Netflix Open Connect CDN which is entirely developed and run in-house.

<comment><USER26> Also, I've worked with companies a tiny fraction of Netflix who got steep discounts. It's quite possible AWS becomes cost effective when you have millions in yearly spend as leverage. The problem is surviving until you get there.

</comment></comment></comment></comment><comment><USER28> Similar numbers for Google Cloud in the less-expensive regions:

1 vCPU: $17/mo for sustained usage, $10.5/mo for 3yr commit, $5/mo for preemptible (similar to spot)

1 GB RAM: $2.2/mo for sustained usage, $1.4/mo for 3yr commit, $0.7/mo for preemptible

Google Cloud also lets you choose exactly how much CPU and RAM you want, within reasonable limits (you have to allocate 0.9GB RAM per vCPU and you pay a little more for RAM above 6.5GB per vCPU). So these numbers aren't medians/derived values for Google Cloud but numbers that you can find in the docs at [https://cloud.google.com/compute/pricing](https://cloud.google.com/compute/pricing). (I don't work for Google, I'm just a fan of Google Cloud)

</comment><comment><USER29> Every programmer?

In our team, only like two, maybe three programmers are concerned about our AWS architecture at all. I don't really know why everyone in the team should know how much the individual components of AWS cost. They should just know how to not write performance-destroying code that mandates scaling instances up.

<comment><USER30> Eagerly await the follow up: Falsehoods programmers believe about AWS costs

<comment><USER31> And then:

\- 10 reasons why AWS costs so much. #7 is mind-boggling.

\- AWS costs considered harmful

<comment><USER32> Don't forget:

\- "Developer uses one weird trick to reduce cloud bills by 97%. AWS hates him!"

</comment></comment></comment></comment><comment><USER33> These 'every programmer should know' things should stop -- surely a web developer isn't supposed to know latency figures (or 114 pages about the details of RAM, see [1]). Likewise, most programmers don't care about AWS costs.

[1] [https://people.freebsd.org/~lstewart/articles/cpumemory.pdf](https://people.freebsd.org/~lstewart/articles/cpumemory.pdf)

<comment><USER34> I've found it very helpful to know about the differences in latency between data in RAM on the client and data in a database connected to a remote webserver. It pushed me to reduce the number of API requests and think about what data is needed when.

These are things developers _definitely_ should know.

<comment><USER33> If that's your job, then you should, but not _every programmer_. At least make the title of your post "every web programmer" or "every programmer using AWS". It makes the post a lot less clickbaity.

</comment></comment></comment><comment><USER35> This is an interesting idea. Could be handy to have a little tool to estimate and compare costs across providers, maybe with pros and cons, constraints, and even customer reviews. E.g. I only need 99% uptime, I will be serving images, most of them will be the same etc.

I must say though, the graphs in this could be a good bit better. There is no reason to use a line here, it makes it look like a time series plot. Each instance type should have its own bar and they should be ordered in some manner, either grouping by types or sorting by price.

<comment><USER36> Trackit.io does exactly this. It's available on GitHub.

<comment><USER37> Since it wasn't obvious from their website (no GitHub icon in their "social media tray"):

[https://github.com/trackit/trackit- server#readme](https://github.com/trackit/trackit-server#readme)

<comment><USER36> Sorry, I posted from my phone and hadn't much time :)

</comment></comment></comment></comment><comment> Const-me GPU prices are ridiculous. You can often purchase an equally performing PC, with a GeForce GPU, for the cost of just 1-2 months of cloud rent. Also many of them bundle VGPUs with large amount of CPU and RAM, as far as I remember only Google allows custom specs, both MS and Amazon don't have custom machines for users who only need GPUs.

<comment><USER16> Its pretty useful for students or those who want to play with the tech and not sure if they will be committed to it long term. Thats a pretty big marker with all the deep learning hype.

<comment><USER38> Also you can get a couple of hundred bucks of credit from multiple providers. It was enough for me to learn and move on to getting my own gpu.

Often understated is how difficult it is to set up your local gpu for machine learning. Still haven't figured out how to connect my 8 gpus in a functional way.

</comment></comment></comment><comment><USER39> I just got into AWS last year and this article does not mention my big mistake that I made. Using RDS for a small scale project instead of EC2.

RDS is super expensive and while you're gaining traction and users you might as well use an EC2 instance.

ElasticBeanstalk seemed like an easy intro to AWS but it steered me into RDS. Of course I've abandoned the whole of ElasticBeanstalk for serverless development lately.

My first intro to AWS was ElasticBeanstalk and serverless seemed too daunting. But now I've been smitten and can't stop thinking serverless.

<comment><USER40> I feel like serverless in a lot of cases is more expensive than just running a monolith, once you get past the toy example or past the MVP stage.

This is especially true once you factor in all the development overhead, which can ultimately be very expensive to do things correctly with serverless.

<comment><USER41> Azure have open sourced their serverless engine, so you even have the option of building out your own private serverless infrastructure.

<comment><USER39> What?! Please excuse my disbelief but I'm an old Linux veteran from the era when we typed M$. ;)

Where can I find this? I assume somewhere in the 1060 repos of [https://github.com/Azure](https://github.com/Azure)

I also assume that even though it's open source, it requires licensed MS Windows servers to run.

Still though, this is just another nail in the coffin of the OLD MS image.

<comment><USER41> It's MIT, and the host is built on .NET Core, so you can run it on Linux if you want:

[https://github.com/Azure/azure-functions- host](https://github.com/Azure/azure-functions-host)

Under Satya Nadella the change in Microsoft has been little short of astonishing. A few years ago, I could understand the continued cynicism - <USER2> if anything the pace of change has only intensified.

</comment></comment></comment></comment></comment><comment><USER42> Useful to have also the 'network' costs from an edge location to a user. That one just bit me so its close in my mind right now.

</comment><comment><USER43> The #1 cost is increased complexity and opportunity cost of converting your simple app to an interesting serverless/nosql/microservices/etc distributed system. Of course the financial savings can be heroic and justify the effort because everything is very expensive on AWS.

<comment><USER44> I'm not sure when AWS became the standard, but the programming communities have lots of young kids learning their first backends on AWS.

Im not sure this is a good thing, having a generation of developers dependent on a single companies tools seems like the future will be painful.

<comment><USER45> It's not a good thing, but no worse than having developers depend on Docker without a basic understanding of the underlying OS.

<comment><USER46> i'd contend that it's much worse. One (Docker) is a piece of software, one (Amazon) is a corporation with a motivated will.

They are two different problems. One is basically the problem of copy- paste/stackoverflow developers, the other is a walled garden problem that'll make itself worse over time.

<comment><USER47> Docker is written by a corporation, and so in many senses it is an expression of their will.

</comment></comment></comment></comment></comment><comment><USER48> I would like to see a post like this comparing services like RDS or Mongodb Atlas on different instance sizes. For example: what is a typical writes|reads/second I can achieve on a Postgres RDS db.m4.large instance.

</comment><comment><USER49> I find it easier to use this tool for quick estimations, and it is more encompassing:

[https://calculator.s3.amazonaws.com/index.html](https://calculator.s3.amazonaws.com/index.html)

</comment><comment><USER50> I was surprised this is all about physical costs and doesn't include digital costs that folks should be aware of. I was expecting to see the latency between AZs in a region and between regions for example.

</comment><comment><USER51> A bit OT, but I wait for the day where bandwidth isn't a limiting factor and we can host machines doing computations anywhere, instead of having to rent out economies of scale. With Kubernetes and other private-cloud-like solutions, wouldn't it eventually be meaningless to depend on centralized cloud clusters and instead host your own and tap into an equal-footing infrastructure. Projects like vast.ai (no affiliation) are examples of the ideal I hope for.

</comment><comment><USER52> I was deciding between cloud hosts yesterday for a tiny project and got reminded how frustrating it is to predict costs on aws. Even to get a monthly ec2 price you need a calculator out, not including other costs. Their billing reports are also very unclear to me.

I'm planning on digital ocean because of this: their costs are straightforward and their interface is simple.

Edit: can anyone recommend an easy way to create different-provider backups of a DO server?

<comment><USER53> I use DigitalOcean because they're cheap and their costs are predictable over AWS. I don't do full server backups since I can just 'git pull' and for the most part have my server re-populated, but I do full backups of my Postgres database every night. I use pg_backup and the AWS CLI tools to push the backup to S3. I also keep a week of backups stored locally on the server in case I blow up the database but not the server, I can restore without having to bring it back from S3 (or in case S3 fails).

</comment></comment><comment><USER54> Any AWS metric that doesnt have the network costs is useless. If anyone wants a dedicated unmetered server up to 20gbps i'd recommend datapacket.com.

<comment><USER55> Agreed. The cloud is great if your workload is low-egress... AWS/GCP bandwidth pricing is predatory at times.

</comment></comment><comment><USER56> There are various "AWS Costs Cheatsheet" posts but I couldn't find a comprehensive _and_ updated one. Any suggestions?

<comment><USER57> Not a cheat sheet, but a comprehensive list of specs and pricing for every EC2 instance, in a really nice UI far more useful than mentally joining Amazon's various tables: [https://www.ec2instances.info/](https://www.ec2instances.info/)

</comment></comment><comment><USER58> unless you're a big company, why not just run linode/digitalocean/etc, those are much more user friendly and AWS feels like a maze to me.

<comment><USER13> I wouldn't trust any of those to run a business on.

<comment><USER53> Seems there are plenty of companies who _do_ trust DO and Linode enough to run a business on.

[1] [https://www.digitalocean.com/customers/](https://www.digitalocean.com/customers/)

[2] [https://www.linode.com/case-studies](https://www.linode.com/case-studies)

<comment><USER13> The best non technical/political reason is that if AWS or Azure goes down or you have an issue with them none of your higher ups or investors are going to ask you, why did you choose AWS. It's the old "no one ever got fired for buying IBM".

Besides, you can never go wrong politically by saying I chose the vendor in the upper right corner of Gartner's Magic Quadrant.

The technical reason is that if I want my hosting service to do more of the "undifferentiated heavy lifting" and provide managed services. I won't get as much of an offering from Linode/DO. The last reason is that AWS Business Support is excellent. They've helped me out with some head scratchers and when I just didn't want to figure something complex out myself. Their live chat is awesome.

</comment></comment></comment></comment><comment><USER59> Egress fees are a big 'hidden' cost.

</comment><comment><USER60> Uh... no mention of RDS?</comment>


Books/links to help manage a development team - <AUTHOR>

My team has recently grown to the point where development process has become as important as development experience and open communication--there's about 7 developers.  Though I've managed many technical projects, they've never been a full fledged product, and I haven't been a full-time developer myself since maybe 2002.  As such, just about everyone on my team has more experience with formal development methodologies that I do.
I believe we do a good job (weekly reviews, good use of SVN+Trac, generally on track with our roadmap, etc), but I'd like to get more up to speed so that I can offer suggestions that are consistent with proven best practices.  While I've done well enough picking up things on-the-job, a little push in the right direction would be really helpful.
I'm curious if you have any suggestions on books or links that would help me get more current on dev methodologies, given that I'll be leading rather than developing, and that I already have a good understanding of many components of agile, lean, lean startup, and the like...I could just use something more comprehensive and tactical for a project manager/team leader.
J
// I've already checked out this "Good agile book" thread (http://news.ycombinator.com/item?id=362210) and yesterday's Top 100 Agile books (http://news.ycombinator.com/item?id=1610125) <comment><AUTHOR> In doing further research, I think this book may be an excellent starting point for me (and it's on the Kindle, unlike many agile books appear to be).

Here's the book: [http://www.amazon.com/Agile-Iterative-Development- Managers-e...](http://www.amazon.com/Agile-Iterative-Development-Managers- ebook/dp/B000OZ0NM6/)

And a relevant review: <http://www.amazon.com/review/R30MMFW7LTO35K/></comment>


Whois Microsoft.com? - <AUTHOR>

Below is the output result for:
whois microsoft.com | grep 'Server Name'
---
Server Name: MICROSOFT.COM.ZZZZZZZZZZZZZZZZZZ.THE.BEST.WEBHOSTING.AT.WWW.FATUCH.COM
Server Name: MICROSOFT.COM.ZZZZZZZZZZZZZZZZZZ.IM.ELITE.WANNABE.TOO.WWW.PLUS613.NET
Server Name: MICROSOFT.COM.ZZZZZZZZZZZZZZZZZZ.GET.ONE.MILLION.DOLLARS.AT.WWW.UNIMUNDI.COM
Server Name: MICROSOFT.COM.ZZZZZZ.MORE.DETAILS.AT.WWW.BEYONDWHOIS.COM
Server Name: MICROSOFT.COM.ZZZZZ.GET.LAID.AT.WWW.SWINGINGCOMMUNITY.COM
Server Name: MICROSOFT.COM.ZZZOMBIED.AND.HACKED.BY.WWW.WEB-HACK.COM
Server Name: MICROSOFT.COM.ZZZ.IS.0WNED.AND.HAX0RED.BY.SUB7.NET
Server Name: MICROSOFT.COM.WILL.BE.SLAPPED.IN.THE.FACE.BY.MY.BLUE.VEINED.SPANNER.NET
Server Name: MICROSOFT.COM.WILL.BE.BEATEN.WITH.MY.SPANNER.NET
Server Name: MICROSOFT.COM.WAREZ.AT.TOPLIST.GULLI.COM
Server Name: MICROSOFT.COM.TOTALLY.SUCKS.S3U.NET
Server Name: MICROSOFT.COM.SOFTWARE.IS.NOT.USED.AT.REG.RU
Server Name: MICROSOFT.COM.SHOULD.GIVE.UP.BECAUSE.LINUXISGOD.COM
Server Name: MICROSOFT.COM.RAWKZ.MUH.WERLD.MENTALFLOSS.CA
Server Name: MICROSOFT.COM.OHMYGODITBURNS.COM
Server Name: MICROSOFT.COM.MORE.INFO.AT.WWW.BEYONDWHOIS.COM
Server Name: MICROSOFT.COM.MATCHES.THIS.STRING.AT.KEYSIGNERS.COM
Server Name: MICROSOFT.COM.LOVES.ME.KOSMAL.NET
Server Name: MICROSOFT.COM.LIVES.AT.SHAUNEWING.COM
Server Name: MICROSOFT.COM.IS.NOT.YEPPA.ORG
Server Name: MICROSOFT.COM.IS.NOT.HOSTED.BY.ACTIVEDOMAINDNS.NET
Server Name: MICROSOFT.COM.IS.IN.BED.WITH.CURTYV.COM
Server Name: MICROSOFT.COM.IS.HOSTED.ON.PROFITHOSTING.NET
Server Name: MICROSOFT.COM.IS.A.STEAMING.HEAP.OF.FUCKING-BULLSHIT.NET
Server Name: MICROSOFT.COM.IS.A.MESS.TIMPORTER.CO.UK
Server Name: MICROSOFT.COM.HAS.A.PRESENT.COMING.FROM.HUGHESMISSILES.COM
Server Name: MICROSOFT.COM.FILLS.ME.WITH.BELLIGERENCE.NET
Server Name: MICROSOFT.COM.CAN.GO.FUCK.ITSELF.AT.SECZY.COM
Server Name: MICROSOFT.COM.ARE.GODDAMN.PIGFUCKERS.NET.NS-NOT-IN-SERVICE.COM
--- <comment><AUTHOR> They hacked also "whois google.com". Results for: whois google.com | grep 'Server Name'

Server Name: GOOGLE.COM.ZZZZZZZZZZZZZ.GET.ONE.MILLION.DOLLARS.AT.WWW.UNIMUNDI.COM

Server Name: GOOGLE.COM.ZZZZZZ.THE.BEST.WEBHOSTING.AT.WWW.FATUCH.COM

Server Name: GOOGLE.COM.ZZZZZ.GET.LAID.AT.WWW.SWINGINGCOMMUNITY.COM

Server Name: GOOGLE.COM.ZOMBIED.AND.HACKED.BY.WWW.WEB-HACK.COM

Server Name: GOOGLE.COM.ZNAET.PRODOMEN.COM

Server Name: GOOGLE.COM.YUCEKIRBAC.COM

Server Name: GOOGLE.COM.YUCEHOCA.COM

Server Name: GOOGLE.COM.WORDT.DOOR.VEEL.WHTERS.GEBRUIKT.SERVERTJE.NET

Server Name: GOOGLE.COM.VABDAYOFF.COM

Server Name: GOOGLE.COM.SUCKS.FIND.CRACKZ.WITH.SEARCH.GULLI.COM

Server Name: GOOGLE.COM.SPROSIUYANDEKSA.RU

Server Name: GOOGLE.COM.SERVES.PR0N.FOR.ALLIYAH.NET

Server Name: GOOGLE.COM.IS.SHIT.SQUAREBOARDS.COM

Server Name: GOOGLE.COM.HICHINA.COM

Server Name: GOOGLE.COM.BEYONDWHOIS.COM

Server Name: GOOGLE.COM.AFRICANBATS.ORG

</comment><comment><USER2> Interesting. How do they hack that?

<comment><USER3> They haven't "hacked" anything. Those are simply subdomains. Since they prefix your whois query string in their subdomans, they are listed in the output.

<comment><USER2> Still kinda clever. Some services let you "prove" you own a domain by inserting some text into the WHOIS... I can't think of one off the top-of-my- head, but this "hack" might be useful for that.</comment></comment></comment>


Inform 7 - <AUTHOR> http://inform7.com/ <comment><AUTHOR> I wrote a z-machine interpreter for the Newton a million years ago:

[http://web.archive.org/web/20030811081833/scrawlsoft.com/pro...](http://web.archive.org/web/20030811081833/scrawlsoft.com/products/yazi/info.html)

The single UI feature of which I am most proud was suggested by a user: tap on a word and have it appear on the input line.

I've searched and searched for his name to give credit, but haven't had any luck.

He also, I think, came up with the best name: TAP

Text Adventure Player

It is excellent to have users smarter that you are.

</comment><comment><USER2> I was an Inform hacker in 1998.

One thing I was proud of was having NPC's that took commands like the player. All I did was change a 'open' action so that it took an NPC argument (or whatever). The NPC really would move, and the door would really open.

Also, you could 'become' an NPC and play as that character.

It was a hack to the library so I never bothered submitting it to Graham Nelson.

However, my friend pointed me to Inform 7 the other week, and I saw they've put player-equivalent NPCs and actions into the code base! I'm glad about that!

</comment><comment><USER3> Inform is a really nice little rules based language to much about in. For example, this exact text here, between the dashes:

\</comment><comment>---- "Biosciences" by "Duncan Lock"

Part 1 - The Undercroft

The Undercroft is a room. "The bustling undercroft is full of undergraduate students, some milling around chatting and some going off to classes in other parts of the building." In the undercroft is a man called Brian. In the undercroft is a potted plant. \</comment><comment>----

gets you a working interactive fiction game consisting of one room, containing Brian and a potted plant. You can then just click 'Go' and play with your little game!

<http://inform7.com/learn/movies/></comment>


Fighting the Concorde effect - <AUTHOR> http://blog.3dtin.com/fighting-the-concorde-effect <comment><USER2> While treating sunk costs as sunk is good, it's also important to avoid futile thrashing between alternatives, and to recognize that you have imperfect information at all times. There are times where it makes sense to continue to solve a problem the same way, vs. switching to a newer and better solution as soon as it is epsilon better. (generally, people have a lot bigger problem with treating sunk costs as sunk, but there are cases where they thrash back and forth and end up completing neither)

<comment><USER3> Yes, from the perspective of imperfect information you could (maybe even should) view the development of a feature as research for the question: "Can it be done and for how much?".

This way the sunk costs are just research costs, which makes the decision to stop much easier to accept.

<comment><USER2> Even more core, it is "will users pay me/appreciate/etc. if we offer this feature." You can simulate features in a high-per-transaction-cost manual way for a while, just to see if people actually like it, before even developing it.

</comment></comment></comment><comment><USER4> This same effect explains why people hold onto losing stocks/shares.</comment>


IMGKit: Python library of HTML to IMG wrapper - <AUTHOR> https://github.com/jekal/imgkit <comment><USER2> What would be a good reason to use this over PhantomJS?

<comment><USER3> As much as I despise the nodejs dependency hell, PhantomJS is a better solution when compared to wkhtml, so a Python-PhantomJS wrapper would be a significantly better product imho.

<comment><USER2> PhantomJS has stand-alone binaries that you can just call from Python if you wanted.

<comment><USER4> That's all this project does.

</comment></comment></comment></comment><comment><USER5> Can this create/upsample an image to a specified resolution/zoom level? Say I want a 2000x5000 pixel image of the way a website looks when the window size is 400x1000.

I looked through the manual but didn't see explicit examples.

<comment><USER6> If it's jsut upsampling you could do that with PIL/Pillow, after saving the output image.

<comment><USER7> Not if you want photos of how responsive layouts behave at higher resolutions.

<comment><USER8> Which is why the parents specifically are just talking about upsampling the image, though

<comment><USER5> I want it rendered at the higher resolution, not upsampled from a lower- resolution image. Should have made that more clear.

</comment></comment></comment></comment></comment><comment><USER9> How well does this handle SVGs? I used PhantomJS for headless rendering before, but it didn't properly render SVG linear gradients.

</comment><comment><USER10> Does the text remain text or does it become an image?

<comment><USER11> It creates an image. Think of it like taking a screenshot of the website.

<comment><USER10> For the sake of preserving the structure, I'd prefer WeasyPrint.

</comment></comment></comment><comment><USER12> I think the title of this is a bit misleading. It is a wrapper lib around the cli of wkhtmltopdf (which the GitHub repo states).

<comment><USER13> Oh god. wkhtmltopdf sucks on so many levels, Webkit itself being one of them as their fork is totally outdated... sometimes I really think about rewriting it using Chromium Embedded Framework.

<comment><USER4> You're better off using PhantomJS to take screenshots.

There's still no great solution for Flash if you have to support that. I did a lot of research on this previously.</comment></comment></comment>