Celebrating Daemontools – two tools for managing Unix services - <AUTHOR> https://blog.infinitenegativeutility.com/2015/2/celebrating-daemontools <comment><USER2> I can't hear daemontools without thinking of how much I used it back in the day for games... ;)

The lengths we used to go just to play a game...

<comment><USER3> Different daemontools but same here.

The daemontools OP is talking about is also very useful, though.</comment></comment>


The Verge Tech Survey - <AUTHOR> https://www.theverge.com/2017/10/27/16550640/verge-tech-survey-amazon-facebook-google-twitter-popularity <comment><USER2> This may be a perfect example of the tech cage, where we fail to understand the opinions of those outside of the industry. I would be very interested to see the same survey provided to those who work in technology, especially in the Bay Area, to see how it skews.

<comment><USER3> Talk about selection bias. Amazing how they extrapolate this to headlines like "A third of America wouldn't care if Twitter disappeared". Would be nice to know how the study was performed, and how the population sampled.

<comment><USER4> 80% of the US population doesn't use Twitter. Why is it hard to believe that 33% wouldn't care if Twitter disappeared?

<comment><USER5> And "care" is ambiguous. People who loathe Twitter "care", not just people who like it.

</comment></comment></comment></comment><comment><USER6> In my opinion, Graphs seem hard to understand. For example, I wanted to compare "Greatly Trust" part of Amazon vs Your Bank in the first graph, but since both "Greatly Trust" sections start and end at different places, I can only compare them based on my raw senses but to me they look exactly equal, even if there might be a small difference.

<comment><USER7> Yeah the graphs here are a visual mess. Who decided on displaying the data this way?

</comment></comment><comment><USER8> The utility of Amazon is really apparent in these responses. I think the "how much would you care if company {x} disappeared" is one of the most interesting questions. I imagine this is how people feel about that:

Apple - iPhone / Mac loyal users would be disappointed but know that Android devices and PCs have gotten pretty good lately, not the end of the world.

Google - has all your online accounts, mail, calendar, docs, etc. Losing this would be a massive pain, but it's not too hard to imagine some other software company stepping into the void.

Amazon - the everything store. If this goes away many people will have to reorganize their physical lives. More trips to the store, back to waiting weeks (not days or hours) for niche items, paying more for almost everything. Most importantly, it's very hard to imagine any other company developing a logistics network like Amazon has.

I would have liked to see Uber/Lyft included in the survey. Although I have a pretty negative opinion of the "ridesharing" industry I can't really imagine what would happen to urban and suburban life if they just disappeared. So many of my plans these days rely on the fact that I can summon a car to take me somewhere without any planning, and it's not going to be horribly expensive.

<comment><USER9> I was surprised by Microsoft. I could see IT department heads missing Microsoft and XBox loyalists, but other than that, who would miss Microsoft? Did they survey a lot of people in Seattle?

<comment><USER10> Some of Microsoft overlaps Google (lots of people still use Outlook/Hotmail/Contacts/Calendar/Office/OneDrive/Skype/etc, it's kind of a pain to switch over everything)

The gaming community would loose an awful lot too. Not just PC Gamers, but most of the industry development tools and workflow are largely Windows- centric.

</comment></comment></comment><comment><USER11> Almost all of those graphics are needlessly difficult to understand.

<comment><USER12> not so.

The amount of info being conveyed is not trivial. It's nature requires reader to think about what is being presented. There is likely not too many other options to more clearly visualize the data.

6 values for each company, and you want to convey relativity between the values of each company..

<comment><USER13> I was confused by the lack of consistency across each of the graphs.

"No Opinion/Don't care" is sometimes gray, sometimes yellow, and sometime orange.

And for some graphs, the negative reaction is on the left and others the negative reaction is on the right.

</comment></comment></comment><comment><USER14> I wish these graphs weren't rasterized. They're a bit fuzzy on my display

<comment><USER15> First world reply to a first world problem.

</comment></comment><comment><USER16> Oh poor Twitter... practically useless.</comment>


Jack Daniel's Sends the Most Polite Cease-and-Desist Letter Ever - <AUTHOR> http://mashable.com/2012/07/22/jack-daniels-trademark-letter/

<comment><USER2> <http://news.ycombinator.com/item?id=4279063> <Many comments

<http://news.ycombinator.com/item?id=4280953>

<http://news.ycombinator.com/item?id=4282486>

<http://news.ycombinator.com/item?id=4286471></comment>


Ask HN: Why this package gets about 200 downloads after each release on npm? - <AUTHOR> I put a command line tool on npm recently.
Usually, it gets a few downloads per day, but as soon as I push a new release suddenly it gets about 200-300 downloads.
Are there any bots downloading each release? I'm just curious to know, as this is my first serious npm package.
Talking about: https://www.npmjs.com/package/moro <comment><USER2> This can be useful: [http://npm-stats.com/~packages/moro](http://npm- stats.com/~packages/moro)

</comment><comment><USER3> I have the same with a package on nuget. Bots or updating the packages</comment>


What makes a great API ? - <AUTHOR> http://kirang.in/2013/11/07/what-makes-a-great-api/ <comment><USER2> A few comments:

> valuable service

I'd think that what makes a good api should be characterized independently of what makes a good service. You can have a useless service, with a fantastic api. No one is going to use it, maybe, but the api itself can still be objectively seen as being good.

> Design

I agree with most of the points, although I'd say that the single most important part of the design is ease of use. I'm not a professional software engineer, so I don't have a good appreciation for the other elements.

> Documentation

Now that is the most important part for me. I recently had to use the API of a service I won't name. It was a disaster. The API was documented, but the documentation was basically unreadable and confusing. It took me a long time to figure out how to work with it. A good documentation, with extensive examples can save the user an awful lot of troubles

Addenda: I have two other considerations, somewhat unrelated:

\- The last API I had to deal with use OAuth. I assume that there's a module or package for it in most languages. But this is the kind of thing you want to check, because if your language of choice doesn't have it, then good luck implementing it.. (rule number one of cryptography: don't write cryptography code yourself). I'm sure there are other examples of tiny details like this that make certain languages harder to use.

\- Support for JSON is usually appreciated (by me anyways).

Edit: Fixed typo (without->with)

<comment><AUTHOR> Thank you for your thoughts and suggestions ! :)

The "valuable service point" struck me now. I'll refactor that section on the blog soon.

</comment></comment><comment><USER3> I'll add two points that I think are more important than any that have been mentioned so far.

1) A good API is hard to misuse. – Documentation is great, but what's even better is when the naming conventions and responses from the API are so well named and their purposes are so obvious, that it's hard to misuse.

2) A good API is helpful and informative when you've misused it. – Sent in an invalid key? The response should say so, and not just return an anonymous error. Is your request missing a required header field? A required parameter? An incorrect type for a parameter? Unused parameters? Yes, good documentation can help with these issues. But one of my biggest problems with documentation is that you can't look up what you don't know you need to look up (especially when there aren't any good tutorials).

</comment><comment><USER4> One thing not mentioned here: discoverability.

As there are more and more APIs, more devices, and in general more complexity, computers should be talking to each other and figuring things out between themselves more. Right now almost all APIs require developers to explicitly code the interaction details. If 'RESTful' web services were actually 'RESTful' (i.e. reached level 3 on the Richardson maturity model) then we might be able to do that.

People keep saying that HATEOAS isn't ready for use yet, I'd say the only reason for that is that no one is using it. I would consider a level 3 qualifying web service to be technically far superior to the majority, and I would be more likely to use it.

<comment><AUTHOR> Any good resources to read more about "discoverability" that you'd recommend ?

<comment><USER4> The original thesis on REST by Roy Fielding is a good start, to understand the real motivation behind creating it.

Martin Fowler has a good article on the Richardson Maturity Model, and what each part means: [http://martinfowler.com/articles/richardsonMaturityModel.htm...](http://martinfowler.com/articles/richardsonMaturityModel.html)

Finally, for an example of why HATEOAS is a good thing, and why APIs should be at level 3 on the RMM, Jon Moore has a good talk on Hypermedia APIs here: [http://vimeo.com/20781278](http://vimeo.com/20781278)

As far as I am aware, there are very few companies actually applying these principles. GitHub is doing something close to it in their API, and I have heard that Google uses this sort of system for their APIs extensively on internal projects. In terms of new-web-API-startup like companies, Twilio/Stripe et. al, I have yet to see many take it this far.

<comment><AUTHOR> I see. Thank you for the resources :)

</comment></comment></comment></comment><comment><USER5> I agree with the list and I would like to add a couple of points to it.

1\. don't return error formats in a different data format (e.g. xml error response with a json api)

2\. don't use custom request signing (hey, mixpanel and seomoz)

3\. when rate limiting, use headers to communicate how much requests are left, how many were made before and how many seconds until the rate is refreshed.

4\. Authentication, there's an obvious ease-of-use vs. sensitivity of the data. but when in doubt pick basic auth (or api-token) with https and close port 80 on the api host.

5\. use REST

6\. be fast

7\. provide as much nested data as you can without violating 6

8\. maintain your own client libraries

I agree with the comments here that documentation and consistency is a good thing, I also agree with having an option to batch requests if you have a rate-limit in-place.

</comment><comment><USER6> I don't think the points mentioned are very discriminating. Planning, design, documentation... Aren't these traits indicative of all "great" software?

Maybe the best way to express what makes an API great is to lead with an example. Show me a great API.

<comment><AUTHOR> Yes, those points are general traits of all great software. But there are still API's out there that fail to conform to those basic principles and that's why I wanted to write the post as a checklist to self, to remind as to what makes an API great.

<comment><USER7> Yes, but the world is now reading ;)

I found this very simplistic, but then I didn't realize it was a checklist for yourself. A few notes, if I may:

first, it is generally recognized that it takes 3 or so tries to get an API/framework correct. Meaning used in 3 or so representative projects. Oh, if you are essentially copying another API in a different language or whatever than the hard work is done. But recognize there will be a lot of refactoring going on in greenfield development. So you probably don't want to do big design up front, as you will get it wrong. Accept that. I'm sure we can point out some genius counter-example, but for us mortals the rules apply.

You touch on some of that in the post, but for me it is always the biggest, most important thing, and it almost always get treated as the elephant in the room that no one really acknowledges, or kind of just hand waves away, and go off and invest 3 months of work into something that is broken and unusable even before public release.

Second, I'm not sure you captured another vital component - that of safety. One of the MS Press books (I forget which one) uses a vending machine analogy. There was a vending machine in the office that had you punch in 2 numbers to get a product. A digit for the row, another for the column. Perfectly usable, except there was endless cursing because inevitably people would punch in the #s in the wrong order, and get something they didn't want. The trivial fix is to use letters for columns, and #s for rows, so the KitKat bar is A3, and the gum is C1. Fixes are often a lot less trivial in code APIs. But you need to design them so the user cannot easily shoot themselves in the foot. How many hours have you struggled with a 3rd party library because it _looks_ like you connected everything up correctly, but nothing works? The API allows you to do things that have no meaning.

</comment></comment></comment><comment><USER8> Consistency and logic.

POST /users

GET /users

PUT /users {:id => 2, ....}

GET /users/2

GET /users/2/friends

POST /users/2/friends

etc

<comment><USER9> PUT should reference the object ID in the URL though.

<comment><USER8> Yup, I agree. But I just wanted to show how the params might look like.

</comment></comment></comment><comment><USER10> I agree on all this points, although all of them should apply to any type of software you build. Anyone know of great APIs of non-tech startups?

</comment><comment><USER11> I can't remember what it was, but someone posted here before with a library to help build clean, easy to use API Docs. The one that looks like a split-pane with examples on one side and details on the other.

</comment><comment><USER12> Documentation and consistency. I almost don't care if your API sucks as long as it's documented well.

<comment><AUTHOR> Precisely the reason why I enojoy integrating Stripe or Twilio. They've abstracted a complex service with a beautiful API and awesome documentation.

</comment></comment><comment><USER13> A great API helps me print cash.</comment>


Looking at the facts: Sarah Sharp's crusade - <AUTHOR> http://www.preining.info/blog/2015/10/looking-at-the-facts-sarah-sharps-crusade/ <comment><USER2> And this is why some people shouldn't even have flagging rights I think.

Now this story is off the front page and lost for the majority of the audience here.

</comment><comment><AUTHOR> Seems hugged to death already, sorry.

Wayback machine link:

[http://web.archive.org/web/20151008102734/http://www.preinin...](http://web.archive.org/web/20151008102734/http://www.preining.info/blog/2015/10/looking- at-the-facts-sarah-sharps-crusade/)

</comment><comment><USER3> "Social Justice Warrior"

Really? That's the level of discourse the author is gonna set the bar at? At least I know that I'm not missing any substance by having just skimmed the post.

Seems like a cheap scramble for pageviews with an incredibly biased article, starting with the title.

<comment> vezzy-fnord Loaded language is extremely common, and you doubtlessly use plenty of snarl words without even realizing it: [http://rationalwiki.org/wiki/Snarl_word#Snarl_words](http://rationalwiki.org/wiki/Snarl_word#Snarl_words)

Though, much of a word's snarl comes from its tone and context. Where SJW is indeed a pejorative, it should not be construed as an intrinsic thought- terminating cliche. It's merely a shorthand to describe a certain flavor of people subscribing to academic ideologies that interpret the world as an identity struggle.

<comment><USER3> Yeah, I don't doubt that I do.

But presenting an entire post full of loaded language that also rails against "Social Justice Warrior Sarah Sharp's crusade" while claiming to be just "looking at the facts" is disingenuous at best.

</comment></comment></comment><comment><USER4> "crusade", "SJW mode", "ranting", "meandering", "here [sic] arguments are so weak I had nothing but good laugh" and it goes on and on.

The author couldn't be less objective and unprejudiced, if he tried. Obviously though he doesn't and couldn't give less of a fuck, not even remotely making an attempt at actually merely "Looking at the facts".

No matter where your opinion on Sarah Sharp, this piece makes one thing abundantly clear, the author is an asshole. Which I'm sure he doesn't mind being called, I mean this is all just fun right?

<comment><USER2>
      The author couldn't be less objective and unprejudiced, if he tried.

As far as I understand it was a collection of things that had been written on the LKML.

<comment><USER4> It's not just a series of quotes (not that it would this would necessarily be objective). He provides clearly heavily biased commentary, starting with the title.</comment></comment></comment>


Holy levitating Slinky - <AUTHOR> http://kottke.org/12/06/holy-levitating-slinky <comment><USER2> Well, saying that it's "compressing as it falls" doesn't quite explain the observation either. What you see is the original slinky with a sudden catastrophic "sound wave" at the top propagating across it. It doesn't look like the whole slinky is contracting as it falls.

It's important to understand that as you hold it, each part of the slinky exists in a condition of equilibrium: since it is not accelerating upward or downward, the force of gravity and the force of the spring above pulling the slinky up must exactly cancel. If you slice the Slinky up into different "chunks" as a function of height, each chunk (call it dm) has the relationship (up-force) - (down-force) = g dm. It only connects to the parts immediately above and below it, and only moves when those parts move.

So actually, as long as that bottom part remains stretched just the way it is, the bottom edge cannot go up or down; the tension up top is the same as the tension on the bottom.

With a little more reflection, the stretching changes _must_ propagate at the speed of sound in the slinky. That's just what sound is -- changes of pressure (in this case, tension) propagating through the solid. If you could send this sort of information faster, then sound would propagate faster commensurately. The speed of sound is fundamentally thus seen to be a connection, for the spring force, between "wiggle this side over here" and "that side wiggles."

Now, there are other forces at play in the slinky in theory which can propagate much faster: I imagine that a sound wave which spirals through the plastic (using it as a waveguide) for example, might actually travel to the bottom much faster. The top pushes on air which has a faster speed of sound, and emits light which goes even faster. But, those forces are not the crucial force which holds the slinky up and therefore needs to change.</comment>


Firefox Focus 3.0 – 27 Languages / Ability to change search engine - <AUTHOR> https://blog.mozilla.org/blog/2017/01/25/firefox-focus-now-available-in-27-languages/ <comment><AUTHOR> Source code at [https://github.com/mozilla- mobile/focus](https://github.com/mozilla-mobile/focus) \- let us know what you think.</comment>


H.264 is magic (2016) - <AUTHOR> https://sidbala.com/h-264-is-magic/ <comment><USER3> The interesting thing about H.264 is that it achieves excellent compression rates not by a single "miracle trick", but by combining multiple small-gain techniques. Every technique lets you gain on the order of 1-3%, but when you combine them all, you get fantastic results.

Another tidbit of (potentially) interesting information: it took many years for H.264 to achieve its full potential, for two reasons:

1) Implementing a decoder is really, really complicated, especially if you want it to run fast, and if you think a miracle "hardware decoder" solves the problem, you need to remember that H.264 places heavy requirements on memory bandwidth and caching (B frames), so it's not like you can just feed a stream through a chip and get video frames on the other side.

2) Implementing an encoder is really, really complicated :-) — and there are many ways to encode a video stream. H.264 delivers the tools, but does not specify how you should use them. It took many years for decoders to get to a state where most of H.264 is used efficiently.

<comment><USER4> It's a tremendous effort to simply ensure standards. There are so many knobs and dials on h264 encoders, and features in each profile, is hard to appreciate the compatability we have today across devices - <USER2> considering how much is in silicon.

I remember sponsoring a feature in x264 (the best software encoder) and the level of complexity in that code is mind bending. The patch was to allow better CPU utilization when running with over 16 cores - something the authors hadn't run into at the time and had no systems to test on. I did a simple profile during encode, sent them the trace, in 2 days the patch was tested and working and it hit main in a week's time. Boom, all 48 cores used.

Made me realize just how much talent is behind these technologies and I still shake my head at how much money is generated with this and how little of that makes it's way back to the developers at the core of it.

<comment><USER5> > Made me realize just how much talent is behind these technologies and I > still shake my head at how much money is generated with this and how little > of that makes it's way back to the developers at the core of it.

If it's any consolation, not much more makes it to the talented developers in commercial products, either. It mainly ends up going to executives and sales.

<comment><USER6> I think in New York and California at least, developers are getting paid closer to what their work is worth. Salaries are often hitting the $300k ballpark for highly talented developers. Average pay itself is now around a $150k.

I think it's _primarily_ developers in other countries[1] that are getting severely underpaid. Hopefully that changes in the coming years.

[1] The most egregious example might be India. Around 7 years ago, I was on a plane from Kerala (a state in India), and the guy sitting next to me happened to run a software consulting company (in Kerala). His customers were from Japan and some other countries (can't remember), and his company wrote Linux drivers for proprietary hardware. He hired kids right out of college. I asked him how much he paid them, and he said without blinking, Rs. 15000 per month. Type in "INR 15000 to USD" to Google, and you're in for a shock. It's around $2600 per year, but probably closer to $3000 per year at the exchange rates back then.

American developers doing the same job could earn up to 100 times that, and at the very least 40 times that amount. I can't understand why on Earth salaries are 40 to a 100 times higher _for the same exact work_ , and for work that could be done by anyone anywhere on the globe, but I hope this massive difference in pay converges and disappears soon.

<comment><USER7> What kind of developers are you referring to? I've noticed a huge discrepancy between companies and industries.

</comment></comment></comment></comment></comment><comment><USER8> Interesting introduction. Too bad it doesn't really get into some of the more complicated things an advanced H.264 encoder like x264 is doing, e.g. adaptive quantization methods. There's also a mistake or two, for example

> P-frames are frames that will encode a motion vector for each of the macro- > blocks from the previous frame.

Actually, P-frames can use data from multiple previous frames, not just the last one.

I think it's worth pointing out, as well, that even though we've technically surpassed H.264 with newer codecs like H.265, VP9, and AV1, all of which are roughly 20%-50% more efficient, this has required tremendous increases in encoding complexity. H.264 is special - it seems to occupy a kind of inflection point on the complexity / efficiency curve. It's _far_ more efficient than previous codecs like Xvid, WMV, and so on, but at the same time even a lot of underpowered devices from over a decade ago can easily play it. We're not likely to see tradeoffs that good again in the video codec space.

<comment><USER9> When you control for image quality VP9 outperforms H.264 in single-threaded decoding:

[https://blogs.gnome.org/rbultje/2015/09/28/vp9-encodingdecod...](https://blogs.gnome.org/rbultje/2015/09/28/vp9-encodingdecoding- performance-vs-hevch-264/)

You get the same image quality at a lower bitrate and faster decoding.

VP9 encoding is much slower, but Intel's SVT-VP9 encoder is getting over 300 frames per second on the right hardware:

[https://www.phoronix.com/scan.php?page=news_item&px=SVT- VP9-...](https://www.phoronix.com/scan.php?page=news_item&px=SVT-VP9-Open- Source)

<comment><USER8> That's an interesting claim. I have to say it doesn't match my experience at all, e.g. when playing VP9 videos on Youtube. Several thoughts:

* I'm not sure what the "ffh264" decoder they're talking about is. My understanding is that pretty much everyone is using libx264 for both encoding and decoding. Maybe the latter is more performant?

* Using SSIM to pick "equal quality" files will probably not give very accurate results.

* SVT-VP9 is, if I'm not mistaken, a hardware encoder. One of the issues with these is that they require some significant tradeoffs to get better speed, so they'll have worse quality than a software encoder at the same bitrate. If you go with software you really pay the price when it comes to encoding time.

<comment><USER10> Some corrections:

ffh264 is FFmpeg's decoder. libx264 only does encoding, not decoding.

SVT-VP9 is a software encoder, specially optimized for Intel Xeon Scalable and Intel Xeon D processors.

</comment></comment></comment></comment><comment><USER11> Overall, this article is a pretty good overview of how lossy image/video compression works. However, what the author describes as "quantization" is actually a low pass filter. With quantization, you do not simply zero out the high frequency components. You "snap" them to specific intervals. For example, if you have some data varying between 0 and 100, like [7, 39, 97, 42, 13], quantizing that by a factor of 5 would give you [5, 40, 95, 40, 15]. This gives you an approximation of the fine details, rather than simply throwing them away.

</comment><comment><USER12> > Let's say you've been playing a video on YouTube. You missed the last few > seconds of dialog, so you scrub back a few seconds. Have you noticed that it > doesn't instantly start playing from that timecode you just selected. It > pauses for a few moments and then plays. It's already buffered those frames > from the network, since you just played it, so why that pause? Because > you've asked the decoder to jump to some arbitrary frame, the decoder has to > redo all the calculations - starting from the nearest I-frames and adding up > the motion vector deltas to the frame you're on - and this is > computationally expensive, and hence the brief pause.

Uh, is that actually what's going on there? Because for some reason, seeking is basically instant when I'm playing locally-saved videos, including ones I downloaded directly from Youtube and didn't re-encode.

Actually, this is one of the reasons _why_ I download videos before watching them, instead of using the normal Youtube player.

<comment><USER13> Yeah. VLC for example doesn't have a frame-by-frame backward seek, because they argue it would blow up the memory consumption to keep all those frames around just in case you want to jump backward. And since videos are mostly hardware-decoded on the GPU these days, you'd be wasting all that memory in your VRAM.

Here's one of many threads where Jean-Baptiste Kempf addresses the feature request. [https://forum.videolan.org/viewtopic.php?p=390778&sid=1a571d...](https://forum.videolan.org/viewtopic.php?p=390778&sid=1a571d0ef612114307fdd41229a6278f#p390778)

The amount of pausing is going to vary a lot by video. If you have an I-frame followed by 250 P-frames, there will probably be a noticeable pause.

<comment><USER12> It has been a _long_ time since I used VLC—I generally use mpv on Windows and QuickTime X on Mac because I like their minimal UIs—but I remember it being instant too!

Side note, using VRAM to make seeking instantaneous seems to me like a perfectly good use of resources, at least if my device has the VRAM to spare. On my desktop, I have a 1080 Ti, and it mostly sits idle when I'm watching a video...

<comment><USER14> You may want to revisit VLC. It now has the option for you to run it with with minimal UI or even with no UI at all (just a video frame), and if you are brave enough you can also design a custom UI for it.

The only thing I _really_ don't like about VLC is the traffic cone icon, but I get that's their brand so I have to live with it.

<comment><USER15> I'm another mpv user and I switched from VLC to mpv just because mpv is plain better than VLC. VLC would give me some slowdowns and would hang sometimes, whereas mpv has always played everything perfectly.

<comment><USER16> But VLC has many many more options and possibilities, if you need it.

<comment><USER12> This is kind of why I don't want to use VLC.

I want my media player to do exactly one thing: open the video or audio files I tell it to open, and play them. Plus the option to pause, seek, enable subtitles, or change audio tracks. Anything else is extraneous.

I'm something of a software minimalist. I've spent a lot of time finding and hiding UI elements on my Jailbroken iPhone. If I don't need something I really want it out of sight.

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> Chroma subsampling is a terrible idea in the digital era. You should just assign a different compression factor to the chroma channels instead. I don't understand why we keep using this system of discarding 75% of the color data BEFORE applying the lossy compression algorithm. (Well I do know of one argument - apparently it reduces the amount of CPU time required to compress and decompress the data, but I think that's a pretty lame reason)

<comment><USER18> > You should just assign a different compression factor to the chroma channels > instead [of subsampling chroma].

The good news is that we don't need to change the H.264 standard for this.

Supposing that:

\- most of the already deployed embedded H.264 decoders support 4:4:4 profiles

\- non-chroma-downsampled input content is available

Then, it's only a matter of changing encoder implementations to do exactly as you say.

<comment><USER17> I'm pretty sure there is plenty of 444 content. Movies are likely filmed/edited at 444, they are provided to theaters at 444, but they are generally only made available to consumers at 420.

I believe the reason is that most hardware decoders sold to consumers do not support anything above 420. As Blu-rays and such are all currently 420, the chip makers don't have much incentive to support 444. It's a shame because your TV fully supports 444, and the difference can be huge on some content.

</comment></comment></comment><comment><USER19> But it isn't free so it's a pity it's taking over the world. Chrome plays h.264 / mp4 because they are licenced. Firefox counts on OS support. (The Cisco plugin only supports webrtc calls) so you can't legally watch MP4 videos on Linux.

Webm seems to be as good and free.

Can some comment if I am mistaken about this?

<comment><USER20> It's all capped at a max payment and not terribly expensive.

VP9 and its new versions will take over but it will be a while until it's implemented in hw fully.

<comment><USER19> Non-free standards are bad for open source and innovation in general.

<comment><USER21> The good thing about standards is everyone can have their own.

</comment></comment></comment></comment><comment><USER22> It is pretty amazing that H.264 is not one algorithm that just popped out and changed the world, instead it is accumulation of collection of tricks developed over several decades.

I distinctly remember few folks saying that you would need to violate laws of physics to transmit 1080p @ 60 fps over wifi. Now we are all doing it many times over and billions of dollars are being made every year. This is one of the unsung achievement worthy of Nobel-prize level awards but no one seems to know who these people were.

I wonder if there is any detailed history of how various components of H.264 came together, who led this effort, how projects were funded for such a long time?

</comment><comment><AUTHOR> Previously discussed:

[HN 2016] - [https://news.ycombinator.com/item?id=12871403](https://news.ycombinator.com/item?id=12871403)

<comment><USER23> I wrote a comment there about frame differencing and wanting to try writing a video decoder after having written a JPEG one. Now over 2.5 years later, I can say that I did write a toy H.261 decoder, along with one for MPEG-1 and the beginnings of another for H.262/MPEG-2 in my spare time, and it wasn't all that difficult. In fact, because of the need to not only decode an image for each frame, but to do it quickly enough to keep up with the framerate on the limited hardware of the time, these early video codecs are in some ways _simpler_ than JPEG --- e.g. all the Huffman tables are static, the colourspace and subsampling are either fixed or have only a small number of variations, etc. As evidence in support of this, my JPEG decoder is around 750 LoC (in C), while the H.261 one is slightly smaller at just under 700 LoC. The MPEG-1 decoder is a bit more complex, at close to 1kLoC. I haven't finished the H.262 one and it is more complex (interlacing is a pain...), but it's probably doable in less than 2kLoC total for a decoder that understands both MPEG-1 and 2/H.262 --- the latter is somewhat a superset of the former. I wasn't really optimising for size, nor did I have a good idea of how much code it'd take before I finished, so these should be quite "realistic" estimates of complexity.

I've seen and given recommendations on here to write a JPEG codec as a learning exercise, and a search of GitHub reveals plenty of others who've done it; but the same situation doesn't seem to hold for video. Nonetheless, having done it and realised it's not that much work (i.e. should be doable in a weekend or two), I now recommend trying to write codecs for H.261 and MPEG-1 too. There's not that much media now in those two codecs, but if you get to MPEG-2, you can experience watching DVDs using your own code.

<comment><USER3> The complexity in H.264 gets you later on, when you try to build a complete (e.g. fully compliant) decoder. There are some killer features which seemed like a good idea at the time, but which make the complexity staggering.

MBAFF is a good example (Macroblock-Adaptive Frame/Field encoding). What this means is that the encoder can choose for every block whether it wants to encode it as a single frame or two interlaced fields. Combined with B-frames (and remember that B-frames reference frames both in the past and in the future) this makes for really interesting memory access patterns, which then kills your cache locality and murders performance.

</comment></comment></comment><comment><USER24> One under-appreciated advantage of H.264: it's the last video codec where the artifacts look obviously artificial. Modern codecs have artifacts that look too much like real objects, so they take more brain power to ignore. After watching low bitrate H.264 for a few minutes I stop noticing the artifacts, which isn't the case with modern codecs.

<comment><USER25> What codecs are you talking about? I'm interested to see these artifacts for myself.

<comment><USER24> Comparison videos from: [http://video.1ko.ch/codec- comparison/](http://video.1ko.ch/codec-comparison/)

[http://video.1ko.ch/codec- comparison/videos/vp9_562.webm](http://video.1ko.ch/codec- comparison/videos/vp9_562.webm)

[http://video.1ko.ch/codec- comparison/videos/x264_562.mp4](http://video.1ko.ch/codec- comparison/videos/x264_562.mp4)

At 15 seconds, in the VP9 version, it looks like smoke is pouring off the top- left metal pipe of the machine. It immediately draws my attention because it's so out of place. But in the H.264 version, the whole image is noisy, so nothing distracts me.

<comment><USER26> I'm not really sure which is which but the webm version looks way way way better to my eye. Not even any comparison really.

</comment></comment></comment></comment><comment><USER27> General question about video codecs: why are codecs tied to resolutions? Why can't some codecs be sued for resolutions larger than some maximum? This is probably the one thing I never understood about codecs.

<comment><USER28> Mostly it's just because the standards say so. There's nothing about how MPEG-2 works that inherently makes it impossible to encode 4K video with it, the standard just says that MPEG-2 video shall not exceed 80Mbit/s, 1920 horizontal pixels, 1152 vertical pixels, or 62,668,800 luminance samples per second.

The limits are just there for interoperability. As is, if you encode a video that complies with MPEG-2 High Level, then you can be pretty confident that any MPEG-2 High Level decoder will decode it.

Without the limit everything becomes a mess. You're writing a decoder, what max res should you support? 4K? 8K? The people writing decoders don't agree and then people trying to distribute these high resolution files find that they work in some decoders but not others.

<comment><USER27> Thanks! What do you mean by "not being able to decode" something though? Do you mean time-wise (not enough bandwidth to keep up) or do you mean the algorithm would fail in a different way? The latter doesn't make sense to me, and the former seems like it's entirely system-dependent? Like isn't it the exact same problem if I have an underpowered computer?

<comment><USER12> It means hardware decoders won't be able to decode the videos. (Some software decoders that are overly rigid about following the spec may refuse as well, but most won't, and none actually _have_ to from my understanding.)

You can absolutely use h.264 to encode 8K@120fps if you want to.

<comment><USER27> Ohh hardware decoders! Makes sense.

</comment></comment></comment></comment></comment><comment><USER29> Chroma subsampling is a terrible idea for non-photographic images. People are not "terrible" at color perception. I can clearly see its harmful effects to on-screen captures, line art, anime, 2d video games, or even Super Mario 64's hat.

Youtube's chroma subsampling makes colors bleed, Mario's hat turn into chunky red blocks approximately filling Mario's hat, screen captures discolored and grainy around text, and sharp colored anti-aliased lines turn into a discolored mess.

(Mario's hat was pixelated on pannenkoek2012's emulated SM64 videos at native 640x480. Maybe newer video decoders antialias the chroma channel, so Mario's hat is not pixelated but "merely" blurry.)

I upload oscilloscope videos with colored lines on black backgrounds, and stopped using brightly colored lines partly because color was being blurred and discolored by Youtube chroma subsampling.

</comment><comment><USER30> I bought a laptop camera module and attached to a USB cable and it pegged my raspberry CPU at 90% and soon I started receiving temperature warnings through that thermometer icon on the screen.

I wonder why webcams don't directly use the onboard graphic card.

Even the cheap webcams all suffer from the very same problem.

The reason I bought laptop card, I thought it would be higher quality and lower price and would be good for monitoring my 3d prints.

I've no found any Raspberry Pi camera with autofocus with IR filter intact ( I need it for full light use and better colours during day time)

<comment><USER14> Try the Microsoft LifeCam range. AutoFocus, Supported by Linux, Really good image.

</comment></comment><comment><USER31> I like this high-level summary. One thing I'd change though is that the photo loses details in the frequency domain only. In real view it actually gains unnecessary details/noise - it's not that the grill details disappeared, it's the whole image that gained an anti-grill layer. (You can now see it on the smooth part of the laptop)

</comment><comment><USER32> > You wouldn't say HHHHHHHHH. You would just say "10 tosses, all heads" \- > bam! You've just compressed some data! Easy. I saved you hours of mindfuck > lectures. This is obviously an oversimplification

Oversimplification and exaggeration. Understanding Huffman encoding and deflate algorithm are just two five minute articles away.

<comment><USER33> It's also not compressed at all - the english prose is 21 characters, over twice as long. Also the English prose requires at least ASCII, while the string of H can be encoded with a single bit each. 168 bits > 10 bits.

To be fair, it is faster to _say_ , out loud, because our verbal "character set" is so much richer. A unit of pronunciation is roughly a syllable. "10 tosses, all heads" has only 5 syllables, while "ach ach ach..." takes 10. But this insight is a bit much to ask of someone new to the concept of compression.

<comment><USER34> It's difficult to describe but I read it more like converting the raw data (a list of results) to a generator. Akin to the "compression" described in this xkcd: [https://www.xkcd.com/1155/](https://www.xkcd.com/1155/)

</comment></comment></comment><comment><USER35> The chroma subsampling is why 4k video played on a 1080p/1440p monitor looks so much better/clear/detailed on monitors. Typical 1080P native encoding has less chroma detail for size saving.

</comment><comment><USER36> I've seen this article before, and this part always stuck with me:

> I captured the screen of this home page and produced two files:

> PNG screenshot of the Apple homepage 1015KB

> 5 Second 60fps H.264 video of the same Apple homepage 175KB

> Eh. What? Those file sizes look switched.

> No, they're right. The H.264 video, 300 frames long is 175KB. A single frame > of that video in PNG is 1015KB.

The article does go into how and why this is possible (tl;dr H.264 is lossey, png is not), but for the difference in human-detectable quality, H.264 is astonishingly more efficient.

<comment><USER37> It's a good example but the difference doesn't quite have to be as dramatic as the article makes out.

Screenshot tools rarely produce optimised PNGs. A quick test of the image in the article shows that optipng (lossless) can bring the size down to ~570kb or pngquant (lossy) to ~240kb. Zopfil could further compress the images produced by optipng or pngquant, but it's a case of diminishing returns at that point.

To be clear the article makes a good comparison but I think it perhaps overstates the case to make a point.

</comment></comment><comment><USER38> I worked on this project where we needed to analyze the motion of objects from the camera in real time. We were trying to find an algorithm that was fast enough to run on the hardware, but to no avail.

One day, it dawned on me that since our hardware encodes the video in h264, we should be able to get some of that info from the video itself. So we tried extracting the motion vectors from the video and sure enough, it was good enough for our use and we got it basically for free.

</comment><comment><USER39> FFT is the real magic there and has been since the 90s.

Convince me otherwise.

<comment><USER40> The FFT is lossless, so there's no compression as a result of using it.

The compression is gained by perceptual modeling and dumping parts, and by powerful entropy modeling, like CABAC and such.

Also more advanced things are usually used in modern compression: DCTs, sliding window stuff, wavelets, CABAC stuff, motion prediction schemes, etc., none of which were used in the 90s (except DCT and rudimentary prediction).

<comment><USER41> "FFT is lossless" \- is this true without infinite precision values?

<comment><USER40> The FFT using real numbers is not lossless (except in certain numerically coincidental cases that are not mush practical use), but there are lossless approximations used routinely in compression to avoid error at this point.

For fixed input and output (say 8 bit to 8 bit data), it's easy to make lossless FFTs, since the final truncation or rounding can be designed to never lose information. So in these cases, even a FFT based on , say, IEEE 754 doubles can be made lossless when using integral input and output.

A similar example is converting 8 bit RGB to 0-1 floating point by dividing the color channel by 255.0, which is lossy as real numbers since the resulting floating point is truncated (except precisely for inputs 0 and 255). However, multiplying the result by 255.0, and rounding appropriately, you can recover each input without loss. So this is an example of a lossless transformation that has lossy steps in between. This can be done in almost any case you need to do it.

FFTs in compression are not used to lose values, but to change signal domain, where perhaps interesting and lossy things can be done.

Google lossless FFT or reversible FFT and dig around.

At the end of the day, however, it's not FFTs that are useful very much in compression. They're only used for signal changes at best, and they've been made obsolete for the most pary by other techniques.

<comment><USER21> Can you give some details re:

> The FFT using real numbers is not lossless

The DFT over the reals is invertible, right? How can it be invertible but not lossless?

<comment><USER40> > The FFT using real numbers is not lossless

FFTs use terms of the form e^(2 pi i k / n), and these terms cannot be represented exactly except in rare cases with finite precision floating point numbers (follows from Gelfand's Theorem).

Thus, as soon as you try to use or compute such a term, you've made an approximation, losing information.

The transform is approximately invertible using finite precision, and if your inputs are some fixed set you can make sure and do error analysis to ensure those terms come back out via careful rounding.

But the FFT, and it's inverse, involve infinitely precise real numbers that cannot be represented as floating point.

The DFT fails for the same reason.

 _However_

As I explained above, if you have a limited set of inputs, say byte values 0-255, that get converted to floating point via these approximations, then the inverse approximation is applied, _then the final values are appropriately rounded,_ you can make the DFT and FFT (and almost any approximation algorithm) on this limited subset of inputs 0-255.

As a simple example, consider turning a byte color channel 0-255 into a 32 bit float in 0.0f - 1.0f via dividing by 255.0. Now every one of the values except 0.0f and 1.0f are approximations, since the only exactly representable floats are dyadic (denominator is power of 2) and these denominators are 255 (no powers of 2).

So this is lossy. But the limited inputs means there are 255 different floats possible.

Now multiplying by 255.0, which is (int this case) not lossy on floats, puts each back to close to an integer (but not all are integers). Rounding to closest will restore the original 0255 byte values.

So the roundtrip is lossless here. But the same transforms dividing by 255.0f then multiplying by 255.0f are not lossless for all floating point inputs.

In each case for any algorithm, one needs to carefully design the inputs and outputs and transforms carefully to ensure it behaves as desired.

This is the tip of the iceberg when dealing with floating point algorithms :)

The DFT over the reals is not lossless. Using floats, for example, and applying it to max_float overflows, so cannot be inverted. It's not even bit invertible for almost any set of inputs. It's only invertible on very limited specialized sets of inputs

<comment><USER21> I find it very confusing that you seem to use "real number" and "floating point number" interchangeably.

Like this:

> The DFT over the reals is not lossless. Using floats, for example, and > applying it to max_float overflows, so cannot be inverted.

I don't see why you make statements about the reals based on what happens with floats. They aren't the same, and the DFT exists independently of actually computing it.

</comment></comment></comment></comment></comment></comment></comment><comment><USER42> Still what got into all MacBooks > mid 2015 is a hardware HVEC encoder(if I'm not mistaken), anyone knows why HEVC and not H.264 ?

EDIT: *HEVC not HVEC

NOTE: I found out HEVC is H.265 so my question might not make sense :)

<comment><USER43> Fairly sure hardware H.264 decoding has been included with every Mac, usually on the GPU, for years and years now. A couple years after the iPhone, I think.

<comment><USER42> Decoding sure, but I was saying about encoding, my Macbook mid 2015 comes with "Intel Iris Pro 1536 MB" and takes 25% cpu to capture screen with Quicktime, while newer MacBooks >=2016 come with HEVC, which after some searches I found out is also called H.256[1][2], and is not just in Macs[3]

[1] [https://support.apple.com/en-us/HT208238](https://support.apple.com/en- us/HT208238)

[2] [https://www.macrumors.com/guide/hevc-video-macos-high- sierra...](https://www.macrumors.com/guide/hevc-video-macos-high-sierra- ios-11/)

[3][https://www.techspot.com/article/1131-hevc-h256-enconding- pl...](https://www.techspot.com/article/1131-hevc-h256-enconding-playback/)

EDIT: I see now the label 2016, so opening question didn't make sense :) sorry.

<comment><USER44> They also have encoding. The software might not be using it properly though. [https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video](https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video)

I think screen recording is slow not due to the video encoding but since it defeats standard GPU acceleration of the GUI in order to be able to capture the contents

<comment><USER45> It really shouldn't. You just feed the framebuffer both into the DMA output, and in the H.26x encoder. If needed, use another framebuffer space if the encoder is slow, so you don't _have_ to recycle the buffer after this frame, and risk the encoder not being done yet.

</comment></comment></comment></comment></comment><comment><USER46> In the article near the top it says

"1080p @ 60 Hz = 1920x1080x60x3 => ~370 MB/sec of raw data."

Can someone help me understand the 60 Hz part of this? Was it meant to be fps or does it really mean Hz? And why?

<comment><USER47> The average refresh rate of a computer display is 60Hz. In this context it's interchangeable with framerate but we think of it in terms of Hz because of the monitors.

</comment></comment><comment><USER48> Nice summary!

I just wish he wouldn't have used pounds for comparison... I mean there are 3 countries left on this planet that don't use the metric system officially. Even the scientists in the US use it. Reminds me of [http://www.joeydevilla.com/wordpress/wp- content/uploads/2008...](http://www.joeydevilla.com/wordpress/wp- content/uploads/2008/08/king-henrys-foot.jpg)

But given how he managed to simplify such a complex technology to an easy-to- read article I don't really want to criticize him for using the wrong unit system.

</comment><comment><USER49> I have to say I really love sid's style of writing. Humurous and entertaining yet brief and to the point.

Hope there's more like this to come.

<comment><USER50> Written in 2016, it is the only entry on the blog. Odds are slim, at least coming from Sid.

</comment></comment><comment><USER51> I'd be interested to see this same analysis with H.265 given that its the way forward and the standard on the for many devices and media.

</comment><comment><USER52> Thinking back on the different dead ends, I figure that H.264 is just a reasonable evolutionary stage.

Now, Pixelon...that was probably magic.

<comment><USER31> Somehow I missed this dot-com crash story. The number of things wrong with it is amazing: [https://www.wired.com/2000/05/perilous-fall-of- pixelon/](https://www.wired.com/2000/05/perilous-fall-of-pixelon/)

<comment><USER53> If stories like this interest you, I highly recommend Nat Geo's Valley of the Boom. I don't even like "history shows" but it's a funny and self-aware take on SV in the 90s with solid acting and a pretty low budget.

</comment></comment></comment><comment><USER54> Fascinating. I love these accessible dives.

</comment><comment><USER55> Neat! I'd love to read another post like this about H.265.

</comment><comment><USER56> What a nice article! Love finding these on hn

</comment><comment><USER57> Don't use blockquotes instead of section headings… please. This makes my skin crawl, but more importantly, it messes up assistive technology such as rotor and text-to-speech.

</comment><comment><USER58> I believe the "magic" is happening in the visual cortex.

</comment><comment><USER59> We have the tech today for even wildly better compression, at least for natural looking images. Using neural nets. There is a big computational cost to them, but if history is any guide, that won't be an issue forever.

<comment><USER60> Got any links to that?

<comment><USER61> Its sort of obvious if you see stuff like this:

[https://www.pcgamesn.com/nvidia/ai-gaugan-app-painting- neura...](https://www.pcgamesn.com/nvidia/ai-gaugan-app-painting-neural- network)

The neural network basically has a huge lookup table of "natural" looking stuff. Just point into that table and get what you want out of it: grass, trees, a crowd, etc and the NN will produce it to your specifications.

</comment></comment></comment><comment><USER62> Judging from the comments, the article mostly resonates with people who already know how the codec works, but it doesn't seem to be very useful to engineers who, like me, want to understand it in a bit more technical detail. The author could have removed the casual conversation noise in the text (very annoying in technical articles!) and filled it with some more useful bits instead.

<comment><USER63> I enjoyed the reading and think that it is not a technical article rather entertainment. Maybe that is wrong too and it is both. Why should every technical article by definition be written in a boring language?

For technical details use the terms and search them in your favourite search engine. There are literally tons of articles about every aspect of H.265.

<comment><USER62> > For technical details use the terms and search them in your favourite search > engine.

Thanks for lecturing me, but the article in question is neither too entertaining nor too informative. That was my point. Other than that, of course I can find better materials on the net.</comment></comment></comment>


Ask HN: What methods, tools etc. do you use to validate your business ideas? - <AUTHOR> Do you even validate your ideas or start implementing right away? What does your process of validation look like? How do you generate leads? (landing/coming soon pages, emails, etc.) How do you make the final decision whether the idea is worth implementing? <comment><USER2> I get on the phone and call people even before building anything, especially with B2B products since their phone number is easier to find. Email works too but it's not immediate and it's easier for people to not reply than it is on the phone. I used a book called Talking to Humans (free) [1] that talks about how to validate ideas.

The main way to do so is to listen to the potential customer and not even mention your idea or that you are working on something. You must first understand their true problems, not your idea of what their problems might be, which many technical people especially do and rush into building a product that people may not even want. Ask them about their problems in their daily life and if you keep hearing the same thing over and over and it aligns with your idea, then build the product. Even if it doesn't, a repeatedly mentioned problem is still one that could have a good solution.

[1] www.talkingtohumans.com

<comment><USER3> If you aren't mentioning your idea or that you are working on something, what is the subject of the call? I think I'm not understanding how a cold call like this would work.

<comment><USER2> When you're validating an idea, you don't want to lead people on and get vacuous "yes"es, as in if I ask someone if they like my idea, they will most likely say yes, for one of three reasons:

1) they want to appease me, especially if they're family or friends 2) they like the idea in their minds but that may not be exactly what you have in mind 3) their idea is the same as in your mind, but they don't have the propensity to buy.

You want to listen to what their true needs are instead of shoehorning your product into their problem. You want to build the product from the ground up so to speak. Sometimes, you guess their needs completely right and your product is exactly what they're looking for. However, this is extremely rare because the odds that ex nihilo you got something that fits their needs exactly is 1/infinity. This does not mean that you cannot tweak your existing product to fit their needs, but it is simply better for both you (no prior sunk cost) and the customer (can get something that actually solves their problems) in the long term.

<comment><USER3> yes, the goal behind what you are trying to accomplish makes sense.

I think I'm still not understanding how you are making a cold call, over the phone, to a complete stranger, and somehow getting them to start talking about problems they may have without there being a purpose (from their point of view) to why they're talking to you in the first place.

<comment><USER2> I usually tell them that I'm doing research in the field and need first hand interviews, or something similar. If you can find a story or reason that is plausibly true about why you want to interview them, then that works as well.

</comment></comment></comment></comment></comment><comment><USER4> The best validation is a deep understanding of the target group and their problems. Let me give you an example.

First, pick the target audience you are either part of, or familiar with. In my case, I chose new and aspiring managers.

Second, learn about their pains. Talk to them, see what they discuss on Reddit, Quora, and wherever else they gather. In my case, I see questions about communicating and dealing with difficult people and dealing with various corporate processes.

Third, figure out what they pay for. Some groups buy books. Some pay for SaaS. Some prefer webinars, screencasts or courses. The options are endless, but the focus should be on what the customers already buy, not what we can easily make. In my case, new managers often buy books.

Four, pick one pain and fix it. Now you don't really need validation in the conventional sense of the word because now you _know_ what the people want and you _know_ what they pay for. I picked the communication challenges new managers face because I have studied this topic extensively before.

Five, implement. In my case, I started writing a book, even though I have never written a book before. But I know there are people I can help, so there is a chance that I actually will. My progress so far (shameless plug, accept my apology and please remove it if you consider it inappropriate) [https://www.thenewrole.com/](https://www.thenewrole.com/)

This process is a somewhat simplified version of what a marketing expert Amy Hoy talks about. I suggest you check her website [https://stackingthebricks.com/](https://stackingthebricks.com/) if you are considering starting a side business.

Hope this is useful! :-)

<comment><USER5> I find the "talk to them" part kinda hard for some audiences.

Some people make money with "talking" to people and want to be paid for answering any questions,even if you want to help them.

Some people are rather protective of their knowledge, too.

</comment></comment><comment><USER6> Perhaps against the grain, I sometimes like to build an MVP for myself before any significant validation; something I'll use even if others won't. Then if others don't use it, I will. It can be a good opportunity to experiment with certain technologies as well. One such example is a service I made [http://EmailMeTweets.com](http://EmailMeTweets.com)

When I first made it public I submitted it to ProductHunt and tweeted at marketing folks, with large follower numbers on Twitter, to please try it and help promote it. There was traction but not nearly as much as fast as I had hoped. In fact, just the other day I created an Indiegogo campaign to gauge the interest in paying for the service. At this time, there are 3 contributors for $12 each. Without a big surge it obviously doesn't seem poised to stay alive... for the public. However, like I said, I'll continue to use the service privately, freely. So, it's validated and minimally viable for myself; unfortunately not for the public.

<comment><USER7> Curious about your use of Indiegogo - what made you decide to go that route to gauge interest? It doesn't strike me as a place where folks go to find Saas apps though I could be wrong.

</comment></comment><comment><USER8> A few days ago someone posted a side project marketing checklist to HN that has many great ideas: [https://news.ycombinator.com/item?id=14942902](https://news.ycombinator.com/item?id=14942902)

</comment><comment><USER9> I like to imagine that the product already exists, then attempt to sell it to a customer face to face / in person.

Let's say I'm doing some kind of SaaS for accountants. I would meet with dozens of accounts with a sales pitch for "x software". This will quickly help you figure out if what you're planning on building is actually valuable.

Anybody that takes you up on the sale gets to be an early tester.

</comment><comment><USER10> Feedback from others is absolutely critical. I'm just one person and I usually have some sort of abnormal preference even if I don't know it.

So I'll implement a quick version of the idea that gets the point across to others and roll it out to generate feedback. People will likely utilize it in ways you didn't expect or point out flaws in concept or execution- this is good because even if it doesn't validate your idea it could point you towards developing something else.

This works for smaller features within a project as well. Just roll out a rough cut of it, get feedback, and refine. The product I'm working on ([https://www.jqbx.fm](https://www.jqbx.fm)) has a live chat feature so it's easy for me to roll out a feature to a subsection of users and ask them about it directly. But even if it's as basic as sitting behind someone at your laptop it's almost always worth your time.

</comment><comment><USER11> I'm a huge fan of the interview process Ash Maurya recommends in Running Lean. I'd add to that the understanding Jobs to Be done (Read: Competing Against Luck by Christiansen) for an interview process that really get's to the base progress a user/client is trying to make in a given circumstance.

</comment><comment><USER12> Small-scale test of the general idea. Not even an MVP - test the basic idea as an extreme rough draft. If people respond positively to the general theme, keep testing and building up for that responsive audience. If it's good enough, they'll keep engaging

</comment><comment><USER13> In my former life I was a product developer.

I liked to tell as many different people about my ideas and get their feedback for if it is dumb or not. In that list of people will at least be a couple who would be in the intended audience.

If the idea is at least positively received, I might make an MVP if it is easy enough, if it isn't, I'll probably abandon it.

If the MVP is stable enough, I'll probably point Facebook or Google Ads at it to drive traffic.

If any traction is gained, I look at the numbers to see if it is worth it to finish building it, or just leave it as it is running.

I'm not sure if the Google/Facebook Ads are still a good traffic driver, but they used to be.

</comment><comment><USER14> There is only 1 metric I believe we all need to test during ideation. Will people use this? If so, why? This may involve various ways of answering that question. Things like, figuring out your target audience, asking around, taking surveys, asking people to sign-up for updates etc.

I happen to hate searching for such answers, and end up creating MVPs only to realize not enough people want to use it. But I think even before MVP, one must pursue getting some early adopters excited to try it (even if it is for free). For my next project I plan to be thorough (hopefully).

</comment><comment><USER15> One thing to really watch out for is if the product can be sold profitably or not. Especially in the SMB B2B market many technology products are in great demand, but the CAC is just too high for the product to be viable.

You can get sucked into to creating a product that your customers love, but which can only be sold at a loss once the cost of acquiring the customer is taken into account. After making something that nobody wants (to pay for anyway), this is probably the biggest mistake made by entrepreneurs.

</comment><comment><USER16> If I get a idea for a project og business idea One of the first things I do is checking if where is existing business or similar and go through what they offer and what where price is or if they make any Next write down what they offer now compare your own idea and ask yourself how can I be different and why should customers choose me instead of the compitors

</comment><comment><USER17> If it's helpful, I wrote a blog post a while back that can help when deciding which idea to put deeper validation efforts into:

[https://blog.nugget.one/upstart/your-ideas-dont-suck-your- fe...](https://blog.nugget.one/upstart/your-ideas-dont-suck-your-feedback- loop-does/)

<comment><USER18> FYI your website is down. Can't access [https://nugget.one/](https://nugget.one/)

<comment><USER17> Thanks for the heads up. Should be good to go now.

</comment></comment></comment><comment><USER19> Step 1: Create a "reverse" income statement to test whether the basic concept is financially viable.

Step 2: Talk to at least 10 potential customers to assess the idea. Make sure most are people who don't feel obligated to be nice to you.

</comment><comment><USER20> Nothing can beat Excel for checking initial financial assumptions

</comment><comment><USER21> Discuss about your idea with people, if none of them tells you that they would use your product (without you asking if they will use it or not), then probably no one will ;)

</comment><comment><USER22> [https://www.youtube.com/watch?v=M_tKYvPKN7U](https://www.youtube.com/watch?v=M_tKYvPKN7U)

</comment><comment><USER23> * Google Insights/Trends

* Google Adwords Keyword search tool

These two help in calculating demand of a service or product.

* If you've got contacts, Random Sampled Survey

</comment><comment><USER24> I look at big companies road map, and do what next thing, only better.

</comment><comment><USER25> Back testing with stock market data...

</comment><comment><USER26> My own intuition.

</comment><comment><USER27> Customers.

</comment><comment><USER28> Amazon Mturk has been a good way for me to get the opinion of people on potential products.

<comment><USER14> Interesting. How would you use it though? Since most Mturks are random people with random background which may lead to incoherent response?

<comment><USER27> Better than validating it with people you know - who are too homogenous to your own background to be able to give you an accurate picture (assuming your product is for general consumers).</comment></comment></comment>