Infallible Formula to Create a Money-Making Startup - <AUTHOR> http://diegobasch.com/infallible-formula-to-create-a-money-making-s

<comment><USER2> It's a great method to get inspiration for potential problems in the B2B space. The challenge however, after you've come up with a solution is the sales. Enterprise sales are notoriously difficult especially due to "big business" culture that creates inefficiencies to solve in the first place.

PG has this to say in his essay (<http://paulgraham.com/start.html>) :

"Start by writing software for smaller companies, because it's easier to sell to them. It's worth so much to sell stuff to big companies that the people selling them the crap they currently use spend a lot of time and money to do it. And while you can outhack Oracle with one frontal lobe tied behind your back, you can't outsell an Oracle salesman. So if you want to win through better technology, aim at smaller customers."

</comment><comment><USER3> As someone who works (not in IT) at a government-owned BigCo (2000 employees), let me tell you: building the software that will solve our problems will be the easy part. Navigating the bureaucracy of BigCo's procurement processes will probably take you more time, skill and money than actually building the product.

And it's not enough to have a product that solves a problem. BigCo never fixes things all at once. So whatever your product is, it'll have to work with innumerable poorly-documented legacy systems. Integrating with BigCo's existing, proprietary systems might take longer than developing the core of your actual product.

But the upshot is that BigCos are used to dealing with BigSoftwareCo that is just as slow and unresponsive in its development processes as BigCo itself. BigSoftwareCo doesn't want to fix our problems, they want to sell us an off- the-shelf product at a price tag that implies thousands of hours of dev work. So if you can show them that you can accommodate their needs quickly and without the usual obstructionism, technology dogmatism and nickel-and-dime-ing practices of BigSoftwareCo, there's definitely an opportunity to build a great relationship.

<comment><USER4> Not to mention BigCo's managers are highly averse to risk and won't put their bonuses and careers on the line by vouching for a small company that may or may not be around in a few years in case they need support or maintenance.

</comment></comment><comment><USER5> Heed warning - it's not this simple. The only way to properly do this is to build a strong network with the stakeholders who buy this IT software. Mainly the CTO/CIO (or CFO). That's the only way to make money in the enterprise. Sales cycles are long and arduous, so be prepared for the long haul.

<comment><USER6> In most companies, the CTO or CIO comes nowhere near the kinds of purchasing decisions made by the kinds of companies started by a process like this. There are many tens of dir/it or dir/ops people in almost any Global 2000 company that routinely make six-figure purchasing decisions.

Many of those decisions do percolate to the CIO/CFO level for approval, but there is a world of difference between "purchase approver" and "decision maker". Directly targeting the purchase approver is often a very bad sales strategy.

<comment><USER5> Good point. A VP of Operations/Infrastructure (of IT) who would report to a CTO/CIO would be more suitable. I guess I should have clarified that a bit better. It's also, as you allude, not as simplified as this.

Either way, my point was more about the fact that working day to day-to-day IT work won't give you the edge unless you have some sort of in with the guy/girl who ends up actually making the purchasing decision. It's also not as simple as this, as IT/Business organizations can be muddied with politics that don't always serve the IT personnel's productivity interest.

It also doesn't hurt to also get in with procurement. Which is another barrier I've found in "BigCo". Many large (Global 200) companies have a shortlist of companies they will work with and won't even entertain a conversation unless you are a certain size or have a pre-existing contract with them.

</comment></comment></comment><comment><USER7> BigCo will refuse to buy your software purely because you ARE a startup.

Until you have a decent amount of employees and cash reserves, building any dependency on your product would be deal killing risk.

The assumption is you are likely not going to exist in 2 years.

</comment><comment><USER8> "Before I started IndexTank I already knew that people would be willing to pay $$$ for such a service because I'd been making money with search consulting for a while."

I started off the in a similar manner, doing lots of consulting work. I know there is demand (in my case, its selling structured data), but however I am not too sure on whether the same demand would exist if I were to 'productize' my consulting work (ie. launch an api for structured data).

As one of the commentors, mentioned its really hard targeting the big companies and most small companies simply don't have the technical expertise to use an API (as in my case).

</comment><comment><USER9> True ... the alternative is to stay where you are and actively look for problems. They're not limited to BigCo and if you have the entire breadth of human suffering in view, you're bound to find at least one idea that can be sold (for money ... not users) _and_ excites you.

</comment><comment><USER10> Basic idea is that there are lots of problems in BigCo that can be solved with software.

<comment><USER11> I think more clearly it is a suggestion to find a problem that needs solving, and then solve that. This is solid advice.

Many people will 'dream up' a new product/idea/concept, fall in love with it, and then try to build a company out of it. This has a much much lower chance of success. (great for including in a work of fiction, less useful when you're trying to convince people they need something they never heard of before). So don't do that.

But it also works for life, the 'Just Landed' app is a good example of something that I do infrequently, FlightAware made better (you can watch as the plane comes in) and this could be even better. But you have to have to consciously make the step of stepping outside 'living' your life and look at like as a process to be improved.

Some people do this naturally, others have to work at it, and some never get it. Diego's advice to talk to people and find out their pain points (everyone loves to complain) is a good way of seeing the process outside of that actors implementing that process.

</comment></comment><comment><USER12> Sounds like a good idea, however to sell at BigCo:

\- you really need contacts. As the OP points out, this involves working with them some time (in the inside, as contractor, selling to them, whatever). If you haven't, find a salesman who has the right contacts and experience: "don't call X, he's retiring next year and doesn't want to rock the boat with a new purchase. He'll just string you along. Call Y instead, he's new to the company and wants to prove himself - I play golf with him next Saturday".

\- decision making at BigCo is at a frustrating, glacial pace. If you're used to the instant feedback of throwing something up in an app store or HN or whatever, you'll be disappointed.

\- lots and lots and lots of meetings. Incredibly inane and the lead times are ridiculous.

\- once you've proven yourself, it can be a real gravy train.</comment>


State of the Union Address for 2012 at 5by5 - <AUTHOR> http://blog.5by5.tv/articles/2012/11/18/state-of-the-union-address-for-2012

<comment><USER2> I'm very sad that Hypercritical ends. Siracusa was a podcast making machine, always great at picking out interesting topics, always great at explaining and finding the right analogies. Sure, not all episodes were always genius (as is bound to happen with a weekly podcast), but it's not like that has happened more often recently.

I'm not really at all in tune with American pop culture, so his appearances on The Incomparable are sadly not as interesting to me.

I'm guessing doing an interesting podcast week after week is just hard and he wants a break?

<comment><USER3> According to Dan, Siracusa will make appearances in the new show called "The Crossover" (<http://5by5.tv/crossover>), aleit irregularly. All is not lost for Hypercritical listeners.

<comment><USER4> The problem I foresee is that, by its very nature, The Crossover is intended to have on multiple hosts from other shows, and as such it's likely that he would have to share the episode with people other than Dan. While this could work for some things, I don't think it would work as well for subjects where he has deep, complex opinions or observations that he needs to impart without interruption. Still, I'll take what I can get.

<comment><USER3> Very true. I'll miss the Siracusian-length rants.

</comment></comment></comment></comment><comment><USER5> Random question but I've always wondered this: I don't yet understand the market for 1.5+ hour weekly podcasts like some of 5by5's. Do listeners generally have long commutes or even listen during work? Do listeners generally only listen to 1 podcast a week?

I'm intrigued because I listen to a few far shorter podcasts each week but would find even a single 2 hour episode almost impossible to get through on my schedule.

<comment><USER2> You might have very atypical media use. I fit podcasts in by replacing other media use.

I was just looking at media use data in Germany† and people spend tons of time using different media: 242 minutes of TV (that's four hours!), 191 minutes of radio, 83 minutes on the Internet, 23 minutes reading newspapers, 31 minutes listening to audio recordings, 22 minutes reading books and 6 minutes reading magazines. That's minutes per day (numbers from 2012, Van Eimeren, Frees 2012).

Those who are younger (14–29) use media a bit less – but still quite a lot: 138 minutes TV, 146 minutes radio, 150 minutes Internet, 10 minutes newspapers, 72 minutes audio recordings, 30 minutes books and 4 minutes magazines.

So if people just forgo a little bit of TV use they will easily be able to fit in a weekly 1.5 hour podcast. If you look at the time when people use different media (radio use peaks in the morning at 7, probably when people are in the bathroom or on the way to work, TV use massively peaks around 8 in the evening, Internet use is pretty evenly spread throughout the day) you can also see that radio use is also a prime candidate for replacement by podcasts.

By the way, the data I was looking at can tell you very little about how people use podcasts, because in the general population very few do. Only two percent use audio podcasts at least once weekly.

—

† I'm sorry, but I'm too lazy to look up numbers for the US right now. The paper with the data from Germany is the one I have handy. As far as I remember the details of US media use are different but they are broadly similar (as would be expected for two similarly developed countries).

—

Source: Van Eimeren, Birgit; Frees, Beate (2012): 76 Prozent der Deutschen online – neue Nutzungssituationen durch mobile Endgeräte. In: Media Perspektiven 7–8/2012 (p. 362–379).

<comment><USER5> Just wanted to thank all of your for your responses - this is very enlightening!

I can't listen to speech while doing something else (other than driving or sleeping) so that probably limits my opportunities time-wise.</comment></comment></comment>


FFmpeg and a thousand fixes - <AUTHOR> http://googleonlinesecurity.blogspot.com/2014/01/ffmpeg-and-thousand-fixes.html <comment><USER2> It is interesting that YouTube isn't mentioned in this blogpost, despite there being good evidence that ffmpeg has been used there[1].

The fuzz testing they mention is based around constructing malformed (or at least "exotic") input files and then monitoring for failures... ie simulating exactly the kind of attack someone might use against YouTube's transcoding infrastructure.

[1] [http://multimedia.cx/eggs/googles-youtube-uses- ffmpeg/](http://multimedia.cx/eggs/googles-youtube-uses-ffmpeg/)

<comment><USER3> I'm not proud of this legacy code, but... it exists because it was a real world issue that would cripple a conversion server in production envs when fed certain files with timing/syncing errors as part of an automated upload and conversion process. When it would crash it would consume 100% of the cores and eat up enough RAM to force swap.

This cron has been keeping ffmpeg in check for over 4 years (not 6, whoopsie) in a production environment at this point... it processes thousands of videos a day using a custom queuing and reviewing system.

      1 <?php
      2 /*
      3 **  Cron responsible for detecting failed/hung ffmpeg
      4 **  instances and killing them.
      5 */
      6 require_once '/lib/class/dbmysqli.php';
      7
      8 $output = shell_exec('ps -aeo pid,etime,args | grep ffmpeg | grep -v grep');
      9
     10
     11
     12
     13 preg_match_all("/^[ ]{0,}([0-9]*)[ ]{0,}(.*?) .*?([0-9]*?)\-[0-9]*?\.flv /m", $output, $preg_out, PREG_SET_ORDER);
     14
     15 if (!empty($preg_out)) {
     16
     17     $db = new DBmysqli('dbmaster');
     18
     19     foreach ($preg_out as $process) {
     20         $pid = intval($process[1]);
     21         $etime = $process[2];
     22         $queue_id = intval($process[3]);
     23
     24         $etime = intval(str_replace(':','',$etime));
     25
     26         // elapsed time >= 60:00 (1 hour)
     27         if ($etime >= 6000) {
     28             $fail_sql = "DELETE FROM media_upload_queue WHERE queue_id = $queue_id";
     29             shell_exec("kill -9 ".$pid); // kill hung ffmpeg process
     30             $db->query($fail_sql); // remove file from DB
     31             // debate if life is worth living...
     32         }
     33     }
     34 }
     35 ?>

Yes I know there are better ways to do this now at the OS level, but it was a quick hack over a half decade ago and continues to work... ain't broke, don't fix it kinda deal.

<comment><USER4> "I'm not proud of this legacy code,"

"This cron has been keeping ffmpeg in check for over 6+ years in a production environment at this point... it processes thousands of videos a day using a custom queuing and reviewing system."

why do we programmers always feel we need to apologize for something that we did quickly, but has been running without incident for a number of years.

take a bow my friend. that was an awesome patch you came up with all those years ago.

<comment><USER5> > why do we programmers always feel we need to apologize for something that we > did quickly, but has been running without incident for a number of years.

The script is only treating the symptom, not the problem.

It would be better to detect the files before they waste an hour of time, so you could tell the user instead of having them silently disappear. Maybe there is something you could do to fix the files. The programmer part of me says there is a real fix that needs writing.

It's obviously a great script if it's worked that long. Who knows how long it would take to track down and fix the bug(s) causing the issue. If they haven't _needed_ to fix the bug in all these years just writing that script was obviously a good decision.

<comment><USER6> > If they haven't needed to fix the bug in all these years just writing that > script was obviously a good decision.

Exactly this. There are an unbounded number of bugs which will cause this single symptom. I think detecting the symptom is exactly the right solution.

<comment><USER7> In theory. In reality, after treating one or two causes, the occurrence rate may fall to once a month or year.

I agree that mitigating the problem is a good _first_ step though.

</comment></comment></comment></comment></comment></comment><comment><USER8> The wonders of C:

\- NULL pointer dereferences,

\- Invalid pointer arithmetic leading to SIGSEGV due to unmapped memory access,

\- Out-of-bounds reads and writes to stack, heap and static-based arrays,

\- Invalid free() calls,

\- Double free() calls over the same pointer,

\- Division errors,

\- Assertion failures,

\- Use of uninitialized memory.

But hey, any good programmer always writes perfect C code.

<comment><USER9> Really breaking new ground here. What should FFmpeg have been written in? I can think of a few candidates, but I think they share a number of those "wonders".

<comment><USER10> On the 30C3 there was a talk about a C Compiler (actually an llvm optimization plugin) that can eliminate nearly every memory management related vulnerability by adding memory checks. The penalty is only a 100% increase in runtime.

> If you're a C programmer and somebody says like: I've that optimization that > makes your program 3% faster you go: wow! Then you come and say: well, now I > make it half as fast you go: w00t?! But then you remember that people > actually use ruby to serve web pages.

I'd say this would be actually worth it for a far more secure FFmpeg.

[https://www.youtube.com/watch?v=2ybcByjNlq8](https://www.youtube.com/watch?v=2ybcByjNlq8)

<comment><USER11> The difference is that FFMpeg is very, very CPU-bound, taking four hours instead of two to encode a movie is a big hassle. On the other hand, not many people use a world-facing FFmpeg instance to submit jobs to. I don't see why security should trump speed here.

<comment><USER10> You mean, not many people use VLC?

[https://www.videolan.org/developers/vlc.html#thirdparty](https://www.videolan.org/developers/vlc.html#thirdparty)

<comment><USER12> For realtime playback locally sure. I don't think there are many/any public- facing systems backed by VLC. Presumably VLC prioritizes speed over security.

<comment><USER10> Oh, _that_ was the point being made. My view is that VLC essentially _is_ a public facing system, in the same sense that my browser is a public facing system. Sure, it is not as easy to do a targeted attack against my VLC as against my mail client (or against the nginx on my server). But it is fed a lot of untrusted data. The same goes for Chrome, which also uses FFmpeg according to the article.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> This is cool. I wonder how many of the bugs led to code execution?

If you managed to execute code, what privileges do you have in Chrome? I hope that Chrome was using OS sandboxing for video playing. After all, if they found 1000 bugs, there are probably a few more zero day's available.

Does playing Flash video in Chrome/Firefox end up using ffmpeg? I'm not that knowledgeable about how video works, but there are probably at least two execution contexts: playing .mp4 natively in Chrome, or playing it via a Flash container.

<comment><USER14> > I wonder how many of the bugs led to code execution?

Quite a few. We're often affected with VLC, and code execution is easy to get to. But with VLC, you're "only" in userland.

<comment><USER15> Chrome is also in userland - and it has a sandboxing system. Assuming they're sandboxing ffmpeg, these bugs are more risky for VLC users than Chrome users. Plus, Chrome is more diligent with security updates and the auto-update mechanism is fully automatic.

A sandboxing system provided by either ffmpeg or VLC would be a very good idea, though it would be some work... encoded data in, decoded frames out via shared memory. Negligible performance impact.

<comment><USER13> Yeah definitely. Really there is a gross violation of the principle of least privilege here. A video player is a great thing to sandbox because all it needs is a video output, very limited file access, and very limited gui input. It doesn't need to read all your files, open network connections, or start processes.

Given that there are apparently thousands of bugs in the video parsing code, it seems like a no-brainer.

Section 5.2 in this DJB paper talks about (portable) isolation of plain transformations. Video playing is already close to "pure" or could be made pure pretty easily.

[http://cr.yp.to/qmail/qmailsec-20071101.pdf](http://cr.yp.to/qmail/qmailsec-20071101.pdf)

<comment><USER14> > A video player is a great thing to sandbox because all it needs is a video > output, very limited file access, and very limited gui input. It doesn't > need to read all your files, open network connections, or start processes.

Not sure if serious or sarcasm, to be honest, since this seems very far from what we see.

A media player is not simple to sandbox, (as the MacOS X sandbox showed us for example), because:

\- you need to open files by yourself, without user interaction, to support playlists,

\- you need to open connections by yourself to support video protocol like RTSP, RTMP, RTP,

\- you need raw device access to support Webcams, Capture devices, DVDs, DVB tuners,

\- you need to access GPU buffers for direct rendering, and/or shaders to do fast filtering or just plain chroma-conversions,

\- you need to be able to access the audio output, at low-level, for libsync which is not always doable with the simplified APIs,

\- and I don't understand what you mean by "very limited gui input"; how is that less than other programs?

Sure, it can be done, with performance costs but it's clearly not a "no- brainer".

<comment><USER13> I'm talking about a multiprocess architecture like Chrome.

[http://www.chromium.org/developers/design-documents/multi- pr...](http://www.chromium.org/developers/design-documents/multi-process- architecture)

The entire app isn't sandboxed -- just the code that does video parsing, i.e. with the thousands of bugs and hundreds of remote code execution exploits (!).

See my other comment on this topic. ffmpeg is already very modular, and used in many video players (user interfaces), so this separation is more than natural -- it _already_ exists in the codebase.

BTW, some people seem to be unfamiliar with the multiprocess/Unix design approach (usually people with a Windows background, which I came from as well). I recommend [http://www.catb.org/esr/writings/taoup/](http://www.catb.org/esr/writings/taoup/) for a great intro to this design philosophy.

<comment><USER14> > The entire app isn't sandboxed -- just the code that does video parsing, > i.e. with the thousands of bugs and hundreds of remote code execution > exploits (!).

The exploits are usually on the protocol (access) level, the demuxer (format) level but also the decoder level.

While in theory the first 2 are what you call parsing, many security issues appear also at the decoder level.

And if you want to split the video decoder from the rendering, you need to introduce an additional memcpy (or two) of full decoded frames, which has an important impact.

I agree it would be nice to try, with a correct, separated processes architecture, but it means changing also a bit the usual architecture, where there is a direct-rendering between the decoder and the output.

<comment><USER16> What you do is split the file I/O from the rendering. Drop privs on the renderer, and feed it data from another process that handles the file access, setup, and other privileged operations.

You can open your X11 socket (or whatever your GUI uses) in the I/O process and hand it off to the renderer before dropping privileges, and hand it off without any loss of performance. Another thought -- although I don't generally touch 3d rendering, or know enough to know if it's possible off the top of my head -- is something like cross-process pbuffers: you render into one, and composite the result to the screen from another process. Since your OS already does this sort of thing (X11 compositors, for example, do this sort of cross- process compositing), this can't be too expensive.

You might need to use the XACE extension to restrict the renderer to just the rendering output window in case it gets exploited, so that you don't have access to the rest of the UI.

The places where vulnerabilities can do damage are when communicating with the rest of the OS: File I/O allows you to get user data, permanently install malware, and delete things. The window system allows you to snoop keystroke/mouse data, synthesize input, and watch what the user does. Drop privs on those, and your app is fairly effectively sandboxed.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> _Until we can declare both projects "fuzz clean" we recommend that people refrain from using either of the two projects to process untrusted media files. You can also use privilege separation on your PC or production environment when absolutely required._

Well, it's not like I've got a whole lot of options for video processing. Note that they didn't recommend alternatives that they say _are_ fuzz clean.

I use FFmpeg quite extensively and will continue to do so.

</comment><comment><USER18> Interesting that they are pushing fixes for both FFmpeg and the libav fork. I guess they decided not to pick sides in that war.

<comment><USER19> That is good. But this reminds me of the first time I installed 'ffmpeg' on Ubuntu only to have it not work with my well-tested parameters and also declare ffmpeg 'deprecated'. Only after some head scratching did I figure out it had installed libav and an ffmpeg wrapper that tried but failed to be compatible with the real ffmpeg interface.

I wasn't even aware of the split at the time, but this shenanigan definitely gave me a strong negative initial impression of libav.

<comment><USER18> For sure. The story of FFmpeg and libav is full of sneaky shenanigans like that.

<comment> vezzy-fnord A good entry on this, for those interested: [http://blog.pkh.me/p/13-the- ffmpeg-libav-situation.html](http://blog.pkh.me/p/13-the-ffmpeg-libav- situation.html)

<comment><USER20> That is written by the FFmpeg side. Do you have a complementary view?

<comment><USER21> The mpv developers have published an overview of the situation from their independent perspective which (FWIW) is very accurate from my vantage point as well: [https://github.com/mpv-player/mpv/wiki/FFmpeg-versus- Libav](https://github.com/mpv-player/mpv/wiki/FFmpeg-versus-Libav)

</comment></comment></comment></comment></comment></comment><comment><USER22> Once again I am happy that 'Segmentation fault. Core dumped.' is a perfectly reasonable reaction to malformed user input in my field of work, which only increases my respect for those who write real-world software. Thanks! :)

</comment><comment><USER23> So my paranoia has finally taken over. My first read through and I only take in these features:

        * I presume all this infra is used for more than just ffmpeg so why this software specifically? Im conscious its included in everthing from web browsers to games. If this was an attack vector, its a BIG vector.
        * 1121 (or whatever it was) is a curious number to make a post about. Why not at the 1k mark or 1250?
        * Explicitly calling out use of an unpriv user to a readership likely already well versed in basic security practices but notorious for casually ignoring on personal machines

Google, blink twice if theyve got ya tied up in some legal requirement that you cant disclose.

</comment><comment><USER24> A _thousand_ , and all the categories they list are things the computer should catch for you. How many more will it take before we switch to better languages?

<comment><USER25> Be aware that these are open-source projects. I don't see _you_ starting a fork in a better language.

<comment><USER26> I find this, 'don't complain, fork/fix' mentality ridiculous. Why should he? If his goal is get people off C, communicating/persuading in a developer community comments is probably more effective than him creating a project in some other language that no one would likely ever use.

<comment><USER27> In what way is asking someone, somewhere, to move a project to another, unnamed language, going to be more effective than actually starting a project and putting code into it? Sure, "go do it yourself" is not a pleasant response, but "go take the tens of thousands of lines of code that mostly works, and spend person-years rewriting it in another language so that a certain set of bugs aren't an issue anymore, instead of fixing the mostly working code" is in no way a reasonable request. "What can be done to help reduce the bug count, by a casual user?" might be. Contacting Coverity or another static analysis company that occasionally runs their tools on open source programs to help the world (and get the free press out of it...), might result in a huge list of subtle (and hideously obvious) bugs getting squashed.

</comment></comment></comment></comment><comment><USER28> I get a kick out of the fact that they recommend not using FFMPEG but they say that Chrome uses it.

<comment><AUTHOR> > I get a kick out of the fact that they recommend not using FFMPEG

More specifically Google is recommending you don't use it to "process untrusted media files"

<comment><USER29> I don't trust any media files... Especially ones from Google who are doing most of the research... seriously though, media zero-days are unheard of, probably because they would be extremely valuable.

</comment></comment></comment><comment><USER30> This must be one of the reasons it seems like there's a new update to VLC every time I launch it...

<comment><USER14> Yes. That's sad, but we need to update often, because of that.

And our update mechanism suck. We're working on improving the process because of that...

</comment></comment><comment><USER31> > At Google, security is a top priority - not only for our own products, but > across the entire Internet.

Hm, privacy was not on top, I guess - or we're toasted.

<comment><USER32> Can't deliver privacy guarantees unless you have a secure system in the first place. That's not to say Google does care about privacy, but security being top makes sense.

</comment></comment><comment><USER33> Bit off topic but I may as well bring this up here.

      1. flvrunner uses ffmpeg
      2. biggest result for flvrunner searches is how to remove flvrunner (including browser toolbars)
      3. doubleclick runs webads for flvrunner.com (google own doubleclick)
      4. such ads even run on onlinebehavior.com, which is owned by Google's analytics guy.

There's a lot of malware on the net that google could do more to reduce.

Someone more conspiratorially minded than me might make other deductions.

</comment><comment><USER34> The software testing course in Udacity is an excellent introduction to the fuzzy testing techniques that the article talks about. Here is the link: [https://www.udacity.com/course/cs258](https://www.udacity.com/course/cs258)

</comment><comment><USER35> Years ago (I think it was ffmpeg) it would crash constantly on tv input, but it ran in a different process and restarted automatically.

</comment><comment><USER36> It sounds like about the right level of effort required to maintain something developed by Fabrice Bellard.

</comment><comment><USER37> "Until we can declare both projects "fuzz clean" we recommend that people refrain from using either of the two projects to process untrusted media files."

BAHAHA What a joke.</comment>


Show HN: CoveTrader – Combines multiple crypto exchanges into one trade platform - <AUTHOR> Our dev team recently launched CoveTrader at https://trader.covemarkets.com/. Using our extensive experience from traditional financial markets, we connect to many crypto exchanges (Coinbase, Kraken, Bitstamp, etc.) and show aggregated order books, trade lists, and analytics in both real-time and historical formats. We currently support BTC, ETH, LTC, BCH, ETC, EOS, and XRP, but are adding more. It's currently just analytics, but we'll be adding order sending in Q2. Feedback and feature requests are welcome and can be sent to firstname.lastname@example.com. <comment><USER2> Looks pretty cool, gives us a try too! Nice to see more diversity and choices in the market.

[https://kloudtrader.com/narwhal](https://kloudtrader.com/narwhal)

</comment><comment><AUTHOR> Please visit: [https://trader.covemarkets.com/](https://trader.covemarkets.com/)

Thanks!</comment>


Bodega Isn't Just Bad Branding, It's Bad Business - <AUTHOR> https://www.eater.com/2017/9/13/16302386/bodega-startup-corner-store-silicon-valley <comment><USER2> That name in danish is literally used to describe cheap bars with pool tables</comment>


The Amazon forest is the result of an 8,000-year experiment - <AUTHOR> https://arstechnica.com/science/2017/03/the-amazon-forest-is-the-result-of-a-8000-year-experiment/ <comment><USER2> Well. Not really an experiment, which has specific meaning.

But like any other environment it's affected by those who inhabit it, so no big surprise here.</comment>


Iranian authorities updated account of UIA 752 shoot down - <AUTHOR> http://avherald.com/h?article=4d1aea51/0000&opt=0 <comment><AUTHOR> Two things surprising to me: the SAM system was relocated and not recalibrated. Those running the system couldn't communicate with their command center and fired without approval.</comment>


Sierra On-Line Documentary - <AUTHOR> https://www.youtube.com/watch?v=l_1TZoNiW6U&feature=youtu.be <comment><USER2> Pharaoh was developed by Impressions Games which was bought by Sierra On-Line in 1995. It remains one of the best historical strategy games ever made. Its cinematics were extraordinary for a game turning 21 this year ([https://www.youtube.com/watch?v=1u0js7at9uc](https://www.youtube.com/watch?v=1u0js7at9uc))</comment>


Making QUIC Quicker with NIC Offload - <AUTHOR> https://dl.acm.org/doi/abs/10.1145/3405796.3405827 <comment><USER2> > We find that the kernel to userspace communication, ... often the cause of > application performance degradation

Assuming they're talking about Linux here, I wonder if they used io_uring.

<comment><USER3> >Lesson #1: Data copy between user and kernel space costs around 50% of total CPU usage. This can be avoided by using kernel-bypass techniques as adopted in Quant.

>Lesson #2: In the presence of a kernel-bypass optimization, crypto operations become the new most expensive operation, requiring up to 40% of CPU resources per connection.

I suppose they did an equivalent optimization ? I have no experience with io_uring, but I assume the gains are similar to kernel bypass optimizations.

</comment></comment><comment><USER4> This gets reinvented as a system matures, I think.

IBM mainframes had TCP offload in the early 90's at least. (The NIC in that case was a PC running PS/2 plus some routing software. Worked great.)

<comment><USER4> and they surely weren't the first to do that either, I assume now.

<comment><USER5> FYI, it's possible to edit comments to add further thoughts. :)

<comment><USER6> Only for two hours.

As it turns out, my average time to return from distraction to proofread is a little bit more than two hours.

<comment><USER5> Indeed, that does seem to happen often. Though in GGP's case the reply was posted within a minute of the original comment.

</comment></comment></comment></comment></comment><comment><USER7> Have we addressed the privacy issues with QUIC?

<comment><USER8> What do you mean by privacy issues with QUIC? As I understand it, it's just another application layer protocol.

<comment><USER7> Iirc, Brave browser began disabling QUIC because the connected server was able to fingerprint the connecting party, and they deemed it an intentional part of the protocol design. Essentially connecting anonymously to a server is not possible. I'll try and dig up the blog post.

<comment><USER6> This one?

[https://content.sciendo.com/configurable/contentpage/journal...](https://content.sciendo.com/configurable/contentpage/journals$002fpopets$002f2019$002f3$002farticle-p255.xml)

</comment></comment></comment></comment><comment><USER9> Hmm, researchers with the 'National University of Defense Technology' propound on the benefits of offloading encryption/decryption to a separate processor/FPGA on the Network Interface Card. Just for speed, I'm sure!

<comment><USER10> This has been done for years, eg with Chelsio NICs, and indeed it can speed things up quite a bit.

<comment><USER11> Wiretapping has been done for years, yes. NOBUS encryption compromise has a long history, and speeds things up a lot. (for the NSA)

<comment><USER12> How do you imagine this wiretapping works? This NIC is installed in your servers, and is fed AEAD keys that are derived from key exchange with a client by your TLS stack (and so private keys exist on the host, not on the NIC). This allows the NIC to decrypt/encrypt flows that pass through it and free up CPU cycles. QUIC requires forward secrecy for key exchange, so every flow will use a different AEAD key already, meaning any snooped key can only be used to decrypt the current flow, not any others. Every modern offload NIC uses this basic design, more or less. Where's the wiretap? Is the NIC going to somehow store every intermediate AEAD keys and escrow it to the NSA somehow? What does NOBUS have to do with any of this? And why wouldn't they just backdoor the motherboard/OS/CPU itself to acquire private keys directly?

<comment><USER13> Since NIC is used to encrypt the traffic, it can purposefully have vulnerability that NSA knows about?

Fixing NIC is not as easy as fixing software.

<comment><USER12> Okay, but I'm asking what does it look like? What kind of vulnerability? They presumably aren't using SuperDuper Secret Wifi to mirror data wirelessly to the moon, right? There are a limited number of outcomes at some point. They can't exactly change the encryption algorithms, otherwise clients fail to connect, and modern TLS (and QUIC) are designed to reduce algorithm agility in the name of preventing downgrade attacks and insecure suites. And they can't just break AES with an alien computer, because if so, why bother with the NIC at all? And again: How do they _escrow the data_ they want out of the network, considering the extremely variable (and potentially secured, unknown, hostile) network conditions? If they can do that with some kind of host exploit or whatever, why not just take private keys in the first place? They could just snip ground cables then and be done with it, which is exactly how they got Google. (The most realistic case I can think of is somehow compromising entropy generation, perhaps.)

And finally, why do _any_ of this when you can almost definitely just issue a gag order to a legal council, or behind-the-door threats to a foreign government agency to tow the line, or any number of things? You're dealing with governments who have immense global influence, not scrappy hackers who only have their wits and old laptops about them.

I'm not saying agencies don't have exploits, or they don't use them, or they don't spy on a lot of data, or that even some backdoors aren't real. But if you're looking a NIC offload device, immediately claim "Wiretapping", and can't actually explain how it wiretaps anything or what the attack model is, it's really just random speculation and fear mongering.

<comment><USER14> While I don't think it's likely, it's not hard to conceive a scenario where the NIC purposely weakens the security for attackers in the know.

Purely theoretical (and I'm not a crypto guy, so please do correct me if this is nonsense), but imagine a scheme whereby the IV is chosen to be the first few bytes of the private key xor the port tuple.

This could reduce the difficulty of brute forcing the key, and no extra traffic need be generated - we already know that the NSA operates passive observers, and has even placed such systems inside corporate networks in the past.

EDIT: As to why they'd do this instead of getting a gag order - because they can? Because there's less oversight? Safest to assume that any technical capability will be abused sooner or later.

<comment><USER12> > but imagine a scheme whereby the IV is chosen to be the first few bytes of > the private key xor the port tuple.

Again, the NIC doesn't choose the IV. It is given an IV by the host system, which is derived from key exchange in software, and that IV must match what the other side of the link derives from its own key exchange operation. It has no choice but to use the IV given. Otherwise, the two parties can't communicate. So the NIC would have to attack the host system somehow to engage in this attack, but then it could just steal a private key anyway and get all communications forever. This is basic Diffie-Hellman/TLS 101.

This kind of "I'm not an expert, but let me make up a scenario completely divorced from reality..." thing is what I'm talking about when I say speculation/FUD. It sounds sufficiently "techie smart" to pass a trivial smell test but otherwise instantly falls apart.

> As to why they'd do this instead of getting a gag order - because they can? > Because there's less oversight? Safest to assume that any technical > capability will be abused sooner or later.

Any person in your life that you know could suddenly commit a horrible crime, just "because they can." Do you think they will? Is that reason to _assume_ they will? "Because they can" ignores a basic aspect of how decisions are made, which is understanding their motivations and reasoning.

And less oversight from what? These gag orders are already enforced in secret courts. Governments exert pressure on each other, behind closed doors and through agreements like trade sanctions, to force other governments to comply. Theres's _already_ "no oversight" in the process, by design it avoids oversight. Spooks can literally walk into your datacenter and pull a rack out of the cage and there's nothing you can do about it unless you want to get thrown in a dark hole for 500 years. Even if they had to resort to techie tricks, why is the scenario you imagine any more plausible than a thousand simpler, alternative options? Multi-million dollar corporations get ransomware'd all the time, and it's not like the culprits need hardware backdoors to do it.

Again: these agencies have exploits, and for a reason. They certainly use them. They have backdoors. That doesn't mean we just get to turn our brains off the instant something we don't understand mildly spooks us and assign complete impossibilities as the culprit. You're not far from just doing high- brow "lizard people control society" stuff at that point.</comment></comment></comment></comment></comment></comment></comment></comment>


GlaxoSmithKline whistleblower awarded $96m payout - <AUTHOR> http://www.guardian.co.uk/business/2010/oct/27/glaxosmithkline-whistleblower-awarded-96m-payout

<comment><USER2> The article doesn't mention any criminal charges. So, the cost of this payout will simply be passed on to (particularly, captive -- think IP monopolies) customers.

I'm glad the problems were finally acknowledged, but very sad that no one is going to jail for them. The responsible parties simply buy their way out, using their customers' (aka the victims') money.</comment>


Colorado farmers bringing in nation's first hemp harvest since the 1950s - <AUTHOR> http://www.washingtonpost.com/lifestyle/food/legal-or-not-colorado-farmers-bringing-in-nations-first-hemp-harvest-since-the-1950s/2013/10/12/97c7fa6c-3349-11e3-ad00-ec4c6b31cbed_story.html

<comment><USER2> At the risk of sounding like an insufferable hippy, it is absolutely ridiculous that hemp has been banned. Even if you consider pot to be harmful, it is _stupidly_ obvious that hemp is harmless.

On another note, the headline of the article gives the wrong impression: [http://en.m.wikipedia.org/wiki/Alex_White_Plume](http://en.m.wikipedia.org/wiki/Alex_White_Plume)

On yet another note: hemp has been largely surpassed by other products and isn't really a big cash crop even in countries were it has remained legal. It may make a comeback with a nontraditional demographic, much like PBR, but that remains to be seen.

</comment><comment><USER3> What is ridiculous is that industrial hemp is grown in Canada and then imported by American car companies, to use in the production of certain car parts.

So Ford (for example) can import industrial hemp to the USA, but it was not allowed to be grown in the USA...

</comment><comment><USER4> I didn't know growing industrial hemp was also outlawed in the US.

<comment><USER2> Technically it is regulated, not banned. Of course you could say the same of cocaine...</comment></comment>


Ask HN: Tool to sort/filter bookmarks? - <AUTHOR>

In the last 3 years as an employee I collected loads of bookmarks in Firefox that I know could really use. Problem is, I wasn't really strict with my sorting and filering them at times. This means, there are lots of different structures and even more folders called "stuff". Not the best starting point if I want to catch up on some topic.<p>Are there any good tools out there to sort or filter an exisiting base of bookmarks? <comment><AUTHOR> 1 person upvoted this, so maybe this will be seen:

I found a nice extension to Firefox called Bookstack [1]. It basically enables you to go through your bookmark folders as if they were stacks and remove every item from them by looking at them while sorting them into new folders.

Not perfect, but a first step. Now I will have to find an addon that makes adding sites to some folders easier...

[1] <https://addons.mozilla.org/en-US/firefox/addon/bookstack/></comment>


An Electric Aviation Pioneer Who Aims to Crack the Pilot Shortage - <AUTHOR> https://www.forbes.com/sites/jeremybogaisky/2019/10/24/bye-electric-pilot-shortage/#48069f3553ea <comment><USER2> It's not so much the cost of learning to fly that is dissuading people from becoming pilots. Yes it's much more expensive than it was just 20 years ago, and way more expensive than 40 years ago... but the biggest problem was that until these current pilots started retiring, it was actually a slow climb (pardon the pun) to get to a good flying job.

You had to start with whatever small regional airline or cargo company you could get to accumulate hours, and eventually you could get into the airlines. Until the last 10 years or so, there wasn't really a shortage of pilots in the airlines.

Plus, as we know, non-executive salaries haven't been keeping pace with inflation. And perks and retirement benefits have been shrinking. So it's less attractive now to become a commercial pilot than it was 20+ years ago.

A couple of other factors are also involved. 1. The US economy has shifted away from trade/skilled jobs and into information jobs. 2. Many pilots of the past learned to fly in the military; but since WW2, we've had less and less need for pilots - especially now that we are moving a lot of activity to remotely piloted drones.

I expect that as commercial pilot jobs become more attractive (due to not enough applicants), the salaries may go up, and the interest will redevelop.

<comment><USER3> Pilots make 6 figures, don't they?

<comment><USER4> Eventually. First you have to pay for your training. If you want to work for a major airline you also need a college degree. After that, you become a flight instructor and earn $20-30k and rack up some flight time. You then need to spend several years coming up with 1500 hours of flight time, either through instructing, flying charters, or simply paying to rent a plane. If you need to pay to rent or buy the plane, you're looking at $80k minimum to get buy the final 1000 hours.

Assuming the economy is doing well, you can go get a regional airline job. Today it will pay $40k to start, but if the economy falters, there will be an excess of pilots, and those that are lucky enough to have an airline job will be getting $25k like they used to.

Finally, after some years of this, you'll move to a major airline and start finally making good money. All the while, you're often away from your family, living an unhealthy lifestyle, exposed to radiation, and if you ever have any significant illness or require most any medication, your career is over, and you have to apply all of those jet pilot skills to something else.

If you're 18 and starting your training today, you have 47 years to retirement, and it's very likely that less than half way through your career, automation and remote piloting will have advanced enough that many planes will only require one pilot, causing a collapse in the demand for airline pilots and wrecking salaries.

But hey, you get to fly for a living!

<comment><USER5> There's no chance that automation will advance such that you only need one pilot. The pilots job is to babysit the machine, hardware and software. And you need two for the obvious reason that sometimes one is incapacitated. More complex software _increases_ the need for two pilots.

<comment><USER4> Airbus is openly working on it. [https://www.cnbc.com/2019/06/28/airbus- pushing-ahead-in-tech...](https://www.cnbc.com/2019/06/28/airbus-pushing- ahead-in-tech-as-it-aims-for-single-pilot-planes-cto.html)

If you go back to the 1940s, some big aircraft had five flight crew for long- range flights. They progressively dropped the navigator, then the flight engineer in the 80s, because computers could do their jobs. Many small business jets already can fly legally with a single pilot.

When automation reaches the point that the computer can fly the entire flight, and the pilot merely needs to monitor, as we are close to with this new Garmin autoland system, the pilot incapacitation argument for having two crew members goes away. It's inevitable and just a matter of time.

<comment><USER6> There are some strange corner cases out there:

> _Already requested and had the data linked chart for our landing weight sent > up to the aircraft: we require 5,671 feet on a wet runway, good braking, > zero tailwind. Each knot of tailwind adds 150 to the distance required, so > even one knot of tailwind exceeds the runway length._

* [https://jethead.wordpress.com/2014/12/13/flying-a-jet-in-the...](https://jethead.wordpress.com/2014/12/13/flying-a-jet-in-the-los-angeles-storms-december-12-2014/)

<comment><USER4> Interesting story, but the FMS already can calculate the required distances in various scenarios. I'm not aware of an FMS that can do the what-if type analysis automatically about the range of acceptable wind directions and speeds, but that can certainly be built.

Instead of the F/O flying the approach, he could simply have let the autopilot do the whole thing and focus himself, as he did, on the strategic aspects of the flight.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> Forbes is garbage, don't post their articles here. This is a fawning, analysis-free piece that reads like a long-form press release

<comment><USER8> Not just Forbes. This has happened to Entrepreneur, Newsweek and many others.

</comment></comment><comment><USER9> Wages and conditions in the industry suggest if anything there is a glut of pilots. The pilots that do have jobs work on short term contracts with no job security. It is very much a buyer's market with a few legacy exceptions such as BA where the union is very strong.

</comment><comment><USER10> This whole mandatory retirement after is ridiculous and needs to end. If someone is physically able, and still had good mental capacity as well as senses why stop them? Looking at how out of shape most millennials are your average 65yr old is probably a better bet!

<comment><USER11> ok boomer

This may seem fine on the micro level -- an individual working into their 70s, but is a disaster at the macro level -- a significant part of a workforce suddenly in their 80s.

Senior workers tend to hold senior positions. A 65-er retiring frees a slot for someone in their 40s to move up, which frees a slot for someone in their 20s to take their prior role. Fewer retirements slow advancement for the rest of the workforce and drives out younger workers with skill to seek their fortunes elsewhere.

When that cadre of octogenarians finally starts dropping, the organization will be rushing to fill senior roles with mid-level employees, leaving a big gap to fill for the middle and entry levels.</comment></comment>


A Developers Notebook - <AUTHOR> http://haristauqir.com <comment><USER2> This isn't an article. It's your personal url. I think this is spam.</comment>


Female tech leaders out earn males for the first time – recruiter - <AUTHOR> https://www.channelweb.co.uk/crn-uk/news/3072234/women-tech-leaders-out-earn-men-for-the-first-time-recruiter <comment><USER2> There's no link to the research source in the article.</comment>


Apple's Bitcode Telegraphs Future CPU Plans - <AUTHOR> https://medium.com/@InertialLemon/apple-s-bitcode-telegraphs-future-cpu-plans-a7b90d326228 <comment><USER2> I managed to ask Chris Lattner this very question at WWDC (during a moment when he wasn't surrounded by adoring crowds). "So, you're signaling a new CPU architecture?" But, "No; think more along the lines of 'adding a new multiply instruction'. By the time you're in Bitcode, you're already fairly architecture-specific" says he. My hopes for a return to big-endian are dashed. [Quotes are approximate.]

<comment><USER3> Why on earth would you ever want to _return_ to big-endian?

<comment><USER2> Because big-endian matches how most humans have done it for most of history ("five hundred twenty one" is written "521" or "DXXI", not "125" or "IXXD"). Because the left-most bit in a byte is the high-order bit, so the left-most byte in a word should be the high-order byte. Because ordering two 8-character ascii strings can be done with a single 8-byte integer compare instruction (with the obvious generalizations). Because looking for 0x12345678 in a hex dump (visually or with an automatic tool) isn't a maddening task. Because manipulating 1-bit-per-pixel image data and frame buffers (shifting left and right, particularly) doesn't lead to despair. Because that's how any right- thinking person's brain works.

The one place I've seen little-endian actually be a help is that it tends to catch "forgot to malloc strlen PLUS ONE for the terminating NUL byte" bugs that go undetected for much longer on big-endian machines. Making such an error means the NUL gets written just past the end of the malloc, which may be the first byte of the next word in the heap, which (on many implementations of malloc) holds the length of the next item in the heap, which is typically a non-huge integer. Thus, on big-endian machines, you're overwriting a zero (high order byte of a non-huge integer) with a zero, so no harm done, and the bug is masked. On little-endian machines, though, you're very likely clobbering malloc's idea of the size of the next item, and eventually it will notice that its internal data structures have been corrupted and complain. I learned this lesson after we'd been shipping crash-free FrameMaker for years on 68000 and Sparc, and then ported to the short-lived Sun386i.

<comment><USER4> > Because big-endian matches how most humans have done it for most of history > ("five hundred twenty one" is written "521" or "DXXI", not "125" or "IXXD").

Actually, it is possible that that was nothing more than an accident. We use Arabic numerals, and Arabic languages are written right-to-left. Then there are languages like German where digits are read in reverse, so "42" is read as "two-and-forty".

<comment><USER5> The German "decade flip" is restricted to the tens and unit places; otherwise the order is English-like, with larger terms leading.

The cardinal number systems for most major languages lead with larger terms (as in English). I don't think there's anything deep about this, it's probably an accident. And there are languages which lead with smaller terms, such as Malagasy (the national language of Madagascar).

The ordering of digits in Arabic is not obviously relevant, per se, since spoken English ("one hundred twenty one") matches the order of the Arabic numbers, too.

<comment><USER6> It's funny how the Germans and Dutch (rightfully) ridicule Americans for writing dates in middle-endian order like 9/11/2001, yet they say numbers with the decade flip "two and forty". That's just as ridiculous.

<comment><USER7> MM/DD/YYYY is simply a direct transliteration of spoken English, which makes it easy to read and write dates. In other languages, the spoken version is little endian or big endian, and the written version aligns accordingly. (At least for the languages I know.)

<comment><USER8> ITYM "... spoken American".

<comment><USER7> Are you saying it is commonly referred to as "The 11th of September, 2001" in England?

<comment><USER9> We would normally refer to that as September 11 because it's much more talked about in the US, and that's the phrase used there.

Any other dates will likely be in the same order as written. For instance, the rhyme for bonfire night is 'remember remember, the fifth of November'. I believe that many in the US also talk about the fourth of July, rather than July fourth, so it's not like English has the hard-and-fast rule you were proposing.

<comment><USER7> Ok, fair enough. What do you say for non-special dates like July 3rd, 2015?

<comment><USER9> 'Third of July, 2015' or more likely 'third of July'. The date format really isn't lying to us in UK English.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> I thought that just being LLVM bitcode wasn't enough to guarantee portability like the author assumes that it is.

There's ABI specific pieces that are still not abstracted in the bitcode like struct packing rules.

<comment><USER11> Not only that, but "[target data layout] is used by the mid-level optimizers to improve code, and this only works if it matches what the ultimate code generator uses. There is no way to generate IR that does not embed this target-specific detail into the IR. If you don't specify the string, the default specifications will be used to generate a Data Layout and the optimization phases will operate accordingly and introduce target specificity into the IR with respect to these default specifications."

Much of this article is simply inaccurate speculation.

[1] [http://llvm.org/docs/LangRef.html#data- layout](http://llvm.org/docs/LangRef.html#data-layout)

<comment><USER12> note that data-layout used to be optional, but is now mandatory.

</comment></comment></comment><comment><USER13> How does dynamic linking work in a scheme like this? Would any pre-compiled libraries need to be distributed as Bitcode as well?

Due to the concerns above, IMO Bitcode is less about compatibility and more about app thinning. It's pretty easy to go from Bitcode to 4 different variants of ARM; but another entirely to go from Bitcode to x86 and ARM. Currently, developers have to ship binaries compiled for multiple architectures, which increases app sizes. I suspect Apple is just building a workflow that creates a device-specific version of each app, and having developers compile to Bitcode simplifies app submission.

<comment><USER14> Yeah, according to Apple "If you provide bitcode, all apps and frameworks in the app bundle need to include bitcode."

[https://developer.apple.com/library/prerelease/ios/documenta...](https://developer.apple.com/library/prerelease/ios/documentation/IDEs/Conceptual/AppDistributionGuide/AppThinning/AppThinning.html#//apple_ref/doc/uid/TP40012582-CH35-SW2)

</comment></comment><comment><USER15> This is a smart move. It's essentially what the System/38 (later AS/400 & IBM i) did. They had an ISA or microcode layer that all apps were compiled to. Then, that was compiled onto whatever hardware they ran on. When IBM switched to POWER processors, the just modified that low layer to compile to POWER processors. That let them run the old apps without recompilation of their original source. They used this strategy over and over for decades. Always one to keep in mind.

Going further, I think a team could get interesting results combining this with design-by-contract, typed assembly, and certified compilation. Much like verification condition generators, the compilation process would keep a set of conditions that should be true regardless of what form the code is in. By the time it gets to the lower level, those conditions & the data types can be used in the final compile to real architecture. It would preserve a context for doing safe/secure optimizations, transforms, and integration without the original source.

<comment><USER16> Based on other comments here, it sounds like the LLVM Bitcode representation is more closely tied to a specific architecture than the System/38 intermediate representation was. Same idea, though.

<comment><USER15> I agree. That's either an advantage or another opportunity for modern IT to learn from the past. Those old systems were full of so many tricks they're still outclassing modern efforts in some ways haha.

</comment></comment></comment><comment> Ruud-v-A Microsoft has been doing a similar thing with [.NET Native]([https://msdn.microsoft.com/en- us/vstudio/dotnetnative.aspx](https://msdn.microsoft.com/en- us/vstudio/dotnetnative.aspx)) for a while now, though MSIL is much higher level than LLVM IR. With .NET Native, you _can_ submit your app once an run on ARM and x86, 32-bit or 64-bit.

</comment><comment><USER17> I really hope this bitcode feature isn't going to cause a lot of trouble for app developers. Up until now, the app you built on your machine (and tested on your devices) is the app you submitted and which is running on your customers machines.

In the future, when your app crashes on customer's machines and doesn't on yours, how are you going to debug much less explain this to apple and have them fix the issue for you?

This is especially scary when you consider the turnaround time of ~2 weeks before your new build becomes available in the app store for you to test.

<comment><USER18> You can download the newly compiled binaries to your test devices. The App Store won't release it to customers until you say it's okay.

</comment></comment><comment><USER19> With Bitcode, Apple could change OS X into something like the old TAOS jit based OS. Except for a small kernel, all TAOS executables were intermediate representation files. This IR could be translated to real machine code at the same speed as disk access, and resulted in code running at 80-90% native speed on most platforms.

With software like that, Apple could become independent of any particular software architecture.

(TAOS dates from the 90's and is hard to google, but is mentioned in some papers. And yes, the JIT translator could do that even on 90's machines.)

<comment><USER20> Sounds somewhat similar to the IBM AS/400 (renamed many times). Applications were shipped as byte code and translated to the local machine architecture. The translated byte code was appended to the application, a but like NeXT fat binaries or OS-X's universal binaries.

The native byte code was 64bit, though iirc the first implementation used a 32bit address. The result was you could ship an application once and when it came time to move architectures all you had to do is one command to retranslate the byte code.

Very neat. I'm not sure why this approach hasn't been used more often.

EDIT: Looks like they're going exactly the AS/400 route.

<comment><USER10> Actually AS/400 byte code has 128 bit pointers. It was originally implemented on a 48 bit processor (weird 70's architectures FTW!).

<comment><USER20> I stand corrected. Thanks.

AS/400 had some really cool technology built into it.

</comment></comment></comment></comment><comment><USER21> Note that gcc considers their monolithic design a feature to encourage companies to contribute back code rather than a painful lesson to be learned from...

[http://gcc.gnu.org/ml/gcc/2004-12/msg00888.html](http://gcc.gnu.org/ml/gcc/2004-12/msg00888.html) [https://gcc.gnu.org/ml/gcc/2007-11/msg00460.html](https://gcc.gnu.org/ml/gcc/2007-11/msg00460.html)

<comment><USER22> The article was talking about the Bitcode design in LLVM, not the plugin situation. Yes, gcc considered making plugins hard a feature, but as for a design that separates the frontend and backend in order to make each more modular, then gcc in fact does have such a thing, gimple,

[https://gcc.gnu.org/onlinedocs/gccint/GIMPLE.html](https://gcc.gnu.org/onlinedocs/gccint/GIMPLE.html)

I'm far from an expert, but from what I've seen and heard, gimple is pretty good for what it does. In other words, the author of the article is wrong to say that LLVM learned from a painful lesson in this area.

</comment></comment><comment><USER23> When I read something like this my mind wanders and I imagine a future MacBook Pro that, instead of a single Intel processor, contains many ARM processors. 10-15 ARM processors acting as one could offer a whole lot of performance when you needed it (use all of them), and a whole lot of energy saving when you didn't (use 1 of them). With the current trend of multi-core CPUs, I see this as the ultimate form of the architecture.

Now, whether Apple will do something like this is or not is anyones guess, but its nice to dream of the possibilities. :)

<comment><USER24> > _I imagine a future MacBook Pro that, instead of a single Intel processor, > contains many ARM processors. 10-15 ARM processors acting as one could offer > a whole lot of performance when you needed it (use all of them), and a whole > lot of energy saving when you didn 't (use 1 of them). With the current > trend of multi-core CPUs, I see this as the ultimate form of the > architecture._

Doesn't make sense. What's the difference between "10-15 ARM processors" and 4-8-12 Intel based cores?

I mean apart from the fact that we aren't going back to multi-processor architectures, since there's no benefit from that compared to cores (latency, etc).

</comment></comment><comment><USER25> I wonder how Bitcode will play with profile guided optimization. Will you also provide pgo information to apple or will they generate it.

</comment><comment><USER26> My suspicion is that bitcode allows the App Store team to provide Watch/iOS specific binaries for an individual device. Right now the solution is to create fat binaries that eat precious space. The watch being even more constrained can use all the help it can get.

</comment><comment><USER27> I wonder if this could also be a way to protect compiler-tech and silicon trade secrets, even after they're widely used in the field? Perhaps only Apple ever compiles the final, deployed versions of apps.

</comment><comment><USER28> Yeah, and their patent applications telegraph the iMac with the fiber-optic shell that's been just around the corner for 10 years.</comment>


Ask HN - I have an idea but no means of creation; where should I present it? - <AUTHOR>

I am no programmer, nor do I know everything about transmitting protocols or encryption. But I am quite passionate about my privacy, and the privacy of those activists and journalists that need the secure communication provided by such protocols.
This being said all of the recent happenings with the gov on both sides of the oceans has had my ticker in overtime on a solution, or the start of one. Where would be a good place to present this sort of idea for creative and constructive criticism? (Note that I am not looking to make money from my idea, Id rather everyone just be able to communicate safely, if my solution has any validity at all that is.)
Thanks. <comment><AUTHOR> The way I see it, we have two core issues at hand with email and privacy. 1) The NSA is intercepting email in transit and/or 2) They are obtaining access to where ever the email records are stored via legal or non legal means.

My solution is a local application installed on the computer itself used for email functions, similar to that of Outlook or Thunderbird. Except with no centralized server for transmitting or storage. Instead store all communication logs and copies locally on the clients computer. Along with encryption certs unique to each persons computer. When generating your certs, you would generate with a specific person in mind, so that when you put in parameters for generating a new, unique key, the other person that you are wanting to communicate with can create a corresponding cert that will decrypt the message upon arrival.

One problem I have already is to have keys that reflect the ability for each persons client to read each persons emails. How do you share something you want to keep secret, to keep things secret, without exposing the keys to flaws.

Anyway, moving on, the actual trick I have come up with though is splitting. I want to send an email to person B, so I type it up, I hit a button that says "prepare to send" which sets into action a few steps. One, it fills the email with random crap that doesnt matter to make it larger. Two, it splits this new, larger email, into say, 20 smaller files, but in random splits that will have to be remerged once it has been received. Because it has been split into pieces, we will also want to send each piece through a different transit server before it hits its destination. In this way, even if the emails get caught in a dragnet, or are even targeted, they would have to be lucky enough to get all 20 pieces, know how to decrypt them, put them back together, and get rid of the BS that was filled into them. With them split and going in different ways before going to the same place, and the gov entity not knowing how many pieces there were, they wont even know how many they are looking for.

So

Message - process - FILL with random (Im thinking this is more like salting if im not mistaken). Split into x number of emails Send through multiple servers so not as easily obtained, tracked, or pieced together. Received by the recipient software that can reassemble, take out the BS, and decypher the encryption locally. None of it ever stored anywhere but the 2 persons computers.

When I say fill with bs, I mean like you typed-

Dear John,

Marry had a little lamb. Daddy killed it dead. Marry took it to school next day between two pieces of bread.

Love Bob

You take this and add characters all throughout the email, jumbling the sentences before the messages themselves are even encrypted by some other standard means. And before that standard method is used, the email has been shattered into many many emails of random number before you even wrap it up to be used for transit.

Sorry for the wall of text and probably repeating myself, but I am trying to be thorough in my explanation so as to prevent misunderstandings. I am also typing in a hurry because I have to leave the lab and wont be on for a little while.

Anyway, Im just throwing it out there. Thanks.

</comment><comment><USER2> I would start by reading Bruce Schneier's superb book _Applied Cryptography_ , to see if you're already on a well-trodden path. Computer security and encryption as a technical matter is a very deeply analyzed problem with lots of real world experience.

So basically, don't assume your solution is novel. If it seems novel and you're not an expert, then odds are a: it's not actually novel, or b: it won't work.

 _Applied Cryptography_ is a pretty accessible book. It helps to have a lot of CS and math, but it's not absolutely necessary to get the gist of most of it.

<comment><AUTHOR> Well I have a very basic understanding of the methods I would think to employ, but I appreciate the info. I will look into it. Truth be told, I just happen to try to approach all problems with a fresh mind. Poke around and see if it has been done. I am under no illusion that its the grand fix all, but I always hope when I sit down with someone that they can help me flesh it out, or it at least puts their gears in motion. My limitation is actually going and doing it myself with my shallow knowledge pool. The pitfalls of tryin to dabble in everything and gaining expertise in nothing ;)

Thanks again though!

</comment></comment><comment><USER3> So if it's not commercial, why don't you discuss your idea here, in public? You may also ask a question on Quora.

<comment><AUTHOR> The problem with my ideas usually isnt that they are bad, its that they wont work or they have been done in some way without my knowledge. But Im fine with presenting it here, but I will do so in a non responded thread above.</comment></comment>


Netflix Blog: Tips for High Availability - <AUTHOR> https://medium.com/@NetflixTechBlog/tips-for-high-availability-be0472f2599c <comment><USER2> Interesting they choose to rename blue/green deployments to red/black. Is there some Netflix specific reason for that? Like their logo colors maybe? Red/black is more commonly meant for insecure/secure ([https://en.m.wikipedia.org/wiki/Red/black_concept](https://en.m.wikipedia.org/wiki/Red/black_concept))

Not trying to be a pedant, just curious.

<comment><USER3> Red black is used for all sorts of things. (See red-black tree, roulette, a checkers board, etc). This deployment style has been labeled red-black elsewhere also.

<comment><USER2> Okay, fair enough. We are though, headed into an era of overloaded terms. Which drives confusion and unnecessary arguments. Language is funny and fickle.

<comment><USER4> I'm going to start calling them "sarcoline-coquelicot" deployments now.

</comment></comment></comment></comment><comment><USER5> The article makes it clear Netflix uses Spinnaker a lot for these things. Anyone willing to comment on their Spinnaker usage? Especially interested in smaller deployments.

<comment><USER6> Takes a while to get it going but it does seem very good once it's set up. I had trouble trying to find any guidance on ways of running DB migrations during pipelines, so I'd be very curious as to how other people solve that (as I'm tempted to try it out again).

<comment><USER7> You could trigger a DB migration using a Jenkins step. Alternatively could have a web hook which triggers a migration as a step in the pipeline. Third option would be to run the migration inside the container.

</comment></comment></comment><comment><USER8> Another Netflix piece on failover: [https://opensource.com/article/18/4/how- netflix-does-failove...](https://opensource.com/article/18/4/how-netflix-does- failovers-7-minutes-flat)

<comment><USER9> Thanks for the plug. I'm the author of the failover piece. Happy to answer questions.

<comment><USER2> _" When we failover US-East, we send traffic from the Eastern U.S. to the EU and traffic from South America to US-West. "_

Interesting that you shift US-East to the EU. Any more color on why that direction, versus say, US-West? Like maybe less common components in AWS? At face value, it seems like an odd choice.

<comment><USER10> See slide 5 of this presentation: [https://www.slideshare.net/mobile/InfoQ/chaos-kong- endowing-...](https://www.slideshare.net/mobile/InfoQ/chaos-kong-endowing- netflix-with-antifragility)

<comment><USER11> That "this is what success feels like" slide is mind-boggling. All Netflix clients on all platforms on all devices simultaneously react.

I can only imagine what debugging misbehaving clients looks like, it's got to be north of 1,000 hardware/software profiles.

</comment></comment></comment></comment></comment><comment><USER12> This post should be called "Deploying while maintaining HA" or something.

I thought of some questions:

How is the dev/test env kept up to date for integration/canary tests? When a new app/service is pushed to production does its build/image (AMI?) become the new dev/test env base image for everyone?

Do engineers decide which metrics are tracked by Kayenta? Does ops? How does this come together?

What about post deployment service monitoring / alerting? When is the dev team off the pager hook?

Do you assume that a successful deployment in us-west means it will be successful in us-east or is there an [integration] test done per region?

Just curious.

</comment><comment><USER13> The blog post had a graph in the middle of what I assume was some sort of load metric. It's surprising how regular it was across the week and on the weekend.

<comment><USER10> It appears to be an SPS (stream-starts per second) graph.

[https://medium.com/netflix-techblog/sps-the-pulse-of- netflix...](https://medium.com/netflix-techblog/sps-the-pulse-of-netflix- streaming-ae4db0e05f8a)

<comment><USER14> Hey, One of the authors of the SPS blog post here, as well as the engineer responsible for real-time alerting on SPS.

It is an SPS graph (as indicated by the title). Spinnaker displays the graph on that page to give engineers a visualization for what SPS looks like during their deployment windows. If your service is critical for streaming you'll have a preference for deploying during lower traffic hours to minimize potential impact. Fortunately as the post mentions it is very regular, and has different praks in different aws regions, which allows regional deployments to be staggered.

</comment></comment></comment><comment><USER15> I work at one if the big tech companies. Some of these practices are ingrained in me - even for services that do not promise top tier availability. I just realized I take much of this for granted even though it may not be common knowledge.</comment>


Square Stops Gun Sellers From Using Its Payment Technology - <AUTHOR> http://www.forbes.com/sites/abrambrown/2013/05/14/square-stops-gun-sellers-from-using-its-payment-technology/

<comment><AUTHOR> This is pretty upsetting, I had a lot of respect for Square and what they were doing but it sounds like some lawyer that doesn't like guns decided that to them, 'guns are scary, evil' and "cause physical injury" and therefore should not be a part of Square transactions.

I understand banning illegal services like drugs but are they not aware of the 2nd amendment and that guns are simply a tool? As a firm believer in the 2nd amendment and competitive shooter I know I won't be recommending them anymore.</comment>


Dozens of elephants killed near Botswana wildlife sanctuary - <AUTHOR> https://www.bbc.com/news/world-africa-45396394 <comment><USER2> "Dozens" seems to minimize the true count. The true count killed in this incident is 87 elephants. Additionally, five white rhinos have been poached in three months as-well.</comment>


Michael Arrington should totally hire this idiot? - <AUTHOR> http://www.arringtonpleasehire.me

<comment><USER2> Seems a bit desperate. Not the best way to promote yourself.

<comment><AUTHOR> Not desperate. Just hungry. :)

</comment></comment><comment><USER3> Why not? He doesn't seem any worse than most of the digerati out there. He'll be pivoting at Le Web in no time.</comment>