PhantomJS 2.5.0 beta - <AUTHOR> https://groups.google.com/forum/m/#!topic/phantomjs/AefOuwkgBh0 <comment><USER2> This is just my opinion, but I can see PhantomJS lose a lot of its appeal when the --headless switch for Google Chrome hits the stable channel (quite a way off still, sure). This was already discussed to some extent here: [https://news.ycombinator.com/item?id=11839303](https://news.ycombinator.com/item?id=11839303). Add a couple of porcelain libraries on top of chrome-remote-interface, and you have an even more powerful instrumented headless browser, plus you don't need to worry about security, performance and keeping up with crazy new web features. Also, Chrome headless will encompass features aimed at automation/CI like:

\- "Virtual" time (fast-forward time) \- Session isolation (even for same origin) \- Full page screenshots out of the box

Slides for Chrome headless: [https://docs.google.com/presentation/d/1gqK9F4lGAY3TZudAtdcx...](https://docs.google.com/presentation/d/1gqK9F4lGAY3TZudAtdcxzMQNEE7PcuQrGu83No3l0lw/edit#slide=id.g13338d98ec_0_25)

Shameless plug: I'm using Google Chrome to drive [https://urlscan.io](https://urlscan.io) and will make the switch to the headless version once it's included in the Beta channel.

<comment><USER3> I always disliked working with Phantom. Slow, buggy, crashy, limiting and weird subset of Javascript support with an awful ecosystem of hacks trying to get it usable in Node. I can't wait until a more stable tool superceeds it.

<comment><USER4> Casperjs, which runs on top of PhantomJS has been nicer to use for me. Also Nightmare([http://www.nightmarejs.org/](http://www.nightmarejs.org/)). The problem I always had with PhantomJS is that it's more difficult than it should be to automatize very simple website interactions. In Nightmare's homepage you can see a comparison between a script made in PhantomJS and one in Nightmare.

<comment> fake-name NightmareJS isn't headless, which is a dealbreaker for me.

I'm automating in environments that don't have X.

<comment><USER5> Yeah, that's the problem with Nightmare. It's great otherwise.

<comment><USER6> Witty comment about it being a Nightmare.

</comment></comment></comment></comment></comment></comment><comment><USER7> I wish the PhantomJS project was more upfront about the version of Qt (and therefore Webkit) that ships with each version, like they are in this email. That information is very hard to track down otherwise, and I think it's the most crucial part of each release (since feature support in Phantom tends to lag browsers by several _years_ ).

Additionally, it would be cool for sites like caniuse or MDN to track Qt Webkit feature support as well.

<comment><USER8> I built QtWebKit with roughly the same version used for Phantom 2.5, here's the support from html5test.com:

[https://rm891swy536139ttwzee.mnutt.sandcats.io/853e0e85-170e...](https://rm891swy536139ttwzee.mnutt.sandcats.io/853e0e85-170e-4282-a447-c72633854edf.png)

I specifically left out some audio/video support from my build because I didn't need it. And Qt 5.8 will be released this month with HTTP/2 support.

</comment></comment><comment><USER9> Congrats and thanks to the phantomjs devs for dedicating their time to develop this useful software.

I've used PhantomJS for web app testing, but not as general-purpose browser automation tool. Imperfect, insufficient, and under-maintained as it may be, I appreciate the existence of the Webdriver API and its main implementation based on PhantomJS. It defines a baseline model for web test case development, and a choice of implementations for test execution (recently also directly supported by Safari, I believe).

</comment><comment><USER10> An alternative to PhantomJS is Nightmare ([http://www.nightmarejs.org/](http://www.nightmarejs.org/)), based of Electron.

We're currently using PhantomJS for some headless stuff, but are considering a switch.

<comment><USER11> The trouble with Nightmare and Electron is that they can't be run truly headless, you need to use xvfb with its additional overhead.

There are various requests to have electron made truly headless but its not a core feature that most users want. However we may see it supported (I hope) when the new headless features of Google Chrome make there way down into libchromiumcontent then electron.

<comment><USER12> Not really ... You can run it in software rendering just fine. That is what we do.

</comment></comment></comment><comment><USER13> Also worth mentioning is SlimerJS, which is not headless but allows you to do WebGL screenshots: [http://slimerjs.org/](http://slimerjs.org/)

</comment><comment><USER14> Does anyone have a working version of phantomjs 2.1.1 with ghostdriver that connects to selenium grid?

I tried this one for example but it just crashes randomly when the jobs are running: [https://github.com/AkeemMcLennon/docker-selenium-node- phanto...](https://github.com/AkeemMcLennon/docker-selenium-node-phantomjs)

The latest 2.5.0 beta doesnt have the updated ghostdriver yet.

How much ram does chrome --headless use (true headless not XvfB), because I want to run about 200 of these in parallel and keeping the ram down is important obviously.

Does anyone have any suggestions? I figure by using selenium grid I can run them in docker containers and even on multiple hardware machines.

Currently, I am running about 50 in parallel with Magellan [https://github.com/TestArmada/magellan](https://github.com/TestArmada/magellan)

But its hitting some kind of system limits or sync issues because each process is started before run time rather than ready and waiting in a pool.

Any help on running 200 headless browsers via selenium grid would be greatly appreciated. I think the current selenium chrome Xvbf docker container uses like 300mb+ per instance so that would be 60gb of ram which is a fair bit.

<comment><USER15> I got 2.1.1 working on my raspberry pi. The binary I have is an arm build but I can confirm that the issue can be solved by editing the binary as detailed in [https://gist.github.com/neuro- sys/41e368839a9b20dafb34](https://gist.github.com/neuro- sys/41e368839a9b20dafb34)

I forked the binary from [https://mecrazy.net/2016/06/06/i-built- phantomjs-2-1-1-for-r...](https://mecrazy.net/2016/06/06/i-built- phantomjs-2-1-1-for-raspberry-pi/)

it's available here: [https://github.com/ShamariFeaster/phantomjs- binaries](https://github.com/ShamariFeaster/phantomjs-binaries)

</comment></comment><comment><USER16> How about networking improvements? I gave up on PhantomJS a long time ago because it did not support SNI and was therefore unable to talk to most HTTPS sites. Is that situation better now with this new release and the newer Qt underneath?

<comment><USER17> this is a beta release so you can test those features and provide feedback.

</comment></comment><comment><USER18> Noteworthy -

For an exhaustive list of alternatives, also see: [https://github.com/dhamaniasad/HeadlessBrowsers](https://github.com/dhamaniasad/HeadlessBrowsers)

</comment><comment><USER19> [https://bitbucket.org/lindenlab/dullahan/src/default/example...](https://bitbucket.org/lindenlab/dullahan/src/default/examples/README.md)

Working on this - Dullahan which is a headless browser SDK based on Chromium Embedded Framework and hope to have something like PhantomJS working soon albeit using a modern browser rendering engine.

</comment><comment><USER20> Awesome! I was just doing a PhantomJS using project few weeks ago and the QtWebKit 5.6 upgrades were something I sorely needed. Keep up the good work!

</comment><comment><USER21> I haven't used the new version of QtWebKit yet, but I've heard the changes are pretty huge and they basically bring QtWebKit back up to par with other modern web browsers. It's pretty exciting to see one of the lesser known web browser engines get development like this, especially since it results in improvements to things like qutebrowser (and PhantomJS.)

</comment><comment><USER22> Happy to see this project updated! I use it regularly and currently there's just somethings it can't handle, maybe it can now.

</comment><comment><USER23> This is good news - looking forward to it!

I'm hoping this will eventually speed up my tests during deployment a bit as I currently have to pipe my ES2015 code through a Karma transform to have it readable by my PhantomJS instance.

</comment><comment><USER24> I have really benefited from using this tool on a couple of side projects. If any Devs are reading, thank you for continuing to work on such a useful tool!

</comment><comment><USER25> Do you know why devs is no unresponsive on mailing list and gitgub? There are many pull request to fix bugs and they are simply ignoring them.

<comment><USER26> Volunteers with finite amounts of time.

</comment></comment><comment><USER12> Electron is headless when you pass "show" to be false. Works great and pretty quick.</comment>


Coding school 42 plans to educate 10,000 students in Silicon Valley for free - <AUTHOR> http://techcrunch.com/2016/05/17/coding-school-42-plans-to-educate-10000-students-in-silicon-valley-for-free/ <comment><USER2> So they have a 4-week coding competition, and then at the end, the best students ... get to study programming for 3-5 years? Aren't those the students that _don 't_ need to spend 3-5 years studying programming and could just get a job now?

<comment><USER3> So I spent 3 years in Epitech, which forked into 42, and the first 4 weeks aren't a competition but more like an onboarding thing. You basically learn C in 4 weeks, with coding tests pretty much every day. It sets the base and it turns out to be a great way to quickly know if you really wanna be a software engineer. After these 4 weeks, students start working on group projects and regular CS classes (although teachers are senior students).

</comment></comment><comment><USER4> Oh cool, it's just like Hunger Games. The kids love Hunger Games!

</comment><comment><USER5> What's the catch aka business model?

</comment><comment><USER6> Living in San Francisco for 4 weeks or 4 years costs a lot.

</comment><comment><USER7> dupe: [https://news.ycombinator.com/item?id=11713423](https://news.ycombinator.com/item?id=11713423)</comment>


Alan Donovan and Brian Kernighan Answer Questions on Go - <AUTHOR> http://features.slashdot.org/story/15/11/18/1748247/interviews-alan-donovan-and-brian-kernighan-answer-your-questions <comment><USER2> Go is as far removed from "extreme abstraction" as any other language I know. I don't know how it can even for a second be considered to have that desireable attribute.

I, for one, don't like abstraction and power just because I'm a PL geek. I admit those are also reasons, but the truth is I use them to make my codebase smaller, simpler, and easier to reason about.

For some, achieving that goal means writing for loops. For me, it's never having to write explicit loops.

<comment><USER3> Agreed. There's abstraction and then there's abstraction. I might have been in the camp of Go enthusiasts if my experience with abstraction only included C++ or Java. But once I discovered Standard ML and OCaml I found out how wonderful abstraction can be.

<comment><USER4> I'm not sure many people are arguing that abstraction isn't good for _those who are writing the code_ \--that's the whole point of abstraction after all, to make the developer's job easier.

The problems tend to crop up for other people trying to decipher and read it. Sometimes also for the machine trying to execute it.

</comment></comment></comment><comment><USER5> _> > ..., and extreme abstraction capabilities in high level languages like Common Lisp, ..._

 _> Go has everything you mention in both of your lists of desirable attributes (depending perhaps on what you mean by "extreme abstraction")_

"perhaps"? By whose definition would it ever even come close to "extreme abstraction"? All of us assembly programmers, maybe?

Don't get me wrong, I use Go where it makes sense. It has pros and cons. I'm literally programming Go right now. But "extreme abstraction"... you need more than "perhaps" to qualify that.

Or was that his point and did I just completely miss the joke? Actually, that's probably it.

<comment><USER6> You missed what Donovan said, which answered the question. Obviously PL- enthusiasts (give me all the shiny! all the time!) wont like the answer though.

<comment><USER7> I don't really understand the claim that generic programming is "shiny". It is an old idea that has been implemented many times. It's totally fine that the Go team left out generics – it's their decision – but denigrating people who lament that decision as shiny-chasing PL-enthusiasts is unfair.

</comment></comment></comment><comment><USER8> Kernighan's response to the C question was rather odd -- where is this myth that C is only used in embedded systems and drivers coming from?

The Linux kernel is still written in C.

Ben Klemens, author of 21st Century C, leads the statistical computing group for the research arm of the U.S. Census Bureau. He models complex systems and computations in C.

The entire Python scientific computing stack is resting on C.

A huge portion of the GHC compiler is written in C.

Many AAA game developer studios are writing their engines in C (with vendor supplied C++ compilers, but C none-the-less).

And do people really write C89 for greenfield projects today? C99 is pretty amazing and has added some great features to the language. C11 _might_ be the trivial change having only added some atomic operators (depending on whether you think this is trivial) and rescinding VLAs (though most compilers will probably continue to support them anyway).

I think Go is a perfectly fine language but it seems like perhaps the group is a little out of touch with the rest of the world judging by replies to questions like this.

<comment><USER9> Kernighan wasn't the one claiming that C was mostly used for embedded systems and drivers; that was the guy who posed the question. Kernighan only agreed that C is still popular in that space.

As for C99, I would love to see some hard stats about rate of adoption. Personally I can't imagine starting out on a new project and _not_ use C99, but then the hard-core C contingency is a pretty conservative bunch. (Are there still systems where you can't get a C99 compiler?)

<comment><USER10> > Are there still systems where you can't get a C99 compiler?

In windows it's pretty annoying to get a C99 compiler. The "standard" compiler to do Windows development is Microsoft's, and it doesn't support C99.

Of course you can install MinGW (gcc); most open source software on Windows uses it, but it's very unusual for the average Windows developer to even have gcc, and completely unheard of in the commercial space.

I will also note that installing MinGW in Windows is a pretty painful experience as well, which doesn't help.

For most Windows developers, if it doesn't ship in Visual Studio, it doesn't exist.

<comment><USER11> Don't forget about the Intel C++ Compiler and Clang, never mind more obscure alternatives like TinyCC, Pelles C and Comeau C/C++.

There _are_ options.

Also note that Microsoft has finally started improving C99 support with Visual Studio 2013 and 2015.

<comment><USER12> > Also note that Microsoft has finally started improving C99 support with > Visual Studio 2013 and 2015.

Only as far as C++ standard requires C compatibility.

For Microsoft the future for native programming on Windows are C++ and .NET Native.

For compatibility with open source world and ISV that still care about C, the answer is the integration of Clang frontend with Visual C++'s backend, that is coming with Visual Studio 2015 Update 1.

</comment></comment></comment></comment></comment><comment><USER13> I never knew that the decision to not have versioning built into go was because of the diamond dependency problem. This makes a lot of sense now that I think about it. I wonder what alternatives to manually versioning things the go team is considering.

Also, at the beginning the Go team wasn't all that enthusiastic about an IDE, but its great to see that Go will get an IntelliJ grade IDE soon, so that works for me :)

<comment><USER14> I really am not a fan of the lack of versioning, even with that explanation. Of course diamond dependencies are a problem. But they haven't actually fixed that problem; they've just declared a method for picking one of the two dependencies that is ill defined ("whichever one you happen to have pulled in first"). At least with versioning I get told that there's an incompatibility in the expectations between two libraries.

With Go's removal of version numbers, they haven't removed any of the complexity of resolving incompatibilities, they've just hidden the fact that there is one. I would have much preferred a strategy of maintaining version numbers, and resolving them via "either you specify what version you want pulled in, or we'll build with the most recent and throw a warning".

<comment><USER15> I think that the solution to this problem is one of the few things that nodejs got right. Instead of declaring a global dependency on `require("D")`, you assign it a variable name. Then, each package can declare its sub- dependencies, and even when the versions differ, the different versions are kept in a different scope.

[https://nodejs.org/api/modules.html](https://nodejs.org/api/modules.html)

      // Package A:
      import B from "B"; // A/node_modules/B/1.0.0
      import C from "C"; // A/node_modules/C/1.0.0

      // Package B:
      import D from "D"; // A/node_modules/B/1.0.0/node_modules/D/1.0.0

      // Package C:
      import D from "D"; // A/node_modules/C/1.0.0/node_modules/D/2.0.0

<comment><USER16> I'm convinced this is the right approach: dependencies should be purely local to a module. The tooling should make it easy to detect you have a diamond, but let each module get what it needs. As generally everyone moves towards more continuous update and integration, at the end of the day it's all gonna be sha's anyhow.

<comment><USER17> But what do you do in a typed language? For example, suppose you have modules A and B, which depend on module C. Let's also say that module A has a function foo that return a value of type C.t, and module B has a function bar that accepts a value of type C.t.

Now let's say that module A is depending on version 1.0.0 of module C, and module B is depending on version 1.5.2 of module C.

Does B.bar(A.foo()) still work? Is type t the same in C version 1.0.0 and 1.5.2?

<comment><USER18> Good point. But if B.bar accepts an interface that C.t implements, then everything should work just fine and that should be the way to go on public APIs.

<comment><USER17> But what if the interface is defined in C? The interface has to live somewhere, and wouldn't it be versioned too?

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> Here's a solution to the diamond dependency problem: Imports can only be done for a specific lexical scope. Also, imports must specify a particular version. Lastly, calls to a package go to a specific version of the package, and types are defined in a specific version. (This is why all imports must be done for a specific lexical scope.)

The same scheme could be used to resolve diamond dependencies in multiple inheritance as well.

<comment><USER20> That's essentially the solution Rust and Cargo adopt. It occasionally causes strange errors like "expected Foo, found Foo" (where the first Foo was, say, Foo version 1.1 and the second was Foo version 1.2), but the compiler now detects this situation and tries to explain what's going on.

I'm certain I'm biased, but in my experience it's been the most robust way to handle versioning that I've dealt with.

<comment><USER19> I proposed this in a Smalltalk newsgroup over a decade ago. The reaction? I was ridiculed.

Imports naming specific versions would also allow better integration of code/build into a version control system. Actually, the entire OS should be built with version control in mind from the ground up.

<comment><USER21> No truly good idea will ever escape ridicule.

</comment></comment></comment></comment><comment><USER22> The recently open sourced VSCode also has Go support now. Good first impression. The docs and type inference stuff are more convenient compared to the oracle plugin + GoSublime for sublime it seems.

<comment><USER23> I thought it was just syntax highlighting. Any link documenting VSCode's Go suppport?

<comment><USER12> There is a video of the presentation:

[https://channel9.msdn.com/Events/Visual-Studio/Connect- event...](https://channel9.msdn.com/Events/Visual-Studio/Connect- event-2015/060)

<comment><USER24> Jump to 2:15, great to see!

</comment></comment></comment></comment><comment><USER25> "The reason it took God only six days to create the universe is that he didn't have to deal with the embedded base."

</comment><comment><USER26> Nice interview. Plenty of good details. I mean, there was this slip:

"The languages... are either long gone (PL/1) "

[http://www-03.ibm.com/software/products/en/plicompfami](http://www-03.ibm.com/software/products/en/plicompfami)

[http://www.fujitsu.com/fts/products/computing/servers/mainfr...](http://www.fujitsu.com/fts/products/computing/servers/mainframe/bs2000/software/programming/pl1.html)

[http://www.iron-spring.com/about.html](http://www.iron-spring.com/about.html)

Next he's going to be telling you COBOL is gone, too. Big time writers' definition of dead/gone in IT has always seemed different than general usage. ;)

</comment><comment><USER27> This us utterly hypnotic. Nice work.

</comment><comment><USER28> > In general, Go strongly encourages being explicit about errors.

That's completely incorrect: it's trivial (and common) to just ignore errors in Go:

        ok, _ := Foo()

Checked exceptions don't let you get away with this kind of sloppy programming.

<comment><USER23> It's also trivial to "just ignore errors" in a language based on checked exceptions:

        try {
            ok = foo()
        } catch (Exception e) {
            // Do nothing
        }

Checked exceptions don't save you from "this kind of sloppy programming".

<comment><USER28> But you have to write the `catch` (or a `throws`). That's the point.

Now if you choose to handle this stupidly, that's entirely your fault, but at least the compiler did its jobs by forcing you to think about the error case.

In Go, the compiler doesn't enforce anything.

<comment><USER23> How many lines of Go have you written before writing this?

In Go, as in Java with checked exceptions, the compiler forces the programmer to handle the error.

Using _ in Go is not idiomatic. It's the exact equivalent of using a try/catch block with nothing in the catch block in Java.

<comment><USER20> If you call a function that returns nothing but an error, Go does not force you to acknowledge the error's existence and will silently allow you to drop it.</comment></comment></comment></comment></comment>


A Simple Explanation: VLC.js - <AUTHOR> http://ascii.textfiles.com/archives/5089 <comment><USER3> At what point will the current trend to treat the browser and WebAssembly as a compile target for traditionally fat client applications raise the need for a browser package manager ala Apt or Yum? When will dynamic linking become a necessity because due to memory usage, disk space, and patches for 0day exploits in upstream libraries?

It seems like the more the browser takes on the role of an OS the more it will need all the stuff that OS's have built up over the years to make it manageable.

<comment><USER4> We have (or had) that. Chrome is removing web app support (the Apps button on your favorites bar). Firefox removed their app support after FxOS died.

Nobody wanted these things. Users didn't understand the idea. The APIs were clumsy. The framing of the apps added little value over a traditional web app. Portability was a joke.

The best thing to happen to the web is service workers and the new world of "offline" APIs, like the push notification API. They allow pages to get the goodness of a desktop app without the downsides of installing a "native" app.

(source: I helped build the Firefox Marketplace)

<comment><USER3> Those are more of an app store and less of a package manager. Apt and Yum solve a problem users don't know they have. The app stores solve the problem of obtaining and installing the application. They don't solve the problem of managing shared libraries and quickly updating those shared libraries.

For an application like VLC this sort of thing become really important. More important than your typical email client or document editor.

<comment><USER4> On the web though, you don't need the user to manage libraries. In the same way you don't install a web page, libraries are simple script tags pointing at URLs. Pages will never reference incompatible versions. Browser caching makes "sharing" of libraries transparent. What's there left to manage?

<comment><USER3> Is this still true for an application like VLC compiling to WebAssembly that uses other downstream libraries?

What if VLC uses downstream libx and Audacity also uses downstream libx and downstream libx has a critical 0day. I'm not super clear yet on how the WebAssembly build pipelines works. Is dynamic linking done through script tags? or are the binary blobs statically compiled? I'm genuinely curious here how this is all supposed to work. How does one ensure that downstream libx is updated for _all_ of the WebAssembly applications that use them.

</comment></comment></comment></comment></comment><comment><USER5> > I firmly believe that this project, fundamentally, would change the > relationship of audio/video to the web.

Aren't they banging open doors? HTML5 video and audio is great and with WebAudio and WebGL you can manipulate every aspect of the experience. EQ, filters are possible. What else do you need? Subtitles?

I just don't understand what problem are they trying to solve because "I can't run VLC in the browser" certainly isn't one.

<comment><USER6> You can even get subtitles on top of HTML5 videos with simple positioned elements with text shadows for borders. I can't say I understand the motive of the project either, other than "see, everything can be done in JavaScript!"

<comment><USER7> Or just use a `<track kind="subtitles">` and reference a vtt file?

<comment><USER8> Which doesn't allow SRT subtitles, or XML subtitles (hello German public broadcasters!), or ASS subtitles (hello anime fandub groups!)

Doing this natively in JS via canvas allows you exactly that - rendering these things natively, in real time, with all the support for rare features you could ever need.

</comment></comment></comment></comment><comment><USER9> From the previous discussion:

vespakoen noted that there is already ffmpeg library compiled with emscripten:

[http://bgrins.github.io/videoconverter.js/demo](http://bgrins.github.io/videoconverter.js/demo)

My speed test of webm to mp4 conversion (it doesn't display the video, it's just ffmpeg) showed the JS version _around 10 times_ slower than the native ffmpeg.

The test was on relatively recent i7 and the latest Firefox (certainly having as fast asm.js as possible) then (around a month ago). I haven't investigated the cause for such significant speed difference.

<comment><USER10> I have a project that decodes mpegts packets and renders the frames to a webGL canvas using FFMpeg. [https://github.com/colek42/streamingDemo](https://github.com/colek42/streamingDemo). Full disclosure much of the client code is copy/pasta from [https://github.com/opensensorhub](https://github.com/opensensorhub)

Edit: add attribution

<comment><USER11> is there a live demo some/anywhere?

<comment><USER10> Sorry, I am using it to proxy UDP streams which requires a backend component due to browser restrictions. If I hosted a demo it may end up costing a nontrivial amount of money. I am developing this closed source for my employer, but if there is enough interest I can push some of the changes to the open source repo.`

</comment></comment></comment></comment><comment><USER12> Power consumption and performance are big concerns, because JavaScript (software) decoders can't leverage hardware acceleration.

But archive.org wants to preserve and publish many formats that are not high resolution, high frame-rate video, and we can expect both performance and power consumption to improve over time.

<comment><USER13> HTML5 canvas can use hardware acceleration, of course. If the codec being used is suitable for HW acceleration (i.e. h/x624 yes, VP9 no - generally)

</comment></comment><comment><USER14> But wouldn't a browser plugin be easier to develop and also easier to use for the end user? Exactly what is gained by having it run in js? (Better portability maybe? But VLC already runs on almost all platforms natively...?)

<comment><USER15> Because every browser has different plugins and their concept tends to change every few years, for good (security) or bad (politics) reasons. Once VLC.js runs, you can reasonably expect it to run forever and be just another compilation target.

<comment><USER16> Exactly.

For example, Mozilla is deprecating NPAPI just as fast as it can, because it long predates e10s (Chrome-style process isolation) and they do not play well together. e10s has been so long delayed already, and Firefox's performance problems are so intractable to solve without it, that from the perspective of a very long-time Firefox user like me this is very obviously the correct course of action.

This deprecation will break the existing VLC plugin for Firefox, but that's not a huge deal because it never has worked very well in any case - <USER2> a reliable NPAPI plugin is a seriously hard problem in general and has been since browsers got complicated enough to do interesting things in the first place. You also have to restart the browser to make any changes to the set of installed NPAPI plugins, and that's a lot more of a problem than it was before the browser became an application runtime. Now that the JS runtime is fast and capable enough to run "real application" code, we derive a much better experience by leveraging that power than we do by targeting a legacy interface that is relatively ill documented and much easier to get wrong in ways that break the whole browser rather than just one page or tab. We also get cross- browser compatibility of a sort that NPAPI never could offer.

I get that a lot of people think it's weird for the web browser's page scripting language, which was nothing more than a cute toy a couple decades ago, to be eating the world like this. I think it's weird, and it's how I earn a living! But, weird or not, it's also the best application platform we have or likely soon will have for a broad and widening variety of use cases, and that is because, while no more perfect than anything else, it actually works really well. Railing bitterly against technology that works really well seems potentially counterproductive to me. But I appreciate there exists a diversity of opinion on the subject.

</comment></comment></comment><comment><USER17> Does this mean that developers will go as far as maintaining a port of FFMPEG to JavaScript? Being able to decode 10bit videos in pure JS sounds both insane and insanely cool

<comment><USER9> > a port of FFMPEG to JavaScript

It already exists, see my other comment for more details.

<comment><USER18> There are a number of ports powered by Emscripten ([https://github.com/Kagami/ffmpeg.js](https://github.com/Kagami/ffmpeg.js))

I wrote about some stuff here [https://paul.kinlan.me/ffmpeg- ideas/](https://paul.kinlan.me/ffmpeg-ideas/) when building [https://paulkinlan.github.io/deviceframe.es/](https://paulkinlan.github.io/deviceframe.es/) (I do a lot of screencasts and need a simple way to scale this out to other people that doesn't involve installing ffmpeg and scripting)

EDIT and Note: Also I just saw your comment about videoconvert.js

<comment><USER9> Do I understand it correctly that you want the users to do javascript ffmpeg processing on their android devices? I admit I have no idea what deviceframe is, so I'd also like to read some explanation on that too.

Have you compared the speed of Android javascript ffmpeg vs PC native? What happens with the battery?

<comment><USER19> Javascript and as soon as possible WebAssembly, as I understand it.

I think mobile is a secondary consideration in this at this point. Considering it is connected to the internet archival efforts and simply ensuring the survival of numerous video/audio formats I imagine they are fine if it isn't efficient but simply effective at this point.

Edit: on second thought, you might not be commenting on the VLC.js thing, I haven't read about the ffmpeg js stuff ;)

<comment><USER9> Sadly, that answer is a completely missed conversation topic.

I've asked about

[https://paulkinlan.github.io/deviceframe.es/](https://paulkinlan.github.io/deviceframe.es/)

</comment></comment></comment></comment></comment></comment><comment><USER20> This is one of the reason I'm truly excited about the future of javascript in its ES6 or Typescript form.

If they manage to make this run and use emscripten and manage to build all the video computation optimizations.. how far of a stretch is it really to combine lodash+emscripten+BLAS/LAPACK to build a high performance computing platform based on javascript ?

<comment><USER21> Honest question: Why is building an HPC platform on a language designed in less than 2 weeks something to aspire to?

Why not aspire to replace javascript entirely with something better?

<comment><USER20> huh?

ES6/emscripten/Typescript was not designed in 2 weeks. could you talk about what you mean.

<comment><USER22> He's talking about the first JS specification which was done in 2 weeks IIRC. I've seen such comments several times which completely discard all the effort made to improve JS until now.

</comment></comment></comment></comment><comment><USER23> Wouldn't webxmp (also emscripten based) be a better match for MOD/S3M/XM support:

[http://www.wothke.ch/webxmp/](http://www.wothke.ch/webxmp/)

<comment><USER24> That's really neat! I still have a few music modules I can't part with.

</comment></comment><comment><USER25> This is very exciting and to a lot of people, doing things with only a browser is so much enabling!</comment>


Report on the technical vulnerabilities found in Huawei 5G [pdf] - <AUTHOR> https://finitestate.io/wp-content/uploads/2019/06/Finite-State-SCA1-Final.pdf <comment><USER2> "On dozens of occasions, Huawei engineers disguised known unsafe functions (such as memcpy) as the "safe" version (memcpy_s) by creating wrapper functions with the "safe" name but none of the safety checks. This leads to thousands of vulnerable conditions in their code."

Things like this everywhere. Just stupid programmers or method?

<comment><USER3> To actually _fix_ this, you have to call memcpy_s instead of memcpy, and then _not_ create the wrapper, which seems like almost the same amount of work, but slightly less. So why create the wrapper? Their library didn't have a memcpy_s function?

<comment><USER4> Huawei is actually one of the very few vendors which do provide the Annex K functions. I can only name Cisco, Intel, Embarcadero, HP and Microsoft who also do so. But Intel very badly.

The report is right to call out Huawei engineers to bypass these checks. They probably had the same attitude to bounds checks as all open source developers and industry practioners.

</comment></comment></comment><comment><USER5> It seems odd to have a report with a 5G label and then find information in it about core routers. Yes they play a part but it strikes me as odd that there is no focus on mobile switching, gateways, HLRs, signaling vulnerabilites etc..

When looking at the boxes they compare:

Huawei: CloudEngine 12800 scales up to 2000 * 25GBit or 500 * 40GBit Juniper: EX4650 Ethernet Switch scales up 48*25GBit

With all due respect but this is comparing apples with orange seeds. A carrier class switch is more complex than an enterprise switch and has a much larger attack surface.

<comment><USER6> Its probably a shared codebase, I wouldn't be surprised if they had a monolithic OS and just had specific daemons to run things.

<comment><USER7> Shared codebase + more offload ASIC processors is exactly what I'd expect

<comment><USER6> Exactly.

Most network forwarding happens on ASICs, its the only reasonable way to get performance.

</comment></comment></comment></comment><comment><USER8> > While we cannot prove malicious intent through a technical analysis, we can > concretely state that 55% of tested devices had at least one potential > backdoor

<comment><USER9> And: "In virtually all categories we examined, Huawei devices were found to be less secure than those from other vendors making similar devices."

<comment><USER10> I was wondering about how bad this was against industrial standards, considering they've been articles about TP-Link and other Internet-of-Trash devices that have a lot of unpatched issues.

The article seems to suggest Hawaii is exceptionally bad at patching or best practices in security.

<comment><USER11> It's a low bar, but apparently they managed to not clear it.

</comment></comment></comment></comment><comment><USER12> Not since the days of Microsoft astro-turfing have I seen such a campaign against a tech company. Some of these metrics are very subjective and I've seen so many bone-headed security bugs in equipment over the years (Cisco, Nortel, TP link, Dlink...) that I really have difficulty believing the narrative of state sponsored malice.

Regardless, Huawei needs to clean up their practices. Ineptitude isn't a defense and this level of sloppiness is unacceptable; We have too much at stake with these essential networks, and I think legislation of security practices and standards in these products should be considered.

<comment><USER13> > Regardless, Huawei needs to clean up their practices.

Sure. So does every crappy home router manufacturer, Android manufacturer, security cam manufacturer etc: singling out Huawei is retarded.

Personally I suspect that biggest issue is that the US doesn't wish to lose their own NSA phone/txt backdoor access (thumbs in everyone's pies). They have access to most US TCP/IP traffic, so they don't need to target router manufacturers.

<comment><USER12> Yes, but this is actually an opportunity for Huawei. If I was them, I would take these findings and hire a team of hackers/sec researches to rip their products apart, fix the issues and then put then up for independent comparison with competitors. They have all of the resources one could dream if to be the best for security. It would be nice if anyone would set a high bar.

</comment></comment></comment><comment><USER14> Some of these items seem like they need more analysis, and that the report was rushed. We agree as a community that hard coded default passwords are bad when unchanged, but that implies that the device doesn't require them to be changed on first boot. Basic setup processes often require these credentials to change after the first provision.

I would expect a report which produces claims about such comprehensive backdooring and negligence to at least demonstrate how that behavior would play out in the real world. It seems much more like they did static analysis of the binaries, identified any strings with default passwords, and then reported on them. That's okay to do, but you can't conclude that's a problem until you confirm shoddy behavior after provisioning and deployment in practice.

</comment><comment><USER15> The process and practice findings are the most damning in my opinion. This isn't a opps we were rushing to market, this is a problem with values and in my mind damns the whole organization.

<comment><USER10> Basic security tooling to remove hard coded keys of any kind need to be in the process. I bet shops like Cisco/Linksys, Neatgear and others at least make some attempt of this in their CI process.

<comment><USER16> No

[https://nvd.nist.gov/vuln/detail/CVE-2019-1804](https://nvd.nist.gov/vuln/detail/CVE-2019-1804)

</comment></comment></comment><comment><USER17> The huawei user account hardcoded in the firmware that has sudoers permissions for insmod/modprobe is really blatant. I would have guessed the Chinese would at least attempt to hide their way in?

So they sell enterprise infrastructure equipment with such an obvious backdoor? It seems the US boycott of Huawei was fully justified after all, I didn't really believe them.

<comment><USER18> That doesn't have to be a backdoor. Remember the wave of bugs caused by hardcoded secrets in Cisco hardware?

Might just be some leftover debugging account or for service.

<comment><USER19> >That doesn't have to be a backdoor.

It is a backdoor. The definition of the term doesn't require intent.

</comment></comment></comment><comment><USER20> The report says that Huawei devices have unsafe functions like "memcpy" and "strcpy". Is this a coding preference or dangerous practice? To what extent can these examples reflect on code quality?

<comment><USER21> strcpy() - sure it is fairly often used unsafely, strlcpy() or snprintf() or similar should generally be used. But memcpy()? It takes an explicit length, which you have to calculate. What's the alternative? memcpy_s() does part of the check for you, but you end up writing more code around it. It's trying too hard and not achieving a net positive. I have seen "use of unsafe function memcpy()" show up in some dumb security scans recently, and it's a strange development. There is lots of C code where avoiding memcpy() would be quite awkward and really not help anything at all, it's just a fundamental operation.

Here's a dry technical report on how the new-ish _s variants are not really helpful: [http://www.open- std.org/jtc1/sc22/wg14/www/docs/n1967.htm](http://www.open- std.org/jtc1/sc22/wg14/www/docs/n1967.htm)

<comment><USER4> This technical report is nonsensical, and asks for continuation of widespread insecure practices. The authors (the glibc maintainer) should be removed from the committee for spreading such harm. He simply doesn't like secure practices, and continue to use sometimes checked calls. If he would have some technical competence he would implement the needed safety checks seperated into compile-time (no performance penalty) and runtime (when the compiler doesn't know). He only does the first, and leaves all the dynamic cases unchecked. But then he would stumble over the inability of gcc to properly handle compile-time expressions.

See eg. [https://rurban.github.io/safeclib/doc/safec-3.5/index.html](https://rurban.github.io/safeclib/doc/safec-3.5/index.html)

</comment></comment></comment><comment><USER6> I'm so glad this isn't about how "5G is dangerous zomg".

We're lucky to get this sort of report!

(In case you were looking for an article about the safety of 5G -> [https://medium.com/@tomsparks/is-5g-dangerous-405a19e9ea88](https://medium.com/@tomsparks/is-5g-dangerous-405a19e9ea88))

</comment><comment><USER22> It's hard to say to which extend this "assessment" is honestly backed by solid evidence, or if it was written first and foremost for political reasons. The broad claims, apparent lack of factual evidence (only statements about what has been discovered), and rather overarching nature of the whole report, sure do suggest it could be the latter. Pure speculation on my own part, of course.

I have not gone through the full report (yet), but while reading just a few sections my sales-pitch-bullshit-meter went on full tilt, at least a few times.

The timing of this report is also peculiar, to say the least. Who is Finite State? What is their track record, to date? Who owns the company? What business relationships/interests do the company and its owners have?

Yes, I realize/know these are suggestive questions. Questions that should nonetheless have satisfying/assuring answers, in order take this report seriously.

There's a lot at stake here, so at least examining the validity of this report/summary isn't just a luxury.

Are the factual findings behind this report publicly available?

<comment><USER23> They did it using their analysis automation software (iotasphere) and the list of devices is in appendix A.

Someone knowledgeable about forensics with access to the same devices probably can validate those claims.

<comment><USER22> I sure hope that someone will. I do not have access these devices, neither do I have any inclination to buying their commercial product in order to verify their claims.

Maybe their product is brilliant, and/or maybe they did a lot of manual verification of their automated scans, in which case their report could be accurate (though still hard to verify).

That said, my experience with both static analysis and runtime security scanner is less than stellar. They are without a doubt useful tools, if not indispensable. But they usually generate more leads for further (manual) analyses, rather than generate (accurate) final verdicts.

Granted that pen-testing is a different animal (but one I have more experience with), I often have had to explain to customers that the reports they got from security auditing companies were full of false conclusion. More often than not because those companies were relying/trusting too much on automatically generated results, in addition to them trying to sell additional services. So, forgive me if I'm skeptical.

That Finite State apparently has a stake of their own here (promotion of their product), which makes me only less inclined towards giving them the benefit of a doubt.

As I mentioned before, this is of course all speculation on my part. Still, until Finite State publishes their factual/detailed scan results, I believe I have every reason to consider that this might be either a political motivated report, or more likely be an attempt to use/abuse the current political climate to promote their own product and services.

</comment></comment></comment><comment><USER24> _• 29% of all devices tested had at least one default username and password stored in the firmware, enabling access to the device if administrators don't change these credentials.

• We identified 76 instances of firmware where the device was, by default, configured such that a root user with a hard-coded password could log in over the SSH protocol, providing for default backdoor access.

• 8 different firmware images were found to have pre-computed authorized_keys hard coded into the firmware, enabling backdoor access to the holder of the private key.

• 424 different firmware images contained hardcoded private SSH keys, which can enable a man-in-the-middle to manipulate and/or decrypt traffic going to the device._

What a witch hunt... This is state of the art in the industry. Everybody does it like that. No intelligence agency has to be involved at all, it's basic negligence. If you're behind a NAT, your device is unlikely to be attacked via these vectors.

<comment><USER25> Exactly, every vendor has default passwords. If you are doing things right all default credentials are changed or disabled. Access to any service is firewalled off or better yet only accessible out of band on a dedicated management network.

<comment><USER10> This is changing. I've seen a number of ISPs no longer have default passwords. Each router or modem has a random password string set to the device, it's printed out and pasted as a sticker on the modem (or some print directly to the plastic). A lot of big name devices do this now too.

Sure it's a password written on the device, but it's random, you need physical access to see it, and people who are security conscious can change it.

This bad practice isn't excusable, especially not by a company as big as Huawai, not if they want to be taken seriously.

<comment><USER22> It's definitely a good development that ISPs have started to deploy routers and modems with randomized passwords. However, please do keep in mind the deployment of consumer equipment and enterprise hardware is different. Or at least it should be, in theory.

Enterprise equipment is usually not supposed to be just dropped into place, without oversight. It usually needs proper configuration/management, by qualified people.

Whether this also happens in practice can be a different story altogether. Still, the security of enterprise equipment usually involves more policy and procedure than it does with consumer equipment. With the latter, security has to come more or less by default, because the people handling the devices usually have little expert knowledge.

</comment></comment></comment></comment><comment><USER26> It is not the bug now but bug in the future. It is whether you can trust a country that respect nothing human.

Can you even a campaign against anything once they take over your information infrastructure.

Yes NSA may do it but cf NSA with PLA ... what is your leverage. Hear of the word totalitarian...</comment>


Ask HN: What framework/api do you use for building Augmented Reality apps - <AUTHOR> <comment><USER2> For our latest project we use a new framework from Pikkart. It's really good and it's cheaper than other commercial alternatives. You can find it @ [https://developer.pikkart.com](https://developer.pikkart.com)</comment>


Newlib-nano – Embedded C library for smaller systems - <AUTHOR> https://keithp.com/newlib-nano/ <comment><USER2> I'm confused. The ARM toolchain already offers newlib-nano and ARM is maintaining it since several years ago. What's the relationship between OP and ARM's newlib-nano?

If OP is a independent work from ARM, then he choose a bad name.

[1]: [https://community.arm.com/iot/embedded/b/embedded- blog/posts...](https://community.arm.com/iot/embedded/b/embedded- blog/posts/shrink-your-mcu-code-size-with-gcc-arm-embedded-4-7)

</comment><comment><USER3> I looked around a bit and quite a lot of this appears to be from NetBSD?

</comment><comment><USER4> What's this for? It doesn't seem to be the same as musl or uclibc, and the only time I've come across newlib (I think) is when I was compiling wget.

<comment><USER5> I think you were seeing gnulib, not newlib.

<comment><USER4> Thanks</comment></comment></comment>


Ask HN: Programmers who worked at small and big companies, which do you prefer? - <AUTHOR> As someone a little disillusioned by the advertised advantages of working at a startup, I'm curious to hear the perceived trade-offs as well as how people's preferences led them to prefer startups, mega-corps, or anywhere in-between. Or, if company size isn't a factor to you when seeking out a company I'd be interested to hear why.<p>Obviously mileage will vary based on the company, but try to generalize if possible. <comment><USER4> Depends on what you're after. Small companies can be very nimble, enabling you to have more of an impact within the company as you wear many hats. Sometimes less is more and you'll perform better when you're forced to work within staffing, budget or time constraints. Often a small company can feel like family. Sometimes stock options can turn into a golden ticket. But with all that comes additional stress and sometimes not great base pay.

Large companies some of the above can still remain true, you get job security through company stability, total comp packages can be much higher and more reliable. Company budgets are bigger so your products can sometimes have a much larger customer impact globally. More availability to move between teams and meet new coworkers or switch offices without much risk.

Medium sized companies are my favorite where the hair on fire foundation was already laid, product market fit is establish, but company is still private and growing quickly. Less risk here but still decent upside and less of the large company regulatory and compliance annoyances that impede velocity.

And in my experience sometimes the size of the company has less to do with quality of life than work culture.

</comment><comment><USER5> I prefer the big ones. They have a lot of money, so they can afford to move slower. Also, there's ton of inefficiences inside which means that they you really can't be expected to perform too well. All this adds up to ending the day much more rested that in a startup.

</comment><comment><USER6> I worked at FAANG as well as several small startups (<100 people).

The perhaps biggest downside of working in a big company is politics. You are no longer working to provide value to customers. Instead, you are playing a game of politics to get ahead of your peers. Sometimes these are correlated - you can get ahead by providing value to end-users, but often there is no correlation at all. After a while this really started to get to me to the point I could no longer stand it.

In terms of learning, I found both environments to be similar _if you are motivated_. It's true that in a startup you are forced to wear many hats and learn about things outside of your comfort zone. However, big companies will often give you these opportunities as well, they just won't force you take them. It's easy to become complacent. In a startup environment you can't do that.

In terms of risk-adjusted payoff, working for a big company is probably a much better choice than working for a startup.

Personally I prefer startups or small companies at this point, I don't think I could deal with the politics in large corporations again.

</comment><comment><USER7> Startups are faster paced and provide faster general skills growth.

For example I was running a team of five devs a year after graduating - <USER2> would not have happened at a big company.

I also got the chance to architect complex systems within my first 2 years - again that would not have happened at a big company.

On the other hand by moving to a large tech company my pay _quadrupled_ and I was able to focus on growing my pure coding ability.

I've also had the opportunity to solve complex dev problems that we would have just ignored/hacked around in a startup.

Probably my most frustrating experience is that big tech companies are biased towards big tech company experience. My extensive management experience was heavily discounted and I had to join as an IC. It took me almost 5 years to get to the same management level (director) at my current big tech (non-FAANG) company.

</comment><comment><USER8> I worked at two Fortune 500 (> 50,000 people) public companies and now work at a medium-small sized private company (~500 people). I prefer medium-small company. Also - what number would be small? like < 100?

The only advantage to large companies is if there is a large presence in your area, moving internally between jobs is relatively easy if you want to change things up.

Advantages of small company:

\- Privately owned so no stockholder BS or the constant downsizing/offshoring accounting games.

\- Most time-wasting corporate stuff is very minimal at a small company. At large corp you could spend hours upon hours doing the mandatory annual security training which ironically required Flash installed to run the big brother tracking video course. Same for ethics, HR and other corporate-y trainings.

\- No real politics. Large corp it was common to run into people that didn't contribute anything to the products/solutions. It appeared they existed to slow things down and act out Dilbert comic strips. This can really wear on you. Watch the movie Office Space.

\- Want to try a new tech out? Talk with a couple people, give it a whirl - <USER3> has to have some purpose though. Large corp: "no, we use XYZ, we've always used XYZ"

</comment><comment><USER9> I personally hate big companies. I don't like red tape, and they've got lots of it. Also, I hate politics, and they have lots of that too.

I _love_ small companies. Clearly, not _any_ company will do - there are small companies that generate quite a bit of profit, so you don't need to worry about tomorrow as long as you're doing your job.

I'm not into startups - AT ALL - (at least, the US version of them), which basically pretty much promote "work your ass off - overtime, and the like -for pretty much low pay", with the possibility of making it big being insanely small.

(disclaimer: I do have a company, with a single employee - me, but I don't see myself as a startup; I do have collaborators; I don't ask for overtime, we always agree on "this is the task at hand, and this is what that will cost me")

</comment><comment><USER10> You want to join medium-size organizations (< 1000). Big companies have too much red tape, small companies either do not pay very well or do not have enough ability/knowledge to execute.

In medium-sized org, you have more room to grow in terms of projects, responsibility and money. Usually, you have fewer limitations and your projects would have a higher impact. General engineering culture is still the most important factor following compensation. You want to find a place where you will get paid well and you will learn the best practices and tech.

</comment><comment><USER11> I prefer to have more control over my career than some (and less than others.) By control I don't mean that I get to choose what I am currently working on but rather that I can work towards what I will be working on in the future. Mid to large companies that are both well managed and filled with highly capable people tend to enable and encourage this as it can bake in resiliency and helps different parts from becoming siloed.

</comment><comment><USER12> I prefer working for tiny companies. The last company I worked for had only 3 employees including myself. (Not startups though.)

With a small company, the software is not a "cost center".

Everyone has to pull an oar. You can tell pretty quick in say a 4 person company if someone isn't contributing.

No room for politics.</comment>


Understanding the Node.js Event Loop - <AUTHOR> https://nodesource.com/blog/understanding-the-nodejs-event-loop

<comment><USER2> This is a nicely consumable lightweight summary of how the node event loop works. In my experience this is one of the hardest things for new JS developers to actually understand, especially those coming to the language with a limited asynchronous programming background. Here's a good link to how the browser event loop works for those interested: [http://blog.carbonfive.com/2013/10/27/the-javascript- event-l...](http://blog.carbonfive.com/2013/10/27/the-javascript-event-loop- explained/)

IMHO, there's just a few things you need to understand to start getting productive with js fast, #1 on the list is to know how the event loop in the browser/node works and this is a really good intro to the node side of things. #2 is event bubbling/capturing in the DOM ([http://javascript.info/tutorial/bubbling-and- capturing](http://javascript.info/tutorial/bubbling-and-capturing)), and #3 would be everything about functions / function scoping.

Those are the parts I always try to explain first to devs I've mentored who were just jumping in to javascript.

</comment><comment><USER3> The only thing that annoys me with this (and many other popular posts in our field recently) is that the author talks about "magic". I believe this started with the advent of Rails where some things seemed "magical" at first due to Ruby's meta-programming features. Many writers and speakers have adopted it since.

However, there is nothing occult about software development and we shouldn't use "magic" as an excuse for not understanding or describing things thoroughly. Just imagine your doctor explaining you a surgery in these terms.

Otherwise, a very good and refreshing read.

<comment><USER4> Right, I agree with this. There's nothing magic about any of this -- ultimately it's all just software, hardware and physics.

To describe something as magic in a serious context is toxic. We're not sleight of hand performers, we're engineers. It's more helpful to state something like, "this library is complicated and the internals are beyond the scope of this post". A pointer to documentation and source code on libuv would be even better. In face, this series of articles on libuv are a great start for those who are interested: [https://nikhilm.github.io/uvbook/introduction.html](https://nikhilm.github.io/uvbook/introduction.html)

<comment><USER5> If all occurrences of "this is [magic]" are treated as "this is [sufficiently complicated internally and beyond the scope of this post]," then what's the difference? The former makes for a point that's expressed more succinctly and eloquently at little real loss.

<comment><USER4> I went and looked in the Oxford English Dictionary and did not find your definition of magic there. [http://www.oed.com/search?searchType=dictionary&q=magic&_sea...](http://www.oed.com/search?searchType=dictionary&q=magic&_searchBtn=Search)

<comment><USER6> Definition 2a seems related: "Producing [...] remarkable results, like those attributed to magic [...]; effecting or permitting change, success, etc., as if by magic."

Also the 'magic box'.

The Urban Dictionary (the place where I look for colloquialisms) has this definition: "with or through complexity either spurios to the contemproary context or to long to be reasonable or desired of explanation". [1]

1: [http://www.urbandictionary.com/define.php?term=magic&defid=9...](http://www.urbandictionary.com/define.php?term=magic&defid=994242)

</comment></comment></comment></comment></comment><comment><USER7> There's a great talk from jsconfeu that describes the event loop in more detail. It helped me understand what I subconsciously had picked up about how Javascript works: [http://2014.jsconf.eu/speakers/philip-roberts-what-the- heck-...](http://2014.jsconf.eu/speakers/philip-roberts-what-the-heck-is-the- event-loop-anyway.html)

<comment><USER8> Thank you. That was a fantastic talk. Short, to the point and useful :)

</comment></comment><comment><USER9> Another important concept for JS devs to understand is (macro)tasks and microtasks. This is becoming especially important in ES6 and beyond (due to Promises leveraging microtasks).

[http://stackoverflow.com/questions/25915634/difference- betwe...](http://stackoverflow.com/questions/25915634/difference-between- microtask-and-macrotask-within-an-event-loop-context) has a decent overview

<comment><USER10> Case in point which tripped me up recently. Knowing that promise.then() callbacks aren't called synchronously[1], even if the promise is already resolved, I assumed that this:

        var tick = Promise.resolve();
        async function cpuBoundJob() {
          for (...many iterations...) {
            // crunch some bits
            await tick;
          }
        }

...would spread out a heavy calculation and allow other things to break in occasionally and do work. But based on tests, nothing ever got a chance to run until the whole job finished.

My current understanding (correct me if I'm wrong) is that promise.then() calls you back from a microtask, which means "await tick" doesn't teleport you into the next tick, just out of the current call stack into a new one at the end of the _same tick_ , regardless of how many things are piled up waiting on the next tick. In other words the entire calculation remains one big CPU- blocking chunk. This modified version ended up working as expected, since setImmediate is (apparently) not a microtask.

        var tick = () => new Promise(res => setImmediate(res))
        async function cpuBoundJob() {
          for (...many iterations...) {
            // crunch some bits
            await tick();
          }
        }

[1] [https://promisesaplus.com/#point-34](https://promisesaplus.com/#point-34)

</comment></comment><comment><USER11> Nice description. The Node event loop pretty much defines the general shape of all APIs that work in Node. When we were porting HyperDex to Node, the greatest challenge was to integrate the data store with the async event loop [1]. The resulting callback oriented programs look strange at first to people coming from a traditional thread-oriented background, but they have an elegance of their own.

[1] [http://hyperdex.org/doc/latest/NodeAPI/](http://hyperdex.org/doc/latest/NodeAPI/)

<comment><USER12> btw, the params order for callbacks is not very node-y in HyperDex. It should be (err, result) instead of (result, err).

Putting the error first would enable Async or Promise libs to work...

</comment></comment><comment><USER13> > emitter.emit('fire')

I think it should be this.emit('fire')

</comment><comment><USER14> Great

</comment><comment><USER15> > _Node utilizes as much of what 's already available from the operating > system's kernel as possible._

And yet we've seen a proliferation of fibre/coroutine/green thread libraries for Node which all reinvent the concept of threading poorly.

<comment><USER16> Node's event loop is inherently cooperative multitasking. It's not trivial to simulate preemptive multitasking without writing/compiling your code to giant state machines. There are different use cases for the two, especially since any CPU-bound task performed in the event loop will cause stuttering in your app.

</comment></comment><comment><USER17> At first glance the lower half of the P was obscured by dust on my screen. 'Understanding the Node.js Event Looo' is a much better title.</comment>


Security of BIOS/UEFI System Firmware from Attacker and Defender Perspectives - <AUTHOR> https://github.com/advanced-threat-research/firmware-security-training <comment><USER2> An this is why I'm trying very hard to get coreboot to work on my systems, and I'm very eager to see Libreboot-type of FSP/BSP-free images in the future so we can actually verify our boot chain.

Keeping all of the firmware code secret makes no sense, and seems to be artificially enforced by 'patents', and 'trade secrets' while most likely the vendors are trying to keep buggy code secret and hope that obscurity alone will keep them safe.

<comment><USER3> In my opinion, vendors are oblivious to the notion of the buggy code. From the releasing-a-product standpoint, there is a tested or an untested code. While personally I wouldn't be surprised that someone could knowingly release firmware containing known "stability issues", I doubt any vendor hides source code out of fear that someone else will discover some hypothetical bugs. Among other problems, finding bugs in an unknown firmawre is not so simple, as it requires an intimate knowledge of the underlying hardware (and a fair share of the bugs is closely related to its quirks), which might be not so familiar as, say, well-established (and relatively well-documented) Intel Architecture.

</comment></comment><comment><USER4> Seeing slides in BIOS-UEFI-Security.6-Mitigations.pdf that imply that there's critical crypto being done in SMM mode makes me feel unfathomable hopelessness. The whole x86_64 platform security model (which includes all the privilege levels and the corresponding access control mechanisms) is one hell of an overgrown clusterfuck that could not be _more_ hostile to formal verification.

There's lots of wantonly convoluted stuff going on -- a random example is the access control mechanism for the PCH's GPIOs (search "GPIO registers lockdown", in quotes). This isn't, of course, implemented with a register which contains bitfields determining which privilege level can write to which sets of registers. That would be too simple. The GPIO Lockdown-Enable bit _can_ [sic] be changed by the same software that the lockdown mechanism is meant to deny access to! This seems like utter pointlessness -- what use is an access control mechanism if the agent being restricted can change the parameters of the mechanism willy-nilly -- but Intel has a solution! Changing the lockdown-enable bit triggers an SMI and shunts control to SMM mode, which is, naturally, expected to include code that figures out that this bit should not be disabled, and is to flip it back on (and return from the interrupt, having trashed the caches a bit).

This is, of course, a pathologically needless level of complexity -- in the ARM world, we have registers with silly names like "NSACR" that higher- privileged execution modes can set to restrict access to certain resources. There's certainly no BIOS-OEM-provided code that needs to exist and be correct in order to implement such a basic task. In the end, this level of access- control is equivalent to a bloody Boolean operation or two, for heaven's sake! All the CPU needs to do is to decode the instruction, realise it's a potentially-privileged instruction, decode what the instruction tries to modify, look up in a table which register holds the relevant permissions bitfield, and do the relevant boolean operation between that register's contents and an appropriate bitmask, and fault depending on the result. Since there's an extremely close match between the properties that needs to hold (the truth table of all combinations of "privilege level _x_ can modify resource _y_ iff bit _n_ in register _z_ is set") and the mechanism that enforces it, it's easy to reason about this scheme and not difficult to either prove an implementation correct (or find counterexamples).

Meanwhile, the "wake up SMM and hope it'll countermands the illegal write" scheme depends on a lot more machinery. How does it work on a multi- core/multi-socket platform? How does this mechanism interact with the caches or the memory model? Is it possible to set up a race condition where the illegal write ends up going through uncountermanded because SMM mode can be made to not see the register in an illegal state? This is orders of magnitude of orders of magnitude more complex than analysing a lookup-table and a bitmask -- we need to understand the semantics of memory reads/writes, of caching, of mode switches to SMM and the SMI interrupt, and how all of this clusterfuck is affected by the fact that there's _multiple cores_ in our system. LANGSEC people will call this a "shotgun parser" \-- when input data checking / recognition is interspersed with processing logic.

Even if all of this miraculously works and there's literally no way that all the cores working together can send an illegal write that SMM code won't countermand -- there's still the issue of making sure that the specific SMM blob that our BIOS OEM wrote cooperates properly with this. Indeed, making BIOS OEMs implement these sorts of convoluted and critical mechanisms and expecting them to get all of them perfectly right requires a level of optimism that doesn't yet exist. The situation has devolved to the point that there is literally a tool called "chipsec" that lets you test for the presence of a handful of well-known security-critical things (from time to time someone discovers a new one, of course, UEFI/ACPI and the x86_64 privilege model is too complex for people to be sure that we found all the issues) that UEFI programmers are _notorious_ for messing up. That this tool needs to exist is shameful. Of course, the security of the x86_64 platform doesn't just depend on a bunch of magic access control registers being set right, there's Turing- complete code that needs to be implemented by the BIOS OEM (and runs in the most privileged execution mode that isn't Intel's "management processor") that is _security-critical_ , and, well, it's hard to prove that arbitrary Turing- complete code behaves correctly.

The auxiliary CPU mode (SMM) initially meant to hide APM and emulate PS/2 mice in 90s-era computers is now _critical_ to platform security, and does dangerous stuff like crypto and handling pointers from UEFI / the OS. Every few months someone I follow on Twitter finds some new way to trick some widely-deployed SMM code into writing to a memory region it shouldn't, it's quite depressing. Great. Another pointless defender/attacker arms race that the defenders could decisively win had Intel thrown away the spitefully complex intricacies of SMM and the x86 security model and replaced it with a clean, formal-verification-friendly set of privilege levels whose correct operation doesn't depend on platform firmware code. Even AArch64 is less broken when it comes to this.</comment>


Ask HN: valuing a non-existent company - <AUTHOR>

Hi guys. Occasional commenter posting anonymously. We are in an interesting situation and I would appreciate advice.
Our core business is in a niche area, but we have the opportunity to get some angel funding to enter a related but much larger market. We are planning to enter this market in the long run anyway. Moving now would provide some risk diversification, give us the cash to cover upfront costs and provide some miscellaneous administrative support. Also first mover advantage. The business model would be validated early, or fail early. We'll know in a few thousand dollars so it isn't really high risk.
We could take this step on our own, but we're bootstrapping and can't afford to split our resources to support two products. So the advantage is really avoiding a delay of at least six to eight months. There is no risk of dilution to our existing business, so that is not a concern.
The question is how to value an angel investment in a business like this where there is a small initial investment for market testing and an agreement for how to move forward if it makes sense. To me this means figuring out a reasonable value for our own contribution. But what is that? The costs of building the tech side from scratch? The opportunity cost of the people who need to manage/execute? Does the market value of our labor even matter if we are taking equity?
I understand that the answer is always "we need more details", but it is hard to do that without being overly specific. Apologies for being obtruse. I'd appreciate hearing about any relevant factors that came into play in any similar agreements any of you have run into. <comment><USER2> I heard an Angel say "pick a number between 3 and 5 million and move on". I think the point is that the investors need enough equity to have a good upside (50x $) and the company needs to maintain enough equity to be motivated (the majority).

So, if you are shooting for the fences, how much do you think the company can be worth in 5 years? What percentage will give the angel 50x their investment?

I am looking forward to what others have to say.

</comment><comment><USER3> Why not skirt the question and take the funding on a convertible note?

<comment><USER4> Exactly.

I think a reasonable deal is a 30% discount, ~5% interest, and a big bonus if there's never a valuation.</comment></comment>


Gobs of data - <AUTHOR> http://blog.golang.org/2011/03/gobs-of-data.html

<comment><USER2> I find the discussion of misfeatures in protocol buffers to be interesting. I'd always found default values and required fields to be superficially appealing but questionable in practice, and it's kinda nice to see that validated by someone with an insider perspective.

</comment><comment><USER3> Since I started watching Go evolve, I've stopped wondering what I would want from a new programming language. Seems like they have a pretty good idea already.</comment>


RESTful API Design: can your API give developers just the information they need? - <AUTHOR> http://blog.apigee.com/detail/restful_api_design_can_your_api_give_developers_just_the_information/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+ApigeeBlog+%28Apigee+Blog%29 <comment><USER2> REST is the "literally" of the web world.

I think they mean "RPC over HTTP API Design".</comment>


Find Tech jobs from companies sponsoring visas - <AUTHOR> https://techmeabroad.com <comment><USER2> I think that this website is a very great idea... I am a web developer, but I am also passionnate about career advices... I can give a real sample to explain why I am so happy to talk about this new website.

I have a great network of web developpers, system administrators... living of North Africa... Here in Paris, we need them, but it's not so easy to hire them...A website that focus only on companies that are ready to help you sponsoring visa is a really great idea.

In another way, On my linkedIn account, I often receive propostions from recruters (dubai, asia,...) or startups in other countries that are looking for webdeveloppers. Now if I receive one, I'll forward them to techmeabroad...

Regards, Ben Younes Ousama.

</comment><comment><USER3> The whole approach of finding a job which sponsors a visa seems very wrong to me.

I'd rather focus on working on what I'm passionate about.

Once you got that figured out, if you just put your time and effort to become one of the best at what you do, the bureaucratic obstacles become merely a distraction.

</comment><comment><USER4> nice one</comment>


How do you add value to your network? - <AUTHOR> Hey guys,<p>What does "adding value to others" mean to you? How do you maintain relationships with clients, mentors, peers, and so on? <comment><USER2> It takes work!

While you don't hit it off with everyone, you'll likely find that there are some amount of folks you meet at conferences, in the community or otherwise that you could see spending time with or getting to know.

Keep updated with those people, follow them on social media, send them articles or links of things that you know they're interested in and call them from time to time.

Beyond that, if you can place facilitator of needs, awesome. You know x person has a need for a person with skill y, and you z person with y skill. Link them up!

This doesn't come without some level of intentionality. The main thing though is to build relationships and community with no expectation of a return. Genuinely do it because you want to model the type of behavior you want to see in others.

</comment><comment><USER3> How do you add value to people who give you money?

That's the same way you'll add value to others as well.

I'm a problem solver. I'll help people that do not give me money by solving some of their problems. It could be code, it could be knowledge or it could be connections.

I'll do that for free because I like helping people and because someday they might help me.

</comment><comment><USER4> The easiest thing you can do is be really nice and supportive. There's a shortage of genuinely nice people, and it helps whether you're at the top or bottom.</comment>


Show HN: Bit Gold and Bitcoin - <AUTHOR> https://medium.com/@insearchofsatoshi/bit-gold-and-bitcoin-9357176cd420 <comment><AUTHOR> In 2005 Nick Szabo drafted but did not publish a Bit Gold whitepaper. For a short time it was available on his website, and although he did not publish it, it is still available on archive.org. This essay discusses the light this previously unknown paper sheds on the relationship between Bit Gold and Bitcoin and how, if Szabo were Satoshi, he might have evolved the ideas in the Bit Gold whitepaper to developer Bitcoin several years later.</comment>


Experimental C++11 multi-platform 3D engine - <AUTHOR> https://github.com/floooh/oryol <comment><USER2> This is by the same person that was behind "The Nebula Device" and "Nebula 2" and "Nebula 3" at Radon Labs.

Oryol is a great new take on some of the core concepts of Nebula 3, but in a C++11 environment and being able to rely upon more modern compilers, libraries, etc.

<comment><USER3> BruceM(itchener), I presume?

If so, I remember you from my nebula hacking days 10-15 years ago. Man, I'm so old.

<comment><USER2> That's me!

</comment></comment></comment><comment><USER4> I've spent the last few weeks to move the python build scripts into its own project ([https://github.com/floooh/fips](https://github.com/floooh/fips)), that's why the Oryol master branch is a bit quiet at the moment, the build system stuff happens on the 'fips' branch (but this is only tested on OSX so far).

<comment><USER5> I was intrigued by the bullet in your manifesto about using C++ "because Rust is not yet ready to replace C++".

Obviously not disagreeing given that Rust is still pre-1.0, but I'd be interested in hearing what you see as the big stumbling blocks; is it just library/tooling/ecosystem maturity, or do you have major reservations about the language as it stands?

<comment><USER4> Rust is currently developing so fast that this bullet point in the manifesto is nearly obsolete. C/C++ currently still has the edge for multiplatform- support (emscripten, PNaCl, iOS, Android(?), game consoles), but that's about the only remaining point. With Rust it is also easy to integrate existing C libraries (e.g. middleware libraries for game development), which is usually the main problem when choosing a language other them C/C++ for game development.

<comment><USER5> Yeah, I'm not sure what the current outlook is for Rust+Emscripten. I'd hoped that the recent runtime removal might put that back on the table, but haven't seen any explicit confirmation of that.

Rust programs also seem to produce big binaries, even with LTO, which would be another strike against it for web delivery.

<comment><USER6> Rust's big binaries (these days, about 500kb for hello world) are due to the combination of two factors: 1) it statically-links by default (if you opt for dynamic linking, you get the typical C-esque 10kb binaries) and 2) it ships jemalloc as a custom allocator rather than relying on the system allocator. Considering that jemalloc would be unnecessary for any program targeting a web-based platform, you'd be able to strip down the compiled size considerably.

<comment><USER5> Is jemalloc's footprint really in the hundreds of kilobytes?

<comment><USER6> As source I cite this PR that just landed today:

[https://github.com/rust-lang/rust/pull/20218](https://github.com/rust- lang/rust/pull/20218)

 _" It's quite possible that small programs don't use all of jemalloc, and building with -ffunction-sections and -fdata-sections allows the linker (via --gc-sections) to strip out all unused code at link time. This decreases the size of a "hello world" executable for me from 716K to 482K with no measurable impact on link time. After this patch jemalloc is still the largest portion of our hello world executables, but this helps cut down on the size at least somewhat!"_

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> That'sa some nice code you've got there! You should share this over at [http://www.reddit.com/r/gamedev](http://www.reddit.com/r/gamedev) They might appreciate it more than the HN crowd.

</comment><comment><USER8> This looks nice. I'd like to try it out. Seems to be taking a sort of opposite approach to Polycode's maximalism. But if you're interested in this lib, I'd say, give Polycode a look over too.

It runs and builds on OS X/Windows/Linux, and includes a built-in content IDE, and full lua integration, with a lua script editor built in the IDE— theoretically you can make whole games from within Polycode's IDE, but I personally prefer to work with it in raw C++, and use the IDE for creating and importing content for a game.

[https://github.com/ivansafrin/Polycode](https://github.com/ivansafrin/Polycode)

Polycode is currently lacking support for web, iOS, and Android, but it's on the roadmap.

Anyway, nice engine you've got going! Gonna have to try out the demos and see how it works!

</comment><comment><USER9> This looks super cool, I love that it works with both emscripten AND pnacl.

</comment><comment><USER10> Being a fan of things such as MOAI and openresty/luvit, which are essentially C/C++ engines within a Lua body, I am of the opinion that C/C++ frameworks still have a lot of life in them. And there is something very fresh about oryol .. its indeed a very usable framework as well as a decent code base, from the perspective of contributing ...</comment>


Taking Control of Container Placement - <AUTHOR> https://medium.com/containership-articles/taking-control-of-container-placement-dafed5b54383

<comment><USER2> This seems really powerful compared to other solutions I've used. Placing containers using ECS has been a huge pain point. I'd be interested in trying it out and look forward to more constraints in the future.

<comment><AUTHOR> Thanks! We have experience with ECS as well and have found tags and constraints to make life a lot easier when you are trying to containerize everything.</comment></comment>


Predicting the future of the Internet is easy - <AUTHOR> http://cdixon.org/2011/01/13/predicting-the-future-of-the-internet-is-easy-anything-it-hasnt-yet-dramatically-transformed-it-will/ <comment><USER3> Some day, real people (non-techies, non-Westerners, people who are not 20-something males, etc) will almost certainly be about as plugged in as the mean HNer is currently. I cannot conceive of this happening without generating staggering amounts of wealth.

Groupon just invented a new industry with yearly sales which are quickly going to eclipse 10^10 dollars by figuring out how to do distribution for local businesses over the Internet. I think in 2020 that will be remembered by historians with special interest in modestly successful advances. Somebody is going to do something as revolutionary as Groupon/Google/Facebook to a market as big as "healthcare" or "India", and (to misquote an amusing line from Social Network), they'll be able to buy Harvard and use it as their pingpong table.

</comment><comment><USER4> It's easy to forget how young and immature the internet and even the software industry is. Software development is still in its alchemy stages, with as many (if not more) snake oil salesmen as there are people doing sound, repeatable engineering and science. And as much content as is on the internet today it's still a far cry from all of it, which is a near inevitability at this point. Every book, every magazine, every audio recording, every movie, every tv show, every home video, every bit of information that was ever typed or written onto a piece of paper, everything. As we get closer to that things will change, we have only just barely and experimentally started putting anything of substance online and the result has been staggering.

Nobody has seen anything yet.

</comment><comment><USER5> Im not sure the last time Chris made the front page of HN, but for some reason reading this was absolutely a breath of fresh air for me. The internet will keep doing its thing whether facebook IPOs or the 'bubble' bursts. Focus on your own work, the stuff that really matters, and you will inevitably succeed. Whatever the next big wave is, you'll miss it if you're too busy fretting about what has already come and gone.

</comment><comment><USER6> I'm not sure finance & law are of no value. Whatever may be wrong with the financial system, it isn't entirely maintained for the benefit of $GS (or hate figure du jour). I can imagine some of its functions replaced by less formal structures though.

Law is much harder to see replaced, even if flawed & incredibly inefficient. Law of little value? Shall we try without?

<comment><USER7> With simpler laws, you might get by with more stupid lawyers.

Like, managing an index fund can be a part time job or even done by a computer, compare that to the hot shots that hedge funds attract for their active trading.

</comment></comment><comment><USER8> _convincing the upcoming generation to innovate in sectors that have a direct impact on the quality of peoples' lives (Internet, healthcare, energy, education) instead of wasting time on sectors that were historically prestigious (e.g. finance and law) but add little to negative economic and societal value._

And that, my friends, is why we're in this mess to begin with - <USER2> that don't generate any progress, and therefore should not become profit centers. Just like ISP's and other utilities, in an idea world they should be regulated and forced to compete on a fair and open market.

<comment><USER9> that was my favorite line as well. now that startup pursuits have become semi- mainstream, i think you'll see the next generation of parents less inclined to push their kids down a carved out path.

if parents really want to see their kids "make them proud," they won't push them towards a law degree...but rather encourage them to change the world.

one of these days, these law students will figure out that there's just not as much work as there once was and we could really use their expertise to move the ball forward instead of slowing things down.

</comment></comment><comment><USER10> He highlights finance as one of the industries inevitably disrupted. I hope he's right but I have personally always viewed finance industry profits as a tax that cannot be avoided at any cost less than the cost of starting over. Is there hope that I'm mistaken?

<comment><USER11> One possible example that already happened: The internet contributed to the more-or-less death of the traditional full-service stock broker in the late-90s, with the rise of electronic discount brokerages like E*Trade.

I do agree that it's not clear it has or will in aggregate reduce the finance industry's take, though. More comprehensive disruption of the finance sector by technology would find a way to reduce finance's total cost, the way VOIP completely disrupted traditional telecoms. Finance firms seem remarkably adept at keeping the total take from falling overall, though, even when particular fees are forced downwards.

<comment><USER12> Much of the mathematics of finance is actually as new as the internet. It has yet to be automated...

Also, the mathematics of finance is still very, very much under development. Volatility of volatility model, anybody?

</comment></comment></comment><comment><USER13> Ha, I liked this one: "we'll look back in amazement that in 2011 we still used brokers to help us find houses" - I haven't used a broker to _find_ a house in 12 years, actually.</comment>