Product Hunt's Response to Accusations of Exclusivity Is to Increase Exclusivity - <AUTHOR> https://medium.com/@minimaxir/product-hunt-s-response-to-accusations-of-exclusivity-is-to-increase-exclusivity-3758ac23f152 <comment><USER2> My main issue with Product Hunt is that most of the products are so trivial that it almost seems like a parody site.

A sample from right now:

'The funniest dog name generator ever'

'Fast and easy way of collecting email addresses'

'Finally, an emoji keyboard for the modern desktop'

Don't people have more interesting things to do with their time, or try to create more interesting, useful products?

<comment><USER3> Product Hunt's greatest success is being the perfect reflection of the Silicon Valley industry and mindset that surrounds it. Whether you consider this a feature or a bug is a personal judgement.

Personally I'm glad it exists because it seems to divert at least some of the vacuous SV sycophancy away from Hacker News.

<comment><USER4> Your first paragraph deserves to live in a poster.

</comment></comment></comment><comment><USER5> >Submitters without any network at all are _screwed._

If success in becoming featured on Product Hunt is contingent on the submitter's network, then the submitter's product is probably of such quality that they're screwed anyways.

That, or their product is solid but it simply isn't appealing to the PH crowd. You see cool stuff every day that fits that description, usually accumulating a mere handful of votes. Generally speaking, the more niche the product is, the more likely it is to end up in that category.

Product Hunt is ruthless curation. It isn't designed to be fair. Suffice it to say, exceptional products—or products that make waves elsewhere—are almost assuredly bound to do well on Product Hunt, regardless if the author even has an account there, let alone a network.

To be clear, I'm not defending insider voting rings, I'm just saying the best strategy is to trancend them entirely.

<comment><USER6> The difference is where the product shows up on the PH page.

With a network, you automatically get access to the front page. Without it, you won't even show up on the site even if it's the best product in the world for the PH crowd.

<comment><USER5> It'd still be posted on Product Hunt though, despite no self-submission.

That said, banking on PH to carry your initial launch is a bad idea if you've no network there, let alone posting ability.

</comment></comment></comment><comment><USER7> This is exactly what happened to Digg. You had people getting paid to get stuff up on Digg's front page. So Digg introduced friends and the ability to see what friends are digging. And they banded together to really ruin the experience for others.

During the time it happened, I did a bit of offhand calculation and found that of all the posts on Digg's front page, most had the same 100 or so "power diggers" in the first 200 likes of the submission. And these 100 users were authors of at least half the submissions on the front page, many of which were clear promotional ads.

</comment><comment><USER8> So people are more worried on being featured on PH than having a profitable business? Am I the only one who couldn't care less about PH?

<comment><USER9> Way too many idiots playing entrepreneur. My customers aren't on PH. Most, if any "customers" on PH are short lived.

<comment><USER8> I honestly don't get it. Maybe is my fault, I don't know, maybe I'm missing something.

<comment><USER10> You're not missing anything. Product Hunt will fade away as the VC bubble continues to deflate.

<comment><USER9> This. Don't get attached to PH, when the bubble is a poppin' PH will be gone.

</comment></comment></comment></comment></comment><comment><USER11> It's sad. The response PH gave in December dodged the true question of _how do things get to the homepage_ [1]. Defenders stuck to the concept of "Great products will make it to the homepage"–in reality, everyone knows that's complete BS.

I find myself going to PH weekly now instead of daily. It's almost predictable the things you'll see there now (yet another mockup tool, "ProductHunt for X", trivial toy app, etc.).

Maybe a site like this simply can't scale and keep this format.

[1] [https://twitter.com/rrhoover/status/679150836862218241](https://twitter.com/rrhoover/status/679150836862218241)

</comment><comment><USER12> Product Hunt is the mirror image of Silicon Valley startup VC scene. It's all about connections, nepotism, pedigree, ass-kissing and brown-nosing.

</comment><comment><USER10> Product Hunt's problem is that even though a less exclusive/biased system would create more value for users, such a system wouldn't allow PH to capture much, if any, of that value for themselves. By enabling biases in their listings, they're able to shift some of that value from the users over to themselves. It's good for the companies that get more attention than they deserve, and most consumers won't find out due to information asymmetry and apathy.

It's the same reason why you can't trust Yelp reviews - they needed to generate revenue somehow, and the only real option was to offer to hide negative reviews in return for money.

<comment><USER13> _the only real option was to offer to hide negative reviews in return for money._

I don't understand this. Why couldn't they sell "you seem to be looking for a dentist... this other dentist (who paid for this advert) has a higher rating than the one you're looking at right now"? It seems that Yelp could have been in the enviable position of selling advertising which is actually useful to the viewer.

<comment><USER14> Because then why would any dentist who is not at the top pay for advertising?

<comment><USER13> Because they would gain customers who were previously looking at inferior-to- them (according to Yelp ratings) dentists.

</comment></comment></comment></comment><comment><USER15> It's too bad OpenHunt didn't take off. But of course why would it? With no public commenting system it simply isn't "sticky" enough.

[https://www.openhunt.co/](https://www.openhunt.co/)

HN discussion: [https://news.ycombinator.com/item?id=10759879](https://news.ycombinator.com/item?id=10759879)

<comment><USER16> Yeh but it also takes a substantial, sustained effort to build a community and after the initial launch the OpenHunt crew stopped.

[https://www.openhunt.co/audit](https://www.openhunt.co/audit)

Maybe no spam for all of Feb?

</comment></comment><comment><USER17> > The front page real estate taken by the in-network posts comes at the > expense of those out-of-network in a zero-sum game. Submitters without any > network at all are screwed.

I don't know if that assessment is true. I don't have a network. If I submit something to PH and someone with a network upvotes it, their network is now more likely to see it. So if you create something cool, that a few people with networks discover on the upcoming feed, you're chances to reach even more people and get the foot in the door (and land on the frontpage) are higher now, with this new front page algorithm, no?

<comment><AUTHOR> Granted, if you don't have a network, your submission is stuck in the "upcoming" purgatory, so that issue already blocks the probability of a serendipitous upvote.

</comment></comment><comment><USER18> Thanks for the analysis. I'd be curious to see PH's traffic. I get the vibe from checking # of retweets/hearts on their tweets that PH has peaked and are now on the descent.</comment>


SQLite4: The Design of SQLite4 - <AUTHOR> https://sqlite.org/src4/doc/trunk/www/design.wiki <comment><USER2> When should I use SQLite3 and when SQLite4? Are there any special use-cases for one or the other? Or should I just use 4 from now on, since it's simply "better" than 3? Didn't read like that's the case.

<comment><USER3> I'm really not sure what the status of SQLite4 is at the moment. Has it been released?</comment></comment>


A 1400€ metronome, gently offered by Dell - <AUTHOR> http://foxtn.tumblr.com/post/47548667423/a-1400eur-metronome-gently-offered-by-dell

<comment><USER2> You know, it might be more useful to write up the entire episode in a single relatively brief article, rather than keep it in its present disconnected cliffhanger form.

It's an interesting story, but it would be more useful if it were written as a single article with no repeated sections.

<comment><AUTHOR> ok, this story is going on now, I can't write the future (for now).

I'm keeping the first post updated but I added a shorter one for describing this last episode.</comment></comment>


Domain profile reports on the "Decisions that cost money" - <AUTHOR> http://thechangelog.com/post/2911083036/domain-profiler-get-domain-profile-reports-on-the

<comment><AUTHOR> I think this would be best delivered as a gem too. Forking and pulling down a copy is a good start, but having this as a gem would make me smile.

</comment><comment><USER2> (Just an FYI.. it doesn't seem to work on Ruby 1.9.2, though 1.8.7 was OK.)

<comment><USER3> Please send me a pull request if you're able to figure out what is going on :(

<comment><USER2> I didn't really have time to dig into it but.. I'll have a quick look for 5 minutes now anyway. I'm on 1.9.2 by default so might easily be able to see the problem.

UPDATE: Ok, you have a pull request now ;-) Got it working. Specs now pass.</comment></comment></comment>


Doing work locally minimizes problems with Electron apps - <AUTHOR> https://jlongster.com/secret-of-good-electron-apps <comment><AUTHOR> I'm the author of the post and I should also say that I don't love Electron. It's a practical choice for getting an app of the ground, but I want something more lightweight eventually.

I see it as a stop-gap until I can invest more resources into something more lean. But I'm focused on the product right now, not rewriting the world and wasting time because of engineering ideals.

<comment><USER2> I agree with your statement but just have never really seen it work that way in real life.

Slack is the best example. An app that just 'works' in Electron and the stop- gap became the permanent solution for eternity et voila -> still huge valuation and IPO.

I am afraid once you go Electron, most will stay Electron.

<comment><AUTHOR> Whatever it ends up being it has to be web-based. I'm definitely not going to rewrite the app.

However, if something were to come along that uses the user's local webview instead and provided a way to bundle a node server with it, I could absolutely switch to it.

In fact, I've played with the idea of providing an app that only sits in your menubar and runs the server, and opening the app just opens a tab in your browser. But most users don't care as much about all this as we think and would find that odd.

You're right though - if something is working well enough it probably will end up sticking. Slack just goes to show that it's probably Good Enough despite what we feel about it.

<comment><USER3> Someone tried that with Electrino, but there hasn't been any activity in 2 years.

That only deals with memory and disk use though. CPU and battery use under 'load' will still be through the roof compared to native apps. I put load in quotes because even entering text or scrolling creates massive energy consumption spikes on Electron. Oh and your UI will still be sluggish too.

To be clear: I'm not hating on your app or choice of Electron, just be aware most of the drawbacks are here to stay. Even Microsoft with all their engineering prowess can't seem to fix them (see VScode vs Sublime), and they are already using a heavily customized and optimized Electron.

</comment></comment></comment></comment><comment><USER4> The problem I see with many electron applications isn't just that they're bloated (they probably are in about the same relation to average RAM that Tcl/Tk apps were or early Gnome/Mono ones). [Edit: To clarify, I mean the average RAM at the time they were accused of bloat]

It's that they're quite non-native. And not just in the "button borders are 1 pixel wider than usual" sense -- they're usually styled to resemble web applications, with big data views, Material UI, etc.

Going from this to something more lightweight/native doesn't just require a different API, it requires a completely redesign.

In other words: The Big Rewrite That We're Definitely Going To Do Someday(tm).

<comment><AUTHOR> I agree with this too but this problem can be addressed by caring. I care. You can find a bunch of non-native interactions in my app but every time I work on it I make it better and implement more of a native feel (like disabling selecting text when you shouldn't be able to, etc).

It's a balance - personally I think apps that focus on a specific platform is exclusionary. So accepting some non-native feel means more people can use my app.

I agree that browsers make it hard though.

<comment><USER5> > like disabling selecting text when you shouldn't be able to

This is the worst thing in computers. Why would you EVER _block_ the ability to select text? I am engaged enough to want to select something you made! Embrace that, don't stop it!

<comment><USER6> Think of all the thousands of tiny snippets of text and single Unicode characters that make up a UI. You definitely don't want to be able to select snippets within most controls, because selection _interferes_ with using the control.

Examples: the text in a menu item, the text in a button, the Unicode x in a close icon, the bar between menus.

This is really noticeable in a few web applications where you can select the wrong thing and it interferes severely with using the UI (I have experienced this with Windows and Android).

Yes, you usually want to be able to select the main text. However many mobile UI frameworks just disable all selection, because that is the easiest way to also disable selection within UI controls (managing this issue is actually quite difficult from my experience writing a HTML UI framework).

<comment><USER7> In other words, HTML UI frameworks are still far behind native frameworks and actually not really usable yet.

<comment><USER6> In other words, poorly written components don't work well: which happens in other GUI toolkits and not just HTML.

Good HTML UI frameworks don't make this mistake.

<comment><USER7> Are there any good HTML UI frameworks yet?

<comment><USER6> So long as you don't want an OS native UI, yes I think so.

For example I use Visual Studio Code every day and the UI seems pretty good to me (panels, menus, tabs, check boxes, combos etc). I am presuming it uses a component framework (although I admit I haven't looked at the source).

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> So, I love the role that Electron fits, but boy do I hate how much we discuss Electron itself.

What we should be discussing is providing the functionality of Electron without the cost. A protocol that behaves like React native. Ideally a spec, which any language can implement and provide a native UX to the end user.

Some projects try (ProtonNative, for example), but I've not seen one go quite far enough. I don't want to be forced to write a JS app to get benefits here. After all, with ReactNative, it's basically a process RPC to a bunch of widgets. `process <-> UI`. So why be forced to write JS? An open spec would do wonders here.

I've not invested cycles on solving this problem, of course. Yet I can't help but wonder why so many people are focused on solving UI issues with very narrow solutions like Electron, ReactNative, ProtonNative and so forth. They're great projects, but a slightly wider vision would make them amazing projects.

Thoughts? Am I missing something obvious?

<comment><USER9> >I don't want to be forced to write a JS app to get benefits here.

Although most major Electron apps are written in node, it's pretty easy to have a backend in another (compiled) language. You can simply use the subprocess module in node and build your own bridge, for example to a Go CLI that accepts events from electron and does whatever you want it to do. It's a workaround for sure, but the point is you could do it in a pretty stable way.

>After all, with ReactNative, it's basically a process RPC to a bunch of widgets

>Yet I can't help but wonder why so many people are focused on solving UI issues with very narrow solutions like Electron, ReactNative, ProtonNative and so forth

I've used React Native in the past and I'm working on a Flutter app now. The problem with RN is that every platform has its own unique widgets. It's neat that you have one codebase, but you still need to build a separate UI (at least for certain pages) for Android and iOS. It just doesn't look right otherwise.

Flutter gets away with this because it owns the drawing layer as well as the widget layer. I know that something drawn in Flutter is going to look exactly the same no matter where I run it. This means that widgets fail gracefully by default - in RN, things just straight up break or look weird. The worst that can happen in Flutter is that the user has to deal with an iOS picker on Android or vice versa.

Electron is more akin to Flutter than RN in this way. It controls the drawing/rendering because of Chromium. Slack on Mac looks exactly like Slack on Windows. That's more enticing than needing to build new UIs for every platform you want to support.

The only way to solve this is to basically invent another cross platform drawing layer. But why bother? HTML/CSS already exists. There's effort and tooling and docs and community and everything else you'd need for a successful framework. It's not worth the effort.

I think the best solution will be just embedding Electron at the OS layer, so apps don't have to bring along their own Chromium runtime. This way it runs closer to the metal, which should help alleviate RAM and storage concerns (these seem to be the only major concerns most people have with it).

<comment><USER10> > Flutter gets away with this because it owns the drawing layer as well as the > widget layer. I know that something drawn in Flutter is going to look > exactly the same no matter where I run it.

Isn't this pretty unfriendly to users? As an iOS user, I occasionally encounter apps written with Material guidelines in mind, and it's a bit jarring because they don't fit the guidelines of the device I'm using.

These are all great solutions for developers, but for consumers, it all kind of sucks to varying degrees.

<comment><USER11> > Isn't this pretty unfriendly to users? As an iOS user, I occasionally > encounter apps written with Material guidelines in mind, and it's a bit > jarring because they don't fit the guidelines of the device I'm using.

Personally I care a lot more about what app I'm using than what OS it's running on; I'd far rather have Slack look the same on my phone as it does on both my computers than have Slack on one computer look like apps on that computer but different from Slack on the other computer. Sometimes we forget that the OS exists to facilitate the apps, not the other way around.

<comment><USER12> > Sometimes we forget that the OS exists to facilitate the apps, not the other > way around

For OS where the base operating system didn't offer much functionality to apps beyond the basic API for building controls and so on, this might be true.

For Apple's systems, especially macOS and iOS/iPadOS, this isn't quite true. On iOS, apps need to be built according to system conventions and standards in order to be able to take advantage of a lot of _services_ that the OS provides (such as accessibility, automation, i10n).

When a macOS or iOS ignores system conventions, the result can range from completely unnoticeable, a slight inconvenience, or the app being next to unusable after the OS updates due to underlying features being changed without the developer having updated their (needless) custom code to account for it.

I notice this with the Sensibo app which follows Material design guidelines. It took me a long time to figure out how to edit a particular bit of information because the icons that the app used were not what I was expecting. On iOS, an editable table view should be accompanied by a button at the top right that says "Edit"; Sensibo instead provided a pencil icon. After I finished editing, I wasn't sure what to do — on iOS, I expect to see a "Done" or "OK" button where the Edit button was; instead, Sensibo required me to click on the pencil again even though it was greyed out and looked inactive.

When an app doesn't conform to the guidelines _and I have a choice about whether or not to use that app_ , I'll generally prefer to find something else where I'm not expected to memorise how to use it.

Sometimes we forget that apps exist to facilitate getting tasks done, not forcing us to remember their deliberately-included inconsistencies compared to every other app on the system (including third party ones).

</comment></comment></comment></comment></comment><comment><USER13> I'm not an electron hater, but when only using 300mb of memory while idle is something to boast about, they may have a point.

<comment><USER14> It's relative. Outlook is using 2.37 GB of RAM as I type this.

<comment><USER13> That's odd, OUTLOOK.EXE is using 44MB of ram as I type this and has been open for 7 hours.

<comment><USER15> Maybe a Mac. My Outlook on mac is idling at 150MB.

<comment><USER14> Yes, this is Office 365 on macOS. Neither the computer nor Outlook have been restarted in weeks, which is surely contributing to its current RAM footprint.

</comment></comment></comment></comment></comment><comment><USER16> Electron discussions often focus around the battery and memory impact, one other thing I'd mention is the poor access to native features. You get the stuff a web browser can do, but for anything else offered by the OS you might need to wait a while.

For example, Apple shipped their first TouchID laptop in 2016, and Electron's support for that landed in v6.0.0 beta 1 just last month.

[https://electronjs.org/releases/beta?page=2#6.0.0-beta.1](https://electronjs.org/releases/beta?page=2#6.0.0-beta.1)

I'd looked at switching off 1password to BitWarden and that's a pretty major feature to be missing for three years.

Surprisingly, for as much as everyone craps on the touchbar, that got supported by Electron a lot faster than TouchID did. I guess because not all apps need authentication and they can rely on passwords instead, while it's really obvious when an app doesn't have touchbar support.

<comment><USER17> Not with nativescript.. you just use the brand new api

<comment><USER18> Does nativescript do desktop? Their site seems squarely focused on mobile (iOS/Android).

</comment></comment></comment><comment><USER19> If electron apps' claimed memory causes your computer to go slow due to swapping, you should complain. But most complaints I see on this topic stem from a discomfort over the reported figure by the operating system. With SSDs and modern virtual memory systems the claimed memory figure is not a sufficient measure to infer the resource utilization of a process. For example, if slack has a bunch of allocated memory that is rarely accessed, it's going to (at worst) land on your SSD and sit there, having no impact on your user experience.

If you want to complain about applications' engineering, you should complain not about claimed memory but about things that are directly affecting resource utilization or user experience like power draw or excessive CPU utilization due to poor programming. Claimed memory must be combined with a profile to show that cold reads are common enough for the claimed memory figure to be correlated with a performance hit. But of course, this often isn't readily available without profiling for all but the most extreme cases, and so engineering types fixate on easily visible measures like claimed memory that likely have no actual resource utilization impact in most cases.

<comment><USER20> >If electron apps' claimed memory causes your computer to go slow due to swapping, you should complain. But most complaints I see on this topic stem from a discomfort over the reported figure by the operating system.

I tried that once. But I'm sick of having to fight that my usecase is valid, that my laptop isn't too old, and that it isn't broken/misconfigured in some other way. It's 4 years old, and I can't run 2 electron applications and firefox at the same time without swapping so hard I can't get anything done. Fuck me for asking so much from a computer, right?

<comment><USER19> I very specifically cited someone like you as being a person who has a valid complaint. I'm calling out people who get upset at the idea of a computer doing "wasteful" work, when that wasteful work has no meaningful impact on latency, responsiveness, power draw, etc. This is common and is a form of "visibility bias". For example, computers do not report their CPU cache miss rates in a highly visible place. I'd imagine if they did, you'd have countless people agonizing over them, despite the fact that in most cases a higher cache miss rate would not result in perceptible difference when using the computer.

<comment><USER12> > I'm calling out people who get upset at the idea of a computer doing > "wasteful" work, when that wasteful work has no meaningful impact on > latency, responsiveness, power draw, etc.

But who gets to make that call?

For every person out there who complaints that using voice chat in Discord on a MacBook Pro on battery causes the chassis to heat up, fans to kick into turbo boost, and the battery to dwindle — but __only __when using the .app, not when using the website through Firefox, there 's another person who says "oh, but my computer doesn't do that".

So then everybody who has a similar complaint as the first person is branded a liar and whiner.

</comment></comment></comment></comment><comment><USER21> Thanks for the post.

> I want something more lightweight eventually

Check out [https://github.com/revery-ui/revery](https://github.com/revery- ui/revery)

<comment><AUTHOR> That's a very cool project (I've used OCaml before) and I hope it takes off. I should try it, but I admit I'm a little skeptical because there are so many details to get right like focus management, accessibility, and more.

Unfortunately since I can't rewrite my project entirely, I'm going to need something that reduces bloat by using a local webview instead, so it's still web-based.

<comment><USER22> The local web view is the bloat. The rest of electron is not the problem.

<comment><AUTHOR> ? A local webview usually means that it reuses the user's already installed browser and uses that instead.

<comment><USER22> The browser I have installed is the single most resource intensive program I have on my computer.

<comment><USER23> Why? Because you picked a particularly bad one or because you have 200 open tabs? Usually the browser is running anyways so adding another open tab or window increases resource usage much less than another program shipping its own browser engine.

<comment><USER24> Do you know of a browser that isn't massively resource intensive?

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> 300mb of memory to do a bit of money-counting, which you could probably do equally well with MSVC in 2mb.

Such a wasteful world.

<comment><USER25> My favourite example is Etcher[1] which is effectively a GUI for "dd" and ships as a 227MB "app" that uses >200MB of RAM idling.

"dd" is 37KB.

[1] [https://www.balena.io/etcher/](https://www.balena.io/etcher/)

<comment><USER9> I've used Etcher a bunch of times. It's pretty and gets the job done.

But it's also heavily recommended within the Raspberry Pi community, and used by Balena's customers for starter IoT applications. We can assume most of the users of this application are beginners.

So, do you think these beginners would care about the size of this "app" that they'll probably delete as soon as it finishes? Do you think they'd care about 200MB of RAM when they're almost certainly running it on any modern PC instead of some ancient EeePC from hell? As a beginner, would you feel more comfortable in front of Etcher or dd?

The crux of your argument basically boils down to "a GUI takes up more RAM than a CLI." Which is obviously true, but it doesn't invalidate the need for more GUIs in the world. Could Etcher have been built with a native GUI? Sure, but it's free software, there are no expectations. If you wanted to, you could clone the repo and build it natively yourself.

But why would you? Etcher already exists, and it would save you from hundreds of hours of debugging and learning new frameworks. Knowing HTML/CSS/JS, you could build Electron Etcher in a fraction of the time it would take you to build Qt4 Etcher, and the compromises are acceptable given the target audience.

And now you understand why the Etcher devs chose Electron.

<comment><USER26> > So, do you think these beginners would care about the size of this "app" > that they'll probably delete as soon as it finishes

Well, looks like enough people care (or at least noticed) [https://imgur.com/LV6AT7d](https://imgur.com/LV6AT7d)

<comment><USER9> The dev's justification:

[https://github.com/balena- io/etcher/issues/2365#issuecomment...](https://github.com/balena- io/etcher/issues/2365#issuecomment-394702621)

</comment></comment></comment></comment></comment><comment><USER27> Obviously bundling nearly all of Chromium with an app is inherently inefficient, but out of curiosity's sake, has there been any objective analysis of _exactly_ what makes this use a seemingly disproportionate amount of resources? Are JavaScript runtimes inherently inefficient? Is it Electron's node.js part? Is it the huge amounts of code needed to parse and render HTML and CSS? Is it the multimedia features, like WebGL and the video and audio players? Is the problem actually just badly written code running on top of Electron? And if so, what _exactly_ are are most Electron apps doing wrong that one could avoid? Is it, as this blog post seems to suspect, that they aren't sufficiently taking advantage of Electron's node.js integration?

In addition, once the actual bottleneck has been identified, doesn't this open the door to fix things on the WebKit/Blink engine side of things? Before, when Electron was managed by GitHub, one could say this would be unreasonably difficult, but now that GitHub is owned by Microsoft, isn't tweaking Blink to be more suitable for deployment environments outside of Chrome exactly the type of thing they would have a vested interest in doing, and the resources to actually pull off? After all, with VS Code, they are one of the highest- profile Electron users, and using Blink in Edge presumably involves working with the Blink code to some degree anyway.

It's always struck me that a lot of Electron apps likely never use a single line of WebGL, while a lot of WebGL apps likely only use DOM APIs to pretend the DOM doesn't exist by plastering everything in a huge <canvas> element, and both likely don't want their binaries to include the Chrome devtools, but Electron includes all of these things anyway. I'm not sure how much space or resources it would actually save, but is Chromium/Blink refactoring their code and build process to facilitate Electron building different Electron "profiles" that include different sets of features even remotely a possibility?

</comment><comment><USER28> Not only memory is the problem with electron, but also app feeling. Skype left is the only electron app I'm using. And comparing to desktop messaging app (telegram): 1\. Skype unresponsive. Opening chat takes up to few seconds. 2\. I see how opened chat is literally updating -- messages, avatars, emoticons emerges from dust 3\. Selecting messages as messages in native vs selecting messages as text with surrounding artefacts (chat is a html page with all what comes with it)

Summarizing, electron apps are just feels like foreigners.

</comment><comment><USER29> I have never had an issue with electron apps being terrible because of networking - although clearly there are apps that do push more work than is necessary over the network, there's plenty of native software that does too.

My problem with electron apps is that they use way more memory and cpu than they should be - the architecture fix for that would be to not be a full browser.

The problem I have is they lack basic app behaviors from the platform. My experience is mostly for the Mac, but my experience using vscode on Linux has been similar.

But here we go, for a Mac:

* cmd-e/f/g are so wide, not just app wide. Vscode goes even further and seems to achieve per-view behavior.

* document editors and viewers are expected to have an icon for the file being edited, either in the window or in the tab. That is a draggable link to the file, and a drop down on right click for the directory hierarchy.

* drag and drop of files into and out of the windows don't work

These are basic behaviors that even the simplest apps manage to get right on OS X, yet no electron apps seem capable of meeting this low bar.

</comment><comment><USER30> People love to bash Electron, but what is the alternative?

Is there any other platform that provides:

\- Support for the 3 major desktop platforms from one code-base;

\- A decent choice of high-level programming languages (there are excellent compile-to-JS options now);

\- Reasonable tooling and community support;

\- MIT (or similar) license

<comment><USER31> Qt. Supports Linux, macOS, and Windows. Supports at least one high-level programming language very well (Python). Has good tooling and a lot of support.

It's LGPL, which is fine for most cases. But you could always buy a commercial license.

</comment></comment><comment><USER32> I use VS Code and it never felt sluggish. But it's probably one of the best Electron apps out there.

Honestly, if I had to develop a Desktop app today and wanted it to be cross- platform, I'd probably use Electron. No need for hacky workarounds like in a web app to make it run on every major browser.

<comment><USER33> I use VS Code too and it's extremely sluggish compared to a text editor like Sublime. I use it only because the plugin experience is much better than Sublime and the responsiveness is infinitely better than JetBrains' products.

<comment><USER34> I find it not fair to compare VSCode to JetBrains' product: one of them is an IDE, the other one is an editor, different features and different targets!

<comment><USER33> I disagree. VSCode is really a hybrid product. With plugins, it does intellisense and debugging, which are really the two major value-add features of an IDE for me. It's still light years faster than IntelliJ (and light years slower than Sublime). I switched from IntelliJ to VSCode and it minimally affected my workflow, so they're objectively not different targets in that regard.

<comment><USER35> Are there are VS Code intellisense plugins that beat Webstorm?

</comment></comment></comment></comment></comment><comment><USER36> I honestly don't know why people waste time building Electron apps.

It's the worst possible combination. All of the downsides of web apps (poor performance, memory use, poor integration, etc.) and native apps (security problems, hard to update, etc.), without the advantages of either one.

Why would I want to download your specially packaged browser, without adblock or uMatrix or any of that, just to basically view a website anyway?

<comment><USER37> Ah, but you don't waste time, that's the whole point of the idea. You just repackage your existing web app and distribute it as a "native" app, no effort required. You also have a cross-platform application with no extra work (the auld "Write once, run anywhere" slogan), and can use web developers instead of people who know QT/Java/what have you.

Then the user can run your chat/music/text editor in the background, have it appear in the taskbar, and start boot, at the cost of requiring a few gigabytes of RAM and 25% CPU utilization while idling.

Personally I'd rather have Progressive Web Apps but for some reason, those are still in an awkward stage.

<comment><USER7> > and can use web developers instead of people who know QT/Java/what have you.

Is that supposed to be an advantage? ;)

</comment></comment></comment><comment><USER38> I've never noticed Electron apps negatively impacting my computer. I use Slack all the time and I don't notice a difference when I have it running vs when i don't. Do most people not like Electron because the number seems high or does it negatively impact their computer in some way? (I have a MBP with 8 gig RAM, that doesn't seem out of the ordinary)

<comment> yoz-y As for impact it depends. Slack is not that bad except that it is slow (for example switching teams was sluggish when I was using it). But when I run Slack, Skype, Insomnia and VS Code at the same time I definitely saw the impact. I had 16Gigs of ram but half of that was eaten by a VM all the time.

</comment></comment><comment><USER39> I am an electron hater but this is a really good setup. Server reloading with that debugger is rad. I'm going to use these methods with a platform I don't like.

</comment><comment><USER40> Well to me it is thing that should not be done at all, instead of doing it diligently.

I will put this in category of 'The secret of good user tracking on web', 'the secret of doing quality journalism on ads supported model' or 'the secret of nuanced political debate on Twitter' etc.

</comment><comment><USER41> Unrelated: I really hope software like [https://github.com/revery- ui/revery](https://github.com/revery-ui/revery) becomes mainstream!

This node IPC debugging technique is pretty nice!

<comment><USER42> What's the size of the built binaries?

</comment></comment><comment><USER43> The hate for Electron is strong in this thread.

For Electron haters, here are the alternatives: OpenJFX (some people hate Java), Qt5 (it can be pricey in some situations), wxWidgets (not popular enough?).

However, I have a lot of hope for Flutter in desktop. I hope the developers of Flutter framework can pull off the challenge.

In my case, for opensource projects, I definitely use Qt5 (using PySide2). But for proprietary projects, I have to admit that most likely I will choose Electron.

<comment><USER44> AFAIK, as long as you don't modify Qt itself, you can use it just fine for proprietary projects without having to buy a commercial license.

</comment></comment><comment><USER45> I'm building an application using Electron and I've found the majority of memory consumption occurs when instantiating new components. When using a UI framework like Material, this can happen frequently, such that the simple act of hovering over a menu and showing an indicator can cause an increase of 3 MB of RAM. I love Electron, it allows me to create applications in an intuitive way using a language meant for building UIs, rather than slapping a library on a native codebase, and I can deploy it wherever Chromium is supported, so I'd like to make it efficient, wherever possible.

Two solutions I have found during my development:

1) Keep your components light. Most of what I used from Material can be done completely with raw HTML and CSS in a simpler fashion without much going on in terms of state, so I created my own components that serve my purposes. A lot of libraries and frameworks are heavy because they are used on webpages viewed by multiple different browsers and require complex logic to work across different environments. Electron sandboxes your code in Chromium + node, you code only for 1 browser engine, so I hope more devs take advantage of that fact.

2) Load what you know you need in the beginning, and keep your cache to a minimum. Most Electron apps do not have view transitions, they are built as SPAs. One of the downsides of this is that you need to be mindful of your cache since it is not cleared when you display a new component. Clearing the things that don't matter to the view or data has been essential in keeping my memory footprint small.

With a large app that does many things and loads many views, I'm able to keep the RAM usage to under 200, most times around 150, which, while larger than something like Excel running idle, is a small price to pay to have intuitive design features and access to a large and constantly growing library of reusable components and packages.

</comment><comment><USER46> The type of developer that chooses Electron isn't going to care about squeezing out the best performance possible. They already made a conscious trade-off to sacrifice performance and memory usage in exchange for productivity. They won't do some hacky things like use Rust instead of nodejs.

</comment><comment><USER47> It sounds like a generally good approach for keeping the UI process responsive at all times. I wonder if the serialization cost behind all those IPC calls becomes an issue at some point, especially when passing large structures back and forth, have you found this to be an issue for your use case?

<comment><AUTHOR> I haven't noticed an issue - the only alternative would be to block the renderer and do it in the same process. The IPC uses a local domain socket which is very high performance (better than a web socket) so any overhead is quickly made up for with UI responsiveness. There may be specialized cases where it is a problem and you'll have to figure something out.

<comment><USER48> How do you secure the socket and prevent a malicious process from connecting to it?

</comment></comment></comment><comment><USER49> I feel like the key idea here is somewhat unrelated to writing Electron apps. It's that you can set up any Node server app to run inside an Electron window during development and then you get cmd+r and all the devtools for free. Kind of like a node inspector on steroids. Pretty nifty!

</comment><comment><USER50> If you need a background process to call Node APIs because "it's the only place where it's safe to access them", what unsafe way is this meant to contrast with? Does Electron normally (but unsafely?) call Node APIs from the browser process?

<comment><AUTHOR> You can enable "node integration" with any renderer process and call node from their. But you definitely should not enable it in your primary renderer process if any untrusted code is running their (if you pull in any JS from anywhere else)

</comment></comment><comment><USER51> Thanks for sharing! Your post is full of things that might seem obvious in hindsight, but all too likely to be ignored absent a compelling writeup like yours. Bookmarked the linked repo, will revisit w/ my next Electron app.

</comment><comment><USER52> What are the 83 threads used for in that trivial budgeting application?

</comment><comment> smith-kyle I made something similar a couple years ago: [https://github.com/smith- kyle/electron-process](https://github.com/smith-kyle/electron-process)

But I love the addition of defaulting to a node server for lower memory usage.

The Atom editor (the first Electron app) has slowly been rewriting a lot of their modules in C++ for speed and lower memory usage, binding to them through NodeJs. Kind of ironic since GitHub created Electron to build desktop apps using web technologies.

</comment><comment><USER53> Vote for Linux support in Actual here:

[https://trello.com/c/A7XzJgJX](https://trello.com/c/A7XzJgJX)

<comment><AUTHOR> Linux support was added in 0.0.99! See here (and download it): [https://github.com/actualbudget/releases/releases/tag/0.0.99](https://github.com/actualbudget/releases/releases/tag/0.0.99)

Somehow that card didn't get moved to done, sorry! Let me know if it doesn't work.

</comment></comment><comment><USER54> The argument about bloat is so weak tho. Engineering about trade-offs. If you can't tell why people use Electron, you're not a good engineer.

</comment><comment><USER55> IMHO, good electron apps feel and behave like native apps. Or, in the worst case scenario, they feel like an equivalent web app.

Specifically - I should not have to think of having to "ctrl-R" for a desktop application.

Good electron apps do exist, but they feel like a vast minority. Most just feel like a bloated version of their web application.

<comment><AUTHOR> I completely agree! Are you referring to the "ctrl-r" to reloading the server in the article? That's only for development - the user never has to refresh the app like that of course!

<comment><USER55> I think you're saying "a user should never have to refresh the app". Because ctrl-r has returned completely borked applications back to working. Specifically, I've had to use it almost daily to make slack recover from a freeze.

<comment><USER56> I'm not sure what the point is, here. Is it that Electron apps can lock up or crash? That they can be refreshed seems pretty immaterial, given that any application in any language can crash

</comment></comment></comment></comment><comment><USER57> My simple rule of thumb for reducing the memory usage of Electron apps is to not run any of them. Problem solved. It's like why I never ever installed java which was subpar garbage. Gosling has a lot to answer for.

</comment><comment><USER58> I would just skip Electron and make a PWA (add2desktop) instead.

</comment><comment><USER59> I miss native apps.

</comment><comment><USER60> The secret is don't use electron.

</comment><comment><USER61> I find the problem lies with the trade-off you are making.

You are making your life as a developer easier in exchange massively increasing the resource requirements and battery usage on end-user machines.

It's just bad practice and Electron encourages it.

I feel like if we ever want to be taken seriously as a profession the way structural or mechanical engineers are for example, then efficiency and reliability need to be primary considerations, otherwise we are basically making toys.

<comment><AUTHOR> I would easily forego the niceties explained in the article if it meant there were something better.

The main reason to use Electron is that I can have happy macOS, Windows, and Linux users. Apps that focus on a single platform are exclusionary.

<comment><USER62> You can have that already - use web, the real one, not Electron. Everyone then can use their favorite browser for your app, not forced to use Chromium.

And if you want to make an efficient desktop app, look at telegram source code.

<comment><AUTHOR> The web's offline support sucks, and don't tell me to use service workers. Actual is a truly offline 100% local app. Most users love that they can simply launch a desktop app and it always runs no matter their network connection.

</comment></comment></comment></comment><comment><USER63> First rule: if you need Electron, change your requirements/design and move to something more cross platform like Qt or libui/C++.

<comment><USER45> You forgot a number of additional tips:

\- Hire a C++ programmer

\- Learn C++

\- Adapt knowledge in your current language of choice to C++/Python

\- Find a designer who can use Qt

\- Adapt common tools normally found in npm to C++

\- Rewrite your application logic to run in C++

Should be simple for any dev to do considering the pay off is lower RAM and "native" look and feel

</comment></comment><comment><USER64> Some people seem to think that unused ram is good, quite the contrary actually. If you have 50 of these apps running doing work while playing a game while having a pc from 10 years ago, then I can actually see a problem. Laptop and battery-driven devices is a whole other matter though.

I like to think the reusability and speed of development with electron is just terrific and is a trade-off that is worth it in the end.

<comment><USER55> Unused RAM _is_ good. It lets the OS do things like pre-fetching and caching disc reads. That is, it speeds everything up to not be using all of your RAM.

Plus, when you do fill your RAM, you have to start contending with swapping, which slows everything down significantly.

<comment><USER64> Swap is not enabled by default by any OS that I know of. And I doubt any OS requires you to have several gbs free in order to work smoothly.

My point being is, we are not stuck with 2 gbs of ram anymore and like games, apps will scale to use more resources for ease of development or better features.

<comment><USER65> Literally every major OS in existence has swap enabled by default. Linux, Mac, windows.

The alternative is for things to start crashing when you run out of memory rather than swapping out.

<comment><USER66> > Linux, Mac, windows.

But not Android, iOS or iPadOS. (And for good reason - these OS's run on devices that use bottom-of-the-barrel flash storage and can't sustain the wear-and-tear that comes with using swap.) That's not "literally every major OS"!

<comment><USER55> Instead, they suspend applications to the self-same flash storage when the the application is not being actively used and there is memory pressure.

Doesn't seem a lot different from swapping the memory to disc until its accessed again.

Not to mention, it's kinda irrelevant to a discussion of electron applications.

<comment><USER66> They don't "suspend applications". They can _prompt_ an application to serialize an image of its "core" memory for storage on the flash media and to subsequently restore its state from the serialized memory dump, but very few applications are written to do the job properly.</comment></comment></comment></comment></comment></comment></comment>


What do you look for when hiring a freelancer - <AUTHOR> Ive built an e-learning website linked to a freelancing website. Users take courses in the e-learning part of the website and complete tasks for companies in the freelancing part of the website. But they can only apply for a task if they have completed its required courses.
So, I would like to know what other companies look for when hiring a freelancer. Are courses enough to validate that a user has the knowledge they say they do?
Also, I would like to know what issues you might have hiring on other freelance platforms so that I can try and solve them with my app.
Ive talked with potential users and they seemed interested but I want to hear a company's perspective on my idea. <comment><USER2> This is a really cool idea -- though to be honest, I would have some reservations depending on what the "courses" entail.

The biggest thing that I would look for in a freelancer is demonstrable experience doing the particular thing I'm hiring for. For example, if I want someone to help me implement a game using the Unity game engine, I'd look at the freelancer's body of work of past Unity projects. I would be very hesitant to work with someone who is learning Unity for the first time using your platform because that would imply they have no prior experience using that technology.

It sounds like your courses are intended as a way to validate a freelancer's knowledge -- almost more like a test or certification than a course. I see in another thread that you intend to have users upload a previously completed project, which could be a great way to check their past work, but then I'm a bit confused as to why they would then need a course.

So I guess I feel a bit confused about what I should expect from the people completing the courses -- will these people be freelancers with experience and the courses are just validating that? Will they be freelancers with no prior experience in a particular technology, and they're learning it for the first time via your site? Will it be a mix of all of the above?

I think a focus on validating freelancer's existing skills would probably give the most useful data for the folks that are trying to hire them.

<comment><AUTHOR> Anyway if its not asking too much could you check out my site and tell me what to improve: [https://enrole.herokuapp.com](https://enrole.herokuapp.com). For companies: [https://enrole.herokuapp.com/companyhome](https://enrole.herokuapp.com/companyhome). Im just starting out with this idea so my website isn't as polished. So, any feedback to improve it will be appreciated. You can give feedback here: [https://discordapp.com/invite/wcQSPww](https://discordapp.com/invite/wcQSPww) or email me at firstname.lastname@example.com

</comment></comment><comment><USER3> Someone who has passion in the field is something I've seen companies prefer, almost more than a specific skill set as as person who has passion has a much greater willingness to learn/adapt. I am not directly associated with the hiring process.

If you have 100 candidates who all can pass the test to get to the point where there are tasks to work on. Let's assume 10 all apply for the same task at a company. If all applicants pass the test, how to you figure which would be a better fit?

What could be used is to allow an individual to showcase past projects they have done to help a company determine which of the applicants would be better suited for what they are trying to accomplish.

I want to clarify to that this would be for personal/pet projects they worked on in their own time, not for another company or where they were paid to do it. This helps show a level of genuine interest if you are doing a project for the love of learning or to self motivate and fix a problem where there is no solution.

<comment><AUTHOR> In my website, to complete a course a user has to submit a project which can be anything related to the course. Companies use this and tasks they have completed in the past to figure out who to give the task too.

Would this be counted as personal project or would companies think that users are just submitting a project to complete the course rather than because of their passion

<comment><USER3> If the project that is related to the course was created specifically for that particular course and your site/service then I personally would lump that into paid as the reason for that particular project is the hope to acquire more paid work.

It still would be a good way of seeing how different people's projects may differ and be a good way to narrow down candidate selection choice.

I was thinking more along the lines of a resume page, and being able to showcase a project they had done in the past, independent from this coursework, that relates to the subject/task at hand they are applying for.

<comment><AUTHOR> So, if there was an option where users could upload personal projects they have completed in the past (not related to courses) to their profile, would that be ok.

<comment><USER3> Yes, when an employer is looking for a potential candidate/user and they review/pick out 10 potential hires based off the coursework and an obvious candidate doesn't jump out at them, the employer could turn to the personal projects listed in the profile to compare how they work without direction/on their own which may help them decide which user to hire.

<comment><AUTHOR> Ok thanks a lot for your suggestion. Ill add this feature to my site.

<comment><USER3> I checked out your site briefly, it seems like a really great concept! A little bit of feedback coming from a non-designer, so this may not be true for everyone.

On the about page, rather than have the top be enroll now, I would reorder things.

Currently you have: -Enrole -Problem -Solution -Vision

I would put: -Vision -Problem -Solution -Enrole

Keep it up!</comment></comment></comment></comment></comment></comment></comment>


Apps for children must offer privacy by default - <AUTHOR> https://www.bbc.co.uk/news/technology-53985421 <comment><AUTHOR> I don't see why privacy by default should be limited to children's apps. All apps should be private by default.</comment>


Canadians may now apply for a TN visa before they reach the border crossing - <AUTHOR> http://www.uscis.gov/portal/site/uscis/menuitem.5af9bb95919f35e66f614176543f6d1a/?vgnextoid=c130f4c88ad0a310VgnVCM100000082ca60aRCRD&vgnextchannel=e7801c2c9be44210VgnVCM100000082ca60aRCRD

<comment><USER2> I've crossed the border on a TN for 16 years now. I've been refused twice. The first time (my first time) because I was really unprepared. The second time, about two years later, when we used a law firm for the first (and only) time, I was at Netscape. The border agent didn't like the look of my education credentials, and I ended up in Toronto (I'm from Vancouver) - because Netscape thought that was a "Good Place to cross") - I got supporting documentation and I've never been refused again (14 years running now. Knock on wood.)

Ironically, I'm heading back again to re-apply this weekend, flying back home for thanksgiving weekend. The cool thing is you only have to do it once every three years - though, now that I do the paperwork myself, it's a one page, 4 paragraph letter, describing my job, where I work, what I do, what I make, what my experience and education are, and when I'll start work, and for how long.

For those who fall within the system (2 year+ diploma, appropriate job category) - it's a 5 minute interview at the Airport/Border. I don't even arrive more than 15 minute earlier at the airport anymore.

</comment><comment><USER3> I was rejected for my TN at the border on my first attempt. The customs agent thought I was working illegally for some reason. I always hated how so much of the decision process was based on whether the guy at the border liked you or not. I ended up having to cancel all my flights, hotel and car reservations and spent the whole day talking to lawyers.

Tried again a week later, and the customs agent I met this time said I should have been let through the first time. -.- _sigh_

<comment><USER4> My first time applying at the border for a TN, I was refused. The lawyer said they always do that first time to make you work harder.

The excuse was I didn't have copies of my university degrees (I had transcripts showing the granted degrees).

So this has to be a good thing, reducing the power of border agents.

<comment><USER5> > _"The excuse was I didn't have copies of my university degrees (I had > transcripts showing the granted degrees)."_

That's not really an excuse, that's a stated requirement. Not all border officers screen so thoroughly, but you _should_ have had the original of your degree.

If there's one thing I've learned in dealing with work visas, is to not trust your lawyer. Most immigration lawyers I've seen are, to be polite, less than amazing at their work. Double check, verify everything they do.

<comment><USER6> I got accept the first time, and didn't even have an official degree paper yet, since the ceremony was six months ahead of time. I just had transcripts and a letter from the school that I completed the requirements of my degree. It really is the mood of the officer. I had a really nice newbie mom type for my application.

</comment></comment></comment></comment><comment><USER7> This is so helpful.

When I was planning to apply for TN, I was very hesitant to pack all of my things, buy a plane ticket, go to the airport and have to deal with a random unfriendly customs agent, then have a good possibility of getting declined for arbitrary reasons.

<comment><USER8> >have to deal with a random unfriendly customs agent, then have a good possibility of getting declined for arbitrary reasons.

This not only hurts the candidate, but harms American companies so much more. Some technical recruiting process take months to find a good lead for a hire. More specific the criteria, even tougher to find a good candidate. Naturally, American company have to consider beyond American pool of hires and check Canadian candidates as well.

The company easily spend between $2000 to $3000 for TN application, lawyer fee, flight, hotel, other accommodations for interviews per candidate. It's devastating when you try to hire a Canadian after spending so much time and cover all the cost, just to be denied by border officer.

<comment><USER9> That's interesting. I've never been refused for a TN application (have applied four times). And I'm not anyone special either, I just have printed credentials and have written the application letters myself (with approval from the U.S. company that hires me). In fact, one time when I was applying, the agent merely skimmed my write-up and we ended up talking about nothing for fifteen minutes.

</comment></comment></comment><comment><USER10> I'm so jealous I was denied my TN last year because the interviewing agent didn't like my credentials, I'm a Mexican citizen and we don't have it that easy, we have to make an appointment for a paperwork review process (1 month minimum) then they schedule your for interview ( 1mknth here too ). At the time I was just married, my employer had just moved us to Chicago, and suddenly I was in Tijuana ( my hometown ) without a job or any personal belongings. Haven't applied again since I haven't had the luck of getting another employer try to hire me since they are all scared ( with justification )of the process, wish it was easier for us too, but I know we have a higher track of abusing the system (especially on immigration). Still. I would try again if opportunity came.

Edit: typo

<comment><USER11> Really? I'm sorry about your experience but mine was completely different, I scheduled my interview in the embassy in Mexico City and I just had to wait 1 week. It was a 2 day process, first I went to a CAS and they captured my biometric information, the next day I went to the interview with the consul which was like 15 minutes or less (but I had to wait for 2 hours), he only took a look at my employer's offer letter and asked for my cedula profesional and that was it. I got my TN visa 1 week later.

<comment><USER10> Thing is I already had mine, I was just renewing, hope everyone else's experience is far better than mine

</comment></comment></comment><comment><USER12> Question for the HN community: As a software engineer considering work in the US I am covered under the list of professions for H1B & TN visas and can probably find a job where a company will sponsor me. However, my girlfriend went to school for a BCom in marketing. If the 2 of us considered moving to the US for work, what would be the easiest way for her to get a visa? Would she be able to get a TN visa as a marketing / management consultant (she graduated only 2 years ago)? Any tips are much appreciated.

</comment><comment><USER13> Good news. I hope they start implementing this type of thing (ie not having to show up at the border and hope you've got everything all set) in more visa situations.

If you require a visa to enter the country you should be able to apply ahead of time and know the outcome/be able to appeal if appropriate. Not just hope at the border.

</comment><comment><USER14> This is really great for Canadians. Less risk packing up all your stuff (paying for all the bag fees) and potentially get denied.

It does look like it costs $325 to file this I-129 (Petition for a Nonimmigrant Worker). This is in contrast to applying at a port of entry where the cost is $50/$56. Am I mis-reading?

</comment><comment><USER15> This is just beautiful. Finally, some added sensibility. My gratitude to both governments for pushing this through.

</comment><comment><USER16> Luckily I didn't live too far from the border so I drove down ahead of my move to get approved. Then when it came to move I already had my TN visa.

This would've been nice though!

<comment><USER14> I've heard that sometimes they will deny unless you are heading straight to your destination and starting work within 3 days. Good to see it worked out for you.

</comment></comment><comment><USER17> Any risk of being totally banned from the states after getting rejected with a TN too many times?</comment>


Self-serve banner ad design tool Canned Banners exits beta - <AUTHOR>

After launching in January 2010, Canned Banners (www.cannedbanners.com) has left beta. Our self-serve app lets anyone quickly create Flash banner ads using pre-designed templates. We're a 3-person, self-funded startup based in San Francisco. Our full press release is here: http://bit.ly/eCrXcq <comment><AUTHOR> We're definitely pursuing partnerships with publishers, ad networks, and miscellaneous online marketing solution providers...our growing list of partners is here: <http://www.cannedbanners.com/partners>

But I'd never heard of PageGage. Thanks very much for the suggestion!

-Myles Younger

</comment><comment><USER2> You've made a pretty nice banner creator. Might be worth looking at partnerships with publishers who have self service ad buying platforms or platforms like PageGage.</comment>


What I wish systems researchers would work on - <AUTHOR> http://matt-welsh.blogspot.com/2013/05/what-i-wish-systems-researchers-would.html <comment><USER2> Computer systems, which are _deterministic_ , have long since passed out of the realm of being _understandable_.

They're chaotic systems[1]. Parameters which are below the threshold of detection can drastically affect their trajectory in phase space. Accidental complexity has zoomed off to live a life all its own, and that takeoff has been continuous for decades [2].

Everything we build around software that's not directly bearing on the problem domain is chipping away at this or that aspect of the ungoverned bits. As we corral each level of new unpredictability, we merely set the scene of the next level of complexity.

 _There is no universal solution_. There is no final boss which, once defeated, leads to the scrolling names and 8-bit themesong. It never stops, ever, because the value of systems that frequently fail unpredictably is lower than the cost of flawless systems and higher than the value of not building such systems.

Edit: sounds very unhelpful, even defeatist. I know. I'm deeply ambivalent about what _can_ be done versus what we should _assume is possible_. I'm a fan of serious software engineering and tool support and all that jazz ... yet thoroughly pessimistic about the limits of the possible. Deep sighs for every occasion!

[1] [http://chester.id.au/2012/06/27/a-not-sobrief-aside-on- reign...](http://chester.id.au/2012/06/27/a-not-sobrief-aside-on-reigning-in- chaos/) (warning, self linkage)

[2] [http://www.cs.nott.ac.uk/~cah/G51ISS/Documents/NoSilverBulle...](http://www.cs.nott.ac.uk/~cah/G51ISS/Documents/NoSilverBullet.html)

<comment><USER3> They're not even deterministic. Even if your input is totally ordered, entropy from the memory subsystem will bubble up to communicating threads if you have more than one. Sometimes one thread will read the newly written value, sometimes the old. That said, there is some clever research out of UW into how to make such systems deterministic while still recovering almost all parallelism.

<comment><USER4> I think the more important point jacques_chester was making is that even in the case where computer systems are 100% deterministic it doesn't matter because they are so complex they become chaotic systems. And then hypothetical determinism won't save you because there are too many variables in play simultaneously, and even the slightest variation in any one of them could change the behavior completely.

<comment><USER5> Sorry guys, but I don't buy your defeatist arguments for one second. Sure using traditional tools, you may be right. However, all we have to do is look at the brain with its 100 trillion Connections (axioms) or a CPU with its billions of transistors to see that your logic is flawed. If it was not possible to design fault tolerant systems on large scale neither of these two examples could exist.

I don't claim to have the answers, but a few years ago, I had to help out some EE's write some testing software in LabVIEW. The first thing that shocked me was the system was immune to bad data. Secondly, how elegantly it took advantage of our multiprocessing and multithreading hardware.

What I wish systems researchers would work on is concurrent, signal-based, synchronous languages. This is a winnable game, but we must swallow the red pill and change the rules of the game with new tools and fresh ideas.

</comment></comment></comment></comment><comment><USER6> Speaking as a student studying systems (particularly security and OS development) . . . the second point sounds really interesting.

Recently I've been working on a simple microkernel (called Sydi) that should actually make this possible. It's a distributed system straight down to the kernel: when two instances of the kernel detect each other running on a network, they freely swap processes and resources between each other as required to balance the load. This is a nightmare from a security perspective, of course, and I haven't figured out the authentication bits yet -- too busy working on an AML interpreter because I don't like ACPICA -- but that's not the issue here. (It's far from finished, but I'm pretty confident that I've worked out the first 85% of the details, with just the remaining 115% left, plus the next 95% of the programming . . .)

Due to some aspects of its design (asynchronous system calls via message- passing, transparent message routing between nodes in the network, etc) I feel that it will be entirely possible to take a snapshot of an entire cluster of machines running the kernel. It would be expensive -- requiring a freeze of all running processes while the snapshot is taking place to maintain any level of precision -- but I'm confident I could code that in during the space of a week or two . . .

I haven't thought much about what kind of analysis one could do on the instrumentation/snapshot results, though. I'm sadly too inexperienced with `real-world' systems stuff to be able to say. Anyone have any suggestions for possible analysis avenues to explore?

<comment><USER7> This sounds interesting! Have you published anything more substantial on it? I yearn for the day when I can have a Plan 9-ish resource sharing and file system connecting my personal computer to my cloud computers.

</comment></comment><comment><USER8> I want to register a complaint. _Everyone_ talks about wanting ground-breaking research and innovation, but in my own personal experience I see little-to-no support for it.

It reminds of the scene from the film _Election_ , where a character intones, after you've seen him ski down a double-black diamond slope and crash, "Last winter when I broke my leg, I was so mad at God."

We're spending money to squeeze the last drops of gold out of the research done in the 60's and 70's, while research starves, and then complaining that there's nothing new. I don't see how we can expect anything different.

<comment><USER9> i originally opened the link expecting to conclude the same thing, however if you read the article again i think the guy is a) highly qualified to complain (after all he chaired HotOS's PC, no small recognition of his stature in the community) and b) he raises good points.

systems are about interactions, and he's identified the challenges in analyzing systems.

this isn't so much about work from the 60s and 70s, but about modern architectures and the new things they enable. we're still working with models form the 60s and 70s and ignoring various aspects of new system architectures and wondering why we still suck.

<comment><USER8> Thanks, I think you're right that I need to read it again. I have a personal issue with this and it leads to knee-jerk reactions. Thanks for that.

</comment></comment></comment><comment><USER10> i find it "interesting" that all the complains are about useability from a "userland" developer pov. Generally what comes from that are patched up systems to "run stuff in kernel" because "its safe and fast"

Which of course is not true. There are however various attempts to make a "proper" "modern" OS but all of these have failed due to the lack of commercial backing. There is apparently no financial interest into having a secure, scalable, fast, deterministic, simple OS right now (which, oh, requires rewriting all your apps, obviously)

Even Microsoft tried. Yes, Microsoft did an excellent such OS.

<comment><USER11> Well, there is a lot of interest in a secure, scalable, fast, deterministic, simple OS which is also backwards compatible. ;)

What do you mean by "safe" though? How is kernel-space safer then user-space?

<comment><USER12> By being written in systems programming languages safer than C.

<comment><USER13> This could catch some programming bugs, sure, but it wouldn't make the OS much safer. You can still have attacks on integrity, privacy and availability.

<comment><USER12> Agreed, but the attack window would be smaller, that is what counts.

If you really want to attack a system, nothing beats social engineering anyway.

</comment></comment></comment></comment></comment><comment><USER14> CS research has been pathological for a long time.

Anybody near the field has their own list of "taboo" research topics that are very interesting but just don't get funding.

Then you go to the library and realize they have 120 shelf-feet of conference proceedings about Internet QoS that have gone precisely nowhere.

</comment><comment><USER4> Configuration, performance, package management, reliability/robustness, security, monitoring, virtualization, and distributed computing. Those are the big problems and missing pieces that people doing systems level engineering are facing today. And, no coincidence, these are the things where you see the most repeated instances of tools that address these issues.

Consider package management. You have an OS level package manager, you have OS level application management (on android, for example), you have node package management, you have ruby gems, you have perl and python packages, you have wordpress themes and plugins, etc. Obviously it doesn't make sense to have a single global package manager, but what about standards for package management? Competing package management infrastructures which serve as foundations for all these lower level systems? A mechanism for aggregating information and control for all of the package management sub-systems?

As the article points out a lot of systems folks and OS folks are still retreading the same paths. They want to pave those paths, then build moving walkways, and so forth. That's all fine and good, but there's a lot of wilderness out there, and a lot of people are out in that wilderness hacking away with machetes and pickaxes building settlements, but for some reason most OS folks don't even imagine that roads can go there too.

<comment><USER15> As far as the ruby/python/perl package problem goes, I think we're heading towards a standard with virtualenv-style tools and potted environments. The simplest possible standard there is a <foo>/bin/activate script to set the right environment variables; you can build pretty much anything else you want on top of that.

</comment></comment><comment><USER16> I think all of these areas are making great progress.

> An escape from configuration hell

debconf. chef, puppet. git. augeas. gsettings. In the Windows world, the registry.

There's tons of work being done in this area. You may say that more can be done, or that it isn't good enough, but I think you're wrong to claim that nobody is working on it.

> Understanding interactions in a large, production system

How about the Google Prediction API? Send the data in, and look for changes when you have a problem that you're trying to fix, such as in your example.

> Pushing the envelope on new computing platforms

This is incredibly vague. I find it difficult to understand your definition of "new computing platforms". Based on my interpretation of what you mean, I think a modern smartphone might qualify: a new form factor, more ubiquitous than before, novel UX (capacitive touchscreen), and an array of interesting sensors (accelerometers, compasses, GPS).

I think your post highlights the lamentation of AI researchers: we don't have a good, static measure of success. Once we've achieved something, our definition changes to exclude it.

<comment><USER17> None of those configuration management projects really advance the state of the art beyond cfengine which has been around for 20 years. They're still just pushing text files around.

</comment></comment><comment><USER18> I think people replying here aren't quite appreciating the complexity of configuration properly here. One might think that configuration doesn't get harder with scale, but it does. Suggesting that the solution is simply using a general purpose language for configuration or storing configs in a key-value db or in a version control system is almost adorable. It's basically completely missing where the pain points are.

And you can be sure that a company like Google will have done configuration using flat files, key-value stores, DSLs, general purpose languages that generate config files, general purpose languages that actually execute stuff, and probably many other ways :-) If there were simple solutions like that, they'd already be in use.

Here's a small list of desirable properties:

\- Configurations must be reusable, composable and parametrizable. When running a system with multiple components, you need to be able to take base configurations of those two components and merge them into a useful combination, rather than writing the configurations from scratch. You must also be able to parametrize those base configurations in useful ways -- which external services to connect to, which data center and on how many machines to run on, etc. And note -- this can't be a one time thing. If the base configurations change, there needs to be some way of eventually propagating those changes to all users.

\- Configurations must be reproducible. If the configuration is a program, it shouldn't depend on the environment where it's run on (nor should it be able to have side-effects on the environment). Why? Because when somebody else needs to fix your system at 3am, the last thing you want them to do is need to worry about exactly replicating the previous person's setup.

\- Tied to the previous point, configurations also need to be static or snap- shottable. If a server crashes and you need to restart a task on another machine, it'd be absolutely inexcusable for it to end up using a different configuration than the original task due to e.g. time dependencies in programmatic configuration.

\- It must be possible to update the configuration of running services without restarting them, and it must be possible to roll out config changes gradually. During such a gradual rollout you need to have options for manual and automatic rollback if the new configuration is problematic on the canaries.

\- Configurations need to be debuggable. Once your systems grow to having tens of thousands of lines of configuration or hundreds of thousands of programatically generated key-value pairs, the mere act of figuring out "why does this key have this value" can be a challenge.

\- It'd be good if configurations can be validated in advance, rather than only when actually starting the program reading the configuration. At a minimum, this would probably mean that people writing configurations that other people use as a base be able to manually add assertions or constraints on whatever is parametrized. But you might also want constraints and validation against the actual runtime environment. "Don't allow stopping this service unless that service is also down", or something.

Some of these desirable properties are of course conflicting. And some are going to be things that aren't universally agreed on. For example there is a real tension between not being able to run arbitrary code from a configuration (important for several of these properties) vs. reuse of data across different services.

My thinking on this is already a few years out of date, it's probable that configuration has again gotten an order of magnitude more complex since the last time I thought about this stuff seriously, and there are already new and annoying properties to think about. The main point is just that this really is a bit more complex than your .emacs file is.

<comment><USER19> The properties that you listed reminded me of the Nix package manager <http://nixos.org/nix/> and NixOS that uses it. I came across it originally while looking into package managers that would allow rollbacks or at least make it easy for multiple installed versions of a package.

<comment><USER20> I had the same thought. (I haven't used Nix, only read about it.)

</comment></comment></comment><comment><USER21> Dealing with configuration files is a problem that extends well beyond distributed systems, or even systems programming in general. It's certainly a very important issue.

One of the fundamental problems with configuration files is that they are written in ad-hoc external DSLs. This means that the config files behave nothing like actual software--instead, they have their own special rules to follow and their own semantics to understand. These languages also tend to have very little abstractive power, which leads to all sort of incidental complexity and redundancy. Good software engineering and programming language practices are simply thrown away in config files.

Some of these issues are mitigated in part by using a well-understood format like XML or even JSON. This is great, but it does not go far enough: config files still stand alone and still have significant problems with abstraction and redundancy.

I think a particularly illuminating example is CSS. When you get down to it, CSS is just a configuration file for how your website looks. IT certainly isn't a programming language in the normal sense of the word. And look at all the problems CSS has: rules are easy to mess up, you end up copying and pasting the same snippets all over the place and css files quickly become tangled and messy.

These problems are addressed to a first degree by preprocessors like Sass and Less. But they wouldn't have existed in the first place if CSS was an embedded DSL instead of a standalone language.

At the very least, being an embedded language would give it access to many of the features preprocessors provide for free. You would be able to pack rules into variables, return them from functions and organize your code in more natural ways. Moreover, you could also take advantage of your language's type system to ensure each rule only had acceptable values. Instead of magically not working at runtime, your mistakes would be caught by compile time.

This is particularly underscored by how woefully complex CSS3 is getting. A whole bunch of the new features look like function calls or normal code; however, since CSS is not a programming language, it has to get brand new syntax for this. This is both confusing and unnecessary: if CSS was just embedded in another language, we could just use functions in the host language.

I think this approach is promising for configuration files in general. Instead of arbitrary custom languages, we could have our config files be DSL shallowly embedded in whatever host language we're using. This would simultaneously give us more flexibility _and_ make the configuration files neater. If done correctly, it would also make it very easy to verify and process these configuration files programmatically.

There has already been some software which has taken this approach. Emacs has all of its configuration in elisp, and it's probably the most customized software in existence. My .emacs file is gigantic, and yet I find it much easier to manage than configuration for a whole bunch of other programs like Apache.

Another great example is XMonad. This is really more along the lines of what I really want because it lets you take advantage of Haskell's type system when writing or editing your configuration file. It catches mistakes before you can run them. Since your config file is just a normal .hs file, this also makes it very easy to add new features: for example, it would be trivial to support "user profiles" controlled by an environment variable. You would just read the environment variable inside your xmonad.hs file and load the configuration as appropriate.

Specifying configuration information inside a general-purpose language-- probably as an embedded DSL--is quite a step forward from the ad-hoc config files in use today. It certainly won't solve all the problems with configuration in all fields, but I think it would make for a nice improvement across the board.

<comment><USER22> You really don't want to use a Turing-complete language for configuration. It makes all sorts of things impossible such as:

\- Automatically scanning for insecure configurations (eg. OpenSCAP)

\- Parsing the configuration in other programs.

\- Modifying the configuration programmatically (cf. Puppet et al)

Also, <http://augeas.net/>

<comment><USER21> Those are the things that would actually get easier: your configuration would be a data structure in the host language rather than a text file. So you wouldn't have to parse it at all. Other programs would have to either be in the same language or have some way of communicating with the host language, but that isn't too bad a restriction.

Similarly, this would make modifying configuration programmatically better. Instead of dealing with serializing and deserializing to random text formats, you would just provide a Config -> Config function. This would also make tools modifying configuration parameters more composable because they wouldn't be overwriting each other's changes unnecessarily.

</comment></comment></comment><comment><USER23> All the three points in the "wishlist" are real-world problems, something which academic conferences will shun straight-away. All work in areas like configuration management must be either a)Evolutionary work based off existing tools. This unfortunately doesn't qualify as good-enough research OR b)A radically new approach, which is almost useless unless a robust enough solution is developed, deployed, and tested. Researchers prefer multiple papers over perfecting the engineering of software tools.

Hence there is no incentive for academic systems researchers to focus on these (it's no fun having paper after paper shot down).

</comment><comment><USER24> In the OP, the

> The "bug" here was not a software bug, or even a bad configuration: it was > the unexpected interaction between two very different (and independently- > maintained) software systems leading to a new mode of resource exhaustion.

can be viewed as pointing to the solution:

Broadly in computing, we have 'resource limits' from the largest integer that can be stored in 2's complement in 32 bits to disk partition size, maximum single file size, number of IP ports, etc. Mostly we don't pay really careful attention all of these resource limits.

Well, for 'systems' such as are being considered in the OP, do consider the resource limits. So, for each such limit, have some software track its usage and then report the usage and especially when usage of that resource is close to be exhausted. And when the resource is exhausted, then report that the problem was the resource was exhausted and who the users of that resource were.

For a little more, can make some progress by doing simulations of 'networks of queues' with whatever stochastic processes want to assume for 'work arrival time' and 'work service times'.

For more, do some real time system monitoring. For problems that are well understood and/or have been seen before, e.g., running out of a critical resource, just monitor for the well understood aspects of the problem.

Then there are problems never seen before and not yet well understood. Here monitoring is essentially forced to be a continually applied 'statistical hypothesis test' for the 'health and wellness' of the system complete with rates of 'false positives' (false alarms) and rates of 'false negatives' (missed detections of real problems).

For the operational definition of 'healthy and well', collect some operational data, let it 'age' until other evidence indicates that the system was 'healthy and well' during that time, and, then, use that data as the definition. Then decree and declare that any data 'too far' from the healthy and well data indicates 'sick'.

Then as usual in statistical hypothesis testing, want to know the false alarm rate in advance and want to be able to adjust it. And if get a 'detection', that is, reject the 'null hypothesis' that the system is healthy and well, then want to know the lowest false alarm rate at which the particular data observed in real time would still be a 'detection' and use that false alarm rate as a measure of the 'seriousness' of the detection.

If make some relatively weak probabilistic assumptions about the data, then, in calculating the false alarm rate, can get some probabilistically 'exchangeable' data. Then get to apply some group theory (from abstract algebra) and borrow a little from classic ergodic theory to calculate false alarm rate. And can use a classic result in measure theory by S. Ulam, that the French probabilist LeCam nicely called 'tightness', to show that the hypothesis test is not 'trivial'. See Billingsly 'Convergence of Probability Measures'.

Of course, the 'most powerful' test for each given false alarm rate is from the Neyman-Pearson lemma (the elementary proofs are not much fun, but there is a nice proof starting with the Hahn decomposition, a fast result from the Radon-Nikodym theorem), but for problems never seen before we do not have data enough to use that result.

For the statistical hypothesis test, we need two special properties: (1) Especially for monitoring systems, especially complex or distributed systems, we should have statistical hypothesis tests that are multi-dimensional (multi- variate); that is, if we are monitoring once a second, then each second we should get data on each of a certain n variables for some positive integer n. So, we are working with n variables. (2) As U. Grenander at Brown U. observed long ago, operational data from computer systems is probabilistically, certainly in distribution, quite different from data from most of the history of statistics. He was correct. Especially since we are interested in data on several variables, we have no real hope of knowing the distribution of the data even when the system is 'healthy and well' and still less hope when it is sick with a problem we have never seen before. So, we need hypothesis tests that are 'distribution-free', that is, make no assumptions about probability distributions. So, we are faced with calculating false alarm rates for multi- dimensional data where we know nothing about the probability distribution.

There is a long history of hypothesis tests for from a single variable, including many tests that are distribution-free. See old books by S. Siegel, 'Nonparametric Statistics for the Behavioral Sciences' or E. Lehmann, 'Nonparametrics'.

For multi-dimensional hypothesis tests, there's not much and still less when also distribution-free.

Why multi-dimensional? Because for the two interacting systems in the example in the OP, we guess that to detect the problem we would want data from both systems. More generally in a large server farm or network, we are awash in variables on which we can collect data at rates from thousands of points per second down to a point each few seconds. So, for n dimensions, n can easily be dozens, hundreds, ....

For such tests, look at "anomaly detector" in 'Information Sciences' in, as I recall, 1999.

If want to implement such a test, might want to read about k-D trees in, say, Sedgewick. Then think about backtracking in the tree, depth-first, and accumulating some cutting planes.

For monitoring, there was long some work by Profs Patterson and Fox at Stanford and Berkelely, in their RAD Lab, funded by Google, Microsoft, and Sun. The paper in 'Information Sciences' seems to be ahead.

More can be done. E.g., consider the Rocky Mountains and assume that they are porous to water. Let the mountains be the probability distribution of two- variable data when the system is healthy and well. Now pour in water until, say, the lowest 1% of the probability mass has been covered. Now the test is, observe a point in the water and raise an alarm. Now the false alarm rate is 1%. And the area of false alarms is the largest we can make it for being 1% (a rough surrogate for the best detection rate -- with a fast use of Fubini's theorem in measure theory, can say more here). As we know, lakes can have fractal boundaries, and the techniques in the paper will, indeed, approximate those. Also the techniques do not require that all the lakes be connnected. And for the dry land, it need not all be connected either and might be in islands. So, it is quite general.

But may want to assume that the region of healthy and well performance is a convex set and try again. If this assumption is correct, then for a given false alarm rate will get a higher detection rate, that is, a more 'powerful' test.

Still more can be done. But, really, the work is essentially some applied math with, at times, some moderately advanced prerequisites from pure and applied math. Theme: The crucial content of the most powerful future of computing is in math, not computer science. Sorry 'bout that!

<comment><USER25> All of this is done at Google - there's extensive monitoring for all production systems, with alerts firing once parameters move outside of their statistical "normal" range. In practice the range tends to get set more by "How tight can we make it before we get really annoyed by these pagers?" than any rigorous statistical method, but the run-time operation carefully measures average & extreme values and determines when a spike is just a momentary anomaly vs. when it's worth paging someone.

The problem is that inevitably the _next_ problem is exhaustion of some resource that nobody even thought of as a resource. In my time at Google, some of the resource limits I've faced have included (besides the obvious RAM/disk/CPU): binary size, size of linker inputs, average screen size of websearch users, time required for an AJAX request, byte-size of the search results page, visual space on the monitoring UI, size of a Javascript alert box, maximum length of a URL in Internet Explorer, maximum length of a request in Google's load-balancers, time until someone submits a CL touching the same config file as you, and time before a cosmic ray causes a single-bit error in memory.

File descriptors are a comparatively obvious resource, but nobody thought of them because in previous normal operation they never even came close to running out. Should someone set an alert on "wind speed running through the Columbia River Gorge" (which would've been responsible for an actual outage had there not been significant redundancy in the system, BTW)?

<comment><USER24> > All of this is done at Google - there's extensive monitoring for all > production systems, with alerts firing once parameters move outside of their > statistical "normal" range. In practice the range tends to get set more by > "How tight can we make it before we get really annoyed by these pagers?" > than any rigorous statistical method, but the run-time operation carefully > measures average & extreme values and determines when a spike is just a > momentary anomaly vs. when it's worth paging someone.

Really, "All"?

First question is, 'parameters'. Usually, and likely at Google, they are monitoring just one parameter at a time. This is a really big bummer. E.g., even if do this for several parameters, are forced into assuming that the 'healthy and well' geometry of the parameters is just a rectangle. Bummer that hurts the combination of false alarm rate and detection rate.

Next, the "normal range" is a bummer because it assumes that what is healthy and well is just a 'range', that is, an interval, and this is not always the case. The result, again, is a poor combination of false alarm rate and detection rate.

Again, yet again, please read again, just for you, as I wrote very clearly, to do well just must be multi-dimensional. I doubt that there is so much as a single large server farm or network in the world that is doing anything at all serious working with multidimensional data for monitoring. Not one.

Next, your remark about false alarm rate points to a problem I point out is solved: The method, with meager assumptions, permits knowing false alarm rate in advance and setting it, actually setting it exactly.

For how many and what variables to monitor, yes, that is a question that can need some overview of the server farm or network and some judgment, but there is some analytical work that should help.

For "rigorous" statistics, the point is not 'rigor' but useful power. Being multidimensional, knowing false alarm rate and being able to adjust it, etc. are powerful.</comment></comment></comment>