Kim Wall Was Stabbed After Boarding Submarine, Danish Prosecutor Says - <AUTHOR> https://www.nytimes.com/2017/10/04/world/europe/kim-wall-peter-madsen-submarine.html?module=WatchingPortal&region=c-column-middle-span-region&pgType=Homepage&action=click&mediaId=thumb_square&state=standard&contentPlacement=1&version=internal&contentCollection=www.nytimes.com&contentId=https%3A%2F%2Fwww.nytimes.com%2F2017%2F10%2F04%2Fworld%2Feurope%2Fkim-wall-peter-madsen-submarine.html&eventName=Watching-article-click <comment><USER2> This has to be the most cut and dry murder that will not result in a conviction since OJ.

<comment><USER3> Why do you anticipate no conviction?

<comment><USER2> While I don't know what the burden of proof is is in Denmark, if it is similar to that in America where guilt needs to be proven beyond "reasonable doubt" I don't see a conviction coming. There are no witnesses, yes she was on the sub and yes her DNA was on the suspect. But his lawyers can claim that he had a co-pilot on the sub who lost it and killed her then abandoned ship. The reason her blood was all over him was to try to save her, then he panicked when she died so he sank the ship. Defense attorneys love to play these games (since its their job to get their client the lowest sentence possible, regardless of their guilt) and I can see a jury or lay judges being able to come to a majority verdict of guilt.</comment></comment></comment>


I Predicted the New Microsoft Ads - <AUTHOR> http://blogs.jobdig.com/wwds/2008/09/10/taking-a-deep-bow-and-kicking-the-dirt-under-my-desk/

<comment><USER2> Coincidentally, I predicted that someone would take credit for predicting that Bill Gates would appear in the advertisements. :-)</comment>


People "probably won't" need discrete graphics cards anymore – Intel - <AUTHOR> http://www.tgdaily.com/content/view/36758/135 <comment><USER2> Meanwhile, Intel looks into taking in nvidia...</comment>


Jumpsuit – A minimal React and Redux framework - <AUTHOR> https://github.com/jumpsuit/jumpsuit <comment><USER2> This really won't teach you either react or redux.

React and redux have been abstracted away to the point that they no longer really serve a purpose... You may as well have used preact, or another toolkit that uses JSX templating.

By abstracting it all away, you loose all the flexibility that higher level components and redux plugins bring to the table.

</comment><comment><USER3> I understand the motivation for reducing react/redux boilerplate, but what does the CLI achieve that isn't already provided by existing tools?

<comment><AUTHOR> Well, for starters, it gets you up and going with a new project in like 60 seconds, as opposed to hooking everything up yourself (package.json, scripts, dependencies). The cli can also do awesome development watching and great production builds with about 10% of the config that webpack does. It's optional, but man, it kicks butt and gets the job done! :)</comment></comment>


How to Make Showering an Amazing Experience - <AUTHOR> https://howitzer.co/the-top-5-weird-fun-things-to-do-in-the-shower/ <comment><USER2> The shower can be a dangerous place - a friends died in the shower (slipped and hit his head). I've seen people slip in the shower and, in attempting to recover, launch themselves across the bathroom (potentially into cabinet corners et al). So I avoid dancing and exercise while showering.

See also

"That Daily Shower Can Be a Killer (nytimes.com)"

[https://news.ycombinator.com/item?id=5145268](https://news.ycombinator.com/item?id=5145268)

Make sure your shower has grab bars, a non-slip pad and is, in general, a safe place to bathe. Occasionally consider, if only for a moment, what you might do were you to slip.

Bath-tubs merit mention in the National Safety Council's data on accidents. See "The most common causes of death due to injury in the United States:" at

[https://danger.mongabay.com/injury_death.htm](https://danger.mongabay.com/injury_death.htm)</comment>


Estimating Coefficients in Linear Models: It Don't Make No Nevermind (1976) - <AUTHOR> http://www-stat.wharton.upenn.edu/~hwainer/Readings/Wainer_Estimating%20Coefficients%20in%20Linear%20Models.pdf

<comment><USER2> Any commentary on why would anyone consider this paper of interest today? The peculiarities of the least squares fit are no secret, the assumptions of the "generalizability of fit theorem" presented are highly unrealistic (compares equal weights to weights chosen uniformly at random), and we are now living in the age of things like Support Vector Machines, Structural risk minimization, VC dimension, boosting and so forth, which all address the issue of overfitting very well. This reads like prehistory of the field, and not very significant at that.

<comment><USER3> The paper is interesting because it is a highly readable and extremely elementary introduction to the topic. Most HN readers don't even know the prehistory of the field, so I think it's useful to them.

<comment><USER2> The rudiments of modern theory can be understood with little more difficulty, and it is clear from those that any significant restriction of the number of degrees of freedom of a model reduces the chances of overfitting occurring, but also decreases the fraction of predicitions the model will get right, so the real issue is where exactly do we draw a line, and this is now understood quite well - the approach from the paper, for practical purposes, throws out the baby with the bathwater. The first few lectures of the course of machine learning by Yaser Abu-Mostafa are a really engaging introduction to those topics:

[https://work.caltech.edu/](https://work.caltech.edu/)

By the way Howard Wainer is a noted author of semi-popular (some formulas actually appear etc.) statistics books, so if enjoyed the writing, maybe a better use of time would be to read his newer and more general stuff:

[http://www.amazon.com/Howard- Wainer/e/B000AP7SUU/ref=sr_ntt_...](http://www.amazon.com/Howard- Wainer/e/B000AP7SUU/ref=sr_ntt_srch_lnk_1?qid=1401098828&sr=8-1)

</comment></comment></comment><comment><USER4> This is extremely interesting. In the forecast domains I work, (I use neural networks on standardized inputs) I typically use 50-100 epochs at most for a few thousand rows (with some approx 10 input neurons) and I don't see networks improving very much after that.

However, our industry peers often use a few thousand up to several ten thousands of epochs, yet our software is regularly best or second best when compared by clients.

I think this may be the same effect: Network weights should be "ok" after a few dozen epochs and any training beyond that maybe wins a percent of variance explained but already risks overfitting.

We use the spare time to train more networks (on slightly different input data) whose results we aggregate after that.

Something else: can someone explain the "I don't make no nevermind", I suppose it's funny but I don't get it.

<comment><USER5> [http://www.urbandictionary.com/define.php?term=it+don%27t+ma...](http://www.urbandictionary.com/define.php?term=it+don%27t+make+no+nevermind) though I don't get how it relates to the paper

</comment></comment><comment><USER6> Given the almost 40 years since this was written, how much has actually changed?

<comment><USER7> Data set sizes have changed. That can make this issue (the tendency of OLS regression to overfit) less or more of a problem.

If you have 100 observations and 10 dimensions (N = 100, p = 10) then OLS can overfit, especially if the predictors are highly correlated or there's a lot of noise on the response. If you have N = 10^6, p = 10, then probably not. If you have N = 10^6 and p = 10^5, then overfitting is again a danger. If you have N = 10^5 and p = 10^6, then you can't even do an OLS regression because, when p > N, it involves inverting a singular matrix (and, to boot, a gigantic one). The rough guideline is:

        N/p > 100: very low danger of overfitting.
        N/p < 100, > 10: low danger of overfitting.
        N/p < 10, > 1: high danger of overfitting. Use regularization (ridge, Lasso, etc.)
        N/p < 1: OLS not even well-defined. Regularization mandatory.

Equal weights is a primitive way of regularizing. You lose accuracy in the zero-noise case but get a model that is more likely to be "wrong" in a way that is harmless in prediction (because of correlations in predictors). These days, people are more likely to use ridge regression or, if they want a parsimonious model (few nonzero coefficients) they'll use Lasso or the elastic net.

Though you almost never see equal weights in a _regression_ setting, you do see equal-weight constraints in neural network architectures (e.g. convolutional neural nets) to prevent overfitting. In neural nets, this usually can be done with absolutely no loss of accuracy because ANNs are already so highly parameterized that some form of regularization (weight decay, equal-weight constraints, early stopping) is mandatory.

<comment><USER4> I was asking about that in another thread. Do you have any references about you N/p rule of thumb? Thanks in advance!

<comment><USER7> That N/p rule of thumb is ultimately subjective, except for the fact that when N/p < 1, you can't use OLS anymore (because there is an infinitude of zero- training-error solutions). It depends on correlations of predictors and signal/noise ratio as well. Those are just approximate guidelines.</comment></comment></comment></comment>


Show HN: An idiot-proof crypto library for Python 3 (feedback, please) - <AUTHOR>

I really couldn't find a simple wrapper to pycrypto that would encrypt data correctly in Python 3.
So I tried to follow the instructions at http://www.daemonology.net/blog/2009-06-11-cryptographic-right-answers.html (written by HN's cperciva) and wrote https://github.com/andrewcooke/simple-crypt
Is that OK?  Given the problems Tarsnap had with CTR mode, was it wrong to follow those instructions anyway?  Have I got the offset handling right?
It seemed like it would be better to write this once, get it revised here, and then publish it (I'll make an egg for pypi once it seems OK) than quietly write my own and let others do the same (it was surprisingly complex - I mention the Tarsnap issues because after reading about them realised I had the same error...).
Apologies if I overlooked an existing package.  And thanks for any constructive criticism. <comment><USER2>
        def encrypt(salt, password, data):
                    ^^^^^^

You're the crypto library. Don't ask the user to choose the salt, they won't do it right.

        key = _expand_key(salt, password)

Most crypto people would advise you to generate two separate keys for encryption and MAC.

        hmac = data[-HMAC_HASH.digest_size:]
        hmac2 = HMAC.new(key, data[:-HMAC_HASH.digest_size], HMAC_HASH).digest()
        if hmac != hmac2:
           ^^^^^^^^^^^^^

Possible timing attack. Consider doing another HMAC of each HMAC and comparing those instead. See [https://www.isecpartners.com/news- events/news/2011/february/...](https://www.isecpartners.com/news- events/news/2011/february/double-hmac-verification.aspx)

        offset = getrandbits(AES.block_size*8)
        Counter.new(AES.block_size*8, initial_value=offset, allow_wraparound=True)

I don't like that you allow wraparound counters, in fact, I'd go so far as to call it broken. This is not allowed by NIST-specified CTR mode.

Bad Things^TM will happen if multiple messages (or different parts of a single message) are ever encrypted with the same key. Seriously, screwups like this have shortened wars.

         return prefix + encrypted + hmac

Put some type of magic constant in front of this. You'll be glad you did later when you have to change the format or simply the PBKDF2 count.

         HMAC_HASH = SHA256

but later... def _expand_key(salt, password): ... return PBKDF2(....)

I don't see where you're telling PBKDF2 what hash to use for its HMAC. It's probably defaulting to something like SHA-1.

<comment><AUTHOR> whoa. thanks. about to go to bed(!) but will address that (dual hmacs) tomorrow am. thanks again!

edit: on the wraparound counters, i can replace it with a nonce (prefix) and smaller counter (and no wraparound). i thought they were equivalent, but people do seem to prefer a prefix (my reasoning was that if you combine nonce and counter with addition rather than concatenation then you need wraparound since you may start at the end of the counter).

edit2: so what's a reliable way of providing salt? i can't think of one that's context-independent except for simply generating some random junk and appending it to the message. is that ok? (i guess so)

ps, also, while here, i will drop this link which so far has no replies [http://codereview.stackexchange.com/questions/19910/simple-c...](http://codereview.stackexchange.com/questions/19910/simple- crypto-library-in-python-correct-and-secure)

<comment><USER2> Yeah, sorry for all the edits.

Just pick a secure random number of 128 bits for the salt.

And read the NIST docs on CTR mode. Merry Xmas.

</comment></comment></comment><comment><USER3> _Given the problems Tarsnap had with CTR mode_

FWIW, the Tarsnap bug wasn't a cryptographic mistake, it was a refactoring mistake -- I knew what the code should be doing, and the code was doing exactly what the code should be doing, right up to the point when I decided to tidy it up.

The lesson to be learned from that isn't "crypto is hard" or "CTR is dangerous", but rather "pay attention when you rewrite existing code".

<comment><AUTHOR> ok; in my case i was confused by the way pycrypto handles ivs for ctr mode (it ignores the iv and expects an explicit offset or prefix to the counter). so initially i had the same encryption every time.

but anyway, ignoring that red herring, if you have the time, confirmation that it seems to do what you described would be appreciated...

[edit: fwiw, other concerns i have include: is there any reason why hmac shouldn't use the same key as the main cipher; is it better to use a random counter offset or a smaller counter and a prefix / nonce?]

</comment></comment><comment><AUTHOR> Clickable links:

[http://www.daemonology.net/blog/2009-06-11-cryptographic- rig...](http://www.daemonology.net/blog/2009-06-11-cryptographic-right- answers.html) (not me!)

<https://github.com/andrewcooke/simple-crypt> (main page, inc example)

[https://github.com/andrewcooke/simple- crypt/blob/master/src/...](https://github.com/andrewcooke/simple- crypt/blob/master/src/simplecrypt/__init__.py) (source)

</comment><comment><USER4> Very cool! I've been using m2Crypto, Tomcrypt, and PyCrypt manually over the last year or two - I've tried various variations, and changed it up a few times.

Most recently, though, I've moved to GPG, because frankly, I don't trust myself to do it right.

Your library looks cool, but with all respect, I don't trust you either yet ;)

Good luck with this - It seems really useful, and I hope it gets some more exposure, and can cook for a bit.

</comment><comment><USER5> As the saying goes: "Someone is always going to build a bigger picture idiot."</comment>


The Psychological Price of Entrepreneurship - <AUTHOR> http://www.inc.com/magazine/201309/jessica-bruder/psychological-price-of-entrepreneurship.html

<comment><USER2> Being a first-time bootstrapper, I am striving very hard to minimize financial exposure. However, this means that I had to convince my cofounders and team to work for free; or rather, they agreed to join because they believe in the vision and for the learning experience and journey rather than for immediate monetary compensation.

Although they admittedly joined on happily and believe in our potential, I do struggle with doubt and accountability to them. The article seems to emphasize the mental strain caused by blows to self confidence or finances, which can certainly be crippling and terrifying.

However, by minimizing financial exposure and trying to keep a perspective that protects self-confidence, those are less of a concern to me compared to facing the possibility of failing my friends who contributed their time freely and signed on to what was originally my vision.

The pressure and anxiety over prospect of disappointing the friends who became teammates based on their belief in you and your vision seems hard to avoid. To be blessed by forgiving loved ones, friends, and teammates is priceless and the articles recommendation to "Most important, make time for your loved ones...Don't let your business squeeze out your connections with human beings...When it comes to fighting off depression, relationships with friends and family can be powerful weapons." resonates the most.

</comment><comment><USER3> This really rings true for me. I literally found out today that my co-founder is quitting so he can have a better lifestyle. I'm glad he was honest with me and I don't harbor ill feelings but it definitely is making me feel very alone. I'm not sure if I should give up and get a day job or continue on my own. I feel like taking a job would just cement my failure but maybe that has already happened. Anyway, this article was pretty timely for me. I think I'll be scheduling some lunches with friends who I can vent to and try not to keep it all in.

</comment><comment><USER4> I really wish, as a culture, we celebrated honesty as much as bravery. Good on Feld and Inc for starting.

<comment><USER5> > I really wish, as a culture, we celebrated honesty as much as bravery.

But if we did that, we would have nothing to celebrate. Everyone lies, but not everyone is brave. I invite you to divide bravery by honesty, but watch out for a divide by zero error.

<comment><USER4> Interesting. Admittedly it gets hard with employees, investors and markets, which are all prone to emotional decisions. But I think we're messing with our happiness if we conflate bravery with false courage. Courage/Bravery isn't about denying fear, it's about not letting fear lead to bad decisions.</comment></comment></comment>


RubyMotion build system is now open source - <AUTHOR> http://blog.rubymotion.com/post/24197887535/community-open-source-updates

<comment><USER2> I would love to see RubyMotion find a way to open-source its entire stack and still be able to monetize.

Ideas:

\- Only charge a license fee if you intend to publish to the app store. True, this would be hard to check, but I think it would let the ecosystem flourish

\- Give away the platform, sell premium documentation, screencasts, books, etc. (Although I can't see this generating enough revenue. Maybe when combined with some other approach?)

\- Make one kick-ass IDE that's leaps and bounds better than XCode (including a better interface builder). Sell that. I favor this idea :)

<comment><USER3> Laurent Sansonetti (creator of RM) was on the Ruby Rogues podcast ([http://rubyrogues.com/055-rr-rubymotion-with-laurent- sansone...](http://rubyrogues.com/055-rr-rubymotion-with-laurent-sansonetti/)) and made a compelling case for the current pricing scheme. I won't paraphrase Laurent, but if you're interested, listen for yourself.

</comment></comment><comment><USER4> Glad to see Laurent open sourcing this piece - hopefully it will lead to even deeper community involvement. In the few short weeks since it was released, RubyMotion already has a really impressive community behind it and I'm excited to see what the future holds.

</comment><comment><USER5> Nice, I had not heard of RubyMotion. How does it work? Is there anything similar for Android?

<comment><USER6> Have you tried Ruboto (<http://ruboto.org/>) ?

<comment><USER7> This is not the same, because RubyMotion compiles directly to native code.

The best would be to have something that could make use of Android's NDK.

</comment></comment></comment><comment><USER8> I just bought my copy of RubyMotion, does open sourcing the build-system means that I can now build iOS apps using RubyMotion for Free? If so, I just wasted $150 dollars...

</comment><comment><USER7> It would be very nice to have a native code compiler for Ruby with support for more than just iOS.</comment>


How the modern containerization trend is exploited by attackers - <AUTHOR> https://kromtech.com/blog/security-center/cryptojacking-invades-cloud-how-modern-containerization-trend-is-exploited-by-attackers <comment><USER2> If you didn't build the container, then you are putting all your trust and your companies private bits in the hands of Joe Random.

Please do not blame the technology. This problem existed long before containers and will exist long after they are gone. This is people trusting unknown anonymous third parties to build things that will run in their datacenter.

<comment><USER3> Isn't that always true though? This is just one additional layer of trust. Sure, there are reasonable layers we should care about, but you're rarely, if ever, going to be doing everything and trust everything.

Ie:

> If you didn't build the container..

> If you didn't build the package on Debian..

> If you didn't verify the source code when compiling by hand..

etc.

<comment><USER4> > _to be doing everything and trust everything_

Also, it's sort of weird how often people conflate these two things. There's this idea that home-rolling is naturally safer, and it's simply not true.

Everyone doing anything with software is relying on layers someone else built, and we should keep doing that. Layers I handle myself are layers that I know aren't malicious, but that doesn't mean they're secure. The risk of malice doesn't just trade against convenience, but against the risk of _error_. Using somebody else's programming language, compiler, crypto, and so on doesn't just save time, it avoids the inevitable disasters of having amateurs do those things.

We live in a world where top secret documents are _regularly_ leaked by people accidentally making S3 buckets public. I'm not at all convinced that vulnerable containers are a bigger risk than what the same people would have put up without containers.

<comment><USER5> There's this idea that as long as everything is not rigorously proven secure, we might as well grab binaries of file sharing sites and run them in production.

This argument tires me. Every time some smug developer asks me if I have personally vetted all of gcc, with the implicit understanding that if I haven't we might as well run some pseudonymous binaries off of docker hub, I extend the same offer to them: Get a piece of malware inside gcc and I will gladly donate a month's pay to a charity of choice.

Sometimes I have to follow though the argument with the question if they will do the same if I get malware on docker hub (or npm or whatever) but the discussion is mostly over by then. Suffice to say, so far nobody has taken me up on it.

The point is, that there's a world of difference between some random guy on github and institutions such as Red Hat or Debian or the Linux kernel itself. Popular packages with well functioning maintainers on Debian will be absolutely fine, but you probably shouldn't run some really obscure package just because some "helpful" guy on Stack Overflow pointed to it, and you certainly shouldn't base your production on some unheard of distribution just because the new hire absolutely swears by it.

<comment><USER6> Right. All-or-nothing thinking is the bane of analysis, and philosophy in general.

</comment></comment></comment></comment></comment><comment><USER7> The headline is misleading. It's not "the modern containerization trend" that is the root cause of this. I expected to read about something about container breakout or the difference between container confinement and VM confinement.

Instead, it turns out that it's the "store model" (Docker Hub in this case) and malware injection into that store that the article is really talking about.

The article also seems to talk about misconfigured systems permitting some level of admin access to everyone. That's not really a new "container" class of vulnerability though; it's the equivalent of leaving root ssh open with a weak password or similar.

<comment><USER8> Even that isn't quite it - this is not a case of people accidentally downloading and running malicious containers.

People are leaving kubernetes/docker/whatever open to the world, and attackers are instructing their servers to download and run these containers.

The complaint is that Docker Hub is hosting the attack code for the attackers. They could have hosted it on their own custom registry server if they wanted. (But why bother if you can just host it on Docker Hub.) In the same vein, they could use GitHub to host their attack code. Or they could put in in an S3 bucket...

</comment></comment><comment><USER9> Whilst this article has some decent points, I feel it overblows/mis-understand others.

It's fair to say that downloading and running images from Docker hub without establishing trust is a dangerous practice.

Similar in danger to using npm, rubygems, nuget, Maven central etc. In that there is only limited curation of content.

That attackers have "malicious" images on Docker hub isn't that relevant, unless they can get people to execute them. If they were typo-squatting or otherwise trying to trick users into running those images that would be more relevant. Instead what seems to be being described is the use of Docker images as part of attacks on other systems (e.g. Kubernetes installs with poor security)

The bit around running a malicious container instantly leading to root on the host is just wrong. With a standard Docker install, no customization, there are some risks, however unless you do something like run --privileged, or mount a docker socket inside the container, you're not guaranteed to be able to get root on the host.

(BTW anyone who reckons this is trivial should give contained.af a look)

</comment><comment><USER10> _disclaimer: Security Engineer at Docker_

It is _VERY_ hard to do runtime detection of mining apps for two reasons:

1) it's mostly CPU usage intensive work and only if you know what's the average amount of computer power needed by your application upfront will you be able to make a policy decision on which image to stop and how to adjust Cgroups resources. If you don't, you'll have to build a reference profile of a trusted image anyway to be sure of what's the expected behavior.

2) There is no other "malicious" activity that might be reported by runtime security tools (it generally doesn't trigger anything blocked by your seccomp/LSM/filesystem-integrity profiles).

\</comment><comment>- How to protect against this </comment><comment>-

The best protection is at the build chain level. There are tools out there to "bless" and/or verify an image's content/creator. Notary and Docker Trust (higher-level abstraction based on Notary inside Docker) are two tools that allow you to do:

      - key management

      - signer management

      - trust management

over Docker images.

It is crucial for people out there to make sure they only deploy trusted images and make decisions on what to run (CI or Prod) based on signature integrity of trusted images.

For a quick tutorial on Docker Trust and Notary, check this out: [https://github.com/dockersamples/dcus-2018-hol/tree/master/s...](https://github.com/dockersamples/dcus-2018-hol/tree/master/security/trust)

Stay safe and do not run unsigned/untrusted images!

<comment><USER11> Isn't the real problem mentioned in this article that people are running their docker daemon unauthenticated on public endpoints? That's not the default behavior right? So people have actually gone out of their way to make themselves insecure.

Look at the names of the containers in the article. Nobody is pulling these themselves. The problem is attackers compromising docker hosts and pulling arbitrary containers.

What safeguards does docker provide against exposing the daemon publicly, accidentally or otherwise?

<comment><USER10> The daemon is listening by default on a non-networked unix socket so if you're exposing listening on the network, you're already out of the default behavior (which is totally normal but that means that you've started regarding the instructions/doc on how to do so, and our doc page on this matter also includes security guidelines to enforce TLS verification/whitelisting daemon- side).

There is currently no "superduper-safe-mode" that enforces `--tls-verify` at the daemon-level to prevent lack of client verification/whitelisting. This can be discussed, the issue obviously being the UX (that means getting proper certs, specifying them in the config etc..).

</comment></comment></comment><comment><USER12> Containers were cool. No config, single deploy, etc.

But now I have to configure many yaml files, launch config and build script that do god knows what to databases and other config scripts hidden inside the container.

Some projects are really cool and easy to install while others are just pile of hacks.

<comment><USER13> Stateless services are all nice and dandy, and all the marketing you'll see is about stateless services. The trouble starts when you:

\- run a stateful service that's not explicitly designed to work well in a distributed fashion.

\- have services that need to be started in a precise order.

</comment></comment><comment><USER14> > By default, docker containers run as root which causes a breakout risk. If > your container becomes compromised as root it has root access to the host.

Is this really true, unless you start container with `--privileged`? Incidentally, I just read plan for better security defaults to avoid `--privileged` (which is not default, AFAIK) on lwn: [https://lwn.net/Articles/755238/](https://lwn.net/Articles/755238/)

<comment><USER15> It's about UID mappings between namespaces. When you are UID=0 in namespace X and manage to get out of namespace X, then you are still UID=0 outside X, so you're root.

It's possible to remap UIDs such that root in namespace X has UID=12340, and when root gets out of X, then he's nobody.

<comment><USER9> it's not quite as straightforward as just UID mapping. Assuming a standard install of Docker, the container processes only have a limited set of capabilities, have an AppArmor/SELinux profile applied and have a seccomp filter also applied, which makes it harder to break out the the underlying host.

</comment></comment></comment><comment><USER16> Maybe this isn't clear, but something I read a while back on this topic:

Running a container from dockerhub is basically the same as curl piping into bash.

<comment><USER17> What? No.

Curl piping into bash will trivially steal all of your data at once.

Running a container from dockerhub is much safer, provided you do not give it privileges using --privileged or bind-mounting system files like docker control socket.

If your system is up to date and there are no docker 0-days active, the worst "docker run --rm -it RANDOM-CONTAINER" can do is to use too much resources -- your local secrets would be safe.

<comment><USER18> ...unless said docker container is running an app server that has direct access to your database.

</comment></comment></comment><comment><USER19> If a container system were formally-proven to provide all of these:

\- hard limits, priorization and accounting metrics of all resources, incl.: IO, storage, compute, mem, net, kernel structures

\- provable isolation / no side-channel leaks

\- SELinux

\- Live migration of processes and storage to different hosts, suspend/resume

\- Type 4 hypervisor containers for different kernels, OSes, etc. configured and managed seamlessly with the same API

Then and only then can the jumble and complexity of containers running on hypervisors go away and be more like SmartOS with an ability to run bare-metal without losing devops flexible capabilities of running T4 hypervisors under everything.

<comment> yjftsjthsd-h Okay, quick search failed me: what is a type 4 hypervisor?

</comment></comment><comment><USER20> One interesting point here, that Matt Levine has pointed out, is that previously, upon penetrating a company's network, hackers would do malicious user-harming things like stealing credit card or personal information.

Now, (some) criminals are just running mining software.

As a company, your bottom line might care more about the latter than the former, but as a consumer, this is great news; if hackers lose interest in our data and start stealing and re-selling compute cycles, then the chances of catastrophic identity theft could go down dramatically. Sure, prices for web services might go up a little, but they'll do so across the board.

</comment><comment><USER21> I feel like the guys in Office Space here, but I'll buy some subscriptions to Vibe if somebody can explain theoretically how you can convert a large sum of ill-gotten cryptocurrency to usable money without getting immediately caught?

From the article it sounds like the attackers' wallet id is hard-coded into the malware. I'm not familiar with monero, but aren't all transactions in cryptocurrency public and permanent?

Wouldn't it be obvious from following that who is ultimately benefiting from this?

<comment><USER22> Monero has the "laundry" part built in.

I also don't think these criminals necessarily convert to fiat to aquire what they need/want.

<comment><USER23> When you're talking about millions of dollars of bitcoin or other altcoin stolen...well, it's hard to smoke that much crack.

<comment><USER24> You can buy gold with crypto.

<comment><USER23> Millions of dollars worth? Or is it some penny-ante exchange like most Bitcoin-to-fiat services? Or will it have a strong know-your-customer rules that make it problematic for these actual criminals?

<comment><USER25> I don't see why not. Transactions worth tens of thousands of dollars are relatively common as I understand it.

</comment></comment></comment></comment></comment></comment><comment> squeaky-clean I think I'm missing something about this attack, how does one actually get attacked? From the article it sounds like a combination of bad firewall configuration and an exposed Docker Daemon allows the attacker to install any image they want into your host?

Why are so many commenters here saying things about running untrusted Docker containers then? From my interpretation, the people affected didn't go to dockerhub.com/docker123321/maliciousCopyCatImage and run it. Instead it was injected into their otherwise-safe containers and they had no idea this ever happened until things started acting weird. Am I missing something?

<comment><USER26> I also had trouble following this article.

The Tesla vulnerability at the top seems to be simply a misconfigured kubernetes cluster, through which the attackers were able to get at AWS credentials. But then all of the subsequent examples are malicious images hosted on dockerhub. Were malicious images involved in the Tesla exploit in some way? What's the connection?

</comment></comment><comment><USER27> Here is one thing I have trouble understanding, despite working in cybersecurity.

If I run a VM, I have to harden that VM. If I run a Docker container on top of that, I now have to harden the Docker container as well. This is more work, and a greater "surface area" of attack.

Forgive my ignorance, but do most ppl run Docker through managed PaaS services now, so that they don't have to worry about the double work of hardening the VM? That's the only way I see it making sense long-term, where the Cloud providers manage the physical infrastructure like they do now _as well as_ the EC2 / IaaS later.

<comment><USER28> Most people just assume that Docker is a magic box that solves all of their problems. They build Dockerfiles that depend on a grotesque cascade of hardly- vetted parent images, because then their Dockerfile is "only three lines! Ha! Take that, old guys!"

If people had been asking any of the basic system engineering and security questions, we wouldn't be talking about this, or to be frank, most of the stuff that passes for "DevOps" these days.

</comment></comment><comment><USER29> Take WordPress press for example. If you've never before setup a LAMP stack to run WordPress, you probably be running a WordPress container. And if you don't regular check the log files and monitor system resources, then you're asking for these problems.

I wouldn't run a server with a pirated version of Windows and I wouldn't run a dodgy container from an untrustworthy source. Yet people do. I wouldn't blame MS for a pirated Windows that came with malware like I wouldn't blame containers for this either. It's all about Trust.

</comment><comment><USER30> Although I understand reading the sources of all software we're running would take way too much time to be reasonable, running a quick docker inspect / docker history on all images we use is, in addition to be interesting, probably a big first layer of protection.

Having a tool doing this for us - i.e a sort of docker anti-malware, that would inspect images and containers for us, without necessarily go through all the security stuff the official tool checks - would also be very handy.

<comment><USER31> There is an entire genre of such tools now. Some names to look up are BlackDuck Hub[0] (commercial) and CoreOS Clair[1] (opensource).

At Pivotal we use both -- BlackDuck is built into a number of our pipelines and Clair is shipped as part of PKS (in the Harbor registry[2] contributed by VMWare).

A lot of our customers also use other security scanning tools that have expanded to include container scanning.

[0] [https://www.blackducksoftware.com/products/hub](https://www.blackducksoftware.com/products/hub)

[1] [https://github.com/coreos/clair](https://github.com/coreos/clair)

[2] [https://github.com/vmware/harbor](https://github.com/vmware/harbor)

<comment><USER30> I didn't know them. Thanks for bringing them up!

</comment></comment></comment><comment><USER32> You can easily avoid this by only using Official Images and Certified Content from the Docker Store: [https://store.docker.com/](https://store.docker.com/)

</comment><comment><USER33> I would like to get more research of this sort.

Would someone kindly point me towards it?

Thanks.</comment>


CSS image sprite easter egg from BankSimple's signup form - <AUTHOR> https://www.banksimple.com/img/sprites.png

<comment><AUTHOR> just put the name "drop table" in the form.</comment>


Airbnb Project Lighthouse Methodology [pdf] - <AUTHOR> https://news.airbnb.com/wp-content/uploads/sites/4/2020/06/Project-Lighthouse-Airbnb-2020-06-12.pdf <comment><USER2> Here is the announcement Airbnb is sending to users:

         </comment><comment></comment><comment></comment><comment>--

Subject: _How you can help us fight discrimination_

Now more than ever, we need to fight for equality and justice. That's why we recently announced Project Lighthouse—an initiative to uncover, measure, and overcome discrimination when booking or hosting a home on Airbnb. We'd like to use information from hosts and guests in the US to help, so we want to make sure you understand how it works.

Here's what you need to know:

• _We'll only use information you already share_

This project will address discrimination that's based on perception—so we'll use first names and profile photos from hosts and guests to help us understand the perceived race someone might associate with them.

• _We'll use it to uncover patterns of discrimination_

We'll use this information to help us understand when and where racial discrimination is happening on our platform. Any insights will be used to help develop new features and policies that create a more equitable experience for everyone.

• _Information won't be tied to your specific account_

We know your privacy is important, so we analyze trends in bulk and Airbnb won't associate perceived race information with your account. We won't use this information in marketing or advertising, and it will only be used for anti-discrimination work.

• _We consulted with and solicited input from leading civil rights and privacy organizations to guide us_

We know how delicate this work is—so we developed this work with support and input from leading civil rights organizations like Color Of Change and Upturn, along with privacy organizations like Center for Democracy & Technology, to make sure our approach is both thoughtful and respectful of your privacy.

You can opt out in your Privacy Settings if you don't want to participate. And though you can change your mind at any time, your first name and profile photo may be used if you don't opt out by August 31, 2020.

Read our frequently asked questions: [https://www.airbnb.com/resources/hosting-homes/a/a-new- way-w...](https://www.airbnb.com/resources/hosting-homes/a/a-new-way-were- fighting-discrimination-on-airbnb-201)

By including your information, you're helping us identify and prevent discrimination from happening on Airbnb—and you're taking part in an initiative to better understand how and where discrimination happens on platforms like ours. This is an important step for our industry, and we're making the methodology publicly available to other companies who want to join in the fight.

In solidarity,

The Airbnb team</comment>


Spore Prototypes (2008) - <AUTHOR> http://www.spore.com/comm/prototypes <comment><AUTHOR> I never cared much for the final game itself, but the 2005 GDC presentation on Spore was and still is magical. One of the best demos in history:

[https://www.youtube.com/watch?v=N4ScRG_reIw](https://www.youtube.com/watch?v=N4ScRG_reIw)

You can hear the sheer awe in the audience whenever Wright would zoom out to the next stage.

<comment><USER2> I wonder what would have happened if EA went with Will Wright's vision instead of the disaster that was released

<comment><AUTHOR> Probably the same thing that would have happened to SimCity (2013) had EA not also ruined that.

On that note, I happened upon some concept renderings yesterday, and it broke my heart:

[https://www.eywong.com/projects/0WRr5](https://www.eywong.com/projects/0WRr5)

[https://www.eywong.com/projects/dD5Ae](https://www.eywong.com/projects/dD5Ae) (scroll down)

I don't know how someone wakes up and goes "Oh boy, we better close Maxis." It boggles the mind.

Were it not for their interference, Maxis would've probably been incredibly profitable for EA, even today.

<comment><USER3> Wow this looks amazing.

I don't exactly understand, how the texturing works:

"Buildings were textured using a procedural system which utilized a shader that could overlay 2 UV regions from a single atlas. Each UV tile could be spread apart with an offset to get the proper spacing between windows and other detail elements"

This sounds like magic to me.

<comment><USER4> I _think_ I get it, it's pretty smart. So you have a texture atlas - a big texture with many small textures packed into it. Each small texture occupies a rectangular UV region in the big atlas, and when you are looking up texture coordinates x,y of the small texture, you remap them from (0,0)-(1,1) to the coordinates of the region in the atlas. This lets you avoid having a ton of small textures and instead use one big texture, only changing the UV region before drawing each model.

What he's talking about is a more advanced version that lets you do multitexturing from a single atlas - use two small textures and blend between them, with each having different tiling settings to make everything line up. The way I imagine it works is like so:

You're trying to draw a concrete cube with windows on each side, spaced 1m apart vertically. You load up a concrete texture and a window texture in the atlas. You set up the concrete texture to tile continuously. You set up the window texture to repeat with an offset that lines it up with the building's UV coordinates such that the windows appear horizontally and vertically where they should. The shader then chooses the window texture if the UV coordinates fall within the window tile, or the concrete texture otherwise. Depending on how complicated you make the tiling, you can get a lot of effects this way.

P.S. Look at the second link's first screenshot. The building that's second from the bottom, facing towards us. The windows are all spaced equally.

</comment></comment></comment></comment></comment><comment><USER5> The biggest complaint about Spore was that it felt like a collection of minigames without an unifying vision. Maybe developing your game by aggregating a bunch of minigames is a risky idea.

<comment><USER6> The minigame aspect doesn't really matter -- most games can be considered a bunch of minigames wrapped up, where each component is really its own subgame (eg weapon crafting vs fighting vs town-building vs etc), all threaded together to produce some overarching value.

The problem with spore is that it had a very clear, and obvious, and intentional thread -- the evolution of your creature. Each minigame was _supposed_ to impact the way your creature developed -- the constraints of your environment, your skill, your choices, etc, were _supposed_ to guide your creature down a path such that your choices from minigame 1 would change what you'd have to work with by minigame 3.

The problem was that nothing mattered. It wasn't that there was no unifying vision -- it was that the game _pretended_ everything mattered, but you'd chug along and realize there was literally nothing affected by anything you did -- the whole thing was an empty shell.

It was supposed to be a (perhaps goofy) cross-millenia simulation, and instead it was.. 5 fairly worthless games mashed together to produce one large worthless game.

The crime too is that mario party _knows_ their minigames are shallow -- so they last for at most minutes, and offer like 60 of them. Spore, of course, somehow believed it had some kind of depth, and run you through each minigame for hours.

<comment><USER7> >The problem was that nothing mattered. It wasn't that there was no unifying vision -- it was that the game pretended everything mattered, but you'd chug along and realize there was literally nothing affected by anything you did -- the whole thing was an empty shell.

This is how I felt when I first played Spore, but since then, I've come to feel this way about all single-player games and most multi-player games, honestly. The only games I've played where what you do _really_ affects the world in a lasting and true way are sandbox MMORPGs. (Examples: EVE Online, Ultima Online, Shadowbane, Darkfall, etc.) Imagine some of the sandbox aspects of Spore, but with tens of thousands of other players in real-time.

I think sandbox MMOs are a massively untapped niche. The problem is most of them have been developed by companies lacking funding and experience (and often competence), so they're usually rife with game-killing issues, and most people don't get interested in them or stick with them. Another inherent issue is that a lot of players aren't comfortable with the idea of semi-realistic consequences upon death, like other players taking everything in your gear and inventory, or perhaps even taking your clan's/nation's/corporation's city/territory that you painstakingly built over months. If you're just a casual video game player, harsh consequences and ruthless risk vs. reward often won't appeal to you.

I suspect there'll one day be a "killer app" of sandbox MMOs that'll put them more into the mainstream, but I think it may be a decade or more away.

<comment><USER6> The question of games as simulation is actually a very interesting one, and in my opinion, the one true way of viewing games -- every game is a simulation of some fictional universe, and the player's primary goal is to understand, interact and finally abuse that simulation to achieve his goals.

Sandbox MMOs are the end-game, but I don't think we exactly need them just yet -- we can get 80% of the way there in a much simpler fashion -- you don't need true simulation; you just need sufficient simulation. You don't need world- modification, you just need sufficient world-interaction. You don't need environmental destruction and creation, you just need environmental reaction. The true goal is a reactive, but self-stabilizing, environment.

For example, the mistake Ultima Online made with their simulation was to make the environment self-containing -- which lead to catastrophe as soon as the assumptions were violated (players hoarding resources, and not feeding it back into the system). They had the simulated, editable environment.. but it wasn't self-stabilizing (or even stabilized by GOD aka Origin Systems), and it badly needed to be.

Dwarf fortress makes the interesting modification of allowing, accepting and even _encouraging_ catastrophic environmental reaction -- _losing is fun_. A sandbox MMO obviously can't do the same, because it'd be too detached from any individual player's interaction, but there's clearly variation that can occur in how you run the simulation

But currently most games don't understand that simulation is the one true goal, so they don't even get close. We're not at 80% of the way there.. we're more like 10%, except in a very few titles. EVE is probably the only thing that makes any real progress towards it.

But anyways that's far more advanced than what Spore needed to be a _good game_. Spore didn't need to be an advanced simulator in each of its stages -- it just needed to have some legitimate reactions to the choices you made. It needed a basic simulator in each stage, but really the only simulation with any merit was stage 1 (the bacteria or whatever). After that.. nothing you did mattered except for stats.

</comment></comment></comment></comment><comment><USER8> Some people are working on an open source spiritual successor [0]. If you were in awe of the original 2005 demo and what could have been you should definitely check it out. The cell stage is in early stages (heh) but is already quite enjoyable.

[0] [https://github.com/Revolutionary- Games/Thrive](https://github.com/Revolutionary-Games/Thrive)

</comment><comment><USER9> Spore was an amazing experiment. It was a terrible game, but I'm glad they tried it.

<comment><USER10> I felt the same way. My friends and I _obsessively_ waited for it to come out back in high school, but were all pretty disappointed when we actually played it. It was kind of nostalgic when No Man's Sky essentially followed the same trajectory a decade later.

<comment><USER11> No Man's Sky had an epic redemption story after its disastrous launch.

<comment><USER12> They added a lot of what was missing for sure, but No Mans Sky still feels like it lacks depth when I last played it (granted, I haven't played it in about a year so don't know what it's like _right now_ )

</comment></comment></comment></comment><comment><USER13> I wonder what a modern Spore would look like. Both the procedural aspects and the graphics could could be made much better with the processing power available today.

The world of Spore after the water stage felt a little empty, I'm sure modern computers could generate more variety of plants and animals. Moreover, it always felt a little claustrophobic with the limited rendering distance. Spore should've been all about the beauty of ecosystems/worlds— I'm imagining Minecraft-like random weather events and biomes, caves and mountains into the distance. At the planet-level, I'm imagining something that feels more like a planet vast and diverse in weathers, features, etc. with a lot of detail if you zoom in.

Video-game aesthetics and video cards have also progressed a lot since Spore's original release. It could also take on a different style, less cartoonish and maybe more realistic, or more artistic, or both.

Again, I'm thinking of Minecraft, with sunsets, a vast sky, inspiring piano music playing in the background, etc. Maybe something in the style of Journey, with a well-thought out color palette, or like Breath of the Wild, with a unique art style and shading.

<comment><USER14> It sounds like that would have a lot of the same issues that No Man's Sky did. It's hard to make a procedurally generated world be meaningful. I don't think that problem will be solved by adding more processing power - we need a breakthrough in game design or theory

<comment><USER15> Gameplay is often hard, they played it safe by using well tested concepts. There is however a "hack" you can do if you you have a cool engine but lack gameplay - just make it a multiplayer sandbox experience, and the "gameplay" will be invented by the players and community.

</comment></comment></comment><comment><USER16> The only part that I thought was any fun was the initial ocean stage, swimming around collecting food and upgrading your animal. Reminded me a lot of some of the old 2D space shooters I played as a kid. It would have made a really nice browser game.

<comment><USER17> > It would have made a really nice browser game.

flOw is a free Flash game released in April 2006. Where "the player navigates a series of two-dimensional (2D) planes with an aquatic microorganism that evolves by consuming other microorganisms". The author/developer of the game was also involved in the development of Spore [1].

There are videos on YouTube showing the gameplay (which is circa an hour) of the PlayStation 3 version (released 2007) and PlayStation 4 version (released 2013).

Thesis about Flow in Games: [http://jenovachen.com/flowingames/thesis.htm](http://jenovachen.com/flowingames/thesis.htm)

If you read the Wikipedia article about the Flow video game you can also find a link the the archived version of the games' source code.

[1] "I wasn't hired for Spore just because of Flow. There's a lot of other game design philosophy involved in Spore, and they liked my philosophy, so I was hired to help with design.", [https://v1.escapistmagazine.com/news/view/64984-Flow- Graduat...](https://v1.escapistmagazine.com/news/view/64984-Flow-Graduates-to- PlayStation-3)

</comment></comment><comment><USER18> I've played Spore when it came out, when I was a lot younger too. Back then I enjoyed it a lot. Nowadays, I realize how much of a clusterfuck that game was, with lots of things that were just plain broken or bugged.

</comment><comment><USER19> I think this article has important context for the discussion here.

The Man Who Made Spore Suckier [https://v1.escapistmagazine.com/news/view/87123-The-Man- Who-...](https://v1.escapistmagazine.com/news/view/87123-The-Man-Who-Made- Spore-Suckier)</comment>


Google Code Jam 2012 Schedule - <AUTHOR> http://code.google.com/codejam/schedule.html

<comment><USER2> From the eligibility guidelines:

"You are not a resident of Quebec, Italy, Saudi Arabia, or anywhere that the contest is prohibited by law."

Does anyone know why you can't register if you live there?

<comment><USER3> here's why: [http://business.financialpost.com/2011/09/08/why-many- contes...](http://business.financialpost.com/2011/09/08/why-many-contests- exclude-quebec-residents/)

<comment><USER4> Interesting. Years ago I noticed a clause on the MLB all-star ballot that Canadian residents would have to pass a time-limited math-based test in order to qualify for some drawing related to the ballot. I wonder if this was related.

In fact, a quick google search reveals an almost identical clause in the 2012 NHL ballot:

 _If the potential winner is a Canadian resident he/she will be required to correctly answer a mathematical skill-testing question without assistance of any kind, whether mechanical, electronic or otherwise, in order to be eligible to receive the prize._

<http://vote.nhl.com/public/fulfillment/rules.pdf></comment></comment></comment>


How many Mechanical Turk users lie about their location - <AUTHOR> http://behind-the-enemy-lines.blogspot.com/2011/03/do-mechanical-turk-workers-lie-about.html

<comment><USER2> Ok,so ~10% aren't indicating their true location. Why does it matter? Maybe they just said whatever was needed to get the HIT?

If the HIT is not actually locale specific, there is no reason to expect all takers to truthfully divulge where they are located.</comment>


Resurrecting the extinct frog with a stomach for a womb - <AUTHOR> http://phenomena.nationalgeographic.com/2013/03/15/resurrecting-the-extinct-frog-with-a-stomach-for-a-womb/

<comment><USER2> This is seriously cool and great to see coming out of Australia.

I know there have been attempts to clone several other species in recent years, what are the most common issues with attempting to do this? I suspect not having a sufficient DNA sample, so how was one obtained for this case? Are the DNS repositories out there for extinct animals or those nearing extinction just so that one day we can bring them back? If not, i think this wouldn't be a terrible idea...

</comment><comment><USER3> He injected the DNA of an extinct frog into the egg of a living frog.

I wonder if Jurassic Park is the next step.</comment>


Hackers Attack on City Water Station Destroys Pump - <AUTHOR> http://www.wired.com/threatlevel/2011/11/hackers-destroy-water-pump/ <comment><USER2> My favorite part about this hack is the fact that the initial attack vector was PHPMyAdmin. Really?

I know SCADA systems used by the electric utilities are required to adhere to security guidelines defined by NERC-CIP (Critical Infrastructure Protection), and I'd be shocked if there weren't massive violations of whatever the water utility equivalent is.</comment>


Cicada: Dependably Fast Multi-Core In-Memory Transactions (2017) [pdf] - <AUTHOR> https://hyeontaek.com/papers/cicada-sigmod2017.pdf <comment><USER2> Source Code: [https://github.com/efficient/cicada- engine](https://github.com/efficient/cicada-engine) Evaluation Code: [https://github.com/efficient/cicada-exp- sigmod2017](https://github.com/efficient/cicada-exp-sigmod2017)

</comment><comment><USER3> How does this compare to Adaptive Radix Tree (ART) with optimistic lock coupling?

Or is it more about transactions isolation than thread safe access ?</comment>


Show HN: Calculate monthly salary in Slovenia in 3 steps - <AUTHOR> http://izracunplace.si <comment><AUTHOR> Hi,creator of this mini site here. While this mini site is in Slovene language, it could benefit to foreign freelancers or 9 to 5 job seekers who are currently in Slovenia. First input field is brutto payment, second is cost for lunch and commute (in Slovenia companies must pay you for that) and the third field is number of kids you have. Calculation result is what you get on you bank account on monthly basis. If you have any questions or suggestions, please don't hesitate to ask me.</comment>


Subversion Vision and Roadmap (spoiler: no DVCS) - <AUTHOR> http://svn.haxx.se/dev/archive-2010-04/0047.shtml

<comment><USER2> _Subversion has no future as a DVCS tool. Let's just get that out there._

It's fortunate that the developers are smart enough not to go down that road.

The big problem with svn is that the simplicity of the underlying model does not translate into robust features on the front-end. The idea of directories as subdirectories and branches and tags is the result of muddled thinking that will make it impossible for svn to ever have reliable merge tracking. Partial checkouts are a useful feature, but they are not worth botching the architecture so as to make it unusable for basic version control needs like history tracking, diffing and merging.

The idea of an atomic repository with an identifying hash is so powerful, as exemplified by git and other DVCSes, _even for mundane development tasks and workflows_ , that svn is basically a dead-end. They can't fix the situation without breaking backward compatibility, and they can't break backward compatibility without a huge uproar from existing repo administrators.

<comment><USER3> Could you please elaborate on how the use of hashcodes rather than counting numbers for identification on an atomic repository botches history tracking, diffing, or merging?

<comment><USER2> Well the hashcode effectively signs the full repository state, so that you know that if a repo has the same hash code it has the same contents, however that's not the main problem with svn.

The main problem is that svn has no notion of a project to attach it's merges or comparisons against. A directory in a repo may be a subdir of the project or it may be a branch. This leads to an explosion of edge cases and legal commands that make no logical sense. As long as everyone follows certain practices (such as not committing changes to two branches at the same time) then a heuristic approach sort of works, but that's hardly the way to design a robust (and ostensibly simple!) system.

If you can merge subdirectories in a project, and if you can merge branches which are just an ancestor directory of that, and if a given merge only affects certain subdirectories in a repository, how can anyone expect subversion merge tracking to be viable? Even if they somehow munge it to work in 99.9% of real world cases, think of the complexity and mental overhead of maintaining this solution compared to what a DVCS with clear-thinking primitives can achieve. It's time for the successor to svn in centralized version control systems to be built from scratch. svn itself is hopelessly hamstrung.

<comment><USER3> When there's only one repo, there's no ambiguity about the content represented by a revision number. Revnums are perfectly adequate for the many situations in which frequent coordination with a central repository is completely feasible.

Git also does not require every file and subdirectory to be modified by every merge, so based on that criterion it's no more astonishing that SVN merge tracking can work than it is that Git merge tracking can work.

SVN's merge tracking was an after-thought, and it was implemented using the general, user-visible, metadata facility (properties). So historically svn was prone to problems such as repeated merging, and today it's possible to make things complicated, and then manually edit and botch the complex merge history, and suffer.

Fundamentally, svn:mergeinfo summarizes history rather than pointing only to merge-parents. I suppose the latter is what you mean when you speak of clear- thinking primitives. I wasn't involved in the mergeinfo design, but it seems less likely the result of muddled thinking than of a design decision recognizing that the entire revision graph isn't locally available to svn users and that the summary is sufficient under reasonable restrictions on usage.

Your condemnation of svn as a "hopelessly hamstrung" "dead-end" seems derived more from your dismissal of centralized version control generally than from the particular details of svn's design or implementation. In that dismissal I think you're taking too narrow a view of the ways in which people work together.

<comment><USER2> Wow, what I said just sailed right over your head.

 _Git also does not require every file and subdirectory to be modified by every merge..._

Uh, nooooooo... git always merges the whole repository. This makes merge tracking easy to implement in a complete way, and easy to reason about as a user.

 _Fundamentally, svn:mergeinfo summarizes history rather than pointing only to merge-parents. I suppose the latter is what you mean when you speak of clear- thinking primitives._

No, I'm referring to the fact that in git has a strict definition of both branches and the project tree, the notion of a merge isn't a primitive, it just sort of falls out of the primitive definitions naturally. Each revision in a git repo contains one and exactly one copy of the whole working tree. In Subversion the repository just has directories, some of which are project tree directories, and some of which are branches, and some of which are tags. _That_ lack of clarity leads to all sorts of problems with basic functionality that VCSes should have.

 _it seems less likely the result of muddled thinking than of a design decision recognizing that the entire revision graph isn't locally available to svn users and that the summary is sufficient under reasonable restrictions on usage._

What it seems like is that the designers had only ever used CVS, so what they were working on seemed so advanced to them at the time, that they thought that "cheap copies" and partial checkouts were just icing on the cake and they had no idea of what they were trading away.

However in hindsight it's _clearly_ not worth munging everything together. You are far better forcing users to actually define branches and subprojects. The amount of work that svn saves you is insignificant next to the degradation of the information stored in the repository data structure.

 _Your condemnation of svn as a "hopelessly hamstrung" "dead-end" seems derived more from your dismissal of centralized version control generally than from the particular details of svn's design or implementation._

It might seem that way because you don't appear to have understood a word I said. Nothing I've said had anything to do with distributed development. Everything I've said is specifically about svn's primitive concepts, and dead- end is a perfect way to describe it.

This ignorant defense of subversion needs to be stopped. I understand there are reasons people need to use subversion. I understand there are use cases that DVCSes don't fit. I certainly don't think git is the end-all-be-all of VCS. But to defend subversion without adequate understandings of its failings just makes you look bad. It's no different than a Java developer talking about how they don't see what's so great about Lisp macros. If you don't grok the concepts then any arguments you make are just noise.

</comment></comment></comment></comment></comment><comment><USER4> more detailed spoiler:

 _Subversion exists to be universally recognized and adopted as an open- source, centralized version control system characterized by its reliability as a safe haven for valuable data; the simplicity of its model and usage; and its ability to support the needs of a wide variety of users and projects, from individuals to large-scale enterprise operations._

 _A shorter, business-card-sized motto might be: "Enterprise-class centralized version control for the masses"._

<comment><USER5> All I can say is thank goodness I can use Git with Subversion without anyone being the wiser. Local feature branches are one of the godsends of DVCS.

<comment><USER6> Yeah. But it is hard to send local branch to review to other team members when Subversion is used on the project.

We use Savana to add proper branches to Subversion. Tool lacks polishing, but it is helpful. It is simple to use: you can create private branch, sync your private branch with parent branch, and promote your private branch to parent branch ( <http://savana.codehaus.org/gettingstarted.html> ). It is not a git replacement, of course.

</comment></comment></comment><comment><USER7> in my opinion, subversion fails to properly support many features it claims to support. when people find that some things do not work as advertised _by design_ , people naturally get pissed.

for example, subversion claims to do path-based authorization. we found that it only works when not using repository-wide branches/tags. when one wants to assign different rights to different users on different parts of a project, and also wants to tag the whole project, the policy file needs to be adjusted accordingly for the new branch.

so the best way to do it is to hold these parts on separate repositories, which breaks subversion's main selling point over cvs. so one defines svn:externals, which in turn needs to be updated for every branch/tag with revision numbers.

and this is a just minor annoyance in the whole stinking pile of mess that is branching/merging in subversion.

</comment><comment><USER8> "Enterprise-class centralized version control for the masses" is the right way to go for svn IMO. With fast reliable networks, if svn implements the said roadmap features then existing organization probably won't need to move to DVCSes.

Distributed model works really well with open source. So I suppose svn has chosen its niche. I know I would have been happier using svn over ClearCase so many times.

</comment><comment><USER9> So when is it going to start preserving filesystem modification times? That's only been on the TODO list for what, close to ten years?

I'd really like to see that (which has existed in branches) before some sort of massive effort to keep up with the DVCS Joneses.

</comment><comment><USER10> They don't need to go the dvcs route, just get merging working.

<comment><USER11> I agree. They can keep the centralization, distributedness isn't all that useful for in-house development and leads to problems when people forget to push after committing. The real power of git is super cheap, super easy branching and merging. If that existed in svn, we would never have switched.</comment></comment>


Show HN: Make Your Photos Untraceable with Sensor Noise Cancellation - <AUTHOR> https://www.esenbil.com/photoclean <comment><AUTHOR> Every camera sensor has a unique fixed pattern noise, called Photo Response Non Uniformity (PRNU). Our new application Photoclean estimates PRNU and removes it from photos. This unique noise not only degrades the photo, but can also be used as a fingerprint to associate a photo with a camera. Once you remove it, you obtain a cleaner photo and eliminate the possibility of camera and photo association. Any feedback will be appreciated.

<comment><USER2> Interesting, seems like something every digital camera should have the ability to do by default.

Although, the privacy issues could be a larger concern in some instances. Such as a source to news paper ect... if not applied it could be used to identity photos taken by the same individual, which could then de-anonymize them.</comment></comment>


The Pitch: Behind the scenes of venture-capital financing (2001)  - <AUTHOR> http://www.nj.com/business/ledger/index.ssf?/news/stories/0909thepitch.html <comment><USER2> Follow-up: the company they're talking about (Global Locate) was just acquired by Broadcom: <http://www.globallocate.com/PRESS/Press_Releases/Broadcom_To_Acquire_GL.htm></comment>


Ask HN: How would you keep 2 APIs in sync? - <AUTHOR> Since Mulesoft's acquisition I've been thinking how I would build something similar. A couple of projects I'm involved in have mentioned "We need bi-directional sync between Salesforce and X, Y and Z", which has always been followed by "but MuleSoft and PieSync cost serious $$$$$".
Have have/would you build it? The resources I've found are about syncing files between two endpoints, or databases between mobile devices and the server (which is similar, but often encapsulated in the storage).
I already have a page of edge-cases where data would be incorrectly overwritten. E.g.:

  09:00:00 Data polled from systems A and B (for these are 'enterprise' systems without webhooks or notifications or changefeeds)
  09:00:01 A record (which is in the polled batch) is changed in system A
  09:00:07 The changes are written back to each system, OVERWRITING THE CHANGE IN SYSTEM A MADE AT 09:00:01
 Without the ability to lock system A while the sync is done, I don't see how this can be prevented? References appreciated; I'm on holiday soon and up for a lot of reading. StackExchange has been very silent; 'ESB' and 'bi-directional sync' aren't the right keywords?
Finally, any open source implementations I should look at? <comment><USER2> > Without the ability to lock system A It can be done using watermarking. Let > us take the database example. Assume we have a timestamp column in database > A. The column can be used for syncing records to other storage system.

Typical implementations use a query (select records where timestamp >= 'initial_val' and timestamp <= 'max_val/desired_val') and then get the data. Next run typically starts from max_val of previous run.

For open source you can look at Sqoop[1] or Gobblin[2] which do this

[1] [https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html](https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html) [2] [https://gobblin.readthedocs.io/](https://gobblin.readthedocs.io/)

<comment><AUTHOR> Thanks for the info re watermarking and the links.

Re watermarking, would it prevent the overwriting example? If you could do conditional updates via the API then yes, because you could say "WHERE the record timestamp = the timestamp we have". But if not, the change in System A will be overwritten because no one knows it has changed.</comment></comment>


Exercise to improve hunchback posture forward head carriage correction - <AUTHOR> https://www.youtube.com/watch?v=LT_dFRnmdGs <comment><USER2> I can personally attest to how well the wall angels(and its variations) work. The thing is, you've probably spent years in that wrong posture so you need to be diligent and disciplined about doing your stretches and the change won't happen overnight. Any changes to your computing setup, which you spend hours on, would also be of benefit. Having a desk job will destroy your body so anything you can do to delay this is a good thing to investigate.

Upvoting because this is useful for any programmer out there. If you're not in pain now, you will be...

<comment><USER3> Is office work really a curse on your body? Are there any alternatives? Is standing all day more beneficial? I find it difficult to get conclusive answers on this by myself.

<comment><USER4> > Is office work really a curse on your body?

Yes.

> Are there any alternatives?

Yes, jobs that provide more physical activity as part of your work.

> Is standing all day more beneficial?

No.

<comment><USER3> What if I take a 5-minute break to walk outside every 45 minutes? It's very difficult to find reliable sources on this. It seems everybody is just parroting whatever they currently believe. How can I find an authority on this subject?

<comment><USER4> [https://www.cnn.com/2017/09/11/health/sitting-increases- risk...](https://www.cnn.com/2017/09/11/health/sitting-increases-risk-of- death-study/index.html)

[http://annals.org/aim/article/doi/10.7326/M17-0212](http://annals.org/aim/article/doi/10.7326/M17-0212)

> Take a movement break every 30 minutes, say experts. No matter how much you > exercise, sitting for excessively long periods of time is a risk factor for > early death, a new study published Monday in Annals of Internal Medicine > found.

> There's a direct relationship between time spent sitting and your risk of > early mortality of any cause, researchers said, based on a study of nearly > 8,000 adults. As your total sitting time increases, so does your risk of an > early death. The positive news: People who sat for less than 30 minutes at a > time had the lowest risk of early death.

In short, sitting time accrues in a health debt you can't pay back with breaks.</comment></comment></comment></comment></comment>


True Match: Going undercover in Match.com - <AUTHOR> http://www.technologyreview.com/web/24177/?a=f

<comment><USER2> Match.com doesn't have a problem with its matching algorithm as it does with the fact that - like the author - there are more fake profiles on it than real ones, and not all of those were made by third parties.

It's a site best avoided.</comment>


Results: iOS Game Revenue Survey - <AUTHOR> http://www.streamingcolour.com/blog/2011/09/28/results-ios-game-revenue-survey/

<comment><USER2> Interesting how median lifetime revenue increases with the number of developers on a game up until 20+, where it dips.

From this it would seem 10 to 20 person teams are the sweet spot for mobile game development.

Now I just need to find 9 to 19 more developers for my project :)

<comment><USER3> The median revenue _per developer_ per month is an indicator of profitability and probably a better metric to decide the right team size.

So if you look at figure 7b, the sweet spot seems to be a team size of 6 to 9 people (where the median revenue per developer per month is $550 to $820. In contrast, the median revenue per developer per month for 10 to 19 person teams is $320 - $620).

If you can hire 19 people, you should probably split them to run two projects of 6-9 people each. :-]

<comment><USER4> If you consider what a capable developer can bring in just in salary in a month you realize what a ridiculous chicken-scratch game this is. Making indie apps may be fun but it's a dumb way to try to earn money.

</comment></comment></comment><comment><USER5> Take-home lesson:

To be successful,

        (1) Practice.
        (2) Network.

<comment><USER4> And, if you look at the games that are _really_ doing well in the store, write gimmicky, addictive games and make a fortune selling virtual coins/tokens/badges/clothes to people psychologically locked in to your scheme.

Freemium is king in the app store now.

<comment><USER6> I think is better to try to go for quality rather than gimmicks. In the long run it will succeed more often than not. Anecdotal Example: Apple

Going for gimmicks is like going for get rich quick schemes.

<comment><USER7> Apple started out as a get rich quick scheme, not a "lets change the world" scheme.

<comment><USER6> I don't know if that is true but it is irrelevant since they became successful NOT by using gimmicks or schemes but focusing on quality, which is the point I was trying to make.

</comment></comment></comment></comment></comment><comment><USER8> To me, this just shows that the iPhone app is totally hit based

The medium lifetime revenue is right beside the 90th percentile, which means those top 10% earn the vast majority of the revenue

<comment><USER4> This has been my experience too. I have one fairly successful app, with good reviews and a long stint on the featured page in its category. It's still not even close to paying for the time it took to develop it. I would have been much better off financially if I'd spent that time consulting.

Writing apps is appealing because it's so much easier than bootstrapping a software business from scratch outside the store but I'm not convinced it's a good bet for the average developer.

<comment><USER9> Am I reading that right? 100k lifetime value seems to be near the top end of what you can expect from one app. A reasonable salary at a normal job for a senior-level developer is near or a little better than that. If you want to succeed better than you could by working for someone it seems like you either need to be completely serious about releasing new games every 3-6 months(and release hits constantly), or, maybe, not write games (do business apps fare better?)

Has anyone tried the minecraft approach on the app store?

<comment><USER10> This assumes you only care about money and don't care about creative control and the satisfaction of running your own business. For a lot of indie developers making equal to or slightly less than they would as a salaried developer is doing well, even if you don't have a massive hit that makes $1M+.

<comment><USER9> I admit that's the way I phrased it, but I meant to say "Why iOS when you have other avenues if you want to work on your own thing?"

In other words, iOS seems to be a very bad choice for someone who wants to make money, when you could do a, say, SAAS web app, or a traditional app. Sure, you probably won't START making the same money as a traditional job with those, but the cap is much higher.

</comment></comment></comment></comment></comment><comment><USER11> I'd be interested to see the distribution of revenue from sales vs ads.</comment>


Celebrity suicide on Twitter: Activity, content and network analysis - <AUTHOR> https://www.sciencedirect.com/science/article/pii/S0165032718316410?dgcid=raven_sd_search_email# <comment><AUTHOR> The full title is "Celebrity suicide on Twitter: Activity, content and network analysis related to the death of Swedish DJ Tim Bergling alias Avicii" which is way too long for HN.</comment>


Solving a maze with Perl and DFS - <AUTHOR> http://desple.com/post/118014845597/solving-a-maze-with-perl

<comment><USER2> I used to love perl but switched to python almost 10 years ago. It is beautful how perl code can be so terse - at the same time I was constantly having trouble to understand my own code 6 months after I wrote it.

IMHO Python has a better compromise between expressiveness and maintainability but of course it's a matter of personal taste.

<comment><AUTHOR> I do agree with your point, however when writing quick code on windows and unix I think perl really does the job. It is just the matter of using the languages available and that you are most fluent in.</comment></comment>


Square open-sources Tesseract: fast filtering for coordinated views - <AUTHOR> http://square.github.com/tesseract/

<comment><USER2> Please don't call it tesseract, we already have a prominent open source project with that name: <http://code.google.com/p/tesseract-ocr/>

<comment><AUTHOR> Apologies; I wasn't aware of the Tesseract OCR project until very recently, and then I hoped there would not be much harm given that the two projects are so unrelated. (The name "Tesseract" was the natural progression from "Square" and "Cube".) What's the saying? There are only two hard things in Computer Science: cache invalidation and naming things.

<comment><USER3> Hypercube?

<comment><USER4> n-dimensional hypercube

<comment><USER5> hypercubes are already n-dimensional, that's not adding any extra information.

</comment></comment></comment></comment></comment><comment><USER6> @mbostock: Why is there Dart code in the project? Just because that's where you borrowed one of the sort functions from?

Thanks for another useful library!

<comment><AUTHOR> Yep. Typed arrays don't have a built-in sort method, and even the built-in array.sort is extremely slow. I ported Dart's dual-pivot quicksort implementation, which reduced the time to sort 1M floats from ~2.5s to ~350ms (timed in Node v0.6.2).

[https://github.com/square/tesseract/blob/master/src/quicksor...](https://github.com/square/tesseract/blob/master/src/quicksort.js)

<comment><USER6> Good to know. I'll have to consider switching away from Underscore's sort, which is probably equally slow.

By the way, quicksort.js is code you wrote, right? I assume that's too nice and concise to be from the Dart generator...

<comment><USER7> Underscore doesn't have a "sort" apart from the native Array.prototype.sort, but it does have a "sortBy", which is something else entirely.

<comment><USER6> The sortBy method is just producing an integer array that is then sorted using each browser's implementation of Array.prototype.sort. Is that a proper reading of the code?

<comment><USER7> Bingo. You got it.

</comment></comment></comment></comment></comment></comment><comment><USER2> Update: Square have changed the name to Crossfilter:

"Renamed to Crossfilter, partly in homage to Chris Weaver's work on multidimensional visualization. It may not have the intrigue of "tesseract", but it does describe the library's function succinctly."

\-- [https://github.com/square/crossfilter/issues/1#issuecomment-...](https://github.com/square/crossfilter/issues/1#issuecomment-4706390)

</comment><comment><USER8> That's timely. Recently I've been looking around for various widgets interfaces to explore multidimensional data. This looks quite useful - way up next to good old pivot table.

What do you think of parallel coordinates as a widget? (F.ex. <http://exposedata.com/parallel/veggie/>)

</comment><comment><USER9> This is really great work, and I can't wait to see some of it filter into Square's Cube project.

</comment><comment><USER10> I got a preview of this project from Square's CTO 3 weeks ago at Stanford. The fast-filtering is MIND-BLOWING.

</comment><comment><USER11> It'd be cool if it could say, show the data for all weekends at once, skipping over the weekdays.

</comment><comment><USER12> So mbostock works for Square?

</comment><comment><USER13> Thank you for an release. I've been heavily promoting the web stack for over a decade and a half, yet still I'm surprised by what it is capable of. This also provides a clear demonstration of the power of algorithms even on an imperfect implementation. Excellent, clean API as well.</comment>


Ask HN: What's your definition for 'being hacker'? - <AUTHOR>

<comment><USER2> I think the Jargon File (<http://www.catb.org/jargon/html/index.html>) puts it nicely.. there are several definitions, most having to do with technical adeptness and a delight in solving problems and overcoming limits.

</comment><comment><USER3> A hacker is someone who isn't afraid to win the ball even though it requires falling in the mud or stepping on a more technically skilled opponent's foot.

</comment><comment><USER4> Someone who solves problems or produces work in unconventional ways, not working via the usual channels.

</comment><comment><USER5> to succeed...

<comment><AUTHOR> Do you think that anyone succeeding can be called 'hacker'?

<comment><USER5> Nope, you're right to point that out. But a certain amount of success is an important requirement.

You can still fail sometimes and try again and have success or find out that something is just not possible.

So trying to put it in a single word was maybe a weak attempt. There are more factors, like do something new (and hopefully have success with it), learn a lot, hard work...

<comment><AUTHOR> Seems like 'succeding' and 'innovation' are key components, according with the other comment.

<comment><USER5> Funny, after responding to your post I was watching that IBM video on youtube [http://www.youtube.com/watch?v=XrhDaAmn5Uw&feature=chann...](http://www.youtube.com/watch?v=XrhDaAmn5Uw&feature=channel) somebody had posted on HN in the meantime. And "to invent" was what I actually wanted to add here, after I heard Bijan Davari in the video, saying "Our job is to invent" (around 26:30)

</comment></comment></comment></comment></comment><comment><USER6> Someone who improves the world through innovation.

<comment><AUTHOR> Interesting, this is similar to the Schumpeter's vision of an entrepreneur.</comment></comment>


Keurig mea culpa: 'We were wrong' - <AUTHOR> http://americasmarkets.usatoday.com/2015/05/08/keurig-mea-culpa-we-were-wrong

<comment><AUTHOR> I admit to more than a small amount of schadenfreude over the graph of their stock price about halfway down the page.

It warms my heart to see a company that pulls aggressive anti-consumer DRM shenanigans get taken over the knee and spanked by the market.</comment>


Xamarin acquires RoboVM - <AUTHOR> https://robovm.com/robovm-acquired-by-world-leader-in-cross-platform-mobile-xamarin/ <comment><USER2> And they've already raised the prices.

Edit: Actually, they've just changed their "startup" pricing and it isn't publicly available anymore. It might actually be better, if they allow you to have more than 3 devs.

<comment><USER3> More importantly, they've removed IntelliJ and Eclipse integration from the Solo/Indie plan to match Xamarin keeping Visual Studio integration out of their Indie plan.

<comment><USER2> Exactly. However, it looks like their IntelliJ based IDE is still included in Solo, which I missed initially because they put it at the top of the chart. That's a considerable improvement over MonoDevelop

</comment></comment></comment><comment><USER4> RoboVM has been free for libgdx users. Xanarin is disliked by independent developers for its pricing structure. Will RoboVM remain free to use? I hope so

<comment><USER5> libGDX and PlayN folks get a free license with debugger support for commercial and non-commercial games.

<comment><USER4> Thanks badlogic. Can you provide any details about why libGDX gets special treatment? I'm curious

<comment><USER4> badlogic, I can't reply to a comment thread going as deep as your last comment. Did Xamarin specifically put in the acquisition that it will remain free for libGDX developers? Because my faith in Xamarin is not high that they won't remove free integration with RoboVM, as Xamarin loves charging developers high prices, and thus me developing games using libGDX and RoboVM becomes a risky decision.

<comment><USER5> It will stay free for libGDX folks.

</comment></comment></comment></comment></comment><comment><USER6> Interesting business move. Since it appears the plan is to get RoboVM customers to use Xamarin Insights and Xamarin Test Cloud does that mean that Xamarin's original product of C#/XAML to native iOS and Android dev environment is going to come to be seen as the razor to the services metaphorical blades if you will?

</comment><comment><USER7> Kudos to the Xamarin management for this strategic move.

(CIL BYTECODE / JVM BYTECODE) => LLVM => NATIVE

</comment><comment><USER8> Isn't RoboVM in the same place license wise as Android? Start with loosely defined subset of Java. Transform to something that isn't Java byte code and doesn't adhere to the Java VM specification. Doesn't pass the TCK and never will.

Up until now RoboVM has been featured frequently by Oracle in an attempt to mask their weakness in the mobile space (since Oracle doesn't acknowledge Android). I wonder how quickly this will stop.

</comment><comment><USER9> I never really looked at RoboVM, would anyone whose used it care to weigh in on the positives and negatives of RoboVM?

<comment><USER10> It has limitations on runtime introspection and modification, but once you learn them, you can use idiomatic java and/or scala. That said, the limitations make it difficult to easily port.

<comment><USER5> The only limitation is the lack of runtime byte code genration. Reflection and proxies work as intended.

</comment></comment></comment><comment><USER10> How easy would it be for a competitor to use their code? It seems to me to be difficult given Xamarin's positive reputation, but are there any legal or technical barriers for a competitor to offer support, patches, etc., and contribute upstream?</comment>


Mixpanel Streams: Watch What Your Users Are Doing On Your Site, In Real Time - <AUTHOR> http://techcrunch.com/2011/06/14/mixpanel-streams-watch-what-your-users-are-doing-on-your-site-in-real-time/

<comment><USER2> It's funny to see this b/c I've been a user of Mixpanel almost from the beginning and built my own stream. I honestly don't know if they allow this now, but back then you couldn't export your data. So as a believer in making data-driven decisions, it simply made sense to save off every single Mixpanel event before sending it over to them.

Customers would call on the phone saying I'm looking at X, and it was kind of hard to figure out exactly where they were on the site by their verbal description. Where they'd come from (if they were referred from Google AdWords, they might have different expectations than from somewhere else). We'd have to awkwardly ask callers for the product ID number, or stuff like that.

To help with customer service, I built a real-time AJAX stream of all our Mixpanel events. (Which was easy since we were saving them anyway.) I found this to be essential for customer service and learning about our users in general. Sure it may seem creepy when you first learn about this, but honestly, what developer doesn't tail their server logs just for fun? With the right intention, it can be a seamless way to be extremely helpful to users. With a real-time stream, you're just suddenly more helpful to people who need it.

By anecdotal evidence, we have found that letting people know you were following them is not a good idea, b/c you may cause them to feel creeped out. But the truth is, it's more likely than not that this is already happening (to you, right now, on this very site). We would never deny it, but we tend to just not bring it up / volunteer the fact. A real-time stream just gives visibility into the data that's always been there.

What was great about having our own stream was that we can pull in any user account data that's helpful. In other words, once a user logs in, you can display any relevant user account info in the stream. How long has a user been shopping? How many products has he/she looked at? Is he/she a return user? Has he purchased before? You can essentially link your real-time stream with your customer relationship management, aka CRM. If you choose, you can reach out to them in real-time on the site, either triggering an event on their next page load, or popping up something like one of those embedded chats.

<comment><USER3> Yes, you can export your data: <http://mixpanel.com/api/docs/guides/api/v2>

</comment></comment><comment><USER4> Streams seems really similar to this project I saw a couple of weeks ago. <http://news.ycombinator.com/item?id=2560151>

</comment><comment><USER5> Real-time clickstreams are one of Clicky's most popular features. This is definitely an attempt to win over customers that might otherwise choose the competition.

As far as the creepiness goes... unless you think request logging should be totally disabled, anyone can simply tail their web server's logfile to see real time click data. How is it more of a privacy violation, just because it's easier to read? In fact, the anonymization Mixpanel is doing does _more_ to protect privacy than a simple tail would do.

<comment><USER6> I _could_ trudge around in my databases to read direct user to user messages. Should I? Should I sell a tool to facilitate doing so?

edit: re: downvotes, Yes I was using argumentum ad absurdum, but I was trying to show that just because you have data doesn't mean analysis of it isn't intrusive.

</comment></comment><comment><USER7> I'm a huge fan of real time analytics so gave Mixpanel a shot just now and after playing around with it for a few minutes removed it from the site right away. I really didn't like the color scheme to start with but it just didn't feel very useful. It just showed a bunch of users with colors as their names rather then giving totals or anything like that.

I use <http://woopra.com> quite often but give them too much money (keep running over in pageviews and its getting expensive) and was looking for an alternative but this just doesn't feel right for me. Currently trying the other one they mentioned in the article chartbeat and though I'm not sure I like it better then woorpa yet it does seem more useful.

Update: I'm going to still keep using chartbeat until my free trial runs out I think but it seems like Woorpa = chartbeat + mixpanel + more just based on a quick 5 minute use of both chartbeat & mixpanel.

Update2: <http://reinvigorate.net> looks cool too, also mentioned in article. Will have to try it out as well.

</comment><comment><USER8> That's not creepy at all.

Edit: Everyone should stop for a moment and think about how they'd feel if they knew someone was watching their every move on a website, even if that person worked for the website. Even anonymized, I don't like it. I would probably avoid any website that I knew used that tech.

<comment> brown9-2 Stop to consider how much data about how you use their site is already available to the operators of the sites you visit, I think this is only a small extension of that.

Heck with mobile apps, some sites/apps can even see where you are using them from.

</comment></comment><comment><USER9> Hey, what are those other two icons in the screenshot? (in the left side nav). They aren't in my account!

<comment><AUTHOR> You'll just have to wait and see ;)

</comment></comment><comment><USER10> great way to see typical user behavior!</comment>


Ask HN: Deciding Between Yahoo and Amazon - <AUTHOR> I am a current graduate student graduation in March 2020. I have software engineer university grad offers from Amazon, Seattle and Yahoo, Sunnyvale. I interned at Yahoo in Summer 19, and have the verbal return offer from them. I am facing a hard time deciding which company to join. Compensation-wise, there won't be much difference. But technically speaking, which is a better place to work at in terms of my career growth as this will be my first full time job? I enjoyed the internship at Yahoo! and my team. With that being said, I heard Amazon is good for initial career shaping. (Yahoo is a sinking ship, so I have heard). I know the team I will be joining at Yahoo, and it's the domain I wanna progress in, as opposed to Amazon where you get to know the team after you accept the offer. Considering every possible factor (excluding compensation), which is a better place to work at? <comment><USER2> I would be cautious buying into the sinking ship metaphor, not the least because you may owe your future employer the intent to keep the ship from sinking. You're early stage career, and don't need an employer with a 20+ year horizon.

Yahoo may not be in the dotcom glory days boom phase, but its not going away, and it has some increadibly smart people. I know somebody vaguely who worked in europe on Yahoo backed stuff and it was front-line innovative work.

Amazon is huge and complex and has many rulers and conflicting rules, and also has this nasty 'Screw the worker' back-taste about how you, the digerati get the candy, but the people in the warehouse are urinating in bottles to make pay.

Do you really want that on your personal live choice resume?

</comment><comment><USER3> Amazon is the rising star vs Yahoo. Amazon is the company with new ideas and a better technical reputation. But you should be comparing jobs and career advancement rather than companies. A big chunk of the people that starts at a company don't stick around forever so I would be thinking of how the position will advance my career and life.</comment>


Ask HN: To ask or not to ask? - <AUTHOR> I sometimes find myself in this situation where I have a very nice idea or I have some doubts about something and I'd love to share with people from hacker news to know others opinions but I am under the constant feeling that I will not get any response or just negative feedback (which is better than no response).<p>Am I the only one who is hesitant about posting on HN? <comment><USER2> You're not the only one who's hesitant. I actually tend to put a lot of thought into my responses because I know from experience that sometimes my ideas don't bode well with the community. Maybe they were bad comments, maybe they weren't well thought out, who knows. I've gotten better at this, but it's taken practice.

Sometimes you won't get any response, in fact the majority of mine don't receive any feedback whatsoever. I've learned to live with this, it's not a major problem at all. The battle for me was realizing that a some of the folks here are very critical (in a good, analytical way). It takes time and mistakes to learn, and trust me, you will learn as long as you try. This goes with anything, but I think the hacker news community is a truly remarkable one and has helped me grow in many ways.

Embrace the comments that give negative feedback, understand where the posters of the replies come from and try to step back and realize where you may have been in the wrong.

</comment><comment><USER3> > I am under the constant feeling that I will not get any response

You might not, but consider that if you don't ask _you won 't get any response either_.

</comment><comment><USER4> What's so awful about posting something and getting no response?

In fact, what's so awful about posting something (especially on a forum that offers something pretty close to anonymity) and getting a negative response?</comment>