Why Your Business Plan Just Killed Innovation - <AUTHOR> http://paulbromford.wordpress.com/2013/08/01/why-your-business-plan-just-killed-innovation/

<comment><USER2> "But what do you think? Do you agree that business plans and risk assessments can stifle innovation?"

No, I don't agree. I agree that badly thought-out business plans and risk assessments stifle innovation. I've worked in environments where a lack of planning killed innovation because the leadership team failed to keep track of changing technologies that required a departure from business as usual.

There's nothing inherent in planning or risk assessments that kill innovation. Anyone who has extensive planning experience (and is actually good at) will tell you that the purpose of a business plan is not to perfectly predict the future, but rather to create a compelling vision of where an organization is heading and how and why it's possible to get there. Without planning, confusion and misunderstanding can set in, leading to organizational chaos.

Formal planning is often not needed in smaller teams (e.g. most startups) simply because the team is small enough and close enough that nothing really needs to be written down, but a close-nit team still has a definite plan - even if it does exist in the sort of "mind meld" that exists among members. But even if it isn't written down - a plan is still there.

Indeed in the amazon case: ([http://paulbromford.wordpress.com/2013/08/01/why- your-busine...](http://paulbromford.wordpress.com/2013/08/01/why-your- business-plan-just-killed-innovation/)) - they have quite a bit upfront planning. The construction of press releases, FAQs, user manuals(!), etc. are an example of excellent long term planning. Once buy-in is achieved, the small team then goes into action figuring out how to make it happen.</comment>


Fibonacci Hashing: The Optimization That the World Forgot - <AUTHOR> https://probablydance.com/2018/06/16/fibonacci-hashing-the-optimization-that-the-world-forgot-or-a-better-alternative-to-integer-modulo/ <comment><USER2> Don't do this. Use a real hash function that guarantees a highly random distribution, make your hash tables power-of-two sized, and map from hash value to table index using (hash & (size-1)).

The fibonacci constant thing will help clean up the distribution of a bad hash function, but it does nothing for collision resistance if the underlying hash function is weak.

-Austin, author of Murmurhash and SMHasher

<comment><USER3> He's clearly, and obviously not talking about using it as a hash, or even to consider it as a secondary hash as someone mentioned.

The use case according to the article is strictly to replace integer modulo to map into buckets for cases where that operation is the limiting factor. In his case that's when 9ns per key is too much, and roughly 1ns is good.

For small hash tables sometimes the worst case of a linear scan of the entire table is perfectly acceptable, and the additional performance the other 99.999% of the time is a welcome bonus.

<comment><USER4> Fibonacci hashing takes the form of "h * k>>(64-b)", where k is determined by golden ratio and b is the bit size of the table. This involves one generic multiplication, which is the bottleneck. This multiplication is not strictly necessary. You can replace it with k=1033 for example, which can be implemented as "h+(h<<10)+(h<<3)". This will be a good enough safeguard against naive hash functions but is much faster to compute.

With "h * k>>(64-b)", the result has a cycle of 2^b. Suppose b=3 and you have input 1<<3|1, 2<<3|1 and 3<<3|1, Fibonacci hashing will put them to the same bucket – it is not that effective. A safer strategy is to use a proper integer hash function like Thomas Wang's 32-bit hash function. Although it involves more steps, it only involves plus and bit operations and probably can be computed faster than generic multiplication.

At the end of day, however, Fibonacci hashing or similar ideas only helps when you hash keys to similar integers but has no effect when you hash different keys to the same integer. You have to use a reasonable hash function anyway.

<comment><USER5> > Although it involves more steps, it only involves plus and bit operations > and probably can be computed faster than generic multiplication.

It the most recent popular CPUs the multiplication is exactly "one step" long, that's how wonderfully fast they got to be. See e.g. Agner Fog instruction tables. And where not, the number of "steps" is typically not more than 2 or 3. The multiplication is implemented very efficiently today, unless the CPU has to be with a very low transistor count (linke in some embedded systems). The more exotic CPUs can, of course, be different.

</comment></comment></comment></comment><comment><USER6> Summary: The article is looking for an efficient way to map a hash code into a smaller power-of-two-sized range, for use as a hashtable index. It dismisses the common solution, masking off the high bits, because it discards information, and proposes Fibonacci hashing: multiply by the golden ratio and shift down. It gives measurements suggesting this gives better performance in practice, and some theory as to why this might be.

But later it mentions, in passing, a simpler approach: just shift the high bits down and xor! The author only tries doing this as a preprocessing step in front of Fibonacci hashing to avoid "bad patterns". So I'm left wondering: might shift-down-and-xor be good enough on its own?

<comment><USER7> The article presents Fibonacci hashing as an operation to map a hash code into a smaller range, but it isn't that, really. The operation that does that is still just taking some bits from the hash code.

What Fibonacci hashing actually is is a way of stirring a hash code before use, to spread its entropy out more, so that the bits you end up taking a more likely to be well-distributed.

If your hash codes are already well-distributed, then this is pointless. But if they aren't, it's useful. So, it seems to me that rather than applying Fibonacci hashing to all hash codes, it would be better to use it as the original hash function for types which currently have bad hash functions. This is something a C++ standard library could easily do.

For example, LLVM's libc++ implements string hashing using MurmurHash2 on 32-bit machines, and CityHash on 64-bit machines:

[https://github.com/llvm- mirror/libcxx/blob/master/include/__...](https://github.com/llvm- mirror/libcxx/blob/master/include/__string#L854)

[https://github.com/llvm- mirror/libcxx/blob/master/include/ut...](https://github.com/llvm- mirror/libcxx/blob/master/include/utility#L954)

But hashes all sizes of integers to themselves:

[https://github.com/llvm- mirror/libcxx/blob/master/include/ex...](https://github.com/llvm- mirror/libcxx/blob/master/include/ext/__hash#L94)

Changing that to a Fibonacci hash, or a simpler shift-and-xor, could be a quick win.

Provided that libc++'s unordered_map uses power-of-two table sizes, that is. The code is labyrinthine, but i think, rather gloriously, sometimes it does, and sometimes it doesn't:

[https://github.com/llvm- mirror/libcxx/blob/master/include/__...](https://github.com/llvm- mirror/libcxx/blob/master/include/__hash_table#L2131)

[https://github.com/llvm- mirror/libcxx/blob/master/include/__...](https://github.com/llvm- mirror/libcxx/blob/master/include/__hash_table#L125)

__constrain_hash is a simple but entertaining bit of bit-dickery (reformatted slightly):

        size_t __constrain_hash(size_t __h, size_t __bc) {
            return !(__bc & (__bc - 1))
                ? __h & (__bc - 1)
                : (__h < __bc ? __h : __h % __bc);
        }

The x & (x - 1) tests whether a number is a power of two, because for any number that is not a power of two, subtracting one leaves the top bit set, so the bitwise and will contain at least one set bit. If the bucket count (number of slots) is a power of two, use a mask to extract the bottom bits of the hash. If it's not, do a modulus - but spend a branch to avoid that if the hash is already in the right range, which i'm surprised is a win.

<comment><USER5> > LLVM's libc++

Great find!

So the library solutions are still often suboptimal, and it's even more easy to hide bad decisions in the C++ sources, so whoever has the approach "just use the default library" should be aware of that once the performance is important.

Yes, even the simple multiplicative constants can significantly improve the hash if it by default doesn't do anything with the input! The libraries definitely should be fixed, and adding the multiplication step is really a simple and fast change for a great benefit.

As an inspiration, Kernighan and Ritchie in their book about C used a simple number 31, and that simple hash is still quite good compared to much more complex and more recent solutions as K&R also haven't used the (I guess misleadingly named) "open addressing" for their hash table. Their solution is amazingly minimalistic and in that context amazingly good for chain hash tables. I wouldn't be surprised if just changing

        return __c;

to

        return __c * 31;

in the functions discovered would result in great improvement. The good side of such a constant is that it can give the fast and small code even on the old architectures where the "normal" multiplication is slow (e.g. even if there's no fast multiplier the result can be obtained by one shift and one subtraction!). Also on modern architectures using this constant can't result in any performance degradation but improving the hash behavior of these formerly unprocessed inputs guarantees speedup. And there are surely use cases when using more complex functions is much better, e.g. those suggested by aappleby:

[https://news.ycombinator.com/item?id=17330787](https://news.ycombinator.com/item?id=17330787)

Back to the "open addressing", if you are rolling your own hash table and don't plan too much hash tables to be present in memory at once, it's often much faster to use "chains" (like in the K&R C book) than trying to store everything only in the table (which is misleadingly often called "open addressing" even if "closed hashing" is a better term) and jump through the table in the collision case. Maintaining lists per entry is typically much faster when the table is fuller, provided the allocation routines are fast.

[https://www.strchr.com/hash_functions](https://www.strchr.com/hash_functions)

By the way, MurmurHash2 or 3 and CityHash are definitely very good functions, the problem is when they aren't used in the library, like, it seems, in libcxx. And in the cases where the simpler code is needed, even a simple * 31 is much, much better than nothing!

And note, it seems there are even problems with these good functions, security wise: apparently the language implementations or the services accepting uncontrolled inputs also have to care about the security aspects of their hash functions:

[https://131002.net/siphash/](https://131002.net/siphash/)

"Jointly with Martin Boßlet, we demonstrated weaknesses in MurmurHash (used in Ruby, Java, etc.), CityHash (used in Google), and in Python's hash. Some of the technologies affected have switched to SipHash."

"SipHash was designed as a mitigation to hash-flooding DoS attacks. It is now used in the hash tables implementation of Python, Ruby, Perl 5, etc."

"SipHash was designed by Jean-Philippe Aumasson and Daniel J. Bernstein."

<comment><USER7> > "SipHash was designed as a mitigation to hash-flooding DoS attacks. It is > now used in the hash tables implementation of Python, Ruby, Perl 5, etc."

It's also the default hasher in Rust.

Rust's hashing is interesting. Types that want to be hashable implement the Hash trait. What the Hash trait requires is that a type knows how to feed its fields to a Hasher, as a sequence of primitives - it doesn't require that it actually computes a hash itself. It's the Hasher which computes the hash. This is nice, because it's very easy to implement Hash; indeed, so easy that it can be done automatically using a derive macro. The downside is that it's not possible for a type to implement a custom hash that takes particular advantage of its own structure, and so to get a particularly good tradeoff of distribution against performance. The only place to make that tradeoff is in the choice of Hasher, where it has to be made generically across all types.

That said, you can choose the hasher used for individual HashMaps, so if you have a HashMap where you know the keys are integers, you can use a Hasher which just does a Fibonacci hash.

</comment></comment></comment></comment><comment><USER8> It's not really fair to compare a custom implementation against the standard unordered_map implementations, which need to be fully general. See [https://news.ycombinator.com/item?id=9675608](https://news.ycombinator.com/item?id=9675608)

Still, I'm shocked that GCC, LLVM, and boost all assign buckets using modulus, which is very slow. I would love to know the reasoning. I assumed that they mask the high bit (or & with the table size).

Fast hashmaps use xor to mix information from low bits (examples below). Fibonacci hashing amounts to running a second multiplicative hash over your input, which is only worthwhile if you're paranoid about your input distribution.

Java SDK: [http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/s...](http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/util/HashMap.java#l320)

Android (C++): [https://android.googlesource.com/platform/system/core/+/mast...](https://android.googlesource.com/platform/system/core/+/master/libcutils/hashmap.cpp#83)

<comment><USER7> std::unordered_map is a bit of a red-headed stepchild in the C++ world. I'm not surprised that it hasn't had nearly the amount of tuning that hashmaps have had in other languages.

When i asked some C++ers about it, they warned me off using it, for reasons i didn't fully understand, but i got the impression that there are structural reasons why it can never be really fast, so anyone who needs a really fast hashmap uses some non-standard one anyway.

</comment></comment><comment><USER9> The Golden Ratio is proposed as a special multiplier for what is otherwise known as 'linear congruential' psuedorandom number generation. I'm not clear on whether the non-repetitive property of the ratio will benefit its performance at all.

This decimal is computed for 2^64 /1.618... 11400714819323198486 -it looks like this in binary: 1001111000110111011110011011100101111111010010100111110000010101

The runs of 7 ones, 5 zeros and 5 ones could be sub-optimal.

[1] Donald Knuth himself discovered this one for 64bits: 101100001010001111101000010110101001100100101010111111100101101

It contains a run of 7 ones, and 5, but maximum zeros in a row is 4.

I did once mine multipliers for LCGs of different bitlengths by comparing quickly measured ratios in their output to those precomputed from good psuedorandom sequences (average deviation etc) Then having found numbers which achieved the basic signature of random data, they were tested with Marsaglia's old 'diehard' battery of tests - and often passed.

Here is a multiplier discovered for a 32bit LCG 110010011101110100001011

I had a list of them I meant to examine here, but have lost it :(

Anyway, there is plenty of academic work to read on this subject:

1 - [https://en.wikipedia.org/wiki/Linear_congruential_generator#...](https://en.wikipedia.org/wiki/Linear_congruential_generator#Parameters_in_common_use)

</comment><comment><USER10> It's a strangely distinct memory that when I needed to implement a hash table in 1992, I read Knuth--in 1992 I guess there wasn't much else to do--and apparently read it correctly. The beauty of the golden ratio technique made it memorable and I've been doing "multiply by 2^n/phi" ever since! In fact just last week I launched Ruby to calculate it again...

</comment><comment><USER11> I wonder if, under some circumstances, one ought to use this instead for optimality of this or something similar:

[https://en.wikipedia.org/wiki/Plastic_number](https://en.wikipedia.org/wiki/Plastic_number) for a similar use. It shares a property with phi which no other irrational shares with it (They are known as the only two Morphic numbers, which one must avoid confabulating with a similarly named concept whose name I can't recall right now.). _And_ Knuth liked it (well, the reciprocal of it's square anyway, albeit the cubed version seems more interesting to me.), but never found any application for it. (He even made a special TeX symbol for the square of it, 'High Phi', see Wikipedia for details.)

<comment><USER12> As @twic and the OP discussed, the equal distribution of numbers within a defined range is naturally achieved through low discrepancy sequences (eg recurrence,Halton, Sobol, etc..) Furthermore, in ultra high-speed / low-level computing situations the additive recurrence methods are often preferred due to the incredibly fast and simple method of calculating the each successive term simply by adding (modulo) a constant value to the previous term.

For the one-dimensional case, it is well known, and relatively easily proven that the the additive recurrence method based on the golden ratio offers the optimal 'evenness' [low discrepancy] in distribution [1]. For higher dimensions, it is still an open research question as to how to create provably optimal methods. However, one of my recent blog posts [2] explores the idea that a generalization of the golden ratio, produces results that are possibly optimal, and better than existing contemporary low discrepancy sequences. In the one dimensional case, the critical additive constant is of course, the golden ratio. In the two dimensional case, the additive constant is based on integral powers of the plastic number. The generalization to even higher dimensions follows other Pisot numbers.

[1] [https://en.wikipedia.org/wiki/Low- discrepancy_sequence#Addit...](https://en.wikipedia.org/wiki/Low- discrepancy_sequence#Additive_recurrence)

[2] [http://www.extremelearning.com.au/unreasonable- effectiveness...](http://www.extremelearning.com.au/unreasonable- effectiveness-of-quasirandom-sequences/)

</comment></comment><comment><USER13> If you need to map a uniform b-bit integer x into the [0, n) range, using ((x * n) >> b) will work just as well as the modulo in terms of (quasi-)even distribution, and it avoids integer division.

The idea is just to imagine x as a b-bit fixed-point uniformly random number in [0, 1), and rescaling it by multiplication.

This works great for hash tables whose size is not a power of 2, provided that you start from a good hash function.

</comment><comment><USER14> Moral of the story: Always _always_ consult Knuth.

<comment><USER15> Unless Knuth is outdated. With hash tables Knuth is seriously outdated. With hash functions you can consult his test functions, but he is also outdated.

E.g. the CRC scheme is the fastest by far (one even exists in HW), but too easily attackable. Trivial really, any 10 year old can do that, due to some unfortunate CRC properties.

<comment><USER16> > any 10 year old can do that

Is this kind of hyperbole really necessary?

<comment><USER15> Yes, because almost nobody knows. But when you see it it's super trivial. Knuth would be ashamed.

</comment></comment></comment></comment><comment><USER17> The article attempts to explain this property of phi:

> Maybe you have a picture of a flower, and you want to implement "every time > the user clicks the mouse, add a petal to the flower." In that case you want > to use the golden ratio: Make the angle from one petal to the next 360/phi > and you can loop around the circle forever, adding petals, and the next > petal will always fit neatly into the biggest gap and you'll never loop back > to your starting position.

And points to a video. I think there's a much clearer explanation which shows how this is because (in some sense) phi is the "most irrational" ratio, stemming from its derivation from continued fractions: [https://www.youtube.com/watch?v=sj8Sg8qnjOg](https://www.youtube.com/watch?v=sj8Sg8qnjOg)

I freely admit I have not a clue how this does or doesn't improve hash algos, but it's a cool video! :)

</comment><comment><USER18> If you have a reasonable hash function, why is power-of-two and bitmasking bad? If you're uniformly distributed over a range N you'll be uniformly distributed over N/2

<comment><USER19> It isn't. You are correct. This is useful to support the underlying hash function but at that point you might as well just improve that and take the power of two, bitmask approach.

<comment><USER20> I think that boils down the problem nicely: if you're using a hash table or writing your own for a specialised use-case, you should pick a good hash function.

But if you're writing a general purpose hashtable implementation you have to deal with the fact that a lot of users won't use a good hash while some will, so you need to find a tradeoff between using their hash as-is and mixing it up to improve on the bad ones.

The latter need to come almost free, however, or you'll ruin performance for those who actually do their homework.

</comment></comment></comment><comment><USER21> I think the search part is implementing the Fibonacci Search Technique[0]. It's related to Golden ratio search[1].

[0] [https://en.wikipedia.org/wiki/Fibonacci_search_technique](https://en.wikipedia.org/wiki/Fibonacci_search_technique)

[1] [https://en.wikipedia.org/wiki/Golden- section_search](https://en.wikipedia.org/wiki/Golden-section_search)

</comment><comment><USER22> On a related note, growing a hash table by a factor of two might not be optimal due to interactions with the memory allocator, more specifically the new allocation will be larger than all previous allocations combined so that you can not reuse previously allocated memory. Starting with m buckets one will allocate m * 2^t buckets after t growing operation, in total m * (2^(t + 1) - 1), but that is one bucket less than the m * 2^(t + 1) buckets required for growing again [1]. Whether this might be an issue obviously depends on the memory allocator used and other factors like the usage of a compacting garbage collector, phi appears again when looking for a better growth factor, and I will just leave this link to a Stack Overflow question [2] as a starting point because I am unable to find the article I had initially in mind.

[1] Assuming you can somehow incorporate the current allocation, previously freed were only m * (2^t - 1) buckets.

[2] [https://stackoverflow.com/questions/2369467/why-are-hash- tab...](https://stackoverflow.com/questions/2369467/why-are-hash-table- expansions-usually-done-by-doubling-the-size)

</comment><comment><USER23> Light travels around a foot in a nanosecond.

Makes thinking about processing times a bit easier when you can visualise it as a distance.

<comment><USER24> [https://www.youtube.com/watch?v=JEpsKnWZrJ8](https://www.youtube.com/watch?v=JEpsKnWZrJ8)

</comment></comment><comment><USER25> This sounds quite similar to the fastrange [0] method. I've used it for random sampling but not a hash table. It'd be great for non-power-of-two tables while avoiding integer modulus.

[0] [https://github.com/lemire/fastrange](https://github.com/lemire/fastrange)

<comment><USER26> fastrange is discussed in the article.

</comment></comment><comment><USER27> Am I correct in thinking that using this Fibonacci method of mapping hash values to buckets will solve the "accidentally quadratic" behavior in a Robin Hood hash table? Rust's default hash table suffered from the issue but their fix was much less elegant.

</comment><comment><USER28> The comments here have given me an idea for my own hash tables.

Rather than accounting for poor hash functions using phi or fmix, I'm going to measure the hash function's distribution at run time and throw an error if its bad. For release builds I'll disable these checks.

<comment><USER29> Runtime errors seem a little bit fussy, but "try a fast approach that usually works, detect worst-case behavior, then fall back to a thing that's usually slower but avoids the worst case" is a common implementation pattern (think introsort). In principle, you could start with something trivial as your hash, then rehash your key with a better-behaved-but-slower function if your normal probing strategy is going on far longer than it ought to given the load factor.

But we rarely see that, and there are probably good reasons. Hash tables are more rarely the bottleneck in the real world than in benchmarks, and when they are an issue, other factors (size, concurrency, weird pathologies, mem latency) may matter more often than hashing time.

</comment></comment><comment><USER30> is there a way to use this technique for consistent hashing?

</comment><comment><USER31> Great article, really blow my mind</comment>


NSA says it can't search its own e-mails - <AUTHOR> http://arstechnica.com/tech-policy/2013/07/nsa-says-it-cant-search-its-own-e-mails/?fb_source=ticker&fb_action_ids=10201357828524764&fb_action_types=og.likes

<comment><USER2> Previous submissions of this story:

[https://news.ycombinator.com/item?id=6093243](https://news.ycombinator.com/item?id=6093243)

[https://news.ycombinator.com/item?id=6092485](https://news.ycombinator.com/item?id=6092485)

[https://news.ycombinator.com/item?id=6091261](https://news.ycombinator.com/item?id=6091261) <\- Discussion</comment>


OPM – Open PostgreSQL Monitoring - <AUTHOR> http://opm.io/

<comment><USER2> There's no install instructions in the .zip file, but there is a github wiki. But that has obsolete instructions. But there's an open issue, and the comments for that issue include fixes to that.

I stopped messing with it once I saw how they wanted me to install something into root's crontab.

My hypothesis: postgresql has none of these jazzy monitoring tools because it never crashes.

Sure, as traffic grows, slow queries need to be optimized, crappy N+1 ORMs need to be replaced, and boxes need more RAM and faster disks, but when was the last time you heard of anyone complain about postgresql or blame it for any downtime?

<comment><USER3> I want to be able to scale out by adding more machines. I want to be able to failover automatically to another data center when the first one goes down. I have yet to see a straight forward way to accomplish this with PG. Their wiki lists a bunch of tools related to this but they are either abandoned or does not cover this as far as I can tell. I can't understand all the positive I read about PG..

<comment><USER4> I'd say the way failover/HA is done at Heroku is straightforward. Will (the designer) or I plan to write about it some day, pending laziness.

It took some time to figure out because it required breaking some orthodoxy, but I'm happy with the result.

The thresholds are documented at [https://devcenter.heroku.com/articles/heroku-postgres- ha#fai...](https://devcenter.heroku.com/articles/heroku-postgres-ha#failover- conditions).

The promotion is done by rebinding the URL of the database and restarting the app. This shares mechanism neatly with changing of passwords, which is one reason we decided it was worth throwing out network-transparent orthodoxy when it came to HA: the clients must be controlled anyway to deal with security considerations.

<comment><USER3> > I'd say the way failover/HA is done at Heroku is straightforward.

> It took some time to figure out because it required breaking some orthodoxy

If it's hard to figure out, it does not sounds too straight forward.

<comment><USER4> Only because the idea of manipulating clients of the database to do HA is somewhat heretical.

</comment></comment></comment></comment></comment><comment><USER5> I'm not sure how these two projects are different, but they seem to be worked on by the same people/person.

[http://dalibo.github.io/powa/](http://dalibo.github.io/powa/)

<comment><USER6> Just noticed that as well.

I wonder why two different projects for the same thing? Are there any differences between OPM and PoWA? Is OPM built on top of PoWA?

<comment><USER7> Hi !

I'm involved in both projects. Here' s some insight :

OPM and POWA are not related. They were born at different times and developed by two different R&D teams at DALIBO, a French PostgreSQL company. The fact that they've been released publicly almost at the same time is merely a coincidence

OPM is designed to help a DBA to manage a very large number of servers. It provides answers to the questions : "Is there a problem somewhere on my PostgreSQL servers ? Which instances need to improvements ?"

POWA is more focused showing the "real-time" activity of one specific instance. It provides answers to questions like : "Which queries are slowing down my server right now ?"

If you're familiar with Oracle products, the difference between OPM and POWA is more-or-less the difference between Oracle AWR and Oracle Grid Control.

In a nutshell, if you only have one or two PostgreSQL servers in production, then POWA is easier to install and probably enough for you. If you want more stats and an overview of your servers, then OPM is a better choice.

Both projects are autonomous and moving fast. We're just pushing code on github without direct economic goals or marketing plans :-) In the long run we'll see which one finds its user base and how they evolve.

That being said, we also have other guys working on other similar PostgreSQL tools, in particular :

pgBadger ( [http://dalibo.github.io/pgbadger/](http://dalibo.github.io/pgbadger/) ) is a log analyzer which provides rich reports of the database activity. Basically it answers to the question "Which queries should be optimized ?"

pgCluu ( [http://pgcluu.darold.net/](http://pgcluu.darold.net/) ) is a audit/reporting tools, more focused on system and resource utilization. It answers to questions such as "What's the load on my CPUs ? Is my memory swapping ?"

pg_activity ( [https://github.com/julmon/pg_activity/](https://github.com/julmon/pg_activity/) ): is like linux "top" command but for PostgreSQL queries.

pg_stat_kcache ( [https://github.com/dalibo/pg_stat_kcache](https://github.com/dalibo/pg_stat_kcache) ) will gather statistics about reads done by the filesystem layer

So basically regarding PostgreSQL monitoring, we're just moving in every possible directions at the same time... Some say it's a "waste of time", we prefer the term "Darwinian Approach" :P

<comment><USER6> Thanks for the clarification!

</comment></comment></comment></comment><comment><USER8> Since we (Zalando) are heavy PostgreSQL users, we also had the need for monitoring and part of the result we came up with is [http://zalando.github.io/PGObserver/](http://zalando.github.io/PGObserver/) and [https://github.com/zalando/pg_view](https://github.com/zalando/pg_view)

</comment><comment><USER9> Minor grammar nitpick: "Open PostgreSQL Monitoring is a free software" is incorrect English, as software is an uncountable: we don't say "a software", but a program, a system, or something like that. You could also say "is free software".

<comment><USER7> fixed.

</comment></comment><comment><USER10> htop like application for PostgreSQL server activity monitoring:

[https://github.com/julmon/pg_activity/](https://github.com/julmon/pg_activity/)

</comment><comment><USER11> I am unsure what some of the graphs measure and labels mean when viewing the demo. Tooltips, descriptions, etc would be helpful in my opinion. Overall it seems like a great start though.

<comment><USER12> For all PGSQL graphs, the signification of each graph and label is (only :/) documented in the check_pgactivity "\--help" option, as it's the direct source of these graphs. Sadly, we didn't have time yet to write a proper documentation on the wiki page, or on the UI. We'll add a better documentation as soon as possible.

For the SYSTM graphs, they correspond to standard nagios system checks.

</comment></comment><comment><USER13> Is there anything similar for Newrelic out there? Nice project btw.

</comment><comment><USER14> The "fork" link goes nowhere, FYI.

<comment><USER12> This has been fixed. Thanks for the report !</comment></comment>


Rio Olympics Drug-Testing Lab Is Suspended by WADA - <AUTHOR> http://www.nytimes.com/2016/06/25/sports/olympics/rio-drug-testing-lab-is-suspended-by-wada.html <comment><USER2> Urgh, paywall. Far too many of these in HN recently, one could be forgiven for thinking they were spam to sell subscriptions.</comment>


Litecoin Sees Huge Price Jump After Launch Of 'Atlantis' - <AUTHOR> http://cryptojunky.com/blog/2013/04/02/litecoin-hits-new-highs/

<comment><USER2> I know it's been asked before, but what's stopping the creation of more of these 'currencies', effectively devaluing the rest?</comment>


Python implementation of passcode hashing algorithm used on Samsung Galaxy S4 - <AUTHOR> https://gist.github.com/hubert3/8560499

<comment><USER2> No practical hashing algorithm, even Scrypt, would be able to securely hash a 4 digit PIN. If they don't have a hardware security module they may as well store it plain. The amount of cryptography fail in mobile apps is stunning... from Snapchats static encryption key to pointless obfuscation like this.

If you want to talk about weak passwords though, consider that millions of broadband customers in the UK are likely using cable modems/routers with the default Wifi password that came printed on the base... currently 8 random lowercase characters on Virgin Medias newest gear for example. WPA2-PSK handshakes only add 12 bits during key stretching, so that's only 2^50 hash iterations to compromise potentially millions of wifi networks.... which can be done in about ~2 minutes to a few hours with a bit of hardware investment.

<comment><USER3> The interesting part is whenever they use the same algorithm for hashing passphrases (there's an option to lock phone using a proper passphrase, not 4-digit pin).

<comment><USER2> Good point! Although really, breaking the unlock password on a phone isn't that interesting to an attacker unless the owner has encrypted their phone or, more likely, they suspect the owner uses the same password for other things.

I think most people just use face or pattern unlock these days though?

Android actually does quite well in this regard: KitKat now uses Scrypt... although both PIN and Face Unlock can still be used with encryption, which very secure. Oddly you can't use pattern.

<comment><USER4> >I think most people just use face or pattern unlock these days though?

Most average users, sure. I've seen a couple of family members using face unlock, and many random people and coworkers using patterns. Myself, I use a password.

<comment><USER2> Patterns are terrible. I've clocked a few friends patterns accidentally, let alone actually _trying_ to shoulder surf. Far too visible.

<comment><USER5> Ah, but it's a matter of what you're trying to protect against.

I'm having a number of devices, from 'swipe to unlock' via 'pattern unlock' and full passphrases.

My phone uses a lock pattern, because

\- swipe to unlock would unlock the phone in my pocket. Rare, but happened a number of times

\- my phone is the one device that consistently lies around the house and gets carried away by my 1.x year old son. Swipe to unlock was very easy to understand...

So - pattern aren't secure, agreed. They are just a way to lock the keyboard against accidents and for that they work quite well..

<comment><USER6> This. Arguably, if you're not using full disk encryption, your adversary is someone that (at least) hasn't entered high school yet... ;-)

I'd add that it also does add a bit of "intent" (in the same sense that (quite easily pickable) door locks do -- if your child/friend/coworker unlocks your phone with your pattern, that's clearly crossing a boundary - more so than just looking at your unlocked phone.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> Is it possible to get the hash and salt off the device while the phone is still locked and not in developer mode? If not this doesnt seem too serious as a security risk.

<comment> fission-fish You can root the phone from a sdcard and then have access to it. You'll need root rights anyway to access the hash and salt. This might not work with every Android version.

<comment><USER8> maybe I'm wrong but don't you need to unlock the bootloader before, and doing the phone will be erased as a security measure?

</comment></comment></comment><comment><USER9> If you have enough access to the phone to dump the hashed pin code then why would you care what the pin code is to begin with? This seems like a non- problem unless people are reusing their phone pin code for them ATM cards but no amount of security is going to fix that kind of stupid.

</comment><comment><USER10> Question : how did you get the salt and algorithm used ? By retro engineering the code ?

<comment><USER11> "Get salt in signed numeric format by doing sqlite3 query SELECT value FROM locksettings WHERE name = 'lockscreen.password_salt' on /data/system/locksettings.db"

<comment><USER12> That does not answer his question.</comment></comment></comment>


Need feedback for my startup skanr.com please - <AUTHOR> http://skanr.com

<comment><USER2> I have no idea what business problem this solves. That's sad, because you used the word "powerhouse" to describe it, and that got my attention; that was good writing!

Learn the difference between a "feature" and a "benefit". A benefit is a pain I know I have even I don't know what any of your features are or mean. _LEAD WITH BENEFITS._

<comment><AUTHOR> This tool has the potential to drive down the costs of integrating mobile feedback. You can just use my app to integrate a range of input and media, as opposed to writing your own app from scratch to collect input.

<comment><USER2> Nope. Keep going. "Integrating mobile feedback" is a feature. See if this helps: a benefit is something my mom would understand, or a CEO would understand. What basic business function is improved by "integrating mobile feedback"? Why do I want a range of input and media, and why mobile?

Feature/function/benefit language is tricky, but it's extremely important.

Here is a _fantastic example_ of what I'm talking about:

<http://www.appointmentreminder.org/>

<comment><AUTHOR> Website owners don't have to create an iPhone application to interact with a mobile phone. Hows that?

<comment><USER2> Getting warmer, but are you sure you want to assume that every business owner that owns an iPhone knows immediately why they want a custom iPhone application? You have 15 seconds to sell them.

</comment></comment></comment></comment></comment><comment><USER3> Ok, when I read the copy on the home page and "How it Works" I didn't get it at all.

When I watched the video it made a ton of sense and made me really excited. Problem is when I come to a site like this I don't ever click on the video.

I would work on the copy, a lot. The tool is definitely powerful (I can think of several use-cases where a client wanted to use cheaper HTML forms but augmented with mobile device capabilities like camera & geo location) but the copy isn't expressing what it does in terms of the problems it solves for me, the random visitor. You need to paint the verbal descriptions with broader strokes.

<comment><AUTHOR> Thanks rwhitman, based on the comments I can tell I need to go back to the drawing board with my pitch. What could I have told you to make you more inclined to have watched the video? Thanks again for your input!

<comment><USER3> I think getting users to click video play buttons tends to be an effort in futility - video is a time commitment. Its a good supplement to the copy, but I probably wouldn't depend on it as your main tool to explain the product...

</comment></comment></comment><comment><USER4> I might be a bit slow between the ears, so I don't really understand what it is. What is it exactly? Is it a webservice, is it a piece of technology? Is it a javascript library?

The site looks very nice and you absolutely have my attention.

<comment><AUTHOR> Thanks for your interest, Skanr is the iPhone client that allows your service to leverage this new functionality. The interface used to capture data is represented by a simple JSON string, hope that helps!

</comment></comment><comment><USER5> Just a suggestion -- use DisplayRecorder (from Cydia) on a device ( <http://rpetrich.com/cydia/displayrecorder/> ) or SimFinger (on GitHub) with the simulator ( [http://blog.atebits.com/2009/03/not-your-average-iphone- scre...](http://blog.atebits.com/2009/03/not-your-average-iphone-screencast/) ) when taking your videos -- it makes it a lot more pleasant and less confusing than seeing a mouse cursor over the iPhone's screen.

</comment><comment><USER6> I don't know what it is. I read the "How It Works" page and don't understand how it works. There's something called a metric, which is encoded as a JSON string and somehow sent to a phone which renders a UI to send something back. What is communicating with the phone? Is it a web service you're going to run? Is it a library for my application to use?

<comment><AUTHOR> The HTML link includes a launch parameter telling the phone where to download the Metric (the json string), allowing the Skanr client to parse it, render the interface, and send the results to the defined destination.

<comment><USER6> What HTML link? A URL pointing to some kind of service you'll run? How will this service be connected to a phone? Is it something I'll build into a mobile application? Do you register a phone through a website and the phone runs some generic Skanr application? You've only lead me to more questions!

<comment><AUTHOR> The link that launched the Skanr app contains a variable with a URI to the Metric file. The Metric file can be stored anywhere, it just has to be a valid HTTP URI. The data is collected by the Skanr client and sent directly to your web service via XHR. Keep the questions coming if you have them :)

<comment><USER7> How does a link from my website launch the Skanr app? What is the NS class or method?

<comment><AUTHOR> The browser recognizes the protocol of the skanr:// link and launches the application.

<comment><USER7> I found it, a custom URI scheme - cool.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> Is this how it works?

I have a web app that keeps inventory of your physical book collection (not really). My users would like to use their iPhone as a bar-code scanner to add new books to their inventory. I don't want to create a native application so I decide to integrate with your app. My user installs your native app and then comes to my site to add a book and clicks a scan link. That scan link calls your native app to use the camera on the device to scan the bar-code, decode the image and send me back an ISBN.

I'm not sure I would have taken the time to figure out that's how it works without seeing this on HN.

"Go where your html form can't."

That makes sense to you and me now, but meant nothing to me when I first entered your website.

<comment><AUTHOR> That's exactly how it works, if you have any ideas on how to repackage the sell I'm all ears! It sounds like this is my new priority :)

</comment></comment><comment><USER9> I don't get targeting the iPhone yet featuring Flash on the home page. Your audience can play HTML5 or gracefully degrade to QT.

It's about iPhone, but I can't see it on iPhone or iPad.

Make pitch1.swf a .mp4 file, and use the embed stack here:

<http://camendesign.com/code/video_for_everybody>

Try the second one from the bottom, w/o the Ogg lines.

</comment><comment><USER10> In terms of the website design you should find something to put in the whitespace to the left of the video and below the description - maybe a quote from a well-known person, or make the video more square and pull up the content below the fold.

</comment><comment><AUTHOR> Hello HN, I would be super happy if you share some feedback on my project! I'm hoping my tool can help some folks make some cool mashups, thanks again for your time :)

</comment><comment><AUTHOR> I reworked the copy in the main section, hope that clarifies some confusion. I'll continue working on the copy to sell the benefits, thanks everyone!

</comment><comment><USER11> Site isn't loading for me, anyone else having this problem? I'd love to give some feedback if I could visit the site.

<comment><AUTHOR> Ahh one sec, I'm resizing the server.

</comment></comment><comment><USER7> Very well done. This is a new model, at least to me, and an easy way to access the mobile space without creating a native app. I think this will be very effective.

The website and newsletter signup process are very polished.

Congrats

<comment><AUTHOR> The goal is to make the mobile space more accessible, thanks for the comment!</comment></comment>


This robot maintains tender, unnerving eye contact - <AUTHOR> https://techcrunch.com/2018/08/14/this-robot-maintains-tender-unnerving-eye-contact/ <comment><USER2> Is anyone else getting tired of every minor announcement in robotics being described as creepy? 95% of the time I think it is more cool than creepy.

<comment><USER3> I find this cool as well. I felt (not thought - I'm talking about gut reaction) that the robot was aware just when looking at the video. I suspect the feeling would be magnified when interacting directly.

It's not just the robotics, although I think I interpret a video of a real object differently than a 3D animation.

It's that the device by mirroring a human through sensing and software can trigger my human brain's mirror neurons to emotionally engage with the robot as if it were another sentient being.

My sentience detection has been hacked. Fascinating.</comment></comment>


Teach Programming in Africa - <AUTHOR> Hi,
I'm an engineer in Mechatronics and have been working as a Software Engineer for about 5 years. Lately I have got the desire to go out and teach others and do it in countries where I could make a difference, and I am looking for Africa especially.
Do anyone of you have experience from that? Did you go through any organization, government funded projects, schools or similar?
Would like to hear your experiences! <comment><USER2> I had a phenomenal experience building a raspberry pi lab in a secondary school in the soweto-kayole slum on the outskirts of Nairobi. We spent 6 weeks training academic facilitators to maintain equipment and enrolling students in programming courses through Khan Academy, Code Academy, and other online platforms. The hardest part was teaching young adults that have never consistently used a computer how to use a keyboard and mouse. Also, how to navigate a desktop environment. It's been successful and we have seen a few programmers come out of the school with marketable skill. Students usually leave to search for a hustle to get food and clean water.

Kenyan culture and programming appeared to be oil and water for a lot of students and some teachers. They are incredibly relational and always run into to friends and family on the way to important meetings. Sometimes forgetting to show up completely because they were hanging out. We called it "Kenya Time". All the clocks ran on Kenyan Time. I had teachers get upset at me because they showed up 4 hours late to a meeting and I wasn't there.

The teachers and students that got the importance of deadlines, methodical approaches, and being on-time excelled and usually came out of the slum with a decent little programming gig.

</comment><comment><USER3> Get in touch: jkuria gmail. I can connect you with people who are already involved in this.

</comment><comment> africa-toto Good luck! Will be following this thread</comment>