Two's complement – You beauty - <AUTHOR> http://everyogi.in/c/twos_complement/ <comment><USER3> This all is suddenly less surprising for people who learned some _modulo arithmetics_ in school (or university). That is, calculating with the remainders of division. For example:

\- Calculating "modulo 60" means calculating time with a round clock in mind, considering only minutes and ignoring hours (and seconds).

\- Calculating with angles (in degrees) means just calculating "modulo 360".

\- "modulo 1000" means calculating with the last 3 decimal digits of an integer, ignoring the front digits.

The fundamental result here is: No matter modulo which number you calculate: addition, negation, subtraction and multiplication work "out of the box". And you'll quickly notice that "two's complement" just means calculating modulo 2^n, where n=8,16,32,64 or 128. But this all really works for any m >= 2, not just m = 2^n.

(One drawback though: division doesn't work here, it works only if m is prime, and even then it is slightly different from what you'd expect, although completely logical.)

In short, the elegance comes from modulo arithmetics. It has nothing to do with "two" or "binary", it would e.g. work with 3-state logic machines the same way.

 _EDIT: To those who downvoted this: Do you care to elaborate? The author did 't mention modulo arithmetics with a single word, although it is an essential part to truly understand how and why two's complement works._

<comment><USER4> I didn't downvote you. However, I am a math person and your post _feels_ wrong for some reason. It isn't wrong, but it feels like it should be; and I am not entirely sure why.

CPUs use modular arithmetic for both signed (two's complement) and unsigned values, so that cannot be the defining feature of twos complement. The insight with two's complement is that we can interperet the "upper" half of numbers as negative. As you say, this should be very familar to anyone who has worked with modular arithmatic. For those who have not, this just means that (in 3-bit twos complement/ integers mod 8). We interperate 7 = 8-1 = -1 and so on. As a result of this, operations on signed and unsigned integers are literally identical. Almost.

CPUs differ from modular arithmetic in 1 way: multiplication. Specifically, CPUs do not do modular arithmetic for multiplication. However, the result of the multiplication of two n-bit numbers could be as big as 2n-bits. When CPUs do this multiplication, they store the result in two registers. If you only look at the bottom register, the result is equivalent to modular arithmatic. However, if you consider the top register, the result is not (this is why there is a MUL and IMUL instruction).

Arguably, the same thing happens for addition. However, because there is at most 1 extra bit needed, it is not given its own register, but rather, the upper digit is stored in the carry flag.

The other insight of two's complement is encoding. Under the construction I presented above, if given the 3-bit twos complement number 0b111, we would have to compute that:

        0b111 > 0b011
        0b111 = 0b1000 - <USER2>

The insight of two complement is a way performing this computation using primitive bitwise operations. Specifically:

       If the first bit is 0, we are done
       Otherwise, perform bitwise negation, add 1, and consider the result "negative".

There is no obvious equivelent of this method for other bases.

<comment><USER3> _> CPUs use modular arithmetic for both signed (two's complement) and unsigned values, so that cannot be the defining feature of twos complement_

I disgree, because this still fits nicely into modulo arithmetics.

Signed versus unsigned just means that we choose a different set of representants for certain operations (such as). For unsigned, we use the smalles non-negative representant. For signed, we use the representant with the smallest absolute value (and prefer the negative one if there is a tie). Still, nothing with binary.

Except for one single detail: The "tie" is solved in favour of the negative number, because that way, the first bit always denotes the sign. This little details is binary-specific, but that's it.

 _> CPUs differ from modular arithmetic in 1 way: multiplication. Specifically, CPUs do not do modular arithmetic for multiplication. However, the result of the multiplication of two n-bit numbers could be as big as 2n-bits. When CPUs do this multiplication, they store the result in two registers. If you only look at the bottom register, the result is equivalent to modular arithmatic._

Good point, but in most (non-assembly) code that I see, the result of a multiplication is stored in a same-size integer. So I'd argue this is used as much. I agree that this is still a difference, though.

 _> The insight of two complement is a way performing this computation using primitive bitwise operations._

I believe that negation is _not_ what this is all about. To the contrary, the negation is more complicated for two's complement than for other representations. For example, in other representatios you just flip a single bit to negate a number.

No, the point is that there _no special cases_ for increment, decrement, addition, subtraction and multiplication (with the small difference discussed above). And there is not even a difference between signed versus unsigned arithmetics except for comparison (also discussed above). This is what works out perfectly well in modulo arithmetics, and has nothing to do with binary.

<comment><USER5> "Two's Compliment represents a _linear_ automorphism over a cyclic group" is what I think you're trying to say. russdill & co are hinting that Modular Arithmetics are cyclic groups.

I think for most people, the aha-moment comes when they realize One's Compliment double-counts the zero. In contrast, linearity is simply assumed. Otherwise, why would anyone make a number system out of it?

</comment></comment></comment></comment><comment><USER6> I program a UNISYS 2200 mainframe at work. It uses 1's complement.

Yes, there are two zeros. Not a problem in practice because all arithmetic operations normalize -0 to +0 at no extra cost in execution time, so -0 practially doesn't happen. IIRC from Assembler class, addition is implemented as subtraction of the negative operand. Just in case anyone ever needs it, e.g. for bitmaps, the SZ (store zero) assembler instruction is complemented by a SNZ.

Programming in a high level language, the representation of negative numbers is all but transparent to the programmer. When reading dumps, not having to perform an extra addition (subtraction?) when changing a number's sign is pretty sweet! Also, abs(-MAXINT) == (+MAXINT), reliably. The asymmetry of number ranges always bothered me on 2's complement machines.

What _is_ annoying about the UNISYS boxes is the 36 bit word format, though. Characters are stored in 9 bit quarterwords that map pretty awkwardly to bytes containing 8-bit ASCII. Binary data formats are essentially incompatible with anything.

<comment><USER7> > What _is_ annoying about the UNISYS boxes is the 36 bit word format, though. > Characters are stored in 9 bit quarterwords that map pretty awkwardly to > bytes containing 8-bit ASCII. Binary data formats are essentially > incompatible with anything.

This is why the FTP protocol has a byte size command. If all you have is 8-bit bytes then that seems strange. But at the time FTP was designed the most common machines on the ARPANET had 36-bit words (mostly PDP-10s and their derivatives) and bytes (the term was used in the more general sense) were just bit strings of 1-36 bits. 7-bit ascii was common (5 characters would fit in a word, like my username GUMBY), as were six bit bytes (pack six characters into a word). I never used 9-bit _characters_ though arrays of nine-bit bytes were not unreasonable.

BTW the PDP-10 had 18-bit addresses so each word of memory held a Lisp cons; CAR, CDR, RPLACA etc were machine instructions. Gordon Bell and Alan Kotok designed the -10 (and its predecessor the PDP-6) with Lisp in mind. The first Lisp Machines.

> Binary data formats are essentially incompatible with anything.

Well, that's true today, but look at it the other way around: Unix was really developed for an 8/16-bit machine. It was a reimplementation of Multics that ran on a 36-bit machine (GE 645 & Honeywell 6180) written in PL/1\. Unix was famously written for the PDP-7 (an 18-bit machine) but it was written in assembly. The famous PDP-11 version was written in a BCPL derivative you might have heard of called "C" and, since PL/1's level of machine abstraction was still new, the derivative modeled the PDP-11 architecture. So nowadays all CPUs are C machines and C runs well on them. Probably the most common non- PDP-11-like machine most programmers will program these days is a GPU.

<comment><USER8> > 7-bit ascii was common (5 characters would fit in a word, like my username > GUMBY), as were six bit bytes (pack six characters into a word)

There were a bunch of different six bit character encodings, often (though not always strictly correctly) called "BCD". The horror show of IBM's EBCDIC was an eight bit extension of one of these.

Then there was 5 bit Baudot code, and...

The last time I checked, many *nix systems will still assume that you're on a 5 bit Baudot (uppercase only) teletype (i.e., a genuine physical tty) if you attempt to log in using all uppercase in your user name.

Some systems hacked in more characters by having special "shift in" and "shift out" characters. If a "shift in" character appeared in the stream, the system would switch to the alternate character set until a "shift out" character was received.

<comment><USER9>
      > … *nix systems will still assume that you're on a 5 bit Baudot (uppercase only) teletype …

_Akshully_ the original 1963 version of ASCII¹, which was a 7 bit code but did not include lower case. The Model 33 teletype² (one of the terminals used by UNIX developers³, and probably a contributing factor to two-character command names) was a 1963-ASCII device. Even after 1967 ASCII added lower case, popular low end video terminals⁴ did not include it so that they could get away with 6 bits worth of printable character ROM.

¹ [http://worldpowersystems.com/archives/codes/X3.4-1963/index....](http://worldpowersystems.com/archives/codes/X3.4-1963/index.html)

² [https://en.wikipedia.org/wiki/Teletype_Model_33](https://en.wikipedia.org/wiki/Teletype_Model_33)

³ [https://commons.wikimedia.org/wiki/File:Ken_Thompson_(sittin...](https://commons.wikimedia.org/wiki/File:Ken_Thompson_\(sitting\)_and_Dennis_Ritchie_at_PDP-11_\(2876612463\).jpg)

⁴ [https://en.wikipedia.org/wiki/ADM-3A](https://en.wikipedia.org/wiki/ADM-3A)

</comment></comment></comment></comment><comment><USER10> The python result should be expected. Python's integer type isn't sized, that is python will happily give you factorial(100), despite it being much larger than 64 bits. It can't then give you twos complement, because it can't know the size with which to complement the two.

<comment><USER11> That's a reasonable tradeoff for Python, but it's worth noting that there is a way to deal with this.

Quoting from [http://reference.wolfram.com/language/tutorial/IntegerAndNum...](http://reference.wolfram.com/language/tutorial/IntegerAndNumberTheoreticalFunctions.html)

        Bitwise operations are used in various combinatorial
        algorithms. They are also commonly used in manipulating
        bitfields in low‐level computer languages. In such
        languages, however, integers normally have a limited
        number of digits, typically a multiple of 8. Bitwise
        operations in the Wolfram Language in effect allow
        integers to have an unlimited number of digits. When an
        integer is negative, it is taken to be represented in
        two's complement form, with an infinite sequence of ones
        on the left. This allows BitNot[n] to be equivalent
        simply to (-1 - n).

<comment><USER12> Yes, but how would you print that infinite sequence of 1s?

<comment><USER13> same as the infinite number of zeros in front of a positive number.

<comment><USER14> Just trimming 1s would be ambiguous. Is b11 3 or -2? You would have to add a 0 prefix to all positive numbers or some other to negative.

<comment><USER11> Just prefix a + or - just like you do for base 10.

<comment><USER12> So then it would not make much of a difference anymore with what Python does. Which was what they wanted to avoid.

</comment></comment></comment></comment></comment></comment></comment><comment> white-flame Looking at limited-digit odometer style devices gives the best example of why 2's complement is saner.

      ...    or  ...
      9998       1110
      9999       1111
      0000       0000
      0001       0001
      0002       0010
      ...

What number is before 0 in binary? 1111. So that's where -1 is. The number before that? 1110, so that's -2. The whole XOR + 1 thing can be derived from this shape.

That, and simple addition of both signed and unsigned numbers actually works. :)

The only question is where you draw the line between underflowing negatives and overflowing positives, and going halfsies on the top bit seems to make sense. For an 8-bit number, there are 128 numbers on each of the negative/non- negative split, but zero mucks it up by being not mathematically positive. 1's complement evens it out by having 127 numbers on each side plus two zeros, but messes up signed math.

</comment><comment><USER15> the [https://en.wikipedia.org/wiki/Method_of_complements](https://en.wikipedia.org/wiki/Method_of_complements) article explains a more fundamental piece of information about this operation - i often see articles explaining two's complement, but doesn't say anything about this general 'complement' method (which works for all bases, not just binary).

<comment><USER16> > In the decimal numbering system, the radix complement is called the ten's > complement and the diminished radix complement the nines' complement. In > binary, the radix complement is called the two's complement and the > diminished radix complement the ones' complement. The naming of complements > in other bases is similar. Some people, notably Donald Knuth, recommend > using the placement of the apostrophe to distinguish between the radix > complement and the diminished radix complement. In this usage, the four's > complement refers to the radix complement of a number in base four while > fours' complement is the diminished radix complement of a number in base 5.

That makes sense. The names one's complement and two's complement are kind of confusing otherwise, since they actually refer to totally different things, and it's not clear how they would generalize to higher bases.

</comment></comment><comment><USER17> > There would be two ways to represent 0, as +0 and -0.

IEEE 754 floating point actually has this (due to having a dedicated sign bit). I think most code doesn't care (IIRC they are defined as equal for comparison purposes even though the bits are different in memory), but apparently it's sometimes handy to have for some functions that have a discontinuity at zero or otherwise need to preserve the sign through a multiplication by zero.

<comment><USER18> Because of this, javascript has a +0 and a -0. They make a fun trivia question because there are very few ways to distinguish them since most of the ways of checking equality (even ===) will report that they are equal.

In fact, I only know two ways to distinguish them: divide something by them, and you get positive infinity for +0 and negative infinity for -0, or you can use Object.is(-0, 0) which will return false.

<comment><USER19> Weird. I just tried it out. So in Javascript you can have two variables, `a` and `b`, such that `a === b` and `1/a !== 1/b`.

</comment></comment></comment><comment><USER20> If you're looking for true beauty, look no further than negabinary. ([http://mathworld.wolfram.com/Negabinary.html](http://mathworld.wolfram.com/Negabinary.html))

E.g.

         3 = (-2)^2 + (-2)^1 + (-2)^0 = 0110_-2
        -3 = (-2)^3 + (-2)^2 + (-2)^0 = 1101_-2

There is no signed bit, you don't have to worry about sign; everything just works like "normal" numbers because that in fact is what it is.

A pity it was never used except a few times in early computing. I'm never sure why 2's-complement won.

<comment><USER21> _...why 2 's-complement won._

Maybee conversion to/from character code is easier. Notice the conversion code in the linked article.

<comment><USER20> > conversion to/from character code is easier

(1) How often do you convert from machine integers to binary character representations?

(2) If you're referring to

            for(j = 0; j < bitlen; j++) {
                bin[j] = (i & 1) ? '1': '0';
                i >>= 1;
            }

that's the exact same for a negabinary machine.

<comment><USER21> I meant to/from numbers in text form. Usually decimal.

</comment></comment></comment></comment><comment><USER22> One thing that I find very fascinating about Gustavson's Unum work is that he proposes a lot of interesting ideas - not all new, as he is happy to remind you of himself - for encoding numbers:

> _Type 2 unums are a direct map of signed integers to the projective real > number line. The projective reals map the reals onto a circle, so positive > and negative infinity meet at the top._

[http://deliveryimages.acm.org/10.1145/3010000/3001758/ins01....](http://deliveryimages.acm.org/10.1145/3010000/3001758/ins01.gif)

He also proposes to include the reciprocal of every included number in this projection, leading to a very nice property:

> _To negate a unum, you negate the integer associated with the bit string, as > if that integer was a standard two 's complement number. Flip the bits and > add one, ignoring any overflow; that gives you the negative of an integer. > It works with no exceptions. But get this: To reciprocate a unum, you ignore > the first bit and negate what remains! Geometrically, negating is like > revolving the circle about the vertical axis and reciprocating is revolving > it about the horizontal axis. And yes, the reciprocal of zero is ±∞ and vice > versa._

[http://ubiquity.acm.org/article.cfm?id=3001758](http://ubiquity.acm.org/article.cfm?id=3001758)

[http://www.johngustafson.net/presentations/Unums2.0.pdf](http://www.johngustafson.net/presentations/Unums2.0.pdf)

</comment><comment><USER23> While two's complement is quite clever, it's not an obvious choice when you are building things out of tubes or relays.

One's complement has two very nice properties:

1) the range is symmetric

2) "end-around carry" makes all the bits look identical in terms of implementation

<comment><USER24> But isn't that drawback of 1's complement that 0 in an 8 bit number can be represented two different ways?

0000 0000 and 1111 1111

<comment><USER23> Is it a drawback? If your "compare to 0" is "are all bits the same", then it would be an advantage.

An asymmetric range is a "drawback" of two's complement. Is that a drawback?

It depends upon what your building blocks in the technology are. For example, we don't use J-K flip flops anymore because they are a pain to make and use when MOSFET's are your building blocks (J-K wants bipolars).

</comment></comment></comment><comment><USER25> A widget I built last year for interactively visualizing a number circle with various binary interpretations: [https://thesis.laszlokorte.de/demo/number- circle.html](https://thesis.laszlokorte.de/demo/number-circle.html)

<comment><USER26> That's really cool! However, I can drag the SVG's viewbox around; something I wouldn't expect to be able to do.

EDIT: It makes sense when you zoom in (like you would with 7 bits), but if I'm zoomed out all the way, I wouldn't' expect to be able to pan around. I'd expect it to prevent panning past the edge.

<comment><USER25> Makes sense - thanks for the suggestion :)

</comment></comment></comment><comment><USER27> It's an elegant construct, but this used to be a part of an entry-level course in every computer school I know of. Have things changed now?

<comment><USER28> I suspect lots of people here have little or no higher education. Mine was 10+ years ago, so an occasional refresher can be nice; I remember the general concepts, but not necessarily the details.

<comment><USER29> The category of Things I Once Knew but Have Since Forgotten because I Don't Them Often Enough probably includes 95+% of everything I've ever learned about both mathematics and computers/programming.

The people on here who rattle off the names of various mathematical theorems like it's nothing and act like it's weird not to remember how intro-level algorithms work without thinking really hard and doing some trial-and-error for a while or consulting a reference must have much more interesting jobs than I _ever_ have. :-/

</comment></comment></comment><comment><USER30> Would it be possible to use - 0 & +0 in useful ways, like determining "direction" of previous operation? No clue how that could be useful, but I'm betting there's a use case out there.

<comment><USER31> Yes! Although I'm not sure if it would ever be useful for integers, it's vital in floating point: [https://people.freebsd.org/~das/kahan86branch.pdf](https://people.freebsd.org/~das/kahan86branch.pdf)

</comment></comment><comment><USER32> Probably the most interesting part of 2's complement is how to negate numbers. Flip all the bits and add 1. Which is strange, because to negate the number again, flip all the bits and add 1. It feels like accidentally adding 2 doesn't it?

</comment><comment><USER33> There is that wierd number 100000...0000 which is that extra negative number with no positive analog. I'm currently working with a noninteger value representation system that hijacks this value and assigns it as +/\- infinity (negation is two's complement)

</comment><comment><USER34> Shitty C code: the binary printing function is needlessly complicated.

        void print_bin(int x){
    	for(unsigned m=~(~0u>>1); m ; m>>=1)
    		putchar(x&m?'1':'0');
    	putchar('\n');
        }

<comment><USER35> Shitty C code: I've seen Perl golfs more readable than this.

The article's code is more explicit and verbose, which makes it easier for non-C-programmers (like myself) to actually understand what's going on.

<comment><USER36> Understandably confusing for a non-C-programmer, but it is idiomatic C. It's not code golf; it's just the way one does machine independent bit twiddling.

A mask is used to test each bit position in turn. The tests look like this if written using binary literals:

        0b1000 & x
        0b0100 & x
        0b0010 & x
        0b0001 & x

The '&' is the bitwise AND opperator in C. Of course, we'd have to do as many of these as the word size so _1010111_ uses a for loop that starts with the first mask, a 1 in the leftmost position, and shifts it right one position every time through the loop (using the C right shift operator >> on an unsigned mask value). When the one bit is eventually shifted out the right side of the mask, the mask is all zeros so the loop terminates because zero acts like false in the for loop test.

The only other tricky thing is initializing the mask. To set only the leftmost bit in a word the code uses the bit complement operator ~ of C. Breaking it down for a four bit example looks like:

        0u          == 0b0000
        ~0u         == 0b1111
        ~0u >> 1    == 0b0111
        ~(~0u >> 1) == 0b1000

This is the expression that appears in the for loop initializing the mask value, and it works for any word size.

The original article's code was definitely not idiomatic, efficient, or safe (the memory allocation for an array of characters could fail and segfault). The book _Hacker 's Delight_ is a great reference for those wanting to understand how to do low level coding, a requirement for close to the hardware work like writing device drivers.

</comment></comment></comment><comment><USER7> > I was not curious to ask what is the need for one's complement or two's > complement.

A problem with school, not the student (a common problem IMHO).

Good write up!

</comment><comment><USER37> The Universe is two's complement. Item 154 of HAKMEM: [http://catb.org/jargon/html/H/HAKMEM.html](http://catb.org/jargon/html/H/HAKMEM.html)

</comment><comment><USER38> you could as well provide an explicit mask to help (how far does the sign extend) python f.e a

             'bin(-5 & 0b1111)' gives
             '0b1011'

which is what you want

</comment><comment><USER39>
        int bitlen = sizeof(i) * 8;

I wish life was this simple.

</comment><comment><USER40> This is halfway to p-adic numbers and quote notation.</comment>


'Zen curtain' saves birds from hitting glass windows - <AUTHOR> https://www.abc.net.au/news/2019-10-28/the-curtain-saving-birds-from-hitting-glass-windows/11638774 <comment><USER2> They should make one for people.

A Midwest bank was remodeling their in-supermarket branches with glass walls. It looked really spiffy and let in a lot more light. Then a few weeks later they had decals up on some of the glass panels and they were frosting others and I asked a teller why they changed.

It turns out that many people were slamming into the glass walls, usually when turning a corner quickly, (my guess: while staring at their phones!) and hurting themselves, so the bank decided it was better to get ahead of the inevitable lawsuit and make the clear glass more visible.

How we've managed to survive this long as a species is beyond me.

<comment><USER3> The African savanna is very coal- and coke-poor -- for most of our species' early history, our natural predators, like lions and cheetahs, were unable to produce sheet glass in large quantities. When humans started moving north into Eurasia, where coal deposits are more common, they were saved by the fact that the ursine glass industry was depressed by terrible protectionist laws up until the Bearton-Woofs Agreement in 1944.

<comment><USER4> The comment was funny enough, the animal puns sends it to another dimension. Heads up:

> 1944 Bretton Woods agreement established a new global monetary system. It > replaced the gold standard with the U.S. dollar as the global currency.

</comment></comment></comment><comment><USER5> The glass walls that line motorways in Poland have black decals stuck on them that look like the outlines of birds of prey. Supposedly, birds can't tell the decal from the shadow of an actual threatening species that they would prefer to avoid, and so they don't fly anywhere near the walls.

<comment><USER6> > _" glass walls that line motorways"_

I've never heard of this, why is it done? For aesthetic reasons? It seems like it would be marginally less practical than solid barriers which would block out headlights at night.

<comment><USER5> These barriers are put up for sound-proofing reasons. Mostly the barriers are opaque, but one often encounters the glass ones at the points along these routes when they run through towns. Here, the glass barriers provide soundproofing without blocking one's view of the surrounding cityscape.

<comment><USER7> sound-reducing reasons*

Sorry for the nitpick, but I live near a highway with one of these barriers and it's in no way sound-proofing.

<comment><USER8> Were you able to experience it without the barrier?

<comment><USER7> No, but with it I can still hear the highway. I'm sure it's certainly not as bad as it is without it, and I'm thankful it's there, but it's still audible. It sounds like it's maybe 2 or 3 blocks away vs right in your back yard.

</comment></comment></comment></comment></comment></comment><comment><USER9> My current desk faces a large wall sized window, ground floor, all glass building, surrounded by trees. At first I was excited to finally face a window, but I have seen SO many birds slam into it and die on impact. Walking around the perimeter, depending on the time of year, you'll see dead birds every 20 feet or so. this sounds promising

</comment><comment><USER10> Just that small building causes 4-5 bird deaths per day? Good lord.

The picture in the article reminds me of an old youtube video that suggests painting thin white vertical stripes on your windows. Supposedly the birds avoid flying between the vertical gaps. Horizontal lines aren't as effective. I wonder who first discovered this method.

<comment><USER11> > Just that small building causes 4-5 bird deaths per day? Good lord.

He did say that it only happens during "this time of year". Collisions with buildings kill up to 1 billion birds per year in the US [1], so that sounds about right.

[1] [https://en.wikipedia.org/wiki/Environmental_impact_of_wind_p...](https://en.wikipedia.org/wiki/Environmental_impact_of_wind_power#Birds)

</comment></comment><comment><USER12> This reminds me of the awareness efforts by the Nature History museum in Rotterdam with the celebration of Dead Duck Day [1].

The case study's[2] abstract documents:

_On 5 June 1995 an adult male mallard (Anas platyrhynchos) collided with the glass façade of the Natuurmuseum Rotterdam and died. An other drake mallard raped the corpse almost continuously for 75 minutes._

Quite an interesting—albeit disturbing—read.

[1] [https://www.hetnatuurhistorisch.nl/activiteiten/dead-duck- da...](https://www.hetnatuurhistorisch.nl/activiteiten/dead-duck-day/dead- duck-day-2019.html)

[2] [https://www.hetnatuurhistorisch.nl/fileadmin/user_upload/doc...](https://www.hetnatuurhistorisch.nl/fileadmin/user_upload/documents- nmr/Persberichten/Persberichten/persberichten_2013/DSA8_243-248.pdf)

</comment><comment><USER13> Perhaps less visually distracting to the people inside the buildings, would be to create the same "striating patterns" on the building using those large decals that are used for public-transit advertising.

<comment><USER14> Yes, this is also common and covered in the article...

</comment></comment><comment><USER15> Nice to see Twitter finally being used solve problems for birds themselves.

</comment><comment><USER16> I wonder if the same tactics can be applied for apple developers...

<comment><USER17> We did it to ourselves with this modern "glass walls and doors everywhere" design trend.

</comment></comment><comment><USER18> I wonder how this actually works?

I regularly place seeds on my deck railing to feed the birds (and the squirrels). They knock some of the seeds to the deck floor.

Some birds go for the seeds on the deck floor. Those birds frequently fly between the balusters, both when coming and going. The balusters are 1.5" across, with 5" gaps between them.

Based on watching those birds, I would not have expected a "zen curtain" to help. I'd expect birds to think they could simply fly through the gaps.

<comment><USER19> The birds aren't aiming for seeds on the other side and have no motivation to fly through the ropes?

</comment></comment><comment><USER20> > Mr Hines has been burying the birds that die in the nearby garden, and has > been taking the birds that survive to a local vet.

I thought Lorikeets were classified as pests in both Western Australia and Queensland. A few years ago my mother and I found an injured baby Rainbow Lorikeet in the middle of the road (some kind of wing injury). We put it in a box and went to the vet. Right before handing it over, I happened to ask the receptionist: "You guys aren't going to just kill him or anything, right?"

That is when the receptionist decided to tell me that because Rainbow Lorikeets are considered pests, that is exactly what they would do. They wouldn't even look at the bird's injury.

I took him home instead; he recovered and went on to live with me. I really hope that this person had some kind of agreement or assurance from the vet that they would actually help the birds. Of course not every bird that flies into a window can be saved, but if he was taking them there expecting them to get some kind of general diagnosis or treatment those birds may not have gotten the ending he was expecting.

</comment><comment><USER21> My place of employment desperately needs something like this. We have a number of sky bridges between buildings that are just a suspended hallway with floor to ceiling windows on both sides, so to birds it looks like they can fly through. We're located at the edge of the city, so there are just rolling hills of farmland and more natural space for birds to approach from. I regularly see dead birds under the sky bridges.

<comment><USER22> Maybe stick something on the inside of the window?

</comment></comment><comment><USER23> What I find fascinating about this is that there's probably a set of neurons in bird brains that specifically fire when looking at vertical lines. Here's a live recording of this effect from a cat[0], also known as the Hubel and Wiesel[1] experiment. Now those are being triggered, causing the bird to avoid the glass. This is an exceedingly simple solution to the problem. Very impressive!

[0] [https://www.youtube.com/watch?v=Cw5PKV9Rj3o](https://www.youtube.com/watch?v=Cw5PKV9Rj3o)

[1] [https://en.wikipedia.org/wiki/Torsten_Wiesel](https://en.wikipedia.org/wiki/Torsten_Wiesel)

</comment><comment><USER7> This is an old trick used on homes, except window paint (typically white) is used to make thin vertical lines about 6 inches apart. It doesn't affect your view and it doesn't look ugly if you use a straight guide and thin lines.

</comment><comment><USER24> My previous house one year a male Cardinal thought he saw a rival in a window and attacked it over and over again for what seemed like hours (his reflection). The female sat on the fence and I could swear was enjoying it.

<comment><USER25> That happened at my house (though with a different species). I closed the shade, which disrupted the reflection, so that the bird would not harm itself or waste its energy unnecessarily.

</comment></comment><comment><USER26> My house has large glass windows. The temporary solution (nothing more permanent than a temporary soultion) to the frequent dead birds was to stick little pieces of bright green masking tape on the window.

</comment><comment><USER27> I wonder how subtle the lines could be, and still work? Could a stripey near- invisible 'shadow line' be silkscreened on the glass before its installed?

</comment><comment><USER28> Looks like a valid and clever solution for a problem. I wonder how this will stand wind, or if will create noise hitting against windows.

<comment><USER29> I recall a short video [0] gaining traction on this topic a while back with the simple solution of drawing the lines on windows with a marker pen.

[0]: [https://www.youtube.com/watch?v=UC9xQkUtQ98](https://www.youtube.com/watch?v=UC9xQkUtQ98)

<comment><USER10> That's the video! I instantly thought of that when I saw the picture in the article. Thanks, I wasn't able to find it.

</comment></comment></comment><comment><USER30> Related-ish Planet Money episode:

[https://www.npr.org/sections/money/2017/07/28/540084544/epis...](https://www.npr.org/sections/money/2017/07/28/540084544/episode-786-rest- of-the-story-2017-vol-1)

Go to 16:40 for the related part.

</comment><comment><USER31> I need a zen curtain on the front of my phone, my television, my laptop screen ... heck I need some zen everywhere

</comment><comment><USER32> There was a particularly invisible glass wall in a parking lot near me. I guess enough people walked into it that they did the low-tech human version of this by sellotaping a piece of paper onto it that simply said "window"... I can't imagine birds understanding that, but it was hilarious to see.

</comment><comment><USER33> It's interesting how one community sees the rainbow lorikeet as an invasive pest that needs to be eliminated and another community is carefully redesigning buildings to save the poor rainbow lorikeets.

<comment><USER34> All native wildlife in Australia is protected -- cute cuddly and deadly poisonous animals equally.

Lizards, spiders and insects found in a home are often caught and released outside in the garden rather than killed (especially the huge huntsman spiders we get here which are harmless to humans). I'll even catch and release cockroaches rather than kill them.

<comment><USER33> Yes, but they're not native to Western Australia. They were introduced by accident in the 1960s.

They were also accidentally introduced to New Zealand in the 1990s by some idiot in North Shore who released large numbers of captive ones.

</comment></comment></comment><comment><USER4> I've seen a church in Kerala, India use plastic chain curtains over their doors to prevent birds flying inside. Seems they are commonly used for this purpose.

</comment><comment><USER35> Twenty years ago a friend of mine sold something similar for houses. We live in the mountains, lots of birds, lots of bird-window collisions.

</comment><comment><USER36> Why is it called a 'Zen curtain'?

<comment><USER37> See related: [https://news.ycombinator.com/item?id=21386788](https://news.ycombinator.com/item?id=21386788)

Saving you a click: because "Zen" is a marketing term meaning "calm or peaceful". Birds not slamming into glass is considered to be peaceful. Not be confused with the "Zen" of Zen Buddhism, where it doesn't mean that at all.

<comment><USER34> > Not be confused with the "Zen" of Zen Buddhism...

Or motorcycle maintenance.

<comment><USER38> Many consider "Zen and the art of motorcycle maintenance" a modern-day Buddhist parable though the author states it specifically not to be..

</comment></comment></comment></comment><comment><USER39> Zen it's all about being in concordance with nature</comment>


When a Memory Allocator Hides Security Bugs - <AUTHOR> https://blog.fuzzing-project.org/65-When-your-Memory-Allocator-hides-Security-Bugs.html <comment><USER2> Instead of switching to the debug allocator, might it instead be possible to teach ASAN about the pool allocator? I think I did this at some point in the past for Valgrind. I don't know if it would be sufficient, but with ASAN it's at least possible to "poison" and un-poison memory manually with macros: [http://blog.hostilefork.com/poison-memory-without- asan/](http://blog.hostilefork.com/poison-memory-without-asan/)

<comment><USER3> I really like APR. While pooled allocation isn't always the best approach, it makes a lot of sense in server software, and reduces the risk of many classic memory problems (e.g., I don't have to worry whether I pair 1000 mallocs with 1000 frees; it's acceptable to pair 1000 apr_pallocs with a single apr_pool_free).

ASAN is certainly helpful when using classic memory management techniques, but since so much C code uses custom memory management of some kind, it has to handle more than malloc and free.

I certainly hope people improve ASAN instead of abandon things like memory pools. I would consider "malloc and free with ASAN" to be a step down from APR's pools.

<comment><USER4> ASAN does handle this. As nkurz noted, custom allocators need only poison/unpoison memory regions to benefit from ASAN and MSAN. For example see how the absl "SwissTable" does: [https://github.com/abseil/abseil- cpp/blob/master/absl/contai...](https://github.com/abseil/abseil- cpp/blob/master/absl/container/internal/raw_hash_set.h#L1587)

</comment></comment></comment><comment><USER5> Heartbleed was also an example of a bug which was hidden from the various validation/static checking tools because OpenSSL was using custom memory allocation, wasn't it?

<comment><USER6> Yes. There's a good article here explaining why various static analysis tools didn't catch Heartbleed: [https://dwheeler.com/essays/heartbleed.html](https://dwheeler.com/essays/heartbleed.html)

>It is known that OpenSSL does not directly allocate and deallocate memory directly using standard calls. Instead, it used a caching freelist system internal to OpenSSL to reuse allocated memory. Since OpenSSL does not return (deallocate) memory back to the underlying system once it was done with it (in some cases), special tools could fail to detect some common weaknesses such as use-after-free or double-free that they would otherwise find. It also sometimes prevented some operating system and run-time mitigation mechanisms from working. In short, it is widely agreed that this OpenSSL memory allocation approach prevented many mitigation and weakness detection mechanisms from working.

</comment></comment><comment><USER7> Somewhat ironically, a major secondary reason we used custom allocators in games (performance bring the primary reason) was to find and diagnose memory bugs. Long before ASAN existed I worked with some very powerful in house memory debugging tools at EA, some of the functionality of which I still haven't seen matched in commercial or open source tools.

Security wasn't the primary concern back then, it was avoiding memory leaks on platforms with no virtual memory and tight memory constraints and squashing bugs.

</comment><comment><USER8> I increasingly feel we need to apply more "sledgehammers" to security. If I was running servers with security I cared about, I would use a malloc which cleared all allocations, and the new llvm/clang flags which clear all stack variables to fix unitanialised problems. Maybe even try turning on the more advanced clang sanitizers, if I could afford the cpu time.

If we, as programmers, want to be respected, we need to make every reasonable effort to improve security.

<comment><USER9> I believe OpenBSD does most of these things. The primary problem is that security is a tradeoff -- for instance OpenBSD completely disabled hyperthreading when it was discovered to be fundamentally insecure. That was a 50% performance hit right off the bat -- not everyone can afford to make such a tradeoff.

</comment></comment><comment><USER10> Expect many of these bugs with WebAssembly.

</comment><comment><USER11> Misleading title... why all bugs are "security bugs" nowadays? It's just a bug.

Also, the allocator does not hide the error. It's not a problem generated in or from the allocator. It's a problem generated by the programmer and the impossibility of static analyzers to know every allocator here and there.

</comment><comment><USER12> > and that indeed it seems some people did run the pool debugger in production > (OpenBSD).

.. another remote exploit for OpenBSD ?

<comment><USER13> OpenBSD doesn't include the Apache HTTP server, so while their default settings might expose Apache bugs if you choose to use it, it's not fair to say it's a vulnerability in OpenBSD

<comment><USER14> This reasoning is getting annoying. If you do nothing with your OS, it's secure! If you use software it bundles it's not secure and it's not their fault (but they'll fix it anyways)!

That is just oh so gimmicky. It reflects quite poorly on the project.

<comment><USER15> I think I'm missing something here. How is a bug in Apache an issue with OpenBSD?

<comment><USER16> I believe what's being referred to is the first sentence on the OpenBSD site: "Only two remote holes in the default install, in a heck of a long time!"

A more cynical reading of it would be: "OpenBSD is secure as long as you don't run software on it." I share that cynicism.

The discussion is relevant if you believe that an OS should have some responsibility for memory-related security issues.

<comment><USER17> > The discussion is relevant if you believe that an OS should have some > responsibility for memory-related security issues.

Pedantically speaking, your point is valid. But as a practical matter, OpenBSD is the operating system that takes these issues the most seriously. They formalized the notion of Unix-based privilege separation, and systematically employed it across the system. They regularly perform system-wide code audits (including of bundled software, such as Apache back when they actually bundled Apache) to identify and remediate high-risk source code patterns (e.g. strcpy -> strlcpy, realloc -> reallocarray, etc). While people [still] debate the merits and performance of things like seccomp or Capsium, OpenBSD has incorporated pledge(2) into the vast majority of binaries on the system. They were the first to apply stack canaries, ASLR, W^X, and most recently PIE _globally_ by default. And apropos the article, they were the first to implement their system malloc in a robust way that maximized the chances of catching memory bugs, sometimes sacrificing performance in a way that would be inconceivable for Linux distributions. And all of these things have resulted in a steady stream of bug fixes upstream to third-party packages from the ports collection.

One of the reasons OpenBSD forks projects like openssh or libressl, or implements from scratch services like iked, httpd, ntpd, relayd, smtpd, and many others, is precisely so they can assume full responsibility and control for things like memory management.

See [https://www.openbsd.org/innovations.html](https://www.openbsd.org/innovations.html)

When a researcher points out a potential issue on OpenBSD, they're emphasizing their point by leveraging OpenBSD's credibility and the hard work put into the system.</comment></comment></comment></comment></comment></comment>


Ask HN: How much time do you spend on HN? - <AUTHOR>

I use to spend at least 1 hour/day on HN. Sometimes, when things are slow, I end up spending 2 hours. But today I noticed that the board is really active, and wondered.. How much time do you spend on HN? Do you stay here for more time on fridays? <comment><USER2> Don't really count as I'm on and off throughout the day but I'd say probably a couple hours per day.

</comment><comment><USER3> around 2 hours

<comment><AUTHOR> 2 hours every day?

<comment><USER3> yea :) .. sometimes it gets interesting reading the tech stuff</comment></comment></comment>


Ask HN: Should I stay in a company if I know they are selling smoke? - <AUTHOR>

I've recently started working for a medium-sized start up (~100 employees, 8 years old) as a junior developer. Up until now I've been working as a freelancer in direct contact with the client/end user, and all my projects translated into something useful: a product or some extra functionality on a product that satisfied a particular demand. Very small contracts lasting only a few months and comprising just a few thousand lines of code each.
Now I just do whatever management throws at me, which is usually work handed to them by account managers who get contracts to work on big projects (hundreds of thousands or even millions of lines of code and several months or even years of development).
One of the first things I've noticed on my new job is that a good number of projects never reach production or, if they do, they are useless products retired or restarted from scratch by a different company a few months later. Specially when it involves government contracts or collaboration with other companies.
Should I worry about this smoke/functional product ratio and start looking for a new job, or is it what I should expect from any software development company handling many big projects at once?
What should I do if I stay between a few months and a couple of years and become project manager to avoid any products I am put in charge of ending up as vaporware?
What should I do if despite my best efforts I still can't avoid it? <comment><USER2> I suspect what is driving your question is that you're mystified how you can be paid to produce work which isn't meeting some need, or being tangibly important in some way. If you're working on stuff which never reaches production or is immediately retired or restarted then it probably seems to you like a bad way to run a business.

I can sympathize. I worked for a bank in a role doing tasks which, to me, clearly did not meaningful add to the companies bottom line. In my estimation, my (and my department's) existence was a drag on the bottom line. I constantly found myself amazed that the company spent so much money in inefficient counterproductive ways. So I wasn't honestly all that surprised when my department was wiped out last week and I was laid off.

I think ultimately you're right: when it's all vapor then there will likely come a day or reckoning. But you never know how soon they day will come. Maybe it's a proactive restructuring of the company and they downsize and cut because they realize the error of their ways (and pivot to producing more value), or maybe it happens when the economy tanks and money gets tight.

I think you'd just be smart to continue to diversify your options however long you work in your current position. Play your cards in such a way that if you ever wanted to leave or had to leave, you would be able to frictionlessly pivot to something else. If you can do that then I say don't sweat making a buck off your employer's vapor production. When and if the gravy train comes to a halt, you'll be ready to do other things.

On the other hand, if you're not happy in your job--i.e., if you don't like your work, your colleagues, the environment, or your boss then consider staying there just long enough to make it legit (probably about a year) and then go looking for something else.

Good luck!

<comment><AUTHOR> Thanks :-) I'm actually quite happy. The pay is decent, the colleagues are great, smart and they have a good deal of experience. Plus the office is not far away from home. From what I know from my college partners, such a combination is quite difficult to achieve.

I enjoy what I do and I am learning a lot. I see many things I like and many other I would do on a very different way, such as:

\- There is no automated testing at all,

\- No separation of duties either,

\- The version control system is ridiculously outdated, misused and underused,

\- The deployment strategy is practically non-existent,

\- The upper management constantly tries to push technical decisions that sell great to the client but make little to no sense on the project context, adding many man hours to achieve the same results.

Fortunately everybody identifies this issues and they do what they can to fix them (except for the automated testing, it seems like they all think it's worthless), but the company is growing and external image is all they can care about for most of the time...

</comment></comment><comment><USER3> I currently work for an MLM business. It literally is a pyramid scheme.

I'm shopping around for something better, but the pay is decent.

Standard Advice for your situation. Do the best you can in your current job, while looking for something better. You already have a job, so you can be somewhat picky about your next project. Knowing that you will have an new job in 1-12 months means that it's easier for you to put up with their foolishness for now.

<comment><AUTHOR> Good advice, indeed. I don't think I'd have any problems getting a new job right know, because everybody wants junior developers, and practically none of them are as experienced as I am.

A few years from now it might be very different, so if I intend to switch I should better do it ASAP.

The only problem is: What do I do to avoid ending up in the same situation? I did my research on this company before and during the interview, and it's almost impossible to know what happens inside the office beforehand (Even people you might know already working there usually won't tell you anything about what's going wrong, because it can damage the company image...)

That's why I want to know if that's something every company has to deal with, and what are the chances of finding one where it doesn't happen.

<comment><USER3> There are no guarantees. It's hard to size up a place until you're there a few weeks/months.

For example, at my current job, the boss admitted on the interview that their system was a mess and they wanted to clean it up. But, once I was working there, I'm doing 100% maintenance with no real opportunity to refactor or clean up the old system.

Also, after a management change, a good job can turn bad. When my current boss was hired, all the programmers hired by the previous boss quit in the next few months.

I had a job turn sour when a coworker was fired. I had double the workload, and the easier tasks my ex-coworker was doing wound up on me.

It does get a lot harder as you get older. If a job ad says 2-5 years of experience, then 15 years of experience makes you ineligible. Also due to technology churn, your experience in older languages has a market value of zero (even though it does have real value).

I have had jobs that I liked. In one job, most of the team interviewing me seemed awful. The guy I wound up working with was very nice, but he only spoke for me for 2 minutes during the interview. I thought I flunked the interview and it would be an awful job, but it wound up being nice.</comment></comment></comment>


A New Perl.org (Redesigned) - <AUTHOR> http://www.perl.org/

<comment><USER2> Finally it has an easy-to-find explanation of Perl 5 vs Perl 6 that newcomers can understand:

<http://www.perl.org/dev.html>

</comment><comment><USER3> those fonts don't look too well on mac os/firefox:

<http://imgur.com/R75yG.png>

<comment><USER4> Looks like they're using sIFR. Maybe you have flash or javascript disabled?

</comment></comment><comment><USER5> Nice, but a little heavy on the dark blue.

</comment><comment><USER6> Looks fantastic!</comment>


Ask HN: Why hasn't Rap Genius been shut down? - <AUTHOR>

Given the news of the recent raid of Undertexter, I'm curious how a service like Rap Genius stays online?
Undertexter profited, via banner ads on the site, for the community sourced subtitles.
Rap Genius, while not currently profitable, did take a 15 million USD round of funding for their site hosting community sourced song lyrics.
Undertexter created derivative work by creating translation of copyrighted subtitles to native languages.
Rap Genius created derivative work by creating an annotation layer on top of copyrighted subtitles.
In both cases, neither party had obtained permission or licensing.
So how is Rap Genius's model or execution different from Undertexter? <comment><USER2> I believe because:

1\. Lyrics are user generated content, UGC + DMCA Safe Harbor makes it easy for a site to keep operating, even if they have to keep taking some content down.

2\. They have done a seemingly good job of getting major label rappers on board with the service, so maybe the labels actually get this this could be helping them make money.

<comment><AUTHOR> 1\. So are translated subtitles. Is the premise that Undertexter was not responding to DMCA requests to take down reported content, but Rap Genius is?

2\. This confuses me. Unless the labels are licensing/legally approving of the use of the content, isn't just allowing it to happen weakening their case against sites they are not so friendly with?

</comment></comment><comment><USER3> Undertexter was profiting from simply translating subtitles whereas, Rap Genius falls under fair use because, they're creating a new layer on top of the lyrics.

</comment><comment><USER4> Because Rapgenius hasn't profited out of the lyrics yet. DMCA focused on monetization of other people's content as one of the main criterias for violation. It doesn't matter whether it's UGC or not.

<comment><USER5> If I recall, this is basically the surest way to lose your DMCA protection. The key is for them to prove that you knowingly profited from the copyright infringement. Another issue is what percentage of your service is legit/infringing.

As much as people hated it at the time, the DMCA is a pretty reasonable law (which is why the bad guys at various companies are trying to replace it).

<comment><USER6> The DMCA has no such consideration of profiting from infringement or the percentage of the service that is legitimate versus infringing. A service that contains nothing but infringing content, and puts ads on every page hosting that content, can be completely free from liability for the infringement. No need to speculate based on bits and pieces you think you've heard places -- read the bill! It's short, it's not hard to read, and it's something you should know given your profession.

<comment><USER5> This is common and incorrect view of the legal system in the US. The law as it is written is only half of the picture. You need to understand the relevant case law to have any idea how a judge might actually rule on a given case. This is one reason why attorneys make so much money.

</comment></comment></comment></comment><comment><USER7> Users submit lyrics, sometimes the actual artist of the song verifies it too, it's a completely different theme than 99% of lyric databases, kind of like, the reddit of lyrics in a way?

<comment><AUTHOR> This is the same model as Undertexter (minus the "sometimes the actual artist verifies it part), the subtitles are sourced via community user submissions.

What do you mean by it being different than 99% of lyric databases?

Are you implying that 99% of the lyric databases copy their content from somewhere or the site admins themselves enter the content themselves?

Does that really make a difference since the end result would be identical (both in content and in execution) in that they are hosting and displaying copyrighted content that they are not licensed for, regardless of what the originating source was.

</comment></comment><comment><USER8> rap genius is not derivative. its transformative, and therefore falls under fair use. Its also UGC, which as noted is specifically granted safe harbor under the DMCA.

<comment><USER9> Is UGC always a safe harbor? To qualify for safe harbor in the case of file sharing services, it's not enough not to deliberately populate the site with copyrighted content and to honor takedowns; there's also a requirement that the operators of the site not run it with foreknowledge of infringing copyright, so that if they became aware of copyvio content, they'd still be required to take it down.

</comment></comment><comment><USER10> I'm more curious about how they plan on monetizing "annotating stuff".

</comment><comment><USER11> Because the RIAA isn't as crazy as the MPAA

<comment><AUTHOR> Or are they?

[http://en.wikipedia.org/wiki/International_Lyrics_Server](http://en.wikipedia.org/wiki/International_Lyrics_Server)

"On January 14, 1999, Swiss police raided de Vries' apartment and the service's ISP and seized equipment in pursuit of a criminal copyright violation complaint filed on behalf of eight music publishing companies including Polygram, EMI, and Estefan Enterprises. Although the industry succeeded in taking down this service (and turning many music fans to dislike the industry), the Internet proved more powerful than the industry, and song lyrics are now freely available on many similar sites."

</comment></comment><comment><USER12> Shhhhhhhh.</comment>


Eloquis: Early in the market and targeted the wrong customer segment - <AUTHOR> https://failory.com/interview/eloquis <comment><USER2> I don't know if there is a lot of useful information to glean from this post, but I did enjoy guessing about a few things that were (domain) name-related.

Under "What would you do differently?"

> I would definitely not put a lot of emphasis on domain name or invest a lot > of money without researching the potential of the name.

I was tickled by this, because at the outset I was actually interested in reading an article about the failure of a highly-marketed blood thinner of the (almost exact) same name. I guess this guy probably learned it's hard to win mindshare & SEO points when you're up against Pfizer, who owns Eliquis.com.

</comment><comment><USER3> Not going to lie - first thought from headline was "how does a blood thinner target the wrong customer segment?? Have you ever been to a coumadin clinic?" And then I realized the issue.

<comment><USER4> Yes, hard lessons and the funny thing is we were oblivious to that name and paid a lot to acquire the domain name.

</comment></comment><comment><USER5> After reading the article, I can't figure out what their product does at all in practice.

Apparently it turns "hello user" into "Hello Bob, it's great to meet you on a wonderful December day" but it's not at all clear how that's packaged into something that's useful and non-trivial.

That seems like it could have been one of the main problems.

<comment><USER4> Well, the example is a very simplistic one for the purpose of this interview. The vision, if you could call it that, was to package other signals e.g. weather, geography, and demography into personalizing a message. Having come from a background of e-commerce I definitely know personalization has a positive impact on cart values, engagement, and conversions. Considering all of that our hope was that we would be that kind of a personalization software for mobile apps.

<comment><USER5> But how does that work in practice?

Does their system determine those signals or are they fed to it as input?

How can it possibly provide a generic solution to "personalizing a message" that works across all apps and is not just something trivial like the example in the interview?

On the web the kinds of "personalization" I can think of are public account profiles, targeted advertisements, recommendation engines ("you may also like...", "products like this...", etc.), algorithmic newsfeeds and history- sensitive search. If their software does any of those, maybe they should just say what it does.

</comment></comment></comment><comment><USER6> > Our brand name was a disaster and most of the traffic that we got, for free, > was looking for an alternate name called "Eliquis" a drug from Bayer. There > was no way we would win the SEO game. We had no SEO strategy or any other > ecosystem strategy to piggyback our marketing on.

First thought was definitely to the blood thinner... Which is an interesting commentary on the state of direct to consumer pharmaceutical marketing in the United States...

<comment><USER4> Absolutely the blood thinner won and it's a hard lesson that we learned.

<comment><USER6> My point was not so much about losing a battle over name recognition, that's something that happens relatively all the time. I mean specifically losing it, even to the general public that isn't on blood thinners (like myself), to a blood thinner. In a lot of developed countries, where direct to consumer pharmaceutical advertising isn't legal this obviously would not have happened...

</comment></comment></comment><comment><USER7> If I may be so bold:

TL; DR: "I spent 20k on writing code"

Why?

Because there's not a single sentence that I read addresses the most important question for any bootstrapped $THING:

* why would anyone want to pay me for this $THING.

<comment><USER4> You bring up a good question. The $ amount while painful was spent over getting some design work,domain name etc... The value we thought the product would add was to make apps more engaging. Personalization is a big deal in e-commerce world and I naturally translated that as a consequence to mobile app - I do realize that was a mistake. So yeah, we definitely expected to get paid for such a service.

<comment><USER7> > Personalization is a big deal in e-commerce

That's a claim that is not backed up by anything. Most personalized service one can get is in boutique non-ecommerce companies. Those companies are struggling.

That in turn means that personalization at this time is a _gospel_.

</comment></comment></comment><comment><USER8> To the interviewee: Having learned these lessons, why are you uh, making the "Product Hunt for Marketing Tools" next? Have you applied your learnings from the failure of Eloquis? This seems like an even harder segment to make money in.

<comment><USER4> Yes the lessons learned here were translated. As of now the product hunt for marketing is a place for discovery and we realize that our new model will build a community and we have to build active monetization around value-based product around the same premise - yes product hunt for marketing won't make money by itself.

</comment></comment><comment><USER9> I hadn't come across failory.com. Like autopsy.io - with multiple sites like this around, it seems like we rather like revel in failure...

<comment><USER10> I believe the father of them all is [http://fuckedcompany.com/](http://fuckedcompany.com/) by [https://news.ycombinator.com/user?id=pud](https://news.ycombinator.com/user?id=pud)

Reading how others failed is a good way to avoid failing the same way. At least your failure will be original!

</comment></comment><comment><USER11> Interesting, he recommends the book:

> The Black Swan - The Impact of the Highly Improbable: Achieve deep thinking > and not arrive to simplistic explanation.

When I first read that in 2013 it spurred my interest, which eventually lead to my current company:

[https://projectpiglet.com/](https://projectpiglet.com/)

The goal of which is to actually "identify" black swan events. The premise of a black swan is that you can't predict it, but I'd argue that quite often in a world of several billion people - someone has predicted it (typically those most in the know).

Anyway, it's a good book and I've actually been using the system I built ([https://projectpiglet.com/](https://projectpiglet.com/)) recently to watch the decline in hype around cryptocurrencies. Which just goes to show, as the author notes - you can often be too early (or late) to the game and it makes all the difference.</comment>


Wall Street Applied to Technology Entrepreneurship - <AUTHOR> http://www.centernetworks.com/wall-street-startups

<comment><USER2> really good article - especially the part about patience and not being a pig because pigs get slaughtered.</comment>


Hawaii Extends Thirty Meter Telescope Permit Amid Protests - <AUTHOR> https://www.npr.org/2019/07/31/747056053/hawaii-extends-thirty-meter-telescope-permit-amid-protests <comment><USER3> I think a lot of people in La Palma would be quite happy to have it built in the Roque de los Muchachos obvervatory [1] if Hawaii doesn't want it.

The local papers are positively salivating at the prospect.

[1][https://en.wikipedia.org/wiki/Roque_de_los_Muchachos_Observa...](https://en.wikipedia.org/wiki/Roque_de_los_Muchachos_Observatory)

</comment><comment><USER4> I visited Mauna Kea as a tourist some years ago.

At the time, there were protesters present.

They appeared to believe that they had been promised there would be a limit on how many telescopes were build, and that the astronomers had broken that promised by starting to build a limit+1th telescope without removing an existing one first.

Anyone know if there's any truth to this?

<comment><USER5> Part of the TMT agreement (from 2015) involved removing some of the telescopes[0]. At least one telescope has been removed (the Caltech Submillimter Observatory, CSO) but aside from the CSO I'm not sure what the status is of other removals. The agreement was that:

> three or four of the mountain's 13 existing telescopes must be dismantled > over the next decade.

This implies the currently agreed upon limit is 10 or 11 telescopes (including the TMT). Unless the agreement has changed, I don't think the promise has been broken since the removal was over the 2015-2025 timescale. That being said, I'm unaware if there was a previous deal.

There had been some controversy in the past regarding "outrigger" telescopes for the 10m Keck telescopes. These would have been used in combination with the 10m telescopes to do interferometry. It was argued that they were thus not "new" telescopes, but rather additions to the existing Keck telescopes. This was controversial and they were never built.

[0] [https://www.nature.com/news/hawaii-prunes-mauna-kea- telescop...](https://www.nature.com/news/hawaii-prunes-mauna-kea-telescope- hub-1.17688)

<comment><USER6> The outriggers did get approval. My understanding is that they were not built primarily for financial reasons.

<comment><USER5> Okay, thanks for the clarification. I hadn't previously heard what the ultimate reason was.

</comment></comment></comment></comment><comment><USER7> It seems to me one of the best ways to protect a sacred site would be to put a serious telescope upon it.

As it restricts access for a host of other activities for many miles around to keep light polution amd vibration to a minimum.

<comment><USER8> Limiting discourse to only your way or the highway is disrespectful and precisely how these protections are ignored. This time it's a telescope, next time it won't be.

Perhaps the community wants to protect their sacred site from building a telescope as well. Ignoring that because you believe your cause to be more valuable than their home isn't justified. It's as nonsensical as claiming that the best way to protect against pregnancy is to get pregnant, it's just doublespeak.

We could level a bunch of suburbia all over the country and put telescopes there, and that would both remove a blight on the American landscape while protecting another untouched one. But this isn't what we discuss because "their" sacrifice is much easier for us to agree on than ours. Like it or not, it's the same colonialist logic that led to the genocide of the American natives.

</comment></comment><comment><USER9> Every source I've found just says "it's sacred in the Hawaiian religion" or "it's a sacred place" but always a vague answer. If you wanted to build on the site of the Wailing Wall or the Kaaba, the opposition would have a very specific list of ways those sites are sacred, with names and events. Additionally, less than 0.5% of Hawaiians practice the Hawaiian religion in question.

Frankly I think this is a bunch of contrarians opposing the construction just to have something to feel self-righteous about.

<comment><USER10> How do you know that the site being considered significant is tied solely to belief in Hawaiian cosmology? I don't know much about Hawaiian culture but couldn't it be the case that a wider proportion of the population considers the site significant for varying reasons? This article looks to imply just that: [https://www.vox.com/identities/2019/7/24/20706930/mauna- kea-...](https://www.vox.com/identities/2019/7/24/20706930/mauna-kea-hawaii)

"Inseparable from Hawaiian culture is a love of the land, 'āina. For Hawaiians — who thrived in the islands before foreigners arrived at the end of the 18th century — care and attachment to the land is central to their identities. The term "aloha 'āina," literally meaning love of the land, has long been a rallying cry for Hawaiians."

I don't have a horse in the race but feel there might be more to it than "...a bunch of contrarians opposing the construction just to have something to feel self-righteous about."

<comment><USER11> _I don't have a horse in the race but feel there might be more to it than "...a bunch of contrarians opposing the construction just to have something to feel self-righteous about._

I wouldn't agree there. If you're a thinking human being, you have a horse in the race. Right now, a bunch of superstitious nitwits are swinging a hammer at its knees.

<comment><USER12> Let's just say you are a member of a group of people who have been in a constant cultural conflict with another group of people. For hundreds of years they've inserted themselves into your homelands, force changes to the culture to fit their needs, and generally been entirely disinterested in actively engaging with you as a coequal resident of their space.

Then, finally, you're able to find some ways to use their cultural system against them to get your needs met and your voice heard in your homeland. Some of this involves appeals to logic, but others involve appeals to emotion. To you, the ends justify the means. The stakes feel very high.

Now imagine some dude on the internet refers to you as a nitwit without really have looked at the larger picture of what your political action within your homeland means within the larger context of your struggle to maintain your culture and be represented as a coequal member of your homeland.

How do you think you'd respond to this dude on the internet?

<comment><USER11> What does any of that have to do with building a telescope? Sounds more like an extortion scheme.

If your culture doesn't contribute to the advancement of humanity, but rather actively stands in the way, then it needs to die.

<comment><USER12> That's a take...

1- This telescope doesn't need to be built here, right now. The world will not end if it isn't built, and all the "benefits" you might claim we gain from this are essentially marginal improvements over the existing half dozen telescopes in that area already. There is no reason that the previous expected agreement of "if you want to build new telescopes you need to decommission an existing one and replace it" can't be honored.

2- Do you know what "extortion" actually is? Do you know what the term "tyranny of the majority" means? Are you familiar with any actual sociological concepts in play in this scenario? Do you actually understand the hypothetical example I tried to walk you through previously, or would you like me to further break down a broad view of what this conflict is about rather than trying to go to snappy talking points and witty one liners? Are you actually just trollin son? These are all earnest questions.

</comment></comment></comment></comment></comment></comment><comment><USER13> The interesting twist about building telescopes in Chile is that it gives the military industrial complex an excuse to build up to 'protect' them. Just like the Panama Canal, the Suez Canal the Hormuz strait and other internationally critical pieces of infrastructure. Sure people wont starve if a Chilean revolution takes over the telescopes and demands 'taxes', but the massive investment will protect itself.

</comment><comment><USER14> Why is this better than launching a telescope into orbit?

<comment><USER15> Mass - cost-benefit still means ground based telescopes are the best option for most scenarios. The protests may change that equation.

As much as I'm automatically inclined to side with science over mysticism, there _are_ other telescopes being built right now (ELT, GMT) in the same class, both in Chile, where the mountains aren't sacred, but just mountains.

So - while they're pushing on for now, it seems there's as much dogma (we must have _this_ telescope _here_ ) amongst scientists as their is amongst the spiritualists - yes, the top of Hawaii is a uniquely good spot for a telescope - _within US borders_ \- there are other spots just as good, however, but those would require international negotiation and cooperation rather than just domestic negotiation and cooperation.

It all comes down to cost/benefit, and I think it's getting marginal for the TMT. They're going to need round the clock security, as the protests won't stop once it's built.

It's not too late to build it elsewhere - even in space. Launch costs have dropped dramatically since the TMT was conceived, and are continuing to do so. Assembly will then be the tricky part, as with JWST.

<comment><USER14> Thanks. So what kind of improvements do they expect with this one?

<comment><USER16> It's planned as a general purpose observatory. Bigger mirror than any ever used would allow sharper, better images.

<comment><USER14> Ok. So a space telescope would be prefered and this is the second best option, limited by budget?

<comment><USER15> Absolutely. The cost, even with current cheapening launch costs, would still be astounding. As long as we're using chemical rockets fuelled from earth we are going to be really limited in terms of what we can put in space - <USER2> again, assembly - we couldn't even get Hubble right first time, and it has taken thousands of hours of spacewalks to keep operating.

It's totally been worth it though.

Stepping stones - if the new class of earth based telescopes finds something worth looking at, there _will_ be space telescopes that make the JWST look like a toy - but not for a while yet. This is basically the "cheap" trial experiment.

<comment><USER17> > it has taken thousands of hours of spacewalks to keep operating.

The total eva time for all five servicing missions for Hubble is under 200 hours.

</comment></comment></comment></comment></comment></comment></comment><comment><USER18> And if the construction location were the western wall? St Peters basilica? The Vietnam war memorial? Arlington National cemetery? A site of equivalent meaning to your worldview? What then?

<comment><USER19> Well, I don't think the Native Hawaiians actually built Mauna Kea. I also suspect that the majority of Jews would not approve of building something atop the Western Wall, the majority of Catholics would not approve of building on St Peter's Basillica, but a 2018 poll conducted by the main Honolulu newspaper found 72% approval for building the TMT among Native Hawaiians ( [https://tinyurl.com/y2qboalr](https://tinyurl.com/y2qboalr) )

<comment><USER18> I looked up the estimated Native Hawaiian population in Hawaii; 298,000 people.

28% of that is 83,440.

A minority of 83,440 native people with shared history and spiritual beliefs are asking to have what is held dear to them treated in a way that does not feel like a violation.

How small a group must they be to make ignoring their plea acceptable?

<comment><USER19> Well, if the other 72% of Native Hawaiians want to see it built on Mauna Kea, why do you dismiss their preferences? If just one person claimed to be offended, does that mean the project has to be stopped? The line has to be drawn somewhere, and 50% seems like a good choice.

</comment></comment></comment></comment><comment><USER20> Maybe its the sacred lands sacred destiny to offer us a view into the cosmos, who has any better clue than any one else to guess divine intents of any kind ?

<comment><USER21> I presume they have some sort of source of truth for this stuff like a religious scripture or oral tradition, but this should be obvious and I fail to see how you could have missed it.

</comment></comment><comment><USER22> If you keep asking they might cave in and say yes?

</comment><comment><USER23> The main problem here is one of incentive. Of course the researchers, university departments, contractors, etc want to build it. But who benefits? A very small number of people. All the locals get is: more traffic, more pollution, more waste, all of which are a direct result of the construction itself plus the increase in visitors. Plus, the locals feel this is imposing on their scared grounds. Again, without ANY benefit to them.

I talked to one person at the protests, who said "The researchers don't want light pollution. We don't want telescope pollution."

<comment><USER6> _Again, without ANY benefit to them._

There is a large benefit to the locals in terms of jobs both during construction and after. Yes, the researchers are not local, but the engineers, technicians, maintenance staff, and so on usually are. Astronomy is the second largest contributor to the economy on the Big Island (after tourism).

<comment><USER19> It also significantly diversifies the economy. Tourism is cyclical. Astronomy research may have booms and busts too, but they are apt to be uncorrelated with the ups and downs of tourism.

<comment><USER24> This is quite literally the best place to study stars in the whole world. Economically sure, not a lot of people may benefit, but scientific knowledge the whole world benefits</comment></comment></comment></comment>


Cost Comparison of Deep Learning Hardware: Google TPUv2 vs. Nvidia Tesla V100 - <AUTHOR> https://medium.com/bigdatarepublic/cost-comparison-of-deep-learning-hardware-google-tpuv2-vs-nvidia-tesla-v100-3c63fe56c20f <comment><USER2> Here is a larger-scale comparison of Cloud TPU and Google Cloud GPU performance and cost (focused on Cloud TPU Pods): [https://cloud.google.com/blog/products/ai-machine- learning/n...](https://cloud.google.com/blog/products/ai-machine-learning/now- you-can-train-ml-models-faster-and-lower-cost-cloud-tpu-pods)

All the code used in that comparison is open source, and there is a detailed methodology page with instructions that you can follow if you want to reproduce the results: [https://github.com/tensorflow/tpu/blob/master/benchmarks/Res...](https://github.com/tensorflow/tpu/blob/master/benchmarks/ResNet-50_v1.5_Performance_Comparison_TensorFlow_1.12_GCP.md)

Also, Cloud TPUs are available to everyone for free via Colab. Here is a sample Colab that shows how to train a Keras model on the Fashion MNIST dataset using the Adam optimizer: [https://colab.research.google.com/github/tensorflow/tpu/blob...](https://colab.research.google.com/github/tensorflow/tpu/blob/master/tools/colab/fashion_mnist.ipynb)

(I work on Cloud TPUs)

<comment><USER3> Are there plans for preemptible TPU pods?

As is, it looks like p3 16xlarge spot instance (or probably preemptible gcp 8xV100) are still by far the most cost effective option. You have to do a bit extra to tolerate preemption, but it's worth the ~80% savings.

Also, the "TPU pod is 200x faster than single v100" comparison is a little goofy. Might as well say Summit or Titan is faster than my desktop.

<comment><USER2> At present, preemptible Cloud TPU v2 and v3 devices are widely available, and they are likely to be the most cost-effective option for training any of the models listed here, often by a wide margin: [https://cloud.google.com/tpu/docs/tutorials](https://cloud.google.com/tpu/docs/tutorials)

Definitely appreciate your point about comparing Summit to your desktop - however, the difference is that you can't rent Summit via any public cloud, whereas you _can_ rent Cloud TPU Pods.

You might find the comparison between 8 x V100 GPUs on GCP and a full Cloud TPU Pod more relevant - in that case, as of the time the Google Cloud blog post linked above was published, a full Cloud TPU Pod delivered a 27X speedup at 38% lower cost for a large-scale ResNet-50 training run, all without requiring any code changes to scale beyond a single device.

<comment><USER3> Thanks for your response.

I'm interested particularly in preemptible _pods_ \- I see here [https://cloud.google.com/tpu/docs/pricing](https://cloud.google.com/tpu/docs/pricing) pricing for preemptible single units, but there is no indication that preemptible pods are available.

</comment></comment></comment></comment><comment><USER4> This doesn't seem like a very informative benchmark to me. They don't mention how/whether they tuned the learning rates and batch sizes to optimize for each different device. Like they mention, they also use a very small network that isn't something you need the power of a TPU to train quickly and may scale differently than a large network.

They also don't post their code so I can't check that their problems with ADAM aren't due to using L2 regularization, which [https://arxiv.org/abs/1711.05101](https://arxiv.org/abs/1711.05101) shows leads to worse performance than SGD and you should use weight decay instead.

<comment><USER5> > They don't mention how/whether they tuned the learning rates and batch sizes > to optimize for each different device.

All networks were trained with the same hyperparameters. Only the batch size was increased with the amount of parallelization used (so increased 8-fold for TPU and distributed GPU).

> Like they mention, they also use a very small network that isn't something > you need the power of a TPU to train quickly and may scale differently than > a large network.

I agree completely here. We unfortunately didn't have access to the TPUs long enough to create more useful benchmarks on networks like Resnet-50 and with bigger datasets.

> They also don't post their code so I can't check that their problems with > ADAM aren't due to using L2 regularization, which > [https://arxiv.org/abs/1711.05101](https://arxiv.org/abs/1711.05101) shows > leads to worse performance than SGD and you should use weight decay instead.

The code is the same for all devices and in the single-GPU and CPU benchmarks we see Adam performing better than SGD. We did not use any regularization besides a dropout layer so I don't think this explains the bad Adam performance on TPU.

I'll make some effort to clean up the code so it can be shared.

<comment><USER4> The fact that you didn't change the learning rates for different batch sizes even following a linear scaling rule let alone tuning for each batch size is pretty important. IMO without tuning the learning rates comparisons across batch sizes are pretty meaningless because it changes the effective learning rate which can make a big difference in training.

<comment><USER5> Good catch! I'll try to rerun the experiment. Hopefully the Google Colab TPUs give similar results to the Google Cloud ones so I can keep experimenting.

Still, since Adam performs worse even for non-distributed TPU (where the batch size and learning rate are the same for all devices) this wouldn't explain what we're seeing. Anyway, I'll try to find some time tomorrow to post the code so you can all have a look at it and hopefully see what's up.

</comment></comment></comment></comment><comment><USER3> Looks like riseml has shut down and taken their comparison post down. I was hoping to compare the results.

<comment><AUTHOR> It's still in Googles cache: [https://webcache.googleusercontent.com/search?q=cache:zZwHCS...](https://webcache.googleusercontent.com/search?q=cache:zZwHCS9zhzAJ:https://blog.riseml.com/comparing- google-tpuv2-against-nvidia-v100-on-resnet-50-c2bbb6a51e5e+) Not in archive.org though.

</comment></comment><comment><USER6> Hmm interesting, is it possible that the batch size for the tpu is larger? I'm guessing they might be using some sort of large batches to populate their giant GEMM cores

</comment><comment><USER7> My understanding is that Teslas have scientific accuracy that is not needed for deep learning... my machine has nvidia 1080s.

<comment><USER8> Tesla V100 has tensor cores which are supposed to massively accelerate deep learning compared to the Pascal architecture of a 1080, which lacks them.

I believe they are present in the Turing products from nVidia too.</comment></comment>