What PHP 5.5 might look like - <AUTHOR> http://nikic.github.com/2012/07/10/What-PHP-5-5-might-look-like.html <comment><USER3> The introduction of list comprehensions is nice and should replace numerous functions. For example, I'm not sure why they're adding the array_column function into PHP 5.5 at the same time as list comprehensions as:

      $names = array_column($users, 'name');
      // is the same as
      $names = [foreach ($users as $user) yield $user['name']];

The only possible reason would be due to a significant speed difference, but I'd suggest improving the efficiency of the list comprehension system (even if just for common cases) than adding more functions.

As opposed to the introduction of list comprehensions, I can't say I like the solution for parameter skipping though... I disagree with the author -- many optional arguments is not a problem, but only if keyword arguments or a similar style are used. Consider the example they give:

      // This create query function in PHP
      create_query("deleted=0", "name", default, default, false);
      // could be represented like this with keyword arguments
      create_query("deleted=0", "name", escape_arguments=false)

I tend to find keyword arguments serve the purpose of self documentation as well -- I still have no clue what the false flag would be for the PHP create_query statement.

Keyword arguments and many optional arguments can allow for beautiful and flexible functions, such as the Python Requests library[1].

      r = requests.get('https://api.github.com/xyz')
      # or we could add a few more complications
      # -- no "defaults" in sight and also self documenting
      r = requests.get('https://api.github.com/xyz',
        auth=('user', 'pass'), timeout=0.5, allow_redirects=True)

[1]: <http://docs.python-requests.org/>

<comment><USER4> > I'm not sure why they're adding the array_column function

Isn't the function call much more readable than the comprehension?

At least as a beginner, I found the large set of built in functions to be one of the strongest points of PHP.

<comment><USER3> One major disadvantage with so many functions is the cluttered global namespace[1]. PHP only added namespaces in 5.3.0 and can't modularize their globals into namespaces without causing massive backwards compatibility problems. PHP is stuck with a polluted global namespace, I'd be wary of adding more. Mind you, I'm well aware that this feeling could just be as I'm primarily a Python coder -- in PHP, you perform array_keys($xyz) whilst in Python it's xyz.keys(). Whilst you might consider Python as cheating by pushing all the functions onto the objects, it results in a far happier global namespace[2].

As far as list comprehensions, I tend to find list comprehensions the readable and self documenting, at least in Python where I most commonly see them. If nothing else, what is the ordering of the arguments for the equivalent non list comprehension? With list comprehensions you don't have that impedance.

      // Python example with filter vs list comprehension
      // Filter requires you to remember the ordering --
      // and does it filter on True or False?
      filter(lambda x: x % 2 == 0, range(10)) --> 0 2 4 6 8
      // The list comprehension is self documenting
      [x for x in range(10) if x % 2 == 0] --> 0 2 4 6 8

I can also convert the majority of list comprehensions into English quite easily -- "output x for each x in range(10) if x is divisible by 2" (though it may be cleaner if it was "for each x in range(10) [output x] if x is divisible by 2").

[1]: <http://php.net/quickref.php>

[2]: <http://docs.python.org/library/functions.html> \+ <http://docs.python.org/library/exceptions.html>

<comment><USER5> >PHP only added namespaces in 5.3.0 and can't modularize their globals into namespaces without causing massive backwards compatibility problems. PHP is stuck with a polluted global namespace

They could do what Python did for 3.0, and make a tool similar to Python's 2to3 that replaces global namespace references.

<comment><USER6> That sounds like a disaster. I don't think anyone wants to follow Python's lead on breaking backwards compatibility without a very good reason. A much better reason than a cluttered namespace.

</comment></comment></comment></comment></comment><comment><USER4> I was confused by the password hashing example since it doesn't appear to use a salt (which was the main reason to implement it at all). Even more confusing, the RCF says it will auto generate a salt if none is passed.

The explanation, a bit further down in the eRFC is that the function password_hash doesn't actually return a password hash, but a string including the algorithm and options used, the salt and the hash itself.

From the RFC:

> It's important to note that the output of crypt() (and hence > password_hash()) contains all the information that will be needed to verify > the hash later. Therefore, if the default hashing algorithm changes, or the > user changes their algorithm, old hashed passwords would still continue to > function and will be validated properly.

</comment><comment><USER7> > PHP 5.5 will no longer support Windows XP and 2003. Those systems are around > a decade old, so PHP is pulling the plug on them.

Given the recent threads about planned obsolescence (Apple) and OS fragmentation (Android) I'm curious what is missing from XP/2003 that is part of Vista+. Are there still exciting things happening in the world of OS APIs that are relevant for server software?

Or does this just mean that noone will actively _test_ the binaries on XP anymore?

<comment><USER8> I can foresee them making a turn on this decision and continuing to support XP until developers actually stop using XP anymore. XP isn't going away anytime soon.

It was nice to see short tags "<?" coming back in 5.4. Before that, they used to say, don't use short tags, it's dangerous, breaks things. But users resisted it, and every ISP kept it open. And now from PHP 5.4 short tag is ON, regardless of whether one setts it On or Off in php.ini.

<comment><USER9> No actually, as of 5.4 <?= is always available. <?= and <? are not the same thing.

</comment></comment></comment><comment><USER10> These sound like useful improvements, especially the new getter/setter syntax and scalar type hinting.

However, what I'd really like to see in next PHP would be the Composer dependency manager bundled in, a bit like Node.js nowadays ships with NPM. This has really brought a new world of cross-project code sharing into PHP:

<http://packagist.org/>

I wrote a blog post on why this is important for improving the state of PHP:

[http://bergie.iki.fi/blog/composer_solves_the_php_code- shari...](http://bergie.iki.fi/blog/composer_solves_the_php_code- sharing_problem/)

<comment><USER11> I think composer is a bit too young to introduce into the language. Perhaps it will prove out to be robust, but I'd wait a little while before introducing it to core...

<comment><USER12> When you look at people complaining at the deprecation of 5.1 and 5.2, I'd rather not have any userland code shipped.

</comment></comment></comment><comment><USER13> 17 years old and a PHP Internals tinkerer...

Awesome work Nikita, keep it up!

<comment><USER11> As someone who has worked with him for quite some time, all I can say is so very much agree. He blows me away left and right with his knowledge, ability and maturity (especially when it comes to code decisions). He's more senior than most developers that I know...

So very much agree, keep it up!

</comment></comment><comment><USER14> I was involved with PHP internals when type hinting came up for 5.3, and helped steer that discussion in some way [1]. It was a truly awful experience - at the time, the internals community was a very negative and poisonousness place. Does anyone still involved know if that has been fixed?

[1] oh look, I'm still mentioned in the RFC :) - <https://wiki.php.net/rfc/scalar_type_hinting_with_cast>

<comment><USER15> Given that you're involved in this - why add "with cast"? I thought the whole point of the feature, in regards to classes and arrays, is to be absolutely strict about function arguments. Once you start down the path of "well, as long as it's sort of like that," then you might as well accept objects that have toString defined in place of strings. Having worked in a large PHP project for years and trying to get it to a point of letting new developers spin up pretty quickly (and use PHPStorm efficiently), STRICT type hinting is one of the best tools possible!

<comment><USER14> So I'm not involved in the current discussion, but I remember some stuff from the last discussion and in general.

To start, you can consider the problem of type checking of scalars in a language with implicit type coercions. You're supposed to be able to treat 0 and 0.0 the same. At the same time, if you have a type check that says "int", you kinda expect to get an int. So "with cast" does that.

I know a lot of people find that ugly, and that you want to strictly that you must receive an int, but that's just not how people use PHP, and it doesn't make sense in the context of the language. As an example, you want to be able to add type checks to all the functions in the standard library, many of whom treat null, "0", 0 and 0.0 to be the same.

For PHP specifically, there is the additional consideration that PHP is supposed to be a newbie friendly language. There was a perception in the PHP internals community that newbie developers could not wrap their head around the concept of "types". I've heard people say that a developer shouldn't be required to think about what types his variable might hold. Clearly this is lunacy for many reasons, but there it is.

<comment><USER14> I just read the RFC, and I think it's very good. The only minor quibbles I have would be using the "compiler" instead of the parser to produce the hint, and whether or not you should cast if losing precision. But these are minor issues that should probably be ignored, since this improves the status quo considerably.

</comment></comment></comment></comment><comment><USER16> The "parameter skipping" functionality is really useful but IMO they should take the same route that C# did with named parameters.

        function create_query($where, $order_by, $join_type='', $execute = false, $report_errors = true) { ... }

        create_query("deleted=0", "name", report_errors: true);

can even specify all parameters by name in different order

        create_query(order_by: "name", report_errors: true, where: "deleted=0");

</comment><comment><USER17> Pretty exciting, though empty() should be deprecated, not improved upon.

The bit about getting the fully qualified class name is important, but it still prevents you from doing something like:

      function builder_factory( $var ) {
        $class = $some_array[ $var ];

        return new $class();
      }

So you have to write a ton of boilerplate for something that used to be easy without namespaces, or write namespace traversal into your PHP (which isn't THAT hard, but is very ugly).

Parameter skipping looks intuitive and useful. Very important as we incorporate more functional paradigms into the code.

I don't believe scalar type hinting will make it in. There's just too much discussion around it. IMHO, it should be super strict. "1" is not an integer.

Getters and setters: meh. I guess it's good. Better than what we have now.

I don't believe PHP will do generators or list comprehensions right, so I'm not holding my breath on those.

<comment><USER18> Better than parameter skipping would be named parameters so you could just do:

        function foo($a, $b="10", $c="5", $d="3") { /* .. */ }

        foo(5, $c="10");

That way $a == 5, $b == "10", $c == "10", and $d == "3". Much better and cleaner syntax in my opinion and a lot of other languages support something similar.

<comment><USER11> The problem with that is it's already valid syntax with a different meaning.

        function foo($a, $b = "10", $c = "20") {}
        foo(1, $c=20);
        var_dump($c); // int(20)

The syntax would have to be unambiguous. Perhaps:

        foo(5, c: 30)

or

        foo(5, $c: 30);

or

        foo(5, $c => 30)

or something like that...

<comment><USER19> What people end up doing in practice is passing in arrays with key/value pairs. The => syntax would be the closest thing to the current way of doing things, except you'd get default values and type hinting. I'm all for it!

I'm still looking for a good way to combine array parameters with defaults and hinting in PHP 5.3. I've had some success using DTO's for primary API's:

      function foo(SomeDTO $data) { ... }
      foo(new SomeDTO(array("a" => "foo", "b" => 10)));

      class SomeDTO extends BaseDTO {
         /** @var string
             @maxlength 10 */
         public var $a;
         /** @var int */
         public var $b = 20;
      }

The BaseDTO class uses reflection to parse the doc comments and figure out how to validate and set its input. It's the same idea as validation annotations in java. It works, but it's quite a heavy syntax, and I wish I had a lighter- weight alternative. I like PHP's loose typing inside an API's class, but when interfacing between API's I want strict typing.

</comment></comment></comment></comment><comment><USER20> This is just a "would be nice" sort of post right? The main feature I'd want (list comprehensions) seems to be just a single programmer's guess at an upcoming feature with no real evidence.

<comment><USER21> Pay close attention to the "status" listed under each header. Some of these features have already landed in master, and others are still in various states of proposal. The author links to the official RFC for most of the proposed features.

I'm not sure if the list comprehensions feature has made it past the mailing list though.

</comment></comment><comment><USER22> Consistency in dereferencing is very important. empty() should at some point be deprecated, but fixing this erratic behavior for now is good imo.

The default value for method arguments is a shockingly bad idea for a new feature, it only supports the old, bashed upon, code quality that have been PHPs greatest legacy problem. If anything in this alley I'd like to see named arguments somehow.

I'm not so sure about property getter/setters as I find the syntax a bit awkward, all while magic methods lets you create getter/setter based APIs.

<comment><USER23> What's the deal with empty()? I've never used it myself but found people at the new place I work do use it.

<comment><USER18> Like the article said, you can't use it with function calls because empty() itself is a construct of the language and not a function. So, empty(someFunc()) is an error. You'd have to put the return value of someFunc() in a variable and test it for emptiness, or do something like if (0 == strlen(someFunc())) { /* .. */ } which is overly verbose and not fast.

</comment></comment></comment><comment><USER24> I think a standard password hash API is a good thing (there was a proposal for such a thing for Python recently on -ideas, though it was essentially rejected in favor of recommending/using passlib), but I'm wary of having default cost factors, that seems unwise: what are the defaults, how are they decided to be ok, and more importantly in what context are they updated over time (and based on what data)?

<comment><USER25> The problem with it in it's current form is it does not allow for an application pepper.

Also I'd prefer it to support scrypt as well as bcrypt.

<comment><USER24> > The problem with it in it's current form is it does not allow for an > application pepper.

Considering:

* no password hash algorithm I know of supports peppers

* that the API allows providing a custom salt

I don't see any issue with the API. If the cryptographic worth of peppers is ever demonstrated _and_ a password hash is built to use peppers, the pepper can be provided as an option to that hash algorithm as the salt and cost already are in the proposed API.

<comment><USER25> Every hashing algorithm allows it, even bcrypt. Very simple example; md5($salt.$pepper.$clearText);

The problem with this API is that if you pass in the "salt" as $salt.$pepper then the output hash also contains the pepper.

The whole point of a pepper is to keep a second salt out of the database. The user salt would be in the database, but the pepper should only be in the application code.

If your database is stolen, but your application code is safe the pepper increases the complexity of brute forcing, as they need to work out what the pepper is

<comment><USER24> > Every hashing algorithm allows it, even bcrypt.

That is not the meaning I intended in my usage of the word "support", but if you equate "support" and "is compatible with", then this API also "supports" peppers, just as much as bcrypt does.

> md5($salt.$pepper.$clearText)

How cute, not just md5 but length-extension vulnerable MD5. I'd recommend not using that scheme for MACs (and more generally not using md5 directly, really, as there are precious few reasons to do so)

> The problem with this API is that if you pass in the "salt" as $salt.$pepper > then the output hash also contains the pepper.

Which just happens to be _the exact same way bcrypt's API works_. Here's an idea: combine the pepper to the password (this is usually done through hmac), not the salt. That's how you use a pepper and remain compatible with the Modular Crypt Format.

> The whole point of a pepper is to keep a second salt out of the database > [blah blah blah]

Contrary to your apparent belief, I am aware of what peppers are, how they are used and what they're supposed to do.

<comment><USER26> A length extension attack is fairly irrelevant for password storage...

</comment></comment></comment></comment></comment></comment><comment><USER27> Regarding the "constant dereferencing" proposal, how do you even make a language parser that can't apply array operations to literals in the first place? I'm trying very hard not to bash on PHP here, and this is a completely honest question. I can't figure out how you'd even go about breaking that if you wanted to. Anyone know the background there?

<comment><USER14> This would probably ban "4[5]" but not "$x = 4; $x[5]".

PHP enforces a lot of things in the parser (as opposed to making bytecode and having a simple type checker). Instead of having a general "expression" type which can be dereferenced, they have different rules for scalars, strings, variables, function calls, etc. For a long time, you couldn't dereference the result of a function call, because the parser didn't allow it, so this is probably the same.

<comment><USER27> Enforcing it in the parser seems so horribly wrong. If you want to disallow stuff like 4[5] at compile time then you should do that in a separate pass on the AST, not in the parser itself. That's ought to be a semantic error, not a syntax error. Treating that separately would require all manner of special cases. But, I guess that must be what they do.

<comment><USER14> Yeah, there's tons of special cases in the parser. I worked on phc (<http://phpcompiler.org>), so I've read the PHP parser and helped write some of our own. We took the AST approach and it worked out pretty well. But PHP was written before its authors knew good interpreter design, and its currently very hard to refactor it out to a better design.

<comment><USER27> Painful. Thanks for sharing your experience. Cool to ask such a question and get an answer from someone who's actually built something related.

</comment></comment></comment></comment></comment><comment><USER28> Yet another array function. But, array_column does seem super useful.

</comment><comment><USER29> The proposals look great! But I'm missing one thing: Less fatal errors, or a better way to catch them. Especially for the case of calling a method on a null value. E.g.

        $foo->getBar()->getBaz()

If getBar() return NULL for some reason, you get a fatal error which you cannot catch and handle without reverting to uglu hacks with a shutdown function.

My personal preference would be to have that statement return NULL and raise a warning or notice, alike to using uninitialized variables.

<comment><USER30> Don't return null from that function. Return NullObject which has __call() { return $this; } - then you could chain as many such calls as you want and they'd be OK with nulls. Then you could check for NullObject as easily as you could check for null - if($result instanceof NullObject) etc.

</comment></comment><comment><USER31> Conspicuously absent from PHP:

       array_copy()

... in order to copy-by-value on arrays containing references. Currently to do that you have to do a very fragile hack like this:

       $copy = array_flip(array_flip($original));

... which is vulnerable to key-value collisions ... or you have to write your own homebrew array copy-by-value function.

<comment><USER5> File a bug report :)

<comment><USER30> Better yet, file an RFC and a pull request.

<comment><USER5> Yep. That's what I'm trying to do for something else :D

</comment></comment></comment></comment><comment><USER32> If there is scalar type hinting I wonder if that will lead to real method overloading by adding methods with different signatures or if devs will have to stick with the __call magic method, which I despise.

</comment><comment><USER33> The getters and setters change is looking great! Overall lots of good ideas!

</comment><comment><USER34> Is PHP JIT at least on the roadmap?

<comment><USER35> As far as I am aware, it is under consideration for Zend Engine 3 / PHP6. But certainly not for 5.5, and probably not in the 5.x series at all. Though how many 5.x releases are left is anyone's guess, personally my crystal ball tells me we will get 5.6 followed by 6. My crystal ball also says we will all have flying cars by now.

</comment></comment><comment><USER36> Laravel already does alot of this stuff ^^^

</comment><comment><USER37> No matter how much PHP is improved upon, until there is a serious contender framework for it like Rails or Django, PHP will continue its route to extinction. Dinosaurs were once big and powerful. They dominated the landscape. They don't exist anymore.

<comment><USER38> _No matter how much PHP is improved upon, until there is a serious contender framework for it like Rails or Django, PHP will continue its route to extinction._

I'm not sure if you're serious, but there are plenty of frameworks on a par with Rails at least (nb: I have zero experience with Django).

At the enterprise level, there's Symfony or Zend:

<http://symfony.com/>

<http://www.zend.com/>

If Rails is your thing, CakePHP shares some concepts (although I'll be the first to admit CakePHP has many flaws):

<http://cakephp.org/>

Want something lightweight that's easy to jump into? CodeIgniter's the one for you!

<http://codeigniter.com/>

And finally, my favourite, Kohana. Absolutely infuriating since they essentially stopped writing documentation for new versions, but if I need to get something written quickly and reliably, this is my go-to call:

<http://kohanaframework.org/>

There are tonnes of "serious" frameworks for PHP (I've certainly missed out a few), and frankly, if you're going to try to attack PHP, this is the wrong angle from which to do so.

<comment><USER39> Please don't recommend CodeIgniter. I'm not normally one for swathing "x is bad" judgements, but CI is simply not in the same league as the other frameworks you mentioned.

<comment><USER33> Tell us why! With the low memory overhead, built-in cache, Active Record (+PDO support) and easy-to-understand MVC approach, I think it is the best framework for writing small to medium-sized web applications.

CI is not Zend, but nothing prevents you from using the Zend libraries with a small CI Library wrapper - I have done so many times in the past, and it works great.

<comment><USER39> CI gives you very little, and what it gives you is mostly categorically broken. The total lack of post-redirect-get and insane choices on the session data front meant I was using a custom session storage library I wrote instead of the bundled one.

We had issues where the router and loader were not playing nicely at all - <USER2> were using the 404 override option in order to do some custom routing (as the router was not flexible enough for even a very basic CMS). However because of how messy the routing code is, and how much of a hack the 404 override was (last time I checked there were several open bugs regarding this on their tracker, some over a year old and completely untouched), it completely blitzed the loaded libraries, so we had to add in more hacks to dodgily clear the cache of loaded libraries and re-run the loader in order to have them available. The only other way to do this would have been to modify the core loader to fix the bug there (but we didn't want to modify core code to make it easier to keep up to date with framework changes), or completely rewrite the router (which really needs to be done, it's a mess).

The database abstraction code is very kludgy and basically useless for anything beyond utterly, utterly basic use cases (there's a reason it's low overhead - it's low everything, including functionality!)

That's just a few of the things we ran into. By the time we went to production I estimate that out of the maybe 30% of CodeIgniter we were actually using for our project, I had re-implemented as custom libraries maybe a third of that amount just to get some sane behaviour, and work around long-standing bugs. Overall I got the feeling that the framework had no solid direction, some of the core components (most critically the loader and router) were quite obviously piles of hacks rather than having been designed and engineered, and honestly the framework was not really much more re-usable or robust than our own custom in-house one. I know it's a very common developer hubris to think you could write your own framework and do a better job, but in the case of CodeIgniter I can confidently make that claim.

On the more technical and abstract side, it's insane that it doesn't use standard PSR loading (so any other library you want to use you have to write a custom wrapper), and the guidance on making re-usable libraries / modules / packages (I can't even remember the right terminology, they use these words in strange ways) is... unclear at best. It uses globals, it's tightly coupled, everything a decent framework should categorically not be. Overall I would say that the level of technical ability on people using and contributing to CI is very low, and most people I've spoken to that think it's great haven't actually used any other framework (which is, frankly, endemic of a large portion of the PHP-using world)

Edit: Also if you care about overhead, you're not writing what I would call a small to medium sized application! _Any_ framework out there worth its salt (and many that are not) will be able to run your application just fine, it's extremely unlikely that the PHP layer will be the bottleneck unless you're writing really terrible code or using a really terrible framework.</comment></comment></comment></comment></comment>


Migrating Kafka to dedicated Zookeeper with zero downtime - <AUTHOR> https://engineeringblog.yelp.com/2019/01/migrating-kafkas-zookeeper-with-no-downtime.html <comment><AUTHOR> Does anyone have any recommendations or learnings to share regarding managing a Kafka cluster for implementing CQRS/CDC patterns.

I was doing a PoC using Debezium ([https://debezium.io](https://debezium.io)) to implement change data capture from a PostgresSQL database but was unable to find much literature regarding the same.</comment>


Secret Ingredients: China's "Fake Food" - <AUTHOR> https://www.thebeijinger.com/blog/2016/08/08/secret-ingredients-truth-about-chinas-fake-food <comment><USER2> When people say we want a free market where government does not mess with it regulating things, they usually dream of some Ayn Randian utopia. In actuality this is what happens every time.

China in a way is more "free market" than America or Europe is. The same stuff happens in Eastern Europe and other less developed countries. So behold, the wonders of unregulated-everything. Anyone headed to the airport yet to live in this utopia? (It is only a half joke, some will actually thrive there).

Yeah sure, on paper all these places have their FDAs, EPAs, FCCs in practice they are not effective, so you do whatever you want.

One can make an argument that eventually the free market will resolve the problem, and those selling gutter oil or mystery white chemical powder as baby formula will be out of business and the true quality products will emerge. Yes, maybe, but do you want to be the failing test case who gets cancer 5 years later, just so others learn that such and such a vendor was putting benzine in their tomatoes. Are you even be able to find them 5 years later to sue them? Or even publicly shame them?

<comment><USER3> I imagine that the hyper-laissez faire libertarian sitting in the passenger seat of a crashing (unregulated) commercial airliner would fondly reflect on the fact that they are about to be part of an event that will transmit information to the market that will permit market participants to make an informed choice regarding their air travel options.

<comment><USER4> Unless the company covers it up somehow, and since they have a very clear profit motive...::crash::

</comment></comment></comment><comment> J-dawg Stories about fake food seem to be everywhere at the moment. China is the source of the most stomach-churning ones ("gutter oil" being my personal favourite - don't Google it if you're planning to eat any time soon).

But there are plenty from the USA and Europe too. Fake olive oil and truffle oil, mislabelled meat and seafood, the horse meat scandal in the UK.

If we can't rely on governments to protect us, I'm beginning to think there's a start-up opportunity for home food testing kits. Is such a thing technically possible? I guess for meat/seafood being misrepresented as a different species a DNA test would be required. But presumably there are chemical tests that can check for contaminants?

<comment><USER5> The tainted milk / baby formula incident is the one that most readily comes to my mind [0]. That and the soy sauce made from human hair.

[0] [https://en.wikipedia.org/wiki/2008_Chinese_milk_scandal](https://en.wikipedia.org/wiki/2008_Chinese_milk_scandal)

<comment><USER6> I was speaking to some chemists recently that claimed some U.S. nutritional supplements have human hair as an ingredient. They went on to mention that the hair is often sourced from China and lately has tested for higher levels of heavy metals.

I can not verify the above claim from an internet search. But does anyone know why the protein in hair might be more beneficial as a supplement than other forms of protein ?

<comment><USER7> That's a pretty common myth spread by articles like this one[0]. Wikipedia's article on Cysteine debunks it[1]:

 _The majority of L-cysteine is obtained industrially by hydrolysis of animal materials, such as poultry feathers or hog hair. Despite widespread belief otherwise, there is little evidence that human hair is used as a source material and its use is explicitly banned in the European Union._

Makes sense too, since ~50 billion chickens are slaughtered per year, the amount of poultry feathers available for processing probably dwarfs the amount of human available.

[0] [http://www.naturalnews.com/032718_L- cysteine_commercial_brea...](http://www.naturalnews.com/032718_L- cysteine_commercial_bread.html)

[1] [https://en.wikipedia.org/wiki/Cysteine](https://en.wikipedia.org/wiki/Cysteine)

<comment><USER8> > poultry feathers or hog hair

Yum! I'm not sure the truth is all that much less disgusting than the myth.

</comment></comment></comment></comment></comment><comment><USER9> Incidently, the most common cancer for males in China is lung cancer (21%), followed by stomach cancer (19%). For females, stomach cancer is (12%).

Cancer kills about 7500 each day in China.

And according to WHO, China the hardest hit by global surge in cancer

[http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3643656/](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3643656/)

[http://www.scmp.com/news/china/article/1422475/china- hardest...](http://www.scmp.com/news/china/article/1422475/china-hardest-hit- global-surge-cancer-says-who-report)

[http://www.wpro.who.int/china/mediacentre/releases/2015/2015...](http://www.wpro.who.int/china/mediacentre/releases/2015/2015020302/en/)

<comment><USER10> I live in China, and a lot of the food here is very honest vegetable dishes with an emphasis on freshness. People also eat a lot of fruit: it's largely a very good diet.

The highest stomach cancer incidence in the world is Korea (at almost double China's rate), followed by Mongolia (1.5x), then Japan (1.4x).

It seems that vinegar, chilli, salt, hard liquor and meat is not a recipe for longevity.

[http://www.wcrf.org/int/cancer-facts-figures/data- specific-c...](http://www.wcrf.org/int/cancer-facts-figures/data-specific- cancers/stomach-cancer-statistics)

<comment><USER9> correct, China is not the top one for stomach cancer. however, it is still at #5 out of all the countries. gutter oil, imitation egg, walnut stuffed with cement, industrial salt sold as table salt, plastic rice, etc, certainly doesn't help

[http://www.wcrf.org/int/cancer-facts-figures/data- specific-c...](http://www.wcrf.org/int/cancer-facts-figures/data-specific- cancers/stomach-cancer-statistics)

[http://althealthworks.com/7943/top-10-toxic-fake-food- items-...](http://althealthworks.com/7943/top-10-toxic-fake-food-items- produced-in-chinayelena/)

<comment><USER10> If you actually read the article, it says those myths are basically all bullshit.

</comment></comment></comment></comment><comment> nommm-nommm Surprised this article didn't mention gutter oil. [https://en.wikipedia.org/wiki/Gutter_oil](https://en.wikipedia.org/wiki/Gutter_oil)

</comment><comment><USER11> Made me recall this investigate report from AlJazeera where they get a chef to prepare fake noodles [1]. They compare the arduous process of producing genuine soup versus the instant recipe made possible with several banned additives which reproduce a similar taste and texture .

[1] [http://www.aljazeera.com/programmes/101east/2014/08/food- tho...](http://www.aljazeera.com/programmes/101east/2014/08/food- thought-201486125625816917.html)

</comment><comment><USER12> Slightly off topic... everyone around me knows what "Tricatel food" means, from the French film "L'aile ou la cuisse": [https://www.youtube.com/watch?v=3Lxk0kxHi80&start=4171](https://www.youtube.com/watch?v=3Lxk0kxHi80&start=4171)

</comment><comment><USER13> here's a Reddit account on finding fake rice:

[https://www.reddit.com/r/China/comments/4u06wk/i_bought_fake...](https://www.reddit.com/r/China/comments/4u06wk/i_bought_fake_rice/)

it's black rice, which if you've never had it, you might've seen it in a pot or shop-packaged cup as dessert with coconut milk

</comment><comment><USER14> > Bees in China, as in the rest of the world, have suffered a catastrophic > collapse in their numbers, due to pesticide use and land clearance.

Bees are dying at higher rates, but in the US there are actually more of them due to beekeepers working to compensate.

<comment><USER15> Are they the appropriate species?

There are more than just honey bees, and beekeepers don't often encourage the other kinds. But the other kinds are important for some major crops.

(I don't remember any more details, I saw a keynote at a conference about this.)

Edit: an overview from Wired [https://www.wired.com/2015/04/youre-worrying- wrong-bees/](https://www.wired.com/2015/04/youre-worrying-wrong-bees/)</comment></comment>


Show HN: Declarative Testing Framework for REST, GraphQL APIs - <AUTHOR> https://github.com/kiranz/just-api <comment><USER2> I wouldn't really say this is any more "codeless" than HTML, SQL, or Prolog, because you're stilling writing unit tests, but in YAML instead of an imperative language like JavaScript.

I think the word you're _really_ looking for is "declarative."

<comment><USER3> OK, we've s/codeless/declarative/'d the above.

</comment></comment><comment><USER4> We attempted to come up with a similar format for Runscope tests (YAML and all) but quickly abandoned it once we tried anything remotely complex (chained requests, control flow, limits of declarative assertions, etc.). YAML quickly becomes far less expressive than code. I still prefer using a UI like Runscope personally, but I'd use code if it weren't available.

Also, any proper GraphQL support would abstract away the payload format. It should simulate a GraphQL client, not an HTTP client.

<comment><AUTHOR> Not sure if you had a chance to go through documentation, but complex stuff like chained requests, dynamic parameter construction etc are supported. You can also provide custom validator functions in YAML to verify responses and a lot more.

Please checkout the documentation.

<comment><USER5> So it is not actually codeless.

</comment></comment></comment><comment><USER6> Neat library, I can see how this could be beneficial for black-box and end-to- end API tests. Node seems powerful here due to the event loop being able to run many parallel HTTP requests with one thread.

Were there any libraries you considered first before creating this? Or parts of an existing library you took influence from?

<comment><AUTHOR> I didn't find any open source frameworks that support testing advanced API flows like multipart forms and uploading files etc. Many companies are actually charging customers for API testing, so I wanted to build a robust open source alternative.

</comment></comment><comment><USER7> This looks like pyresttest.

</comment><comment><USER8> This is cool but I'd rather not add node and npm as dependencies to my application unless its already there. I think this would be better as a static binary or rolled into a CI platform.

<comment><AUTHOR> May be dockerize it?</comment></comment>


How much Americans make in wages - <AUTHOR> https://howmuch.net/articles/how-much-americans-make-in-wages <comment><USER5> I feel this a lot in my own family. I'm in tech at a fairly privileged BigCo. Siblings work retail gigs for a small fraction of what I bring home. I've had raises and bonuses equal to what one of my siblings makes in a year.

<comment><USER6> Out of curiosity, do you subsidize your siblings? I'm going to be in the same situation and I'm not sure what I'm going to end up doing but I think I'm going to end up doing it.

<comment><USER5> Yeah, moderately. This year I gave about $7k to each sibling which is life- changing for them and not really a hardship for me other than slightly delaying my home-ownership goal.

For one sibling, I'm the safety net when crises happen that they can't afford. E.g. can't come up with their car insurance deductible, water heater broke, etc. They don't ask but I always offer when it's clear that a small outlay would spare a huge amount of stress for everyone. Because what am I going to do, let these very minor issues send their financial life spiraling out of control? Of course not.

Other sibling is more financially responsible and trying to work through college part-time. I'm paying the car payment and car insurance while they go because I am 100% on board with this self-improvement plan. I proposed this because these expenses were keeping them from enrolling and I think it's in my own financial interest to invest in broadening the base of financial stability in my family.

Other context: My dad has a decent job (government job paying ~$80k/yr in small town America) but mom is a lunatic when it comes to money so they have nothing saved. Her idea of being able to afford something is being able to afford the payments. So far I haven't had to bail them out but only because my dad is too proud to ask and would rather go to his mother whenever there is a budget crunch.

Edit to add: I see lots of people saying things along the lines of "keep money and family separate". That is a very privileged attitude. It's very hard to enjoy money when people you love are going through hardships that would be solved by an infusion of cash. Obviously it's important not to give more than you can afford but you would be surprised by the number of people in tech who are sending money home in one way or another.

<comment><USER7> You are a good person. Money is really good to keep separate though at least for many people I think. I've done similar things like loaned my older brother enough to get through nursing school as he was stocking at Target at 33 and I thought this was a better path. I was better off then but not well off and this cause some strains. When he was doing better it was hard to get him to pay back anything even at 0% interest (I eventually gave up and made it a a gift though wasn't the original intent).

I also loaned my dad basically my life savings to bail out his company which the great recession ended bankrupting him anyway. This caused a strain with our relationship mostly because he felt and continues to feel extremely guilty about it even though I was able to rebuild my savings since then. Now I pay half his rent and try to avoid tying my finances too closely to him to try and avoid his creditors and possible future medicaid clawbacks. Money and family can be tricky.

<comment><USER8> General advice: treat any "loans" to family or close friends like gifts. Once you hand the money over, imagine it is gone forever.

Then if the loan is paid back, you will be happily surprised. But if it's not you won't be bitter about it, and it won't ruin your relationship.

Corollary: don't lend your family members an amount of money that would cause you hardship if it's not paid back. (Obviously extreme circumstances call for careful consideration of the particulars. This is not blanket advice)

YMMV.

<comment><USER9> This is great advice, I feel. I loaned my sister some money for school, and she made an honest effort to repay me, but started falling back on it after a while. So I just said "consider the rest of the loan a birthday present" and asked her to not pay anymore on it.

Because at the end of the day, I'm not going to let money destroy a relationship, and I gave her that money to relieve her stresses, not to contribute to them. So from now on, if I give money to my family, I let them pay it back according to their conscience and ability. People who need money really don't need more things to worry about.

</comment></comment></comment></comment></comment></comment><comment><USER10> >There is one important caveat to keep in mind when thinking about our dataset. The SSA numbers include any wage earners whatsoever, even part-time workers like students and teenagers.

not leading with this feels like poor reporting

<comment><USER11> They also use the term "earn" but the numbers they are throwing around are entirely after-tax. That's not even poor reporting. That's intentionally misleading. No one thinks that money paid to the government in the form for tax is unearned by the individual.

<comment><USER12> The money that it says I make in my contract has basically been a meaningless number to me most of my life other than a hard number I don't need to calculate when comparing jobs. It is only now that I have a "true" job after graduate where I make enough money to be able to set aside some money before tax.

<comment><USER11> The money your employer pays you is what you actually earn. The fact that you pay taxes on your _earnings_ does not reduce them. If your tax rate doubles next year, your earnings don't go down as a result (though your take-home will).

The IRS specifically defines earned income as what your employer pays you.

[https://www.irs.gov/credits-deductions/individuals/earned- in...](https://www.irs.gov/credits-deductions/individuals/earned-income-tax- credit/earned-income)

</comment></comment></comment></comment><comment><USER13> I honestly wonder are people _that_ delusional that they overlook the ridiculous debt that the country is in? Don't get me wrong, I am not from America myself but I have a lot of friends from there, and I generally don't hear anything good about the 'American dream'.

If you take a hard step back, like way back. And take a moment to look at the "situation" with an outside perspective, don't you think it's a little ridiculous that the debt keeps growing, and somehow the numbers get adjusted, and more loans are given.

And it makes me wonder, if this is not being actively resolved now, then when? In a 1,000 years? I have a hard time believing that society is going to survive that long with all the idiocracy that's going on.

<comment><USER14> Government debt denominated in one's own currency is not really a serious issue because governments can print their own currency to pay it off. Yes this can result in inflation, but the inflationary effects are vastly overstated. Most of the inflation happens at the point of spending.

I recommend reading up on Modern Monetary Theory (MMT)

[http://neweconomicperspectives.org/2014/01/diagrams- dollars-...](http://neweconomicperspectives.org/2014/01/diagrams-dollars- modern-money-illustrated-part-1.html)

<comment><USER15> MMT says that printing money has no effect if you give it to certain people. I've yet to hear of a real-world example where a major economy doubles or triples their monetary supply with no negative effect, which is what MMT will have you believe ("don't give it to the banks - give it to poor people in Idaho!").

If it sounds too good to be true then it is.

<comment><USER16> MMT is economics for morons

<comment><USER17> What is the less moronic theory that I should read up on instead?

</comment></comment></comment></comment></comment><comment><USER18> >48% of wage earners had net compensation less than or equal to the median wage

Isn't that how math works?

<comment><USER19> Definitely a weird thing to highlight as a "surprise" but also not quite right, since it should be exactly half... and also that portion of the pie should be a semicircle (around 180 degrees even if it's 48%)... but it's significantly smaller than that somehow?

<comment><USER20> It does point out that wage is still and predominantly the biggest factor in net compensation.

This is assuming that bonuses, commissions etc. count as compensation, but not wages.

</comment></comment></comment><comment><USER21> This is chart junk. The issue raised in the opening paragraph is a change in wages. Therefore it should be represented as a time-series chart (line, bar, etc...). This is a static view. The different categories are nestled together in a way that obscures their relationship to their neighbors. The income groups are not categorical groups they're subranges. They should be next to each other in a progression. This graphic is confusing and unhelpful.

This probably could have been done well with a steam graph. Maybe a slope graph. I'd have to see the data and it depends on what they're really trying to show.

</comment><comment><USER22> Dual ~30k incomes is still very much middle class in most of the US.

<comment><USER23> Unfortunately most of the population lives in a select few metro areas where, due to the price of rent, greatly reduces the power of 60k in joint income, married or not :(

<comment><USER17> Most?

<comment><USER23> I mean, do you want a specific number? Because if you go to the Wikipedia page for "List of US Metropolitan Statistical Areas", you'll see the following

Amarillo TX population: 265,000

NY, LA, CHI, DAL, HOU, DC, MIA: 70,000,000

<comment><USER24> 21.5% is not what I would consider "most of the population".

<comment><USER25> Well he's not considering in a lot of the other cities that have the same problem. Example: Philadelphia and its suburbs.

<comment><USER23> Additionally, there's only 144m people employed in the USA according to BLS, so 70 million is half the country :)

<comment><USER17> Except that 70 million probably contains a lot of people not in the workforce. So it's not "apples to apples".

<comment><USER23> Then assume 50-60% are working and include the addition multi-million person metropoles I left out like BOS PHI PHX etc

<comment><USER26> Assuming that 50-60% of the population of cities is working is probably too high. As a simple example, the third table at [https://www.health.ny.gov/statistics/vital_statistics/2016/t...](https://www.health.ny.gov/statistics/vital_statistics/2016/table01.htm) data for New York City for 2016. Total population is 8,537,673. Population under 18 is 1,798,888. Population over 64 is 1,155,075. That's 35% of the population right there.

Some of them might be working, of course. For the kids, this is not likely to be more than 10% (which would be _all_ the 16 and 17 year olds, pretty much). For the seniors, let's guess 33% (pretty much everyone under 70). That leaves 28% of the population in the kid-or-senior age range and not-working and 7% in the kid-or-senior age range and working.

To get to 50% overall working with those assumptions, you need 43% of the population to be prime age and working. The prime age population is 65% of the total. So you need 43/65 = 66% of the 18-65 age population working. But in 2017 (yes, I know, a year later; I haven't found the 2016 equivalent) [https://www.osc.state.ny.us/reports/economic/labor-force- tre...](https://www.osc.state.ny.us/reports/economic/labor-force-trends- nys-2017.pdf) claims a labor force participation rate of just about 60% for NYC (see page numbered 14 in the document, page 17 of the PDF). And that's the rate for 25-64, which should be higher than the 18-64 rate, because as the document notes "people between the ages of 16 and 24 are more likely to be in high school or college".

Page 15 of the document (page 18 of the PDF) does show labor force participation rates for 16-24 and 65+ in New York State and our "33%" above for seniors was likely a significant overestimate based on that, though it's hard to tell: NYC labor force participation may not match the overall state very well, obviously.

Anyway, under the (imo optimistic) assumptions above, something like 45% of the overall population of NYC is in the labor force. Or in other words if you ignore the 18% of the population that is children under 15, it's about 55%. So your estimate was pretty close if you ignore the existence of kids...

Or put another way, if the prime-age labor force participation rate is 60%, there's no way that 60% of the overall population is working. Just not possible.

<comment><USER23> I appreciate your statistics but I'm not quite sure how they are related to our discussion.

Most of the kids in NY state are living under the care and support of their parents and aren't working, therefore they shouldn't even be a factor in population/labor force participation metric.

Additionally, many of the old people who aren't working shouldn't be factored in either because they are receiving quite high pensions to live from, as well as social security.

When you factor this in, my initial ratios still stand. Out of the 325m people living in the USA, 45% are working (145m). When subtracting the 30% of Americans who are retired or under the care of parents, you are removing 100m people.

Therefore, as you say, the "prime age" labor force participation rate is 65% (145m/225m).

Applying this ratio to my original numbers, 45.5 million people are laborers in the Northeast. The other 25m either used to be laborers or are protected financially by laborers.

We cannot possibly say that an area like Amarillo TX, with just 100,000 laborers, is indicative of some nationally sustainable trend where cost of living is cheap and wages are still high.

This denies the existence of an extremely large block of humans, who are struggling despite earning decent money.

<comment><USER26> The original context, as far as I can tell, was "what fraction of workers live in the high-cost metro areas?", right?

We're trying to estimate this in all sorts of indirect ways, when the real questions, which I am sure we should be able to find answers for, are:

1) How many workers are there total?

2) How many workers are there in high-cost metro areas?

What we can't easily do is derive the answer for #2 from the total population of said metro areas; it's heavily dependent on the age structure of said metro areas and the specific workforce participation rates for those areas by age.

> Therefore, as you say, the "prime age" labor force participation rate is 65%

On average across the US, plausible.

> Applying this ratio to my original numbers, 45.5 million people are laborers > in the Northeast

Yes, but even in the northeast not everything is in an expensive metro area, fwiw.

> We cannot possibly say that an area like Amarillo TX, with just 100,000 > laborers, is indicative of some nationally sustainable trend

On its own, sure. But there are a lot more areas like that (by count; how the population totals look is not obvious to me) than there are big expensive metro areas.

That is, you might be right in your original claim that "half the country" are in the high-cost situation, but your data is not showing that very conclusively.

That said, I would be shocked if less than 1/4 of the country is in that situation. And yes, we should be aware that these tens of millions of people exist and of their situation.

<comment><USER23> Thank you so much for your well-reasoned response :) I think we've found middle-ground in trying to estimate these numbers too.

It sounds like we could ball-park the number at 33% of Americans being in high cost of living areas.

Additionally, it sounds like we agree that that percentage might change based on if you're looking at urban, suburban, or rural areas, and if you are factoring in dependents or retirees.

I think the question we ask from here is, "how do we increase the number of people who are able to have a large chunk of discretionary income coming in?"

For this the answer is probably political, contentious, and complex. Not everyone can be a programmer, as there's only 4 million computer professionals in the US right now. Not everyone can be a licensed employee either, who tend to make more money (health care, law), because labor supply is strictly regulated for these professions.

I think if we were to somehow reduce rent-seeking behavior in the economy (real estate and health insurance prices are pretty good indicators of state- enforced supply restrictions), then prices and salaries would begin to approach the mean again, which is around 50k per person.

I guess what I'm saying here is I think the roofer who makes 37k a year in Newark NJ is being underpaid, and the physician's assistant who makes 100k a year in Scottsdale AZ is being overpaid. I would wager their labor is of equal value, but is being artificially disrupted.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER27> Would be interested to know the composition of retirees subsisting on social security checks. Also, I believe this is individual income, rather than household. Wondering how much different the picture would be on a household income basis.

</comment><comment><USER28> Perhaps nitpicky, but it bothers me that their data representation is a psuedo-pie chart that isn't even centered.

<comment><USER29> Not nitpicky at all. It's a pointless, obfuscating blob of unhelpful colors violating basic principles of data visualization:

\- Sequential income brackets are not adjacent. The wage of each blob can only be understood by reading every number

\- Irregular shapes make it impossible to gauge size by eye

\- White text on green and pink backgrounds does not score well on WCAG. At least they had the decency to make the text bold.

<comment><USER30> I even think it's intentionally obfuscated. The obfuscation misleadingly suggests the arbitrarily chosen thresholds 30k and 50k are somehow meaningful. This encourages group-association-based / class-divide-based outrage-sharing, reinforcing the infighting between lower middle class and upper middle class.

<comment><USER31> Lumping together $50k-$50M as though that's a meaningful interval is especially misleading.

</comment></comment></comment></comment><comment><USER32> I used to work om many of those undertable gigs in order to get something to eat in the evening. Later I figured out more about taxable gigs and 1099-MISC here: [https://1099-misc-form.pdffiller.com/](https://1099-misc- form.pdffiller.com/) (actually a copypaste from the IRS website) and was puzzled about how some young people actually supposed to live a decent life and pay that amount of their wages to a government as well

</comment><comment><USER33> Anyone know the software package used to create this graph? Was it photoshop :)

</comment><comment><USER34> What a weird way to visualize this data. A simple histogram or CDF plot would have shown the patterns quite well. This way it's hard to follow the bands and blobs, which seem to be based on completely arbitrary cutoffs. It doesn't even seem to follow its own strange logic, jumping suddenly from just under $100K (1% per blob) to over $100K (8%) in a different part of the diagram. Why would anyone do this?

<comment><USER35> I totally agree, it's not a very clear way to show the data.

</comment></comment><comment><USER36> >There is one important caveat to keep in mind when thinking about our dataset. The SSA numbers include any wage earners whatsoever, even part-time workers like students and teenagers. This drags down the aggregate wage numbers for full-time working adults, which reach $61,372 for households last year.

So...what do the statistics look like if we remove part time teenagers and students? I suspect much less attention grabbing than the headline.

</comment><comment><USER37> Yet many parents still think that nerds are boring. Many students still bully those who focus on academic excellence. Teachers still think that students should learn easy peasy stuff because, well, "everyone is f __king unique ", or let's leave no child behind.

The end result? The people in the middle ground, aka most children, will be left behind.

</comment><comment><USER38> 30k is very low. If your company offers 6 holidays per year, that equals out to $14.71 per hour. That's below minimum wage in New York City right now.

<comment><USER39> >30k is very low.

Where you live

>$14.71 per hour. That's below minimum wage in New York City right now

Federal minimum wage is 7.25 and that is the minimum wage for many states as well, mine included. 30k is actually doing decent, after 12.5 years on the job I only make 33k with good reviews every year. If I was suddenly unemployed and had to go work retail I _might_ get lucky and hire in at 8$ an hour.

</comment></comment><comment><USER40> Well let's pile some more random data onto this tire fire and jump to some conclusions. _(edit: comparing 2015 to 2017)_

How much do Americans earn?

2015 data [1]: median _household_ income was $52K. If you're white. If you're black, the median is $34K. So, only like, 1/3 less. Or if you're asian, the median is $72K.

2017 data [2]: median _household_ income apparently jumps to $62k. If you're white it's $65k, black it's $40k, asian $81k.

Then the Social Security data (from 2013) shows the median net compensation for _individuals_ is $28K. And according to tax data, the bottom 50% of taxpayers make less than $34K.

Since the poverty level for a family of 4 (in 2017) is $25k, that means a majority of families of 4 (which, again have to pay 4x for more food, clothing, transportation, education, and miscellaneous) have a bit more than subsistence level money.

2017 data [3]: Census claims the poverty level for a household of 1 is $12k. What? Who the fuck is living in America on $12k a year?

Adding up all the people making less than $15k comes to ~28% of Americans, or 92 million people, just above to below poverty level (SSA data for 2017 [5]). But Census poverty data [6] says only 12% are in poverty. There's a huge gap between the SSA's numbers of people and their net compensation, compared to the Census's formulation of how many people are in poverty. How is there a gap of 52 million people in calculating the poverty rate? Can someone point out what's going on?

GDP is also higher than median household income, which doesn't take inflation into account.

[1] [http://www.mybudget360.com/how-much-do-americans-earn- in-201...](http://www.mybudget360.com/how-much-do-americans-earn- in-2015-household-income-wages-real-income-gdp/) [2] [https://www2.census.gov/programs- surveys/demo/tables/p60/263...](https://www2.census.gov/programs- surveys/demo/tables/p60/263/table1.xls) [3] [https://www2.census.gov/programs- surveys/cps/tables/time-ser...](https://www2.census.gov/programs- surveys/cps/tables/time-series/historical-poverty-thresholds/thresh17.xls) [4] [https://www.census.gov/library/publications/2018/demo/p60-26...](https://www.census.gov/library/publications/2018/demo/p60-263.html) [5] [https://www.ssa.gov/cgi- bin/netcomp.cgi?year=2017](https://www.ssa.gov/cgi-bin/netcomp.cgi?year=2017) [6] [https://www.census.gov/library/publications/2018/demo/p60-26...](https://www.census.gov/library/publications/2018/demo/p60-263.html)

edit: in the Census report ([https://www.census.gov/content/dam/Census/library/publicatio...](https://www.census.gov/content/dam/Census/library/publications/2018/demo/p60-263.pdf)) they mention that you have to adjust all pre-2014 numbers by increasing them by 3.14 percent, due to changes in reporting....

<comment><USER41> The median Asian income is easily explained - only educated/relatively rich Asians can afford to immigrate. Most immigration is relatively recent (60s to present) so their status hasn't really had time to change a lot.

<comment><USER42> Citation? There are certainly rich Asians who immigrate, but I know plenty of people who immigrated and are absolutely not rich at all.

Many Asians who immigrate have little to nothing except an education, some don't even have that. We brought many over during the time frame of the Vietnam war who had nothing at all.

<comment><USER43> > Many Asians who immigrate have little to nothing except an education

Are Indians included in Asians?

</comment></comment></comment></comment><comment><USER44> Anyone else having a hard time with the infograph? Why not just use the usual rectangular shape instead of circle?

</comment><comment><USER45> Would be awesome to see the same chart but broken down by what percentage of income each category represents

</comment><comment><USER46> > 48% of wager earners had net compensation less than or equal to the median > wage

I'm surprised. I expected that 50% of wage earners would make less or equal to the median wage. If for no other reason than the definition of median.

<comment><USER47> If you're going to be this pedantic, 50 is greater than 48. They're not wrong.

</comment></comment><comment><USER48> but hey capitalism, is the greatest invention since we discovered we could wipe our behinds. Problem, with America is there's a lot of embarrassed 'to- be' millionaires. Once everyone accept it's okay to be average and life a average lifestyle safe-guarded by the a social net, a lot of people will be lifted from poverty.

<comment><USER49> What's your point? Can you please elaborate?

</comment></comment><comment><USER50> stopped reading at "net". misleading title

</comment><comment><USER51> The crisis never ended, it just changed shape.

</comment><comment><USER52> Yet many talking heads insist that cheap (under the table) labor from undocumented workers has negligible effect on earnings of poorly educated Americans (rural or inner city). They still think abandoning TPP was bad, that playing tough with China is bad.

Dems used to protect the working class till the 80s, they used to be anti- globalists (till Clinton) then they got the fever for corporate money and sold them down the river and only half heartedly pay lip service to them. The only one left caring is Bernie, but he's too far left for traditional labor.

</comment><comment><USER53> It is a sad situation. But the article doesn't support the title. Because I only see one time point. And "vanishing" requires comparison.

A few decades ago, at what was arguably the peak of the US middle class, relatively few adult married women worked outside the home. But of course, they _did_ work. It's just that they didn't get paid.

Or rather, they got paid through sharing their husbands' income. What's happened since is that both partners often work, but aggregate inflation- adjusted income (aka household income) has stagnated. Maybe even declined.

Anyway, the point is that, a few decades ago, maybe 30%-40% of workers earned nothing. So to get meaningful results, you need to look at aggregate household income over time. Including both partners, and children. And you also need to net out childcare expenses.

Also, I note that TFA relies on W2 data. So what about consultants, whose income is reported on 1099s? But maybe they're a minor factor, and can be ignored.

And finally, it's my impression that executive incomes are typically capped at a $million or so. With the rest paid as stock options and such.

<comment><USER54> One problem with the aggregate household income being added up is there isn't an accounting for worked hours (basically doubling the hours by having 2 people working for a company).

A different, but simple comparison would be to take minimum wage in 1968 ($1.60), adjust for inflation ($11.62 - 2017) and then compare to our current minimum wage ($7.25). So had minimum wage been tied to inflation, it would be right around $12/hr right now.

<comment><USER53> Arguably you're not doubling the hours. Because decades ago, women worked at home, but weren't paid.

<comment><USER55> And then you should also factor in the value of the work women did at home, and weren't paid for. Because they're now having to pay someone else to do that work, which reduces the effective income gain of them working.

<comment><USER53> Yes, very true. And that makes it worse.

I do think that the US middle class is shrinking. But there's a lot more involved than TFA addresses.

</comment></comment></comment></comment></comment><comment><USER56> If $30,000 per year is "substandard" then the rest of the developed world must be positively impoverished: [https://mises.org/wire/when-it-comes-household- income-sweden...](https://mises.org/wire/when-it-comes-household-income- sweden-and-germany-rank-Kentucky). There's a very legitimate criticism to be made about the plight of the poor in the U.S. But the U.S. middle class has it better than almost everyone else in the world. They make a lot more money than in Europe, and pay far less taxes. (Indeed, the poor in the U.S. are so bad off precisely because the middle class pays far less taxes compared to their European counterparts. The difference between Trump's tax code and Sweden's is not corporate tax rates, but individual tax rates, particularly for people outside the top 1%.)

In reality, $30,000 is a pretty decent income for a young single person in Atlanta, where a 1BR apartment near a subway stop can be had well under $1,000. It's tough for someone trying to support a family of four, but by and large those people don't make $30,000. The median income for people 35+ is $50,000. And the median household income of a married couple with kids is $85,000.

There is also the fact that, even if there were zero income inequality (everybody got paid the same), the median income would be about $50,000 (13 trillion in personal income divided by about 250 million working-age people).

<comment><USER57> This is remarkably ignorant of the actual circumstances that lower-class Americans have to deal with, especially when you share almost improbable statistics that are completely out of touch with reality.

30,000 is far from a decent income considering people get gouged on health insurance, which effectively functions as another tax on their post-tax income. When you start adding in the costs of food, internet, phone, utilities etc the end result is much more dire. This is not even counting the fact that Americans work more for less, considering people in Sweden, your example, I guarantee you get far more time off, have more job security and considerably better benefits.

<comment><USER58> According to paycheckcity.com, 30K a year is a net income of $2000 a month in GA.

Rent - <USER2>

Food - <USER3>

Power/Water/Gas - <USER4>

Internet - $50

Cell - $50

These are all reasonable numbers for Atlanta.

It would be a stretch and you should probably have a room mate to give you some breathing room but it can be done. Also if you have roommates, you could afford a car and insurance.

My son is living on his own in Atlanta so I know the expenses are accurate.

We keep him on our family insurance and we can until he is 26.

<comment><USER59> "We keep him on our family insurance and we can until he is 26."

You really buried the lede. Will you also pay the health insurance for all the other hypothetical Atlanteans that you're claiming can get by on 30k?

<comment><USER58> Worse case, insurance on the exchange is $440 a month but I would assume someone making $30K has a full time job with group insurance. If not, and since it would be pretax - add $5280 a year.

<comment><USER59> I wish that were a safe assumption, but many, many--perhaps most--low-wage workers have jobs that keep them just below the full-time threshold so that the employers do not have to provide insurance.

One good way to find our about why it's so difficult to get by on ~$30k is to talk to someone (or multiple people) who make that little. You'll find that the vast majority of the time the fantasy that they blow their money on iphones and drugs evaporates, and the reality that healthcare, housing, childcare, student loan payments, etc are all much more expensive than you've estimated shines through.

<comment><USER58> $30K a year would be $15/hour full time. What single part time job that would have to pay more than 15/hr would get someone to $30K?</comment></comment></comment></comment></comment></comment></comment>


Show HN: Old School Graphics Algorithms - <AUTHOR> https://www.partow.net/programming/bitmap/index.html <comment><USER2> Lovely :) The examples remind me of the short programs for the ZX Spectrum published in magazines like Microhobby, which would create visually interesting images using PLOT, DRAW, and a couple of loops. Come to think about it, that might have been my first contact with Computer Graphics :)

<comment><USER3> My first introduction to graphics was definitely the type-in-program listed in the back of the (orange) manual I received with my 48k ZX Spectrum. It drew a UK-flag.

Here's the source, which gives me a moment of nostalgia:

[http://boriel.com/wiki/en/index.php/ZX_BASIC:Flag.bas](http://boriel.com/wiki/en/index.php/ZX_BASIC:Flag.bas)

I wrote a simple BASIC interpreter recently, in golang, and hacked in support for outputing PNG images of code. I should see if I can get that sample to run (mostly) unchanged!

<comment><USER2> Nice! Some time ago I worked on a ZX BASIC interpreter in Javascript, so it could run on browsers. My idea was that kids these days™ don't have an environment as easily available and immediately gratifying as we did back then, so I wanted to make one!

Have you considered doing something similar with yours?

<comment><USER3> I did embed it in a web-server, so that the user can enter BASIC into a <textarea> and have the output rendered by their browser as an image. I called it "visual basic" ;)

[https://github.com/skx/gobasic/#80-print-visual- basic](https://github.com/skx/gobasic/#80-print-visual-basic)

Just a small example really of embedding, I suspect to make it really useful for new-programmers it'd need a lot of overhaul.

When my child gets old enough I'll probably just give them a BBC-emulator, or similar to get started with.

</comment></comment></comment></comment><comment><USER4> The reminds me of the the days before affordable color bitmap displays. I wrote a unix program to render the mandlebrot set in greyscale. Then a VMS program to render the greyscale image into postscript.

I was in a university lab with a 8 foot long laser printer that normally spit out 200 pages per minutes or so with the first page being a giant ascii rendering of your username. It made a cool _frewp_ noise with each page shot out into a self collating set of shelves. The vast majority of the output was just ascii.

My mandelbrot set kept the printer busy for a few minutes, the operator wanted to reboot. I pleaded with them to leave it run. It finally spit out a page at at least 300 dpi, everyone in the lab came to look and was amazed. Back in those days rending a 3150x2250 part of the mandelbrot set was a substantial amount of compute. These days something like that runs at 60 fps.

</comment><comment><USER5> really cool, I usually draw my stuff on a JavaScript canvas, but I'll try to port one of my codes to c++ using this. thanks for sharing

</comment><comment><USER6> It's fun to play around with the algorithms in JavaScript. [https://codepen.io/collection/AkoojL](https://codepen.io/collection/AkoojL)

</comment><comment><USER7> I did an OpenGL tutorial when it was basically a fancy state machine. Boy how things have changed. Never found a good resource for modern graphics.

</comment><comment><USER8> Going to have a good time implementing a few of these myself. Any other references/resources for neat graphics algorithms on the more compact side?

<comment><USER9> Look for the original Graphics Gems I and II books (circa 1991)

[https://www.glassner.com/portfolio/graphics- gems/](https://www.glassner.com/portfolio/graphics-gems/)

<comment><USER8> Exactly what I was looking for you, thank you:)

</comment></comment></comment><comment><USER10> (This looks great but please don't put Show HN on reading material or tutorials. See [https://news.ycombinator.com/showhn.html](https://news.ycombinator.com/showhn.html) for the rules.)

<comment><USER11> ...but it's his library? The tutorials on the page are just examples for the library, which is linked at the top of the page?

Or rather: How would he present a "Show HN" topic about this specific library? This is the library's actual page, as I understand?

<comment><USER10> I missed that. Sorry! Title restored.</comment></comment></comment>


Losing Focus? Recharge by Giving Your Brain a Break - <AUTHOR> https://spin.atomicobject.com/2017/07/30/recharging-brain-breaks/#.WX5fZ5uRsZU.hackernews <comment><USER2> This Planet Money podcast[1] talks about unlocking creativity, including the use of Brian Eno's Oblique Strategies[2] cards

[1] [http://www.npr.org/sections/money/2016/11/18/502475485/episo...](http://www.npr.org/sections/money/2016/11/18/502475485/episode-736-messy- nobel) [2] [https://en.wikipedia.org/wiki/Oblique_Strategies](https://en.wikipedia.org/wiki/Oblique_Strategies)</comment>


Redeye VC: I Don't Know... - <AUTHOR> http://redeye.firstround.com/2008/03/i-dont-know.html <comment><USER2> I don't know; seems to be a lesson in honesty, do you know? no!? then say so ... If you gained an advantage by slanting the truth then you will almost always end up over promising and under delivering</comment>


Enterprise C# to Self Employed React and Blazor – Followup - <AUTHOR> https://www.towardssoftware.com/blog#blog30 <comment><USER2> Hey, author here. Thanks for sharing on hacker news! If anyone has questions about how to make the transition smoother and how to negotiate higher rates, don't hesitate to email or dm me!</comment>


Ask HN: How to find alternative cable provider to comcast? - <AUTHOR>

Hi HN,
I'm in a New Haven, CT apartment, and we've had lots of trouble with comcast. Is there a resource online somewhere that can find alternative ISP's?
Thanks. <comment><USER2> I don't know of any resources like that, but if I'm not mistaken Frontier serves New Haven.</comment>


Why Concatenative Programming Matters (2012) - <AUTHOR> http://evincarofautumn.blogspot.mx/2012/02/why-concatenative-programming-matters.html <comment><USER2> The elegant minimalism of Forth is inspiring-- how many other languages can fit their REPL and development environment into a few kilobytes?

However, I find implicit arity to be the largest barrier in reading concatenative programs.

To understand what a line of Forth code does, you need to understand both what each function does, and how it manipulates the stack to accomplish it. That's more memory pressure than an imperative language like C, where the flow of data and computations is obvious. It's exacerbated by the tendency to factor a single function into many simpler ones.

In a team, the increased memory burden also requires more communication for shared understanding of code.

Many concatenative languages eventually grow support for local variables, since it's _incredibly_ awkward to implement certain algorithms when you have to do stack juggling to use a variable in a computation.

<comment><USER3> This is true, but it's also important to realise that Forth is not the only direction. ;)

It's been a while since I wrote this article, and there are a number of things it doesn't address, such as the practical concerns you mention. It's much the same issue as comes up when dealing with large assembly projects, or dynamically typed ones for that matter. Knowing what's going on can be difficult to discern without a deep understanding of the overall system, and bugs have a tendency to hide in obscure places.

And speaking of assembly, Forth _is_ essentially assembly code for a virtual stack machine. Or an actual stack machine.

I've been idly working on a statically typed concatenative language for a long time now, partly because I intend it to be taken seriously sometime in the next decade, but mostly as a learning experience. It's designed to address many of these concerns—particularly, enabling programmers to focus on data flow by framing it as a functional language. (E.g.: don't shuffle stacks—use combinators and/or locals.)

However, I continually run into problems with the type system. Typing of concatenative languages is not well researched and is surprisingly difficult. I'm not a type theorist by any means, but at this point I'm probably the world expert on concatenative type systems, simply because there are so few people working on the problem. If anyone would like to help out, I'd love to chat by email (username@gmail).

<comment><USER4> Cat is concatenative and statically-typed. <http://www.cat-language.com/>

<comment><USER3> Sure is. Its type system just suffers from some limitations that I'd like to remedy in my own language.

</comment></comment></comment></comment><comment><USER5> Forth is brilliant, but my favourite concatenative language is PostScript. Most view PostScript as nothing more than a dated binary format for vector graphics. However, writing plots and diagrams in pure PS is incredibly flexible and pretty easy once you build up or find enough libraries to do what you want. Postscriptbarcode[0] is an example of a good library. [1] is a good guide to writing PS by hand.

What is PS does lack, however, is a modern debugger (or interpreter with decent error reporting) and package management.

[0] <http://code.google.com/p/postscriptbarcode/> [1] <http://www.math.ubc.ca/~cass/graphics/manual/>

<comment><USER6> I went on a PostScript kick a few years back. It's amazing what you can tell printers to do: <http://fstutoring.com/~chris/postscript/>

</comment></comment><comment><USER7> One thing I did in the past with concatenative programming that was very fun to work with, was an engine of genetic programming. Among the other features of concatenative programming languages there is that it is trivial to create a random program that is valid, and also to mix different programs together. Not that doing genetic programming with s-expressions is so hard, but it is definitely possible to create very simple and fast implementations with CP.

<comment><USER8> You might be interested in taking a look at PushGP:

<http://faculty.hampshire.edu/lspector/push.html>

</comment></comment><comment><USER9> Warning: Shameless self promotion

I implemented a concatenative DSL for Clojure called Factjor [1] inspired by Factor [2]. Clojure is to Lisp what Factor is to Forth. I used Factjor to implement DomScript [3], which is sorta like jQuery re-imagined as a concatenative language like PostScript. I also gave a talk on both Factjor and DomScript at Clojure/West. Slides are available now [4] and a video will be available on InfoQ soon.

[1] <https://github.com/brandonbloom/factjor>

[2] <http://factorcode.org/>

[3] <https://github.com/brandonbloom/domscript>

[4] [https://github.com/strangeloop/clojurewest2013/raw/master/sl...](https://github.com/strangeloop/clojurewest2013/raw/master/slides/sessions/Bloom- Concatenative_Clojure.pdf)

</comment><comment><USER10> Very important article! Preach it.

I remember discovering the Joy language (and combinatory logic) and being blown away by the elegance of being able to express programs just by composing functions. It was a pretty big epiphany to learn that when functions pop arguments from and push results onto a stack, the "application of a function to a data value" could be treated equivalently to "composition of a function with a 'constant' function that pushes a data value onto a stack". That led to the subsequent discovery that the data stack is extra "scaffolding" that can be removed: using prefix notation instead of postfix allows the program to be represented in memory as an actual composition of partially applied functions, each of which takes the remainder of the program as an argument and returns an output program. This led to the creation of Om [1], which I believe is the "most concatenative" language for this reason.

[1] <http://om-language.org>

</comment><comment><USER11> The only thing that bothers me about the concatenative languages I've seen is the stack. It is a global variable. You don't have the bounding (this is your part of the stack) that you have in languages that have an enforced frame for procedure/method calls.

Sometimes I wonder what a concatenative language would look like if operations could only access things given them by the immediately previous operation and if those things were read-only.

<comment><USER12> One thing it could look like is Backus's FP. I made a concatenative variant of it back in the 80s, and just rewrote the essentials this afternoon: <https://github.com/darius/peglet/blob/master/examples/fp.py>

In my earlier dialect I added a form of local variables, just enough for better readability without really changing the semantics.

</comment></comment><comment><USER13> _This is impossible to do with any other type of language. With concatenative programming, a parallel compiler is a plain old map-reduce!_

This statement (in context) draws a defining line around what is and isn't concatenative, and the statement by Norman Ramsey should have been refuted succinctly with a derived definition: Composable expressions with associativity.

</comment><comment><USER14> This would be much better reordered. The lead 'why concatenative programming matters' is only answered halfway through, after the reader is expected to trudge through lambda notation. Why it matters: it's the basis for interpreters of languages you use. Then explain why, then explain what it is.

<comment><USER15> For some audiences I'd agree, but I think the current presentation works well if the target audience is people who already believe that functional programming matters, but are skeptical that concatenative programming is an interesting or important concept. For that audience I think the presentation starting from the familiar type-signature notation is effective.

</comment></comment><comment><USER16> I especially liked the comparison with unix pipes. Whereas Powershell is kind of an attempt to make command line scripting OOP. This could be applied to create the next generation of unix shells...

I'll have to think about this.

<comment><USER17> Hmm, interesting. I've been thinking about a Haskell-based shell, but the flow of concatenative programming might be a better fit. Maybe I'll have to play with this idea this summer as well...

</comment></comment><comment><USER18> You meant Arrows <http://www.haskell.org/arrows/>

<comment><USER3> I might be wrong, but I think a concatenative language can't be represented using only arrows, because arrows aren't sufficient to express dynamic function application. You can think of monads as arrows with the "bind" function of a type like (a × (a → b) → b). That's precisely the type of the common concatenative "apply" combinator that lets you do anything useful. The lifting of application is what allows you to treat terms (objects) as actions (morphisms). Without it or an equivalent combinator, you can't write context- sensitive computations or manipulate concatenative quotations in any meaningful way.

</comment></comment><comment><USER19> Good stuff. I never thought that the old POP-2 ideas would become fashionable again ;)</comment>


Show HN: FaceGrab – Grab everyone's Facebook profile image without any login - <AUTHOR> http://blog.ankitaggarwal.me/FaceGrab

<comment><AUTHOR> Hi, FaceGrab lets you download anyone's Facebook full-size profile image from their username or userid without any need for login. It uses Facebook's Graph search.

This project was a code sprint. I started it when I needed an image set for a machine learning problem. I came across a Facebook graph feature which returns full-size profile images. I thought I could use this feature to programmatically generate a data-set of actual images. I consulted Facebook about the use of this feature as bug, they responded that the profile pictures are public and its not their responsibility.

So, I created this to generate a data-set of Facebook profile images. I think it could be useful for developers requiring a big image data-set for image processing or machine learning applications, or maybe more. Any comments are welcome.

<comment><USER2> What was the machine learning problem?

<comment><AUTHOR> I was taking an AI course in college, so I wanted to identify objects in an image using Neural Networks, I was moderately successful, I will try to improve in future. But, it was fun :)

</comment></comment></comment><comment><USER3> Someone really needs to create a Hot-or-Not clone using these for input, thus bringing Facebook back to its beginnings.

World without end, amen.

<comment><AUTHOR> Hehe, I didn't have that in mind when I made this, I just wanted some images to solve my problem. If you will use these for such tasks, I don't think you will be let off as easy as Mr. Zuckerberg :)

</comment></comment><comment><USER4> And if you want to scrape a list of Facebook usernames to use with this, Facebook makes that easy too: [https://www.facebook.com/directory/](https://www.facebook.com/directory/)

It's against the ToS to do so, but of course Facebook is willing to expose all that information for anyone willing to break the rules.

<comment><AUTHOR> Actually, one doesn't need to scrape facebook usernames for getting an image dataset, FaceGrab works with userids as well, so you can specify a range of userids one wants to grab profile images of (e.g. 1-10000), and voila you will get all the images saved. Be wary of your hard disk space though.

</comment></comment><comment><USER5> Is there any way of getting the profile photo from an email address instead of username?

<comment><AUTHOR> You can, but that will require additional work, first you will have to use the search feature of facebook graph api to get the userid of the email address and then finally, get the profile image from that userid using FaceGrab.

Now note, while getting the profile photo can be achieved without any login, however, if you want to get userid of a user from email address, you would require authorization token which you can get by logging in and creating a facebook app.</comment></comment>


Marionette.js and backbone, a perfect match? - <AUTHOR> http://blog.netguru.co/post/57690641427/marionette-js-and-backbone-a-perfect-match

<comment><USER2> I really like the minimalistic approach of backbone and it is what has draw me to this framework. Unfortunately the only thing I really miss from backbone is memory management. It really piss me off to have to use another framework just for that.

I currently learning it the hard way with a mobile app/framework I created. My biggest problem is managing views cleanup, beside that I find that backbone provide me with what I need to create a simple app.

The other thing is, when you start creating an app with backbone for the first time, you really don't know what lies ahead as far as memory management goes. It's really not something you expect to have to debug that at 3 weeks from launch.</comment>