Ask HN: Is it important to have accounts on SO/GH for Software Engineers? - <AUTHOR> I'm curious, is it advantage or disadvantage to have accounts on StackOverflow/GitHub for Software Engineers? Or more importantly public speaking?
For instance, I have more than the 55k rating on SO and I contribute several open source projects, some of them have more than 40k starts and relate to one of top IT company.
Last year I received a couple suggestions to work on short-term "freelance" projects from StackOverflow/GitHub sites. So, seems that StackOverflow/GitHub accounts don't give any advantages and LinkedIn works much better.
Also, I noticed that CV which includes SO/GH links doesn't increase chances to get a response (even response, and I'm not talking about job offers) from HR. I'm not located in the USA., maybe SO/GH accounts only useful for the USA, however, statics says that to be involved in open source doesn't give any advantages. <comment><USER2> It depends a lot on what your career goals are. Many startups seem to care about Github activity a lot. Corporations and less trend-seeeking small businesses less so. Though I'm sure companies exist that take StackOverflow activity into account, I've yet to encounter one.

My impression is that participation on these sites only matters if you're an outlier, and that it mostly gets you incoming inquiries rather than helping to have on a resume.

If you're the creator or major contributor to a well known open source project, that will absolutely open doors for you at certain companies that use that project.

Otherwise, participate in SO or put code on Github if it's something you enjoy doing. But don't force yourself to because you think it'll have a major career impact, because chances are it won't.</comment>


Why the doors to America are closing for Chinese tech students - <AUTHOR> https://www.inkstonenews.com/education/chinese-tech-students-us-face-visa-restrictions-amid-trade-war/article/3008298 <comment><USER2> I wonder if Chinese tech students who are unable to find jobs in sensitive sector in America will have to return to China, leading to a reverse brain drain.

The Chinese nuclear weapons program and missile program were both helped greatly when Qian Xuesen[1], one of the co-founder of the Jet Propulsion Laboratory, returned to China during the Red Scare in the 1950s.

[1] [https://en.wikipedia.org/wiki/Qian_Xuesen](https://en.wikipedia.org/wiki/Qian_Xuesen)

<comment><USER3> I think due to the US inability to fix fundamental issues with regards to upward mobility, education and a fuck you attitude on the world stage. Anything they do to counter China will only hasten the rise of China and their decline.

<comment><USER4> A vast majority of the Chinese international students I met in college were:

Already incredibly wealthy (if you think inequality is bad in the US, you should see Shanghai or other parts of China) and had no intentions of staying in the US and were just here for education. It was extremely common to see Chinese students driving around in BMWs, Mercedes, Audis, and very often nicer cars, and it became something of a stereotype with most of the people I knew. Not to mention my university was taxpayer funded.

<comment><USER5> > Not to mention my university was taxpayer funded.

Assuming it was a US university, the Chinese students were paying a special extra-high tuition for international students.

<comment><USER6> I think their point was that it was a public university. At least in my experience, you'll see less (they still exist) obviously wealthy students. It makes the wealthy international students more jarring/juxtaposed.

</comment></comment></comment></comment></comment><comment><USER7> In electronics, the success of areas like Shenzhen and the relative expense of engineering in the US has decimated non-defense industry. In 2002, there were 385,000 electrical engineers in the US. In 2014, there were only 300,000. ([https://www.computerworld.com/article/2487847/what-stem- shor...](https://www.computerworld.com/article/2487847/what-stem-shortage-- electrical-engineering-lost-35-000-jobs-last-year.html)) Why should foreign students expect to work in relatively expensive industries that are losing employees in the US?

<comment><USER8> According to the BLS there were 324,600 electrical and electronics engineers in the US in 2016.

[https://www.bls.gov/ooh/architecture-and- engineering/electri...](https://www.bls.gov/ooh/architecture-and- engineering/electrical-and-electronics-engineers.htm)

A bit of a decline but "decimated" is too strong a word for a decline from 385,000 to 324,600, and much of that is due to smartphones and tablets (and single-board computers like the Raspberry Pi) that reduced the need for custom electronics. For example: in the 90s I worked for a company that made a custom handheld device. Now that product is an app.

<comment><USER9> The rapidly increasing presence of electronics everywhere (cars, "intelligent" fridges, etc), should have caused a dramatic rise in the number of engineers, so I would say "decimated" is somewhat justified.

</comment></comment></comment><comment><USER10> A prominent Beijing scholar who recently fled to the United States has warned that China was sending "spies" to American universities, and urged US institutions to tread carefully on academic co-operation.

[https://www.scmp.com/news/china/article/1437005/expelled- pek...](https://www.scmp.com/news/china/article/1437005/expelled-peking- university-professor-warns-us-universities-over-educating)

</comment><comment><USER10> From Wikipedia: According to Robert Hannigan, former Director of the Government Communications Headquarters, Chinese hackers have engaged in economic espionage against British universities and engineering companies, on behalf of the Chinese government.

What better instrument than a graduate student?

Sure, we want to be open and inclusive to everyone, but don't be stupid. Don't let the country that literally partners up with North Korea ([https://en.wikipedia.org/wiki/China%E2%80%93North_Korea_rela...](https://en.wikipedia.org/wiki/China%E2%80%93North_Korea_relations)) and have hacked and infiltrated other countries before have open access to cutting edge technology.

During university, I've heard from a chinese student (From China), who's dad owned a high tech factory. He told me the plan was to get a job at the most reputable tech firm in that industry, and learn from it, then bring it back to his dad.

<comment><USER11> God forbid, a kid who wants to work in the family business! He must be up to something no good, being from an evil Communist state. Wait, how do family businesses exist under Communism? _head explodes_

</comment></comment><comment><USER12> If China didn't have a massive state-sponsored corporate espionage program these students would have a case. But China does, so the students are going to have to deal with the vetting required to make sure they aren't a part of it.

</comment><comment><USER13> I knew a lot of Chinese students in grad school (computer science) and saw none of this. This article smells fishy.

<comment><USER14> Its actually new. In the past year trump can basically unofficially prevent chinese student from getting visas

</comment></comment><comment><USER15> It is the same situation for Indian students, but due to visa and green card quotas and long processing delays. I'm hoping this leads to a boon for the Indian technology sector, which has already seen a resurgence over the last few years.

</comment><comment><USER16> The American Manhattan Project came about because Germany forced a lot of Jewish scientists out of their research jobs in the thirties.

Those scientists moved to the States, which was America's gain and Germany's loss when lots of Weapons Research happened during the forties.

America seems determined to lose out when Chinese scientists are barred from US jobs, or when Chinese space engineers are refused any interaction with NASA.

<comment><USER14> One they were killing jewish scientist. They were running for their lives.

Two most of these people are students and there a huge line for students to come into US universities. Unless china has a another plan to invest heavily in western free and democratic schooling they won't lose that edge.

</comment></comment><comment><USER11> I've heard that AI and semicon are going on the export control list, and even a US national communicating knowledge with someone is a "deemed export". Does this mean all Chinese H1-Bs are going to be barred from working at Microsoft, Google, Intel, and Qualcomm?

<comment><USER17> It means that AI is going to be sequestered to buildings with big unfriendly signs at the door "No Foreign Persons Beyond This Point" and data will have to be managed in ways similar to medical, financial, legal, or payment data.

Long ago I worked in defense and had some export controlled hardware (unlocked/no restriction high performance GPS and satellite technology)

What it meant was that to work on our stuff you had to be a "US Person" and "export" could be a conversation with the wrong person. In practice it meant the work couldn't be shared with students from a short list of countries. The punishment for infractions could be as little as recording and notification and as severe as significant prison terms. "Export" happens quite a lot and usually consists of innocuous mistakes.

<comment><USER11> But given the broad definition of AI and semiconductor design, however, and inclusion of other technologies like "Logistics" (seriously, the whole category, not a subfield) within emerging technologies list, wouldn't this, in practice, make whole companies no-go-zones for Chinese H1-Bs?

I can't see Intel or Applied Materials or even UPS having Chinese H1-Bs around in the future. While there are some here who wouldn't mind having a Chinese Exclusion Act in their industry, I'm hopeful a majority of voices here will see the problem inherent with moves like this.

</comment></comment></comment><comment><USER11> Qian Xuesen would like to have a word with you.

<comment><USER18> from the grave?

<comment><USER11> Har har... but seriously, look up his story. It's not hard to see other Chinese students today in his (figurative) boat.

</comment></comment></comment><comment><USER19> How many advantages does America have left if China gains a decisive edge in its research capabilities?

I count military and control of the international monetary system. That isn't a winning hand unless the Chinese shoot themselves in the foot with corruption or a return to their communist roots.

<comment><USER20> English is a huge advantage. I don't see the rest of the world clamouring to learn Mandarin any time soon. It's a benefit, sure, but not a necessity.

</comment></comment><comment><USER21> Who, exactly, doesn't understand that the US university system requires a certain number of grad students to have the research output it produces, and a great way to close that gap would be to improve secondary education in the US?

Also, I have a bit of ill will toward US universities that turns down US students in favor of Chinese students. I'm looking at you, UCSD.

<comment><USER22> There aren't many students fit for grad school (in the sciences) that are held back by poor secondary education. Obviously, some specialized programs could accelerate top kids farther, faster, but they're mostly already grad-school capable.

<comment><USER23> This is quite a tangent... But how are you sure about this? I have known many high performing kids that felt severely under-challenged in secondary school, or were even picked on for showing genuine interest in their material. Some of them made it through and turned out fine in the end, but some of them didn't. Some ended up dropping out early, or severely under-performing their potential and throwing away a shot at becoming "grad school capable". (Whether on purpose, due to self sabotage or boredom, peer pressure, or bad habits). The teachers did their best but had to focus on the even more at-risk students.

Not all high-performing students will do well regardless of their environment. Some will crash and burn without support, and that doesn't only mean "accelerating them farther faster". It can be enough to group them together with other kids that are similarly enthusiastic and curious, so that they don't shoot each other down.

<comment><USER22> The majority of high school graduates are enrolling in college. Among people in the top 1% by talent, that number is in the 90%'s, no? So the kids not getting face-time with college and grad school options is a small sliver.

Also, there don't seem to be any other secondary education systems, in other states or countries, that are doing surprisingly better.</comment></comment></comment></comment>


How Grooveshark Uses Gearman - <AUTHOR> http://wanderr.com/jay/how-grooveshark-uses-gearman/2011/03/27/

<comment><USER2> I implemented Gearman at a previous startup and then recently have done a "clean room" re-implementation of it for a personal project.

First: Gearman is foremost a job quque, as others have mentioned. Obviously most jobs have SOME data submitted along with them.

Second: After implementing it as a job queue, another team down the road needed a message queue. So we ran benchmarks. Turns out, Gearman was a faster Message Queue than the dedicated *MQ platforms. We were getting killer performance out of the Gearman Java client. Of course, we had to spend 2 weeks fixing bugs on the Java client to get it to work.

Which brings me to my last comment... Gearman is NOT a mature system. It's brilliantly fast and very good at what it does, when it all works. But you may find yourself, like I did, getting personal with the Gearman code itself to explain wonky behavior. Like, hey, why the heck isn't this job timeout being enforced? Fail.

But as for being awesome, let me give you a taste:

It has a MySQL client. So on our reporting database servers, we installed the gearman client. Which meant we could create UDFs in MySQL that would be actually backed-out by Gearman. In other words... we could do something like this:

Select user_id, UserRiskScore(user_id) from users where user_id = 123;

And UserRiskScore kicks off a Gearman job... which in this case could be just an entry point for a Gearman map-reduce: it dispatches many other jobs, aggregates the result, and then returns it to MySQL and your waiting query.

And, of course, the actual worker code can be written in languages far more useful than SQL.

Now, I'd never install a system like that on the live database. You don't need 200 queries waiting on a gearman response when you've got buckets of data innodb is trying to flush to disk.

But on our reporting db cluster it was brilliant. It made our code wonderfully centralized. A FRONT-END coder can get complex results, having kicked off a map-reduce job without ever having to know what, say, HDFS, or MongoDB, really is.

</comment><comment><USER3> We have a similar setup for our application I have to say that I agree with the article. Some of our workers work in tandem to solve small data chunks, some do async fire and forget types of jobs. All in all, gearman was a pleasure to work with.

As far as the memory job queue not being reclaimed, that might be mitigated by using a secondary storage for it ala mysql via libdrizzle.

</comment><comment><USER4> I'm curious as to why Gearman vs something else, say RabbitMQ?

<comment><USER5> Or zeromq?

<comment><AUTHOR> I don't think 0MQ existed when we made the decision to go with Gearman, but a 30-second glance tells me it would probably be well suited to build a job server on top of, but isn't exactly an out-of-the-box job server like gearman is. For one thing it's not immediately clear to me how you would submit jobs via zeromq for which workers don't exist yet or aren't currently connected. That's just a first impression though since I haven't dived into it very deeply, obviously.

<comment><USER5> I didn't mean to knock the decision about choosing Gearman. I was just interested in your thoughts on comparing the two.

That's correct that ZeroMQ isn't a server. I tend to describe it as a sockets layer that has messaging semantics. If a host isn't up, you can still send messages to the socket. It queues them up, like you'd expect a messaging system to do.

So this also means you can open a socket to a host that doesn't exist. Imagine you turn a server on with no clients up yet. A PUSH socket will block. When you connect a client, by opening a PULL socket, the message exchange starts.

These socket types can be inproc, ipc, tcp or pgm. inproc is an in process queue. ipc is basically unix sockets. tcp is... and pgm is pragmatic general multicast.

To build a broker, you'd probably use something other than PUSH/PULL sockets. You would use REQ/REP (request/reply) sockets. These sockets give you synchronous communication. The request sends a request and is guaranteed a response.

I think their doc explains the rest better than I could, I recommend reading this: <http://zguide.zeromq.org/page:all>

I think this description offers the gist of how REQ/REP, and their nonblocking counterparts XREQ/XREP, can be used to basically script together your ideal broker for many conditions.

 _When you use REQ to talk to REP you get a strictly synchronous request-reply dialog. The client sends a request, the service reads the request and sends a reply. The client then reads the reply. If either the client or the service try to do anything else (e.g. sending two requests in a row without waiting for a response) they will get an error.

But our broker has to be non-blocking. Obviously we can use zmq_poll(3) to wait for activity on either socket, but we can't use REP and REQ.

Luckily there are non-blocking versions of these two sockets, called XREQ and XREP. These "extended request/reply" sockets let you extend request-reply across intermediate nodes, such as our message queuing broker._</comment></comment></comment></comment>


Homakov on Covert Redirect OAuth exploit - <AUTHOR> http://homakov.blogspot.com.ar/2014/05/covert-redirect-faq.html

<comment><USER2> Basically the vulnerability is in the facebook side. Every oauth provider has a list of "allowed redirect uris", a good oauth provider will check the entire url, but facebook doesn't check the query string in the url. If you have a list of allowed redirects like:

\- [http://foo.com](http://foo.com) \- [http://foo.com/foo](http://foo.com/foo)

Facebook accepts redirects like: \- [http://foo.com?anything_here=xx](http://foo.com?anything_here=xx)

And if the client has an open redirect, some query string to redirect anywhere combined with response_type token.. the evil website can get the token.</comment>


Ghidra - <AUTHOR> https://www.nsa.gov/resources/everyone/ghidra/ <comment><USER2> There was a thread on HN when they first announced a public release at RSA[0]. A lot of reverse engineers I know are excited for it.

0: [https://news.ycombinator.com/item?id=18828083](https://news.ycombinator.com/item?id=18828083)

<comment><USER3> Since that was less than two months ago, the current submission counts as a dupe. When it's open-sourced, that will be significant new information, which makes for a new story. It will certainly be discussed by HN then.

[https://hn.algolia.com/?query=%22significant%20new%20informa...](https://hn.algolia.com/?query=%22significant%20new%20information%22&sort=byDate&dateRange=all&type=comment&storyText=false&prefix=false&page=0)

</comment></comment><comment><USER4> My brother is into reverse engineering, he is literally counting the days to the open source release of this.

He said there isn't anything quite like it as it can actually stand toe-to-toe with IDA Pro, the commercial software that apparently nothing yet can really beat.

<comment><USER2> Yup. IDA Pro is the gold standard, and was basically the only choice for a _long_ time. It's also stupidly expensive and priced for defense contractor funny money, with an individual license nearly $3000 [0]. Most freelancers just use a cracked copy or the freeware version.

Recently Hopper and BinaryNinja have been rising in use, with much more affordable pricing plans, but they're still second-rate as far as I know.

(There's also radare2, which for all it's mentioned occasionally I don't actually know anyone that _uses_ it)

0: [https://www.hex-rays.com/cgi-bin/quote.cgi](https://www.hex-rays.com/cgi- bin/quote.cgi)

<comment><USER5> I've always wondered this - there's a real "pay for your useful tools" philosophy on here at least, for things like Sublime Text. Often quoted is the cost per hour of use, etc. which justify the price (which is admittedly low compared to Ida).

Does the same not apply to Ida where you would make the money back fast enough to justify the pricetag? Or is it that there are more hobbyists and less people making actual money from using Ida?

<comment><USER6> This is basically how a monopoly works. And the defense industry behaves like a monopoly even if its made up of a bunch of different companies.

<comment><USER7> I disagree, I think there's an important difference between a monopoly and a company that happens to have no competition at the moment. I think that active anti-competitive actions are one of the defining characteristics of a monopoly, and so far we haven't seen that behavior from hex-rays.

They had a significant barrier to entry protecting them, in that making a decompiler (the hex rays decompiler is the expensive part of "Ida") is hard, and the market is (was?) small.

Consider the difference between Standard Oil and Adobe. Photoshop was the only game in town not because Adobe lost money on it to drive out rivals but because it's hard to make software that does what Photoshop does.

<comment><USER8> > I think there's an important difference between a monopoly and a company > that happens to have no competition at the moment. I think that active anti- > competitive actions are one of the defining characteristics of a monopoly

You are of course free to think so, but be aware that does not align with the common use of the term monopoly.

</comment></comment></comment></comment></comment></comment><comment> amiga-workbench Uhh, what's this about? [https://i.imgur.com/e3kNYTH.png](https://i.imgur.com/e3kNYTH.png)

<comment><USER9> I haven't worked at US government so I am not 100% sure of the details, but this is my understanding:

HTTPS (tls, really) allows clients to present a certificate, just like the server does. This is commonly used, eg, for microservices authenticating to each other in a backend.

It is less commonly used for people to authenticate to servers.

In particular, the "Common Access Card" is the ID badge used by the DoD, various parts of the armed forces, and in particular the NSA (whose website this is). Those access cards have a key and certificate usable for this.

So your keyboard (or laptop) has a smartcard reader in it, and you can insert your ID badge (maybe with a PIN? not sure if usa gov't does that) to log into any website.

Browser UX for this isn't great. Unlike the newer Webauthn specs where javascript (and thus site-specific instructions) can ask you to log in, the browser has to prompt you in a very generic way to present your certificate.

<comment><USER7> I actually prefer the browser UX for client cert authentication; since it's presented by the browser it's harder to do nefarious things with JavaScript to confuse the user as to what site is requesting authentication.

Edit: for what it's worth, the executive agencies are all required to use PIV (essentially civilian CAC) to authenticate, and those PIVs are required to have significant physical controls including requiring a PIN for access. It's a pretty robust way of enforcing 2fa.

</comment></comment></comment><comment><USER10> On a sidenote, looking at their Github, they publish a lot of names / emails in the git logs (~100). Some are encoded / anonymous, but most are real looking names and plausible email addresses.

</comment><comment><USER11> an open source competitor to IDA pro is very welcome, never thought it would come from NSA. have they said anything about their motivation to open-source it?

<comment><USER12> Probably to crowd source maintenance like nearly every other closed to open transition. And ghidra has been leaked a few times (they give it out like candy to contractors) so they're not really losing much.

<comment><USER13> any reports on if it's better than IDA?

<comment><USER12> I've heard that it's about equal in power, but different. Ghidra has a bigger emphasis on working with the decompiled output; you spend nearly all your time in the decompiler whereas in IDA you flip back and for decompiler (if you have it) and disassembler.

</comment></comment></comment></comment><comment><USER14> NSA releasing an open-source tool? My first thought is, better subject it to serious, in-depth security review before installing it locally. Even then, build it from source.

<comment><USER15> Apache NiFi was also released by the NSA & has seen commercial success in the enterprise.

<comment><USER16> Wans't Tor initially developed by NSA/CIA as well?

<comment><USER17> > _" The core principle of Tor, "onion routing", was developed in the > mid-1990s by United States Naval Research Laboratory employees, > mathematician Paul Syverson, and computer scientists Michael G. Reed and > David Goldschlag, with the purpose of protecting U.S. intelligence > communications online. Onion routing was further developed by DARPA in > 1997."_

[https://en.wikipedia.org/wiki/Tor_(anonymity_network)#Histor...](https://en.wikipedia.org/wiki/Tor_\(anonymity_network\)#History)

</comment></comment></comment></comment><comment><USER18> Can't wait to try it and read articles comparing its features with IDA, Binary Ninja, radare2 or Hopper. Sounds really interesting!

</comment><comment><USER19> Good to see NSA has got on board that removing the attack surface of US software is a top priority.

</comment><comment><USER20> I wonder what the NSA does with the analytics on a page like this.

</comment><comment><USER21> I'm still trying to figure out, in this day and age, especially after the Snowden disclosures, why anyone would trust software released by this organization.

you do realize their primary goal is intelligence gathering?

<comment><USER22> It's a _reverse engineering tool_. The community is going to have plenty of ability to do network analysis on it. Also, it's trivial to sandbox it, even if it weren't going to be open-sourced.

<comment><USER23> Sandboxing things is rarely trivial ;)

<comment><USER7> Air gapped RE machines (recall you're probably looking at malware anyway). One way transfer of samples. Print reports and OCR. Done.

<comment><USER20> Is printing and OCRing actually a thing? I'd think you would at least just point the camera (aka scanner) at a screen...

<comment><USER7> It depends how paranoid the security person you're trying to appease is, honestly. There are definitely better options, but that one will always "sound secure".</comment></comment></comment></comment></comment></comment>


Open Automotive Alliance - <AUTHOR> http://www.openautoalliance.net/#about <comment><USER3> Let's make sure to label this [2014], since this "alliance"* was announced nearly a year and a half ago and there's been no movement since.

Was this even covered at I/O 2015?

(* I use "alliance" in quotes because this whole thing reeks of the Open Handset Alliance, which was how Google convinced everyone that Android was a collaborative effort between major vendors and not what it really was - <USER2> field of subcontractors helping Rubin get the first phone to market. I expect just as much from this automotive effort.)

</comment><comment><USER4> "bringing the Android platform to cars"

Does this mean Google?! Would there be any other trustee choice for my car?

</comment><comment><USER5> I prefer the GENIVI Alliance, which is an actual alliance.

</comment><comment><USER6> First of all the German manufacturers should make their cars more secure. There are guys who can open X5/X6 in less than 10 seconds...

And then we can think about adding more technologies inside...

<comment><USER7> You seem to state/believe, that these guys are not able to open US or Asian cars comparatively fast?

For this claim (German cars being inferior regarding security) I would love to see facts, not trolling.

<comment><USER6> Pretty fast for such expensive car, isn't it? [https://youtu.be/U3T_g- KyYTk?t=1m26s](https://youtu.be/U3T_g-KyYTk?t=1m26s)

<comment><USER8> 4 minutes is pretty fast (but longer than "10 seconds" stated earlier). It's also completely orthogonal to the question raised.</comment></comment></comment></comment>


Better Java – Resources for Writing Modern Java - <AUTHOR> https://github.com/cxxr/better-java <comment><USER2> "If you're using Java 8, you can use the excellent new Optional type. If a value may or may not be present, wrap it in an Optional class like this"

While I agree with most things, overuse of Optional like this is an antipattern IMO. Having done a lot of Java 8 development, I find Optional is best for return types, but otherwise forcing callers to wrap values in Optional is unnecessary as opposed to something like Scala. Those that work on the JDK feel the same [1][2] and I think [2] is even overuse with the prevalence of static analysis tools and @Nullable.

Although a tad dated, I find [3] to be a good guide to modern Java all around.

1 - [http://stackoverflow.com/questions/26327957/should- java-8-ge...](http://stackoverflow.com/questions/26327957/should- java-8-getters-return-optional-type/26328555#26328555) 2 - [http://blog.joda.org/2015/08/java-se-8-optional-pragmatic- ap...](http://blog.joda.org/2015/08/java-se-8-optional-pragmatic- approach.html) 3 - [http://blog.paralleluniverse.co/2014/05/01/modern- java/](http://blog.paralleluniverse.co/2014/05/01/modern-java/)

<comment><USER3> Keep in mind that Tony Hoare, the inventor of the null pointer, calls this invention his billion dollar mistake. I generally think that he is right and use of null should almost always be avoided.

<comment><USER4> By all means, have more clumsy, less straightforward ways to represent the lack of a value in your code, because Tony Hoare says so.

<comment><USER3> The problem with NULL is that it can be there for literally any reference in your code. So, unless you are careful, you can get an NPE when calling any single object method. In languages without NULL this can never happen making them much easier to reason about.

<comment><USER4> My initial reaction: programming ain't bean bag. Them's the breaks.

Then I think of it and I see the real problem is languages making all nonprimitive types into pointers without being explicit. In a language like C it's more obvious what can be null.

<comment><USER3> If null can be problematic then certainly making it more obvious when you might have a null is a pretty good idea. I tend to think, however, that getting rid of null completely as a concept is even better. One less thing to worry about.

</comment></comment></comment></comment></comment></comment><comment><USER5> I've seen numerous posts lamenting the use of typical Java Beans.

The problem with the struct approach is that numerous tooling / libraries expects beans with typical getter/setters - Jackson Json, Spring binding, etc. Yes it's possible with configurations and/or annotations to get around this, but IMO, saving a few lines of code in exchange for non-default behavior isn't a valid trade-off.

Adding a dependency on Lombock is even worse.

Honestly, I can spot a Bean class within 1 second of opening it up. Unlike Scala or other languages, Java programmers typically use one class per file, and beans rarely get in the way as they're typically contained in some 'model' package anyway.

<comment><USER6> Another problem with struct approach is inconsistency.

Sometimes you want a computed property and you use non-trivial getter. Sometimes you want to do some work in setter. And then client's code will look terrible: point.x * point.x + point.getY() * point.getY(). There are other problems: binary incompatibility; non-trivial refactoring when I need to introduce a getter or setter.

Java really need lightweight property syntax which would solve all those problems. I don't understand why it isn't introduced yet.

<comment><USER7> The lack of simple property syntax is probably the main reason I keep using Groovy now that Java 8 has arrived. Yes, state is evil. But there are still plenty of situations where you are just schlepping data around and generating thousands of lines of boilerplate to achieve simple data transfer is such a pointless and ugly hack.

<comment> cam- For DTOs we don't user mutators. Just public member variables. It is the only way to keep business logic out of them. That being said, I wish Java has some object/struct syntactic sugar like Javascript Object Literals. They are so powerful.

</comment></comment></comment></comment><comment><USER8> I'm surprised he didn't mention AutoValue [1] or Lombok (using @Data and @Builder) in his section about data objects and builders. I prefer AutoValue, but both approaches cut down on a lot of the boilerplate (and they also generate sensible equals(), hashCode(), toString(), etc).

It's also kind of a nitpick, but I disagree with his preference for Guava's Maps.newHashMap() vs new HashMap<>(). I'm pretty sure the only reason those static methods exist is because pre-Java 7 didn't have the diamond operator.

[1] [https://github.com/google/auto/tree/master/value](https://github.com/google/auto/tree/master/value)

<comment><USER9> He mentions Lombok in the tools section.

And Guava's collection creation methods are part of an intense desire to not use the new key word anywhere in application code.

<comment><USER8> Also, if you look at the Javadocs for some of the creation methods, they explicitly state that they are for Java 6 and earlier, and that modern code should just use the constructor with the diamond syntax.

[http://docs.guava- libraries.googlecode.com/git/javadoc/com/g...](http://docs.guava- libraries.googlecode.com/git/javadoc/com/google/common/collect/Lists.html#newArrayList\(\))

</comment></comment></comment><comment><USER10> After working at two companies filled with Java devs, I don't think any of them will read this article. I can't speak for all Java devs obviously, but it seems unless the dev is a polyglot, they are just fine living in their Spring/SVN/J2EE/Java5 world. They have no idea what's going on in the software development world. I get so excited whenever I hear any of them mention Clojure or even Scala! They just live with this head in the sand mentality. They hate the front end, they think, "these JavaScript developers will go away someday and we can finally go back to writing JSPs". Currently we've updated our JVMs to 8, and that's a big deal, but we've told our devs NOT to use any features from 8 as we're worried about how GC will perform and since we're about to get geared up into shopping season, now is not the time to use shiny new features.

<comment><USER11> This is also how I feel with PHP Developers who work in development agencies or larger companies, either in a small team or by themselves (probably even larger teams, but no first hand experience with that).

Basically they refuse to use version control (FTP their code to the live server), use a terrible dev system, non-existent or terrible staging environments or newer technologies such as composer or other tools which will save them time and effort.

Don't even get me started on the lack of tests, and how buggy and broken their code ends up being - "we don't have time to write tests", yet spend weeks fixing critical issues that would have been found...

/rant

<comment><USER10> I see the lack of unit tests constantly. I did a small stint with PHP and one of the first things I did was learn PHPUnit. Having invested heavily in the TDD Koolaid with Rails/Python, I made sure I learned how to apply the same ideas in PHP. And yes, I've seen quite a bit of the "1999 was just fine" culture among PHP developers too. If you can't deploy your code at the touch of a button, you should be working your way to that goal.

</comment></comment></comment><comment><USER12> There is some good stuff in here, and a lot of bad.

SparkJava (mentioned) is the great hope for sane java web development:

[http://sparkjava.com/](http://sparkjava.com/)

Having to convert out to streams for data structure manipulation with lambdas is insane. It would have been very healthy for the core java team to pull in someone with a lot of ruby experience so they could get that API right.

Too bad: the JVM and associated tools are fantastic.

<comment><USER13> > Having to convert out to streams for data structure manipulation with > lambdas is insane

I don't see why. It's explicit and allows you to chain higher order methods without building up a ton of intermediate data structures and convert to any data structure you like very efficiently.

It's still not efficient as imperative approaches but it's close enough and far better than most collection libraries.

</comment></comment><comment><USER14> Please don't use Java 8's default methods in interface's to do multiple inheritance, and code reuse. Prefering this over 'Util' classes is bad as it is inheritance over composition. The default methods were designed to facilitate easier interface migration

<comment><USER15> Java was designed for TV set-top boxes, but that doesn't mean it's bad to use it on a server. If using a default method in an unintended way leads to clearer, more maintainable code, do it.

<comment><USER14> how would it lead to more clearer code? You would have many interfaces referencing an instance variable, each with different semantics for that variable. Interfaces cant have variables so they should not be used as traits.

<comment><USER15> One good use case I can see is where you want to add convenience "helper" methods to an interface that can reasonably have default implementations in terms of some other method on the interface (E.g. overloads to emulate default parameters). Or for methods that can be implemented in terms of another method, but might have a more performant direct implementation (e.g. Collection#get can be defined in terms of iteration).

I don't quite understand your point about instance variables. Instance variables are and should be encapsulated. An object that implements multiple interfaces must of course provide the correct semantics for all of them. But that's true whether an interface contains method implementations or not.

</comment></comment></comment></comment><comment><USER16> I'm in agreement with the first few comments already - at least some parts of this advice is dubious.

For one - some performance examples (specifically, integer array iteration) is 15 times slower with streams.

Angelika Langer's analysis:

[https://jaxenter.com/java-performance-tutorial-how-fast- are-...](https://jaxenter.com/java-performance-tutorial-how-fast-are-the- java-8-streams-118830.html)

<comment><USER17> > _For one - some performance examples (specifically, integer array iteration) > is 15 times slower with streams._

That's a very misleading example, and even the linked post itself explains why it is so (e.g. not much is done with the integers in the first place).

</comment></comment><comment><USER18> My solution is to use Groovy wherever possible. It has a superb syntax, very low learning curve and compiles into JVM bytecode. There is no issue with getters and setters in Groovy - you just define the fields, which Groovy calls properties and defines implicit getters and setters.

If not for the main application code, I would definitely use Groovy for unit testing in combination with Spock Framework. You just have to try it once to realise what you've been missing all these years with JUnit.

<comment><USER19> Groovy's always been OK for quick and dirty testing of Java classes, though the statement label hack that Spock uses makes Groovy look trashy. The static compilation that came with Groovy 2.0 is what no-one should touch if they don't want to spend countless hours and days debugging and wondering why they didn't use a language built from the ground up for static compilation, e.g. Java, Scala, Kotlin, Ceylon, and others.

</comment></comment><comment><USER20> One thing I'd add, is that Java is very amenable to static code analysis, and there are good tools like PMD and FindBugs for Java. I always encourage teams using Java to use one or both. The biggest downside is that you usually have to tweak the rule-sets to avoid getting lots of false positives. And if you get too many false positives then people tend to stop paying attention to the notifications from the tool.

On a related note, I really encourage Java teams to shoot for a goal of having compiles that finish with no warnings. If there are generic warnings littering your code (like, for example, all the warnings about unqualified references to generic types that you get when using code written before generics existed), bite the bullet and either fix them, or make it explicit that you don't care and add the appropriate annotations to turn them off. At least that way, when a new, interesting warning shows up, maybe people will pay attention to it.

And yes, this is something of a "do as I say, not as I do" thing, as I have definitely written Java code that doesn't follow this rule. Especially on my personal projects or code where I'm not working with a big team. On bigger projects, I do try to push that as much as I can, depending on my role on the project.

<comment><USER21> > [...] Java is very amenable to static code analysis [...]. The biggest > downside is that you usually have to tweak the rule-sets to avoid getting > lots of false positives.

These two statements seem to be in opposition to each other.

(All your advice is spot on. If you let warnings remain in code, you'll just end up accumulating more and more warnings until it's too much to fix. I guess it's a sort of "broken windows" effect.)

<comment><USER20> I don't know if they're in opposition exactly. The Java language is definitely amenable to static analysis to some extent, probably more so than a more dynamic language. But a lot of the "false positives" are just about what rules the developers of the tool ship by default, and what they define as a "bug" (or "possible bug"). For better or for worse, static analysis tools sometimes conflate simple stylistic issues ("don't use variable names less than two characters") with things that are likely to be actual logic errors. And since the tool will always be limited in how much it can understand about the _intent_ of the code, it's pretty much always just applying heuristics and a probabilistic analysis, even when processing Java.

So yeah, those tools aren't perfect, but my subjective experience has been that they help, as long as you tweak the knobs so that they don't just generate something equivalent to the "broken windows" compiler warnings, etc.

<comment><USER21> My point was basically that, in general, imperative programming languages (like Java) are insanely hard to do static analysis on, so I though it was ironic that you mentioned that it (Java) was both a) amenable to static analysis, and b) said that it resulted in a lot of false positives. I think that point b) somewhat invalidates point a), don't you think?

EDIT: False positives tend to indicate that there's something wrong with your analysis.

I suppose this is a general thing: If a warning system alerts you every five minutes, what are you going to do? Are you going to react and go on full- adrenaline-rush every five minutes, or are you just going to turn it off after two or three false alarms?

EDIT: Sorry, OP = you. Edited correspondingly.

</comment></comment></comment></comment><comment><USER22> >> Good alternatives to using Spring is Google and Square's Dagger library or Google's Guice. They don't use Spring's XML configuration file format, and instead they put the injection logic in annotations and in code

Spring has had code-based configurations since 2009[1]; seems a little disingenuous to recommend Guice/Dagger due to this non-existent limitation.

[1] [http://spring.io/blog/2009/12/22/configuration- simplificatio...](http://spring.io/blog/2009/12/22/configuration- simplifications-in-spring-3-0/)

<comment><USER15> Spring has a million and one ways to do anything; that's the problem with it.

Spring with a very strict, carefully enforced style guide is probably as good as Dagger or Guice. But there is a cost to maintaining and enforcing a style guide. Better to just use something that doesn't have the bad options.

<comment><USER23> That's what Spring Boot is for.

Everything is autoconfigured and set to sensible defaults. All the dependencies in the starters are levelled and guaranteed to work together.

Spring Boot on Gradle is the closest thing Java has to adding lines to a Gemfile in a Ruby system. Especially when you add other Spring automagical libraries (Spring Data REST, Spring Cloud...).

</comment></comment></comment><comment><USER24> This is a really great resource and I'll certainly be embracing some of it.

I attend quite a few university hackathons as a mentor/sponsor and one of my goals has been to paint Java in a better light as it doesn't have the best reputation in that circuit. I'm always telling people that their goal should be to build something cool and learn something new.

It's much easier to teach someone how to build their first webapp in a weekend when you can do it in a language that they are familiar with and Java seems to be what's taught at most universities.

The Spark Framework mentioned here is very well done and beginner friendly. I recently wrote an intro blog post [0] about it as an attempt to embrace existing knowledge and showcase that it's easy to build something cool with Java.

0: [https://www.twilio.com/blog/2015/09/getting-started-with- gra...](https://www.twilio.com/blog/2015/09/getting-started-with-gradle-and- the-spark-framework-3.html)

<comment><USER25> Looking at Spark framework I'm wondering about your thought of Spark vs Spring Boot. I've not used Spark but I find Spring Boot lets you get an app of the ground quickly, is part of massive Spring ecosystem which gives you a lot of other advantages (for example, Mongo template might be very useful for Hackathon project). Beyond that RestController automatic integration with Jackson can quickly let you bring up a JSON based webservice.

Also I'm not a fan of how in your example the entire app routing/logic is basically crammed into a single main method. I know it might be simple demo app that you're trying to show off Twilo but I don't think you're doing favor by structuring like that.

What do you feel are the benefits of Spark vs Spring Boot (or Dropwizard I guess). In my mind the only reason I wouldn't use Spring-Boot for a new project in Java is if I wanted to use something like Play! since supposedly using Play!/Scala and futures integrate very nicely into that.

</comment></comment><comment><USER26> I mostly agree with this list but some things I have issues with.

The biggest is DI. Having witnessed what a tangled mess large Guice codebases can become where you really have no idea what is providing what I have to say that Spring here gets a bad rap from the all-XML days and (IMHO) it is actually the cleanest and easiest to work with.

The author mischaracterizes Spring too:

> It has a either code-based wiring or XML configuration-based wiring.

Actually Spring can use a mix of XML code configuration and auto-wiring.

A modern Spring application will have precisely an application context XML file with one line per "bean" eg:

        <bean class="com.foo.SomeClass" />

and that's it. Note: there's no classpath scanning. Auto-wiring takes care of the rest. This has two big advantages (over Guice in particular):

1\. Everything is in one place. If it's not instantiated here it doesn't exist. Guice can be many levels deep and tends to descend into ModuleBuilders and conditional logic in modules (which the docs recommend against but tends to be used extensively anyway); and

2\. Testing. Testing is incredibly easy. I'm talking functional testing here, even integration testing. You simply include another XML file that overrides any definitions from the first. Everything else remains the same.

Testing with Guice can be horrendous and once you start making extensive use of Modules.override() you should abandon all hope.

I'm glad to see this post recommend Maven. It's fun to bash Maven but most detractors ignore the huge benefit that applies to any shop with lots of devs: Maven is opinionated. That's a plus not a minus. Seen one Maven Web application and you've seen them all. There's no really weird directory layout because someone decided they just had to be different.

Lastly Play. I really tried to like Play. I really did. But as of Play 2.0+ it became a Scala not a Java framework. It may still work with Java but some things just require you to know something about Scala. That's a huge negative IMHO. I'm not interested in Scala. Nobody is.

Plus the auto-compile feature of Play tended to randomly break with indecipherable error messages for no readily apparent reasons such that you couldn't tell if it was a bug or not. I get enough of that with C++ template programming thank you.

I also agree that the two books the author recommends are probably the two most important Java books in existence.

<comment><USER27> > Scala. That's a huge negative IMHO. I'm not interested in Scala. Nobody is.

Their use of Scala was the only reason I gave the play framework a second look. Just because you're not interested in Scala doesn't mean nobody else is. Scala is the most popular plugin for intellij by a very wide margin, that's a lot of 'nobody' downloads. Having used both java and Scala extensively I honestly can't understand why anyone would still pick java by choice other than if it's the only jvm language they know.

</comment></comment><comment><USER28> On the struct-like data objects part - this is so much nicer than the getter (and setter for mutable objects) pattern, just because readability is so much improved.

Getters/setters is a pattern that it took me a while to reject, having always been told that public fields were really, really bad. But they're simply not. Having logic in getters/setters is the only justification for this added bloat. In my experience, in the overwhelming majority of cases (never say all) having logic in getters and setters is a clear indication that your design is flawed anyway, so most of the time you should never have them.

On this part:

> Further, this class is immutable unless you extend it, so we can reason > about it easier as we know that it can't be changed.

Well, I'd go further. Make the class final if you really want to guard against people extending it. Of course this is most applicable for writing reusable libraries where communication with consumers may be limited.

If you rely on language features to enforce development practices in a team who can freely communicate, in general you've got a much bigger problems than the few rare bugs that extending an immutable class and making it mutable will cause.

Obviously though, putting guard rails in can only ever be helpful, even when absolutely everyone understands that there's a cliff edge there, and wouldn't conciously try to walk over it anyway.

<comment><USER29> I don't like to use getters, setters either. I like default package access so instance variables are just visible inside a package.

</comment></comment><comment><USER30> I used to be a fan of the Builder pattern (or Fluent Interface), but the ease of use for developers comes at the cost of no compile-time checking. Before, you just had to look up the constructor to see which parameters go where, with the benefit of type-checking. Now, you have to look up the Builder's methods to make sure you've filled in all required fields. The only case where I can actually advocate using Builders is when you have a mess of telescoping constructors.

<comment><USER31> If you have this problem, the library you're using is incorrectly using the Builder pattern. All required parameters should be in the constructor of the builder. Then the other methods on the builder should be optional configuration. For instance on Android you often see a builder that takes a `Context` in the constructor because that's almost always required.

<comment><USER30> This is contrary to the OP's example, where he passes data and num to the constructor, which is how I've seen it used frequently. The Builder pattern is often recommended to "make code easier to read" and wouldn't be useful if many/most of those parameters were required.

</comment></comment></comment><comment><USER32> Re the first bit about 'struct' style, even better (IMO) is to use Lombok; throw an @Data annotation on your class and all you need to do is put your properties in there (private final if need be). Nobody should be writing those boilerplate constructors, getters and setters. If it's generated by your IDE, use Lombok. The best code is no code.

<comment><USER27> Lombok is a crime. Have you seen what happens when someone does a minor reformatting of your code and moves one field declaration above another? You get a nasty little surprise because the two fields of the same type have now reordered in your Lombok generated constructor. When the app runs nobody can figure out why foo is set to bar and bar is set to foo. There's also the issue of Lombok vals which don't conform to the language spec leading to other fun. And it's almost always incompatible with new jdks without code changes to the lombok libs due to the whacky way it relies on implementation specific compiler details.

<comment><USER33> Wow, I didn't know this could happen. I've only recently started using Lombok for some side-project related stuff and this heads up comes at a really good time. Thanks!

</comment></comment></comment><comment><USER34> As a fan of immutable objects, I bemoan the builder pattern. The nice thing about constructors (or factory methods) is that you have a compile time guarantee that you have all required components to construct an object and validate its invariants. With a field oriented builder, only the documentation informs you whether you've met all of the conditions to construct an object. Also, a builder which mutates its own internal state breaks the idea of partial application where several variants can be constructed from a parent.

Unrelated to builders, the stream example could be simplified further with method references

        final List<String> filtered = list.stream()
            .filter(s -> s.startsWith("s"))
            .map(String::toUpperCase)
            .collect(Collectors.toList());

It would be nice if Oracle back filled some predicates to make them more useful in streams (e.g. "s"::isPrefixOf, etc.).

<comment><USER35> You can create immutable "builder" with lambda chains. This also gives you the compiletime guarantee that you have specified the required fields (At the cost of flexibility and ease of adding new fields)

[http://benjiweber.co.uk/blog/2014/11/02/builder-pattern- with...](http://benjiweber.co.uk/blog/2014/11/02/builder-pattern-with- java-8-lambdas/)

Regarding your isPrefixOf - is there a significant benefit to that being in the standard library? You could define a isPrefixedWith("s") that returns a Predicate<String> yourself.

<comment><USER36> This is great, thanks. I've also used something like this: [https://gist.github.com/dwijnand/3741951](https://gist.github.com/dwijnand/3741951)

</comment></comment></comment><comment><USER37> Streams are questionable in performance vs non-stream approaches. I haven't found an article that details performance differences, but I will say that I personally tried doing things using streams and doing the same things using non-streams and have had mixed results, one being faster than the other in some cases.

<comment><USER38> The benefits of streams isn't performance improvements for the CPU, but performance improvements for the developer.

<comment><USER39> i have colleges that claim streams don't necesserily make the code more readable. Imagine a triple nested for-loop - making it a triple nested streams (e.g., where your map method is yet another stream), doesn't seem to really make it any more readable.

The only reason to use streams, i think is the possibility to automatically parrelize, and secondarily to make it more readable.

<comment><USER38> I would take any triply-nested for loops, and reduce each loop to its own method. Inside that method would be nice, easy-to-understand stream logic rather than for-loops.

The Java compiler will unroll the methods anyways, so you get easy readability without any loss of performance.

</comment></comment></comment></comment><comment><USER38> 95% agree with these opinions, but have a few disagreements, or perhaps the author had missed some great stuff.

Two main ones: Mockito is, IMHO, much nicer than jMock. And Immutability is good, but 'final' is... complicated. I find it's often over used by people who don't know why they're using it.

<comment><USER15> jMock isn't great, but Mockito makes it too hard to understand which method you haven't stubbed; I find EasyMock with its explicit replay() step much easier.

<comment> cam- JMock at least blows up if you don't mock something. Powermock and Mockito will let it pass. It is a shame JMock's API is so wordy.

<comment><USER38> That's what I _hate_ about JMock, lol.

You had better mock _everything_ , and better put them in the exact order they will occur in, or your test fails.

Most JMock unit tests I see are essentially the exact same code as is being tested, written a second time in the unit test. So if I want to refactor or modify this code in any way, my unit tests break- even if functionally, my tests are doing exactly what their spec says.

With Mockito, I can say 'Hey, if anyone calls this method with parameters sort of like this, give them this response', and then focus on my unit tests actually testing the output for a given input.

</comment></comment></comment></comment><comment><USER40> Dependency Injection (frameworks) -1

Use constructors. This makes perfectly testable classes and is vastly simpler than encouraging more XML as code. Any proper dependency injection framework should work quite well with regular constructors.

<comment><USER6> Constructors have their problems. First: constructor calling is not readable because Java misses named parameters. It's fine when there are 2-3 dependencies, but not more. Second: circular dependency is not possible. So I prefer to use setter dependencies, as they don't have those problems.

<comment><USER40> I agree on the first. I'm only advocating for keeping the constructor as the DI entry point. A DI framework can then be used for the convenience they provide in wiring things up at main.

Regarding the second: I think circular dependencies should be avoided whenever possible so I view the difficulty constructors create there as advantage rather than a problem.

[https://en.wikipedia.org/wiki/Circular_dependency#Problems_o...](https://en.wikipedia.org/wiki/Circular_dependency#Problems_of_circular_dependencies)

<comment><USER35> By using a DI framework to wire up constructors you tend to use the compile- time checking that your app is wired correctly. Which is unfortunate. You only find out when you try to run it.

Perhaps things being too big to wire together manually is a smell that the application is getting rather large.

There are also other patterns that can be used to make manual dependency injection less of a pain. I wrote about some of them here

[http://benjiweber.co.uk/blog/2014/09/13/frameworkless- depend...](http://benjiweber.co.uk/blog/2014/09/13/frameworkless-dependency- injection/)

</comment></comment></comment></comment><comment><USER41> I agreed with most of these suggestions. I'm not sure the suggestion to use a struct pattern is worth the lack of JavaBeans support in argle support libraries. You can always use final properties and only provide getters.

I think asserting that Fluent interfaces are 'more readable' or better is largely a matter of opinion.

</comment><comment><USER42> Some good suggestions in here and some very very bad ones. For example, using Optional as a member type, tuples etc.

<comment><USER39> this details a good amount of the problems with optional, but yet why it will still be used despite it: [https://developer.atlassian.com/blog/2015/08/optional- broken...](https://developer.atlassian.com/blog/2015/08/optional-broken/)

</comment></comment><comment><USER43> >Java 8 has a nice stream and lambda syntax. You could write code like this:

      final List<String> filtered = list.stream()
        .filter(s -> s.startsWith("s"))
        .map(s -> s.toUpperCase())
        .collect(Collectors.toList());

>Instead of this:

      final List<String> filtered = new ArrayList<>();
      for (String str : list) {
        if (str.startsWith("s") {
          filtered.add(str.toUpperCase());
        }
      }

>This allows you to write more fluent code, which is more readable.

I don't agree. The second snipped is far more readable. This seems like an effort to use lambdas just to be using lambdas.

Not only that, but the non-lambda version is far easier to maintain. When you get exceptions in code that has the form

      this().that().theOther()

it's generally not obvious where the problem is.

<comment><USER44> A matter of personal taste, but I greatly prefer the lambda style (though I wish the example had the .stream() on the following line).

I have been coding Java (and C-style) languages for a really long time. And, I still have some cognitive delay when traversing multiple indented scopes. Not a long delay, but long enough to be slightly annoying.

With the lambda style, I can quickly glance down the text column to see what's going on. Stream -> Map -> Filter -> Collect.

After using lambdas enough, one starts to subconsciously cull the stream() and collect() calls. So, when I look at it, I really see Map -> Filter.

But what really, really annoys me is that exception handling can't be applied to the whole stream processing statement. This leads to code that looks like:

        collection
                .stream()
                .map(o -> {
                    try {
                        return o.oToC();
                    } catch(Exception e) {
                        // handle
                    }
                    return new C();
                 })
                .reduce((c1, c2) -> {
                    try {
                        return c1.combine(c2);
                    } catch(Exception e) {
                        // handle
                    }
                    return c1;
                 });

For that, I have had to write a bunch of helper functions that wrap and throw the exceptions.

</comment></comment><comment><USER45> I note that it's not safe to pass Java 8 streams around as streams can be closed when invoking terminal operations. If you pass a stream to a method and do not know whether the method is going to close the stream, it's dangerous to reuse it after the method return.

<comment><USER46> Well, once a Stream has been consumed, you can't use it again. We have a rule of thumb in my team that our APIs should not consume or return Stream or Optional.

</comment></comment><comment><USER47> I find lauding use of Optional over null for return values, but eschewing checked exceptions somewhat inconsistent. The problem with null is it can be returned anywhere, from any function. It is difficult to generate compile time errors for if they go unhandled. But unchecked exceptions are great because you don't have to then think about them. hmmm.

Addendum: I'm also of the opinion that trying to write java code without an IDE is highly inefficient. With an IDE, you can locate all the throws you're not taking care of quickly, and usually quickly deal with them to get things to compile. Yeah, it's a bit of housekeeping overhead, but once they are dealt with, they are dealt with, even if it is just passing them upstream.

</comment><comment><USER23> I've worked on two Spring Boot projects so far and written somewhere between zero and no XML.

</comment><comment><USER48> Guava is on github now. Might want to change the link.

</comment><comment><USER49> I've been using spring loaded instead of jrebel and it works pretty well and is free.

One thing I'm surprised he didn't mention is exception handling. Namely the growing acceptance that checked exceptions are dumb. I make a point that except for a few rare circumstanes, my code should never have a throws clause. I always created subclasses of RuntimeException (or Spring's NestedRuntimeException) and use them exclusively. Any libs that throw checked exceptions get either handled or rethrown with a custom RuntimeException.

</comment><comment><USER50> As a Java Dev of 15 years, through my own experience, almost all of these points I have discovered through hard fought bugs and trying to find a way to reduce the causes of them in the future.

Final by default is the biggest, bc it removes some of the most basic concurrency errors implicitly. Awesome post. It also is pointing out things that in some other languages (e.g. Python) are implicitly impossible to achieve, and others which some new languages (e.g. Rust) have built in from the beginning, like never use null.

Well done.

</comment><comment><USER51> What web frameworks are you guys using in the Java 8 world that lets you leverage newer language features?

Is it still Spring ? Are you guys doing things like Socket.io , etc.

And is Hibernate the preferred orm ? Confession: I'm an ex Java programmer, who have been working in rails and python for past several years...so am fairly out of date. I have kept up with things like Dropwizard...but have not seen many other startups do Java development.

<comment><USER23> Spring is on Java 7 for the 4.x series.

Spring 5 will take Java 8 as the baseline.

</comment></comment><comment><USER52> Nice! Seeing "enterprise" java code anyways makes my blood boil, and I keep reminding myself that it doesn't have to be that ugly.

</comment><comment><USER53> The only way it seems you can be effective with Java is with an IDE to do most of the heavy lifting. What would be better is if Java shipped with more focused command-line tools and better defaults that allow me to productive immediately without the need to learn something as hefty as IntelliJ / Eclipse. This could lessen the barrier of entry to Java and widen community adoption.

<comment><USER54> Or you could just use modern JVM languages like Clojure.

<comment><USER46> I like Clojure, but anyway to make it start faster in a scripting context? I wrote the same script in Clojure and Kotlin, and the Kotlin one loads so much faster.

<comment><USER54> There's Planck for OS X [https://github.com/mfikes/planck](https://github.com/mfikes/planck)

It uses JavaScriptCore and starts practically instantly. There's some work being done to port it to Linux/Windows as well.

Here's an example:

cat foo.cljs

#!/usr/local/bin/planck

(println "hello world: clojurescript -> planck -> as shell script")

./foo.cljs

hello world: clojurescript -> planck -> as shell script

Planck can be installed using homebrew now as well.

Another promising option is Pixie [https://github.com/pixie- lang/pixie](https://github.com/pixie-lang/pixie)

<comment><USER46> Cheers mate :)

</comment></comment></comment></comment></comment><comment><USER55> Nice compendium. I don't agree with everything, but I doubt you can get an Java devs to agree on everything.

</comment><comment><USER56> For web framework I would also suggest Dropwizard. Its simple easy to use and robust.

</comment><comment><USER57> If you really want to describe modern Java, you should probably mention RxJava and TestNG.

<comment><USER22> TestNG may have been considered a "modern" alternative years ago, but today jUnit4 has caught up with most (if not all) of testNG's features.

</comment></comment><comment><USER58> Nothing on findbugs, checkstyle, EOL and CVEs?

log4j: now log4j2

jOOQ => $, JPA, MyBatis

</comment><comment><USER59> 'Formatting is so much less important than most programmers make it out to be. Does consistency show that you care about your craft and does it help others read? Absolutely. But let's not waste a day adding spaces to if blocks so that it "matches".'

I do agree that it's less important, but it's so easy to do that there is no excuse for not doing it or giving out such rubbish advice. Anyone that wastes a day on formatting is doing it wrong. The problem with advice like this is that people take it to the extreme and ignore formatting altogether. That leads to someone being frustrated and formatting all the files automatically (as should be done anyway) which then leads to possible conflict and crappy diffs. Instead, you could have had professional programmers work on these. In an industry that's touting how code readability is so important (and it is), this advice is toxic. We're finally getting people to realize that readability is the most important part for most code because it is. Honestly, formatting is important in this inconsequential reply, in essays, in everything we write except short chat messages. Imsurenoonewouldliketoreadsentenceslikethis just like no one wants to read obfuscated, unformatted, crappy code. So yeah, I disagree strongly because formatting is important, get one style, get proper tools to automate it, and then you don't have to spend a day putting in spaces (which is ridiculous).</comment>


The Master of the Murder Castle (1943) - <AUTHOR> https://harpers.org/archive/1943/12/the-master-of-the-murder-castle/?single=1 <comment><USER2> The book "Devil in the White City" goes in-depth into both Holmes' story as well as Daniel Burnham's, comparing & contrasting two lives intertwining at the same event (World's Fair), more so contrasting since the two lives / characters could not be more different.

Highly recommend.</comment>


The Long Game of Research - <AUTHOR> https://cacm.acm.org/magazines/2019/9/238955-the-long-game-of-research/fulltext <comment><USER2> Alan Kay talks about this when reminiscing of PARC. All of the cool stuff invented at Xerox PARC (PC, Computer Graphics, GUIs, Word Processors, OOP, proto-networking, laser printers) happened in ~5 years by about ~25 people. But those people were second generation researchers shepherded through the ARPA program, and ARPA had been investing in the effort for well over a decade when it finally paid off. Led by Licklider on the ARPA side and Bob Taylor on the PARC side.

Just the laser printer got back a 250x on all the money invested into PARC. And PARC had to push Xerox to even consider productizing that!

<comment><USER3> Book: Dealers of Lightning: Xerox PARC and the Dawn of the Computer Age (1999)

</comment></comment><comment><USER4> I also remember reading somewhere that the guy for whom the Higgs-Boson is partly named, defiantly responded with "nothing" for many years when his university did their annual (or was it quarterly?) roundup asking all the professors to submit their latest published findings. Later, after winning renown for his work on Higgs-Boson he remarked that he would never have succeeded in today's impatient environment.

<comment><USER5> I don't even think Higgs was being defiant with saying "nothing", I don't think it was that unusual in UK Academia in the 70's and 80's for people not to publish for several years.

</comment></comment><comment><USER6> My mother once asked me why anybody cared about prime numbers - "they're there, but so what?" I told her that prime numbers are actually an important part of public-key cryptography. It occurred to me that prime numbers had been studied for _centuries_ before somebody figured out a way to use them to secure communications.

<comment><USER7> Math is fun. Why do people play frisbee?

<comment><USER8> If someone's experience with frisbee were being forced, day after day for many years, to throw a disc through increasingly arcane set of hoops, and _God forbid_ they try to throw it to another person, then you'd have a lot of work first explaining to that person how frisbee can be fun.

</comment></comment></comment><comment><USER9> Must read: the structure of scientific revolution [https://en.m.wikipedia.org/wiki/The_Structure_of_Scientific_...](https://en.m.wikipedia.org/wiki/The_Structure_of_Scientific_Revolutions)

tl;dr most of the time science is incremental innovation. and big discovery comes as paradigm shifts, usually first started by a group of "non-mainstream" researchers at the time.

<comment><USER10> I've run across mention somewhere recently that Kuhn's work has been somewhat deprecated. Unfortunately I don't recall where I saw this, or in what ways it may have been.

If this rings a bell with anyone, I'd appreciate a pointer or link.

<comment><USER11> Kuhn himself disowned much of his early views, refining his original intuitions in "The Structure". There's an essay on [1] in which he tries to offer a novel interpretation.

[2]: [[https://www.amazon.com/Road-since-Structure-Philosophical- Au...](https://www.amazon.com/Road-since-Structure-Philosophical- Autobiographical/dp/0226457990\]\(https://www.amazon.com/Road-since-Structure- Philosophical-Autobiographical/dp/0226457990\))

<comment><USER12> Working link:

[https://www.amazon.com/Road-since-Structure-Philosophical- Au...](https://www.amazon.com/Road-since-Structure-Philosophical- Autobiographical/dp/0226457990/ref=dp_ob_title_bk)

<comment><USER11> Thanks! Botched that one...

</comment></comment></comment></comment></comment><comment><USER10> Mosche Vardi's essay is provocative, but still leaves a few too many points vague and undefined for my taste.

The nature, questions, and value of research are one element of a larger set of questions I've been exploring.

I've recently run across David Hounshell's work on R&D at DuPont: _Science and Corporate Strategy: Du Pont R &D 1902-1980_, originally published in the 1980s.

[https://www.worldcat.org/title/science-and-corporate- strateg...](https://www.worldcat.org/title/science-and-corporate-strategy-du- pont-rd-1902-1980/oclc/1086231462)

For a ... short-ish ... synopsis, covering main themes, see Hounshell's article: "Measuring the Return on Investment in R&D: Voices from the Past, Visions of the Future"

[https://www.ncbi.nlm.nih.gov/books/NBK45334/?report=classic](https://www.ncbi.nlm.nih.gov/books/NBK45334/?report=classic)

Upshot: R&D is a risk-based activity, and it can have high payoffs, but also high costs. Much of Hounshell's book is about management's attempts to both organise its research and research teams for optimal results, and to manage the costs and risks associated with it. There was a period in which the labs were phenomenally productive, including a roughly decade-long span from the 1920s through 1930s where virtually the entire modern plastics industry ( _Graduate_ fans take note) was invented. But also flops (a fake leather product produced in the 1960s notably).

This fits in with notions I've been noodling at of risks in economics -- an element of virtually all economic activities, but with distinctly different characteristics and scope. R&D tends more toward the dice-rolling variety (you may win), whereas some activities gamble with catastrophic and systemic loss. Some fields appear to be little but raw risk balancing (much of finance, real estate, and of course, insurance) whilst others provide a greater opportunity to directly influence, reduce, adopt, or mitigate risks (engineering, generally).

There's also the risk, or uncertainty, involved in attempting new activities. The recent HN submission "The Drugs Won't Work" [https://medium.com/@belledejour_uk/the-drugs-won-t- work-659c...](https://medium.com/@belledejour_uk/the-drugs-won-t- work-659c6d7a4ac1) ([https://news.ycombinator.com/item?id=20789000](https://news.ycombinator.com/item?id=20789000)) talks of the problems of new drugs discovery, effectively a search through a 10^200 node space for potential beneficial compounds, with very few heuristics in reducing or guiding the search (Lipinski's Rule of Five being one: [https://en.wikipedia.org/wiki/Lipinski%27s_rule_of_five](https://en.wikipedia.org/wiki/Lipinski%27s_rule_of_five)).

 _All_ research (and much coding) is effectively a search though a large possible solution space, subject to constraints and domain characteristics, for possible useful elements or combinations. The problem of huge search spaces means that alternatives to brute-force exploration are quickly necessary.

Back to Du Pont: the chemical search space is also large (pharmacology is effectively a branch of that), and low-hanging fruit was found early with small, simple, and readily-attained compounds: the law of diminishing returns. Worse was the discovery, often much later, that along with useful features came harmful ones -- the law of unintended consequences being another suprisingly general principles that arose from a specific discipline (sociology) but is applicable in nearly all others.

As for management: by Hounshell's account, management largely turned its researchers loose and told them to have fun.

There's the further challenge that the discovery of keys rarely coincides with the knowledge of the lock in which it fits. The phrase "a solution in search of a problem" was first applied (by Theodore Maiman: [https://en.wikipedia.org/wiki/Theodore_Maiman](https://en.wikipedia.org/wiki/Theodore_Maiman)) to a technology of which there are almost certainly numerous instances of in your immediate vicinity: lasers.

Early applications were thought to be in the application and delivery of _power_ (Arthur C. Clarke makes a passing reference in _2001 A Space Odyssey_ to blasting the Lunar monolith with one), but it was the characteristic of lasers as a tightly coherent transmissive blank slate onto which information could be encoded which proved its greatest use.

(This makes me suspect that materials such as graphene -- a uniform, monoatomic plane, might have its greatest application as information storage medium rather than physical materials, much as doped silicon has proved useful in semiconductors.)

Another part of my general study has been in thinking how technology works by considering what _technological mechanisms_ fundamentally exist, and how those interact. My list presently numbers nine (materials, fuels, power transmission & transformation, process knowledge, structural knowledge, networks, systems, information, hygiene -- fuller descriptions elsewhere, still in process), which may or may not prove ultimately accurate. But has been somewhat useful in thinking through numerous problems and applications. Each mechanism has specific characteristics and limitations.

Upshot: I _don 't_ think "technology" is a bottomless bag of tricks. It's definitely useful, and well get more from it, but it also has costs, including complexity (itself a form of risk, which is to say, of debt), but there are bounds. R&D is exploring a space _with_ constraints, in which both benefits and risks may be found. Treating the matter probabalistically and as one to be approached in on a risk-based approach may prove generally useful.

</comment><comment><USER13> TLDR; some research requires couple of decades to bear the fruit but people in power, especially in industrial labs don't have much patience.

I don't see any new insights. This is very well known fact. Typically industrial labs starts with lot of fan fare about long term view, big bets, blue sky research, don't worry about business impact etc. Eventually after few years or decades, people starts counting money spent vs money earned. Then suddenly you have research evaluation meetings where you have to justify everything in terms of business impact. I think this is the life cycle of industrial labs.

<comment><USER14> I wonder what the future is going to be like for research within the next few decades?

Since working in industry for the past five years largely in research environments, I've noticed a profound shift away from research labs run by companies like IBM, HP, Intel, and Sun/Oracle that focused on medium-term research ideas and emphasized publications and prototypes, to a hybrid model of research and product development, pioneered by Google (see "Google's Hybrid Approach to Research" [[https://static.googleusercontent.com/media/research.google.c...](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/38149.pdf\])) and has since become widespread among Silicon Valley's largest and most successful companies. This hybrid research model is more short-term driven compared to the medium-term approaches of the labs of the 1990s and 2000s, and especially compared to the long-term visions of the legendary Bell Labs and Xerox PARC of the 1970s and 1980s. I have no doubt that this hybrid research model has been successful for companies like Google and Facebook from a business standpoint.

However, there is still a need for medium-term and long-term research, for the sake of our field and also for the sake of providing future inventions that industry can sell. There is also still a need for theoretical and explorative types of research that may not immediately have commercial applications. Academia would appear to be a great fit for pursuing medium- and long-term research, but the realities of fund-raising and competing for tenure in academia makes it difficult for academics to pursue risky medium- and long- term projects.

I've been thinking a lot about this regarding my career. I'd love to work on medium- and long-term projects and aim for expanding scientific knowledge, but industry is increasingly focused on short-term gains, and academia is very competitive, from trying to obtain a tenure-track assistant professorship to trying to earn tenure.

<comment><USER5> There is a big problem of feedback with respect to longer term ideas and technology development. In my experience folks at the coal-face (doing delivery, operating the machines) have _vital_ information that would save a huge amount of time in research efforts - not bothering to even investigate certain approaches for example. Sadly this information is often discovered post-hoc! I think that this is what hybrid research taps into, and is the right way to tackle things that will be delivered to market in 5 years or less.

Interestingly there is very little effort in research in general in creating and discovering that qualifying information because there isn't much short or medium term reward in ruling things out - creating negative results.

Academia is now very remote from industry in EU/UK comp sci at least, there are entire communities of work that appear and run and from almost the beginning are known to be very unlikely to create any real or practical result. The poster child for this is the Human Brain Project, but it was apparent that Semantic Web was essentially an open question in computer science very early on, and yet it was sold to the commission and many other funding bodies as incremental technology development. There are other big examples. The problem is that no one is interested in derailing the gravy train and the funding agencies seem to either be under political influence or not to have any critical facilities or institutional memory.</comment></comment></comment>


Box, an advanced Docker builder, v0.5.0 - <AUTHOR> https://github.com/erikh/box/releases/tag/v0.5.0 <comment><AUTHOR> Box is an advanced builder for Docker powered by mruby with many features for manipulating layers and powerful building primitives. box is a single static binary (or container for those of you on OS X).

Docs are here: [https://erikh.github.io/box/](https://erikh.github.io/box/).

This is my first submission to HN so please be kind. :) Been working on this for ~6 months now. I hope to make this a compelling alternative to `docker build`, especially for upcoming OCI needs. Suggestions for improvement are very welcome!

</comment><comment><USER2> Great work Erik. Going to dig in a bit more soon. Best of luck with your first HN post. =)</comment>


Victoria government issues $1700 fines to Uber ride-sharing drivers - <AUTHOR> http://www.smh.com.au/digital-life/smartphone-apps/victoria-government-issues-1700-fines-to-uber-ridesharing-drivers-as-media-gaffe-surfaces-20140508-zr6yp.html

<comment><USER2> As Nelson Muntz would say: ha ha!</comment>