EarthBound's Copy Protection - <AUTHOR> http://earthboundcentral.com/2011/05/earthbounds-copy-protection/ <comment><USER2> If you thought this was interesting, there's a lot more in this vein from the author (Clyde Mandelin aka Mato/Tomato) who is a professional translator as well as a ROM hacker (not to mention, his fame in internet Earthbound/Mother fandom). He has a great blog, Legends of Localization[1], where he talks about how games were localized into English, and streams interesting game hacks and unusual games from Japan on Twitch (usually with his wife, Poe)[2]. For instance, for the past few days they've been playing through an English translation of BS Zelda[3], a 16-bit version of The Legend of Zelda released on Nintendo's Satellaview service for the Super Famicom, which is long gone but the game has been put back together by enterprising programmers.

[1] [http://legendsoflocalization.com/](http://legendsoflocalization.com/)

[2] [http://poematocx.com/](http://poematocx.com/)

[3] [http://en.wikipedia.org/wiki/Satellaview_games_from_The_Lege...](http://en.wikipedia.org/wiki/Satellaview_games_from_The_Legend_of_Zelda_series)

</comment><comment><USER3> This is great.

Reminds me of Game Dev Tycoon's developers releasing a cracked version of the game onto torrent sites where you'd always lose due to players pirating your game [0]. When I heard what they did I sort of thought it was stupid, but the chart at the end where they show that 90+% of the users of the game were playing the pirated version is somewhat chilling. Kind of makes me want to release a version of games I make to pirate sites (well, if I ever went indie... this would be a very hard sell anywhere I've ever worked), just to get similar usage information, although it's probably a mistake, as the information would only bother me.

[0]: [http://www.greenheartgames.com/2013/04/29/what-happens- when-...](http://www.greenheartgames.com/2013/04/29/what-happens-when-pirates- play-a-game-development-simulator-and-then-go-bankrupt-because-of-piracy/)

<comment><USER4> If 90% of players pirate and if your statistics for how many players there are for a given game usually come in the form of "number of sold copies", then you could be severely underestimating the size of the potential userbase; feeling that you are getting a small piece of a small pie, while in fact you're getting a small piece of a much larger pie.

Statistics on pirate behaviour are sort of hard to come by anyway -- if you buy games at all, you're not really a pirate, you're someone who downloaded a game they didn't know if they'd like, or you didn't have money at that moment and wanted to check it out, or any other explanation. Over the years, me and my friends, have basically bought every game we've played. On the other hand, if you habitually pirate and never buy, game developers aren't interested in talking to you; in fact you're basically a criminal in their eyes, stealing their profits. I really wish to see some sort of study disentangling these two groups and finding a way to talk to and understand the latter group. Why they do what they do, what motivates them and so on. Once you know that, you can start addressing the issues that drive such behaviour.

<comment><USER5> You are wondering what is the explanation for people taking readily available free (to them) stuff they enjoy? This is not a deep mystery of the Universe.

</comment></comment></comment><comment><USER6> Spyro YOTD (PS1) also had interesting copy-protection: [http://www.gamasutra.com/view/feature/131439/keeping_the_pir...](http://www.gamasutra.com/view/feature/131439/keeping_the_pirates_at_bay.php)

Both that, and the Earthbound article, have been posted on HN before. Someone might have the links.

</comment><comment><USER7> It doesn't seem like copy protection that makes the game un-fun to play is really a good idea. It seems more likely to backfire as it might generate bad word of mouth for your game.

"Damn, that game sucks. Glad I didn't buy it. Better warn my friends"

IIRC when we made Gunship for the Commodore 64 it had something long these lines in that if the loader was tampered with the helicopter wouldn't get enough power to take off. Whether or not that was a win for anti-piracy I have no idea. I do know the game was completely pirated within 2 weeks of shipping though even though we had gotten some fancy copy protection. At least the copy protection added a fast loader so it was a plus for the user :p

<comment><USER8> Red Alert 2's "all your units explode" copy-protection was incredibly stupid. It would let you type in something like a 30-digit CD-key, and then not prompt if you got it wrong and trigger the "exploding units" code in the game.

</comment></comment><comment><USER9> Another interesting thing about the development of Earthbound was that one of the main programmers of Earthbound was Satoru Iwata, who is now the current CEO of Nintendo.

An Earthbound fan translated an old Famitsu interview with him and another programmer about developing the game and some of the challenges involved.

[https://yomuka.wordpress.com/2011/12/24/weekly-famitsu- septe...](https://yomuka.wordpress.com/2011/12/24/weekly-famitsu- september-9-1994-2/)

</comment><comment><USER10> I feel compelled to post how this was the defining game of my youth. It was the ultimate video game experience in my child brain. I was enamored for years. I wanted to be Ness. My very first website was a GeoCities page, an ode to this game titled "Earthbound: The Best Game Ever." I have scoured GeoCities archives for a copy of it but unsuccessful each time. There was a neat attempt at an Earthbound MMO [http://www.skyon.be/node](http://www.skyon.be/node) but it appears to be discontinued. I love you Earthbound.

<comment><USER11> This summer a fan-made Earthbound (well, Mother) game called Mother 4 will be released for free.

[http://www.mother4game.com](http://www.mother4game.com)

<comment><USER10> that just sent me down a rabbit hole. came back with this: [https://www.youtube.com/watch?v=-hVuv7mDNqM&feature=share](https://www.youtube.com/watch?v=-hVuv7mDNqM&feature=share)

</comment></comment></comment><comment><USER12> You know what... this is really interesting to learn about this now, because I played the hell out of Earthbound when I was younger using an emulator, but to this day I don't recall ever beating it. And it is extremely rare that I would just stop playing a game. The only game I started and haven't completed is Oblivion.

I can't for the life of me remember completing Earthbound or when/why I stopped playing it. I just watched that Youtube clip of the end battle with Giygas and I vaguely remember it, but I have no recollection of some kind of ending. If anything I remember being kind of confused. I bet I ran into that last layer of protection where it randomly freezes and deletes the saved game, and that's why I don't recall an actual ending.

Learning about this now, I guess I feel a little better about not remembering ever beating Earthbound.

<comment><USER13> Eh, as the article points out it's pretty unlikely you encountered that last layer of copy protection from a ROM.

<comment><USER12> Yes, I read that. But it's also most logical explanation for why I don't remember the ending.

</comment></comment></comment><comment><USER14> Method #3 (increasing enemy spawn rate) sounds iffy. It's a subtle change that just makes the game worse, and the player isn't going to know it was because of piracy. They'll just think the game was designed poorly and be less inclined to support the developer.

<comment><USER15> I tend to agree - except there are _way_ more enemies. It's not exactly a subtle change.

Also keep in mind this was 1994. If someone ran into this, it's not like they could go complain on Twitter or Reddit where millions of people would see it and it could potentially damage Nintendo's reputation because people think the game is buggy.

Plus, Super Nintendo piracy was a lot more difficult than modern PC piracy, and it was a lot less common. It's not like today where some people will download a game just to try it out and then buy it if they like it.

<comment><USER8> Yes but here is how your word of mouth for that goes:

A: should I buy this game? B: I played it and it's just stupidly balanced. A: Ok I guess no then.

Notice how in no part of the exchange is anyone going to be communicated any pertinent information about the context, nor do they have incentive to do so.

</comment></comment></comment><comment><USER16> You cannot grasp the true form of copy-protection's attack!

<comment><USER17> The fact that this is horribly downvoted demonstrates how few people have actually played Earthbound.

EDIT: Hurrah! Parent comment is positive again!

<comment><USER15> Or people who get the joke are downvoting it because they like serious discussion and don't want this place to eventually be filled with cheap one- line jokes like Reddit.

</comment></comment></comment><comment><USER18> I really think this old attitude to copy protection is short sighted and why free games are now the top revenue earners on every platform.

If a player is enjoying your game, great! The hardest part of the battle is already won, you made a great game and got it in front of the customers. Now you just need to get them to spend some money.

Sure, back in these days is was probably harder to just ask a user for a credit card number, but punishing people that enjoy your game is crazy.

</comment><comment><USER19> It seems like this would punish unwitting buyers of the pirated carts, not the pirates.</comment>


Can a machine surprise you? We believe so. (SoundCloud) - <AUTHOR> https://developer.soundcloud.com/blog/tracks <comment><USER2> I'm actually finding some pretty good new tracks, I wish I had not to subscribe to get the full Go catalog though!

</comment><comment><USER3> Finally, this was long waited, and it works really well</comment>


Custom domains on GitHub Pages are gaining support for HTTPS - <AUTHOR> https://blog.github.com/2018-10-31-2018-05-01-github-pages-custom-domains-https/ <comment><USER2> The correct URL is: [https://blog.github.com/2018-05-01-github-pages-custom- domai...](https://blog.github.com/2018-05-01-github-pages-custom-domains- https/)

It's a repost of: [https://news.ycombinator.com/item?id=16969111](https://news.ycombinator.com/item?id=16969111)</comment>


Software will not be patentable in New Zealand - <AUTHOR> http://www.nzherald.co.nz/business/news/article.cfm?c_id=3&objectid=10882569

<comment><USER2> "Commerce Minister Craig _Foss_ said today that following industry consultation he had decided to remove the patentability of software from the Patents Bill, which is currently before Parliament."

If anyone is going to support Free and Open Source Software, it's going to be a guy named Foss.

The government asked its citizens their thoughts "a recent poll of more than 1000 Kiwi IT professionals found 94 per cent wanted to see software patents gone." -- As overwhelming as 94% is, I wonder the motivations of the last 6%.

The U.S. is in software patents so deep, the only way I can see it changing is watching other countries move faster without the drain on innovation and the need to pay lawyers when you code.

<comment><USER3> This sounds suspiciously like a government that works for its people.

<comment><USER4> [partisan alert] I wouldn't go too far on that. If you're poor and unemployed here you probably don't feel like the govt. are working for you all that much.

<comment><USER5> I'm continuously amazed by all of the major political parties here in New Zealand who keep using logic and facts instead of merely political stances. Our process of making laws is open, listens to the people as well as lobbyists, and yet allows room for policy differences.

<comment><USER6> NZ makes some great laws (this is one of them) but it makes a few dodgy ones as well - like trying to give Chimpanzees human rights.

Sorry I brought that one up but is one of those laws that has always just made me say - "What the fuuuuuc....."

<comment><USER7> Sure, we all know since Descartes that animals are robot-like objects, have no feelings and no emotion and should be treated as such. So, let's make sure the Law does not take in account any of the discoveries related to emotion and pain in animals, and let's keep business as usual and feel good about it.

Sums up your position ?

<comment><USER6> Wow, over-reaction, much? Look, I could go on and make several well reasoned arguments about how animals are not human but clearly it would be wasted on you.

Just a quick question, but once you have given Chimps human rights and said Chimps go on and kill and eat other apes/monkeys, as they commonly do in the wild - would you have them up for murder and cannibalism charges with appropriate punishments?

<comment><USER7> OK, maybe I took your comment the wrong way. Sorry about that. The way you wrote your comment gave me the impression that you were saying animals are not worthy of compassion, but if that's not the case then it's all good.

<comment><USER8> Compassion...sure. Human rights? No.

<comment><USER9> Why?

<comment><USER10> I would argue that if we are to give chimpanzees human rights then we may as well start enforcing building codes for beavers and ants. What is unnatural (law) does not necessarily work for those the animal kingdom. It should not be forgotten that animals have a certain level of rights to coexist with us on Earth but at the same time it does a disservice to humanity to treat them as our equals.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> It would be very cool if a major software house set up or moved their development shop to NZ so as to be free to develop and test their products in a patent-free environment. Sort of like we have tax havens ... software development havens?

It would be even cooler if a few different countries did this and set up a sort of, software patent-free economic zone.

<comment><USER12> The problem is that any company that sells to people in the US can have their product banned from import if it is found to infringe on patents there.

<comment><USER11> It could create a whole new architecture for software - just the patented parts hosted remotely in software-patent-free zones while the rest is brought local to the customer. At least for the types of products where that works.

<comment><USER13> > _while the rest is brought local to the customer_ //

Sounds like contributory infringement to me. Why not just host the entire thing distally and access via VPN?

The answer lies in where a product is considered to be sold. If you're targeting customers in the USA say then it doesn't matter where your server is you're still considered to be selling in the USA.

YMMV and I'm afraid I don't have any caselaw handy to cite on this.

</comment></comment></comment></comment><comment><USER14> While software that violates other countries' patents cannot be sold to those countries, the sale of software is not the only use of software. The visual effects industry for example can sell their visual effects, and this is a major part of the NZ software ecosystem.

As a kiwi in Europe I can say that this will affect any future decision on where I'll base my business.

That is until the US comes in with their bullying tactics and buy our PM with 5 minutes of facetime with Obama.

</comment><comment><USER15> How can anyone support legalized extortion like software patents? Hopefully this will change in global scale, when new decision makers will replace the old farts who don't have a clue what software development actually is about.

Respect for New Zealand for embracing the change!

</comment><comment><USER16> Wow, that increments the possibility of finding weapons of mass destruction there to the roof!

<comment><USER17> Or big-time torrentors ; )

Oh wait...

</comment></comment><comment><USER18> What does this mean for software companies in New Zealand? If they sell software in the US (and who doesn't, with the internet and all), I assume they're still on the hook for US patent law?

<comment><USER19> No. Technically NZ company can not sell anything outside NZ. And company only operates in jurisdiction of his origin country. Technically "selling in the US" means that NZ company is exporting (and this is regulated by NZ laws) and those who buy that software are importing (which is regulated by US laws) said software into US and US patent holder(s) can try to prevent that import. But even then this means nothing to the NZ company in question. On the other hand if said NZ company has daughter company in US, then yes, that company can be sued.

<comment><USER20> xero.com the beautiful accounting software comes to mind, I wonder what this means for them.

</comment></comment></comment><comment><USER21> The UK and Europe have similar restrictions: [http://en.wikipedia.org/wiki/Software_patents_under_United_K...](http://en.wikipedia.org/wiki/Software_patents_under_United_Kingdom_patent_law)

<comment><USER22> Thats misleading, since software patents are regularly granted anyway. From the article you quoted:

"Patent laws in the UK and throughout Europe specify a non-exhaustive list of excluded things that are not regarded as inventions to the extent that a patent application relates to the excluded thing as such. This list includes programs for computers.

Despite this, the United Kingdom Intellectual Property Office (UKIPO) regularly grants patents to inventions that are partly or wholly implemented in software"

The patent situation is definitely better in UK and Europe than in US, but actual enforcement is key. The problem is using ambiguity of English to justify the software as having a "technical effect" and patentable by that metric. At which point saying that software isn't patentable is at best misleading since you could argue that all software has a "technical effect".

<comment><USER23> It's not "ambiguity of English" -- all (usefulsoftware does have a technical effect. Or would you say software should be excluded from the "tech industry" and people should stop calling themselves "software engineers"? This is why it is mostly patentable in the EU.

About the only thing the "technical effect" limitation prevents is business method patents.

<comment><USER22> I don't know the rules in detail, but the claim is that "software isn't patentable". And yet software with a technical effect is patentable. But all software has a technical effect. Ergo, software isn't patentable because its software, its patentable because it has a technical effect and UKIPO has granted patents on software on that basis. This means that claiming "software isn't patentable" is a misleading claim at best.

</comment></comment></comment></comment><comment><USER24> or it will be superceded by the Trans Pacific Partnership. But I can't tell since the text hasn't been officially released. Leaked drafts don't look good though.

<comment> qu4z-2 "Eternal vigilance is the price of liberty."

There's always another bill someone's trying to force on us :/

</comment></comment><comment><USER25> It will be interesting to see what the practical effect of this is over the long term. The new SOP says that certain processes involving software will still be patentable, which seems remarkably similar to the Supreme Court's finding in Diehr.

</comment><comment><USER17> I've always felt that NZ was a sensible place for startups (in theory, I've never been). A nice clean country, easy of doing business, no capital gains tax, respectable government, no drones or porno scanners, California-like in north (and amazing scenery every in general), impressive # of tech companies in relation to size. They'd be smart to do a Start-up Chile sort of program. If I were a VC and wanted an adventure, I'd set up shop in Auckland and target the promising startups who couldn't get US visas.

<http://ease-doing-business.findthedata.org/>

</comment><comment><USER26> We recently covered patents and basic copyright law in a technical writing class. What my professors taught me was that 'patent is a contract between society and inventors to encourage progress'. And it seems fair on certain levels.

I am a huge proponent of FOSS, I respect Stallman, I've read his book and I think patent trolls are dicks. However, is it a good idea to make software _not patentable_? Will we as a community fail to innovate then?

 _Disclaimer: I only skimmed over the OP_

<comment><USER27> The problem is that the vast amount of computer software doesn't contain an inventive step. The vast majority of software development is taking existing, well-understood techniques and solutions, and applying them to an equally- well-understood problem. Implementation, rather than innovation.

The parts of software development that are genuinely inventive (and thus could be considered worthy of patent protection) are either already held to be not patentable (mathematical algorithms, development practices), or are far better served by copyright and trademark law than by patents (design). Patents serve little-to-no-purpose in protecting legitimate invention in software.

<comment><USER28> Lets also not forget that proprietary software implementations are by design a secret, which almost always are kept forever from the public.

So a genuinely inventive idea can be independent created over and over again. A email sorting algorithm could had firstly been made in then 1950, again in the 1970, 3 times in the 1990, and then claimed in a patent in the 2000. Even if source code was public, a patent examiner would have to compare and understand billions of line of codes to even try to grasp what is genuinely inventive, and was has already been done before.

Examining software patents is an unsolvable problem. Society can either have them an accept that any review is a token effort at best, and a rubber stamp in the common case, or they can avoid the whole mess by not allowing those kinds of patents.

</comment></comment></comment><comment><USER29> New Zealand is such an awesome country. Wasn't it also voted the most free country in the world? I have no sources to back that claim up; I read it somewhere.

</comment><comment><USER30> Really good news. Now, can someone do the same thing in US please?

</comment><comment><USER31> If you're not making the software that you yourself use, you probably want it to be free.

[http://en.wikipedia.org/wiki/Category:Software_companies_of_...](http://en.wikipedia.org/wiki/Category:Software_companies_of_the_United_States)

[http://en.wikipedia.org/wiki/Category:Software_companies_of_...](http://en.wikipedia.org/wiki/Category:Software_companies_of_New_Zealand)

</comment><comment><USER32> This will set a good precedent for other developed countries. Let's hope Australia goes next, and maybe Scandinavia.

</comment><comment><USER33> Is this really that newsworthy? I thought the US was the only country in the world where patents apply to software.

<comment><USER34> In Japan as well.

<http://en.wikipedia.org/wiki/Software_patent#Japan>

</comment></comment><comment><USER35> approval. A software is just like a math formula or expression. Can a math formula be patentable?

</comment><comment><USER36> New Zealand seems like such a sensible place.

<comment><USER37> <http://3strikes.net.nz/information/law-basics> :\

<comment><USER38> That's a nasty one, but such an easy law to avoid. Use direct downloads or download movies, since the movie companies are protesting paying costs by not filing complaints.

I'd rather it didn't exist though.

<comment><USER39> Even in its nastiness it is sensible in its own right - $25 cost for the copyright holder to send a notice. The notice can also be challenged by the accused infringer.

<comment> qu4z-2 Well, also you can't get another notice within 28 days.

And if I recall correctly, you have to get three notices within 9 months.

So basically if you manage to spend less than 1/3rd of your time "On Notice" you should be sweet as.

DISCLAIMER: IANAL

EDIT: Actually, fence-post error. The correct figure is ~2/9ths, as penalties trigger on the receipt of the third notice (so we can't count its period).

<comment><USER40> I think everyone forgot about the 3 strikes law here.

None of the big ISP's enforce it AT ALL.

I've heard a couple of small ISP's kick a couple of people off back when it was first introduced but since then there's been nothing.

Considering most of my friends here all torrent everything off The Pirate Bay (They don't use private trackers :p), I don't think I've heard of a single person I know get in trouble.

Telecom NZ would see that enforcing that law as a cost to them (they are pretty anal about spending money on things other than marketing), so they would probably not do anything until the government takes them to court.

<comment><USER41> My father got a letter from his small ISP saying that they won't be playing along the the three strike thing, and for some reason even if they wanted to they couldn't be sure which of their customers was the infringer. Heavily suspicious, but not 100%. So carry on, we got your back. He is just outside Tauranga.

</comment></comment></comment></comment></comment></comment></comment><comment><USER42> ♫ Pōkarekare ana

</comment><comment><USER43> Craig _Foss_?</comment>


Ronaldo Is Worth Four Times the Entire Hungarian National Team - <AUTHOR> http://www.bloomberg.com/news/articles/2016-05-30/ronaldo-is-worth-four-times-the-entire-hungarian-national-team <comment><AUTHOR> "The Spanish team is the most valuable overall at an estimated $736 million, followed by Germany at $633 million and France at $551 million."</comment>


Comcast, beware: New city-run broadband offers 1Gbps for $60 a month - <AUTHOR> https://arstechnica.com/tech-policy/2019/08/comcast-beware-new-city-run-broadband-offers-1gbps-for-60-a-month/ <comment><USER2> Great going Fort Collins Municipal broadband! way to put the smack down on Comcast. I'm cheering for you, and every other broadband provider that kicks the snot out Comcast and At&t.

</comment><comment><USER3> Why does Comcast continue to fight fiber installations when 5G is on the horizon and likely to draw consumers away to the other big providers (AT&T, Verizon, and others)? To me, that seems like the bigger battle for Comcast.

Secondly, does anyone think it wise to bond $145m as the city did with wireless advances on the horizon? Especially when so far only a couple dozen homes have been connected?

I suppose Comcast can and will fight on multiple fronts, so going after municipal services isn't mutually exclusive to 5G, but both seem like a losing battle for 'ye olde Comcast. I just wonder if in 10 years we'll hear a story about how Comcast "saved" a city $150m by lobbying against a fiber project and prevailing but all the residents ultimately left Comcast for a wireless provider.

</comment><comment><USER4> Rest of world laughs in old gigabit.

Always found it amazing how the US is so far ahead on tech yet is so terrible at actual internet penetration.

My building hit that price/speed level maybe 3-4? Places with actual fast internet like S Korea probably much sooner.

<comment><USER5> If you read the article, Fort Collins had to issue $143 million in bonds to finance the system. The US is enormous, and the cost to deliver fiber to everyone will be high (versus Europe or South Korea).

Arguably, as a muni utility, you don't care about turning a profit versus Comcast and other for profit enterprises. And you have local governance over the infra and how it's operated through local government. But it still takes a lot of effort; effort that can be distributed across the country, but effort that needs to be expended nevertheless.

<comment><USER6> The United States bay be big, but it still has double the population density of New Zealand and yet New Zealand has high avaliability of cheap gigabit fibre.

<comment><USER7> Does every single place and person in NZ have super fast internet? NZ and Australia both have a huge portion of their populations in their major cities or around them. It is not at all the same as comparing to the US's population density.

<comment><USER8> How about the densest place in continental US? NY Manhattan, 1st Avenue - choice is Verizon DSL or Spectrum Business Cable.

"New York Votes to Kick Spectrum Cable Out of the State" [https://fortune.com/2018/07/29/spectrum-communications- kicke...](https://fortune.com/2018/07/29/spectrum-communications-kicked-out- new-york/)

"New York City sues Verizon for not completing citywide fiber network" [https://techcrunch.com/2017/03/14/nyc-sues- verizon/](https://techcrunch.com/2017/03/14/nyc-sues-verizon/)</comment></comment></comment></comment></comment>


Ask HN: What can I do to accelerate scientific research? - <AUTHOR> I love science & tech and how these improve lives. As a software engineer/entrepreneur, in the last years I thought of starting/contributing to some projects which scientists would find useful. Now I'm ready to work full time on this.
Ideas revolve around
* indexing all open research with free unlimited access, similar to arxiv-sanity.com but better; Other projects exist though: Google Scholar; semanticscholar.org, academic.microsoft.com,  https://www.chanzuckerberg.com/science/projects-meta
* generative design
* bioengineering (not sure exactly what, eg microbiota simulator)
* materials simulator (eg how can we get a material having a given set of properties)
I don't need immediate financial returns, but I do need the work to be used & have an impact in real life projects.
What ideas do you have on how one can accelerate scientific research? <comment><USER4> This is a pipe dream of mine, but I would love a Wikipedia of null results: nullpedia! Nobody publishes null results bc they're not exciting, but I think a lot of NSF money would be saved if "failed" experiments were aggregated somewhere in a searchable way.

There's lot of questions. How to organize it? How to encourage participation? How to maximize usefulness while at the same time minimizing volunteer effort? How to encourage discussion (suggesting changes for a better exp design) rather than manipulation (stealing the seed of a bad experiment to publish at your better funded lab)?

I don't know how to do it, but I think if done right people would really like it.

<comment><AUTHOR> Good idea, and great challenges indeed. Taking this further, I'd call it experimenpedia, with experiments published as they are thought out. Be open for comments/review and showing related experiments before the actual experiment being done. This might prevent potential failures and let owners tweak the planned experiment before being actually done. Then do the experiment and publish the results, whatever they are.

I guess secrecy would actually win though, and nobody would use this?

<comment><USER4> eLife is sorta like this. The experiment isn't live, but the publishing process is.

[https://elifesciences.org/about/peer- review](https://elifesciences.org/about/peer-review)

It's pretty interesting

</comment></comment></comment><comment><USER5> Provide funding for permanent positions. I'm currently in the last few months of my first postdoc in condensed matter physics (think superconductors, quantum computers etc.) but will move into industry next year for lack of perspectives towards a permanent position in a reasonable place. As far as I can tell, my research so far was not substandard and the software I wrote has enabled quite a few projects which otherwise would have not been possible or taken much longer. Most people I talk to (both inside and outside academia) express some degree of disappointment over people like me leaving (after 4 years PhD + 2 years Post-doc) but none of them put their money where their mouth is.

To be clear, I can understand that a PhD candidacy justifies a temporary contract and I'm not even asking for a permanent position directly after a PhD (as would be standard in industry), I'm only asking for a reasonably safe perspective towards a permanent position reasonably soon after graduating. Can't exactly start a family if you don't have any kind of job security beyond the next couple months.

<comment><AUTHOR> Excellent point. I cannot afford providing funding, but I can fund myself for 3 years to work on useful software.

> the software I wrote has enabled quite a few projects which otherwise would > have not been possible or taken much longer

What's the common practice with such software? Is that published somewhere, open sourced? Or kept private in hopes of being monetized, with IP owned by the author/university?

<comment><USER5> At the moment it's "available within collaborations". My former supervisor has had some bad experiences with people using his open-sourced software without acknowledgement etc., which is of course not quite ideal if you actually want to build a career in academia for yourself. Monetisation is not really an option.

My toolkit is maybe a bit non-standard in that it has attracted a few external collaborators using it as well and I like to think I have taken better care of upholding coding standards, documentation etc.

Normally software in my field is kept within a group and dies after one or two PhD students have left.

<comment><USER6> This is a very sad state of affairs, therefore I would like to bring your attention to a petition towards open sourcing all scientific (and generally tax-paid) software:

[https://publiccode.eu](https://publiccode.eu)

<comment><USER7> This is a great initiative.It should not be limited to software: research papers, databases and many other things that are currently either not available at all or are behind pay walls should be released.

<comment><AUTHOR> For research papers, there's also this open access initiative which is gaining support: [https://www.coalition-s.org/](https://www.coalition-s.org/) [https://en.wikipedia.org/wiki/Plan_S](https://en.wikipedia.org/wiki/Plan_S)

</comment></comment></comment></comment></comment></comment><comment><USER8> The biggest problem scientists in academia face is that they can't do actual research. They spend most of their time doing stuff that doesn't matter. So probably the biggest leverage you have is to reduce the time they spend doing stuff that doesn't matter.

The most effective way to do that is to establish an alternative form of institution that focuses only on research (and teaching if you like - <USER2> is not the problem). A tough challenge. One line of attack would be to contrast your costs with academia's full economic coating model.

Failing that, here's some things you can do:

\- develop a paper reference system that actually works well. Mendeley is the best there is and it's (IMO) rubbish. Poorly designed.

\- build a typesetting system that's an alternative to latex but is WYSIWYG. Particularly important outside of CS.

\- build a free conference organising / review tool that works. EasyChair is popular and utter garbage.

\- build tools to automate the grant writing process. A step by step system to create a grant proposal, tailored to each grant scheme. Yes, this would potentially damage the grant application process. But it doesn't make any sense anyway, so at least this would free up a few years where academics could churn the required proposals out fast until you were somehow disallowed.

\- provide free slide materials, examples, exercises for all the main CS etc topics. Some kind of "piece it together" kit so lecturers could save time making slides and other materials that tick boxes. Diagrams, for instance, would be very useful. Pseudocode too.

NB you probably won't make any money.

<comment><USER9> >develop a paper reference system that actually works well. Mendeley is the best there is and it's (IMO) rubbish. Poorly designed.

Zotero works wonderfully for me. And it's open, which is a must.

<comment><USER10> I'll preface this by clearly stating that Zotero is absolutely indispensable; I wouldn't be nearly as organized without it. It's very important to me that such tools be open, and Mendeley in particular is a complete disaster in this regard (see their history with encrypting user data).

That being said, Zotero is very much a "least worst" tool in my opinion.

* Overly rigid in how it goes about modeling document types and metadata fields.

* Doesn't handle bookmarks, browsing history, and other various data types. At first glance it's easy to dismiss such things as out of scope, but I find that my typical workflow results in reams of such unsupported data being generated and manually tracked by me. The problem is that this unsupported data is often tightly coupled to the data I'm managing with Zotero, which is frustrating to say the least.

* An incredibly heavy and inefficient piece of software.

* It's far too difficult to set up and manage my own sync server (last time I checked, at least). I don't really _want_ to share all my data with the developers, but it's very inconvenient not to do so.

More on topic with the broader discussion - knowledge and data management in general seems to be a largely unsolved problem, particularly in science and particularly regarding interrelations between and versioning of arbitrary pieces of data.

<comment><USER11> Why don't data versioning tools like git lfs do the job? Is it lack of awareness or is the problem more complex than that?

<comment><USER10> Well I could just be unaware of some functionality they have, but all those tools do is version things. There's no integration with reference managers like Zotero (that I'm aware of), and no tracking of interrelations or metadata.

In contrast, Zotero (and other reference managers) don't do any versioning at all (at least that I'm aware of). Instead, they keep track of the metadata that's necessary to put together a works cited section for an academic paper.

... or at least that's what they started out doing. These days they also try to organize your papers into some sort of category structure, facilitate tagging and notes, provide synchronization between your devices, and probably a few other things that don't come to mind right now.

Feature creep? Sure, but all that stuff is central to the research and writing process. It's also all tightly coupled, so splitting it between multiple tools doesn't work very well. And that's the current problem - how to integrate, for example, a few of your browser bookmarks with your academic literature collection. Or how to track a list of all the papers cited by a particular paper. Or link a specific paper tracked by your reference management software against a specific version of a large data set, perhaps itself tracked by Git LFS.

Generalizing a bit, what about linking experimental notes (typically pen and paper) with data collection software (typically a binary), as well as the collected data (perhaps Git LFS), as well as a specific version of some data analysis scripts you wrote (perhaps Git). Now try to track everything as you work on multiple paper revisions with collaborators, each version of which adds (and sometimes removes) citations and could use a different (likely newer) revision of the collection software, data set, or analysis scripts.

Alternatively, for a data management scenario not directly involving writing papers consider molecular cloning using plasmids. You have a dozen semi- related tubes in a cryogenic freezer that you need to track over many years (ie long term inventory management), each of which has one or more pieces of sequencing data attached to it (so a small data set), they're all interrelated (you create a new one by physically modifying an old one), and each has the typical meta-links to experimental protocols, notes, academic literature, and other things.

I'm not aware of any software solutions that comprehensively address all of this stuff, so people still use pen and paper. But pen and paper is time consuming, it's error prone, it doesn't sync between devices, it's slow and tedious to cross reference - all the typical problems that software is good at addressing.

</comment></comment></comment></comment></comment><comment><USER12> Slightly smaller-scale than most suggestions here, but for the average nonscientific HNer, the best way to help scientists is to improve their programming tools. In the Python ecosystem, for instance; numpy/scipy, scikit- learn, and matplotlib are widely used across dozens of disciplines, and are open-source projects relatively welcoming of new contributors. Julia is a whole new language for scientific computing, where all the fundamental tools are still being built and refined. Raspberry pis, 3D printers and other "hobbyist maker" tools are appearing in research labs to help develop novel instrumentation, so hardware-oriented people can help by contributing to open- source efforts of that kind.

<comment><USER13> Somewhere we computing folk lost our way - nobody needs software just for the sake of it (with the exception of maybe games). Everything we do is supposed to be building tools for other people who do the _real_ work to make their life easier. As opposed to how it currently is which is to provide something for free, then put up artificial barriers to certain parts of it and charge for their removal.

</comment></comment><comment><USER14> Come work with us at Plex Research. A huge problem our founder ran into while doing drug discovery research was that there's tons of data in the world, but no one is using it because it's all in thousands of different places. As a programming, it was crazy to me that even the most advanced organizations in industry (Novartis, Sanofi) or academia (Harvard, Stanford, ...) are still keeping many important datasets in Excel.

We're pulling together all of the world's biomedical research data, structuring it as a graph, and allowing research access all of it as easily as Google does the web:

[https://www.plexresearch.com/products.html](https://www.plexresearch.com/products.html)

</comment><comment><USER15> > indexing all open research with free unlimited access, similar to arxiv- > sanity.com but better

This space is pretty crowded, in my opinion.

I don't know much about biology, but I can tell you that in materials, it's all about data. The materials design problem and predicting new materials comes down to knowing properties of other materials. A lot of progress has been made by using datasets generated by quantum mechanical calculations by the Materials Project, OQMD, AFLOW and NOMAD, but materials design is tricky because what we want to predict are the outliers that we haven't seen yet: materials with the highest strength, etc..

There's value to be created for materials researchers by curating experimental data in a digital, usable form, since so much is locked up in papers, but you really need domain expertise for this and there's another problem that the experiments are so sparse and have so many features (chemistry, microstructure, thermal history, etc) that people have really only been successful when focusing on particular classes of materials.

<comment> andrei-mircea [https://citrine.io/](https://citrine.io/)

You might find this company interesting.

<comment><USER15> I actually know and work with several people there :)

</comment></comment></comment><comment><USER16> Note: The following is an earnest suggestion and not just a recruiting plug, but it is also a recruiting plug. If this is not an appropriate place for this content then comment and I'll remove (I do not frequently HackerNews)!

Personally, I believe the highest leverage thing we can do to promote scientific research is to help connect the dots between related discoveries in the service of bringing new ideas to commercialization/impact as quickly and reliably as possible. I joined Citrine ([https://citrine.io/platform/](https://citrine.io/platform/)) a few years ago thinking that we would help accelerate research by direct support - <USER3> simulation, data, and machine learning tools for scientists. It became clear very quickly that the community was already in a pretty strong position (smart cookies, those research scientists) on that front. The biggest opportunity turned out to be scaling expert knowledge - bridging the no man's land from ideation to scale-up, integration, and manufacturing. At Citrine, we're building infrastructure that helps researchers contribute to an organization- wide knowledge graph capable of supporting inference in the scale-up or manufacturing context based on relationships learned in the R&D phase. This fundamentally changes the ROI calculus for basic research because it can plausibly support the entire product life cycle.

If this sounds exciting to you, then consider joining us! We just raised a Series B and are growing quickly.

If you have an applied math and software eng. background and want to help generalize and scale our property inference infrastructure, then I think you'd enjoy working with me and my team in SSE: [https://citrine.io/careers/#scientific-software- engineer](https://citrine.io/careers/#scientific-software-engineer).

If you have a backend software eng background and want to build distributed services for scientists and engineers at some of the biggest materials and chemicals companies in the world, join us in engineering: [https://citrine.io/careers/#sr-backend-software- engineer](https://citrine.io/careers/#sr-backend-software-engineer).

</comment><comment><USER17> If you want to help software related AI research, start compiling massive high quality training data sets and giving it away for free. Easier said than done though.

No _immediate_ financial return? I hope you can accept _no_ financial return, period. In general the easiest way for an individual to accelerate general research is through generous funding. But even then it's not like a slider in a game where you provide more funding and things get done faster. There's diminishing returns after a point. Not that I'm trying to discourage you, but I hope you're thinking about it the right way before you waste a lot of time and money.

I suggest talking to actual researchers and asking _them_ what they really need, and give them that. Basically the same as a startup going out and talking to customers. The only research probably being done around here is largely software related, and probably not changing the world much in ways that actually matter.

<comment><AUTHOR> > I suggest talking to actual researchers and asking them what they really > need, and give them that.

Will start to do that, thanks!

> No immediate financial return? I hope you can accept no financial return, > period.

That would be ok for the next 3 years.

<comment><USER17> > That would be ok for the next 3 years.

No, I don't mean no returns for three years, I mean no returns _ever_. You must go into this with both eyes open, don't find yourself crippled later because you gave all your time and money away and have nothing to show for it.

<comment><USER18> What if he would sell models? A marketplace for trained models.

<comment><USER19> Some of this already happens. But you also have to realize that that is not in the spirit of science.

<comment><USER18> Got it. It's not, agreed, but nobody should work for free. In fact, I believe that doing what OP wants to do is more harmful than building a business around his intentions.

<comment><USER19> I agree, no one should have to work for free. But there are two competing aspects here. Science is about seeking out knowledge and advancing humanity. But unfortunately we need to buy food to eat.

</comment></comment></comment></comment></comment></comment></comment><comment><USER20> One of the big challenges for university researchers is trying to find talent and dedication in the enormous pool of undergrads and masters students. Nearly every professor relies heavily on the recommendation system or grabbing from the pool of students in a class. The problem with this is that it's often hit or miss with a net-neutral return on investment.

If I send a PhD student to spend a certain amount of time independently training two students, then I am investing that grad student's time into something that could be spent doing research. If one of those students is flaky and is mostly there to pad their resume, it's largely a lost investment (even more so if their work requires extra effort to fix). If the other is graduated by the following semester, the gained productivity might not exceed the other's loss by the time they leave.

The best situation is when you're working with an undergrad who is destined to continue on to the PhD program because you know they'll be dedicated and possibly have a head-start on their thesis work. If you could figure out how to connect these students with advisors using metrics and private social networking, then you will amplify their productivity significantly. You'll improve the likelihood professors will take on undergrads and potentially push researchers into the field earlier.

How you achieve this, I'm not entirely sure. Perhaps you can make it easy to set up competitions which test the skills you need. The winners will be asked to join the lab to work on some project. If some lab on the other side of the country which does similar research also wants to do the same competition, make it easy for them to share and run it.

<comment><USER21> I mean, you could pay them more than approximately nothing...

</comment></comment><comment><USER22> I have a request. Somebody needs to make matplotlib, but with a C API and lots of language bindings instead of only a python API. For example when I'm writing rust, my best option is to use a Python interop library that calls matplotlib...

<comment><USER23> Python is kind of the gold standard in many areas of research, no? Why would you use Rust, or even anything but Python?

I'd only touch Rust (or C/C++) when I need to implement some fast numerical computation that does not already exist in Numpy or Tensorflow, but still call it from Python.

<comment><USER24> I've sped up a simulation 100 fold by porting a 20 line function to rust (which had a terrible access pattern for numpy), and could probably sped it up another 10 (which was closer to reasonable for numpy).

If the thing could have been written in rust in the first place, tons of time would have been saved on trying to optimize python, waiting for simulations to complete before (and to a lesser extent after) I ported a portion of it to rust. Dealing with language interop and build systems.

The main reason I can't suggest that for future similar problems to the person who I did this for is because of the lack of libraries like plotting (plotting is by far the most important one, numpy is second but rust comes a lot closer in that regard).

</comment></comment></comment><comment><USER25> Some interesting companies at the intersection of science/engineering that are hiring software devs:

\- [https://www.ginkgobioworks.com/](https://www.ginkgobioworks.com/) Synthetic biology engineering

\- [https://www.benchling.com/](https://www.benchling.com/) Online LIMS

\- [https://neuralink.com/](https://neuralink.com/) Brain Machine Interface

\- [https://strateos.com/](https://strateos.com/) Programmatic Cloud Lab (disclaimer, I work here)

</comment><comment><USER26> Sci-Hub removes barriers to accessing scientific information. Their infrastructure is slow and has trouble. If you are looking for real impact, help them scale.

</comment><comment><USER27> I don't think this is a comprehensive answer, but if you want a nice summary of how climate change might be impacted by ML/DL applications, this is hot off the press: Tackling Climate Change with Machine Learning, [https://arxiv.org/abs/1906.05433](https://arxiv.org/abs/1906.05433)

<comment><USER28> There was a recent NOAA conference on this topic as well, you can view the slides here: [https://www.star.nesdis.noaa.gov/star/meeting_2019AIWorkshop...](https://www.star.nesdis.noaa.gov/star/meeting_2019AIWorkshop_agenda.php)

</comment></comment><comment><USER29> I think there's lot of room to improve tools in bioinformatics. In practice bioinformatics pipelines tend to be bundles of loosely organised python scripts and I've heard files on the order of a few GB described as Big Data because the processing times are so slow (days for stuff that could take milliseconds)

It would help to pair up with practicing scientists and explore what parts of their workflow can be improved

<comment><USER30> Definitely. A lot of work is done in R or python and could be sped up in a compiled language.

<comment><USER31> Yes. I'm not a scientist, but I think if there were good compiled python/R alternatives, the scientific world would benefit greatly, if not only for the reduced waiting times... Maybe a language with Go simplicity and speed and Python ease of use and appeal... It should have an almost real-time compile mode (with very little optimization) to enable interactive playgrounds, like Jupyter. Of course it'd need also strong optimization modes for final production code.

</comment></comment></comment><comment><USER32> On the intersection of biology and machine learning, there is one of the holy grails of science: protein structure prediction.

I'd recommend starting reading about Google's AlphaFold, since this is currently considered state of the art in the field: [https://deepmind.com/blog/alphafold/](https://deepmind.com/blog/alphafold/)

</comment><comment><USER33> > What ideas do you have on how one can accelerate scientific research?

I work in genetics (as a software engineer).

If there was a major flaw in current scientific research (that involves software), it's that most labs care more about getting published than they do about the reproduce-ability and validation of their work. This means most of the software written in research is ad-hoc, write once, and often never looked at again. It was put together for the sole-purpose of producing some output that could be put in a paper and then lost to time.

A current "holy grail" of software in research would be to fix that: empower other labs to validate and _reuse_ the software written and reproduce the work of other labs with different data sets. And it is actively being worked on in a couple places (that I know of, perhaps more):

* [https://genepattern-notebook.org/](https://genepattern-notebook.org/)

* [https://app.terra.bio/](https://app.terra.bio/)

* [https://software.broadinstitute.org/wdl/](https://software.broadinstitute.org/wdl/)

Some of these are just about giving the community a common framework to use for their software (CWL, WDL, Jupyter), others are about data storage and making it easily accessible for others to use in the cloud for reproducing results.

If you want to have a impact, joining one of these groups would probably put you in a much closer position to doing that.

If you just wanted to work on something in your spare time that would be incredibly valuable, then might I suggest this:

It's amazing how much work is done in the scientific community using CSV/TSV files (usually gzipped). And most of that work is done via perl, sed, and awk. And often these are _huge_ I'm working with a VCF file (TSV) right now that's 2 TB in size ZIPPED! It's crazy. Researchers often don't have the time, resources, or know-how to put together a simple Spark cluster and use it.

A command line tool that allowed someone to run SQL (or SQL-like) commands on a gzipped CSV file FAST would be invaluable. And if it could JOIN across CSV files ... wow!

<comment><AUTHOR> Thanks so much, mussung, for your excellent practical feedback! Both alternatives that you listed are very tempting.

May I please ask you some followup questions? There's no email in your profile. My email is firstname.lastname@example.com

> A command line tool that allowed someone to run SQL (or SQL-like) commands > on a gzipped CSV file FAST would be invaluable. And if it could JOIN across > CSV files ... wow!

What prevents one importing each CSV in a postgres db as tables, creating indexes and then start running queries? Disk space availability? (My local drive is only 1TB)

<comment><USER33> mariushn, I've sent you an email.

> What prevents one importing each CSV in a postgres db as tables, creating > indexes and then start running queries?

There are many reasons:

* Experience/knowledge. Many labs don't have anyone experienced with databases.

* Security. Without proper dev ops a local DB is often out of the question. And when dealing with PII genetic data, [cloud] security can be a major concern.

* Funding. Machines cost money. AWS RDS instance cost money. Maintaining them costs money. Dev ops costs money. etc.

* Often times the queries being done are simple. For example, you may have a giant CSV with cross-ethnic trait data, but only need samples of African decent with a beta value > 0.1. Sure, you could spin up a database, load the entire thing into a table (O(N) + disk space + time), then index it (now it's O(2N) + more disk space + more time), and then _finally_ run your query. Or you can just O(N) run over the CSV once and output the results with no extra disk space or "wasted" (perception of the researcher) time.

Finally, don't fool yourself about the capabilities of researchers. Many are code-savvy, but lack experience. Writing a SQL query is easy. Loading multiple TB of data into a relational database, indexed properly, and done in a manner that won't take _days_ of time is a level higher.

</comment></comment></comment><comment><USER34> This will sound harsh, but it isn't meant to be.

Scientists code better then you do science.

This is simply a consequence Of a weeding out mechanism for those that have no coding skills. The only ppl who get away with no coding skills are important professor with grad students to do the coding.

This isn't to say that our skills are great, but a generic programmers (I.e. CS majors) science abilities are approximately zero (common, no thermo in an "eng" undergrad???)

So what can you do?

Since you mentioned science and not engineering, I'd ignore the AI advice. Science needs models based on mechanistic understanding of the underlying phenomena. A model that merely predicts is useful for engineers, not scientists.

"materials simulator (eg how can we get a material having a given set of properties)"

This is already done, but of limited usefulness. First the Mtls simulators are far from perfect. Then there is the problem of actually synthesizing the mtls. These simulations are more typically done to weed out bad candidates.

"No immediate financial return"

Wrong attitude. Only an attitude of "no financial return" helps science. That's not to say you won't make money off of it, but that can never be a goal since (true) science advances freely (again see the Gaussian jerk vs. Einstein or Landau - who contr. more?)

Instead, focus on making the programming tools scientists use better, easier to use and GPL. GPL is important because an MIT license by itself allows a scientist to use others work while blocking others (see Gaussian).

For example, making python (or Julia?) better would be one of the most important contributions you could make. The matplotlib guy was deeply mourned in science.

The two cents of a physical sciences researcher who once flirted with the Valley.

<comment><USER35> > Science needs models based on mechanistic understanding of the underlying > phenomena. A model that merely predicts is useful for engineers, not > scientists

I'm not sure I agree. I'm aware of quite a bit of supercomputing time that is spent doing lattice QCD calculations (which apparently some scientists find useful), and though I'm no quantum physicist I'm pretty sure there is not much of a "mechanistic understanding" in QCD. I think your claim also doesn't apply to a lot of social science - psychology has a lot of functional models, but I don't think there are many mechanisms described.

I'll also state that modern science that doesn't require any engineering is pretty rare nowadays, so if a predictive model helps engineers that can then help scientists, the model has been helpful to scientists.

Ohm's law existed long before there was a mechanistic description behind it, and though it is mostly used for "engineering," I feel confident that a lot of scientists in the 19th century found it useful.

From [https://www.olcf.ornl.gov/leadership- science/physics/](https://www.olcf.ornl.gov/leadership-science/physics/):

"New Frontiers for Material Modeling via Machine Learning Techniques" \- 40,000 hours allocated on Summit

"Large scale deep neural network optimization for neutrino physics" \- 58,000,000 hours allocated on Summit.

Supercomputers typically do not allocate 58 million hours to things which are not useful.

<comment><USER19> I work with the DOE and was at ORNL before Summit was released (I got to play on Summit-dev). When making these models there is A LOT of exploration happening. There's a whole class of visualization techniques called "in situ" that visualize data as it comes off the press (memory is then dumped because there's neither enough storage space nor can we write to disk fast enough). I'll tell you that there will be a lot of restarting those simulations because the scientists need to explore the data as it is going on. Going in the wrong direction? Made a small mistake that causes cells explode? Realize you're not looking in the right region of interest? You restart the sim (thank god for restart files, right?). Exploration is one of the most important things in research and it is getting more and more difficult. I believe this is what the gp is after. Having these understandings helps you explore the data better. Creating these tools is hard work and takes a lot of collaboration too.

</comment></comment></comment><comment><USER36> I saw this list linked yesterday from another HN article, might give some useful jumping off points: [https://github.com/kakoni/awesome- healthcare](https://github.com/kakoni/awesome-healthcare)

One related thing I found from there was a list of projects for magnetic resonance imaging specifically: [https://ismrm.github.io/mrhub/](https://ismrm.github.io/mrhub/)

I'd assume trying to contribute to those projects would hopefully give greater ROI than building a new thing (without a very specific idea of what to build and the market for it)?

<comment><AUTHOR> Thanks!

</comment></comment><comment><USER37> Find any way to make a lot of money, gain a lot of political power or gain influence over those who have money or power and use it to fund research and make it more appealing culturally.

<comment><USER38> No, the most significant advances in science were done on a shoestring budget (or no budget at all) - just by THINKING. It doesn't require lots of money to support people who do that kind of work, and they are easy to find, especially in theoretical physics and mathematics.

<comment><USER6> Maybe it was true a hundred years ago, but it is not true today, unfortunately. Science became much more complicated.

[https://news.ycombinator.com/item?id=20190468](https://news.ycombinator.com/item?id=20190468)

<comment><USER38> It is still true. Before the tau neutrino was confirmed in a lab, it had been theoretized 25 or so years before that, which required nothing but pencil and paper. Modern quantum field theory doesn't require expensive gear. Math doesn't require any gear at all, and there is a whole pile of crucially important unsolved problems.

The fact that science became complicated is a sign of thought stagnation, not a sign of progress.

</comment></comment></comment></comment><comment><USER39> As a faculty running a lab in human genomics /genetic I would say either join a lab which needs your skills or work for a univeristy in support of their IT and high-performance computing needs. People with computational skills, an interest in doing/enabling research, and willing to accept a sub-market salaery are obviously rare. IT departments at universities are under-staffed and over red-taped, but most researchers depend on them to actually do work.

</comment><comment><USER40> I would go talk to scientists and try to understand deeply what they're working on, how they do it, and what systems (social/technical/political) allow them to do it. I've done this a few times and it's always illuminating and inspiring.

<comment><AUTHOR> Correct. How do you connect with scientists when not working in an university like you do? Getting an email from a nobody offering help for free does sound like a scam.

One answer would be "Enroll into an university". Others?

<comment><USER40> I just emailed them and told them I was doing research. Lots of people, especially students, love talking about their work.

</comment></comment></comment><comment><USER19> My dream would be to have a place I could look at any paper (even if not on a campus internet connection), be able to look at raw data, code, and have a forum to facilitate discussions between researchers.

I know the first won't happen for awhile, but it is a dream (open science).

The second is direly needed but in some cases not practical. But I think there could be a lot to gain from just having a small portion of this. It could help verify results significantly. Also imagine if people could research things without having to have all the fancy instruments. Some of this already happens, but I think it is harder to find and not always easy to sort though. It isn't connected to papers and research. Just having a paper and a link to the data would be tremendous help.

Similarly code. I don't know a researcher that hasn't made an accidental bug in the code that changes results (some slight, some major). I think we need to get over that WE ALL write hacky code. Hacky code is better than a vague description in a paper because you don't have enough room to write an accurate description of your model. Science is supposed to be replicated! Even linking papers with a github account would be tremendous help. Some people don't want to share code, and I think this is a shame and anti-scientific (especially if you are using public money), but that's a rant on its own.

Researchers email one another all the time. Some of these discussions should be public. Papers leave a lot of gaps. An area researchers could add extra notes that couldn't be fit in a page limit, where collaboration can happen, or where people can just ask questions, would be great. Replicating results can be hard and we should be learning from one another's hurdles. That's the point of science after all, for to push the progress of humanity. Lack of ability to communicate should not be a gate.

Along with these things it would be nice to encourage putting up null results. Aside a paper I would love to know what challenges a researcher fought. That's where most of the work is. It is funny, we constantly talk about failure being 80-90% of research. Whole projects failing or just banging your head on the desk because you can't figure out why something isn't working. Let's open this up. Let's help one another. Let's talk about what went wrong and how we fixed things to get to our success. I can't think of anything that would help science more than this.

<comment><USER41> I wonder if people are reluctant to do this because they would be scrutinized more and perhaps their phd, Grant etc might be at risk?

If so, is the only solution to give"credit" towards the phd, grant etc in the form of hours worked towards pushing knowledge acquisition, and not strict results?

<comment><USER19> I think it is partially that (but minor). Partially embarrassment (frequently code is rushed). Partially people view code/data as a trade secret. The latter I find anti-scientific, especially since a lot of funding comes from public money. I'm okay with holding on to it for a small period of time (because we live in an unfortunate world where sharing knowledge can't come first and people need to secure funding), but I don't think this should be a default mode that people do. Luckily it does seem that many are turning to GitHub/BitBucket/GitLab to make their code available.

And everyone that's in a PhD or has one knows that the majority of work you do is failing (but you learn from that failure).

Side note: I wonder if imposter syndrome would decrease if we were aware about one anothers' failures and didn't only see the accomplishments.

</comment></comment></comment><comment><USER42> Learning a lot how science actually works and how flawed it is I came to the conclusion that the biggest boost in scientific progress you could ever achieve is by eliminating science waste and doing more good science.

There are a few tiny steps in the right direction, but it's frustratingly slow. Once you understand a phenomena like publication bias it's hard to swallow that there are still empirical studies published without preregistration. There are so many studies published with such low quality that it's a complete waste, because the likelyhood that they're some statistical fluke is much higher than that they tell the truth.

Though the problems are known since a while, little has changed: [https://journals.plos.org/plosmedicine/article?id=10.1371/jo...](https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124)

<comment><USER19> That is a very click-baity publication (yes, it happens). Title suggests this phenomena is independent of fields. Content talks about the high p value. Not every field uses this metric. So this publication is aimed pretty much at medicine.

While I think the p-value thing needs to be rethought, it is bold to claim "Most Published Research Findings Are False". Honestly it would be more accurate to say "Errors matter and we can't take findings at face value and ignore error bars". Which is pretty much something every self-respecting scientist should know in the first place.

</comment></comment><comment><USER43> So as someone working in application of medicines, I'd agree with the comments that say those of us doing these things have learn to make code work, but are not software engineers (and would not claim to be). This means that if you can improve the tools that are used (and don't care about making a return), the leverage would be massive.

I imagine it isn't trivial (otherwise it would have been done by now), but imagine if R was automatically parallel processing; seldom is the code in my field worth figuring out how to make it parallel, but if it was automatically, then a hell of a lot of time would be saved! I know there are two packages that kind of make it work, but so far I've yet to hit a threshold making it worth it (I just leave thigns running overnight).

Abstract from that to projects like Zotero and you can see how you could have an impact on a lot of people by enabling them to do what they do.

</comment><comment><USER44> I felt the same, and decided to learn how to do that. Followed this one guy i admired a lot. The way he is doing it is to merge software engineering practices into science (in this case AI and robotics), and reduce the cost to do iteration.

For us it means: 1\. 1 click binary deployments 2\. Safer iterations that allow for making mistakes, so that they can do 10s at the same time instead of 1 really safe one. 3\. Logging visualization and whatnot on a unified infra.

We are not scientists, but we know damn well how to scale software and business. It applies everywhere. Think of tensorflow, before most people couldnt do ai themselves, now it is damn easy and more things will happen as a result.

This way they can concentrate on science while we concentrate on scaling it. We are betting on couple breakthroughs as a result of increased enthropy.

</comment><comment><USER45> You can get a job at a company doing research. Improve their tooling and this make them research faster. SRI comes to mind in this regard.

Inside the company you can also push them to try and open source the tools. Unlikely that'll work if they don't do it already, however this skill set will eventually let you contribute more in the future. After some time on the job, start a personal open source project or start a company directed at some of the issues you saw.

It's general advice and it's the long game, but will likely help you have more impact.

The advice above may be more useful for engineers earlier in their career, but you can accomplish that in a handful of years.

</comment><comment><USER46> Forget science: as an "army of one" you're unlikely to make a contribution of sufficient magnitude that some needle might be moved in the right direction. Take advantage of the skills you do have: make money by doing what startup entrepreneurs do, then donate all proceeds to scientific research.

There may be the next Zuckerberg hiding in you. Imagine what would be possible if you could reach that level of wealth and use all that money to propel science forward. I'm not talking about some chickenshit foundation; real impact is made by committing all of your financial means to helping science. That's how a real difference is made.

<comment><USER47> A lot, if not most of the important scientific discoveries were result of work of "an army of one". From biosciences to physics you can do amazing stuff. We don't need any new zuckebergs that siphone all of the great minds that are out there into showing more ads.

<comment><USER48> The individual scientific discovery was common in the early history of science, but has grown less common over time. Looking at this list [1], for example:

\- Prior to roughly WWII, it was mostly individuals, and so we named things after them (Kelvin, Doppler, Joule, Ohm, etc).

\- From WWII until the late 20th century, it was a lot of small teams (transistor, 3; DNA, 4; pulsars, 2; etc).

\- Since then, team sizes have grown so that individuals aren't even named (cloning, an Institute; Top Quark, a Lab; Tau neutrino, a Collaboration; etc).

In fact, in the past 35 years, the only individually named contributors on that list were mathematicians who constructed proofs of long-standing unsolved problems in mathematics.

[1] [https://en.wikipedia.org/wiki/Timeline_of_scientific_discove...](https://en.wikipedia.org/wiki/Timeline_of_scientific_discoveries)

</comment></comment></comment><comment><USER49> Side note: I am the co-founder of Aidlab - a device and a platform that is widely used by biomedical scientists and students in their research ([https://www.youtube.com/watch?v=wY0YPOKNk88](https://www.youtube.com/watch?v=wY0YPOKNk88))

In my free time I am working on my next project - an open-sourced platform where everyone can contribute to help fight death. The platform allows uploading anonymized, structured health records - publicly available. Why? Dying is a number #1 problem that should be solved together.

OP, if interested, drop me an email: jdomaszewicz (at) aidlab.com

<comment><USER50> ...why should death be solved ASAP?

We haven't got the space or resources for exponential growth of population

<comment><USER51> Death has very little to do with the exponential growth of the population. You need to look at the other end.

</comment></comment></comment><comment><USER52> Regarding the last point: try applying libraries like Quantum espresso [1] or CP2K [2] to real world problems and apply machine learning with the solutions they provide to a given problem. There's a tremendous amount of academic research being done in this direction but try and take these libraries and make them useful for real world applications.

[1] [https://github.com/QEF/q-e](https://github.com/QEF/q-e)

[2] [https://github.com/cp2k/cp2k](https://github.com/cp2k/cp2k)

<comment><USER53> As someone working in this field, I disagree. The state of these codes is that if you put garbage in, you get garbage out. Without a strong theoretical background, you're generating junk. Of course, if he's willing to spend several years developing his knowledge of the field, there is for sure room to help. If not in your suggested machine learning area, instead perhaps in improving the quality and usability of the code.

But instead I would recommend he start from something like the NOMAD database, where the calculations have already been run by more knowledgeable people. Then he can focus on the analysis side.

<comment><USER52> Of course you need to develop domain knowledge. But there's tremendous opportunity lying in these libraries, you just have to spend the time looking for it.

</comment></comment></comment><comment><USER54> There are a lot of citizen science initiatives you can participate in including folding at home and so on. Your best bet is going to be to find a field you love and find experts in the field to work with and learn from - theres definitely a lot of lack fo software talent in some areas that you'd be able to make a dent into but outside of citizen science initiatives, you need to start with understanding the problems that need to be solved which can be much more technical and difficult to understand than they ultimately could be to solve. Good luck!

<comment><AUTHOR> Thank you!

</comment></comment><comment><USER55> Provide funding for high school students to spend a summer working in a research lab? The amount of funding doesn't need to be high and it may pique early interest for the next generation .

</comment><comment><USER56> >>> Indexing all open research

I'd also check out analytics platforms such as VIVO. End goal is a universal workflow for all research, discovery and collaboration. Solving this problem will have immediate impact. For example, computational epidemiology and containing Ebola outbreaks in "hot zones" hundreds of miles apart.

Web of Science

[https://clarivate.com/products/web-of- science/](https://clarivate.com/products/web-of-science/)

</comment><comment><USER57> If you have the money to work full time on a personal project you have the time to sign up for a conference or three on the kind of scientific topic you like. Ideally, pick one near your home/in the closest metropolis.

Once there you can apply the following no-social-skills-needed guide to making contacts at a conference.

1\. Watch the keynote.

2\. After the keynote, walk up to pretty much anybody. Ask the following questions: "Do you think (keynote title) is a major area for (conference subject matter)?" and almost regardless of what they answer, you can follow up with "oh, really? Is what you're studying related to (keynote subject)?". Those two questions are enough to make virtually any academic launch in a paragraph-long exposé. Usually by the 6th conversation you have along those lines, you'll have a good summary of which subjects are considered important right now. If the person looks like they like you or are invested in talking about their topic, you can follow up by giving your contact info and saying you're interested in them sending you a paper on the subject they talked about (one of theirs if applicable).

3\. There will be designated poster sessions. The posters are giant sheets of papers with young people in front of them. Walk up to a young person standing in front of a poster that doesn't look that slick but where the subject matter interests you (slick posters are from bigger labs where you are less likely to have an impact). Ask the person what they do, how long they've been doing it for, how big is their lab, and what's the most time-consuming step in their research right now.

4\. If anyone asks you what you do, say you're an expert in (your computing area, web apps, cloud, data analysis, whatever) and interested in the intersection of (your area) and (conference name). Some of the people will say they think (your area) would be great for (their subfield) because of (super niche stuff you wouldn't have thought of). Grab their contact info so you can have a 1:1 meeting with them later where they find you a research subject.

Follow up by having email conversations with a few of the people. If a grad student looks like they can use your help, ask for an intro to the PI. Eventually you'll walk your way into a (possibly paid) research project. It's that simple. Thank you for caring about your world and its future.

</comment><comment><USER58> Build tools to make research more efficient and reproducible. The research community is a small market and has little money. Building tools is often not regarded as scientific activity and can lead to dead ends if publication is pursued.

So, if you do not have to rely on income anymore and want to contribute to science, build tools for the community. Or better, come up with a way to help scientists to build their own tools fast and efficiently.

<comment><AUTHOR> Thanks! Any other details / specifics would be welcome, since it seems you already have some knowledge in this area.

<comment><USER58> With my background in computer science and ML, I worked with genitesists to automate some of their observations.

A lot (almost everything) what they do routinely is still done manually. In genetics there are about 10 model organisms (I forgot the exact number) that most work is done on. Examples I have come in touch with are C. Elegans, D. Melanogaster and mice. A huge amount of work is done on these organisms and a huge amount of time is spent by grad students and post docs, repeating the same boring tasks day in and out. This is a big factor in deciding the breath of studies (you can only compare as many conditions as you can evaluate). Some things biologists do are subjective or prone to bias (even objective tasks like counting stuff becomes kind of biased after fatigue sets in if you have to count a lot of things and have to do it very often)

If you can automate their tasks, what will happen is that you will enable them to increase the breath of their studies drastically and potentially create more consistent measurements as well.

</comment></comment></comment><comment><USER59> Software tools to help with scientific discovery / hypothesis generation. I'm thinking specifically about Literature Based Discovery[1] (LBD), but there are probably other useful approaches in that domain.

I have a little project[2] related to LBD, which I'm working on here and there. There's still a long way to go, but I'm optimistic there's something to be accomplished in this space that could help.

And then you've got things like Robot Scientist[3] which is crazy interesting.

If AI, in the AGI sense, interests you, I might mention the possibility of contributing to / working with one of the popular "Cognitive Architecture" systems like ACT-R[4], SOAR[5], or OpenCog[6].

[1]: [https://en.wikipedia.org/wiki/Literature- based_discovery](https://en.wikipedia.org/wiki/Literature-based_discovery)

[2]: [https://github.com/fogbeam/Valmont-F](https://github.com/fogbeam/Valmont-F)

[3]: [https://en.wikipedia.org/wiki/Robot_Scientist](https://en.wikipedia.org/wiki/Robot_Scientist)

[4]: [https://en.wikipedia.org/wiki/ACT-R](https://en.wikipedia.org/wiki/ACT-R)

[5]: [https://en.wikipedia.org/wiki/Soar_(cognitive_architecture)](https://en.wikipedia.org/wiki/Soar_\(cognitive_architecture\))

[6]: [https://en.wikipedia.org/wiki/OpenCog](https://en.wikipedia.org/wiki/OpenCog)

</comment><comment><USER60> Solve the following problem:

"I want to find all research labs concerned with topic X in this area"

=> Create an (open) crunchbase/patreon for university research groups

Right now, each university has their own website, with their own layout and varying degrees of information. General scraping is impossible. Finding research groups with certain interests without trawling through an endless amount of publications in related journals or randomly meeting people at conferences is difficult. The only comprehensive lists are university ranking sites, but they are not detailed down to the research group level.

\- create a crowdsourced, public index of running projects

\- help labs find each other and collaborate on work

\- help people apply to labs with open positions

\- let donors and investors find and support projects more efficiently

\- make science journalism easier, by connecting reporters straight to the source

The main difficulty is creating the network effect which is why, I suppose, no one has done it yet.

<comment><AUTHOR> > help labs find each other and collaborate on work

Ideally, that would be awesome. But... do they want to? Will they put science & results above all, or they put first funding, egos and credits?

Science is great, but from my experience, these others are even more important for individuals. These support a career & family.

<comment><USER60> The ideas published in journals all depend on humans.

But to an outsider, these connections are completely invisible. Conferences cost thousands of dollars to attend. Subject matter experts are hard to find, so many connections are simply not made.

Which is a good reason to support such a platform, because it may make such views possible and existing deficits more visible and quantifiable :)

Small and medium size businesses outside academia could also use it as to improve their technology and connections.

</comment></comment></comment><comment><USER61> Contribute to TaxonWorks [http://taxonworks.org/](http://taxonworks.org/). Software from a small endowed group that builds tools that support those who describe Earth's species. Software is completely open source, many opportunities to improve what's going on there.

</comment><comment><USER24> I've helped one on one by working with someone who had a problem and needed some code a few times (sometimes getting paid, sometimes for free for a friend). I think that's higher impact than most of the remaining low hanging fruit. These where all 1 coder projects.

There are also larger coding projects in the sciences with teams. Running the database (and doing whatever else needs doing in code) for a telescope for instance. I don't have personal experience with them, but search around and you can probably find them.

The only thing I can think of that would have helped across the projects I've worked on would be better (more convenient, I don't care _that_ much about how the output looks as long as it conveys the information) graphing/data visualization tools.

</comment><comment><USER62> Re: materials simulator, you should totally check Materials Project [1]. I guess that they are open to getting some help in their github repos.

[1] [https://materialsproject.org/about](https://materialsproject.org/about)

<comment><AUTHOR> Thanks!

</comment></comment><comment><USER63> I think the best way to accelerate scientific research is to recreate Bell Labs in a modern setting, free from the desire to succeed.

Get a huge endowment fund (the hard part), hire a bunch of smart and promising people, and tell them you want them to change the world.

</comment><comment><USER64> I work on a project called OpenReview ([https://openreview.net](https://openreview.net)) that aims to improve scientific peer review by encouraging transparency and research on the peer review process itself. Right now most of our activity is within the machine learning academic community, but we have plans to spread into other areas of computer science and more.

We're looking for developers, if you're interested: [https://codeforscience.org/jobs?job=OpenReview- Developer](https://codeforscience.org/jobs?job=OpenReview-Developer)

Let me know if you want to learn more and we can find a way to connect.

</comment><comment><USER65> I'm the engineering director at Quartzy (YC S11). We're focusing on making research more efficient through the laboratory supply room. We help labs through a group buying experience to avoid waste, and offer discounts to their contract prices by working directly with manufacturers.

This isn't the most obvious way to make an impact, but we've effectively saved many labs thousands of dollars on equipment and consumables that can then be reallocated toward their research. Our mission is to increase the efficiency of scientific research — the supply room is just the beginning.

Would be happy to chat - email@example.com

<comment><USER66> >increase the efficiency of scientific research — the supply room is just the beginning.

Efficiency is often compromised institutionally, so I know this is an obstacle.

I've got a lifetime of business models for scientific software, but that wasn't the question above.

One way to accelerate many types of laboratory research would be to offer a workflow alternative where very high performance individuals can make the choice to spend almost all their working time at the bench(es) rather than being distracted or sitting down at desks or office & lab computers. While maintaining overall leadership of the organization, so this requires a different type of team structure as it scales.

That way someone who can really invent a lot doesn't get bogged down by the bureaucracy of their earlier inventions.

</comment></comment><comment><USER67> A shoutout for asking "what can I do" as opposed to "why is research so slow?"

</comment><comment><USER68> A lot of journals (including the one I am an editor at) would probably consider leaving the fangs of elsevier and co and go full open access publishing. However, there are a few obstacles to that, such as missing open and reliable editorial and publishing platforms, contacts to libraries, etc.

In my opinion providing these things and actively encouraging journals to make the jump to open access would be a huge service to academia. Also, I wouldn't be surprised if one could find funding for such a venture.

</comment><comment><USER69> Donating money toward researchers. All of the following situations below may or may not have a useful outcome but direct investment would be the easiest one and one you can leverage.

</comment><comment><USER70> The least/most you can do in this space is likely contribute actively in something you believe in.

Industry wide limitations are largely down to policy & regulations

</comment><comment><USER71> Make programming accessible to scientists.

Today, running any compute/memory intensive experiment requires working through so many tools and terminologies (e.g. AWS, SSH, python, dependencies, virtualenvs). Scientists really need just Jupyter notebook like experience that's collaborative, reproducible, and powerful (i.e. run on any hardware with 1 click).

<comment><USER72> Check out [https://www.dominodatalab.com/](https://www.dominodatalab.com/) \- you've pretty much described their product.

</comment></comment><comment><USER38> > how one can accelerate scientific research

Easy peasy. Help (for example) this guy find funds for continuing his research: [https://scholar.google.com/citations?hl=en&user=Q0w_e84AAAAJ](https://scholar.google.com/citations?hl=en&user=Q0w_e84AAAAJ)

Edit: there are many like him.

</comment><comment><USER73> You mention meta.org, you know you could work with us on it [https://chanzuckerberg.com/join- us/openings/?team=engineerin...](https://chanzuckerberg.com/join- us/openings/?team=engineering) (search "meta" on that page)

</comment><comment><USER74> Shameless plug: come work for Primity Bio. My team builds a bioinformatics web application that (among other things) is several orders of magnitude faster than the other tools, which fundamentally changes the scope of analysis that researchers can feasibly do. Email in bio.

</comment><comment><USER75> to me, the biggest hurdle is the pretentiousness in the research world and the way paper is written. if someone can rewrite papers in plain english, removing all jargons, talking more about intuitions, that will help encourage more people to get involved in scientific research.

</comment><comment><USER76> Here's a journal for publishing null results: [http://www.jasnh.com/](http://www.jasnh.com/)

</comment><comment><USER6> Probably not the answer you would expect, but one very simple action you could do is to sign/share this petition: [https://publiccode.eu](https://publiccode.eu).

</comment><comment><USER77> I think you might enjoy this interview with Elon Musk:

Elon Musk - How to Build the Future

[https://www.youtube.com/watch?v=tnBQmEqBCY0](https://www.youtube.com/watch?v=tnBQmEqBCY0)

</comment><comment><USER78> The job role for you might be as a "Research software engineer"

<comment><USER79> Surprised this comment isn't higher.

OP does indeed sound like the discipline they're looking for is Research Software Engineering.

Some reading: [https://en.wikipedia.org/wiki/Research_software_engineering](https://en.wikipedia.org/wiki/Research_software_engineering) "Research software engineering is the use of software engineering practices in research applications. The term started to be used in United Kingdom in 2012, when it was needed to define the type of software development needed in research. This focuses on reproducibility, reusability, and accuracy of data analysis and applications created for research."

[https://software.ac.uk/](https://software.ac.uk/)

[https://rse.ac.uk/](https://rse.ac.uk/)

</comment></comment><comment><USER80> Put yourself in survival conditions and use science to overcome it

<comment><USER66> Well nature did this to me and the committment to accelerated operation of a wounded laboratory (before it could have been lost altogether) yielded better progress IMHO on the same scientific instrumentation compared to leading PhDs in my field.

I wouldn't recommend it for everyone seeking commercialization because of accompanying roadblocks due to the systematic financial pressure against individuals coming from survival conditions.

But it is good knowing what this gear can really do.

</comment></comment><comment><USER81> Run BOINC, Folding@Home, or like distributed computing project. Many of those projects are run by education institutions.

</comment><comment><USER82> Sign up for a prospective health cohort such as AllOfUs in the US or UK Biobank or equivalents in other countries.

<comment><USER82> [https://allofus.nih.gov/](https://allofus.nih.gov/) [https://www.ukbiobank.ac.uk/](https://www.ukbiobank.ac.uk/)

<comment><AUTHOR> Thanks! Since I'm in EU, I'll also contact biobank and ask if I can help with anything software-related.

</comment></comment></comment><comment><USER83> an idea: create quality libraries for working with genome data.

</comment><comment><USER84> Start a business

<comment><USER43> If you can find a problem, and solve it, a business is a fantastic way to make an impact!</comment></comment>


Protect your downstream clients/upstream services with Istio's circuit breaker - <AUTHOR> https://banzaicloud.com/blog/istio-circuit-breaking/ <comment><USER2> Is this an advert?</comment>