US government commits to publish publicly financed software under FOSS licenses - <AUTHOR> https://k7r.eu/us-government-commits-to-publish-publicly-financed-software-under-free-software-licenses/ <comment><USER2> It seems like they've cut a wide swath for (probably) exempting the most interesting of the software that could have been developed in the open.

> Applicable exceptions are as follows:

> The release of the item is restricted by another statute or regulation, such > as the Export Administration Regulations, the International Traffic in Arms > Regulation, or the laws and regulations governing classified information;

> The release of the item would compromise national security, confidentiality, > or individual privacy;

> The release of the item would create an identifiable risk to the stability, > security, or integrity of the agency's systems or personnel;

> The release of the item would compromise agency mission, programs, or > operations; or

> The CIO believes it is in the national interest to exempt publicly releasing > the work.

[1] [https://sourcecode.cio.gov/Implementation/](https://sourcecode.cio.gov/Implementation/)

<comment><USER3> The important thing is that this changes the default. Instead of the default being closed and needing to justify open-sourcing something, the default will be to open-source things and having to justify keeping something closed. That's huge.

<comment><USER4> It pushes the incentives around as well, though. Unless there's some sort of marginal cost for each line of code they choose to keep proprietary, they'll inevitably do "the least work necessary" when they realize that some project is going to have components fitting these clauses, by by boxing up the entire effort under one of these clauses.

Ideally, with some sort of force _opposing_ propriety, the incentive would be such that the government would think it in its best interests to _granularize_ their code into small libraries, programs, and network services, so that they could open-source as much as possible, leaving tiny "classified kernels" as wrappers for libraries or plugins for apps.

For an idea of such a force: maybe create a "proriety credit" that a government department would to buy for each 1000 lines of code they refuse to release, whose funding goes to this effort to open-source everything else?

<comment><USER5> LOC-based metric tied to money... What could go wrong?

<comment><USER6> An overabundance of semicolons per line? A complete abandonment of python?

<comment><USER7> One-liners that would kill Perl masters by heart attacks half from envy, half from horror

</comment></comment></comment></comment></comment></comment><comment><USER8> I presented on this three years ago to the state government here in Michigan. It makes far more sense for the state's to do this than the Feds.

Why should there be fifty different software programs to register for a state campground? Or deal with the DMV? When the Feds pass a new law some contractor ends up writing the same software fifty different times for each individual state with small differences.

I believe the key is to have a plug in architecture, like WordPress, that allow each state to adopt the program for their individual laws. I think the savings could be tremendous and its entirely possible with multiple states collaborating the software could be continually improved.

I failed to convince them that day but I think the tide is turning.

<comment><USER9> Intuitively makes sense until you realize that there are similarly a lot of line of business apps that are used and implemented again and again across federal agencies.

For example, every classified document has a classification level (Secret, Top Secret, with more details on it). And the functionality to roll up the classification is re-implemented by every contractor every single system. So even within ONE agency, every system's implementation is done from scratch.

It's the perfect kind of thing that could've been an open source library (still in a classified network though), but still extremely valuable. It's something I built and recommended to my superiors ten years ago when I worked for a government contractor, but we didn't get much traction for various reasons. (Cynically, most contractors want the free money to reimplement it, so they aren't going to stop that gravy train.)

So there is a LOT of room even within each single agency, to share code that is common to their business practices.

You'll be hard pressed to find an area in any large organization (public or private) that couldn't benefit from the efficiencies of reuse.

</comment></comment><comment><USER10> Why is this limited to software?

Why not also publish publicly financed: * legal agreements * HR contracts * management presentations, best practices, and training material * marketing presentations and training material * creative output such as artwork and music

so that the rest of the world can use them for free..

<comment><USER11> > creative output such as artwork and music

If I'm not mistaken, all intellectual property published by the US government is public domain by law. I may be wrong or you may be referring to something else.

<comment><USER12> Close. It's not publication by the government that is important. It's authorship.

If a work is a work prepared by an officer or employee of the United States Government as part of that person's official duties, then it is not subject to copyright in the US.

It might be subject to copyright in other countries, though.

If the government pays for a work to be created and published, but it is not a work prepared by an officer or employee of the United States Government as part of that person's official duties, then it will be copyrighted and the copyright owner will be whoever it would be normally. If there is an agreement between the author and the government to assign the copyright to the government, the copyright will still be valid, and owned by the government.

<comment><USER11> I suppose you're right, but isn't that simply an issue of contract negotiation? Often when a private actor hires a contractor to create IP, part of what they're buying is ownership of the IP itself. I have no idea if the US government does this or not.

</comment></comment></comment></comment><comment><USER13> (Next step, open source the government policy, procedure, etc.)

Laws like requiring all commits to be submitted by the author. Authors must disclose potiental conflicts of interest, affiliations, etc. Submitting commits on behalf of another entity is treason; meaning if a funder wants a policy written, they post the funds, select another entity, selected entity assigns staff, staff submits policy, etc.;

All public, all real-time.

</comment><comment><USER14> Licensing problems. For new projects this sounds great, put the code either in the public domain or they pick a license. Which one? MIT/BSD, GPL/LGPL, or something else? The former can generally be incorporated into the later, but what if they pick something else that's incompatible? At least we can see what was done and learn from it I guess. And what if they require a particular license as part of the policy and it prevent government projects from extending existing OSS project? That would be a shame.

The intent seems good, but the details will be everything.

<comment><USER15> GPL is discouraged in government software (speaking from DOE/NSF experience) but BSD or similar is encouraged. The path of least resistance is private/closed source, followed by open source BSD/MIT/Apache, followed by GPL/LGPL, followed by commercialization.

The reason why there is a bit of resistance with open source as a default is because funding has to be traced.

</comment></comment><comment><USER16> Heh. I submitted a FOIA request two weeks ago for code for a $100M system used for storing parking ticket data. They have 5 days to ask for an extension, but they haven't even responded with that.

Guess I'm going to lawyer up again..

<comment><USER17> What is the penalty for not responding in time?

<comment><USER16> For Illinois, two big things - 1\. They can't reject the FOIA with "unduly burdensome" anymore, which is an incredibly common and frustrating response. 2\. They can be sued for not complying, which then becomes a slow drawn out process. The upside is that it's something like 8k if the judge thinks the request was rejected/ignored intentionally or without proper followup.

Example - I did a FOIA that was rejected absurdly unfairly. Going through the attorney general took about 6 months to hear a response back, which Chicago's lawyer then used essentially a mistype to throw that request out. 8 months in total there. Fast forward 6 months later and we have a court date with a lot of small things in between then and now.

<comment><USER18> Why do these get rejected? Is it just people being lazy? Or people not understanding how to handle the request? Seems like a lot of work for them to go to court?

<comment><USER16> A few things -

1\. Definitely a lot of laziness from the FOIA officers. Some are really great, but others not so much.

2\. There's no standardization in how it's handled within the city. Departments in the same buildings have very different idea of what should be rejected, their helpfulness, looseness of FOIA interpretation, etc.

3\. They're truly not technical in the slightest. At one point I asked for a database dump, excluding sensitive columns and tables. The response was "Wikipedia defines a Data as:", so I had some fun explaining that one.

4\. Fighting the process takes a very long time, so a rejection is an easy way to get someone to go away.

5\. Not a reason, but still annoying.. They wait the maximum days to respond to your requests. Of the 50 or so I've submitted, I'd say 45 of them have been last minute. 25 times out of the 50, sure... but most?

It's a lot of work if it gets to that point, yep, but I've only ever needed my lawyer to file a suit. The average FOIA might take me 5 minutes to write, so it's not really that much time - especially considering what you get for it.

Here's an example one with some pretty neat data from it: [https://github.com/red-bin/chitickets](https://github.com/red-bin/chitickets)

<comment><USER18> That's fascinating. It sounds like mostly the requests are rejected because of lack of time / lack of skills / laziness then?

<comment><USER16> Yep. (This post got a lot longer than I was hoping. Hope you find it interesting!)

That ticket data is actually managed by IBM, so that even adds an annoying translation dynamic.

This current suit is because the city thinks it's 'unduly burdensome' to resolve city-owned phone numbers to names that were dialed by Chicago's mayor. Initially I could have understood that, but it became pretty clear pretty soon that they really didn't want me to have those records with these rejections:

Some more detail on the rejections I've been through this past year and 4 months:

-Original request asked for the mayor's records. Rejected for "no such records". Heh.

-Second was for the FOIA officer's phone records. Again, "No such records". (This was just a test to see if the mayor's phone was the only one without any records..... and it was kinda funny ;))

-Sent a request to Chicago's IT dept for their VOIP logs, which got me a file with the first 6 digits of all of non-government phones that were dialed. Government phones were completely redacted for "privacy" reasons. So, so many things wrong there.. especially since FOIA specifically disallows that sort of redaction.

-Sent a "Request For Review" to the attorney general's office, who took 6 months to agree with me and told the FOIA officer to give me the records. Chicago's lawyers then threw the VOIP request out, since they don't have VOIP. The assistant attorney general on the case and my lawyer both said (paraphrased) "They're right, but that's a total dick move and they know it."

-Submitted a cleaned up request, without "VOIP". They sent the same file from before. Same creation time and everything.

-Now we're waiting through the long court process...

About the attorney general's RFR process, though..

You can submit FOIA requests anonymously, but you can't submit RFRs anonymously! Their reasoning was that I need to have a valid signature with my named spelled out in plain text, despite ESIGN allowing pseudonyms. So - don't like that anonymous person? Just don't respond to their request and they'll either have to sue using their name or submit an RFR using their name. I fought that one for about a month and eventually submitted a FOIA request to the AG's office asking for the email chain that led to that decision. That got rejected with something like "deliberation discussions cannot be released."

<comment><USER18> That's really interesting. Sounds fairly hostile. Although I guess from the opposing party's perspective, there is no advantage to sharing information, only potential disadvantage... and its not something you can undo.

<comment><USER16> Not sharing information gets in the way of progress and established laws. Politics really shouldn't have anything to do with it, honestly. But since it does, collectively as a group, we should be doing more besides just voting and walking on pavement ;)

I got involved with this after working with one of the old mayor candidates, Amara Enyia. The plan was to give her the parking ticket data which would be used for her campaign. The day we were supposed to meet, two of the candidates and an ex-senator effectively colluded to get her kicked out of the race by questioning her signatures. The guy who submitted it was related to Dock Walls and friends with Ricky Hendon, iirc. Her campaign manager then went to Bob Fioretti's campaign, where they invited me over, where I gave Fioretti's campaign manager and Amara's campaign manager the full data. Fioretti's manager called it "fucking golden"..... I never heard back. A bit later, he made parking tickets a big part of his campaign.

After seeing Dock Walls, Ricky Hendon and Willie Wilson colluding to get Amara out of the race (seriously, they talked about it on facebook, publicly), it seemed a little suspicious that Fioretti's team didn't bother responding, considering how much they liked it. The phone records request was just a probe to see if the collusion ran deeper. Whether it's true or not isn't the point since, 1. I don't care enough about politics. 2. It's the principal of the entire thing.

For what it's worth, Fioretti dropped out of the race and gave Rahm his support the weekend before the election. :)

<comment><USER18> I agree that not sharing is problem. Sometimes when I describe the opposition's point of view it sounds like I am defending their position. I think in order to fight an opponent, you have to understand their point of view.

Your story is concerning, but I'm not surprised. In my opinion, the system itself filters for this sort of behavior - and it gets worse in the upper echelons of the competition because in the upper echelons it becomes less and less feasible to deviate from an "optimum approach". My assumption being that the "optimum" approach includes participation in unfair and antisocial techniques.

<comment><USER16> Sorry - didn't meant to come off as aggressive or combative.. There's just a surprising amount of hostility from those who like Rahm or think Rahm is a built-in in Chicago. "Why don't you just move?" and calling me unpatriotic were pretty interesting to hear.

The idealist in me thinks those behaviors can be fixed even though it's "turtles all the way down". Though, if enough people do enough tenaciously peaceful civics work, then hopefully things will get slightly better. By treating everything as a mystery as [0] and [1] talk about, things become manageable.

[0] [http://www.smithsonianmag.com/people-places/risks-and- riddle...](http://www.smithsonianmag.com/people-places/risks-and- riddles-154744750) [1] [http://www.newyorker.com/magazine/2007/01/08/open- secrets-3](http://www.newyorker.com/magazine/2007/01/08/open-secrets-3)

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> Looking forward to [https://github.com/nationalsecurityagency/PRISM](https://github.com/nationalsecurityagency/PRISM) et al

<comment><USER20> It's funny you say that because that's the one NSA did voluntarily:

[http://www.informationweek.com/applications/nsa-submits- open...](http://www.informationweek.com/applications/nsa-submits-open-source- secure-database-to-apache/d/d-id/1099972)

[https://accumulo.apache.org/](https://accumulo.apache.org/)

;)

<comment><USER21> Don't forget [https://github.com/simp](https://github.com/simp)

<comment><USER20> Oh yeah, I did forget about that one. Page is kind of clutterer but the glance makes me think this was the tool that automated & checks hardening guides. Am I right?

And does it substitute for system monitoring/config tools that are popular or just complement them?

</comment></comment></comment></comment><comment><USER22> This has been a UK government stance for a while but the problem is that while the UK gov spends something like 8BN pa on software _licenses_ the amount of new development is fairly low (outside of say the GDS - central cabinet office group pushing all this)

So for example, my local council wants to purchase software to help track customer feedback from dozens of sources (web sites, phone ins etc). Sounds perfect for a web era, message queue based project.

But my council is only interested in meeting its fiduciary duty in buying software that already exists, and has been proven to do the job required. Speculative development is out - even though pretty much everyone of 430+ councils, parishes, towns etc of the U.K. Will want something similar in the next few years.

No one is willing to stump up the cash to start the open source alternative.

It's a tragedy of the not commons.

I think there ought to be solutions in some kind of development bonds or other financing arrangements - but it's not clear how.

<comment><USER23> > This has been a UK government stance for a while

As further confirmation, it's cemented as one of the ten design principles behind GOV.UK: [https://www.gov.uk/design- principles#tenth](https://www.gov.uk/design-principles#tenth)

<comment><USER22> Yeah but still... No way to get to the majority of IT spend - even if that IT spend is wasted on licenses costing more than the cost of the capital build out

</comment></comment></comment><comment><USER24> A great start. Now if only the same would be required of all code arising from federally funded research, we could massively accelerate research and development in America.

</comment><comment><USER25> Obligatory mention of 18F ([https://18f.gsa.gov](https://18f.gsa.gov)) within the General Services Administration who develop federal government projects in the open (and have defaulted to open-sourcing since their inception).

[https://18f.gsa.gov/2015/01/16/open-source-for-good- governme...](https://18f.gsa.gov/2015/01/16/open-source-for-good-government/)

</comment><comment><USER26> > _The custom-build software will also be published to the general public > either as public domain, or as Free Software so others can improve and reuse > the software._

> _However, it is encouraged to be retroactively applicable for the existing > custom-build software developed by agency employees in the course of their > official duties._

As much as I love FOSS, anything produced by the government — including software — is public domain, _right_? The government is not eligible for copyright — including FOSS licensing. While I understand that the government sometimes contracts stuff out, and those third-parties seem to enjoy a weird exception to the rule, the article seems to be including stuff developed directly by the government for the government. How do they think they have any rights to it at all?

<comment><USER27> Public domain in the USA, but automatically copyrighted in practically every other country in the world.

</comment></comment><comment><USER28> Several companies have been started by using software developed by people working in governments and it's only fair as it's all funded by the taxpayers

Because many governmental projects are politically sanctioned, done properly, this could become a great way to do R&D where the private market doesn't.

</comment><comment><USER29> I hope this includes voting software.

<comment><USER30> I could be wrong, but in most cases the US government buys appliances from companies like Diebold and they don't develop or contract software for voting. The best that can be done is regulation to force audits of the appliances like what Nevada does with casino slots. Even that would be a huge step up.

<comment><USER13> Might be a reason to long-term require all software used be open source and for that matter make it easy for a user to confirm that the system in use real-time is the system that's open-sourced too.

<comment><USER31> How do you allow a user to confirm this in real time? Hash displayed on th screen?

<comment><USER13> Yes, that's correct, being able to download the code, hash it, and then compare it to the hash displayed real-time would be a first step towards enabling this.

<comment><USER32> the software can easily be modified to display the hash you are expecting

<comment><USER13> To start, something is better than nothing. Long-term remote inbound two part authitacation combine with like measures for hardware via meta-programming would make bypassing this very, very hard for the average hacker.

</comment></comment></comment></comment></comment></comment></comment><comment><USER33> Wow, this is great. I hope it prevents another contractor-related debacle like healthcare dot gov.

<comment><USER34> There was a lot of things that went into that. There was a huge time crunch on the project and until the bill was signed there was only so much groundwork that could be laid because you could only make passable guesses at the final requirements.

</comment></comment><comment><USER35> If this actually works, wow

</comment><comment><USER36> Floating points.... Floating points everywhere!

</comment><comment><USER37> It exempts prior work from this clause though it says it encourages vendors to adhere to it. I have a hard time seeing that anything really interesting will be ever released because it will likely have national security exemptions in it and like taxes many companies will find a way to avoid it.

<comment><USER38> Yeah.. just about anything can be declared under the umbrella of "national security" if you reach hard enough.

For a true to life example, look at the NSA.

</comment></comment><comment><USER39> This is excellent as it puts something (sorta in writing). But the government only has a limited ability to retain copyright. If there is a wholly in-house developed program, one _should_ be able to get the source under a FOIA request.

</comment><comment><USER40> Let's start with open sourcing the effing voting machines.

</comment><comment><USER41> And after the first data journalist wave of analysis reveals an average of 10,000 redundant projects worked on by agencies at any given time, open access will go dark again.

</comment><comment><USER42> It is completely contrary to the notion of an open and free society that the government is permitted to spend taxed dollars developing _anything_ meant for domestic use in secret, much less it being (more or less) the default policy.

Looking at the exemptions carved out of this rule (assuming this source is accurate), this looks more like it is codifying the secret nature of government operations than it is opening up anything substantial.

Hard to regard this as good news, although it may be another straw on the back of the government finally falling and the emergence of an _actual_ open and peaceful world.

<comment><USER43> How exactly is I "contrary to th notion of an open a d free society." I think that's just nonsense. And you think this could only be good if it leads to the government failing and a failed government obviously leads to world peace?

I think some just let irrational hatred make them believe inane fairy tales.

<comment><USER42> The basis for transparency is government, at least in the common law tradition, is the understanding that without it, government actors will tend to act in their own interest rather than in that of the public.

Do you think that this is a reasonable belief?

If so, then it logically and obviously follows both that 1) development of products by the government (to the extent that this is even the reasonable purview of information age government at all) need to be conducted in a profoundly public way and that 2) a failure to do this will result in a properly disaffected populace and shortened moral stature on the part of the government.

It is no fairy tale to suggest that secrecy in government is a substantial part of the formula of government failure more generally. The only assertion I'm making that might not yet be in evidence is that, amidst the information age, this aspect of good government is more important and that its effects will be realized far more quickly.

</comment></comment></comment><comment><USER44> I want more State government to adopt this policy

</comment><comment><USER45> I believe it'll be BSD licence

</comment><comment><USER46> What percentage of US government software is _currently_ developed as FLOSS?

</comment><comment><USER47> One of the current issues is that government-written code is pure public domain, so any contractor (SAIC) can FOIA some code the know their DoD handlers want implemented (ViSTA, written by the VA) and charge $4B. That is a tragedy of the commons if I ever saw one.

</comment><comment><USER48> I want the source code of government cryptography programs.

<comment><USER49> You do realize the security lies with the keys/passwords used to encrypt, and not in the program/algorithm itself?

</comment></comment><comment><USER50> Great news.

</comment><comment><USER51>
        >npm install xkeyscore

        npm ERR! 404 'xkeyscore' is not in the npm registry.
        npm ERR! 404 You should bug the author to publish it

Hm.

<comment><USER52> It probably depends on left-pad</comment></comment>


Kylin: OLAP engine for Hadoop from eBay - <AUTHOR> http://kylin.io

<comment><USER2> The integration with existing BI tools is a big deal. This eliminates the need to export a subset of data into a relational DB for OLAP.

Incremental refresh of cubes is super important especially when working with large data sets.

The ACL support and LDAP integration looks nice. Proper access control on top of BI data is an absolute requirement.

I don't see anywhere this is listed explicitly, but I would guess this works with data in HBASE not individual files stored in HDFS.

<comment><USER3> The Cube File is stored on HBase now, it could be extend to otehr KV value system.

</comment></comment><comment><USER4> HDFS is used to store intermediate files for building cubes.

HBase is used to store data cube and execute the query. HBase co-processor is also used for query processing.

</comment><comment><USER5> I can't seem to find the documentation.

<comment><USER3> the document will coming soon, there are many stuff we have moving from internal wiki to kyliy.io now.

</comment></comment><comment><USER6> what could this mean "extremely large petabytes of data analysis." ?

<comment><USER7> A good sign of 'enterprise' software: the marketing copy was written by someone from Marketing.

And then maybe translated to english by someone from accounting

<comment><USER8> And unthinkingly approved by someone from sales.</comment></comment></comment>


Slowing Down - <AUTHOR> http://leostartsup.com/2012/05/slowing-down-2/

<comment><USER2> I can't overstate how important I think these sorts of habits are. A slow walk. A quiet meditation upon waking. In aggregate, they can really change your life. The problem is, these habits aren't easier to develop than any other. They take practice and time to become innate.

We "make time" for all sorts of destructive habits. I gave smoking hours of each day for a decade. Anxiety, worry, hurry, they'd run my life for more than that. Don't set out trying to replace them. Make it OK to be a few minutes later walking home; a few moments later to work. Make time for these good habits and the bad won't stand in the face of them.

</comment><comment><USER3> Reminds me of the studies done of walking pace in cities. Researchers discretely measure off a known length stretch of sidewalk and then secretly time a bunch of people walking it, with interesting results ("people in fast- moving cities are less likely to help others and have higher rates of coronary heart disease").

Radiolab podcast from 2010: <http://www.radiolab.org/2010/oct/08/>

Paces in different cities: <http://www.richardwiseman.com/quirkology/pace_home.htm>

</comment><comment><USER4> This is a very interesting exercise, I have to try it some time this week!

Here's another one: Go to a quiet place, outside. Maybe to a forest. You should be alone. Now stop and listen. Try to identify all of the sounds you hear. Try to hear more, quieter sounds. Do this for several minutes.

For me it's always amazing how much more sound there is than you'll initially hear. And how I get bored first and then, after a while, can't stop listening.

Edit: Spelling.

</comment><comment><USER5> It's the reason why i prefer walking rather then bike

</comment><comment><USER6> His smugness about the older woman is infuriating.</comment>


Mac OS Crash with CSS (WebKit DoS) - <AUTHOR> https://gist.github.com/pwnsdx/ce64de2760996a6c432f06d612e33aea <comment><USER2> As the user draak567 points out:

> Chrome doesn't crash because the backdrop-filter CSS property is not enabled > by default. You can turn it on by enabling "Experimental Web Platform > features" in Chrome flags, at which point Chrome will also freeze or crash, > at least on all the devices I've tried.</comment>


High Tech Cowboys of the Deep Seas: The Race to Save the Cougar Ace (2008) - <AUTHOR> http://www.wired.com/science/discoveries/magazine/16-03/ff_seacowboys?currentPage=all#

<comment><AUTHOR> Its an old one, but such a great article from Wired (which is a rarity these days).</comment>


Meet Coinye West, the Kanye-inspired Bitcoin alternative for normal folk - <AUTHOR> http://www.pcworld.com/article/2083761/meet-coinye-west-the-kanyeinspired-bitcoin-alternative-for-normal-folk.html

<comment><USER2> When I think of 'normal folk' Kanye is not the first person that comes to mind.</comment>


Why biology students should learn how to program - <AUTHOR> http://scienceblogs.com/geneticfuture/2009/03/why_biology_students_should_be.php <comment><USER2> Biology students should definitely learn how to program.

I'm in my 4th year as a bio major and have been involved with several research projects. The first involved inducing schizophrenia in rats, and recording their movements in reaction to a set of audio stimuli after being given a set of varying medicines. We had folders and folders filled with spreadsheets of data -- data that would have to be manually inputted into some crappily obsolete software. Even with somewhat specialized software, each run was saved as a separate file, adding to the clutter. Our research would have definitely progressed more quickly had we had programmers that built the appropriate software to aggregate our data.

In my second research project, our team was trying to, and still is, find the metrics to 'the beautiful face.' This requires morphing pictures of women together, presenting these synthetic images to a focus group, who then rates the images. After the pictures are rated, they are subject to a natural selection type algorithm we developed, and the beauty evolution is done manually. We would've definitely found much value in recruiting a programmer who could've written a program to automate the selection process, and perhaps built an app that brought the focus group rating aspect of our research to the web. We do use hotornot.com, but it's not ideal.

I'd like to add that additionally, medical students and doctors should learn how to program. There are many many many areas in medicine and healthcare generally that are behind the tech times. Coding doctors would definitely expedite the progression of the medical field from within. At least I think so. That's what my med school personal statement was about. :P

As someone who has studied biology/medicine, and has a definite interest in programming (teaching myself currently), I hope to find myself helping in the progression of the convergence of these two fields.

<comment><USER3> I'd extend your comment to "Everyone should learn how to program". When you look at the number of repetitive tasks that take place in a typical office environment, it's amazing at how much time could be saved with a little programming.

Macros were designed to accomplish this, but they're still way to complicated to be used by most people. It would be great if there were some sort of easy to use third party system that could integrate with a number of products (Word, Excel, Websites) and provide an easy environment to build macros.

Edit: I'm basically thinking of OS X's Automator

<comment><USER4> Construction? Landscaping? Farming? Jobs with lots of secretarial assistance? I think you've taken it a step too far and ignored several sectors of society. Many scientists could benefit from programming, and anyone else who has to deal with information regularly probably could as well. That doesn't equal _everyone_.

<comment><USER5> Arguably VB for Applications could come in handy for almost anyone. Though interestingly most farmers, carpenters, and other handy-workers are more capable of performing arithmetic than the rest of the population. Still the power of a well-tuned spreadsheet is quite amazing.

Farmers are also a bad example these days, agriculture is totally high-tech.

</comment></comment></comment></comment><comment><USER6> Is there a compelling reason why the biologist and the programmer need to be the same person?

It looks to me like researchers have figured out specialization of labor: one well-educated researcher to make the theory, several low-wage grunts who get to feed the rats. (Whether this relationship is fair to graduate students is outside the scope of this post.)

I think we're suggesting people learn programming so that they can do tasks which are of comparative complexity to feeding the rats. Converting data from one form to the other? That is feeding rats. Iterating over data to produce summary statistics? That is feeding rats.

(I know biologists work with really big data sets. So maybe they're not feeding rats, they're feeding monkeys. Disgustingly parallel algorithms are getting easier and arranging the hardware for them is getting easier: in a few years, the monkeys will be about as easy to feed as rats are today.)

Biologists don't have grad students scratch-build their own microscopes, do they? That would be silly. Why not have the scientists and proto-scientists concentrate on science and outsource the feeding of computational rats to someone who doesn't really need a graduate degree in bioinformatics.

Plus, if using cheap outsourced rat-feeders achieves comparable quality to graduate students, there will be less market need for a permanent underclass of rat feeders with crushing debt loads and stipends that make McDonalds look generous.

<comment><USER7> Biologists don't necessarily need to be the one writing the high-end software tools, in the same way (as you note) that they don't build their own microscopes.

But re-formatting, filtering and running basic QC on complex data is something a lot of biologists have to do every day; and the types of data, their quality and their format are constantly evolving. Biologists who have to run down the corridor to their local (probably overworked and underpaid) programmer every time their data format changes, then explain the problem in terms the programmer can understand, then wait for the programmer to write something, and then check the script's output for errors, will be at a huge disadvantage compared to those who can quickly whip up and debug a parsing script themselves on the fly.

<comment><USER8> I couldn't agree more. Grant money is precious. Instead of spending three or four grad student salaries to pay a programmer a competitive wage, it makes more far more sense to fund three or four students on research assistantships where they have time to learn Perl.

</comment></comment></comment><comment><USER9> Math skill will make you a better scientists.

Programming skills could also be useful, but then again maybe not. Read about Richard Dawkins' dangerous addiction: [http://www.guardian.co.uk/theguardian/2007/feb/03/weekend7.w...](http://www.guardian.co.uk/theguardian/2007/feb/03/weekend7.weekend5)

The danger I see in teaching biology students how to program is that they are not, and don't want to be, good software engineers. And that's often worse then not knowing a thing about computers.

A little knowledge is a dangerous thing. Combine a little programming knowledge with a lot of intellect and no small amount of arrogance due the proudly earned Ph.D in biology or chemistry or what ever and some VERY bad things can happen.

I've seen databases where 99% of the data was NULL, because the Chemistry Ph.D author couldn't be bothered with trivial details like what's a one to many relationship. Naturally it was MS Access.

Beware of bad software engineering, even more so if the person coding is otherwise brilliant. That's the most dangerous type.

Software is a very sharp sword, you should use it carefully not try to swing it as hard as you can.

</comment><comment><USER10> Can't agree more. In my undergraduate research in the Cognitive Neuroscience labs at Berkeley, it was disappointing to see how much effort was wasted because people avoided technology instead of embracing it (in neuroscience!).

The work done there was incredible, but as I was passed around to researchers in the lab, programming the experiments they had designed (have to pay your dues to get into research), I got a chance to see the methods they used before they were able to get competent programmers. Manually marking spikes and anomalies in EEG data when there was an api for the program to allow access to recorded data for just such a purpose, dealing with timing issues because few understood the difficulties of achieving high-precision event timing, etc.

It was depressing to think of how much further it could have gone. When I left to pursue my startup, I was working on some automated techniques to remove noise from fMRI data. That type of work will speed up research and allow researchers to spend time on meaningful things.

</comment><comment><USER11> I've always been of the opinion that diversity in your education will help you somewhere down the road. That being said you should still have one thing you focus on. If you're going to be learning how to program as a biologist, you still need to be a biologist first and a developer second.

I'm a developer first and whatever it is I'm programming for second. I learn how to do the job of the people I write code for so it will do what they need.

</comment><comment><USER12> If two semesters of calculus are relevant to biology students, then surely at least one semester of programming is as well.</comment>


Photos of a Sukhoi Su-27 dogfighting with a USAF F-16 around Area 51 - <AUTHOR> https://theaviationist.com/2017/01/06/these-crazy-photos-show-a-russian-su-27-flanker-dogfighting-with-a-u-s-air-force-f-16-inside-area-51/ <comment><USER2> How come the US Air Force has Sukhoi aircrafts? Did they buy them from Russia? I can't imagine Russia selling their top aircrafts to an enemy.

<comment><USER3> And does Russia have any F-16s?

<comment><AUTHOR> I don't believe so, or at least I've never heard about Russian F-16s. Iran still flies some old F-14s though.

The USAF seems to be pretty strict about not allowing more modern fighters to end up in private hands. There are a number of F-86s, F-104s, F-5s and I believe one F-4 Phantom still flying around the US in the hands of private collectors and air museums but nothing newer like F-14s, F-15s, F-16s or F-18s.

Edit: While I've never heard of any F-16s in Russia I'm sure the Russians have had plenty of time inspecting and probably even flying F-16s, it was a very widely exported plane.

<comment><USER4> Does Russia have an equivalent of ITAR? If not, that could partly explain why US aircraft are less commonly sold.

[https://en.m.wikipedia.org/wiki/International_Traffic_in_Arm...](https://en.m.wikipedia.org/wiki/International_Traffic_in_Arms_Regulations)

</comment></comment></comment></comment><comment><USER5> I remember seeing a Su-27 do forward flips, backflips and cobra maneuvers during the 2001 Paris air show.

It looked pretty damn sweet - was the best part of the show for me.

<comment><USER6> I was fortunate that while serving in the USAF we had an alternate for the Thunderbirds on our base. Watching him fly was a good reason to just sit on the ball fields even in the hottest afternoons. One takeaway from a meet and greet was it was real fun for him and all those cool moves are just that, simply cool with little application outside of stunt flying.

</comment></comment><comment><USER7> Don't forget the Aggressor squadron. They play a similar role (tactics, strategy) but with different equipment.

[https://en.wikipedia.org/wiki/Aggressor_squadron](https://en.wikipedia.org/wiki/Aggressor_squadron)

</comment><comment><USER8> The dogfighting scenario is pretty much extinct.

The F-16 is sort of old by now, except the drone version, which is reportedly for use as a training target.

Probably it's a drone F-16 against a Su-27?

<comment><USER9> nah. as sensor packages and missile technology advances, so do countermeasures, while the good ole bullet is harder to fool.

"The F-35 Joint Strike Fighter proponents' riposte has been that the AIM-120 is a 100% reliable and 100% kill probability missile, so each F-35 carrying four AIM-120's will account for four Sukhois or MiGs. This is a dangerously naive assumption. Operationally, the AIM-120 has achieved 10 BVR kills from 17 shots – a Pk or "kill probability" of 0.59 (59 percent) against benign or "dumb" targets. The AIM-120 has been tested in over 200 test firings, with a reliability of about 85 percent, so statistically speaking, about one of those four AIM-120s will be a dud." > [http://www.ausairpower.net/APA- NOTAM-070109-1.html](http://www.ausairpower.net/APA-NOTAM-070109-1.html)

also, once the 6-10 aa missiles you carry are spent, if you aren't faster than the enemy you cannot just disappear.

<comment><USER10> >'Operationally, the AIM-120 has achieved 10 BVR kills from 17 shots – a Pk or "kill probability" of 0.59 (59 percent) against benign or "dumb" targets.'

Fyi, frequencies are not probabilities. That was one of the points that, as he neared the end of his life, lead Ronald Fisher to say that mass confusion over stats would be the downfall of the next set of nations:

"We are quite in danger of sending highly trained and highly intelligent young men out into the world with tables of erroneous numbers under their arms, and with a dense fog in the place where their brains ought to be. In this century, of course, they will be working on guided missiles and advising the medical profession on the control of disease, and there is no limit to the extent to which they could impede every sort of national effort." Fisher, R N (1958). "The Nature of Probability". Centennial Review. 2: 261–274. [http://www.york.ac.uk/depts/maths/histstat/fisher272.pdf](http://www.york.ac.uk/depts/maths/histstat/fisher272.pdf)

Here is another good paper from him on that: [http://www.phil.vt.edu/dmayo/PhilStatistics/Triad/Fisher%201...](http://www.phil.vt.edu/dmayo/PhilStatistics/Triad/Fisher%201955.pdf)

Chapter 9 here also covers it from a different perspective: [http://omega.albany.edu:8008/JaynesBook.html](http://omega.albany.edu:8008/JaynesBook.html)

<comment><USER9> ikr, but there is no much more operational data beyond that. also, not my quote, and the point was just that at some point you need a fallback weapon

<comment><USER10> I had never seen evidence of the "guided missiles" part of Fisher's prediction before. In medicine it is obvious...so maybe I am just not familiar, is there mass confusion in weapons research too?

<comment><USER9> don't know about missiles but if we include rocket science in general there is plenty literature on how they cheated statistics to get the shuttle going [https://science.ksc.nasa.gov/shuttle/missions/51-l/docs/roge...](https://science.ksc.nasa.gov/shuttle/missions/51-l/docs/rogers- commission/Appendix-F.txt)</comment></comment></comment></comment></comment></comment>


Game Companies Oppose DMCA Exemption for 'Abandoned' Online Games - <AUTHOR> https://torrentfreak.com/game-companies-oppose-dmca-exemption-for-abandoned-online-games-180217/ <comment><USER2> "We don't think there's enough interest in the game to make money by still running servers for it, but we also don't want anyone willing to run servers to be able to do it either."

Pick one, either you run the servers or you let someone else do it. (IMO, IANAL, etc)

</comment><comment><USER3> When I'm in a nursing home and suffering from dementia, I hope I can go back and play some of the games I know so well. A whole second time around.</comment>


Mapquest using OpenStreetMap - <AUTHOR> http://open.mapquest.com/

<comment><USER2> When I visit that page it tells me "OpenStreetMap powered by mapquest". Shouldn't that kind of be the other way around? I mean, in this case, mapquest is using the maps from Open Street Map, so IMHO, I'd say that OSM is powering Mapquest here.

Not that it matters much.

I _do_ love Open Street Map though, also because it's so easy to contribute. I could very easily improve the map around where I live. One particular street for example has been closed off two years ago and OSM is _still_ the only map that gets it right.

<comment><USER3> Also.. not particularly surprising. MapQuest hired one of OSM's community managers this summer. Good to see collaboration and innovation on top of good data.

</comment></comment><comment><USER4> The labels are so crazy, how can Elizabeth be shown instead of New York?

<comment><USER5> The labels are choosen automatically by software. It is a bit of an outstanding issue has to how to make that work correctly with USA cities. The whole data set is a wiki, anyone can edit anything. It's not obvious how to place the labels in an objective manner.

<comment><USER6> There's some info here about how the US concept of "city" clashes with the UK concept, which was used as the basis of some data naming decisions, and the effect it has on the US map.

<http://compton.nu/2010/10/city-labels-in-openstreetmap/>

</comment></comment></comment><comment><USER7> This could be really useful. My university campus for example, is shown in much greater detail on OSM.

<comment><USER8> The various hiking and biking trails that I use are likewise far better detailed on OSM. They also have better maps of neat stuff like train tracks. Down to individual tracks at trainyards/stations in some cases.

</comment></comment><comment><AUTHOR> It would be great to have all the biking or hiking trails on the map.

I went mountain biking a few months ago and the maps were hand drawn on a piece of paper. It was embarrassing when I got lost.

<comment><USER9> If you zoom in far enough the trails are visible. Obviously, if the trail isn't mapped yet it won't show up. Feel free to add the trails that are missing!

</comment></comment><comment><USER10> Has some serious issues updating tiles. When I zoom in all the way, it's totally hit or miss if I get image or grey gears.

</comment><comment><USER11> Open... and wrong. It thinks my address in in a city 60 miles away.

</comment><comment><USER12> The easy interface to map-data editing is quite nice.</comment>


Mozilla might launch Firefox premium in 2019 - <AUTHOR> https://www.ghacks.net/2019/06/09/mozilla-might-launch-firefox-premium-in-2019/ <comment><USER2> Jeez, the comments on the linked article are abysmal. If Firefox release a premium version, I'm very open to paying. I prefer to pay for things I find useful and a browser is arguably one of the most important programs on a modern machine.

</comment><comment><USER3> Quick search, here's a 2017 annual report.

> Today, the majority of Mozilla Corporation revenue is generated from global > browser search partnerships, including the deal negotiated with Google in > 2017 following Mozilla's termination of its search agreement with Yahoo/Oath > which required ongoing payments to Mozilla that remain the subject of > litigation.

[https://www.mozilla.org/en- US/foundation/annualreport/2017/](https://www.mozilla.org/en- US/foundation/annualreport/2017/)

Not a great position to be in. I would pay for a premium version.

</comment><comment><USER4> I'm actually genuinely excited for this. Although Firefox's VPN should be free like Opera's[0] or even freemium where users get a more advanced set of VPN features for a small fee each month, and for the free tier, a limited set of options like limited choice of countries, and a bandwidth quota.

[0] [https://www.opera.com/computer/features/free- vpn](https://www.opera.com/computer/features/free-vpn)

</comment><comment><USER5> I'd pay for it.

</comment><comment><AUTHOR> i find this distressing but not unforeseen. im hoping mozilla wont fall down the same we need more revenue slope as a lot of others have before they went full subscription model.</comment>


Apple and Aetna hold secret meetings to bring Apple Watch to millions customers - <AUTHOR> https://www.cnbc.com/2017/08/14/apple-aetna-secret-meetings-to-bring-apple-watch-to-millions-more.html <comment><USER2> Apple, are you sure you want to be associated with the likes of Aetna?

Do a bit of historical research...

And beware becoming associated with "wear this or else" bad publicity.</comment>


Dutch Startup Innovates Bras: Interview with Double Dutch Design - <AUTHOR> http://startupjuncture.com/2014/08/15/double-dutch-design/ <comment><AUTHOR> What do you guys think about this crowdfunding campaign?</comment>


Show HN: Stumbleupon for Inspiration - <AUTHOR>

My mate and I recently built a new website that aims to present quotes and videos in a cool, visual way. Imagine BrainyQuote but visually more compelling.
We've just added a feature - stumble. Users can now stumble their way through inspiration. Simply click this link - http://getinspired365.com/stumble and press the right and left arrow keys. If you feel like you are in need of a bit of motivation it is a great way to spend 5 minutes. If there is a particular topic that interests you, then click a tag and you can stumble through inspiration just for that tag.
thanks! <comment><USER2> I like the idea! My friend and I had a similar idea at one point that we quit working on, but it was a bit more motivation oriented:

<http://stonecoldhustler.com/>

</comment><comment><AUTHOR> clicky - <http://getinspired365.com/stumble>

</comment><comment><USER3> I like the idea, and the design is fantastic, but I question whether the quotes you chose are thought provoking (and maybe even preachy to some) rather than inspirational.

<comment><AUTHOR> Thanks for the feedback. We normally go for a topic (event, person etc) on a particular day - a topic that has relevance to that day. I then select appropriate quotes for that topic and the quotes are determined by which ones are most popular on GoodReads. Will review them though, thanks.</comment></comment>


Magic Lantern - <AUTHOR> https://www.magiclantern.fm/ <comment><USER2> Recently I got a second hand "Canon EOS M" it's an old Mirrorless, for only $200 USD.

Thanks to ML I can get footage at 2.5K in 12-bit RAW, or pseudo-4K in an anamorphic format. If you're a film student in a small budget, the amazing work from the people behind the project it's a gift from heaven, as the quality it's almost film-like (ready for regular LUTs).

Basically you can have RED or BlackMagic Camera color and HDR kinda-quality with a small budget... and a lot or patience and free time.

Still tho... I would not use ML RAW for a client in production projects (like a wedding or commercial), as most builds are in a "Always-Beta" state, with a lot a bugs and slow editing workflow for a professional setting. So in those cases, I would recommend record in old h264 in a flat profile, and use all other ML features like zebras, and live histogram.

Kudos for the ML Devs, who against all logic, are giving ton of work for the indie film community.

<comment><USER3> HN tip of the month ;)

When Canon released the EOS M as its first mirrorless camera, it hasn't gone down well in many reviews as it's a slow performer[0]. Now, this has actually been addressed in a later firmware upgrade, but at that point the damage had already been done. After that, Canon basically dumped this camera on the market at very low prices.

Good thing for us is that this little camera can now be bought very cheap second hand (got mine last year at around $100) Once you load up ML, you get a fantastic fun little camera! I'm not very experienced with shooting video but for photography, it's a wonderful experience. Focus peeking, magic zoom, interval photography, all for free.

Many kudos to the ML devs from this side as well!

[0] [https://kenrockwell.com/canon/eos-m/m.htm](https://kenrockwell.com/canon/eos-m/m.htm)

</comment></comment><comment><USER4> I'm sort of amazed at the cost of high-end cameras, especially when what separates it from a modern smartphone seems to be nothing but bigger/faster sensors and the ability to use big lenses. The camera manufacturers use the lowest powered CPUs they can get away with, running proprietary real-time operating systems with horrible UX and no third party support. For the price of a high end laptop. It really seems like something is off in the market - but there doesn't seem to be any sort of real change happening.

I needed to do some video work last year, so I basically followed Casey Neistat's suggestion for a Canon camera, only to find out it wouldn't allow me to export the video live via an HDMI cable without the camera's overlay on top. Turning all the elements off still left me with a red dot in the corner. Apparently I got the wrong model and the one $800 more expensive would do this. After a while I gave up, went back to the store and got a cheap video camera that could do it, BUT again, I had to get the more expensive version, because the same model in the cheaper option didn't allow this particular use case.

I was really surprised by how backwards this all was! Proprietary OSs, artificially limited functionality, anemic processors, interfaces out of the 1980s. It's crazy!

Has someone slapped Android on a decent camera yet?

<comment><USER5> They're not for everyone, but I think you're overlooking quite a bit. The larger sensor is crucial in low-light situations, and a smaller phone camera just won't handle that. The "horrible UX" is faster by a significant margin, partly by being a dedicated device, partly by having a lot of dedicated buttons, and partly because the response time on pressing the shutter release button is extremely short. If you want to carry a camera for capturing events or moments quickly, I'm not sure that a phone will do that for you.

And yes, the high-end full-frame DSLR costs are on par with high-end laptops, but IMO you get 90% of the benefit for a fraction of the cost with an entry- or mid-level crop sensor.

As for the UIs, I'll definitely acknowledge they could stand some updates, but the way you describe it sounds like an overstatement. What would you improve, exactly?

<comment><USER6> A photo management UI on par with smartphones. Fast, easy to use and easy to do edits and sharing with. Along with GPS, WiFi & LTE connectivity that doesn't suck. Maybe direct app integration with instagram and other apps. Put an OLED screen on the back, not just the viewfinder.

Basically slap a fucking android phone on the back when browsing photos. Make it a separate CPU if necessary.

Embrace the 'casual camera' body style more. The viewfinder should not be a big chunky thing in the center, but on the side and a pop up if necessary. Get rid of grips and make them a uniform rectangle, and add a battery grip in the box to make it slr grippy. People are scared of slr style bodies, while they are not scared of casual camera bodies.

They will continue serving the standard pro market who want the big chunky slr style bodies, this is how they can capture the 'high quality life memory / instagram influencer' market better.

<comment><USER7> The mass market casual camera today is a smartphone, which comes with what you describe.

Canon, Nikon, and others have long decided they can't and won't compete with smartphone makers and are increasingly moving up-market to the prosumer and professional markets who want "big chunky SLR body styles", and they use specialized software like Capture One and Photo Mechanic to view/edit/organize photos anyway.

<comment><USER8> While I will agree SLR body styles are easier to use we don't __want __them, it 's that we accept that the size and weight are the price we need to pay for the performance that you simply can't get out of a phone.

These days when the shot is easy the phone is at least in the ballpark with the big guys. When the shot is hard the phone isn't even in the running.

<comment><USER9> There's also the whole world of non-SLR pro cameras.

</comment></comment></comment></comment></comment></comment><comment><USER10> I now hold the belief that any Canon camera that does not have Magic Lantern installed is broken. I use it on my 5DmkII, and it's an entirely different camera. I love the features that ML brings to me for shooting stills (I don't shoot video on my DSLR). An internal intervalometer is worth its weight in gold. The ETTR ability is also priceless for chasing sunsets/rises. For how I use my mkII, the only thing that I have that's nit picky about ML is that it can't handle intervals less than 5 seconds. When it comes to requiring shorter intervals, I do pull out a wired intervalometer, but that's rare.

One thing that I'd love to figure out is how to have ML control the timelapse, but have the camera signal to an external motion controller. Usually, the motion controllers want to control the camera so everything is in sync. However, as ETTR increases the shutter time chasing a sunset, the motion controller needs to know to delay the move. I have built devices connected to the shutter release port waiting for the voltages to change, but that didn't work. Was hoping that the voltage would drop when triggered internally, but it seems the port isn't wired way. Almost like they might have an opto-isolator on the port or some other method to protect the port but it doesn't allow the voltage to drop when triggered by the camera itself.

<comment><USER11> I'd expand that to "Canon is broken."

This company has needed a management house-cleaning for at least a decade. They're just wandering in the weeds at this point. Does no one there actually pursue filmmaking or photography? It's unreal.

I mean... there's no excuse for any digital camera, especially one made in the last decade, not to have an intervalometer. It's essentially FREE to implement.

But then, Canon's the company that was still pushing interlaced video well into the 2000s and launched the SLR video revolution... with a camera that could shoot 25 and 30 but not 24 FPS.

<comment><USER12> Exactly none of Sony's latest full frame mirrorless cameras have a built in intervalometer, and there is no way to install one. So it's not just Canon.

<comment><USER13> You'll be happy to know that it's supposed to come in the April firmware update, though.

</comment></comment></comment></comment><comment><USER14> I recently purchased an 5d mk iii and installed magic lantern on it. The 14bit raw video is AMAZING. Also, this particular camera can now record at, I think the limit is 3.5k vs 1080p from the factory.

[https://youtu.be/6yKbwXYmpD0](https://youtu.be/6yKbwXYmpD0)

<comment><USER15> That video, with the pre- and post- differences is pretty remarkable.

Here's my stupid question of the day: given that this color-grading of RAW streams makes _SUCH_ a massive difference... why don't companies like Conan and Sony just ship it with the product? Why do their cameras fall short of having something like Magic Lantern? In the end it gives the customer what they want... surely Sony/Canon are capable of producing something like this, so why not produce it?

<comment><USER16> Most probably because it would compete with the Canon EOS product line.

<comment><USER17> The full name of the 5d mark II is Canon EOS 5d mark II.

<comment><USER16> I meant the cinema EOS line...

</comment></comment></comment></comment></comment><comment><USER18> Imagine if Apple and Google made full frame cameras.

Currently we have great hardware with terrible software and vice-versa.

A Sony full frame sensor, Zeiss glass, with an A series chip and good software to make it shine.

<comment><USER19> Apple is allergic to buttons so I can't see them producing a camera with a real user interface (knobs and rings and so on)

Leica T series is probably pretty close to what they would come up with tho. Solid block of metal, and nothing but screen on the back.

<comment><USER20> You can't work like that. I have a 5D and a Sony RX-100, and, given that I've taken good enough photos with my phone, the differences in the sensor are relatively minor. The UX differences, however, are light years apart. It takes me tens of seconds to change the settings like I want them on the Sony, whereas on the Canon they're a knob away.

I'm pretty sure that the major difference between a pro and a consumer camera is the amount of easily accessible buttons and knobs.

<comment><USER21> >I'm pretty sure that the major difference between a pro and a consumer camera is the amount of easily accessible buttons and knobs.

Precisely this. After 10K shots on a Canon 1200D, I upgraded to an 80D specifically to have access to the top controls to quickly change settings while shooting. You can remap a lot of the 1200D's controls in the "Expert Feature"/customizations section, but the 80D's extra ergonomics are a true joy. The camera is molded to my hands and comfortable to wield for hours at a time.

I can't say the same for the smartphones I shoot with. My Blackberry KeyOne's camera has fully manual controls (even the focus distance can be manually set!), but the "slide your finger to control settings" UI sometimes feels like the "Reddit designs volume control" experiment from a few years ago. I empathize with the Java programmers, but it's extremely frustrating to tell the smartphone what kind of picture I want. It should go without saying that it's equally frustrating to be unable to hold the slab phone adequately still.

Reading people's passionate hatred for real cameras makes it quite clear that they aren't the target audience for a simple mid-range DSLR, let alone monsters like the 1D or Nikon D5.

</comment></comment></comment></comment><comment><USER22> Also, for non-DSLRs: [http://chdk.wikia.com/wiki/CHDK](http://chdk.wikia.com/wiki/CHDK)

<comment><USER23> CHDK, with its built-in LUA scripting, is incredible. I've used it to outfit cheap (<$30 USD) point and shoot Canon cameras for time-lapse.

</comment></comment><comment><USER24> Why the hell are they capturing keyboard inputs of the q, v, m, l, and p keys and updating the URI w/ navigational things? Makes CMD+L useless...

Looks like the are supposedly emulating some of the camera's functions? I don't know enough about this camera for it to make any sense to me.

<comment><USER25> Looks like they were trying to create a mock of the user interface that's fully functional in the browser. It's really cool, great to demo, and they probably wanted to put it in the URI so you can share a link to a specific page. Maybe they even use the same UI code for both browser and the embedded system?

But in practice they implemented this poorly. They could update the URI with replaceState so you don't break the back button, and ignore keyboard shortcuts if a modifier key is pressed. One of the reasons I like projects with open- source websites - you can just file an issue on the website itself!

</comment></comment><comment><USER26> 5DMkII owner here. I love ML and have been a canon user for over 20 years. But I recently switched to Sony as have many other Canon users I meet because Canon is not innovating as well as Sony. Better sensors, features, ergonomics and capabilities. The A7Sii is a thing of beauty. Includes many ML features built in like focus peaking, zebra lines, etc. And you van remove the 30 min limit on recording via a hack so no need for MLs features that try to help with that.

Also Sony make sensors for many other camera companies including Nikon.

<comment><USER27> My (relatively) inexpensive Panasonic GX-80 (GX-85 in some markets) has those features. How are Canon so far behind?

The biggest issue with the GX-85 for film is it's lack of audio in, that can be circumvented with a seperate audio recorder but it's certainly not perfect.

</comment></comment><comment><USER28> I'm surprised Magic Lantern hasn't been sued out of existence by Canon. Canon is notorious for holding back simple software features from entry-level cameras to price differentiate. Customisable firmware is a threat to their business model...

For the same reason I'm curious that Sony is not more open - their strategy appears to be the opposite of Canon, throwing in the kitchen sink of features into their entry level bodies. And yet they have removed tha ability to load Android apps onto their latest cameras...

<comment><USER29> Conversely, Magic Lantern is a reason for consumers to pick Canon over Nikon or some other camera brand. I imagine there are also at least some differences in the quality of sensors between high-end and lower-end DSLRs?

</comment></comment><comment><USER30> Why .fm? I instantly felt confident this was internet radio related.

<comment><USER31> Firmware perhaps?

<comment> half-kh-hacker QMK firmware do it too: [https://qmk.fm](https://qmk.fm)

</comment></comment></comment><comment><USER32> Started by Trammell Hudson of Thunderstrike fame (and many other interesting projects), see [https://trmm.net](https://trmm.net)

</comment><comment><USER33> If you're new to Magic Lantern and need a getting started guide, here is an in depth tutorial I created :)

[https://www.youtube.com/watch?v=LHzQkJNMIzU](https://www.youtube.com/watch?v=LHzQkJNMIzU)

<comment><USER34> To repay you here's a little random tip, because the sound on that video is kinda boxy (with my headphones anyway), and this helped me greatly (to be honest, it kinda blew me mind how much this can fix)

With a sample editor that can use VST/AU effects, and the free version of this, or something that has the same functionality: [https://www.tokyodawn.net/tdr-nova/](https://www.tokyodawn.net/tdr-nova/)

1\. take a frequency band, put the Q for it to max or near max, and gain to max.

2\. sweep the frequency until you find the one that makes it sound extremely boxy (it's probably below 1k but don't take my word for it, you _will_ hear it when you found it however)

3\. now set gain to zero, activate the threshold button, and dial the threshold to a value that just makes the boxyness disappear without making it sound _too_ thin

4\. repeat with more frequencies as needed/wanted

5\. compress or normalize

<comment><USER35> What you're mostly doing there is working out the resonances of your room/speakers/headphones. Sorry to say it, but this is terrible advice on how to apply corrective EQ.

<comment><USER34> So, what's a better way? Let's see your not terrible advice.

> resonances of your room/speakers/headphones

Yet the frequencies that stand out depend so much on the input material, and arent't the same between recordings in different rooms or with different mics, even with the same playback setup. Weird.

As the manual of that VST so terribly states

> This is an excellent tool to correct a boxy low-endsound, even out > resonances in a recording or reduce excessive sibilance in a vocal part.

Sure, they don't outline this specific approach, but others do, sadly:

[https://www.soundonsound.com/sound-advice/q-how-do- pinpoint-...](https://www.soundonsound.com/sound-advice/q-how-do-pinpoint- nasty-resonances)

> In my experience visual analysis won't usually help a great deal here, and > you have to adopt more of a 'hunt and peck' approach, sweeping a narrow EQ > boost around the spectrum and then placing an EQ cut wherever the boost > sounds most unappealing

And it just goes on and on :(

[http://indierecordingdepot.com/t/how-do-i-find-resonant- freq...](http://indierecordingdepot.com/t/how-do-i-find-resonant- frequencies/1270)

Of course, since you said it's such terrible advice, my ears noticing the results are probably lying to me, and I can hardly wait to find out how to do it correctly at last.

<comment><USER35> _If_ you are going to use that technique, _don 't_ use extreme Q settings, because as I said you are mostly going to be finding resonances that aren't actually in the signal, but are part of listening in your specific environment. Granted you will find some of the problem frequencies from the source, but with a much lower Q you're at least less likely to find stuff that isn't. EQ should be applied gently, if you require extreme settings then you have a shitty source material, take some steps to correct that. There are no pro-spec EQ hardware units that offer such extreme Q settings, because they are next to useless except in extremely limited circumstances.

Just because something is repeated a lot doesn't automatically make it a great idea: see "Brexit means Brexit" or "Build The Wall" for examples.

</comment></comment></comment></comment></comment><comment><USER36> I apologize if that's a bit off-topic, but I would appreciate some advice on where to start with decent digital photography.

I am in no way a professional photographer, but I have mostly used some relatively budget smart phones to take pictures these past years (my previous experiences were with a canon compact circa 2002-2008, and film). I do not have a great budget for photography, since it isn't my main hobby (and am still studying), but would appreciate some of the features I saw in the discussion thread & website, notably: \- the ability to shoot RAW pictures \- the ability to shoot professional-ish looking videos \- an open firmware I can tinker with and learn from is a great plus \- if the device is moddable or has some available interface ports, that would be nice as well \- maybe a timelapse/timer function? (though that sounds quite simple to implement with the above).

Would compact cameras be the way to go (they're a plus for travel, I think)? Or would a cheap (second hand) DSLR be way better? Any specific brand suggestions?

Thank you in advance for the advice!

<comment><USER37> I'm a lifelong SLR shooter but I was posed the same question recently.

Micro Four-Thirds MILCs appear to be very capable, affordable and portable. I'd suggest you look at Panasonic or Olympus, they seem to have excellent lenses which is what really defines the photo. Fuji are also well spoken of but I think they use bigger APS-C sensors. Check out DP Review website.

Getting into used 'big' DSLR instead would probably cost about $1600 ( used 6D and 24-70 or 24-105 f/4 lens) which is a lot to gamble on a new hobby. And contrary to legend, they don't hold value. I've been quoted less than 50% trade-in value on from-new Canon L lenses. Unless you're buying Leica, photography is not an investment; be prepared to lose money unless you manage to sell your photos in volume.

</comment></comment><comment><USER38> I used this to get a software intervalometer on my T5i and now I can take awesome timelapses of things. It's really a nice set of features.

</comment><comment> gjsman-1000 GoPro once released a camera called the GoPro HERO (2018). It didn't take long for some hackers to realize it was a stock-clearing trick and was actually a GoPro Hero 5 Black with some of the software features disabled. They then figured out a way to install the GoPro Hero 5 Black software on it, re- enabling all of the disabled features.

That... I'm not so sure where that is, morally.

<comment><USER39> The ML folks have sworn off working on higher end cameras (1DX and similar) because of thinly veiled legal threats from Canon, although I'd personally be concerned about bricking a camera that expensive.

<comment><USER40> Which is total BS and allows Canon to drag their feet to add features that the cameras are completely capable of. It's kind of a joke that a camera can do complex things yet the only thing holding it back are firmware updates. Canon could even charge for say a "film" firmware for their DSLR cameras to enable these features, disable AGM, etc. But they have to protect their film cameras that even still miss features ML brings to the table.

<comment><USER39> Agreed.

I wish a development team in a "better" jurisdiction would fork ML and continue on with it.

Sigh

</comment></comment></comment></comment><comment><USER41> Does anyone know if there's an equivalent project for Nikon (D 5000)? On a first quick search, I've found nothing comparable.

</comment><comment><USER42> A poor choice of names imho. When I see "magic lantern" on HN I think of something totally different.

[https://en.wikipedia.org/wiki/Magic_Lantern_(software)](https://en.wikipedia.org/wiki/Magic_Lantern_\(software\))

"Magic Lantern is keystroke logging software developed by the United States' Federal Bureau of Investigation (FBI). Magic Lantern was first reported in a column by Bob Sullivan of MSNBC on 20 November 2001[1] and by Ted Bridis of the Associated Press.[2]"

</comment><comment><USER28> There are similar efforts afoot for Sony cameras but have never reached the level of Magic Lantern.

e.g [https://www.dyxum.com/dforum/emount-electronic-protocol- reve...](https://www.dyxum.com/dforum/emount-electronic-protocol-reverse- engineering_topic119522.html)

[https://sony-pmca.appspot.com/apps](https://sony-pmca.appspot.com/apps)

</comment><comment><USER43> I have a canon rebel t2i sitting on my shelf I bought from a friend and haven't touched. How can I install this and learn some basic photography skills? I used to like making stop motion vids and that's why I bought it but I'm overwhelmed with all of the options.

<comment><USER44> What part of the installation instructions was confusing?

[https://wiki.magiclantern.fm/install](https://wiki.magiclantern.fm/install)

Unrelated to that learning basic photography skills mostly involves learning the relationship between ISO, aperture, and shutter speed. You can google guides, probably attend a course in a local university/college, or adult- educational center, and also experiment via trial and error.

<comment><USER43> None of the installation is confusing -- just the actual camera skills part.

</comment></comment></comment><comment><USER45> I wonder why Canon refrain from supporting this kind of projects by providing documentation on how to interface with the hardware (without going as far as stamping it as official). It seems that it make their products more attractive at no cost for them. Any drawbacks?

<comment> 0-_-0 My suspicion is that so few people are using these features that it's not worth the effort. Disclosing the details of the hardware could also open them up to litigation if a competitor finds something slightly similar to something they already patented.

</comment></comment><comment><USER46> Very cool, I am absolutely shocked that Sony doesn't offer basically any control over bitrates or framerates (on the α7/9), except extremely rigid profiles; I doubt the hardware is actually that restrictive.

</comment><comment><USER47> I will probably never use this but I am delighted every time I see the OS community reverse engineer something and provide a better, open source version while having a fraction of the resources and information.

</comment><comment><USER48> Better headline:

"Magic Lantern: a free firmware add-on for Canon EOS cameras"

</comment><comment><USER49> I'd love to have something like ML for my Sony Alpha :(

<comment><USER50> Pretty sure my A99ii will never see anything cool like this.

</comment></comment><comment><USER51> How did these guys reverse engineer Canon's software?

<comment><USER52> Trammell Hudson in 2009 reverse engineered the firmware of the Canon EOS 5D Mark II because he was frustrated with some of the limitations in the camera's firmware when making short films. That little hack turned into Magic Lantern. On his website [1] you'll find a screenshot [2] of his first success: adding three extra vanity letters to the firmware version number. He states: "Re- writing strings is a good easy technique for figuring out if you have 'won' and your code is running on the system."

[1] [https://trmm.net/Taking_things_apart#Extending](https://trmm.net/Taking_things_apart#Extending) [2] [https://www.flickr.com/photos/osr/16412008471/lightbox/](https://www.flickr.com/photos/osr/16412008471/lightbox/)

</comment></comment><comment><USER53> With magic lantern, is it possible to stream the video from the camera to a host computer? For example to use it like a webcam

<comment><USER54> You might not even need magic lantern for that. I have a 77d and it streams a live view to the Canon EOS utility over WiFi. Quality seems pretty good. I haven't looked into it myself, but I wouldn't be too surprised if someone has written software to stream from it in this mode.

</comment></comment><comment><USER55> Hmm, I see a blank page on mobile (firefox, iOS).

</comment><comment><USER56> sorry for stupid question, does this software support all the EOS M series? M50, M5, M6, M100?

<comment> 0-_-0 The list of supported cameras is right on the front page.

The list of cameras supported by CHDK is on the front page here: [http://chdk.wikia.com/wiki/CHDK](http://chdk.wikia.com/wiki/CHDK)

To summarize, ML supports M and M2, CHDK supports M3 and M10. There was some progress on the rest.

</comment></comment><comment><USER57> obligatory, is there something like this for Nikon cameras?

<comment><USER58> There is NikonHacker [1], but I don't think it's nearly as feature-rich as Magic Lantern.

[1] [https://nikonhacker.com/wiki/Main_Page](https://nikonhacker.com/wiki/Main_Page)

</comment></comment><comment><USER59> One thing I want in full frame EVIL cameras is live photos capability

Record from 1 keyframe to the next keyframe, take full resolution still at the user clicked place in between

And done as automatic as it happens on iOS</comment>