Introducing temporary interaction limits - <AUTHOR> https://github.com/blog/2370-temporary-interaction-limits <comment><USER2> Is this to limit the HN effect whenever an issue is linked here and people just go "yeah +1" even if they don't actually read the discussion?

<comment><USER3> I haven't seen that much since the emoji reaction +1 feature was introduced. I use the latter a lot though.

<comment><USER4> The +1 reaction has helped a lot but I still useless +1 / -1 notification spam pretty regularly from people who haven't gotten the message.

<comment><USER5> _" people who haven't gotten the message"_

You cannot add emoji reaction from mobile version of Github website.

<comment><USER3> That sounds like a good issue to open with GitHub.

I've been using request desktop site to vote in Mobile Safari, but it is one extra thing to do.

</comment></comment></comment></comment></comment><comment><USER6> This seems like a good idea to me.

Nitpick: the captions under the checkboxes are very confusing because they're written with double negatives. "Users that have NOT X will be UNable to Y" could be "Only users that have X will be able to Y".</comment>


Ask HN: How do you describe the zone where you do great work? - <AUTHOR> Describe your feeling of when you do great work - this can be neat design, good code, solution to a really difficult problem, long focused hours, getting multiple things done efficiently.<p>What preparation helps to be in this zone over sustained period of time ? <comment><USER2> I call it the Deep state. If I think about it now, I'm just in it. No other things come to mind.

Obstacle : Not starting already.

Preparation : Trello board of a collection of tasks. Labeling them one by one as done.</comment>


Is OOP dead? Not by a long shot - <AUTHOR> http://techbeacon.com/object-oriented-programming-dead-not-long-shot <comment><USER2> objects are just a feature of FP, pure object oriented programming just seems like the next buzzword OOP people are good at making; I for one would love to put "experience in POOP" on my resume.</comment>


Tile38 – Realtime geofencing and geospatial index, v1.7.0 - <AUTHOR> https://github.com/tidwall/tile38/releases/tag/1.7.0 <comment><USER2> I wonder if a kind of pony-express messaging app could be done with something like this. Like pick a friend in another state, write a message, carry it around in memory hoping that you'll be within a quarter mile of someone else running the app, and see how long it would take for the message to get delivered.

</comment><comment><USER3> I love it when geo stuff shows up here. Its always fun to see what people are doing.

I see that it is in Memory, but are there any practical limits? How much memory is recommended for what size datasets?

And it doesn't look like the documentation has an examples as to how to load data in? Do I need to convert to Web Mercator, or can I give it WGS84 and have it convert? Also, any plans to allow for pure WGS84? I never actually use Web Mercator, so would rather a system that uses WGS84 natively.

<comment><AUTHOR> > I see that it is in Memory, but are there any practical limits?

Yes. The in-memory database is basically a big btree + a big rtree. These trees contain many pointers to geo objects and depending on the complexity of the objects, it may take up a lot of memory. A 16GB machine can typically store 100 million+ points.

> How much memory is recommended for what size datasets?

Depends on the type of object and the length of the keys, but I would conservatively use 128-bytes per point as a general rule. So 16GB machine should support ~135 million points.

> And it doesn't look like the documentation has an examples as to how to load > data in?

Please check out [http://tile38.com](http://tile38.com) for extra info and docs on commands. Tile38 uses the Redis protocol ([https://redis.io/topics/protocol](https://redis.io/topics/protocol)), so it's importing data is similar to [https://redis.io/topics/mass- insert](https://redis.io/topics/mass-insert).

> Do I need to convert to Web Mercator, or can I give it WGS84 and have it > convert? Also, any plans to allow for pure WGS84?

All coordinates are in WGS84 lat/lon, so you should be fine. Under-the-hood the calculations and such are in EPSG:3857 / WGS 84 Web Mercator.

<comment><USER4> > Tile38 uses the Redis protocol

Under the hood, is Tile38 a distribution of Redis with a Tile38 Redis module?

If so, could the Tile38 module be used in "regular" Redis alongside other Redis modules?

<comment><USER5> Lol looks like this database needs its own database. Yeah seems like it should be a plugin or library not sure why it's standalone if it's just backed by redis.

<comment><AUTHOR> It's not backed by Redis. It's fully standalone. I wish I could have made a redis module back when I started the project, but I'm plenty happy how Tile38 has matured.

<comment><USER5> Eh, fair enough then. If you don't have to setup redis yourself it doesn't really matter

</comment></comment></comment></comment></comment></comment><comment><USER6> Looks pretty neat! Since OP is the author, can I ask: how many simultaneous concurrent (client) connections does this support, how many simultaneous moving objects and at what update frequency? Do these limits change if the shapes get complicated?

<comment><AUTHOR> > how many simultaneous concurrent (client) connections does this support?

The client connections are very lightweight. The protocol library is based on [https://github.com/tidwall/redcon](https://github.com/tidwall/redcon). I don't have exact numbers but this link may be helpful (which compares Redcon to Redis): [https://simongui.github.io/2016/10/24/benchmarking-go- redis-...](https://simongui.github.io/2016/10/24/benchmarking-go-redis-server- libraries.html). I've seen it handle 16K+ client connection without issue, but typically a client library that supports pooling should be used (like Redigo).

> how many simultaneous moving objects and at what update frequency?

Depends on the complexity of the objects, the number of nearby objects, and the server hardware. Each collection of objects consists of one btree and one rtree for key/id lookups and spatial queries respectively. So updating a single point is like O(log(n)+log(m)). An in-memory database like Tile38 can achieve 100K+ per second. Utilizing network pipelining can help out too. FYI, I use a 4-core server with 8GB ram for testing.

In the case of a roaming geofence ([http://tile38.com/topics/roaming- geofences](http://tile38.com/topics/roaming-geofences)), following each update there an additional query of the rtree O(log(n)) to retrieve nearby neighbors. If there's a lot of nearby objects, this can result in a lot of data sent over the network. But in general, a collection of millions of points spread over a continent with 100K+ updates per second should work, while keeping with realtimelyness.

> Do these limits change if the shapes get complicated?

Yes. Complicated objects such are MultiPolygons may require additional calculations like raycasting.

</comment></comment><comment><USER7> Neat. Seems like a good weekend is coming up. I was wondering how was the gifs/images done? I would really like to see the code if it was done programmatically.

<comment><AUTHOR> Some programming, some design tools.

I programmatically simulated the movements as paths at 30fps into a series of data points, then imported into Animate CC and tweaked the tweening by hand, then exported to a series of PNGs, then imported PNGs into Photoshop as a movie sequence, finally exported as gif.

The superior tweening capabilities of Animate CC and the excellent gif compression of Photoshop makes tuning the animation super easy.

Perhaps next time I'll programmatically draw gif frames, for the fun of it.

</comment></comment><comment><USER8> Neat! Does someone already used it in production? I'm wondering if it could be useful to replace a medium sized postgres database. I'm trying to replace some clustering currently done with Elasticsearch with postgis, but it's really slow (6m points * 700 boxes, it's taking 20 minutes to scan the table).

<comment><AUTHOR> Yes, it is being used in production.

General purpose spatial queries with Tile38 are very fast. Likely much quicker than PostGIS, but PostGIS will be able to store much more data. It's a tradeoff for sure.

If your dataset isn't terribly big and performance is what you're after then I recommend giving Tile38 a try.

<comment><USER8> Thank you for the details! After all, if I can manage to fit around 6 million points into a single r4.large and get a decent response time when clustering them, I can't see why I shouldn't try something with it as a secondary data store.

<comment><AUTHOR> I can't argue with that. :)

</comment></comment></comment></comment><comment><USER9> how fast is within? does it use ray tracing?

<comment><AUTHOR> > how fast is within?

I replied to another question shortly ago regarding performance. [https://news.ycombinator.com/item?id=13285538](https://news.ycombinator.com/item?id=13285538)

> does it use ray tracing?

No.</comment></comment>


Using regmaps to make Linux drivers more generic - <AUTHOR> https://www.collabora.com/news-and-blog/blog/2020/05/27/using-regmaps-to-make-linux-drivers-more-generic/ <comment><USER2> Observation: Every single piece of hardware that is, or could ever exist, could be represented generically as follows:

1) One or more "meta-registers", that is, hardware registers for the purpose of reading and writing all of the other hardware registers.

2) Some form of standard callback/interrupt from the hardware indicating that registers have changed and need to be re-read.

3) Once #1 and #2 are defined, then a "linux-memory-region as readable/writable hardware-device registers" (such as regmap), can be easily and generically implemented.

regmap, while it implements a generic device abstraction, _is itself not generically implemented_ , and can't be, until hardware designers implement #1 and #2, which first requires the creation of a standard, and then requires a consensus to use that standard for their hardware devices...

(Of course, DMA, as it exists currently, is still part of the model... what we're looking at here is the non-generic nature of hardware registers, from one hardware device to a different hardware device... that's still OK, but there should be one (or more) "meta-register(s)" to read/write all of the others, and some interrupt-driven way of being notified of changes... you know, in J.R.R. Tolkien terms "one register to bind them"... but maybe two or more meta-registers, for this purpose, make sense... I'm not a hardware engineer, I wouldn't know the optimal configuration...)

</comment><comment><USER3> This is def what smart abstractions look like.

</comment><comment><USER4> I see how this is an abstraction of register memory. but how does it help against a constantly moving kernel?

Also, wouldn't this make an otherwise simple code much harder to read and understand?

<comment><USER5> It's not the kernel that's constantly moving, it's the hardware. The idea of regmaps is that when you get a device revision B, rather than having to sprinkle "if(revision == ...)" all over your code you just look it up in a table.

</comment></comment><comment><USER6> How do Linux Devs manage to not accidentally break device support constantly? I imagine this stuff is close to impossible to write automated tests for.

<comment><USER7> Generally/Ideally things break early in the release cycle, bug reports show up on the mailing lists, and fixes are applied. They are marked with a Fixes tag so as for stable releases to pick them up if relevant.

For basic stuff (as in checking if boards still boot) you have projects like kernelci.org that detect regressions on a huge variety of boards. That said, if you need to support a touchscreen, or say, an uncommon USB device, it's up to you to take care of keeping up with the releases.

</comment></comment><comment><USER8> I could see this introducing a performance hit...

Previously a register address was a hardcoded constant. Very small,very fast.

Now, looking up a register ina map involves string matching on the name across every possible register in the map till you find a match. That is much slower, but also makes the kernel quite a lot larger, because you're effectively shipping half the device datasheet in the form of names for every flag bit.

<comment><USER9> There's no string matching that I can see in the linked article?

<comment><USER8> " .name = "dw-mipi-dsi","

Looks like a string in the structure to me. And when you have an array of such structures, you end up scanning through them checking for the name you need.

<comment><USER9> That's the name of the regmap. Clicking through to the definition of the structure says it's optional and thus there for descriptive purposes.

         *
         * @name: Optional name of the regmap. Useful when a device has multiple
         *        register regions.</comment></comment></comment></comment>


The Anti-Capitalist Software License - <AUTHOR> https://anticapitalist.software <comment><USER2> Any license with an ideological component is a timebomb that goes off when the developer decides you are no longer pure enough. You'd have to be insane to use any software with a license like this (if one was enforceable)

<comment><USER3> It's often a joke among far-left communities that garden variety liberals are more contemptible than right-wing organizations, so I believe it.

When you get to the fringes of political ideology, infighting and corruption tend to be insane. There's constant backstabbing (sometimes in a literal sense) at both extremes.

<comment> non-entity yeah I have some ML friends I followed on social media and got a first hand look at so called leftist infighting. Seems mostly counterproductive.

</comment></comment></comment><comment><USER4> This seems related to the _copyfarleft_ concept proposed by Dmytri Kleiner [1] and his Peer Production License as a first-draft implementation of the concept. That's spawned a few license variants and been debated a bit [2]. Would be interesting to know whether the authors of this license are aware of those licenses and debate, and how they see their goals as relating.

[1] [http://telekommunisten.net/the-telekommunist- manifesto/](http://telekommunisten.net/the-telekommunist-manifesto/)

[2] E.g., [https://www.metamute.org/editorial/articles/copyfarleft- crit...](https://www.metamute.org/editorial/articles/copyfarleft-critique), [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2468731](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2468731),

</comment><comment><USER5> I sometimes amuse myself with the idea of creating a software license out of spite for certain organisation, a kind of black list if you will, like Arya Starks kill list but more childish. At any rate you'd probably want to look into all the ways rich people make money with not-for-profits. The difference in intent and word of law is damn difficult problem to solve.

<comment> aaron-santos For all of the GitHub supporting ICE condemnation, I'm surprised there isn't a license which forbids GitHub from using a piece of software.

<comment><USER6> How about something a bit different: a license that prevents Github hosting?

That should be easy.

You create two licenses for your software: a license for the code _per se_ , and a separate one for your _git repo_ of the software.

So that is to say, for instance, your program could be MIT licensed. But your git repository (all the Git objects and meta-data: commit messages, hashes, branches, tags, ...) could have a more restrictive license which asserts "this may not be mirrored on Github".

It's intriguing. If we regard the work you're creating to be your git repo, then that is not free. If we regard the work you're creating to be baselines of an artifact that you're committing to the git repo, then that is free.

</comment></comment></comment><comment><USER7> This doesn't respect the 4 freedoms. If by some odd circumstance, all the FAANG companies became worker cooperatives and adopted all the requirements for this license and released new versions of their software using this license, would that really make the world a better place?

I feel without the 4 software freedoms, like with the GPL, you are not respecting the freedoms of the end user. They mention requiring, "not exploiting non-members". To me that requires the 4 software freedoms.

Is my understanding of this license and the software freedoms incorrect?

<comment><USER8> If each FAANG released backend code:

* Facebook: We'd learn what sorts of data collection and aggregation they do, and how metrics are planned and gamed.

* Apple: We'd learn something about the App Store rejection policy, and we'd also get to jailbreak and repair our phones.

* Amazon: We'd find out how their billing and egress policies really work. We'd also get to see how customers are tracked.

* Netflix: Oh look, tracking of customers, something we'd learn.

* Google: Customers, tracking, etc. But also they know stuff about their long-term hardware vendor partners.

How would this not make the world a better place? It's clearly in line with both the copyleft programme of ending copyright protections on source code, and also more general anti-corporate and anti-surveillance ideologies.

<comment><USER7> This license doesn't require them to share the source though, just limit who is allowed to use it. Is that reading of the license wrong, is my question. They specifically mention not caring if source code is shared but if the companies that utilize software licensed with this license are worker collectives or not.

Which is why I feel it would be better to advocate for the [A]GPL as that would result in the situation you describe.

<comment><USER8> Google has a public AGPL policy [0], so advocating for it will not have the effect that you like.

[0] [https://opensource.google/docs/using/agpl- policy/](https://opensource.google/docs/using/agpl-policy/)

<comment><USER7> The page you linked says Google bans the use of AGPL code in their core products.

Are you saying that since Google has a public policy banning the use of AGPL code in their products there is no use in advocating for a 4 freedom respecting copyleft license?

</comment></comment></comment></comment></comment><comment><USER9> > An organization of people that seeks shared profit for all its members and > does not exploit the labor of non-members

It doesn't make indication of the distribution of the profit. If I were a business owner couldn't I get around this by giving each employee $1 in stock options? If the value of the organization goes up there's still shared profit, no matter how minuscule it is.

</comment><comment><USER3> Forgive my naivety, but why aren't worker-owned cooperatives considered to be profitable enterprises?

Taking an example of a typical factory making widgets, profit is essential so that the organization running the place has a buffer in case of unexpected breakdowns or room for expansion.

This agreement appears to forbid worker-owned cooperatives, since profit is essential to the orderly operation of any factory.

Without profit, there is no emergency fund and there is no preventative maintenance.

<comment><USER10> Presumably a worker-owned cooperative would fall under 2c and 3.

>2c. An organization of people that seeks shared profit for all its members and does not exploit the labor of non-members

>3\. If the User is an organization with owners, then all owners are workers and all workers are owners with equal share.

<comment><USER3> If profit is shared, it's not profit the organization can use as a buffer. The organization itself requires profit. That profit cannot be distributed without putting continuity of the organization at risk.

<comment><USER10> The wording is "seeks shared profit" which does not to me read as "distributes all profit." My read is "seeks shared financial gain" (in contrast to "seeks shareholder financial gain"). Even a non-profit institution is able to re- invest it's income without being a for-profit institution.

<comment><USER3> What is the fundamental difference between a factory owned by a cooperative and a factory owned by a corporation with workers of the factory having shares in that company?

The profit objectives are identical. They have to be, because the basic realities of factory processes mandate profit for ongoing operations and expansion.

If that's true, then why is one excluded and not the other?

<comment><USER11> Those are essentially the same thing, as long as the CEO makes the same as an entry level worker. Because the keyword is equal share. Nobody is elevated financially above another.

You could still have surplus, but workers get to vote on how much that should be, and a flat-style management system still could have managers over others who manage projects, they could pitch budget proposals and vote. Then at the end of the year whatever is left over goes out equally to all (or based on # of hours worked), as a bonus or dividend payout.

But most factories the CEO makes 50 million, the worker might get a 3% match in a 401k, and $15 an hour. In other words the ceo is making 300x more, and is definitely not worth that much. Why are today's CEO's better / worth more than 1980's, by 3-4x as much, when the average worker is not, even though productivity of workers has skyrocketed since the 80's?

<comment><USER12> As I understand it, most co-ops don't even enforce totally equal pay across the board -- they just ensure that there are no extreme income discrepancies. Rewarding workers who do a great job with a bonus makes sense even from an egalitarian perspective.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> First, you have to have software worth licensing, that is complicated enough to be too much trouble to be re-written.

</comment><comment><USER14> I'm about as pro-capitalist, laissez-faire, pro-free-market as it gets. I was a literal card-carrying member of the LibertariaNZ party back in New Zealand. The spirit of this license disgusts me at a fairly deep emotional level.

That said, I think that this specific concern raised by the license is very real:

> ... If you want your code to empower students, artists, hobbyists, > collectives, cooperatives and nonprofits to survive under capitalism while > not contributing free labor to corporations.

I've seen a few cases where I think that open source authors and maintainers are getting it in the shorts. Yes there's value - "Egoboo", learning, personal promotion, etc. - but I don't think it's enough in many cases. For example:

* Free software that contributes to the growth and dominance of proprietary, for-profit ecosystems. E.g. I was recently back on Windows for the first time in a decade or so professionally, building an app I'd written in Common Lisp. Scoop (for package management on Windows) and Rosewell (which now supports Windows) made this super easy and pleasant. But ... why would you spend your time writing Free software to benefit such an ecosystem? Unless you're being paid for it, why not spend your time contributing to a Free ecosystem instead?

* Free software that is productised by cloud providers (e.g. [https://www.zdnet.com/article/aws-hits-back-at-open-source-s...](https://www.zdnet.com/article/aws-hits-back-at-open-source-software-critics/)).

* Companies like Google, who are happy to build upon a plethora of open source products to generate billion-dollar profits, but who then turn around and actively give the middle finger to the communities without which they wouldn't exist (e.g. [https://www.theregister.com/2019/04/03/googles_widevine_drm/](https://www.theregister.com/2019/04/03/googles_widevine_drm/)).

<comment><USER8> I think it's hilarious that you're disgusted. How awful _would_ it be if, when you purchased a computer, you were entitled to all of the code that comes with the computer, and full documentation and toolchains? How much worse would your life really get?

Edit: Please, use your words and not your downvote. Your downvotes just mean that you agree with me: Your disgust is an ironic expression of a crab mentality where you regret that other people might be able to have nice things due to us working together as a society.

<comment><USER14> I think you are attracting downvotes because you completely mistook my point.

I support Free software - financially, by evangelising it, by writing it, and by using it.

My point was that I think some large companies are getting a lot more value from their use of it than the authors and maintainers get from working on it.

(Edited to add: I agree wholeheartedly about downvoting vs. replying. I wish more people would reply instead of, or perhaps as well as, downvoting).

<comment><USER8> You are almost to one of the big tenets of Marxism: Employers have it out for employees; managers do not respect laborers. I think that what's missing from your insight is that _every_ corporation, simply by dint of its legal structure and societal appearance, has an unfair modicum of power which it lords over the folks that actually do the work.

<comment><USER14> ... which is why it's entirely reasonable for labour to unionise, to increase their bargaining power with employers.

One of the fundamentals of Marxism with which I disagree, though - at such a fundamental level that it drives an emotional response to the license we're discussing - is that paid labour is necessarily exploitation. That is, I consider the labour theory of value to be incorrect.

I'm reminded of the famous Reagan quote:

> How do you tell a Communist? Well, it's someone who reads Marx and Lenin. > And how do you tell an anti-Communist? It's someone who understands Marx and > Lenin."

</comment></comment></comment></comment></comment><comment><USER15> Ideological contents aside:

> does not exploit the labor of non-members

This is not good legal terminology and so makes it a bad license. People who want to prevent monetization by a closed-source company are probably still better off with the AGPL.

<comment><USER16> That statement is good enough legal terminology. What makes you say otherwise? They just didn't define members, which are clearly - to me - the people with membership interests in an Limited Company, along with the managing member. Or shareholders in a Corporation.

I have no comment on the rest of the clauses so to anyone passing by, don't base any response to me as if I have an opinion.

<comment><USER17> It doesn't define "exploit the labor of non-members". What interactions with a non-member count as "exploiting" their labor?

<comment><USER7> Does this requirement mean computer components that the software is written and deployed on needs to be "conflict free", and manufactured by other anti- capitalist firms? I don't know of anything that would meet that definition.

<comment><USER17> My _guess_ is that what it is trying to cover is having employees who are paid an hourly wage or a salary only rather than being included in the profit sharing.

If that is correct, there would be no restriction on them buying and using supplies from other organizations that do exploit labor.</comment></comment></comment></comment></comment>


Access Trumps Ownership - <AUTHOR> http://sqlrockstar.com/2014/01/access-trumps-ownership/ <comment><USER2> The point of ownership is to have the greatest certainty of access. Lots and lots of relevant, deal-breaking factors are totally missing from the post.</comment>


Show HN: Cocoapod for 2D liquid fill animation / First pod – would love feedback - <AUTHOR> https://github.com/antiguab/BAFluidView

<comment><USER2> Very nice indeed! For a bit more realism could even explore having the wave front reflect off the side of the container as it encounters the wall. Currently the wave front passes through the wall with no indication that it hit it.

<comment><AUTHOR> Solid idea! I'll definitely be adding this to the to do list!</comment></comment>


Mysterious objects spotted in deep space are unlike anything ever seen - <AUTHOR> https://www.livescience.com/circular-radio-objects-space.html <comment><USER2> What a terrible mobile site. Multiple pop ups, constant interruptions.

<comment><USER3> I strongly recommend using Reading Mode in your browser or a decent Hacker News app with that integrated.</comment></comment>


Personal thoughts about Pyston's outcome - open-source-ux http://blog.kevmod.com/2017/02/personal-thoughts-about-pystons-outcome/ <comment><AUTHOR> Key takeaway: _" the [speed] difficulties come from Python's extremely rich object model, not from anything about its dynamic scopes or dynamic types. The problem is that every operation in Python will typically have multiple points at which the user can override the behavior, and these features are used, often very extensively."_

Also, Kevin and Pyston's team are too modest to state this directly in this article: They made good and steady progress, but it's taken them a long time. A lot of that time was spent making them experts on Python runtime and Python JIT options. With several more years of fulltime work we could have a JIT that we all use instead of CPython.

Pyston showed that certain features of Python make it slow and difficult to JIT. The best case scenario for Python's future would have been for Kevin and Pyston to work with the Python core team to decide how to tweak language features for speed (perhaps marking certain object modifications as being "slow" in the same way rust has an "unsafe" marker). With Pyston shutting down, I think the horse of python speedups is out of the barn and is never coming back.

More discussion: [https://news.ycombinator.com/item?id=13534992](https://news.ycombinator.com/item?id=13534992)

<comment><USER2> I don't get why it seems like languages are designed for convenience then later down the road billions of hours are spent trying to get better performance.

I have seen this happen over and over with PHP, Ruby, js, and now Python.

The best solution is to stop using a slow language for these types of applications. Personally I don't see a point to investing so much in these projects, since all it does is encourage developers to continue their bad behavior.

"Oh don't worry about it, they're working on a JIT if performance is ever an issue" \-- lead dev deciding to write something that is expected to handle 10k transactions per second a few years later in Python.

If performance is ever going to be important just use a high performance language and be done with it. Then you don't have to spend years and millions reinventing the wheel like Facebook did with PHP.

<comment><USER3> Every single company that has invested heavily in JITs for PHP, Ruby, Python etc. has done so because they have vast codebases, often millions of lines, written in those languages and rewriting their codebase is infeasible: getting a 20% performance increase for everything from a dozen man-years is vastly cheaper than trying to optimize their entire codebase similarly yet alone rewrite large parts of the application in another language.

Why start writing something in one of those languages? They have high-quality frameworks that make it quite easy to ship early and ship fast: the vast majority of web-based startups fail, or at most stagnate with at most tens of thousands of users, and for them the performance of the common interpreters is fine. If we want to see a phase shift away from them, we need frameworks that allow quick prototyping written in languages which are more easily optimised (and, honestly, JavaScript _really_ isn't bad in terms of writing a JITing compiler).

<comment><USER2> The point I was trying to make is that the idea that a JIT or compiled version will come along in a few years drives people to use slow languages for high performance projects in the first place. This creates a cycle where the language is used in a way that a JIT coming along is a self-fulfilling prophesy.

The irony is that the companies investing the most money into eventual JIT/compilation are the same ones that wrote code believing it would "come around" one day.

The only way to stop the madness is to avoid using these languages for high performance code in the first place.

Having used many fast/slow languages over the years I'll say I don't really notice any increase in productivity one way or another, and no difference in the richness of the library ecosystem.

Are there any studies that show more dynamic languages are more productive? Usually the argument is made that the more abstracted the language is from hardware the better it is to work in, but I don't notice a productivity difference in C#/Java vs Python/perl

<comment><USER3> I'm not sure people have this idea that a JIT or AOT compiled version will come along; certainly ten years ago nobody really had that impression, has the rise of the JS VMs really changed this so much?

To take FB as an example: AFAIK, it was written in PHP because Zuckerberg knew PHP. There was no expectation it would "come around" and it continued to be in PHP because bills had to be paid and rewriting everything would stop new features from being launched and hopefully reach profitability.

If you're a small startup your first goal is almost always to hit profitability, and then your goal is almost always to maintain that. As a result, language choices are frequently made by what founders know rather than any technical merit.

</comment></comment></comment></comment></comment><comment><USER4> Michael Kennedy mentioned on podcast that Dropbox is considering Python 3 with type hints. Pyston is Python 2 only and considerable effort would take to make it work for Python 3.

PyPy is not that bad as they describe it in incentives to start Pyston. PyPy now has cpyext and Python 3.5 support is funded by Mozilla.

Another reason is pluggable JIT to CPython such as Pyjion. This is already possible in Python 3.6.

Currently you can use Cython for hot spots in your CPython apps and even release GIL.

<comment><AUTHOR> "the [speed] difficulties come from Python's extremely rich object model, not from anything about its dynamic scopes or dynamic types. The problem is that every operation in Python will typically have multiple points at which the user can override the behavior, and these features are used, often very extensively. Some examples are inspecting the locals of a frame after the frame has exited, mutating functions in-place, or even something as banal as overriding isinstance."

It's not typing that's the problem, it's dynamic behavior like monkeypatchable methods which require dict lookups.

<comment><USER4> Nowhere did I state that typing is the problem. I was explaining potential move to Python 3 with mypy.

</comment></comment></comment><comment><USER5> "We needed a high level of compatibility and reasonable performance gains on complex, real-world workloads. I think this is a case that PyPy has not been able to crack, and in my opinion is why they are not enjoying higher levels of success."

It's a little tiring, as somebody who regularly puts production workloads onto PyPy, to hear this bullshit. The truth is that PyPy has been regularly spanking Pyston on benchmarks[1] and has been adding Python 3 support while Pyston has struggled to handle its Python 2 commitments.

It's great that you wanted to build a JIT. JITs are fun and cool. But if you just wanted shit to work, you could have used PyPy _years_ ago and not jumped through all these hoops.

Yet again, the fact is that if you invested any code into C extension modules, you picked the wrong path. Good night, Pyston; you were a cool experiment and I hope that your contributors try sending patches to PyPy next time. (Pyjion, you're next.)

[1] [https://pybenchmarks.org/u64q/benchmark.php?test=all&lang=py...](https://pybenchmarks.org/u64q/benchmark.php?test=all&lang=pyston&lang2=pypy&data=u64q)

<comment><USER6> Not being able to use C extension modules is a killer feature for lots of large Python code bases.

</comment></comment><comment><USER7> I for one applaud Dropbox's devotion to Python. They are really helping to move the Python community forward.

I also think Kevin hits the nail on the head regarding what makes speeding up Python difficult. It's the same thing that makes writing Python so enjoyable: the object model.

</comment><comment><USER8> This is not specific to just Python, but I have often wondered if it would be a productive exercise to write a _static_ compiler for a dynamic language which also supported optional static typing. Presumably this compiler could generate code that is faster than an interpreter like CPython, but slower than what you can get out of a high performance JIT, since a JIT can do dynamic optimizations and function inlining. However, once you add static type notations, the static compiler can (for some cases) generate code that's actually faster than a JIT. Typically you'd only need to add type notations to hot spot code, and you could use profiling from previous runs to figure out what type annotations to use and where to put them.

Using a JIT to generate very fast code for dynamic (even highly dynamic) languages is a really interesting topic. But it seems to involve jumping through a lot of hoops to get performance that is a fraction of what you can get out of an early 1990's era C compiler (or Pascal compiler, for that matter).

<comment><USER9> I'm on the Dart team, so this is a topic that's close to my heart.

> Presumably this compiler could generate code that is faster than an > interpreter like CPython,

People have been writing static compilers for dynamic languages for ages. Lisp and Scheme have been compiled for a long time.

My understanding is even simple static compilation works tolerably well for something like Scheme, but object-oriented languages like Python are a different story.

In object-oriented languages, the majority of "calls" are potentially- polymorphic method invocations. In Python, even `a + b` may call some user- defined "+" method which must be looked up at runtime.

If you statically compile that in a straightforward way, almost all of the resulting "static" code is just going to be calls to dynamic method lookup functions. Something like:

        Obj* temp1 = lookup("a");
        Obj* temp2 = lookup("b");
        Obj* temp3 = invoke(temp1, temp2, "+");

You won't really _do_ anything useful at the level of native code, so you don't gain much from eliminating the interpreter's bytecode loop. In fact, you'll likely lose. That above chunk of C is a pretty verbose encoding of `a + b`.

You can look at bytecode is a compression format for dynamic code of that form. The runtime cost of "decompressing it" \-- the bytecode interpret-and- decode loop may be a net win over the memory wasted by the larger native code size and the cache misses and stuff caused by that.

In order to statically compile a very highly dynamic language, you really do need to do some static analysis to eliminate some of the dynamism. That analysis is either very costly (slow, giant monolithic whole program compiles that take ages as in MLton) or doesn't work very well (failing to infer good static types and resorting back to lots of slow dynamic code), or (most often) both.

With Dart, we have a _very_ mature whole program compiler that does concrete type inference, and it's still both too slow to run and not effective enough at inferring good types.

> However, once you add static type notations, the static compiler can (for > some cases) generate code that's actually faster than a JIT.

You can do that, yes, but it requires a few things:

1\. Your core libraries and idiomatic user code need to actually be fairly "static" in terms of the types they use at runtime. If the code really does use a lot of polymorphism and duck typing, all static analysis can do is tell you, "Yup. You gotta do this at runtime."

Python in particular is well-known for relying heavily on duck typing in its core libraries.

2\. Your type system needs to be sound. Once you are within the bounds of static types, the compiler needs to be able to trust that those types are _correct_ , with no soundness holes. Otherwise, it all falls apart.

3\. You need to deal with the boundary between dynamic and typed code. Because of 1, you need to do validation when an object flows from dynamic-land into the typed area. That validation occurs at runtime and has its own cost. If you aren't careful, that runtime cost outweighs the perf benefits you get once you are in static code.

The "gradual typing" folks have been beating on this for years and it turns out to be _really_ hard. Google "is gradual typing dead" to see some of the latest soul-searching.

With Dart, we had 1 since the language has always had an optional static type so typical Dart code is fairly Java-esque. We did not have 2 and that turned out to be a big problem. We have since come up with a new sound type system [1] to address that.

3 is hard, but our strategy has been to be more restrictive on what kinds of dynamic code we allow to flow into typed code. For example, you simply can't pass a List<dynamic> into a function that expects a List<int>.

By the time you do all of these things, I think you discover that you don't really have much dynamism left in your system. It is handy having a dynamic type for some places, like deserialization, JSON, etc. But if you care about static compilation and perf, actually _being_ statically typed is just so much easier, and dynamic typing doesn't really offer that much value as an alternative.

[1]: [http://news.dartlang.org/2017/01/sound-dart-and-strong- mode....](http://news.dartlang.org/2017/01/sound-dart-and-strong-mode.html)

<comment><USER10> > In order to statically compile a very highly dynamic language, you really do > need to do some static analysis to eliminate some of the dynamism.

Or you can require the user to add type annotations, if/when he wants fast code.

In SBCL (a compiler for Common Lisp):

      (defun count-even (array)
        (declare (optimize speed (safety 0)))   ; <- these are the optimisation settings for this function
        (declare ((simple-array fixnum) array)) ; <- the argument to the function is an array of fixed sized integers
        (loop for x across array count (evenp x)))

This compiles down to quite efficient assembly:

       > (disassemble 'count-even)
       ; disassembly for COUNT-EVEN
       ; Size: 65 bytes
       ; 05A7ABC2:       BB17001020       MOV EBX, 537919511            ; no-arg-parsing entry point
       ;      BC7:       498BF8           MOV RDI, R8
       ;      BCA:       31C9             XOR ECX, ECX
       ;      BCC:       31F6             XOR ESI, ESI
       ;      BCE:       31D2             XOR EDX, EDX
       ;      BD0:       498B70F9         MOV RSI, [R8-7]
       ;      BD4:       660F1F840000000000 NOP
       ;      BDD:       0F1F00           NOP
       ;      BE0: L0:   4839F1           CMP RCX, RSI
       ;      BE3:       7D18             JNL L1
       ;      BE5:       488B5C8F01       MOV RBX, [RDI+RCX*4+1]
       ;      BEA:       4883C102         ADD RCX, 2
       ;      BEE:       4883E302         AND RBX, 2
       ;      BF2:       4885DB           TEST RBX, RBX
       ;      BF5:       75E9             JNE L0
       ;      BF7:       4883C202         ADD RDX, 2
       ;      BFB:       EBE3             JMP L0
       ;      BFD: L1:   488BE5           MOV RSP, RBP
       ;      C00:       F8               CLC
       ;      C01:       5D               POP RBP
       ;      C02:       C3               RET

If you don't add the type annotation (the second declare form), the compiler will tell you that it can't optimise the code because of type uncertainty. Optimising a program becomes a dialogue between the user and the compiler.

<comment><USER11> The entire second half of the comment you've replied to addresses this approach at great length.

</comment></comment></comment></comment><comment><USER12> It's hard for me to believe that it's easier (and not only more comfortable for the engineers) to start pyston rather then rewrite their backend in go or whatever. Is dropbox really so big (not in sales, in the size of their backend) that they should manage their own python implementation? Also choosing python for their backend really seems like the wrong language.

Compilers (especially JIT-compilers) are hard to get right and even harder to get performant. Since i am interested in these areas i would evaluate Graal/truffle or Jython. I only looked briefly at it, but Jython seems very un-optimised and the JVM really improved it scripting-language support in the recent years. Is there really no way to solve the native-extension problem of Jython?

But i maybe wrong, any comments?

<comment><AUTHOR> _It 's hard for me to believe that it's easier (and not only more comfortable for the engineers) to start pyston rather then rewrite their backend in go or whatever._

You are correct, it is easier to rewrite in Go. That's what Dropbox learned after putting two FTEs on Pyston for a few years.

But the larger Python community outside of just Dropbox would have really benefited from these two guys continuing to work on Pyston.

<comment><USER12> Of course, getting a fast JIT-Compiler for Python would be awesome! I have been doing some python lately and it's a nice language. I would guess there is such a broad usage in the industry that a committee funded by various companies could take ownership of the project and greatly increase the assigned resources.

I just question whether it's the right move for dropbox. Seems like not actually tackling the problem and instead trying everything so that it just disappears.

</comment></comment></comment><comment><USER13> "Some examples are inspecting the locals of a frame after the frame has exited, mutating functions in-place, or even something as banal as overriding isinstance"

Ok, they're doing all of this in their codebase?

This seems like "trying too hard". Yes, they can do those. It doesn't mean it's a good idea

<comment><USER5> If you don't do this, then it's not Python.

<comment><USER13> I've yet to see (real-life) code that needs these.

<comment><USER14> It's useful when hacking around someone else's code that you aren't supposed to change. Not sure why Dropbox would need it, though.

</comment></comment></comment></comment><comment><USER15> IronPython uses the Microsoft DLR, which uses a JIT, so it's definitely possible that it would work.

<comment><AUTHOR> They can't support C extensions, which are basically required for any real world Python.

Pyjion is a Microsoft CoreCLR JIT that can support C extensions. That's Python's next great hope for speedup via JIT.

<comment><USER4> Pyjion is C Extension based JIT for CPython. Pyjion only uses CoreCLR RyuJIT as a JIT compiler backend. It is NOT aware of .NET types at all.

Note that you can use pythonnet for interop between CPython and .NET for both embedding and extending.</comment></comment></comment>


America's inequalities can cause arab spring like crises - <AUTHOR> http://www.politico.com/magazine/story/2014/06/the-pitchforks-are-coming-for-us-plutocrats-108014.html#.U7rAX7FGOqk

<comment><USER2> Previously posted: [https://news.ycombinator.com/item?id=7953608](https://news.ycombinator.com/item?id=7953608)</comment>


This Developer Harlem Shakes Your Website - <AUTHOR> http://www.wamda.com/2013/03/jordanian-developer-harlem-shakes-your-website

<comment><USER2> Wow. I am speechless. Such a simple, but highly amusing idea. This is going to be huge, nicely done as well using CSS3 animations. So damn amusing! I tried putting Hacker News through it and it's funny because it doesn't really work that well: <http://www.harlemify.com/?url=news.ycombinator.com>

<comment><AUTHOR> Haha - must admit I hadn't tried that. Still gets Hacker News to shake though! I think it's really smart how Monir built on the Harlem Shake's virality, but made it fun for everyone.

<comment><USER2> Yes, definitely. I used it for a project presentation the other day and it was a hit. I started it off with, "So here's what I've been building for the last few months" and Harlemified the site, people were amused.

<comment><USER3> Thanks for sharing the link, I am glad that you both liked it! :D

as you might know, Technically it's really hard to determine the main logo element in any web page, since HTML coding styles differ from site to another, and there is no standard for such a case.

So to overcome this issue I made some assumptions on how can I find the logo element ( the dance Starter element ).

I assumed logically that I should find the keyword "logo" in 3 possible Locations on a visible HTML Element 1- in the "id" attribute. 2- in the "class" attribute. 3- in the "src" attribute of an html <img /> tag.

if I found any, I will simply get the First element found.

But for some websites non of the assumptions will be correct, for this case, I wrote a simple brute-force like solution to try to find an element that is currently visible on the screen view area and has a size between ( 40x80 and 380x380 ), again I assumed the size range here since the logos are usually in this range; based on a simple test that I made.

And I didn't include elements smaller than 40x80 because animating a small element alone is not as cool as a larger one.

So apparently for Hacker News' website, all of my logo finding assumptions failed... I fixed it by adding a code block that handles special cases based on the domain name, and that's by specifying hard-coded jQuery Selectors :)

So it should work fine now, try it again here: <http://bit.ly/ZcxOIc>

</comment></comment></comment></comment><comment><USER4> Previously on Hacker News, hsmaker: <http://news.ycombinator.com/item?id=5235709>

<comment><USER3> I built <http://harlemify.com/> without knowing about this one... and I'm happy for not knowing about it because seriously If I knew, I wouldn't build a new website. :)

Anyway, I built it in 7 hours just for fun.

After testing the other site, I felt that I didn't waste my time building it for nothing, because I believe that I handled much more cases in mine than the other site :)

Such as: \- Ending the song and animation ( ending the show. ) \- Share functionality at the end of the show. \- Passing the same user's browser headers when crawling requested website. \- Handling most of the webpage character encoding types (including Arabic). \- Handling gzip encoded websites ( for example, try mashable.com on both websites ). \- iOS Browser Support.

Enjoy! :)</comment></comment>


How Your Microsoft Points Will Become Dollars/Pounds/Euros/Whatever - <AUTHOR> http://kotaku.com/how-your-microsoft-points-will-become-dollars-pounds-eu-513044467

<comment><USER2> Blogspam for [http://support.xbox.com/en-US/billing-and- subscriptions/acco...](http://support.xbox.com/en-US/billing-and- subscriptions/account-management/microsoft-points-retire-faq)</comment>


How do I deal with a difficult programmer joining an open source project? - <AUTHOR> http://programmers.stackexchange.com/questions/206321/how-do-i-deal-with-a-difficult-programmmer-joining-an-open-source-project <comment><USER2> I dont know why this is being upvoted, the stack overflow question is a personal rant that doesnt really make any sense.

But to answer the question, the best open source projects I have worked with have had peer review, someone else on the project is the one who actually pushes your code into the repo, this works a great sanity test and catches most of the common implications that you may have missed, but another good part of this is that it works as lazy concencus now all the people who may have issues with your patch have the opportunity to discuss it before it lands while at the same time not adding too much overhead and bureaucracy to get code landed.

The other thing is that the code (or individual parts) have to have owners who have the final say in any disagreements, its impossible to get everyone to agree on every change and sometimes an owner just needs to come in and make a final decision, open source that attempts to work as a democracy has a very hard time in my experience.

</comment><comment><USER3> So it sounds to me like the main issue here is that the original poster did not understand Forking and Pulling, and how important it is to compartmentalize your team using those 2 GIT(hub) features?

You don't go handing the keys over to the main repository to just anyone. You make contributors fork your work, then you pull in changes that you like. This leaves you 100% in control of versioning and the main branch of the project.

</comment><comment><USER4> "How do I deal with a difficult programmer..."

If you're not project lead, you don't.

Something along similar lines came up on Ars Technica a little while back "How should I behave as a developer in a project that's headed for failure?": [http://arstechnica.com/information-technology/2013/07/how- sh...](http://arstechnica.com/information-technology/2013/07/how-should-i- behave-as-a-developer-in-a-project-thats-headed-for-failure/)

There are some good ideas there to better engage something like this.

<comment><USER5> Yes, if you can fire them do it. Otherwise it is none of your concern.

<comment><USER2> Thats pretty depressing, a large part of open source development is learning to work with a wide range of other people without being able to fall back to corporate politics bullshit (not that OSS is exempt)

<comment><USER4> This is exactly how Linus deals with things; being as anti-corporate BS as possible. But note, _Linus_ deals with things, not anyone else. If they have a problem, they can go to him first, but in the end, he's the one who deals with it.

Any other developer who's having problems should go through the leadership first or has no alternative but to leave. It sucks to swallow your pride, but taking matters into your own hands (as some of the comments on the SO post suggest) can lead to anarchy. Then everyone loses.

</comment></comment></comment></comment><comment><USER6> Surprised that this isn't in the thread already.

Brian Fitzpatrick and Ben Collins-Sussman from Google have done a bunch of great talks about exactly this: How Open Source Projects Survive Poisonous People ([http://www.youtube.com/watch?v=Q52kFL8zVoM](http://www.youtube.com/watch?v=Q52kFL8zVoM))

They also wrote a book about the topic: [http://shop.oreilly.com/product/0636920018025.do](http://shop.oreilly.com/product/0636920018025.do)

All of their advice comes from their experience running the subversion project. It is also good general advice for dealing with humans/engineers/etc.

</comment><comment><USER7> Communication with the programmer you have an issue with, and communication with the leadership in the project.

If you have personal issues with the programmer, such as their attitude or behavior, bring them up in a personal setting with the programmer himself.

Your issues regarding the way decisions are made within the project should be raised with the leadership.

If you don't communicate your issues to the responsible parties then you can't expect them to change.

</comment><comment><USER8> He should watch the show The Borgias. He should get enough inspiration on how to handle the situation after this.</comment>


Ask HN: What is a good percentage of Unit and Integration test coverage? - <AUTHOR> Basically the title. To give a little bit more context, our team recently introduced sonarqube and while I was OOO they decided they wanted a unit test coverage of at least 80% for new code.
Now, we are working a product that consistes of two UIs (eclipse and Web) and a java runtime. It's been developed for more than ten years and we have around 45000 tests in total, which combine UI, system, integration and unit tests. Out of all of them, 3200 are unit tests, so less than 10%.
So over time we adapted a style of testing a lot of stuff with integration and system tests, which makes sense, as we are heavily integrated into several systems around us (two application servers, eclipse, multiple messaging broker, different database backends, ...).
I think it is totally unrealistic to have 80% code coverage for new unit tests and it showed already. Two members started writing tests that covered a lot, but still were green when I went ahead and deleted production code from that covered code. Which means, a lot of stuff was mocked.
Now, I'd like to know three things. 1. Your experiences in a similar area? 2. Is there some scientific background to code coverage and it's outcome? 3. What are some good arguments to lower the expected unit test code coverage?
Thanks, Sven <comment><USER2> Personally I don't see the point of doing unit testing to do unit testing when you actually need integration testing. Mocking external services is a waste of time in my humble opinion. It's super time consuming and you can mock it wrongly or have a different behaviour in you mock.

I prefer to use integration tests with sonarqube. 80% is then a good threshold.

</comment><comment><USER3> Depends heavily on the codebase I guess. The last two places I've worked (startups in London) have been very TDD and we get 95%+ coverage on new code. This mostly Ruby so we have the ease of RSpec of course.</comment>


Where is the SF news.yc meet?  - <AUTHOR>

If I remember it was supposed to be some where around North Beach on Friday 27th July 2007 between 7pm and 10pm. Where exactly?
<comment><USER2> Hey bluishgreen - the meetup is at our flat 1203 in 2140 Taylor Street (Y Scraper). Email me if you don't know where that is.

<comment><USER3> How is the parking situation there (or if its bad, how is the transit from caltrain/bart)

<comment><USER4> not great but not terrible either, you should be able to find a place to park within a few blocks, bart is pretty far away

</comment></comment></comment><comment><USER5> <http://usc.facebook.com/event.php?eid=2449811475>

The group says the location is TBA.

</comment><comment><USER4> ok, so facebook stopped showing me event requests so I just made the event open so if you didn't get confirmed you can now add yourself

</comment><comment><USER6> are there any requirements to come?

<comment><USER2> bring drinks/food

</comment></comment><comment><USER7> y-scraper

<comment><USER7> there's a facebook group for it too

</comment></comment><comment><USER8> Enquiring minds want to know!</comment>


Show HN: Off The Roost – MVP – Hungry? Discover. Decide. Dine. - <AUTHOR> http://offtheroost.com

<comment><AUTHOR> If you are on a desktop, pretend to be on mobile and slide the pictures (if any) :D

I built this to put a single restaurant in front of my friends' and I's faces to help make a decision on where to eat. In a nutshell, it reverses the traditional Urbanspoon process and gets smarter.

</comment><comment><USER2> I wanted to give my location manually and there was no way to continue the flow without that. Not sure how tiny a minority I am, but thought you should know.

<comment><AUTHOR> Yeah, that's an oversight on my part. I'll be taking care of that soon.</comment></comment>


Graph: Abstractions for Structured Computation - <AUTHOR> http://blog.getprismatic.com/blog/2013/2/1/graph-abstractions-for-structured-computation

<comment><USER2> Two related things I've been studying in more depth lately: Dataflow programming and behavior trees(the game AI concept).

<http://en.wikipedia.org/wiki/Dataflow_programming>

[http://www.altdevblogaday.com/2011/02/24/introduction-to- beh...](http://www.altdevblogaday.com/2011/02/24/introduction-to-behavior- trees/)

The first comes up anytime you want to make a signal processing chain more modular and composable(graphics and audio are the classic applications) and many of its concepts share space with FP theory. Graph demonstrates a implementation built around certain needs of web apps. Note that it seems like implementations vary a lot with the data types - audio processing, for example, may allow for cyclical feedback loops, and mainly distinguishes between two types of data - multi-channel PCM data(which may be split and combined between nodes) and parameter changes over time.

The second describes a form of concurrent finite states with good compositional properties - parent-child relationships that result in concurrency expressions passed back to parents(success, failure, in progress). Coroutines are comparable in power, but put emphasis on direct control of the concurrency, while BTs use modules of state + logic with pre-designed yielding points. (I think other finite state constructs have applications, too, BTs just happen to be my focus right now)

I currently believe that highly-concurrent applications can be abstractly architected as a combination of dataflow, behavior trees, and asynchronous events - each one of those covers a very distinct set of concepts surrounding concurrency problems, and they present natural boundary points with each other.

<comment><USER3> I'd love to talk with you about this design. I've been looking into a similar kind of build and I'm really curious to compare notes.

<comment><USER2> Shoot me an email. (I just updated my profile)

</comment></comment></comment><comment><USER4> When reading the background on Graph from October ([http://blog.getprismatic.com/blog/2012/10/1/prismatics- graph...](http://blog.getprismatic.com/blog/2012/10/1/prismatics-graph-at- strange-loop.html)), I came across this: _Of course, this idea is not new; for example, it is the basis of graph computation frameworks like Pregel, Dryad, and Storm, and existing libraries for system composition such as react._

I wanted to point out that the programming model behind Dryad and Storm represent computations _as_ graphs, but that the programming model behind Pregel is for computations _on_ graphs. It's a subtle difference in words, but an enormous difference in what you actually do.

</comment><comment><USER5> I'm one of the authors of Graph, and I'll be here to answer questions and read comments. Please let us know what you think, and help us make plumbing and Graph better. Thanks!

<comment><USER6> Hi, do you actually do the bookkeeping for the processing inside graph? How do you do this?

I work on a stream processing/workflow engine used by a few large physics experiments. It's declarative in nature too, but we use XML and let users write the glue. We also have the notion of persistent files and variables, although we don't compile and verify dependencies quite so much.

<comment><USER4> Is this what you work on? "Combining in-situ and in-transit processing to enable extreme-scale scientific analysis": <http://dl.acm.org/citation.cfm?id=2389063>

</comment></comment></comment><comment><USER7> Here's a cool presentation on Graph that I watched a couple of days back.

<http://www.infoq.com/presentations/Graph-Clojure-Prismatic>

</comment><comment><USER8> It doesn't take much of a stretch to see Graph integrated with something like Nathan Marz's Storm (also written in Clojure) to provide the distribution and deployment aspect. Have you guys given that any consideration?

<comment><USER5> For now we're focusing on the in-process use case, which we think is underserved and allows the simplicity of Graph to really shine. That said, distributed Graphs (and possibly, integration with frameworks like Storm) are on the horizon. If this is something you're interested in working with us on, please let us know.

</comment></comment><comment><USER9> What graphs let you do that multimethods and/or protocols/records don't?

<comment><USER5> Protocols and multimethods are great tools to manage _polymorphism_ , whereas Graph is about _composition_. We use both extensively in our codebase, and treat them as separate tools in our toolbox for building fine-grained, composable abstractions.

For example, I don't think protocols or multimethods could easily do any of the things mentioned in the second half of the post (execute part of a computation, auto-parallelize it, monitor the components, etc).

That said, there is actually one case where we use Graphs to solve a difficult polymorphism problem, which I discussed a bit in my Strange Loop talk. Our core newsfeed generation logic used to be composed of protocols/multimethods (we tried both), since each feed type (we have about 10) can define different variants of various steps in the pipeline (but most of the steps are the same). This worked fairly well, but as our system grew more and more complex, we found that there was still a lot of overhead, since the protocol had to contain all the steps that could change, leading to lots of extra complexity.

We've replaced all of this with Graph, where we just define an 'abstract' graph with the most common steps, and each feed type modifies the graph by changing or adding steps -- and we've found this way to be much simpler and easy to understand than what we had before.

This case is special, since it involves both a complex composition and polymorphism. Everywhere else in our codebase, we use (and love) protocols and multimethods for polymorphism.

</comment></comment><comment><USER10> This looks very interesting.

It seems to be similar to something I've been thinking about and trying to build lately, so I'm definitely going to check this out.

<comment><USER5> Thanks! We'd love to hear your feedback -- and if Graph doesn't meet your needs, work with you to fix that.

</comment></comment><comment><USER11> I think this could be used to solve similar problems for event-driven programming. For instance, in Aleph/Lamina (async clojure library), pipelines work great when only one value is returned. But if you want to wait for two remote calls to return in parallel, and feed both results into the next function, the syntax can a bit painful. Here, you could supply something like async-compile which would work similarly to parallel-compile but use pipelines and merge-results under the covers.

</comment><comment><USER12> This initially looks like an IOC Container (StructureMap, etc) with automatic dependency resolution, except you can control the compilation of the internal graph. Is that accurate?

<comment><USER5> Interesting, I hadn't heard of StructureMap. It seems related, but Graph is less complex -- just the dependency and composition parts, without being tied to any particular use case.

</comment></comment><comment><USER13> Brilliant! Been following Prismatic/Bradford for a while now and thought you would not share your 'Graph' library.

If one has not stumbled upon specific use cases like disparate data sources, custom/widely varying transformation logic between these data sources and more then it might be difficult to appreciate your contribution. Thanks for this..even if not right away I hope to utilize it for our startup!

</comment><comment><USER14> This is quite amazing, and frankly quite an eyeopener in the way large clojure projects can be organized. Just curious though: does Graph handle cycles?

<comment><USER5> Thanks!

If by 'handle cycles', you mean 'throw an exception', then yes :). Graph models single-pass data flows, which must be acyclic, and the (graph) constructer and (*-compile) methods throw if you give them cyclic specifications. Do you have a particular use case in mind where cycles are desirable?

<comment><USER14> I was thinking of nodes with feedback loops which is desirable in some data flows. Particularly learning agents.

<comment><USER5> I see. For streaming computations we typically have a Graph behind a thread pool, so a node can always resubmit a datum for another go-round -- there's no concept of sending an updated datum 'back' to another node within a particular execution though.

</comment></comment></comment></comment><comment><USER15> Can graph programs modify the graph they're in, or is that completely fixed? Add new computation nodes, say, if necessary.

<comment><USER5> Any particular execution is fixed once it's compiled. But it's easy to compile different variants of a graph and choose between them based on the input parameters, if that's all you need.

<comment><USER15> I was referring more to "Do this computation and then based on the output, run X or Y" more for automated decision making. When the computation is expensive and you're going for "real time" (people waiting around) then it's nice to shave any measurable fraction of a second.

</comment></comment></comment><comment><USER16> Related functionality as data, I like it.

</comment><comment><USER17> Something which cannot be made out of conses in Scheme or CL?)

<comment><USER5> I'm not sure I follow, can you elaborate? I think something similar could be done in CL, although some of the design decisions might be different because Clojure has nice map literals and function metadata.

<comment><USER17> I'm trying to get what all excitement is about. "We have put functions and data in the same graph-like data-structure because Clojure is so cool"?)

<comment><USER5> No, this isn't just about Clojure. You could do similar things in Scheme, or CL, or even Python or Ruby.

What's cool about this isn't that we've managed to put functions in a data structure. It's that doing this in a particular way allows us to describe computations in a declarative way. This declarative specification opens up lots of interesting avenues to do new things with our code that weren't available before.

Of course the idea of declarative programming isn't new either, but we think this particular instantiation is cool because it's _extremely simple_ and close to the language. Writing a Graph doesn't feel any heavier than writing the corresponding non-declarative code, and this is crucial for making it actually useful in many kinds of situations (rather than just cases where heavy lifting is necessary, like distributed stream processing for example).

<comment><USER17> Yes, using code as data and creation of the code when a program is running is the most powerful feature of Lisps.

My point is that Clojure isn't a Lisp and JVM isn't the best possible platform, and embedded in a Lisp DSLs are even more powerful because of the common (for code and data) underlying data structure - conses.

Of course, I know the counter-points about "Java is everywhere" and "Interloop with existing Java code".

As of heavy lifting or whatever to call it, decent CL implementation would be faster (compiled native code), consume _much_ less resources (predictable memory allocation patterns), and more manageable (behavior much less depended of the system load and how other processes behave).

<comment><USER5> I programmed in CL for several years exclusively, and think it's an awesome language. But I also really love Clojure, and think it's the the most beautiful Lisp (or S-expression-oriented language with a read-eval-print loop, if you prefer) I've had the opportunity to explore. To each their own.

<comment><USER17> Thank you for an alternative definition. In my opinion adding more data- structures into a Lisp ruins it. It is a List Processing, for John's sake.)

More seriously, having exactly one common data-structure for code and data _is_ what holds everything together, the source of power, compactness, elegance and readability.

A small additional effort, a self-discipline of using lists correctly (remembering the costs) everywhere and using hash-tables and arrays only when absolutely necessary is the way to write decent Lisp code.

In case of Clojure it is just a mess.</comment></comment></comment></comment></comment></comment></comment>


How to Become a Hacker - nhc-forum http://nhc.bijayacharya.com/viewtopic.php?f=6&t=3&sid=62271e67c0eb5d95ec35b81cecda5641 <comment><AUTHOR> Feel there is an underlying meaning here.....

</comment><comment><USER2> Forum sign-up required.</comment>


NASA Receives Science Report on Europa Lander Concept - <AUTHOR> http://www.jpl.nasa.gov/news/news.php?feature=6737&utm_source=iContact&utm_medium=email&utm_campaign=NASAJPL&utm_content=daily20170208-4#.WJwUdGujWzA.hackernews <comment><USER2> that's sound good...!</comment>


How to Detect Ad Blocking using Javascript and Google Analytics Event Tracking - <AUTHOR> http://idahospuds.blogspot.com/2013/03/how-to-detect-block-ads-using-google.html

<comment><USER2> I'm sorry, but if anyone's business depends on ad-revenue, find a better way of making income. The ad-block rates will go up massively.

People are fed up with the crapload of popups, layerads, "you will be redirected to X in Y seconds", and other annoying kinds of advertising... for many websites one HAS to use an ad blocker to make the sites actually fucking usable.

The problem for the ad industry is that they opened the box of pandora and it's WAY too late to get the trust of the users back.

</comment><comment><USER3> "This will only work in the cases where Google Analytics is not blocked."

Without Javascript and Analytics: [https://docstrings.wordpress.com/2013/03/31/detecting- adbloc...](https://docstrings.wordpress.com/2013/03/31/detecting-adblock-and- other-adblocking-addons-without-javascript/)</comment>