Build a Serverless Web Applicaion - <AUTHOR> https://aws.amazon.com/serverless/build-a-web-app/ <comment><USER3> Is anyone else slightly peeved by the popularity of this misnomer?

If you are _paying for servers_ , your app isn't _serverless_. Real serverless to me means decentralized p2p and offline apps. I also consider p2p-routed apps with centralized control serverless as long as they continue to function in a read-only state when the control server is down.

<comment><USER4> No, I'm not peeved by the "misnomer". What is getting a bit annoying is every conversation about serverless being dominated by snarky comments about there being no such thing. As software professionals, we all know there is a server involved in responding to requests for our applications. But 'serverless' does accurately describe services like AWS Lambda, because the experience of development is such that you don't really have to think about servers, along with all the complexity of provisioning, configuring and managing them.

TL;DR: It's just a name. Get over it.

<comment><USER5> > But 'serverless' does accurately describe services like AWS Lambda,

Serverless means no server, which is false, because there is a server that runs your code.

> because the experience of development is such that you don't really have to > think about servers, along with all the complexity of provisioning, > configuring and managing them.

That sounds like you are talking about 'web hosting' or 'application hosting', which existed decades ago, don't see any point in inventing a confusing buzzword when we have perfectly fine terminologies already.

> TL;DR: It's just a name. Get over it.

It's a useless and confusing marketing buzzword that makes having intelligent conversations harder. I would love to understand your rationale for defending this confusing buzzword.

<comment><USER4> > It's a useless and confusing marketing buzzword

A marketing buzzword is something like "web scale", which might be used by a non-technical person to describe a technology, but has no real technical meaning. Serverless is not a buzzword because it is the name used for a collection of technologies, even if it isn't a specific product. "Software containers" also falls into this category.

Yes, serverless architectures involve servers. Well done for recognising this, you are very smart. In fact, you are smart enough to realise that the architectures we are talking about here are very distinct from traditional web hosting services. So, it's convenient to use a different name for them.

Finally, it doesn't really matter whether you like the term or not. It is the accepted term and it is here to stay. If you want to engage in conversations on the topic you will need to move past your misgivings.

<comment><USER5> > A marketing buzzword is something like "web scale", which might be used by a > non-technical person to describe a technology, but has no real technical > meaning. Serverless is not a buzzword because it is the name used for a > collection of technologies, even if it isn't a specific product.

What collection of technologies does it name? And how is that different from "web scale" technologies that is also a group of technologies?

> Yes, serverless architectures involve servers. Well done for recognising > this, you are very smart.

If you would have strong arguments, you wouldn't need snarkiness.

> In fact, you are smart enough to realise that the architectures we are > talking about here are very distinct from traditional web hosting services.

That's the exact opposite of what I've said, if you look at it from a server management point of view, traditional web hosting and "serverless" are one and the same, both are used to run the code you upload and both are managed by someone else.

> So, it's convenient to use a different name for them.

It's not convenient, it's confusing.

> Finally, it doesn't really matter whether you like the term or not. It is > the accepted term and it is here to stay.

Accepted by who?

> If you want to engage in conversations on the topic you will need to move > past your misgivings.

If you want to engage in conversations on any topic with anyone, you should read what's on this website [https://yourlogicalfallacyis.com/](https://yourlogicalfallacyis.com/) and stop doing them.

</comment></comment></comment></comment></comment><comment><USER6> I am a believer in "Serverless" and that it will play a primary role in a large portion of developers' lives... _in the future_

My experience with servless is that while this may be true at a certain level:

> No server management

I feel like in Amazon's current toolset, you are MORE aware of "servers" than you ever have been before. It's just abstracted into a different layer (cloudformations, etc). The boilerplate for severless projects it's a complete mess, imo.

Besides that, the Develop/Test/Deploy/Manage lifecycle is really difficult to reason about. I would hate to be the one on call when a serverless service goes down and I have to figure out what's going on!

I look forward to Amazon releasing more tools (or cleaning up the ones they have) to make Serverless a breeze to work in.

[EDIT] clarification of a couple minor things

<comment><USER7> That future is being created, you're probably just less aware of it than you think.

Serverlessconf just announced their NYC conference, expected to be the largest yet. If Austin is any indicator, it will be huge. [1] Tim Wagner at AWS likes to spout the stat that billions of financial trades are already being processed per day on AWS Lambda. The "serverless" market is gigantic and growing rapidly.

And then there's companies like StdLib [2], where we've built an abstraction layer on top of serverless architecture that turns remote procedure calls (and APIs) into first-class citizens of your development environment with a single package and allows you to ship applications without ever worrying about infrastructure. We just closed a $2M seed [3] and are announcing some transformative partnerships shortly. We're tackling some of the exact issues you have problems with, and we're not the only ones.

The future is already here. It's just not evenly distributed. ;)

Disclaimer: I am the founder of StdLib. :)

[1] [https://nyc.serverlessconf.io/](https://nyc.serverlessconf.io/)

[2] [https://github.com/stdlib/lib/](https://github.com/stdlib/lib/)

[3] [https://techcrunch.com/2017/06/27/stdlib-just- raised-2-milli...](https://techcrunch.com/2017/06/27/stdlib-just- raised-2-million-to-connect-businesses-via-apis/)

<comment><USER6> Oh, I know it exists, and I know tons of people are using it.

But like I said, I don't think that the tooling is there right now for mass migration (or anything like it). It's really difficult to test/debug. It's difficult to dianose issues. It's difficult to even set up a "Hello World" API Gateway that uses lambdas. It's painful... as are all new technoligies.

I guess what I mean to say is: I look forward when I just jump on AWS and start using it for a production app, and the boilerplate and amount of config files I have to maintain drops.

I'm also aware of the tool called "serverless" ( [https://serverless.com/](https://serverless.com/) ) which is also a step in the right direction.

<comment><USER8> Not trying to be spammy here, but I want there to be awareness that there are startups working on tools to address these gaps. Our startup, IOpipe[1] is working on one set of tools to ease development, debugging, and instrumentation for serverless applications. Aside from Stdlib and ourselves, there are others working in areas like security and CI/CD as well. These are early days, and tools will soon become abundant, but at least there are a handful to try out today.

[1] - [https://iopipe.com](https://iopipe.com)

</comment></comment></comment></comment><comment><USER9> In principle I really like this way of developing, and especially prototyping, but as it stands right now Google is doing a significantly better job with their tools than Amazon.

Amazon got the same guts, but their presentation of these tools is horrendous. It's aimed at the highly skilled developers who would get the concepts in a blink, with the only problem that these are the same people who can spin out servers with more pleasure and probably in the same amount of time.

Google meanwhile, with their purchase of Firebase, makes some really appealing tools that are not just easy to use, but also easy to understand. Besides, their dev outreach is quite stunning, and so is the email support.

Would love to see Amazon compete, but they have a long road ahead.</2c>

<comment><USER10> > It's aimed at the highly skilled developers who would get the concepts in a > blink,

Do you have any advice on how we could make this stuff more accessible to new developers?

When you say the presentation do you mean the overall marketing, the UX, or just the CSS?

<comment><USER9> All of the above. AWS is the Google Cloud right now, but it's missing the "Firebase" equivalent.

I know a ton of people who have no problem spinning out an EC2 instance to scrape the web for all their needs in a heart beat, and yet I don't know many who just want to make software for people, without all the complexity of setting things up.

Look at Firebase, they have "Authentication", "Database", "Storage", and "Functions". If I want to Auth users into my app, it's quite literally a one line of code. Meanwhile Amazon has "Cognito," which requires setting up identities, rules, connecting it to other AWS services ... _head explodes._

That's a small example, but AWS is built for some serious enterprise business, and if you want to make something like Firebase, you need to repackage it all, and dumb it down to a simple use case. Run whatever 265 services are necessary in the background, but give the user 1 thing to worry about.

I fully understand that would be really hard to do, but that's the point. Gotta do the hard thing for the user, and they will come.

As far as UX/css/visual, it's all of it. Like another commenter mentioned, in order to make things pleasant you gotta move away from the AWS style guide and towards something unique and friendly. Right now the entire AWS website look like it's been built in a spare time by a sys admin (because it probably has been).

IMO it's okay to make "AWS Light" a stand-alone offering, like Heroku, but that means you need to commit to maintaining such a service, and from what I hear through the grapevine Amazon is quite happy to be the piping for the internet, without having to run the consumer side of things.

<comment><USER10> Thanks for all the insight!

Have you seen lightsail or codestar?

</comment></comment></comment></comment><comment><USER11> Sorry what's new about this? Aren't all the pieces existing AWS services? They've just put it in a nice graphic with the architecture drawn out.

Not saying I don't like the architecture (actually think it's quite useful for many scenarios). Not sold on using DynamoDB (I'd rather use a traditional SQL store).

<comment><USER12> I really agree with you on this one, my acquaintances that have actually used this, use it for running bulk jobs that are triggered by S3 changes.

<comment><USER13> I wanted lambda for a bulk data processing task that would have been a nice use of it, but the 5 minute max on a process wasn't going to work for my use case. Maybe someday I'll find a use for it

<comment><USER11> Is the task not splittable?

I had to use lambda at work to import files from s3 ranging anywhere from a few megs up to 10-20gb. Ended up using two lambdas (one for creating byte ranges and another for processing those ranges).

</comment></comment></comment></comment><comment><USER14> Amazon Lambda is like Microsoft Windows was in its early stages. It's a fast- growing, closed/proprietary system for running apps.

I wouldn't be surprised if it achieves a near-monopoly for a while... But I think that eventually more flexible open source solutions will take over. Open platforms like Docker and Kubernetes will likely replace Lambda just as Linux replaced Windows on the server-side.

<comment><USER15> > Open platforms like Docker and Kubernetes will likely replace Lambda

I am not sure if Docker/Kubernetes are comparable with Lambda. But currently there are some attempts to develop an open source alternative to Lambda and other proprietary FaaS (Function as a Service) platforms like OpenWhisk for Cloud Foundry: [http://openwhisk.org/](http://openwhisk.org/)

<comment><USER14> True, not directly. So it's not as clean a comparison as Windows vs Linux so I meant it more in terms of the ability of one to replace/supersede the other.

</comment></comment></comment><comment><USER12> Is there any evidence other that actually backs the notion that you save money on a serverless architecture? As far as I can tell, it feels like once you use this kind of mechanism, you're tied to AWS for the foreseeable future.

<comment><USER10> I did this analysis in my (admittedly crappy and rushed) pycon web talk - <USER2> some cases you can save a lot of money. In other cases you'll pay more. Skip to the 20 minute mark. Also I work for AWS so general disclaimer that I'm biased.

[https://m.youtube.com/watch?v=F8-y_d7l-2c](https://m.youtube.com/watch?v=F8-y_d7l-2c)

An interesting thing to note is that efficiencies in this space only increase. Which theoretically lowers prices. You also get some pretty rapid and massive parralellization when you need it. The pywren project has some insightful examples and benchmarks of teraflops of compute and 10s gigabits/s of data arriving as part of a single function call. It's a different paradigm that will transform a lot of workloads but not all.

It's hard not to get swept up in the frenzy around it.

<comment><USER12> Good talk! But how would you go about creating something like graphql on AWS lambda? Are you aware of any tutorials?

</comment></comment></comment><comment><USER16> \- watchout for latencies (multi-zones apps) time of your lambda functions.

\- keep your lambda functions warmed up

\- node has better warmup time than java lambdas (not sure of python)

\- memory size impact of lambda functions (negligible in most cases)

\- manage lambda timeouts

\- dynamodb latencies are negligible

\- api gateway typically does not introduce latencies

\- multitenancy challenges with single store of dynamodb

<comment><USER17> > api gateway typically does not introduce latencies

I haven't checked my function thoroughly yet, but I think API gateway introduce quite a bit of latency (at least in my case). My maximum invocation time is 600ms, according to AWS Lambda monitoring tool, but network request takes around 1s (checked in chrome dev tools).

Also, worht noting is that my code is hosted in the us-east region and I'm based in Europe. I'm not sure how much latency this introduce.

<comment><USER18> Probably about 100ms or more on each request (to and from). Slower depending on location and network conditions.

<comment><USER16> Yes very much possible if you are requesting across zones. I did extensive testing within zone and couldn't notice anything more than 2-3ms.

</comment></comment></comment></comment><comment><USER19> Amazon wants to control markets by leading in with "convenience" factor but they will sheist you.

Buy stuff from Amazon... but for Pete's sake don't make them a core project dependency for your software projects. You only have yourself to blame when they start ratcheting up their prices.

<comment><USER7> AWS is the system architecture of the web, full-stop. Well, a system architecture, alongside Microsoft Azure and Google Cloud.

Avoid them at your own peril and detriment to your ability to execute as a business.

With the exception of a few masochists, we don't write software in machine code anymore. Same will be said for cloud infrastructure (actually, it's already there). Breathe and accept the future, focus on building what's next and / or adding value to your business.

<comment><USER20> Right now I spend $240-600/mo for 192TB of bandwidth. I just used AWS's calculator and they charge $38,876.48/mo for the same thing.

I don't subscribe to the theory that this is "the future".

<comment><USER21> Indeed. It's a big cost escalator. We have split architecture across two DCs we own and AWS. AWS is a bad deal when you get to our size. If we moved our DCs entirely into AWS we would literally triple costs overnight. On top of that, once it's in AWS, a lot of the services that they provide aren't portable. You're stuck and it's expensive to unstick these. The tooling isn't very mature either. You'd be surprised how difficult it is moving 22TiB of file storage to S3.

The future is bare metal IaaS with something non-provider specific that sits on that as that is 100% portable between Cloud/in house. Docker/openstack that sort of thing.

<comment><USER10> >You'd be surprised how difficult it is moving 22TiB of file storage to S3.

Really? I deal with this daily. Can you email email@example.com - if you're genuinely having or had issues moving 22TiB to S3 please LMK as that's abnormal for our customers and I want to fix it for you and others.

<comment><USER21> Thanks have passed your details on to our expert in this space.

For the public record this was mainly down to the problem of client reliability and migrating data live when there are 8,000 people hammering it with reads and writes. We went for a read/write through system and background synchronisation. The latter we had real reliability problems with and the upload performance wasn't great.

Also ROI didn't stack up in the end versus our SAN so we canned it.

<comment><USER10> Ok, if you guys pursue something like this again definitely check out something like storage gateway. It can help reduce the load in a live migration scenario like this.

</comment></comment></comment></comment></comment></comment></comment><comment><USER22> Mostly stoked about serverless because I won't have to deal with grumpy backend devs anymore

<comment><USER23> Care to elaborate?

<comment><USER24> Typical front end dev drivel.

</comment></comment></comment><comment><USER25> Isn't serverless just the present-day incarnation of timesharing?

<comment><USER26> There is very little "true innovation" in software these days, its usually a rehash of something from years ago, plus some fancy marketing.

</comment></comment><comment><USER27> Amazon has invalid 404 links to the tutorial sections on the main page.

"This workshop is broken up into five modules."

And most of those module links are broken too ;-)

Can only be viewed in a step-by-step mode.

<comment><USER10> Where are the 404s? I'll notify some folks to fix them. I'm not finding any but I'm also on mobile.

<comment><USER28> Modules in the list at the bottom of this page:

[https://aws.amazon.com/getting-started/serverless-web- app/](https://aws.amazon.com/getting-started/serverless-web-app/)

are broken - but the links to the modules at the top of the page work OK.

<comment><USER10> This should be fixed now (shift + refresh)! The links went out missing the "modules" in the path. I think we're going to write a quick lambda function to test for broken links in any post. Anyway, I want to apologize for the poor user experience. We should have caught this!

</comment></comment></comment></comment><comment><USER29> It's great that AWS has a tutorial for serverless. Their docs can be a bit hard to get through.

If somebody is looking for an even more comprehensive guide to building full- stack apps with Serverless and React.js, I created [http://Serverless- Stack.com](http://Serverless-Stack.com).

It's open source and up to date with detailed screenshots and code samples.

</comment><comment><USER30> I'm a little confused as to what the point of cognito is. Wouldn't users typically be managed by the application and data regarding the users typically be stored in the db? Why do we need another service for this?

<comment><USER31> In a microservice architecture, there's usually a users/authorization service. It makes it easier to have multiple services serve a single user pool. If you only have one public service then there's probably no point in having a separate service for users/authorization.

</comment></comment><comment><USER32> Has anyone used Zappa in production? What has your experience been like?

<comment><USER10> I have and I had no issues. Website had peak load of only a few hundred requests per second and was Zappa running flask. Not sure if you consider that production or not.

Disclaimer: I work for aws

</comment></comment><comment><USER33> Can I absolutely control my cost with AWS? I've read several horror stories about runaway costs for various reasons. Certainly the complexity of AWS, the fragmented nature of the service each of which seems a world unto itself make it hard to understand. Then there was a story about AWS changing a policy that ended up costing users $1000's. ????

<comment><USER34> AWS has pricing alerts so you can set limits to specific resources and it will notify you when it tips over. Not that it cuts off the service, you'll still be incurring charges, but at least you can know straight away that something happened.

There's free tier as well if you want to just try out without commitment

</comment></comment><comment><USER35> Has anyone used RDS from lambda? Don't want to use Dynamo but perhaps for some key Val store.

<comment><USER36> Hopefully we will see AWS release a managed serverless SQL db solution at re:Invent 2017. Like DynamoDB but for SQL.

<comment><USER37> > Like DynamoDB but for SQL

This would basically be the game-changer for building backends on lambda.

The absolute key and most valuable piece in most backends is the persistent datastore. I absolutely want to build functional stateless business logic on lambda, but I absolutely will not, no matter the gain from not having to manage servers, have my database be dynamodb or any other non-RDBMS, for any serious application in 2017.

<comment><USER10> Why not? Plenty of serious companies use DDB or MongoDb or plenty of other databases for their "serious applications"... this seems like a rule based on a bad experience. Would love to hear about it.

<comment><USER37> Why not? Well, a caveat: I will always use an RDBMS _when the data is inherently relational_. In a "serious application" (I realise the term serious is hyperbolic here and I shouldn't have used it) this is true most of the time.

If it weren't true, then of course I might consider using DynamoDB or similar. But I think these situations are few and far between.

I'm often wrong though and happy to learn about some counter examples if you have any / have my assumptions corrected.

</comment></comment></comment></comment></comment><comment><USER38>
        > With serverless computing, your application still runs on servers,

...</comment>


Show HN: Integer Programming in the Browser - <AUTHOR> https://www.strandmark.net/wasm/glpk.html <comment><AUTHOR> Despite the URL, this is not Webassembly. I will try that next when it is more supported.

<comment><AUTHOR> Now it uses Webassembly and Web workers to run in the background.

</comment></comment><comment><USER2> Another online service that can solve IP problems is [https://neos- server.org/neos/](https://neos-server.org/neos/)

<comment><AUTHOR> That service is the best! However, it requires submission so it does not have as fast compile-run cycle.

</comment></comment><comment><USER3> GLPK is the GNU Linear Programming Kit.</comment>


Name.com Responds to HTP Security Breach - <AUTHOR> http://pastebin.com/We3xgT4J

<comment><USER2> Interesting that they don't talk about any action they have taken in response (beyond asking their customers to perform some action and implementing vague "security measures"). I would hope that they've identified and fixed the core security issue that HTP exploited.</comment>


800TFLOPS Multicore IC for Realtime Ray Tracing - <AUTHOR> http://techon.nikkeibp.co.jp/article/HONSHI/20090629/172373/

<comment><USER2> There must be a mistake somewhere. 800TFLOPS / 750MHz = 1 million FLOPS / cycle. As there are 130 million gates you would need to do 1 FLOPS with 130 gates. At 6Ghz you still only have 1040gates which is not going to cut it for 32 bit math.

Edit, Ok, [http://techon.nikkeibp.co.jp/article/HONSHI/20090629/172373/...](http://techon.nikkeibp.co.jp/article/HONSHI/20090629/172373/fig1.jpg) has 88TFLOPS _equivalent_ per IC which might be possible if equivalent is loosely defined. And with 9 of those you might hit 900TFLOP's _equivalent_.

</comment><comment><USER3> Thanks for posting, this is very interesting. I wish they included more details.

The realm of realtime computer graphics has been totally dominated by the SGI+Renderman line of thinking, and realtime raytracing has only made occassional appearances in demos or academic research (and possibly the pipe dreams of Intel executives wishing for a disruptive technology to make up for their weakness in graphics).

It's fascinating that they're willing to build custom chips for this purpose. Are Toyota's needs simply so advanced that they couldn't do anything else? Or will this be another Cell processor type project, a chip launched with high hopes and only one major customer?

<comment><USER4> Automotive applications are different in that they're inordinately concerned with the appearance of reflections, especially in areas such as the appearance of the headlights. This makes non-raytraced approaches substantially less valuable in the design process. They really want to know what the shiny bits will look like ahead of time.

Raytracing still suffers form the performance implications of incoherent memory accesses patterns for non-primary rays, but when you want physically correct reflections and shadowing to fall out of the rendering architecture naturally, an implementation of ray tracing in hardware starts to look like a pretty spiffy idea.

Are there other fields that require physically correct models for reflections, etc? None spring to mind for me right away.

<comment><AUTHOR> Designing lenses and mirrors for things like telescopes and lasers, I'd imagine.

Also, a slight adaptation could be good for radiation and electromagnetic research (e.g. how well a material blocks beta radiation, etc.)

</comment></comment></comment><comment><USER5> It's amazing that these chips aren't sacrificing precision for performance, operating on Bezier curves.

While we're on the subject of realtime raytracing, does anyone have insight about the polygons vs voxels debate for normal (i.e. desktop apps, games) programmers?

<comment><USER6> The last time I heard anything about this, the bottleneck on the voxel side was problems with surface texturing. That was some time ago though, and I'm not sure what progress has been made on that front.

</comment></comment><comment><USER7> It seems having followed this over the last several months that many argue that ray tracing is really great, and obviously the way the world works, but scene graphs and filters and such with existing matrix transformations like in DirectX and OpenGL are always going to outperform raytracing because it is just inherently simpler. However I don't do too much development in this area, so I would welcome any other perspective!

<comment><USER8> I don't think it's a matter of performance, but of realism. If you want realistic calculations, you must go to a ray tracing algorithm, there's no substitute.

If you don't need 100% realism, though, you can probably make things look 99% as good with "fake" effects, with better performance.

<comment><USER6> Correct. The "fake" effects have limitations and you can break them under certain circumstances, so raytracing is always a good choice when you aren't sure how the materials will be used and under what conditions they will be rendered.</comment></comment></comment>


Consensus-Driven Development - <AUTHOR> http://www.nczonline.net/blog/2015/04/14/consensus-driven-development/

<comment><USER2> Personal anecdote:

At one of the first programming companies I started at, there were no coding/database standards. When we decided we needed them, we appointed a few people to go over all the commonly used standards and present them to the group.

As we went over each standard, we either accepted it, rejected it, or proposed alternatives as a group. Often, we would have multiple people arguing different sides over each issue. Generally, it was just a case of someone being used to doing things a certain way, but having no real justifiable reason. In those cases we ended up voting on the outcome.

In the end, we had pretty good standards (in my opinion). While this wasn't consensus-driven, it ended up being strongly accepted in the company. I think this was because everyone was involved in the process, and you had a positive say in at least 1 or 2 of the standards that made it. Of course, eventually we added commit hooks to sniff out if you weren't following the standards :p

Sorry, there's no real point I'm making here...

Every company since then (except my current one), management just "decides" how standards/processes are going to be and announces it. Those usually ended up not being adopted.

</comment><comment><USER3> Perhaps less about programming and more about society, I'm really not a fan of simple "majority rules democracy" _or_ consensus building—which feels more like an enhanced version of the former than a distinct approach. Either approach is, in its own way, autocratic and ultimately bulldozes over anyone whose views and preferences don't match up well enough.

Instead, I'm fascinated with systems designed to robustly let people actively disagree but still coexist and work together. The goal is not to sway people's views or push them in a certain direction, but to allow different people to hold to different, possibly conflicting, outlooks and still coexist peacefully and productively. This way, even if you hold some minority preferences—everyone is different, so this is not a surprise—you're still free to do your thing.

That's, coincidentally, one of the reasons I'm a fan of _some sort_ of capitalism: capitalism manages to optimize for _everyone_ without needing a single goal handed down from above—or passed up from below. In a word: decentralization.

And you know, this could be applied to software too. Why does everyone at your company need to do everything in the same way? Sure, you'll get some minor advantages in compatibility, and it will be easier for people to move from project to project, but those advantages are limited—but very visible. I believe they're overvalued for just this reason: experience pushes people away from problems that are _clear_ but not ones that are subtle.

So, from this point of view, I really like how some modern companies (Facebook and Prezi come to mind) are starting to organize small teams in self-contained ways. You expose and support a particular interface to the rest of the company, but can use whatever languages and whatever technologies you want internally. Then, instead of the whole company being a big, uniform blob of interconnections, you have a disparate network of self-contained systems with clear interfaces.

With this approach, each team can work with whatever technology fits their problem—and them—best. Moreover, you get the advantages of low coupling within your internal technologies, because the teams are, by necessity, distinct.

Now all this needs is a catchy name to rival "consensus-driven development". "Decentralized development", perhaps? It's alliterative.

</comment><comment><USER4> This is another name for "design by committee".

The best systems are designed by one hacker working alone, or a small team who share the same vision. (Or if they don't, then the system only reflects a small number of visions integrated into a coherent whole.)

Rather than "take the temperature out of the room", what you want is "go to a different room" (where you can just write code).

</comment><comment><USER5> Good article. I'm planning to write about conflicts in product choices soon and will reference this. Given that it's a common and important issue, I tend to see relatively few articles on contention in teams about code and features (Michael Lopp a.k.a Rands in Repose does cover this topic in his writings.) For me it's one of the most emotional and cognitively intense parts of working on digital products, and requires seriously engaging one's analytical and communication skills.

</comment><comment><USER6> Consensus is not always good or always bad. When to use it depends on the situation. Here's an article explaining:

\- [http://sloanreview.mit.edu/article/when-consensus-hurts- the-...](http://sloanreview.mit.edu/article/when-consensus-hurts-the-company/) [paywalled]

\- [http://ssrn.com/abstract=2579745](http://ssrn.com/abstract=2579745) [open access]</comment>


"Awakenings" in Advanced Dementia Patients Hint at Untapped Brain Reserves - <AUTHOR> https://www.scientificamerican.com/article/awakenings-in-advanced-dementia-patients-hint-at-untapped-brain-reserves/ <comment><USER2> This makes me think of the lucidity I witnessed in the people I knew who later committed suicide.

In the hours, days, and sometimes weeks before suicide, their depression was gone, their insular thoughts vanish, and a deep peacefulness and a period of lucidity that exceeds any capability for such wisdom and thoughtfulness beforehand was visible.

I dislike having seen enough to spot this pattern.

<comment><USER3> Maybe that lucidity was the final push for them. To experience the peacefulness they might always enjoy were they healthy, and then having to go back to the misery of their affliction after.

I'm sorry for your loss.

<comment><USER4> Robert Sapolsky described it this way [1]:

> You get someone who is severely depressed, like to the point of > hospitalization, and when they are absolutely crippled with psychomotor > retardation, that's not when you worry about suicide.

> This is someone who's having enough trouble getting out of bed and getting > dressed each day. They're not going to figure out how to shred the hospital > mattress and make a noose out of it.

> Where you've got your problems is when somebody begins to get better from a > severe depression. When they're starting to come out, that's where the > psychomotor retardation relieves enough that suddenly they've got the energy > to do something catastrophic. That's when people are on suicide watches > (when you have clinicians who are oriented well).

[1]: [https://youtu.be/NOAgplgTxfc?t=514](https://youtu.be/NOAgplgTxfc?t=514)

</comment></comment></comment><comment><USER5> "it seems to be a spontaneous, meaningful event that goes well beyond the occasional "good days" most dementia patients experience. The period of clarity is brief, lasting minutes, hours or possibly a day. It seems to come in the hours, days or weeks before death"

Based on my understanding of "why we sleep" and seeing complicated things fail, I wonder if their brains reach a maximally deteriorated state that somehow cancels out the damage temporarily. It might be helpful to find out whether people in these episodes seem truly "in tact" and able to recall their memories across their whole lives. Or, if it tends to be some minute portion of their memories.

Don't know, but I genuinely wish this disease to perish. How it took my family members was just evil.

<comment><USER6> My first thought was "I wonder if it's caused by a release of DMT?" [1] [2]. Interesting research ahead for sure.

[1] [https://www.frontiersin.org/articles/10.3389/fpsyg.2018.0142...](https://www.frontiersin.org/articles/10.3389/fpsyg.2018.01424/full)

[2] [https://beckleyfoundation.org/2017/07/05/do-our-brains- produ...](https://beckleyfoundation.org/2017/07/05/do-our-brains-produce-dmt- and-if-so-why/)

<comment><USER7> Exactly !!! Someone should notify this team or others in the field. I think DMT controls neurons sensitivity (trigger happiness). In high amounts one gets hallucinations. But for vegetative patients low amount of DMT might restore brain activity to normal level.

<comment><USER7> I shared DMT idea to Lydia (article author).

</comment></comment></comment></comment><comment><USER8> Oh man, the ending to _Coco_ just got even sadder...</comment>


Reverse Engineering Snapchat (Part II): Debofuscating the Undeobfuscatable - <AUTHOR> https://hot3eed.github.io/2020/06/22/snap_p2_deobfuscation.html <comment><USER2> This level of API obfuscation reminds me of forever ago when MSN Messenger figured out AOL's AIM API, so MSN Messenger could send AIM messages, which annoyed AOL. AOL would make API changes to break MSN, but MSN would update the client and stay ahead. Eventually to make the API uncloneable, AOL changed their payload to exploit a buffer overrun in their own AIM clients that wouldn't be in the MSN clients.

[https://nplusonemag.com/issue-19/essays/chat- wars/](https://nplusonemag.com/issue-19/essays/chat-wars/)

<comment><USER3> I think the most important, and this article left it out, is why exactly this makes the API uncloneable - why couldn't MSN just emulate the buffer overflow behavior like it was doing with everything so far?

As the article says, the client also responded with some code. What I think was happening: the client was responding with portions of its own executable memory, which could be checked by AOL servers.

That way for MSN to emulate that behavior, it would need to have the AIM client's executable code inside itself, which would be an easy win in a copyright lawsuit.

<comment><USER4> Why not just send copy written code as part of the payload?

<comment><USER5> Especially trademark violations are very effective for this. For example the original GameBoy used it as DRM. The cartridge had to contain a Nintendo(R) logo which was displayed on boot to work, a legal deterrent for publishing unlicensed games that still works to this day.

<comment><USER6> Except that the use of copyrighted and trademarked data for means of enabling interoperability has been ruled fair use in the Sega v. Accolade[1] case. So I believe Nintendo's use of the logo in this way is not much more than snake oil.

[1] [https://en.wikipedia.org/wiki/Sega_v._Accolade](https://en.wikipedia.org/wiki/Sega_v._Accolade)

<comment><USER7> Gentle reminder that the USA isn't the only jurisdiction. In countries without fair use, for example, this wouldn't even be able to be applied.

</comment></comment></comment></comment></comment></comment><comment><USER8> Hey OP, since you're here:

I find this pretty hard to follow. Would you be open to writing a longform version of this aimed at the tutorial level?

Reading between the lines, I would guess you're trying to demonstrate that you really know what you're doing. Maybe as a proof of concept for possible employment opportunities. If so, that's great! Good luck.

But if I were interested in reverse engineering some other app, I don't think I could understand what you've done well enough to use these techniques on that app. Except maybe the breakpointing within `fuck_debug`, that was pretty slick and easy to follow.

<comment><USER9> Obviously not the OP but I think that a longform version of this would be an entire book/college level course. I wish I could learn how to reverse state of the art obfuscation in a single, long post but that's just not how it works.

<comment><USER10> I would pay for that book.

</comment></comment></comment><comment><USER11> Both iOS and recent Androids have by now a form of app attestation: the server can tell if the caller is the legitimate app or not (with good enough confidence - as everything, it's not unbreakable).

Doesn't that make obfuscation kind of pointless? Even if your knock-off app knows everything about the API of the original service, it won't be able to use it because it is not the genuine app or maybe it is but it is not running in a real iOS/Android device.

Or maybe this is only meant to include non-Android certified phones (= China)?

<comment><USER12> seems like something having a rooted os would fix pretty quickly

<comment><USER13> Seems like the creator of Magisk Manager could not get around Android's implementation: [https://twitter.com/topjohnwu/status/1245956080779198464?s=1...](https://twitter.com/topjohnwu/status/1245956080779198464?s=19)

</comment></comment></comment><comment><USER14> As someone who wrote similar obfuscators (manually) back in 2003-2006 to protect a few indie games distributed on PocketPC (ARM7/WinCE) I found it quite conforting to see that the techniques are still similar.

I wonder about something, how long did it take?

</comment><comment><USER15> For fuckup_debugging, can't you use hardware breakpoints instead?

Also, why not patch the binary? I think iteratively patching out protections (in a repeatable, versioned way) would be my approach. It is then applicable to other binaries as well.

<comment><USER16> Hardware breakpoints are a little complicated on iOS. And patching the binary would of course only work if no other code verified the validity of the page you touched.

<comment><AUTHOR> Are hardware breakpoints even possible on iOS? And correct, you can't patch the binary because there many anti-tampering measures, you could probably bypass those, but that's going a different route.

</comment></comment></comment><comment><USER17> The related previous thread: [https://news.ycombinator.com/item?id=23557998](https://news.ycombinator.com/item?id=23557998)

</comment><comment><USER18> For MBA, there's also Arybo[1] from Quarkslab. Never used it and seeing the reference to SSPAM, I assume the author is aware of the tool.

[1] [https://github.com/quarkslab/arybo](https://github.com/quarkslab/arybo)

<comment><AUTHOR> I came across Arybo while working on the binary but I can't remember why I didn't use it, this is vague memory now. Anyway it does the job in one go, I added an edit.

</comment></comment><comment><USER19> Shouldn't you be able to find any code that scans for breakpoints easily and patch it to be blind?

<comment><USER20> Normally it is more like calculating hash from code piece, then xor result with constant and jump. (In general cases, never reversed snap)

So usually there is nothing to patch.

<comment><USER21> Can you calculate that hash (of the original binary) yourself and patch THAT in the function?

</comment></comment></comment><comment><USER16> I'm surprised that Snapchat doesn't check for the mere presence of a debugger and instead tries to look for breakpoints. Or perhaps you've already found and patched those checks out?

<comment><AUTHOR> It does check for a debugger. But that would be through sysctl, or the csops sys call, which would be trivial to patch and a single point of failure.

</comment></comment><comment><USER22> Anyone else picture Deebo from "Friday" (Zeus from "No Holds Barred") smashing apart source code after reading the title?

Prediction: Just me.

By the way, love both articles. Thanks for taking the time to share.

</comment><comment><USER23> I wonder if the Android version uses the same technique and if not, if it would be harder/easier to break

</comment><comment><USER24> The title is misspelled (s/Debofusc/Deobfusc/).</comment>


Is Microsoft's Edge browser really faster than Google's Chrome? - <AUTHOR> http://cd-rw.org/t/is-microsofts-edge-browser-really-faster-than-googles-chrome/35

<comment><USER2> > For the Edge browser Microsoft has again decided to develop their own > rendering engine, instead of relying on open source alternatives (Blink, > Gecko, WebKit). I have to doubt their decision as a browser engine is a big > development effort and they need to be updating it from now to eternity.

Um, why would they move when they already have an engine? Sure you can argue they should have moved to WebKit or Gecko or even Opera's rendering engine years ago but Trident has been improved significantly over the years. Don't forget Edge is based on Trident (so it's not entirely new or anything as implied in the post; it provided a good base to rip legacy support out of and to improve other things).

Beyond that though it seemed fine just kinda felt like the author had a bias against Microsoft from that first statement where he / she questioned their rendering engine decisions. I'm also not entirely convinced they tuned explicitly for the benchmarks they called out; sure they may have but I don't know that I believe they necessarily would have had to.

<comment><USER3> I sincerely hope they NEVER use another engine. Diversity is our friend, and having multiple engines all on similar terms means that bugs won't end up as "correct" and it helps push everyone forward.

<comment><USER4> If they used an open source engine, I would hope it would be that they would open source their own engine. That would be nice to see. Considering Microsoft open sourcing other projects, and this particular product they give out somewhat freely already. Would also allow a community to build up around it, and report as well as possibly contribute bug fixes.

</comment></comment></comment><comment><USER5> After sadly verifying Microsoft's claims, author goes on to find third party benchmarks where they'd lose to make the point he initially wanted to make.

<comment><USER6> "Well these tests don't meet my expectations so lets keep testing using other benchmarks until I'm validated".

<comment><USER7> Authors explanation:

'It seems like Microsoft has been targeting their optimization effort for the competitor benchmarks in order to show impressive results for their new product. When it comes to more intensive and complex HTML5 benchmarks they are still miles behind the competition.'

Reasonable explanation to me.

<comment><USER8> I think instead they worked in JavaScript performance, so in js benchmarks like sunspider and octane they do well. Meanwhile their page loading was not as optimized, so benchmarks that test loading whole pages such as peacekeeper don't perform as well.

<comment><USER7> Peacekeeper itself says they focus on js.

[http://peacekeeper.futuremark.com/faq.action](http://peacekeeper.futuremark.com/faq.action):

Peacekeeper measures your browser's performance by testing its JavaScript functionality. JavaScript is a widely used programming language used in the creation of modern websites to provide features such as animation, navigation, forms and other common requirements. By measuring a browser's ability to handle commonly used JavaScript functions Peacekeeper can evaluate its performance.

</comment></comment></comment></comment></comment><comment><USER3> It all depends on your workload.

I'm working on a web app that uses typed arrays heavily.

Chrome runs my test case in 2.7 seconds, Firefox in 3.2, and IE Edge in 23 seconds... (ie 11 in 30+)

Now to be fair i developed "for" firefox, checking that it worked in IE and chrome the whole way, but that is a pretty large performance difference.

Plus i've always hated benchmarks. They are so easy to "game" without real improvements in actual code. They are great for development and regression testing, but when used to compare engines i've found they are nearly useless.

All that being said, I applaud the IE team for really sticking to their promise of improving the browser and it seems they are going to give Chrome and FF a run for their money soon.

<comment><USER9> > Chrome runs my test case in 2.7 seconds, Firefox in 3.2, and IE Edge in 23 > seconds... (ie 11 in 30+)

I think you are doing something wrong. I have a light app and very heavy SPA (written really badly with almost 5mb of code, D3 + Highcharts). Still, both app loads in less than 4 seconds in IE 11, FF, Chrome and Safari.

<comment><USER3> It's not page load, its the actual application running.

It does some fairly heavy image manipulation in web workers using typed arrays. The app loads a little faster in IE (the code comes in just under 1mb), but the runtime of my few test cases are magnitudes slower.

</comment></comment></comment><comment><USER10> I read it as - the browser is fast at its core but still needs to optimize for the advanced HTML5 features. I guess it will get better over time. This is a good start. No matter what people think, this is a great step forward. Web as a platform is only as open as the variety of rendering engines providing standards compliant features are. If all the browsers are standardizing on the same rendering engine, it's not the web platform that is standard but that particular rendering engine. Even if it is open source, having multiple competing implementations supporting the same standard is a good thing. Especially when companies would like to push for features that benefit them at the expense of overall community.

</comment><comment><USER11> Issue #1: performance claims based on tests from one device configuration. The best performance data is gathered from as many sources as possible. In the spectrum of statistics, one data point is not enough.

Issue #2: "Personally I have found the Peacekeeper results to be a reliable measurement of web browsers performance." Is there data to back up this claim?

<comment><USER12> I doubt Peacekeeper is representative of real Web pages. It primarily benchmarks things that are easy to measure. Peacekeeper also has several bugs:

* It accidentally benchmarks setTimeout clamping. [https://bugzilla.mozilla.org/show_bug.cgi?id=610077](https://bugzilla.mozilla.org/show_bug.cgi?id=610077) is a dependency of peacekeeper. [https://bugzilla.mozilla.org/show_bug.cgi?id=608648](https://bugzilla.mozilla.org/show_bug.cgi?id=608648) is an example in which setTimeout clamping affects the score a lot.

* Its benchmark of array.splice() is extremely strange: [https://bugzilla.mozilla.org/show_bug.cgi?id=592786](https://bugzilla.mozilla.org/show_bug.cgi?id=592786)

* Its layout benchmarks do not really stress layout and instead either stress basic painting operations or DOM accessors. Most pages do not sit there calling style.top in a loop over and over.

* It sets MozTransform only in Firefox without setting values in Chrome: [https://bugzilla.mozilla.org/show_bug.cgi?id=920659](https://bugzilla.mozilla.org/show_bug.cgi?id=920659)

And so on.

There is no reason to think that Peacekeeper's JS benchmarks are particularly better than V8's; in fact, they're probably worse, due to the proliferation of microbenchmarks. You'd get about the same effect by going to jsperf.com and clicking around.

</comment></comment><comment><USER13> I believe that Peacekeeper/Browsermark give a lot of weight to WebGL and related computations, which aren't currently optimised in Edge.

Octane and SunSpider give more weight to the rest.

It'd be very impressive if Edge was faster all round.

</comment><comment><USER14> _" Also note out previous tests with Linux Mint vs Windows 10 which suggest that Chrome actually runs faster on Linux than it does on Windows."_

Honestly, _any_ test between a microsoft product and a competitor run on microsoft's operating system has to be viewed with a grain of salt. What's interesting here is the degree of trust given to a vendor that has tried to rig/break even hardware to lock out competition: [http://antitrust.slated.org/www.iowaconsumercase.org/011607/...](http://antitrust.slated.org/www.iowaconsumercase.org/011607/3000/PX03020.pdf)

And yes, bg still has a lot of swing at microsoft: [http://www.businessinsider.com/bill-gates-is-back-at- microso...](http://www.businessinsider.com/bill-gates-is-back-at-microsoft- big-time-2014-10)

Worse below, there are people simultaneously believing in a vendor that has long track record establishing their "rig the game" overarching corporate strategy and dissing the blog author for doubting the vendor's claims, even to the extent of accusing the author, who checked only two HTML5 benchmarks, of going on a witch hunt to find benchmarks that disadvantaged the vendor.

<comment><USER15> From the text:

> "If seems unfortunate if we do this work and get our partners to do the work > and the result is that Linux works great without having to do the work."

Maybe I'm being too naive, but this sounds more like he is concerned about giving away the work rather than locking people in.

I would prefer if these kind of initiative are open source but in my opinion each company have the right to release as they please the technology they develop. In my mind, this is the same case that Apple with the Thunderbolt, and I think is fair that if Apple decide to keep its technology as private they should be able to do so.

</comment></comment><comment><USER16> The SunSpider benchmark is bullshit, and you shouldn't take anything from it. In particular, Chrome stopped optimizing for SunSpider with their last JIT, as they optimized for longer running apps (such as ones you'd find on the web) but took a hit on SunSpider.

But SunSpider is a really bad benchmark anyway. All benchmarks are bad and suffer real problems, but SunSpider more than most.

</comment><comment><USER17> > _I have to doubt their decision as a browser engine is a big development > effort and they need to be updating it from now to eternity._

I have to doubt his judgement. Microsoft is a huge software companie, the biggest in the world. Much more than Apple or Google.

Not only they can do that, but they also need to be in control of their stack.

</comment><comment><USER18> Not sure what the point of this is. Microsoft never argued that Edge is faster than Chrome at everything. They specifically said it was faster for certain benchmarks.

But the author is upset because it isn't faster for the specific benchmarks he prefers. If it's slower at Peacekeeper does that specifically mean that the browser is now slow ?

And then weirdly Mint/Linux comes into the argument as though it has any relevance to Edge. Nobody is switching operating systems because of browser microbenchmarks.

<comment><USER19> Why so negative?

The post 1st validates Microsoft claims whether they hold. His tests show that Edge is faster on SunSpider/Octane, but not by the margin that Microsoft suggested. The article also points out that old HW is used as the test bench and that HW will make a difference.

Then the article expands the scope to other benchmarks, and there is a (subjective opinion) that Peacekeepers results hold in real world.

I can't see where the author is upset. Rather I see a typical geek hobbyist experiment and the results posted to a FORUM community - not to a professional or scientific publication.

</comment></comment><comment><USER20> Absent from both the original blog post and the OP is Mozillas kraken test, is firefox simply dismissed as slow these days, and their benchmark just discarded to the annals of time?

From arewefastyet.com they seem to be doing ok, (even if they keep testing some really slow chrome version for some reason) and so it seems strange they are being left out.

<comment><USER21> Yeah it seems rather ridiculous to me that Firefox isn't mentioned here -- Firefox consistently beats Chrome on the three major benchmarks (SunSpider, Octane, and Kraken).

</comment></comment><comment><USER22> I really hope browser races doesn't get boiled down to performance, but also things like resource-hogging, which has made it practically impossible for me to use Chrome on my MacBook.

If Edge can be to Windows 10 what Safari is to OS X, I'll try to do what I can to move my browser workflow to Edge as much as possible.

</comment><comment><USER23> In use on common webpages edge feels very snappy. At least as much as firefox and chrome. Good enough for me.

</comment><comment><USER24> i'd rather they implement more stuff

[http://caniuse.com/#compare=ie+10,ie+11,ie+Edge,firefox+39,c...](http://caniuse.com/#compare=ie+10,ie+11,ie+Edge,firefox+39,chrome+43,safari+8)

</comment><comment><USER25> Security, privacy and usability are much more important to me than raw performance.

<comment><USER26> For those three and web standards support, things look even worse for Edge.

<comment><USER11> Based on what evidence?

IE11 has pretty solid web standards support:

[https://status.modern.ie/?iestatuses=implemented&browserstat...](https://status.modern.ie/?iestatuses=implemented&browserstatuses=&browsers=chrome,firefox,opera,safari&ieversion=11)

And Edge promises a lot more.

<comment><USER26> It's significantly worse than Safari, and that's saying a lot. [http://html5test.com/results/desktop.html](http://html5test.com/results/desktop.html)

</comment></comment></comment></comment><comment><USER27> First thing I thought was the law of headlines

<comment><USER28> Except in this case the answer is somewhere between "yes" and "sometimes"

</comment></comment><comment><USER29> Betteridge's law of headlines hehe

</comment><comment><USER30> TL;DR

> What to make of these results? It seems like Microsoft has been targeting > their optimization effort for the competitor benchmarks in order to show > impressive results for their new product. When it comes to more intensive > and complex HTML5 benchmarks they are still miles behind the competition.

<comment><USER6> As I posted on another comment, the TL;DR is more like this:

"Well these tests don't meet my expectations so lets keep testing using other benchmarks until I'm validated".

<comment><USER30> His conclusion stands, he's got 1/2 of the claimed perf gains from the using the benchmarks the Edge Team advertised, but when tested against other Independent benchmarks Edge's performance falls dramatically short, making it reasonable to deduce Edge is optimized around the benchmarks the PR team is advertising.

<comment><USER31> It's unclear how these additional tests were selected. If the criterium was to find a benchmark where Chrome performs better, then you can't conclude much from this.

<comment><USER32> The IE team has said that they prioritize features based on how they are used on the most popular websites. I'd like to see a benchmark that was just the wall-clock time to load the Alexa Top 1,000.</comment></comment></comment></comment></comment>


Enable Two-Factor Auth on your VPN. - <AUTHOR> http://blog.authy.com/openvpn <comment><USER2> I've been doing this for a while using the Google Authenticator PAM plugin[1]. I use the same module to add two factor authentication to both SSH and OpenVPN connections.

[1] [http://code.google.com/p/google- authenticator/](http://code.google.com/p/google-authenticator/)

<comment><USER3> Can you share some "how to" pointers - may be a blog post describing how to do this?

<comment><USER2> Real quick and dirty. But you should be able to get it to work from these notes. Be warned, I don't go into how to set-up OpenVPN. You can find documentation for OpenVPN at [http://openvpn.net/index.php/open- source/documentation/howto...](http://openvpn.net/index.php/open- source/documentation/howto.html)

You'll need to install Google Authenticator on your phone to complete the set- up.

Here are my notes: [https://invaliddomain.com/openvpn-google-auth- howto.txt](https://invaliddomain.com/openvpn-google-auth-howto.txt) (please excuse the lack of a real SSL cert).

Once enabled, your password becomes <password><OTP> .. e.g. Pa55w0rd!987654

Hope that helps.

</comment></comment></comment><comment><USER4> Better yet, use passwords plus client side certificates. It prevents these scenarios without requiring you to trust a third party with your two-factor authentication tokens.

</comment><comment><USER5> Or, if you want to do the same thing without trusting the security of your VPN to a third party, and you're willing to do a little more fiddling with configuration files, you can just use libpam-google-authenticator: [http://www.howtoforge.com/securing-openvpn-with-a-one- time-p...](http://www.howtoforge.com/securing-openvpn-with-a-one-time- password-otp-on-ubuntu)

<comment><USER6> You are only trusting the half of the security which is better IMHO. The evil person would need to hack/steal both you and the (likely more careful) third party in order to access your accounts. In your scenario it would only need to hack you to get everything.</comment></comment>


The Perl 6 project 10 years old today - <AUTHOR> http://use.perl.org/~masak/journal/40451

<comment><USER2> There's nothing wrong with taking 10 years to create something new. Since they began, I've taken serious ganders into java and ruby. Now that I'm back in the perl camp I find that I know a lot more about programming in other cultures. And now that the community is just about done, I can apply what I've learned using a "language" unprecedented in flexibility and capability.

<comment><USER3> > I can apply what I've learned using a "language" unprecedented in > flexibility and capability.

This right here is what I've always hated about the perl community. It's so rare to find a perl user who can mention the language without spouting these kind of exaggerations and untruths.

Of course you could always prove me wrong by demonstrated how perl is "flexible" and "capable" in a manner that is _unprecedented_ [1].

[1] un·prec·e·dent·ed

        adjective

        without previous instance; never before known or
        experienced; unexampled or unparalleled: an
        unprecedented event.

<comment><USER4> Do you know of another language community with a library or extension system which rivals the CPAN, for one example? How about the existence of PCRE for another? Along those lines, have you seen Perl 6 grammars?

<comment><USER3> The post I responded to said "language" so your comment about CPAN doesn't apply.

On the second part, I did misread the OP as meaning he would return to _perl 5_ (probably a false mental cache hit, since this is usually the context of such comments). Perl 6 does indeed have a few interesting bits.

<comment><USER4> _The post I responded to said "language" so your comment about CPAN doesn't apply._

Somehow Perl 5 made CPAN possible (and by deliberate language design), so that guy in a kilt seems truly a Scotsman to me.

<comment><USER3> Confusing response. If you want to get down to it, CPAN doesn't really provide much more functionality than what you can get in most other languages and provides less than what Java has. CPAN has a tremendous amount of duplication, has a fair amount of toy code, etc., etc.

</comment></comment></comment></comment></comment></comment><comment><USER5> "I count 33 synoptic documents at perlcabal.org. Synopses 2 through 6 tend to be fairly stable, although changes still occur. The remainder of the synopses are still drafts for the most part, awaiting more feedback from implementations and language use."

This is why when people like chromatic invariably comment in a "still waiting for perl 6 thread" that "you can write perl 6 now!!11!" I roll my eyes. (Quietly. So they'll go away.)

<comment><USER4> _This is why when people like chromatic invariably comment in a "still waiting for perl 6 thread" that "you can write perl 6 now!!11!" I roll my eyes._

Almost five years ago now I ported a piece of Perl 5 code used _millions_ of times a day to Perl 6. Where is the hyperbole in my statements?

<comment><USER6> Well, you can't just leave us hanging like that. What's the program?

</comment></comment></comment><comment><USER7> 10 years is also about how long ago I abandoned Perl

<comment><USER8> 10 years too early, I'd say. You're missing out on the fun stuff.

</comment></comment><comment><USER9> One promising thing about Perl 6 was the way it tried to bring parsers to everyday programming the way Perl popularized regular expressions. I'm sad the language seems to have stalled for that reason.

<comment><USER10> You're almost exactly right there. One promising thing about Perl 6 _is_ the way it brings parsers to everyday programming the way previous Perls popularized regexes. :)

Perl 6's approach to syntax, operators, grammars, language extension, etc. is like Common Lisp's reader macros; except that Perl 6 makes them easy _and_ composable. Common Lisp libraries tend to use reader macros very sparingly because it's hard to make them play together nicely. Perl 6 makes it easy to alter the syntax in ways that play together nicely. Many of Rakudo's core operators are defined in normal Perl 6 code. It's trivial to make new ones of your own. The more extensive grammar modifications aren't quite there yet, I think, but it'll happen, and it will be revolutionary.

Fortunately, it hasn't stalled. It's moving along quite well.

<comment><USER9> "Everyday programming" doesn't necessarily use Perl, and it rarely uses prerelease programming languages.

Perl 6 has brought parsers (I guess we should call them "grammars" in analogy to "regular expressions") to certain parts of the (shrinking) Perl community, but the everyday programming in general hasn't been lit aflame. Perl was influential enough to get regexes (in one form or another) into nearly every language in popular use today.

<comment><USER11> > everyday programming in general hasn't been lit aflame

That isn't quite true. A lot of the things that are happening in P6 have been backported to P5 and in general has made everyone excited. It is has also "revitalized" the people working on the core of P5 to think new thoughts as well.

I don't know if I will move to P6 but I am thankful for what it has done to P5.

</comment></comment></comment></comment><comment><USER12> The first production-ready release will be on GNU Hurd, right? ;-)

</comment><comment><USER13> I'd like to mention Dan Sugalski who, before Audrey, was a similar whirlwind for the Parrot project. He too was sadly burnt out long before his time.

I highly recommend perusing his well written blogs that wonderfully explain difficult programming topics.

<http://www.sidhe.org/~dan/blog/>

<comment><USER13> His "What the heck is" and "What the heck was I thinking?" are particularly good.

</comment></comment><comment><USER14> After reading that, it seems like Perl has rather nice community. If Python has engineers and Ruby rockstars, then I'd say that Perl has hackers.

<comment><USER15> > Python has engineers and Ruby rockstars

This meaningless statement really bugs me. People who write Python _and_ Ruby aren't rockstar engineers. They're just programmers/developers/[whatever you like to call them].

People who speak English are simply English speakers. Nothing more can, nor should, be inferred from that fact.

<comment><USER16> I think that people blow it out of proportion, but the primary language choice (and I do mean the language they use when they can chose) of a developer _does_ imply something, however small.

Assembly programming is different enough to Ruby programming that it does imply something about a person that prefers working in either the former or latter environment.

However, the reason I agree with you and upvoted you is that Ruby, Python and Perl are so similar in mindset that anything you might learn from those choices is basically lost in the noise. It's like trying to figure out what a person's choice of colour of parachute means... which is silly, but the fact that he or she skydives _does_ mean something.

<comment><USER15> I like Ruby and assembly (and C, Python, Lisp, JavaScript, ...). So I agree that one's primary language may say something about them, but that something might only be that they happened to land a good job doing X.

</comment></comment></comment></comment><comment><USER17> It's a spring chicken compared to Enlightenment.

</comment><comment><USER18> Didn't someone say it take 10 years to develop great software?

<comment><USER19> You're probably thinking of Joel Spolsky's "It Takes Ten Years To Develop Great Software. Get Used To It".

<http://www.joelonsoftware.com/articles/fog0000000017.html>

However, Joel explicitly states this means you release _early_ , and iterate.

<comment><USER4> Pugs first came out in 2005, and by my count it was the _third_ implementation of Perl 6. Rakudo's a couple of days away from its 31st release. Yes, Perl 6 should have had regular releases much earlier than 2005, but we've used iterative development for _years_.

</comment></comment></comment><comment><USER20> Perl 6 is the Duke Nukem of the programming world.</comment>


Ask HN: What is the next Moore's Law? - Gabriel-Lewis Moore's Law (https://en.m.wikipedia.org/wiki/Moore%27s_law) has led the technological revolution for many decades, but now it looks like it is coming to an end. What are similar phenomenons (inside and outside of processing speed) that either will, or already are advancing technology at the same rate? <comment><AUTHOR> Network bandwidth doubles every X months. (6, maybe? It's really fast.)</comment>


What's replacing P2P, BitTorrent as pirate hangouts? - <AUTHOR> http://www.computerworld.com/s/article/9139210/What_s_replacing_P2P_BitTorrent_as_pirate_hangouts_ <comment><USER2> I really have to disagree with this. Piracy will not be moving to rapidshare - it's just not anonymous and decentralized enough. More likely bittorrent protocols will keep improving with more privacy features tacked on.

<http://en.wikipedia.org/wiki/Phantom_Anonymity_Protocol> <http://www.stealthnet.de/en_index.php></comment>


How are you Celebrating World Backup Day? - <AUTHOR> http://web.synametrics.com/app?operation=blog&st=dispblog&fn=00000000093_1554126110171&BloggerName=catherine <comment><USER2> Honestly could not be more excited about this, my kids have been looking forward to it as well!

I bought a cake, decorated the house with streamers and balloons... it's gonna be a blast!</comment>


Owner of new MacBook Air: 4 GB of RAM is not enough - <AUTHOR> http://forums.macrumors.com/showthread.php?t=1600495

<comment><USER2> I'm going to agree with the commenters and say that a memory-leaking program is likely the culprit.</comment>


The Humble Indie Bundle #4 (pay what you want and help charity) - <AUTHOR> http://humblebundle.com/ <comment><USER2> For anyone curious why this is here, Humble Inc is funded by YC -- beyond it being a collection of cool games people might care for.

edit: I was wrong, not just YC. They have $4.7m from Sequoia[1] too.

[1] [http://finance.fortune.cnn.com/2011/04/22/sequoia-capital- ba...](http://finance.fortune.cnn.com/2011/04/22/sequoia-capital-backs-online- gaming-bundler/)

<comment><USER3> That's very interesting, as I've always thought it was bootstrapped by Wolfire games. Serious question - why does this need funding?

<comment><USER2> The first bundle (and possibly second) was bootstrapped, they took funding after the idea was shown to work. That's all I know, not sure why they have funding.

<comment><USER3> Thanks for the extra info. Seems to me like this is the perfect company that can eschew outside investment; I assume they curate these bundles with no upfront payment, and then everyone gets a cut of digital sales. Not sure where this $4m could be going unless they have big plans beyond the current the current model.

<comment><USER4> Braid can be bought directly with Paypal, Amazon Payments, and Google Checkout as payment options. Corner of the box says "Humble Store". What they do with that is a unknown, but I find it interesting.

<http://braid-game.com/>

<comment><USER5> That reminds me: I noticed that humblebundle.com is apparently involved in the purchase transactions for the recently released 'English Country Tune'. So maybe there's more afoot than just the bundles themselves:

<http://www.englishcountrytune.com/>

</comment></comment></comment></comment></comment></comment><comment><USER6> I just bought this one but wasn't there another bundle just last week? I wonder if too many too soon will lead to bundle fatigue

<comment><USER7> I agree. I used to get really excited about the humble bundle. They had a few noteworthy games per bundle (All the games they have are good, but some are less known than others).

Now that it's a monthly affair I get the initial perception that the bundles have less noteworthy games and that I'm getting tired of so many of them. After a bit of thought I realize I /am/ paying just a few bucks for a lot of games and helping charity and I end up buying them, even if I already own al lthe games. but I fear there's a lot of people who won't make that leap and just say "meh, been there done that". It's sad because the bundle is great for devs and great for gamers.

<comment><USER8> Cave Story+ is an updated-engine version of what is probably _the most famous indie game of all time_ that basically started the modern indie games movement. It would not be unreasonable to say it's one of the best action/platformer games ever made -- a completely indie Metroidvania game that's better than most similar games from major publishers despite being made entirely by one person in his spare time.

Jamestown is a completely awesome arcade-style danmaku shooter inspired by CAVE games and Touhou -- but unlike some recent poor attempts to emulate these styles (e.g. Deathsmiles), it adds loads of creativity of its own, like its so-absurd-it's-hilarious storyline and setting. It was worth every cent on launch, and now you can get it for way less _with a half dozen other games included_.

This might be the best Humble Bundle yet just because of these two.

<comment><USER9> uncontrollably clicked purchase when I saw cave story+. I'll buy anything related to cave story. The creator is really missing out with the lack of merchandise.

<comment><USER10> Plush Balrog? :)

</comment></comment></comment></comment></comment><comment><USER11> Why did they drop EFF from the charities? :(

<comment><USER12> Supposedly the HIB people voted, and the Red Cross came out slightly ahead. I'm kinda bummed. I might go donate a couple dollars to the EFF anyway.

<comment><USER13> They also said that the EFF would be returning as a charity in future bundles.

</comment></comment></comment><comment><USER14> Anyone else experiencing Humble fatigue? When this was a special once a year deal I supported it regardless of the contents of the bundle. Now I feel that there's a Humble Bundle once a week and I've stopped reading their email.

<comment><USER15> I wasn't planning on buying this bundle either, but not because of the regularity of the bundles, more because of the quality of the content. Some of the previous bundles have contained a little much filler/incomplete games/etc.

They've really stepped up the contents with this pack though, I know myself and a few friends feeling much the same bought it immediately. There's an incredible amount of high quality gaming in this pack.

<comment><USER14> Thanks for taking the time to reply. I investigated several of the games and decided to go ahead and purchase it. I agree that the quality has varied over the past few bundles. The novelty of the Humble Bundle has definitely worn off for me (plus I preferred the original set of charities).

</comment></comment></comment><comment><USER16> A good friend of mine wrote Jamestown- kick ass game, and way to go!

<comment><USER17> Wow, send him much thanks. I bought it on Steam when it released (because I'm a sucker for shooters), and it's a marvelous game. One of the finest shooters I've played in a long time. The DLC is awesome, too (Guy Fawkes ship!)

</comment></comment><comment><USER18> I paid the suggested price for this. I'm only running Linux, so it's well worth supporting it.

It's a shame about the EFF though. They're at the top of my list these days.

</comment><comment><USER19> I think this might be the best one yet

<comment><USER20> Agreed, Super Meat Boy alone is worth whatever you may pay, in my opinion :)

</comment></comment><comment><USER21> I always find it interesting that the Linux people pay the most. Then again, the games are probably most impressive compared to games generally available on Linux, and least impressive compared to those available for windows. Not to say anything about the quality of three games, just commentary on the different gaming markets.

<comment><USER22> I've always assume it is more about supporting developers who support linux, and a competition with the other platforms.

<comment><USER23> That's why I usually pay out a minimum of $20 for the bundle with 100% towards developers.

I may sound like an ass saying this but those are popular charities and receive a lot of funding from lots of resources, they can survive without my $5 or $10 while many of these developer groups have only released a couple games. I want to see DRM free indie game development take off so I will contribute to the developers who I feel need it more.

<comment><USER24> You're paying $20 when you could be paying a penny. I would hope that nobody is putting you in the "sounds like an ass" category.

</comment></comment></comment></comment><comment><USER25> Are the Humble Bundles always mostly about games? It would be great to see some non-game bundles.

</comment><comment><USER26> And it's a pretty good spread, too, maybe the best yet - the last few bundles & mini-events have been a bit lackluster, IMO. But that's just IMO.

If you've never played it, Cave Story is one of the best platformers _ever_ , and you _must_ play it if you are a gamer in any sense. The original is available for free (though you might have to English-patch it), but I gather CS+ is a pretty good port.

<comment><USER27> I don't know; between the last three mini-bundles we've had Frozen Synapse, SpaceChem, Dungeons of Dredmor, The Binding of Isaac, Darwinia, Defcon, Voxatron, and Blocks that Matter.

Four of those are strong candidates for indie GOTY (and the others are still pretty good) so it's not too lackluster yet.

</comment></comment><comment><USER28> Does anyone know how to turn off the Paypal "Bill Me Later" popup that appears every time I make a purchase?

I have money in my account. I'm tired of opting out of BML.

</comment><comment><USER29> These games always make me want to make some of my own!

</comment><comment><USER30> Is anyone else not tipping the Humble team anymore? Are you just splitting payment between devs and charities?

<comment><USER31> At least I do the opposite. The Humble team are clearly making my life better by e.g. motivating Linux ports of these games. Meanwhile there are much better ways to donate to charity (and "better" charities).

</comment></comment><comment><USER32> Can anyone post a list for those of us behind a firewall? Thanks!

<comment><USER33> Gratuitous Space Battles (if pay more than average)

Gave Story+ (if pay more than average)

Jamestown

Bit.Trip Runner

Super Meat Boy

Shank

NightSky HD

<comment><USER20> Avg currently 4.70 (was 4.64 about 1 minute ago! climbing fast, damn HN!)

<comment><USER34> If the number of purchases so far were small, it's possible that buying several copies at $.01 would bring the average down by more than the cost in penny purchases (this is a strategy for a purely hypothetical selfish person with time to spend, but not $5). I expect the amount to fluctuate above and below $5.

</comment></comment></comment></comment><comment><USER35> anyone else bummed that this one didn't include desura keys?

<comment><USER36> They might appear on desura - just not day one - it was like this with introversion bundle.</comment></comment>


Alpha release of a PDF, Doc, DocX to Text Service - <AUTHOR> http://txtyo.com

<comment><AUTHOR> It's built with: Node S3 Redis Twitter Bootstrap</comment>


A simple tool to find an open port or domain socket on the current machine - <AUTHOR> https://github.com/indexzero/node-portfinder

<comment><USER2> Wouldn't it be simpler to script `netcat` in shell? Or abuse the output of `lsof`?</comment>


ScalaNLP – A suite of machine learning and numerical computing libraries - <AUTHOR> http://www.scalanlp.org/ <comment><USER2> So, this is (largely) my project.

Not sure why this is on the front page of HN, but I'm happy to answer any questions.

I'm not really giving these libraries the love they need these days. I mostly started them in grad school before the deep learning revolution really hit my subfield (NLP), and I haven't had time to modernize them. They still have their uses, especially Breeze, which is used in Spark's MLLib and directly by a number of companies.

</comment><comment><USER3> Does Scala have a visualization suite like ggplot, seaborn, matplotlib, or something similar?

What benefits does this provide over existing python or matlab code?

> Scientific Computing, Machine Learning, and Natural Language Processing

These seem to be three very different problems. Is there a reason why the group of them is called "ScalaNLP"? If the libraries are generic enough then shouldn't other uses be possible/supported?

Is there a reason this doesn't have a generic name similar to the SciPy stack?

<comment><USER2> Main author here.

Breeze has breeze-viz, which is very basic but at the time there wasn't anything else. I highly endorse using something else. I personally like [http://sameersingh.org/scalaplot/](http://sameersingh.org/scalaplot/)

They're under the same aegis basically because they're all mine. ScalaNLP started out as really being just NLP, but it scope-crept. That said, Epic is a library for _structured prediction_ first and foremost, and one of the main applications of structured prediction is NLP.

Breeze is basically like SciPy and large chunks of it power Epic. It's really the only thing that doesn't belong in the namespace.

<comment><USER3> I'm glad you're bringing something like this to the JVM/Scala-ecosystem.

There are some things that I've been interested in asking for in a high level scientific computing library. If you're planning on continuing your visualization library can you please come up with some solution for layout specification? Whenever I'm plotting something and I spend 30 minutes getting all of the data in order the last thing I want to do is fight with the plotting library's label positions because they overlap. Or if I say "Let me take this plot, add some more stacked subplots, and show different catagories" I don't want my labels to be perfect but my scatters to be given a 10x10 pixel box to draw into.

On the HP/numerical computing side of things have you looked into implicit GPU operation types? Something that would let you queue up operations that can be run on a parallel computing system. Basically describe complex operations with the high-level object's normal operations. The objects aren't actually calculating anything, they just organize a GPU kernel in the background. As the final stage you can turn the

        gpumat a(3, 5);
        gpumat b(5, 3);
        gpumat gpu_op_queue = (a * b) + (a * b) * 5;

        function(a, b) operation = gpu_op_queue.compile();
        mat output = operation(some_3x5, some_5x3);

In the backend you'd hopefully be able to great your own types like 'cpumat', 'computerclustermat', or 'gpuclustermat'.

If you had some easy way to generically express extremely parallel numerical operations, an abstract way of implementing high-performance back-ends that take those operations and compile them to GPU kernels, and a visualization engine that doesn't feel like it's from the 80s then your library will really take off.

Personally I feel GPU-optimization and fighting with visualization libraries are the two biggest pain points in scientific computing.

<comment><USER2> Thanks for the questions.

I am very unlikely to take on visualization. I don't acutely need it for what I do, and I am some-but-not-nearly-enough interested in visualization for its own sake. I started to read about the grammar of graphics stuff at one point and decided it was too far down the rabbit hole.

I have looked more into gpu stuff, and agree specifying a compute graph (and then implicitly optimizing it) is more likely to be the future. FWIW, this is basically what XLA (from TensorFlow) and whatever it was FB announced on Friday are doing.

I wrote my thoughts up recently on the Breeze mailing list here: [https://groups.google.com/forum/#!topic/scala- breeze/_hEFpnI...](https://groups.google.com/forum/#!topic/scala- breeze/_hEFpnI9gog)

I'm starting to think it through but I'm not sure I have time for that either :(. A 4-month old and a startup take up a lot of time.

</comment></comment></comment></comment><comment><USER4> David did a great job with ScalaNLP.

It happens to depend on Breeze. I would point out that Breeze does not support n-dimensional arrays (most tensors), although that is necessary to do in deep learning.

We wrote ND4S and ScalNet to solve that:

[https://github.com/deeplearning4j/nd4s](https://github.com/deeplearning4j/nd4s)

[https://github.com/deeplearning4j/scalnet](https://github.com/deeplearning4j/scalnet)

Moving computation out of Spark's MLlib and into lower level code like C++, as we do with JavaCPP and libnd4j, also improves speed.

[https://github.com/deeplearning4j/libnd4j](https://github.com/deeplearning4j/libnd4j)

[https://github.com/bytedeco/javacpp](https://github.com/bytedeco/javacpp)

<comment><USER2> Thanks for the kind words.

Breeze does a large chunk of (dense) compute via netlib-java, which calls out to "real" lapack if you set it up. Are things really faster than that? Or are you referring to the non BLAS/non Lapack things?

<comment><USER5> Few things about netlib-java.

1: It's a read only repository now. It's retired. Lack of maintenance will hurt its long term prospects.

2\. The license on net lib java's native binaries are not commercial friendly

3\. Net lib java does everything on heap with double arrays, we do everything off heap. There's no copying to worry about, and there's a lot lower latency and flexibility with our data buffers.

4\. Due to javacpp we have better control and interop with other c++ libraries like opencv. This makes it easier to write native code and use it from java later on. This allowed us to write and maintain all of our own c/c++ code with the same api (see: nd4j there) - [https://github.com/deeplearning4j/libnd4j](https://github.com/deeplearning4j/libnd4j)

So yes it ends up being faster in practice for a lot of scenarios. Aside from that, we also have more control over the blas libaries we pick.

This means we also have access to cublas as well as (see below) more configuration and flexibility.

Net lib java tries to be "pure" which, while elegant, isn't practical if you want to benefit from gpus and DL. We implemented the proper shims to make things "just work" from the user's perspective there on top of having more flexibility (see: mkls opemp knobs etc)

Nd4j has its own built in garbage collector and memory management which means we don't have to worry about any strange work arounds when working with cpus/gpus _and_ we can keep off heap buffers in a managed manner.

See:

[http://deeplearning4j.org/workspaces](http://deeplearning4j.org/workspaces)

[http://deeplearning4j.org/native](http://deeplearning4j.org/native)

In general, "just blas" isn't enough. I know from personal experience. I wrote nd4j after trying to use every java library for matrix compute and all of them fell flat in terms of speed, interop with other c++ libraries, and the need to use java arrays was highly limiting. Over the years, we built up nd4j to handle harder scenarios.

This includes other features like distributed parameter servers among other things.

Other things aside: I like what breeze attempted but it ultimately didn't scratch the itch for me when I was looking hard at the various java matrix libraries (I've tried all of them)

When I originally built out nd4j, it has this backend architecture:

[http://nd4j.org/backend.html](http://nd4j.org/backend.html)

It was so we could just use whatever matrix backend we wanted. None of them worked well enough due to the flexibility we needed.

I also had an inherent problem with java based for loops in any setting. We wrote our own forkjoin implementation as well attempting to make it fast and it just couldn't beat plain c.

We've found especially after matrices of size 128 x 128 or so, we hands down beat every JVM out there no matter what language is. The last bit we are working on are smaller matrices.

The other problem we're working on is our sparse support could use some work. The basics are in there but it's not quite ready for prime time yet.

After that, (I'm obviously biased) I don't see how anything could compete with us. Especially after we add our autodiff/pytorch like stack on top of all these primitives.

Hope that helps!</comment></comment></comment>