Launch HN: Batch (YC S20) – Replays for event-driven systems - <AUTHOR> Hello HN!
We are Ustin and Daniel, co-founders of Batch (https://batch.sh) - an event replay platform. You can think of us as version control for data passing through your messaging systems. With Batch, a company is able to go back in time, see what data looked like at a certain point and if it makes sense, replay that piece of data back into the company's systems.
This idea was born out of getting annoyed by what an unwieldy blackbox Kafka is. While many folks use Kafka for streaming, there is an equal number of Kafka users that use it as a traditional messaging system. Historically, these systems have offered very poor visibility into what's going on inside them and offer (at best) a poor replay experience. This problem is prevalent pretty much across every messaging system. Especially if the messages on the bus are serialized, it is almost guaranteed that you will have to write custom, one-off scripts when working with these systems.
This "visibility" pain point is exacerbated tenfold if you are working with event driven architectures and/or event sourcing - you must have a way to search and replay events as you will need to rebuild state in order to bring up new data stores and services. That may sound straightforward, but it's actually really involved. You have to figure out how and where to store your events, how to serialize them, search them, play them back, and how/when/if to prune, delete or archive them.
Rather than spending a ton of money on building such a replay platform in-house, we decided to build a generic one and hopefully save everyone a bunch of time and money. We are 100% believers in "buy" (vs "build") - companies should focus on building their core product and not waste time on sidequests. We've worked on these systems before at our previous gigs and decided to put our combined experience into building Batch.
A friend of mine shared this bit of insight with me (that he heard from Dave Cheney, I think?) - "Is this what you want to spend your innovation tokens on?" (referring to building something in-house) - and the answer is probably... no. So this is how we got here!
In practical terms, we give you a "connector" (in the form of a Docker image) that hooks into your messaging system as a consumer and begins copying all data that it sees on a topic/exchange to Batch. Alternatively, you can pump data into our platform via a generic HTTP or gRPC API. Once the messages reach Batch, we index them and write them to a long-term store (we use https://www.elassandra.io). At that point, you can use either our UI or HTTP API to search and replay a subset of the messages to an HTTP destination or into another messaging system.
Right now, our platform is able to ingest data from Kafka, RabbitMQ and GCP PubSub, and we've got SQS on the roadmap. Really, we're cool with adding support for whatever messaging system you need as long as it solves a problem for you.
One super cool thing is that if you are encoding your events in protobuf, we are able to decode them upon arrival on our platform, so that we can index them and let you search for data within them. In fact, we think this functionality is so cool that we really wanted to share it - surely there are other folks that need to quickly read/write encoded data to various messaging systems. We wrote https://github.com/batchcorp/plumber for that purpose. It's like curl for messaging systems and currently supports Kafka, RabbitMQ and GCP PubSub. It's a port from an internal tool we used when interacting with our own Kafka and RabbitMQ instances.
In closing, we would love for you to check out https://batch.sh and tell us what you think. Our initial thinking is to allow folks to pump their data into us for free with 1-3 days of retention. If you need more retention, that'll require $ (we're leaning towards a usage-based pricing model).
We envision Batch becoming a foundational component of your system architecture, but right now, our #1 goal is to lower the barrier to entry for event sourcing and we think that offering "out-of-the-box" replay functionality is the first step towards making this happen.
.. And if event sourcing is not your cup of tea - then you can get us in your stack to gain visibility and a peace of mind.
OK that's it! Thank you for checking us out!
~Dan & Ustin
P.S. Forgot about our creds:
I (Dan), spent a large chunk of my career working at data centers doing systems integration work. I got exposed to all kinds of esoteric things like how to integrate diesel generators into CMSs and automate VLAN provisioning for customers. I also learned that "move fast and break things" does not apply to data centers haha. After data centers, I went to work for New Relic, followed by InVision, Digital Ocean and most recently, Community (which is where I met Ustin). I work primarily in Go, consider myself a generalist, prefer light beers over IPAs and dabble in metal (music) production.
Ustin is a physicist turned computer scientist and worked towards a PhD on distributed storage over lossy networks. He has spent most of his career working as a founding engineer at startups like Community. He has a lot of experience working in Elixir and Go and working on large, complex systems. <comment><USER2> I've worked on a very similar product in the past and can affirm that there is definitely enterprise interest for a good solution to event replay for orgs that are already doing event sourcing...I'm curious if offering out of the box replay will actually lower the bar and drive more orgs to pursue event sourcing? The CLI search functionality is really cool and useful as well.

<comment><AUTHOR> Hey there!

Re: lowering the bar - we hope so. What we've noticed is that the papers that talk about event sourcing mention replays but don't talk at all about the implementation (or give any pointers). We're hoping that if at least that part is done for you, you've got one less thing to worry about.

As for the CLI tool - thanks! We found it super useful ourselves and figured others would too. I like to think of it as a sort of intelligent `netcat` for messaging systems :D

</comment></comment><comment><USER3> Congrats on the release. We've made a ragtag solution in-house that is complicated but works on those few unfortunate events that we need it. There's a demo on request but it would be helpful if we can have a better way to test the product. Maybe an endpoint where we stream maybe 10000 events and see them replay? What sort of pricing tier are we talking about?

<comment><AUTHOR> Thank you!

Re: ragtag in-house solution that is complicated

^ That's _exactly_ what we're talking about. These systems get complex pretty quick and you end up with duct tape in more than a few places.

As for demo - yeah, our plan is to open up registrations for accounts soon which will allow you to pump data into us for free with a low retention period.

We've still got some pieces to tighten before we can open the gates fully but we'll try to make it happen soon (within next few weeks?). In the meantime, if you want a demo, ping us and we'll make it happen.

</comment></comment><comment><USER4> Wow, this is a great idea. I recently worked on a team building streaming data pipelines, and we built a bespoke system to do exactly this: end-to-end test our software. We had past messages written to a >300TB sharded file, and wrote a microservice to read each shard and publish it to the message queue for our staging instance, and then run data validation/anomaly detection on the output. It was useful but incredibly painful to use and maintain, and Batch would have been a fantastic solution for accomplishing this.

</comment><comment><USER5> Congrats, looks useful! Just an opinion, but I think you should skip the cool large animation on your homepage and just start with the "Our platform is essential in scaling and maintaining your business.". I had no idea what Batch was until I scrolled way below the fold.

<comment><AUTHOR> Yeah... we've heard this before. But the wavy stuff makes me feel so ... _caaaaalm_ :)

<comment><USER5> It is nice and calming, btw your twitter link at the bottom your site is broken (it links to [https://batch.sh/www.twitter.com/batchsh](https://batch.sh/www.twitter.com/batchsh)).

<comment><USER6> Fixed!

</comment></comment></comment></comment><comment><USER7> How does bookmarking work/How do I keep track of how far I've read while replaying from Batch? Will you also index by date? It can take a long time to replay a lot of data; do you have any numbers on the read rates you support per topic?

<comment><AUTHOR> Great questions!

> How does bookmarking work/How do I keep track of how far I've read while > replaying from Batch?

We do not have any bookmarking functionality built (yet) as we currently expect folks to just tweak their search query. Each one of the events has a new id attached to it that you can query and reference during search.

> Will you also index by date?

We do! Every event has a microsecond timestamp attached to it.

> It can take a long time to replay a lot of data; do you have any numbers on > the read rates you support per topic?

We've done some initial replay throughput tests and have been able to reach ~10k/s outbound via HTTP - of course, this is all _highly_ dependent on where you're located. We expect that for folks who need super high throughput, we'll probably need to be closer to them - we fully expect to have to peer with some of our customers and optimize for throughput by doing gRPC and ... batching :)

So far, we've done _most_ of our testing on inbound and we are currently able to sustain ~50k/s (with ~5KB event size). Our inbound is able to scale horizontally and so can go waaaaay beyond 50k/s if needed.

We have _a ton_ of service instrumentation so we've got good visibility around throughput (and thus should know well in advance as to when we're starting to hit limits).

</comment></comment><comment><USER8> Congrats on the launch!

Two questions:

\- If I have some data in Kafka, why would I want to pump it into your platform instead of spawning an Elasticsearch instance and using something like Kafka Connect to write to it and gain visibility?

\- If I use Kafka as a permanent data store (with infinite retention), I can easily replay all events with existing clients (or with plumber). What additional functionality does the "replay" feature offer compared to that?

<comment><AUTHOR> Hey there!

> \- If I have some data in Kafka, why would I want to pump it into your > platform instead of spawning an Elasticsearch instance and using something > like Kafka Connect to write to it and gain visibility?

To avoid having to build, own and maintain the infra you just mentioned. As the number of events on your system increase, you will have to scale ES and other pieces of the system as well.

Our point is just that - if you know what's involved in collecting and indexing the events - that is awesome but maybe you shouldn't have to spend time building the infra around that stuff.

> If I use Kafka as a permanent data store (with infinite retention), I can > easily replay all events with existing clients (or with plumber). What > additional functionality does the "replay" feature offer compared to that?

I think it depends on your definition of "easily replay" \- a kafka replay for a topic that's being consumed by a consumer group would require you to disconnect that consumer group and then run a shell script to move the offsets. You also would not have any way to replay any specific messages - your only point of reference would be an offset (and keyname, if you use it) - not terribly flexible.

With Batch, you get to drill in and replay the _exact_ messages you want (and avoid having to pump and dump potentially millions of messages your consumer doesn't care about).

<comment><USER8> Makes sense, thanks for the clarification!

</comment></comment></comment><comment><USER9> If I'm writing all my messages to durable storage why not work off the durable storage? I'm definitely not an expert in this area so perhaps I'm missing something. My logic is that if you're paying the resource cost to write all your messages why not pay the resource cost to read/write back there?

<comment><AUTHOR> I think you're asking why don't we just be the hosted kafka/rabbitmq/etc and offer all of this stuff in one place. (let me know if that's wrong).

That's a totally legit point - we've talked about offering it all in-house before but it would require us to split our efforts into two - operating a PaaS (for a bunch of different messaging tech) and running the event collection platform.

Operating the PaaS part would be a full-time effort and there's a lot of competition out there. We've decided to focus on the observability/replay part first (since there is a lot less competition) and then later maybe explore the hosted bus option.

LMK if that's not what you meant :)

<comment><USER9> >I think you're asking why don't we just be the hosted kafka/rabbitmq/etc and offer all of this stuff in one place.

The other way around. If I'm not storing my messages today it's probably because it is too expensive in terms of storage or compute to do so. But, presumably, you can't do that any cheaper than I can. And now we are duplicating the work so even more resources are being consumed making it that much more expensive than just doing it myself.

It seems like your service is something I'd want to run pointed towards my Kafka/RabbitMQ/whatever servers. I don't see how duplicating that stream is cost effective.

<comment><AUTHOR> Ahh gotcha. If you need event introspection, doing it in-house is extremely likely to be more expensive (and definitely time consuming) than offloading it.

For example: if you are sending serialized data on your bus - you will need to write something that will deserialize it before inserting it into your elastic search cluster - and now you're managing even more infra (message systems, decoders, document storage).

There is definitely a price attached to the luxury - but we're betting that it'll be _significantly_ less than doing it yourself.

</comment></comment></comment></comment><comment><USER10> This looks interesting! A couple questions (that may also apply to event sourcing more generally):

\- How do you handle events with side effects (sending emails, for example), and ensuring they aren't triggered on replay when they shouldn't be?

\- How do you handle randomness, like uuid generation?

<comment><AUTHOR> > How do you handle events with side effects (sending emails, for example), > and ensuring they aren't triggered on replay when they shouldn't be?

Someone else already addressed this, but to paraphrase: your application should be able to deal with duplicate events (and gracefully handle side- effects).

> How do you handle randomness, like uuid generation?

Are you referring to id generation and tagging in events (ie. aggregate id's)? If so, that'd be an application responsibility - you'd have to determine how to properly attach id's.

Hmm. But that does bring up an interesting idea - what if we provided a way to "group" events and generate aggregate id's on your behalf. Maybe that's what you meant - it's an interesting idea.

We currently don't do anything "extra" in regards to grouping events - we tag each individual event but that's about it.

</comment></comment><comment><USER11> I feel like a tagline like "event sourcing made easy" would hook me more and get me interested in _attempting_ to decipher your marketing page to understand the USP.

Pretty cool idea though. Hope it pans out for you guys.

<comment><AUTHOR> Understood. Yeah, the issue here is that the entire space is pretty complex - it feels like any angle you approach it from, it'll still be complex.

Will try to figure out a way to better communicate what we do.

<comment> mark-ruwt As you're brainstorming, try to imagine what you would shout in a loud bar (remember those?) if someone asked you what your company did. It can be a helpful mental tool to strip things down to the essentials.

Two-sentence pitches are much harder than two-minute pitches.

</comment></comment></comment><comment><USER12> One of my previous co. used Kafka and hacked something similar together on an internal Retool dashboard + DynamoDB. This definitely makes a lot of sense!

Will this work with Celery (python) configured with RabbitMQ as the broker?

<comment><USER13> It will work on any Rabbit queue as long as you are not using the default exchange for the queue.

</comment></comment><comment><USER14> Hi Dan/Ustin,

Congrats on the launch. The pain-point makes sense to me. I'm just curious - what's the big picture for you all? I imagine it must be larger than just replay.

<comment><USER6> Batch is betting that more companies are going to be utilizing event sourcing in order to scale. We want to be a foundation piece in their data infrastructure and support their transition into event sourcing by initially offering replays. We want to be a "One stop shop" for all event sourcing needs.

<comment><USER14> Cool! I don't have much data on how many companies are using events for key workflows, but I do know that many, many companies would _love_ to replay HTTP requests!

<comment><USER6> That's awesome! We support http and gRPC collection as well. Let us know what you have in mind.

</comment></comment></comment></comment><comment><USER15> It seems like you're solving quite a complex problem!

I'm curious how long it took you to build this initial product given the complexity.

YC has a bias for shipping quickly, but my gut instinct is that it would have taken you a while to build this initial version.

Did it only take a few months, or closer to 8-12+?

<comment><AUTHOR> Ha, great observation!

Yes and no :)

We've been exceptionally lucky to have several of our close friends help us out with building an MVP (also helps that our friends have serious experience!). There's a total of 6 of us - 3 people focusing on infra, frontend and Java connector bits, which allowed myself, Ustin and another dev to put 100% of our attention on backend services + arch.

That enabled us to knock this out in a few months. Without the assist, it would probably be closer to your estimate.

Something that may be of interest to some folks: we saved a _significant_ amount of time by not having to run our own k8s - we use EKS, it's very nice. Also, MSK - not having to run/manage ZK clusters and kafka nodes is a (costly) privilege haha

<comment><USER15> Nice, I'm jealous of the extra help you're getting haha (I'm trying to build something solo). It sounds like you're already making a lot of progress.

Good luck building out the rest of the product!

</comment></comment></comment><comment><USER16> I've solved the replaying bit before with a brute approach and AWS Athena. It would ingest all the events from S3, filter the unwanted ones out, and put the rest in SQS ready for consumption. It was definitely expensive though, not something you would run often.

<comment><USER6> This is definitely a valid approach and it is an expensive path. Athena charges a hefty amount per query if you have a large dataset. In addition, this approach won't work if your data is serialized with something like protobuf.

</comment></comment><comment><USER17> Very interesting tool. You're absolutely right that I wouldn't spend my innovation tokens on it. Congratulations for your work !

</comment><comment><USER18> Does this have support for Rabbit pub/sub? There's a bit of confusing wording on the page that makes it unclear.

<comment><AUTHOR> 100% - we use Rabbit internally for our own systems so it has first-class support.

I think maybe we should just list out the messaging systems we support on the front page, so you don't have to dig through stuff... Good point. Let me know if you've got any other suggestions.

</comment></comment><comment><USER19> I hear so much about Kafka, could someone give the two-sentence description of what it is and who uses it and for what?

<comment><USER6> From their website: Kafka is an open-source distributed event streaming platform.

There are many use cases from piping website activity tracking, metrics, log aggregation and stream processing. For us, it's a communication layer utilized by our microservices. An event goes into the stream and any services that cares about that data will consume it. In other words, it's like an ultra- resilient, scalable, redis pub-sub with history that runs on the JVM. You can read more about the use cases here: [https://kafka.apache.org/uses](https://kafka.apache.org/uses)

edit: Sidenote, Kafka is often waaaaaay overkill - if you need messaging, use something simpler like Rabbit or NATS or Redis and only use Kafka if you know why you need it.

<comment><USER19> Thanks. So an event that goes in would be something like, "user logged in," and services that care about that data would be...? Sorry still having some trouble understanding it.

<comment><USER13> Pretty much. A good example might be an online store. Let's say one of your internal services deals with notifying UPS, FedEx, or DHL to pick up and deliver a package from your warehouse and ship to a customer. You could use something like Kafka to store messages about delivers which your internal service will pickup and process and then notify the delivery companies API.

Something like Batch could be helpful in this situation. For example, let's say a dev makes a deploy that breaks only the FedEx delivery notification or the FedEx API breaks in a way your were not expecting. Once the issue is fixed on the dev side or FEDEX side you could use Batch to search for all FEDEX delivers that were handled improperly during the time frame of the issue. This way you are not randomly resending messages to all your delivery companies for an issue that was only related to one vendor.

<comment><USER19> makes sense, thanks. How would this be better than the logs you'd get from each service?

</comment></comment></comment></comment></comment><comment><USER20> Is Pulsar support on the roadmap?

<comment><USER6> We are planning to support as many messaging systems as we can. We will definitely investigate Pulsar. Going to add it to our feature list and make an issue on plumber to support introspection on Pulsar. Cheers!

<comment><USER21> Good to know that Pulsar is on your roadmap. Also, kudos to see user-land tooling around a common painpoint for teams doing any event processing at scale.

<comment><USER6> Thank you! We felt the pain point while actively trying to build observability tools in order to debug our messaging systems. We built plumber to standardize some of our internal tools and then decided to open source it to help others who are feeling the pain.

</comment></comment></comment></comment><comment><USER22> Nice concept and interesting, expect a demo request incoming :)

</comment><comment><USER23> Congratulations on this release....That is really useful!

<comment><AUTHOR> Thank you very much!

</comment></comment><comment><USER24> 'Light beers over IPA' sorry, I'm out.

;)

</comment><comment><USER25> Has anyone ever told you that you're "batch it crazy"?

<comment><AUTHOR> No, but this is now definitely going on a sticker :D</comment></comment>


Google points finger at Facebook hypocrisy, blocks Gmail import - <AUTHOR> http://arstechnica.com/tech-policy/news/2010/11/google-points-finger-at-facebook-hypocrisy-blocks-gmail-import.ars

<comment><USER2> Safe to say the cow is out of the barn here; if Google wanted to have an impact they'd have done this two years ago. How many people in industrialized countries don't yet have Facebook accounts? Like four?

<comment><USER3> I guess I'm one of that four than. I had one briefly, but I never grew sympathetic of them. I deleted it a few weeks later, Facebook is just too creepy in their way of data handling.

<comment><USER4> I don't think you can actually delete a Facebook account. It still exists somewhere...

<comment><USER5> Well u can, you have to contact them! See settings.

<comment><USER6> You have no way of verifying they've completely removed your data from their servers. All you can tell is they no longer let anybody see it through the website. They've publicly stated your data may be retained in "backups" indefinitely.

</comment></comment></comment></comment></comment><comment><USER7> People tend to like Google to keep to standards that would only make sense as a not for profit. Sure data portability is a good thing but when in specific instances your helping your competitor without them giving up anything it doesn't make sense.

I doubt shareholders would really want Google to sit back and give up the game on social.

<comment><USER8> People only hold Google to those standards because Google positions themselves as a champion of openness and user's rights.

As you point out, these standards may not be entirely realistic for a for- profit company operating in a very competitive space.

Google therefore must either take the risk that their standards will actually make them into a stronger company, or gradually compromise them for competitive advantage.

The problem with the compromise approach is that it undermines trust even more than if they'd never held themselves to such high standards in the first place, and that's where the backlash comes from.

</comment></comment><comment><USER9> It seems the 'introduce friends' feature should be an open standard - in the same way hopefully OAuth2 will solve reciprocal login credential validation.

<comment><USER10> <http://portablecontacts.net/draft-spec.html>

<comment><USER9> reference implementation ?

</comment></comment></comment><comment><USER11> I don't like Google putting artificial restrictions on what I can do with my data.

But I also prefer permissive Apache/BSD/MIT licenses to GNU GPL, so maybe that's just me.

<comment><USER12> AFAIK - This is not about the user. It's about other web services interfacing with those APIs.

</comment></comment><comment><USER13> Facebook plays dirty and now they got a little dirt thrown back at them, and I think they deserve it, because they are giving Microsoft the advantage by integrating with Bing. I'm happy that there is search engine competition, but not letting Google do the same is simply unfair.

</comment><comment><USER14> So now its - "its okay to be evil to fight evil" ? I thought googles stance on data portability was that its a good thing and that the users data basically belongs to them. I would guess most users setting up an account at service bar would be interested in importing their existing contacts / settings from service foo that they already use.

<comment><USER15> I would say this is more of a Facebook limitation.

You can still export your Gmail contacts... just not to automatically to Facebook (unless Facebook starts allowing easy export of its own data... which is the bigger problem right now, IMO).

<comment><USER14> When google said in the past that users are free to take their data and leave, I dont remember any caveats being in place. Whether they leave to join a idyllic commune or a kleptocratic dictatorship imo their data should freely go with them.

</comment></comment></comment><comment><USER16> Reminds me of <http://en.wikipedia.org/wiki/Deep_linking>

</comment><comment><USER17> I deleted my facebook account a long time ago.

</comment><comment><USER18> Where is that open-source MapReduce stack? Facebook even gives their HPHP, Hadoop, and other code back to the community for free. The fact is, Facebook cares about Data & Google cares about technology. Google has just perfected making other companies look bad, while Facebook has the attitude of "let's keep working on our features".

<comment><USER19> Sorry, but if you're comparing open source contributions, Google wins by any metric. Consider Chromium, V8, Android, WebM, GWT, Protocol buffers, the Closure Javascript library+compiler+linter, and jillions of other projects (<http://code.google.com/hosting/search?q=label:google>), not to mention the tremendous support they've given Mozilla over the years, their work in defining and promoting open standards like HTML5, plus indirect contributions via the Google Summer of Code program.

<comment><USER20> By "any metric"? How about, "lines of open source contributed per engineer- year"? Google is 10 times larger than Facebook, and has been around for twice as long. You would expect it to have a longer list of open source contributions no matter what.

<comment><USER21> The original article isn't about open source. It's about data portability and openness vs walled gardens.

Producing open source stuff is cool 'n all, but it doesn't really matter as much as being open vs closed in terms of what you can do with your own data.

Personally, I don't particularly care if a company decides to be totally closed source and not participate in open source contributions, as long as they don't start putting up walled gardens and trying to lock in user data.</comment></comment></comment></comment>


Objective-C's niche: why it survives in a world of alternatives - <AUTHOR> http://www.cocoawithlove.com/2009/10/objective-c-niche-why-it-survives-in.html <comment><USER2> I find it surprising that there are so many people bashing Objective-C here in the comments. I have been programming in Objective-C for living since the iPhone SDK came public and I came to love both the language and the libraries.

I think that many people who despise Objective-C syntax are simply not programming long enough in the language to overcome the initial knee-jerk reaction. Mostly the issue seems to be the message passing syntax, which I find very readable and self-documenting. What else should matter if not this?

Then there's comments about speed. I have written two action games for the first-generation iPhones and I have never encountered a performance problem caused by the runtime. Over 99% of the time I could stick to high-level code without resorting to C to gain speed. That says _a lot_ about performance for regular applications.

Block syntax could be better, I can't use closures with such ease as in Perl (for example). But this is one of the things that have to do with Objective-C being a relatively thin layer atop of C. And that I see as a huge win, because you can very easily and naturally mix higher-level Objective-C code with plain C APIs that are very common on most systems. And if you want some lower-level bit-twiddling trickery for performance reasons, you are free to go. For me, C is a very elegant language and Objective-C takes out most of its rough corners.

Saying that Objective-C could be replaced by something "smarter" like Lisp or Haskell is simply ignoring the reality. Objective-C has very low language complexity, both the building blocks and their interactions are simple and there are few surprises. This is a huge advantage both for the newcomers and for programmers that deal with the stack most of their working hours.

The manual memory management first seemed like a chore to me after coming to Objective-C from Java, but the rules are dead simple and after a few weeks you don't think about the memory management any more.

Yes, Objective-C is not hip. So what? It's a pragmatic and reasonably expressive language that gets the job done. I don't think that replacing it would serve any real purpose – and that's a very good reason to survive.

<comment><USER3> My biggest beef with Objective C is that it takes the most retarded feature of C++ -- that you have to write both a header and a implementation file (why oh why can't the compiler compile one?) -- and removes the really nice ability to allocate your object on the stack (which means memory allocation is really, really, cheap and you know when the object is deallocated).

<comment><USER2> As for the header/implementation split, I might have been brainwashed by the language, but I like it :) I take it as a nice division between the public interface and private implementation. And now that we have purely synthesized properties in the modern runtimes, we can keep the header file really clean:

<https://gist.github.com/766381>

I have never missed allocating objects on the stack, I am mostly happy with autorelease pools. They are slower and "less deterministic" than stack-based objects, but I don't mind.

</comment></comment></comment><comment><USER4> Obj-C survives b/c it is the only real way to write iPhone and iPad apps, plain and simple (ignoring Appcelerator and PhoneGap for the moment). If people had options, it is likely they would use the alternatives.

Look at Linux. The "official" language will always be C, but there are toolkits for building GTK apps in Python, Vala, Mono and other even more obscure languages. Some pretty big apps have been written in each and people will continue to use those as long as they are viable options. If, however, someone big enough and with enough clout basically said "only native C/GTK apps will be accepted on our platform", you would see people migrate to that toolchain. It is simple economics.

Apple has the clout (imo, unfortunately) to do that and have done it with Obj-C. They basically killed Java, which was an option up until I think 2 years ago. I can see Apple's perspective, supporting two toolkits is expensive and takes time, resource, energy and focus. In their mind, what benefit to they gain from supporting multiple toolkits? Developers? They know they are popular enough that the developers will take up Obj-C when Java was dropped. Sure there will be grumbling and moaning, but Apple DOESN'T CARE ABOUT DEVELOPERS except that they make apps to attract the users.

I'm not saying what Apple did is right or wrong, but it does make sense from a strategic point of view. However, there is practically no merit to Obj-c on it's own. There are more compelling alternatives at nearly every turn, but NOT if you want to play in the iphone and ipad ecosystem.

<comment><USER2> What would you rather write iOS/Mac applications in if not Objective-C?

<comment><USER5> Me personally? Lua. That language is as simple and as rugged as a cockroach.

<comment><USER6> <https://github.com/probablycorey/wax>

</comment></comment></comment></comment><comment><USER7> Objective-C's niche: it's the core of two popular operating systems.

A lot of strange programming languages survive because of inertia. Emacs Lisp exists because Emacs was written by Lisp hackers. It survives because Emacs ships with a few million lines of Lisp. Replacing it would take hundreds of man-years.

Objective-C survives for the same reason; Cocoa was expensive to write and rewriting it would serve no purpose. So it remains. Is anyone using Objective-C for some reason other than writing OS X or iOS apps?

<comment><USER8> No, and if there was an easy way to bind UIKit with out Obj-C I'd use it in a heartbeat. I'm seriously considering WAX for LUA. Obj-C has the flexibility of C with the speed of small talk.

<comment><USER9> That strikes me as a pretty indefensible statement. Objective-C has an incredibly powerful runtime, which is nearly as flexible as the ruby runtime. It's also much much faster than ruby, and probably every other popular dynamic language.

<comment><USER10> I'm sure the runtime is great, unfortunately the syntax sucks, there is no GC (on iOS) and it can't curry functions, and you can't put an NSInteger into an NSDictionary with out line noise that would do java proud, it doesn't support passing or currying functions.

Sorry, but passing a selector and an object is atrocious, even MS can figure out how to pass a delegate that can figure out whether it needs to pass a this pointer before passing the rest of the arguments.

It's lambda syntax is also attrocious [[ ^ (int x, int x) { return x+y; } copy] autorelease] , wtf is that garbage?

In ruby I don't need to touch 2 different files and put code in 3 places just to define a property, nor do I have to do this in C# or F#.

I doubt it's much faster than C# or F#, especially if it ever involves something simple like putting an integer in a hashtable.

I'd hardly call a language dynamically typed when it will fuck up adding a long and an int. Seriously, define a type that takes an int, pass it a long and see what happens. This is exactly what I mean by the flexibility of C (attrocious type system) and the speed of small talk (incurring a call to objc_msg_send anytime you do anything)

<comment><USER11> * Syntax sucks: I think Objective-C is actually pretty beautiful. Maybe I'm crazy.

* NSInvocation should serve the same purpose in most cases.

* I find the block syntax atrocious if compared to Lisp lambdas. It's not that bad. Blocks are objects, so you're creating an object, with all the same predictable retain/release semantics. Maybe a few gotchas if you're not reading the documentation and realize you should avoid retain loops. Hardly different from Java, only it's easier to miss it in Java, because you never release, anywhere.

<comment><USER12> The lack of namespaces seems silly to me: NSThis, NSThat.

It isn't that Obj. C is a bad language, it's just it doesn't feel like a more modern language, and yet it gets used because of the possibility of $$$. I do understand that you can come to love a language after a period of using (and understanding) it. And I do think a systems-level language should be used for apps on mobile devices, but we don't have to continue to be mired in implementation details that were an artifact of a language from the seventies.

I need to play with it more to get a better feel for it.

</comment></comment></comment></comment></comment></comment><comment><USER13> Based on the comments, seems like most people's objections of Objective-C are really objections about C: header files, manual memory management, function pointers, strong type system, too low level, no native string object, etc. I grew up with C, so learning Objective-C took maybe 10 minutes because it's basically C with some extra syntax for OOP. In general, one only encounters the "strange" syntax when dealing with methods. I personally love the method syntax because it's basically self documenting.

Which leads me to wonder, how many programmers both on HN and in general know C? I'm too young to already be a dinosaur in this industry... Where did all the C programmers go? Seems like programmers nowadays started learning how to code with Java or Ruby/Python and expect all other languages to be just as high level. Objective-C _is_ low level, which makes it harder, but doesn't make it suck.

As a language, I think Objective-C is a fine one, but I must admit that it is a "dinosaur". The real problem is OOP paradigm. Remember, back in the late 70s and early 80s, OOP was new and exciting and C was the dominant language. People were trying to figure out what OOP _should_ look like and how to make C more OO. NextStep (Apple) just happened to bet on the wrong horse. Obj-C went with Smalltalk syntax, while the rest of the world went with C++ syntax and ultimately the Java style syntax we know and love today. Seems like some developers can't wrap their heads around a non-Java syntax for OOP.

All that said, Objective-C survives only because Apple hasn't chosen another language to replace it for Mac and iOS programming.

<comment><USER14> > Obj-C went with Smalltalk syntax, while the rest of the world went with C++ > syntax and ultimately the Java style syntax we know and love today.

It's more than just a syntactic difference. Objective-C implements true _message passing_ which is not the same as "method calling". You can send messages to a nil object, or have an object dynamically respond at runtime to a message it didn't declare at compile time etc.

Some seem to think the brackets are just some kind of whimsy, but it's important that sending a message is not to be confused with calling a function.

</comment></comment><comment><USER15> Why it survives in a world of alternatives? There are no alternatives if you want to distribute on the most popular mobile platform. Personally, I _really_ dislike Obj-C, but since I want to create apps on the iPhone I have no choice. Unfortunately, making a good business decision sometimes means going against your personal feelings. I think that's how a lot of other people approached it too. If the iOS platform changed overnight to Lisp, we would see articles titled "Why Lisp survives in a world of alternatives"

<comment><USER16> > There are no alternatives if you want to distribute on the most popular > mobile platform.

Symbian, Blackberry and Android don't support Objective-C as far as I know. Only iOS does, which is far from the "most popular" mobile platform, especially if you look outside Silicon Valley just for a minute.

[Edit] Also, for the record, you can program in C++ and distribute on iOS with minimal Objective C. I thought I'd point this out given my login name.

<comment><USER17> I think you both are arguing from different sides of "popular". You are stating popular by sheer numbers while he was stating it from a 3rd party developer prospective. I believe he was right in what he said.

</comment></comment></comment><comment><USER18> I've been using objective-c since 2001, when I made the switch from BeOS to Mac OS X. It's not horrible. I don't have any real problems with it. But I think its replacement is long overdue.

Given that Apple has a long history of ditching huge swaths of its technology in favor of more modern alternatives, no matter if their userbase screams bloody murder or not, I'm surprised it hasn't happened already. This would be a good project for those billions they've got sitting in the bank.

I think it would be great if they would anoint some successor as their new systems language, then port Cocoa to it. Programmers could use either objective-c or the new systems language for a few years, just as we used to decide between Cocoa and Carbon. Then objective-c could gradually fade into the past.

<comment><USER19> that's fine, but what would they replace it _with_?

Java? C#? not likely. C/C++? well, you can use them already

I think MacRuby is the closest we have atm to something being groomed as a successor to Obj-C. It's got a way to go yet, but it's already at the stage where it's viable for smaller projects and there's a (small) number of MacRuby apps available on the OSX app store.

<comment><USER20> Any idea how much investment is taking place in MacRuby? If they're really interested in grooming a successor, I'd figure that there would be some serious work going on, behind the scenes.

With the success of the platforms on which Obj-C is used, I don't know if a successor will be another language. An update to Obj-C itself might make perfect sense.

<comment><USER21> Well apparently, MacRuby is already a private framework in Lion, so it serves some purpose in the eyes of Apple.

[http://merbist.com/2011/03/07/hey-apple-please-be-nice- and-s...](http://merbist.com/2011/03/07/hey-apple-please-be-nice-and-share/)

</comment></comment></comment></comment><comment><USER22> meta tl;dr:

The blog post talks about interesting runtime properties of the Objective-C/Cocoa runtime that differentiate it from C and C++/Java.

Hacker News mostly rants about the Smalltalk-like syntax.

</comment><comment><USER23> I wish Objective-C were more popular on linux, I like the way I can mix message passing style OO with C code.

<comment><USER24> There aren't really any good libraries for it. You can compile straight ObjC on linux fairly easily with gcc or clang (use -lobjc), but the thing is without NSString and all of the foundation you'll end up doing a lot of work.

And yes, you can get into GNUStep, but that's a whole nother ballgame.

</comment></comment><comment><USER25> > Objective-C remains an impediment for many programmers

Really!? If that's such an impediment maybe "many programmers" need to look for a new career. Or, go back to copy pasting PHP / hiding their schlubness amongst the other cogs in some Enterprise Java machine.

</comment><comment><USER26> Obj-C, at least in the last few generations of mobile hardware, is a competitive advantage for Apple. It makes it easy to write fast, low-level code in C, and then to move transparently up to a much higher level of abstraction for non-critical code. With the relatively underpowered processors of today's mobile devices, this kind of control is important.

However, Obj-C completely falls on its face for anything of real algorithmic complexity. Working with complex, nested data structures and more sophisticated algorithms is _very_ awkward, and manual memory management only clouds things further. In the longer run this may become a liability for Apple.

<comment><USER2> I fail to see how manual (or, in this context, _explicit_ ) memory management could "cloud things". (It seems to me simply like many programmers can't be bothered to invest the afternoon it takes to learn Cocoa memory management.) Also, could you elaborate why Objective-C makes it hard to work with nested data structures and sophisticated algorithms? In fact, I am struggling to find an example of at least a single general-purpose language that makes working with "sophisticated algorithms" hard.

<comment><USER26> The kinds of algorithms you can express very cleanly in a functional language, like Haskell, for example, are a lot more awkward when you have to worry about managing memory at each intermediate step. Look at all the monkeying around you have to do with __block declarations etc. For typical Obj-C code, where you're just gluing together bits of the Cocoa API, I agree that Obj-C's memory management isn't too bad. When you're trying to work at a higher level of abstraction, it's a distraction.

Writing complex algorithms in Obj-C is a pain because of it's incredibly clumsy syntax for collections and the manual coercion of elemental types back and forth to objects. Look at an implementation of a neural net or something as simple as k-means clustering in Obj-C, then C++, then Scala or Haskell to see how poor Obj-C is for this kind of thing.

For example, imagine how much worse this would be in Obj-C: [https://github.com/cageface/brains/blob/scala/src/main/scala...](https://github.com/cageface/brains/blob/scala/src/main/scala/brains/clustering/KMeans.scala)

Admittedly, Scala is particularly nice for this kind of thing, and Java is no better than Obj-C, but even C++ gives you much better tools for abstracting this kind of code.

<comment><USER2> Thank you, that's helpful.

The algorithm is a bit above my head to rewrite in Objective-C on Sunday, but from my superficial understanding of it the Objective-C version wouldn't be really ugly. Yes, it would be wordier and clumsier, but tolerably so (for me). After all, it's precisely these things where functional languages shine.

I seldom find I have to monkey around with __block. Manual boxing/unboxing sucks, yes. It can be alleviated by a simple macro where this really matters, but I can see such and argument is like pouring petrol into the fire when compared to functional languages :)

Now I understand what was meant by "sophisticated algorithms". Yes, Objective-C is not perfect for those – first class collections are a great convenience here. But I certainly do not see that as a future "liability for Apple" or a huge drawback for the language ecosystem. It's a niche.

<comment><USER27> Its a niche for now. But local intelligence in mobile software is going to matter more and more as ad hoc and sensor networks become more common - otherwise the explosion of data might as well not have happened.

<comment><USER26> And this is just one domain in which Obj-C falls short. For another example, compare doing linear algebra for graphics in Obj-C vs C++ with a nice matrix library.

</comment></comment></comment></comment></comment></comment><comment><USER28> With early iOS they were talking about a pretty serious use of JavaScript. I wouldn't be surprised if they pushed that more in the future...

I've heard there is a lot of work being done to make Haskell play nicely with Xcode [1]. I don't know if Apple is experimenting with that, but they're probably kicking a lot of options around.

[1] <http://hackage.haskell.org/platform/mac.html>

<comment><USER29> Did you paste the link you intended to? I don't see how it supports the idea that there is work being done to make Haskell place nicely with XCode.

<comment><USER28> Work being done by Haskell people, if not Apple people.

<comment><USER29> I'm not even getting that from the link. To me, the link only shows that I can download The Haskell Platform for my Mac.

The only thing there related to XCode is that one needs to have it installed as a prerequisite for running the haskell platform's installer.

This is no surprise because the installer wants to have the "unix development support" package available (it contains gcc, header files, misc command line tools in locations where run-of-the-mill Makefiles can find them...) and it's just incidental that one needs to run the XCode installer in order to get these basic unix amenities on the system.

</comment></comment></comment></comment><comment><USER30> After starting to read a bunch of objective C, I have to say there are some constructs that are very nice compared to an OO language like java -- for example, selectors and the target action design pattern. There's definitely some nice dynamic properties, but at the same time, some of the basic "methods/messages" for the Foundation APIs aren't very convenient to write and are strangely named. For example, compare NSString to strings in Java, Python, Ruby, etc.

</comment><comment><USER31> _This makes the following situations possible:

1) You want to add a convenience method to someone else's object (a quick search of my own posts reveals that about a dozens of my own posts involve adding convenience methods to Cocoa classes, e.g. Safely fetching an NSManagedObject by URI).

2) You want to change the behavior of a class you didn't (and can't) allocate because it is created by someone else (this is how Key-Value Observing is implemented in Cocoa).

3) You want to treat objects generically and handle potential differences with runtime introspection.

4) You want to substitute an object of a completely different class to the expected class (this is used in Cocoa by NSProxy to turn a regular object into a distributed object)._

One can do all of the above with the Go language. In addition, one could also write a reasonably performant compiler in it. (Also JIT dynamic language VMs are good enough now, that I've seen dynamic language parser/compiler implementations perform as well as a naively written Lex/YACC C implementation. And yes, companies paid money for the latter!)

</comment><comment><USER32> Has anyone used pyobjc or some other interface between objective-C and another language? Thoughts?

<comment><USER33> I used PyObjC a few years ago, and enjoyed it much more than straight Objective-C. Far less boilerplate, and the transparent mapping between Python lists and dicts and Cocoa's NSArray and NSDictionary is great.

</comment></comment><comment><USER34> The title of the post is very misleading: Obj-C would crash and burn if there were native, supported alternatives in OSX, it survives exactly because there are no alternatives.

<comment><USER35> Ok, that's simply not true. On Mac OS X, you're welcome to use any language and runtime you wish for your application. Hell, if you want to use a non- Objective-C language and still present a native experience, use PyObjC, RubyCocoa, MacRuby, or any other language that shims into the Objective-C runtime and get access to all of Apple's Cocoa frameworks in the language of your choice.

The same is possible on iOS as well, with the (significant) condition that Apple must first explicitly approve your use of a non-Objective-C runtime before your app can be posted to the App Store.

But in direct response to your post, Obj-C has a rather level playing field on OS X and still survives vs. the alternatives.

<comment><USER36> Welcome to use it, except:

* it's not officially supported by Apple

* it binds to Cocoa (and if you're lucky it's even up to date and doesn't have that many bugs)

* you can't use it on iOS

* you can't get a job doing X instead of Obj-C because almost everyone just uses Obj-C, the Apple supported language

<comment><USER35> > * it's not officially supported by Apple

MacRuby is an Apple project.

PyObjC isn't, but ships in Mac OS X. That's rather officially supported.

> * it binds to Cocoa (and if you're lucky it's even up to date and doesn't > have that many bugs)

Not true. MacRuby is a prime example of a language that taps directly into the Objective-C runtime, eliminating the need for a bridge. And PyObjC is a great example of a mature bridge – it's been around and robust since the NeXT era.

> * you can't use it on iOS

[http://mobileorchard.com/announcing-iphone-wax-native- uikit-...](http://mobileorchard.com/announcing-iphone-wax-native-uikit-iphone- apps-written-in-lua/)

> * you can't get a job doing X instead of Obj-C because almost everyone just > uses Obj-C, the Apple supported language

You can't get jobs programming Ruby, Python, C#, Lua or any other language?

</comment></comment></comment></comment><comment><USER34> Let's face it: in 2011, Obj-C has the sex appeal of COBOL.

I think it's primarily a strategic decision: Just like apple likes to lock-in its customers with itunes, so it does with developers, locking them behind an unpopular language with a slow learning curve.

<comment><USER37> Ignoring the disputable claims that it's unpopular or has a slow learning curve (you can describe Objective-C's additions to C in a single page), there's no evidence to claim that Objective-C is about customer lock-in, especially when you can use so many alternatives including the Apple-funded MacRuby.

Apple uses Objective-C because they have a lot of code written in it, and it has served them very well. iOS is the most popular mobile platform for third- party development. The dynamic nature of the language has enabled the implementation of some useful features like key-value-observing, while the ability to drop to C retains the flexibility of system-level programming.

Even the memory management criticism is overblown. You merely have to follow an easy list of four ownership rules. That's if you're not using the garbage collector that came out four years ago with Leopard.

<comment><USER38> If you follow those "easy rules," how do you handle race conditions between weak references held to avoid retain cycles and the execution of those weak reference's -dealloc methods? There's nothing to stop the weak reference's -dealloc from racing with messages to that weak reference.

Or, how do you handle the fact that -dealloc can be called on whatever thread calls the last -release, even if you make an effort to only send messages to an object on the main thread (Which will trigger implicit retain/release unless you're incredibly careful). Almost all of the GCD examples out there are subtly broken because of this.

As for GC, I look forward to actually being able to use it on iOS.

<comment><USER39> I can't address your first point because I haven't used GC yet, but your second seems like a contradiction. If you're releasing from a background thread then you are sending messages from a background thread. You can easily put code in our object's release method to fix that, though. "If not main thread, call self on main thread."

<comment><USER38> This is a hack, not a general purpose solution, as it requires modifying the class, and and still must be done carefully to avoid accidental retain/release.

Calling retain/release from a background thread will happen simply by using blocks, gcd, or performSelector: APIs, as designed, to attempt to schedule execution on the main thread. Avoiding the issue requires exquisite care, unless you actually want to modify every single class that you may access an instance of from a background thread (such as GCD).

As to the first point, it applies in a NON-GC environment, where you hold weak references to delegates a(nd deregister them in dealloc.)

<comment><USER39> Sorry, the term "weak reference" threw me. So you mean you have pointers to objects you have not retained, and you "deregister" (set them to nil?) in your dealloc? I don't see how that can trigger a race condition or call anyone else's dealloc.

You may have a valid point but I don't grasp it.

<comment><USER38> Think delegates, or observers. There is a race condition between calling the delegate and deregistration of the delegate. Once an object's dealloc method has been called, you must not resurrect it, but there is a race between the execution of -dealloc, the de-registration of the delegate, and the dispatch of messages to the delegate.

<comment><USER39> So you're setting threadedObject.delegate = nil in your delegate object's dealloc, but the threadedObject is sending subsequent messages to your delegate? The doesn't sounds like a retain-release problem. Wouldn't the same thing happen when setting threadedObject.delegate=foo? Messages could still get send to the old delegate if you haven't done your synchronization right.

<comment><USER38> Missing the forest for the trees. The problem is that the old delegate will be invalidated before it deregisters itself, because it will not deregister itself _until_ it is invalid (during -dealloc)</comment></comment></comment></comment></comment></comment></comment></comment></comment>


Too Hot for TED: Income Inequality - <AUTHOR> http://mobile.nationaljournal.com/features/restoration-calls/too-hot-for-ted-income-inequality-20120516

<comment><USER2> TED's argument that the talk is overly political would appear to be supported by the fact that this article appears on a site dedicated almost exclusively to politics. Out of 56 non-external article links the site's home page, only three lead to non-political articles. Those are, respectively, a story about congressmen having their houses burglarized, a take on the BrainGate robotic arm story, and a stub of an article mentioning that 13% of US citizens are born abroad.

Having read the text of the talk, it is in my opinion completely political, and while I may agree with his basic point; he fails to cite examples of where implementation of his propositions had any positive effect. The tone of self- righteous loathing used when describing his ideological opponents does little to encourage sober consideration of his arguments.

As a final point, the speech makes no attempt to address the bear in the room, i.e., the question of whether it is right to take by force from one person and give to another, regardless of how beneficial the effects.

It's an extremely political talk being publicized by an entirely political web site with a title more suited to a Girls Gone Wild commercial than talk on socioeconomics. "Too Hot for TED" indeed.

<comment><USER3> Can you I just question your line: "As a final point, the speech makes no attempt to address the bear in the room, i.e., the question of whether it is right to take by force from one person and give to another, regardless of how beneficial the effects."

I think you're talking about tax. I would have thought that most people would agree that tax is necessary for a functioning civilisation. Those people who don't want to be taxed can use a different currency/road network/law system.

<comment><USER2> More the way the speech assumes that the purpose of taxes and the use of monies thereby gained is to make people more equal, rather than giving them equal opportunity to make more of themselves, or to cushion only those physically or mentally incapable of keeping themselves alive.

</comment></comment></comment><comment><USER3> It seems as if this concept of not discussing politics in 'an election year' is the same as suspending public debate in the US for 1 year out of 4. So that's like the government wasting 25% of its time doing little and saying nothing.

Additionally from my distant from the US point of view I would say that suspending public debate is essentially coming down on one side of the party line.

</comment><comment><USER4> As a European, I'm again and again surprised how many American media and other entitities try to be non-political. In my view, everybody is political, and speaking ones mind advances society. Why is everybody trying to stay neutral?

<comment><USER5> They aren't trying to be neutral : they are hiding behind neutrality. Their paying audience has no interest in hearing that they aren't the ones that make the world go round.

</comment></comment><comment><USER6> non-mobile version of the article:

[http://nationaljournal.com/features/restoration-calls/too- ho...](http://nationaljournal.com/features/restoration-calls/too-hot-for-ted- income-inequality-20120516)</comment>


Redesigning the Scientific Paper - <AUTHOR> https://www.theatlantic.com/science/archive/2018/04/the-scientific-paper-is-obsolete/556676/?single_page=true <comment><USER3> > These programs tend to be both so sloppily written and so central to the > results that it's contributed to a replication crisis, or put another way, a > failure of the paper to perform its most basic task: to report what you've > actually discovered, clearly enough that someone else can discover it for > themselves.

This is the crux of the of the problem IMHO - at least for the fields I study (AI/ML). Replicating the results in papers I read, is way harder than it needs be, i.e. for these fields it should just be fire up a jupyter notebook and download the actual dataset they used (much harder than it seems to actually get your hands on). Very few papers actually contain links to all of this in a final polished manner so that it's #1 understandable and #2 repeatable.

Honestly, I'd much rather have your actual code and data that you used to get your results than read through the research paper if I had to choose (assuming the paper is not pure theory) - but instead there is a disproportionate focus on paper quality over "project quality" at least IMHO.

I don't really know what the solution is since apparently most academics have been perfectly fine with the status quo. I feel like we could build a much better system if we redefined our goals, since I don't think the current system is optimal for disseminating knowledge or finding and fixing mistakes in research or even generally working in a fast iterative process.

<comment><USER4> I've had a paper peer reviewed. It was ultimately rejected but I can't help but suspect that by making all my code publicly available, I hurt my chances of publication. The reviewers comments were about my coding style, my choice of build tool (I didn't use make, but something else which is just as easy to use), the choice of C vs C++...

It's like best practices for computer security -- always strive to minimize the attack surface. :) Without source code there is much less stuff to criticize!

<comment><USER5> > It's like best practices for computer security -- always strive to minimize > the attack surface.

I suspect that's also why some papers are unnecesarely verbose and describe simple things as complicated as possible. Can't criticize something that can't be understood.

<comment><USER6> Then why submit it for _peer review_ at all?

<comment><USER7> Because we need peer reviewed papers on our CVs!

I also detest simple things made complex, though. In my experience (with has covered electronics, epidemiology and geography) reviewers tend to pick up on obtuse issues in text but miss glaring errors in the math. It's sad, and you can see why someone less than scrupulous would exploit that tendency by over complicating things. That said I think plenty of authors are honest but just not very clear thinkers!

</comment></comment></comment></comment></comment><comment><USER8> > _How to integrate billions of base pairs of genomic data, and 10 times that > amount of proteomic data, and historical patient data, and the results of > pharmacological screens into a coherent account of how somebody got sick and > what to do to make them better? How to make actionable an endless stream of > new temperature and precipitation data, and oceanographic and volcanic and > seismic data? How to build, and make sense of, a neuron-by-neuron map of a > thinking brain? Equipping scientists with computational notebooks, or some > evolved form of them, might bring their minds to a level with problems now > out of reach._

The article seems to conflate the praxis of science with the archival of it. Scientists do all of the above on gigantic clusters, not on an IPython/Mathematica notebook. The purpose of publishing papers, on the other hand, is adding to the archival of knowledge, and they can be easily rendered in a laptop with LaTeX.

And they are excellent at archival, by the way. You can see papers from the 19th century still being cited. On the other hand I have had issues running a Mathematica notebook from a few releases back -- and I seriously doubt one will be able to read any of my Mathematica notebooks 150 years from now. The same with the nifty web-based redesign of the Nature paper that is mentioned: I bet the original Nature article will be readable 150 years from now, whereas I doubt the web version will last 20.

<comment><USER9> A group upstairs at the Broad Institute built out a system to use Jupyter notebooks for analysis of genomic data, with backend computation happening on a Spark cluster[1]. Science on large datasets can happen via interactive notebook. In a connection with a recent GWAS on a massive dataset from the UK Biobank, the researchers involved decided not to write a traditional scientific journal article (at least for now) since their analyses will continue to mature. Instead, they've been posting insights online in blog form, with associated code on GitHub[2]. It's a daring move toward publishing at the speed of research. Once their conclusions mature, traditional journal articles may follow to distill and preserve the key findings. In the mean time, those in the field can apply the same code to their data, replicate the analyses, and get an early look at the output of the research. This works partially because the methods (univariate GWAS) are understood in the field and the interpretation and rendering of a particular dataset is the science in this case, rather than a new method (which would still likely warrant a paper).

1\. [https://hail.is](https://hail.is)

2\. [http://www.nealelab.is/blog/2017/7/19/rapid-gwas-of- thousand...](http://www.nealelab.is/blog/2017/7/19/rapid-gwas-of-thousands-of- phenotypes-for-337000-samples-in-the-uk-biobank)

<comment><USER8> > _Science on large datasets can happen via interactive notebook._

I did not claim the opposite, just that it regularly happens without interactive notebooks. This seems like an interesting project though. Regarding the blog posts, it seems that there's a bug that makes all the entries appear as published on September 20, 2017?

</comment></comment></comment><comment><USER10> I used to work as a software developer for a research institute. I wanted to open source our research code and tools, and the department head was in favour of it because it would raise the profile of the research unit.

There were two forces working against us. First many of the grants came from governments, and a stipulation was that we would devote some resources to helping startups commercialise the output of the research. Some felt that open sourcing would remove the need for the startups to work directly with them to integrate the algorithms, and that this would hurt future grant applications by making the research look ineffective.

The main opposition though came from PhD and Postdoc students. Most didn't want anything related to their work open sourced. They believed that it would make it easy for others to pick up from where they were and render their next paper unpublishable by beating them to the punch.

Sadly I think there was some truth to both claims. Papers are the currency of academics, and all metrics for grants and careers hinge off it. It hinders cooperation and fosters a cynical environment of trying to game the metrics to secure a future in academics.

I don't know how else you should measure academics performance, but until those incentives change the journal paper in its current form is going nowhere.

<comment><USER11> > Papers are the currency of academics, and all metrics for grants and careers > hinge off it. It hinders cooperation and fosters a cynical environment of > trying to game the metrics to secure a future in academics.

I honestly can't contemplate who in their right mind would want "a future in academics" where academia is defined as a constant stream of metric gaming rather than actually accomplishing what you originally set out to accomplish.

<comment><USER12> Imagine it like this: maybe you joined your field (be it software or academia) with dreams of making the world better. So it shouldn't matter to you _who_ does the problem solving, as long as the problems get solved and the world gets better in a timely manner. But then, you've found a gig solving an important problem, with ok-ish pay. And you've also found yourself with dreams of a home, a spouse, maybe children. And suddenly, it starts to matter to you whether or not you get paid, so that you can fulfill your dreams. Maybe someone else could do your important work better than you. But then you won't have money to support your family. So it's gaming time, and now you end up focusing on "a future in <your field>".

</comment></comment></comment><comment><USER13> Regarding the article itself: Brett Victor is amazing and so is Strogatz. They are both my heroes actually.

But I do think there is a difference between scientific professionals communicating amongst each other and scientific communication to the public. And if mathematicians understood Strogatz his paper at the time when it was published, and there were enough mathematicians to disseminate the knowledge, then should you require that you create algorithms as animations?

Part of the reason why mathematicians and computer scientists (as researchers) conceive of new algorithms in the first place is because a lot of them are very strong in visualizing algorithms and 'being their own computer'.

Though, if a scientist wants to appeal to a broader group of scientists, then I'd recommend her or him to use every educational tool possible. For example, they could create an interactive blogpost a la parable of the polygons[1] and link that in their paper.

On an unrelated note, it is such a pity ncase isn't mentioned at all in this article!

Also related is explorabl.es, not everything is science communication in an interactive way but a lot of it is[2].

[1] [http://ncase.me/polygons/](http://ncase.me/polygons/) [2] [http://explorabl.es/](http://explorabl.es/)

<comment><USER14> >Part of the reason why mathematicians and computer scientists (as researchers) conceive of new algorithms in the first place is because a lot of them are very strong in visualizing algorithms and 'being their own computer'.

Can you explain this in more detail?

</comment></comment><comment><USER15> We need GitHub for science. But that's not enough. It needs to be combined with a mechanism for peer-review and publishing that funding agencies will find acceptable--that's the key.

<comment><USER16> It's not only about funding agencies, though. Researchers need to be trained in the use of a GitHub like system - as a former graduate student, I knew NO ONE using Github. Furthermore, after working in scientific publishing where it was MS Office or bust, I doubt you will convince Elsevier and the publishing giants that GitHub for Science is the way to go. How do these huge companies make money if scientists are sharing their work for free online? Meanwhile, you have scientists themselves, many of whom are required to hit certain numbers of articles in certain "prestige tiers" of journals in order to get tenure. If you're an extremely busy junior faculty member who will be FIRED when tenure review comes up if you don't have 1 paper per year, will you risk it on a GitHub scheme? I doubt that.

These problems are not unique to science. Take any large group of humans, be it government, the military, a company, a set of companies: they will coalesce onto the use of a certain number of tools, and it will take a LOT of energy and work to switch tools. Getting an entire industry to switch tools is extremely hard, perhaps downright impossible, and I'm not sure if it can be done in the rather slow-moving world of academic science. It simply won't be possible to enact a huge change on the system with so much institutional momentum built up.

It seems to me the best solution is to make incremental changes toward a github like system, training new scientists in grad programs to have better practices. Maybe we can't get researchers to post their full data set for all the public, but perhaps it can be made available to non-competitors, those who researchers aren't actively competing with for the same grants. Maybe researchers can be required by journals to post the peer-review comments/responses, as well as the drafts of the article, alongside the article itself. Maybe we can have researchers posting extremely detailed code/methodological information alongside the article, without forcing them to give over the entire dataset to other researchers?

Finally, maybe the whole incentive structure of academic science needs to change: maybe articles/publishing metrics should be de-valued in favor of teaching skills, mentorship ability, and collaboration within and across disciplines?

<comment><USER17> _I doubt you will convince Elsevier and the publishing giants_

Who says we need to convince them? How about we leave them behind? They are rentiers, gatekeeping society's access to publicly funded scientific knowledge. I can't think of a reason why society should allow this hostage situation to continue.

<comment><USER18> OK, but you need to figure out a way to take over the universities then because scientists are also animals who need food and shelter etc. etc. and depend on grants, stipends, and salaries to buy those things.

I'm not saying this to be dismissive - I'm strongly in favor of faculties organizing to unseat administrators from their privileged positions. It's baffling to me that bright minds on campuses complain at length about the state of higher education but seem oddly averse to _doing_ anything about it.

</comment></comment></comment></comment><comment><USER19> I do not agree that the scientific paper needs to be replaced. It should be complemented with the help of new tools, that is a very good thing, but I still want the article.

I work everyday with papers from decades ago, and I hope people will work with my papers in the future. How can I guarantee that researches of 2050 will be able to run my Jupyter notebooks?

Moreover, it is not uncommon to not be able to publish source code. I can write about models and algorithms, but I am not allowed to publish the code I write for some projects.

</comment><comment><USER20> "... the skill most in demand among physicists, _biologists_ , chemists, geologists, even anthropologists and research psychologists, is facility with programming languages and "data science" packages."

If I wanted to prove to someone this statement was true, what would be the most effective way to do that?

Is author basing this conclusion on job postings somewhere?

Has he interviewed anyone working in these fields?

Has he worked in a lab or for a company doing R&D?

How does he know?

What evidence (cf. media hype) could I cite in order to convince someone he is right?

When I look at the other articles he has written, they seem focused on popularised notions about computers, but I do not see any articles about the academic disciplines he mentions.

</comment><comment><USER21> GitXiv very much worth taking a look at if you're into this kind of thing: [http://www.gitxiv.com/page/about](http://www.gitxiv.com/page/about)

edit: as is Chris Olah's Distill project: [https://distill.pub/](https://distill.pub/)

</comment><comment><USER22> How well does it work to version control Mathematica notebooks in git? For example, is it possible to get meaningful textual diffs when comparing two versions of a mathematica notebook, and can git compress them enough to keep repo size down?

With iPython this is also an issue -- tracking code in JSON is much less clean than tracking code in text files.

It's interesting that Mathematica and iPython both left code-as-plain-text behind as a storage format. I wonder if it would have been possible to come up with a hybrid solution, i.e. retain plain-text code files but with a serialized data structure (JSON-like, or binary) as the glue.

<comment><USER23> I use Mathematica daily and frequently store large-ish notebooks in Git. The format is textual, but the diffs are filled with a lot of noise.

</comment></comment><comment><USER24> as a practical matter, papers will remain relevant as long as they are the metric by which grant applications and tenure decisions are made.

as a philosophical matter, for computation heavy fields, i would love to see literate programming tools become _de rigeur_ in the peer-reviewed distribution of results. In some fields (AI) this basically happens already — the blog post with code snippets and a link to arxiv at the end is a pretty common thing now.

<comment><USER25> Papers (and PDFs) are relevant because they are easy to organize and archive, essentially in perpetuity. Source code is too, so nothing wrong with a "Github for Science". Notebooks, blogs, or interactive dashboards, on the other hand, are an amazing tool both for research and for communication, but they are far more ephemeral than a paper. They need a large overhead to keep them running that cannot be sustained over decades or centuries. Typically, you'll have lots of trouble re-running a 5 year old notebook. That's not to say they're useless, e.g. as supplementary material (quite the opposite). They're just not going to replace papers anytime soon.

<comment><USER24> Those are good points. Journals have to care about this, too, these days -- supplementary information now routinely includes videos (hope you have the right codecs!), word documents, audio, and full or redacted data sets.

I think what appeals to me about literate programming style is that it encourages a return to a more clear and expository style of writing, which has been squeezed out of scientific writing in journals over the years. I don't care what instantiation is required to produce a more uniformly clear and cogent document, I just care that it happens.

</comment></comment></comment><comment><USER26> As is usual for Stephen Wolfram, he has a point, and then blunts it by trying to own the whole thing. Edit: to expand, part of the answer to his question, why don't more people do this, is that it requires his expensive proprietary software. Scientific papers are (nominally at least) a commons.

</comment><comment><USER27> Aside from my other comment, I think that any discussion about the scientific paper and the way knowledge is communicated is incomplete without a mention of Nick Sousanis' _Unflattening_. It is a thesis for a Doctor of Education degree about this very topic, that practises what it preaches by being written as a comic book.

[http://www.hup.harvard.edu/catalog.php?isbn=9780674744431](http://www.hup.harvard.edu/catalog.php?isbn=9780674744431)

</comment><comment><USER28> This is a really interesting article. The use of jupyter as a publication mechanism is a really neat idea! I think this path will be fruitful, and I am all for it. I do think however that some low-hanging fruit should be addressed in parallel - stuff that makes looking through the existing work a total pain:

* Date of publication and dates of research should be required in every paper. It's really difficult to trace out the research path if you start from google or random papers you find in various archive searches. Yes that info can be present but often its in metadata where the PDF is linked rather than the PDF itself. Even worse is the "pubname, vol, issue" info rather than a year that you get... now I have to track down when the publication started publishing, how they mark off volumes and so on. I just want to know when the thing was published.

* Software versions used - if you are telling me about kernel modules or plugins/interfaces to existing software, I need to know the version to make my stuff work. Again - eventually it can be tracked down, but running a 'git bisect' on some source tree to find out when the code listings will compile is not OK.

* actual permalinks to data, code, and other supplimental information. Some 3rd party escrow service is not a terrible idea even. I hate trying to track down something from a paper only to find the link is dead and the info is no longer available or has moved a several hour google journey away.

</comment><comment><USER29> shameless plug for the Popper Convention and CLI tool [http://github.com/systemslab/popper](http://github.com/systemslab/popper) . Our goal is to make writing papers as close as possible to writing software (DevOpsify academia) but in a domain-agnostic way.

</comment><comment><USER13> I love science, but I have a lot of issues with it lately. I'm going to express some of them since they're related to this topic.

The basic function of a scientific paper is understanding and reproducibility (inspired by jfaucett his comment).

I wonder, is reproducibility necessary? Is it even possible when things get really complex? Isn't consensus enough? I feel in the field of psychology (and most social sciences) that is what happens. I suppose consensus can be easily gamed by publication bias and a whole slew of other things. So I suppose as jfaucett puts it, a "discover for yourself" type of thing should still be there. I wonder how qualitative research could be saved and if you could call it science. In Dutch it is all called "wetenschap" and "weten" means to know.

But how should we go about design then? HCI papers use a _a lot of design_ of design that is never justified. The paper is like: we build a system, it improved our user metrics. But is there any intuition or theory written down as to _why_ they designed something a certain way? Not really.

I suppose one strong way to get reproducibility is by getting all the inputs needed. In a psychology study this means getting a dataset. Correlations are fuzzy but if I get the same answers out of the same dataset, then the claims must be true for that particular dataset.

Regarding design and qualitative studies, maybe, film _everything_? The general themes that everybody would agree upon watching _everything_ would be the reproducible part of it?

Ok, I'll stop. The whole idea of that a paper needs to satisfy the criterion of reproducibility confuses me when I look at what science is nowadays.

<comment><USER30> > I wonder, is reproducibility necessary? Is it even possible when things get > really complex? Isn't consensus enough?

If my results are not reproductible, then I'm basically asking for your trust. So now, instead of actually doing the experiment, there's an incentive for me to forge my results, And they don't have to match reality anymore, by the way.

Gotta go ; back to working on my paper about psychic powers.

</comment></comment><comment><USER31> Sometimes you don't need to redo everything from scratch to change things.

There are a number of problems in scientific publishing. Two big ones are:

1) Distribution hurdles and paywalls imposed by rent seeking journals - <USER2> knows how much this has prevented innovation and scientific advancement in the last 20 years

2) Easily replicating experiments / easily verifying accuracy and significance of results - this is related to for instance making data used in research more easily accessible and making it easier to spot p-value hacking

Fixing these might not require a completely new format for papers. Or it could. I can envision solutions both ways.

I really like what the folks from Fermat's Library have been doing. They have been developing tools that are actually useful at the present time and push us in the right direction. I use their arXiv chrome extension [https://fermatslibrary.com/librarian](https://fermatslibrary.com/librarian) all the time for extracting references and bibtex. At the same time they are playing with entirely new concepts - they just posted a neat article on medium about a new unit for academic publishing [https://medium.com/@fermatslibrary/a-new-unit-of-academic- pu...](https://medium.com/@fermatslibrary/a-new-unit-of-academic-publication- on-twitter-cdda2479091e)

</comment><comment><USER32> > His secret weapon was his embrace of the computer at a time when most > serious scientists thought computational work was beneath them.

They still think this.

</comment><comment><USER27> Interesting timing: for the last two years I have worked for a research group headed by Sten Linnarsson at the Karolinska Institute[0]. I was specifically hired to build a data browser for a new file format for storing the ever- growing datasets[1][2][3]. The viewer is an SPA specialised in exploring the data on the fly, doing as much as possible client side while minimising the amount of data being transferred, and staying as data-agnostic as possible.

Linnarsson's group just pre-published a paper cataloguing all cell types in the mouse brain, classifying them based on gene expression[4][5]. The whole reason that I was hired was as an "experiment" to see if there was a way to make the enormous amount of data behind it more accessible for quick explorations than raw dumps of data. The viewer uses a lot of recent (as well as slightly-less-recent-but-underused) browser technologies.

Instead of downloading the full data set (which is typically around 28k genes by N cells, where N is in the tens to hundreds of thousands), only the general metadata plus requested genes are downloaded in the form of compressed JSON arrays containing raw numbers or strings. The viewer converts them to Typed Arrays (yes, even with string arrays) and then renders nearly everything on the fly client-side. This also makes it possible to interactively tweak view settings[6]. Because the viewer makes almost no assupmtions of what the data represents, we recently re-used the scatterplot view to display individual cells in a tissue section[7].

Furthermore, this data is stored off-line through IndexedDB, so repeat viewings of the same dataset or specific genes within it does not require re- downloading the (meta)data. This minimises data transfer even further, and makes the whole thing a lot snappier (not to mention cheaper to host, which may matter if you're a small research group). The only reason it isn't completely offline-first is that using service workers is giving me weird interactions with react-router. Being the lone developer I have to prioritise other, more pressing bugs.

In the end however, the viewer is merely a complement to the full catalogue, which is set up with a DocuWiki[8]. No flashy bells and whistles there, but it _works_. For example, one can look up specific marker genes. it just uses a plugin to create a sortable table, which is established, stable technology that pretty much comes with the DocuWiki[9][10]. The taxonomy tree is a simple static SVG above it. Since the expression data is known client-side to generate the table dynamically, we only need a tiny bit of JavaScript to turn that into an expression heatmap underneath the taxonomy tree. Simple and very effective, and it probably even works in IE8, if not further back. Meanwhile, I got myself into an incredibly complicated mess writing a scatterplotter with low-level sprite rendering and blitting and hand-crafted memoisation to minimise redraws[11].

Personally, I think there isn't enough praise for the pragmatic DocuWiki approach. My contract ends next week. I intend to keep contributing to the viewer, working out the (way too many) rough edges and small bugs that remain, but it won't be full-time. I hope someone will be able to maintain and develop this further. I think the DocuWiki has a better chance of still being on-line and working ten years from now.

[0] [http://linnarssonlab.org/](http://linnarssonlab.org/)

[1] [http://loompy.org/](http://loompy.org/)

[2] [https://github.com/linnarsson-lab/loom- viewer](https://github.com/linnarsson-lab/loom-viewer)

[3] [http://loom.linnarssonlab.org/](http://loom.linnarssonlab.org/)

[4] [https://twitter.com/slinnarsson/status/981919808726892545](https://twitter.com/slinnarsson/status/981919808726892545)

[5] [https://www.biorxiv.org/content/early/2018/04/05/294918](https://www.biorxiv.org/content/early/2018/04/05/294918)

[6] [https://imgur.com/f6GpMZ1](https://imgur.com/f6GpMZ1)

[7] [http://loom.linnarssonlab.org/dataset/cells/osmFISH/osmFISH_...](http://loom.linnarssonlab.org/dataset/cells/osmFISH/osmFISH_SScortex_mouse_all_cells.loom/NrBEoXQGmAGHgEYq2kqi3IExagZjwBYI0R58oA7AVwBs6UMnt5FZ5gB2J1GADl6kYFavUZsWbDjACcQmOyFp42AKy9mq6Z3nw~SfSmFwCepsm0YZwcwaX7S0ZCTTgyAWlgA6Dq0REiABssIhqQeoo3viyRLJq_HEhsvhcatCURFDYTLIqCJZ4mIrZeIRkpvC0DBZS1pxGBo3CzRYmlNWMlEhZiOaIyPhsOZHYOEHQbKRAA), [https://i.imgur.com/a7Mjyuu.png](https://i.imgur.com/a7Mjyuu.png)

[8] [http://mousebrain.org/doku.php?id=start](http://mousebrain.org/doku.php?id=start)

[9] [http://mousebrain.org/doku.php?id=genes:aw551984](http://mousebrain.org/doku.php?id=genes:aw551984)

[10] [http://mousebrain.org/doku.php?id=genes:actb](http://mousebrain.org/doku.php?id=genes:actb)

[11] [https://github.com/linnarsson-lab/loom- viewer/blob/master/cl...](https://github.com/linnarsson-lab/loom- viewer/blob/master/client/plotters/scatterplot.js)

</comment><comment><USER33> This title is horrible hyperbole. Science is more than just machine learning. Hell, even if we just constrain ourselves to "computer science" probably half of it is just math, for which the scientific paper is definitely not "obsolete" nor even deficient in any way.

But outside of computer science you need laboratories to replicate experiments. Scientific papers are perfectly fine vehicles to record the necessary information to replicate experiments in this setting. Historically appendices are used for the extended details. And yes, replication is hard, but it's part of science.

<comment><USER34> What makes you think math papers wouldn't benefit from having interactive bits in the middle? Obviously isn't relevant for all math papers, but often would be extremely helpful. I read a lot of technical papers, and it is quite frequent that I will need to spend an hour or two decoding some formal mathematical statements whose basic idea/intuition could be more clearly conveyed pictorially in a few minutes.

Of course, making interactive diagrams often takes dramatically more work than sketching pictures with a pen (or just writing down equations), and mathematicians are not typically trained to do it, so it would be an uphill slog for many. But I would love it if there was more funding/prestige/etc. available for mathematicians to make their papers more accessible by adding better visuals.

</comment></comment><comment><USER35> This reads like an ad for Mathematica

<comment><USER36> Swipes like this break the site guidelines, including these important ones:

"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."

"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."

Would you please (re-)read [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html) and not post like this here?

</comment></comment><comment><USER37> Average clickbait

<comment><USER36> Maybe look a bit closer? I slog through "average clickbait" for hours a day in the hope of sparing this community of it, and can assure you that is far from the case.</comment></comment>


Adorer: Simple Twitter Marketing with Favouriting - <AUTHOR> http://adorer.co

<comment><USER2> Bad idea...Twitter hates auto-favoriting as they consider it spam.

I know several people that have had accounts suspended for doing this using other services (not this one).

The best way to grow your Twitter audience is via targeted following. Try something like TribeBoost instead.</comment>


The Duct Tape Programmer - <AUTHOR> https://www.joelonsoftware.com/2009/09/23/the-duct-tape-programmer/ <comment><USER2> I enjoyed reading this piece and really benefit from hearing this sort of practical wisdom.

> [Unit tests] sound great in principle. Given a leisurely development pace, > that's certainly the way to go. But when you're looking at, 'We've got to go > from zero to done in six weeks,' well, I can't do that unless I cut > something out. And what I'm going to cut out is the stuff that's not > absolutely critical. And unit tests are not critical. If there's no unit > test the customer isn't going to complain about that.

I hear this sentiment being shared a lot by self-described pragmatists. I don't doubt that this individual made the right choice at the right time given the circumstances, and I certainly can't claim to understand the unit test tools available to C programmers in the Netscape era. But I do know that there is data to strongly suggest that unit testing increases velocity. And I also know that if a module is cumbersome to test, it's almost always a code smell, not an indictment of unit testing.

Anecdotally, I feel way less anxiety shipping production code that is well- tested. That's important, because the root of merge-induced anxiety is uncertainty about the impact on the end user - I'm sure the customers care about _that_.

<comment><USER3> > And I also know that if a module is cumbersome to test, it's almost always a > code smell, not an indictment of unit testing.

I work with a disturbingly large number of exceptions.

What's the correctness criteria for, say, graphics? Is the scene rendered "right"? Accounting for differences in graphics cards, drivers, etc. leading to minor variations in the pixels? Large swaths of the scenes are going to render differently if I change shadow mapping techniques, or adjust the cap or prioritization criteria of shadow casters. Just capturing and returning device screenshots to the test harness is hideously platform specific.

How do I unit test hardware abstractions? I've encountered buggy gamepad abstractions across enough codebases I'm half tempted to set up a physical test rig somehow...

What about the deploy process for a Win8 .appx vs a Android .apk vs an iOS .ipa?

I wrote tests for APK deployment once. ADB kept lying to us and saying it'd deployed when it didn't, so I wrote a wrapper to catch it in the act and cry foul (by checking file metadata.) Tests of the wrapper required spinning up/down Android emulators, skipping it for machines where Hyper-V was disabled, and parsing logcat output to figure out when the emulator was alive enough to start interacting with. Did you know `adb logcat` has multiple text formats, and which one is the factory default can be device specific? There's a reason i started writing tests! We eventually figured out the root cause of ADB lying: loose USB connections. But those are sadly common, so the need for my wrapper remained.

It took me probably 2 weeks to get the wrapper into a solid state for Android project #2. For Android project #1, management wanted a port to iOS _and_ Android in the impossibly small timeline of two months. We had things kinda working by then, and I think we were mostly done with post-launch patching within three months or so - or at least I moved back off the project by then. We lost hours to debugging stale builds and wondering why our new fixes didn't fix anything, but that's still less time than I sunk into getting the wrapper working for project #2, so I wager that saved us time / let us fix more product-visible bugs on project #1.

Project #2 would later be canceled resulting in major layoffs. I doubt my wrapper ever paid for itself as a result - it simply didn't get used for long enough. Part of me wonders if I should've been more "pragmatic" and helped with more customer-facing stuff instead, if that would've prevented cancelation. (But we already had good people doing good work on that front, and I would've risked getting in their way, so I suspect making them be able to trust their deploy process was a good use of my time.)

> Anecdotally, I feel way less anxiety shipping production code that is well- > tested. That's important, because the root of merge-induced anxiety is > uncertainty about the impact on the end user - I'm sure the customers care > about _that_.

While I share your anxiety over poorly-tested code, manual testing and good QA can cover a multitude of sins, and the end user for a lot of stuff can be your fellow developers - not your customer.

<comment><USER2> Your points are well-taken, but some people tend to extrapolate the testing issues with specific cases like the ones you're describing and apply that same ethos to projects that can and should be unit tested, such as web applications. Manual testing and QA are great but cannot provide the sort of architectural guidance that unit testing provides. If a module is difficult to unit test, it will also be difficult to maintain. Manual testing will not readily provide those insights.

</comment></comment></comment><comment><USER4> Its from 2009 but still valid. Keep it simple and choose your tools wisely. Most of the time,the company yoiu work for is nothing like google, so don't try to use all their fancy and shinny tools. For a lot of websites, php, jquery is more then sufficient.</comment>