Estimating Number of Jupyter Notebooks on Github - <AUTHOR> https://kyso.io/KyleOS/nbestimate <comment><USER3> In the same spirit as "Effective Java" and "Effective C++" we need to have a book entitled "Effective Jupyter Notebooks". Here are some of my items below. Maybe this sub-thread can come up with an outline for this book.

Item #1 Writing a notebook is foremost an exercise in expository writing. Make sure the writing is high quality is the first objective when writing a notebook. This is the Knuth's literate programming idea where prose takes precedence over code, which is usually the reverse of the way we usually program; code first, comments second.

Item #2 Don't use notebooks for general purpose programming. Notebooks are supposed to have an audience and clearly explain something.

Item #3 Keep code cells simple and clear. If needing a comment in the code block, consider putting that verbiage in a markdown cell instead and elaborating on the idea the notebook is trying to convey.

Item #4 Don't make notebooks a long series of extended code cells, or even worse, just one long cell. Explain what is going on or see Item #2.

<comment><USER4> I disagree.

While your points are valid for presentations, I believe that notebooks should first and foremost be used for exploratory computing.

Notebooks are my goto tool for whenever I need to do something with a computer and any of the following apply:

* I'm not quite sure what or how.

* It will likely be a one-off

* I need it _now_.

* Someone is watching me, to learn how I do it.

I would go a step further and say that notebooks should probably not be shared directly most of the time, and that you should wrap up functionality into modules/packages for other people to use in their own notebooks.

Edit: Just thought of a corny pun to make my point: They are notebooks, not textbooks. Notebooks are personal, and while it may be useful/insightful to compare notes, it's not the primary function of a notebook.

<comment><USER5> I agree with with your disagreement. I've always approached notebooks as a prototyping tool first and a presentation tool second. I see Jupyter more as an example of an early Interactive programming tool that just happens to be useful for presentation and teaching purposes. ([https://en.wikipedia.org/wiki/Interactive_programming](https://en.wikipedia.org/wiki/Interactive_programming))

And regarding sharing -- particularly with notebooks you should be really cautious about sharing and opening them. The security model is not exactly bullet proof.

<comment><USER6> > _I see Jupyter more as an example of an early Interactive programming tool_

Nitpick, but I hope you meant "young", not "early". Interactive programming is as old as (and today still primarily featured in) Lisp - i.e. twice as old as most of us here on HN.

</comment></comment></comment></comment><comment><USER7> If you ever put notebooks in source control, you owe it to yourself to try the text-based notebooks supported in Visual Studio Code[1]. They're round- trippable with real (i.e. browser-based) notebooks, yet are much better for collaboration, diffing, and editing.

[1] [https://code.visualstudio.com/docs/python/jupyter- support](https://code.visualstudio.com/docs/python/jupyter-support)

<comment><USER8> How does this compare to Jupytext?

I prefer pipenv to Conda, and I don't like having Jupyter(Lab) installed in each venv separately, so instead I only add `Ipykernel` to each venv and then use my system-level JupyterLab to access per-project kernels; seems like that wouldn't work here?

<comment><USER9> Exactly. Conda is the worst of Python world - installing gigabytes of unnecessary garbage every time.

<comment><USER10> But the purpose of Anaconda is to have a easily installable set of frequently used tool for a variety of data science tasks. It's not meant to be a minimalist package.

</comment></comment></comment></comment><comment><USER11> Why doesn't this just use the GitHub public dataset available on Google BigQuery to have much more accurate data rather than "scraping GitHub web search results"? [https://cloud.google.com/bigquery/public- data/](https://cloud.google.com/bigquery/public-data/)

There are a lot of examples of people analyzing public code on GitHub efficiently for patterns and usages with BigQuery and getting pretty accurate data out of it. [https://medium.com/google-cloud/analyzing-go-code-with- bigqu...](https://medium.com/google-cloud/analyzing-go-code-with- bigquery-485c70c3b451)

If you use GitHub on a daily basis, you are unlucky enough to know that web search sadly can't even find words that exist in your repository.

<comment><USER12> One limitation of the BigQuery dataset is they only look at repos with a license on them[1], the scraping approach can look at all public repos.

[1] [https://github.blog/2017-01-19-github-data-ready-for-you- to-...](https://github.blog/2017-01-19-github-data-ready-for-you-to-explore- with-bigquery/)

</comment></comment><comment><USER13> Off topic:

Possibly something in my config, but I've recently got a lot of

        "Sorry, something went wrong. Reload?"

when trying to view Jupyter notebooks on github itself. Seems to be working right now.

I have used this as an alternate: [https://nbviewer.jupyter.org/](https://nbviewer.jupyter.org/)

<comment><AUTHOR> This is what we made Kyso for - the linked post is actually a Jupyter notebook itself, the code is hidden by default to make it readable to non-technical people but you can click on the "code hidden" button on the top right to see the code in full.

If github is not working for you well for your notebooks. You can try Kyso by signing up and importing your notebook from Github directly on this page: [https://kyso.io/github](https://kyso.io/github), or upload it using this page: [https://kyso.io/create/study](https://kyso.io/create/study)

Disclaimer - I am OP and founder of Kyso

<comment><USER14> You made Kyso for making GitHub-hosted Jupyter Notebooks render when accessed with a web browser on github.com?

<comment><USER15> Well, it's one of the ways to post to Kyso, but yeah, you can synchronize your Github repositories to Kyso and choose the notebooks you want to have rendered. When you push changes to the repo, the sister post on Kyso will be automatically updated.

</comment></comment></comment></comment><comment><USER16> If only more people would use org-babel...

If you're on emacs and like Jupyter, there's [https://github.com/dzop/emacs- jupyter](https://github.com/dzop/emacs-jupyter) , which is pretty nice. I've been using it for a few days with Julia, and it works really nice. It also allows you to use different kernels from the same org-mode file, though I haven't tried to pass data between them yet (should be possible, though, at least it works in plain org-mode).

<comment><USER17> I agree. emacs-jupyter is great but I'm eagerly waiting for jupyter notebook server support since my work is on remote clusters now. For local use, it's pretty great.

</comment></comment><comment><USER18> We use notebooks heavily for onboarding devs & data scientists to Graphistry, and I only see that increasing.

Interestingly, for initial use, we increasingly start teams on their existing internal NB servers, and for new ones, they either start on Jupyter included in their Graphistry AMI or use Google Colab. So, very little outside of our quick start notebook skeletons hits GitHub.

So... How many notebooks are actually out there? Probably an even more interesting growth curve...!

<comment><AUTHOR> How do you share and collaborate on the notebooks internally - I'd love to get your thoughts on our Kyso for teams system [1] if you would be willing to chat?

[1] [https://kyso.io/for-teams](https://kyso.io/for-teams)

<comment><USER18> Google Colab has solved the 90% for us. I wish it'd have context sharing across users and better default folder management, and probably other things, but free + usable + sharing has been amazing in + across partners.

Feel free to ping in a ~couple weeks, happy to chat.

</comment></comment></comment><comment><AUTHOR> OP and Founder of Kyso here - we built Kyso to make it easier to blog your notebooks to the public and also to make them easier to share in teams.

The linked post is actually a Jupyter notebook itself - analysing the number of notebooks on Github.

A key element with Kyso is that the code is hidden by default to make it readable to non-technical people but you can click on the "code hidden" button on the top right to see the code in full.

If you want to give Kyso a go - sign up and import from Github directly on this page: [https://kyso.io/github](https://kyso.io/github), or upload using this page: [https://kyso.io/create/study](https://kyso.io/create/study)

<comment><USER19> In my opinion hiding code is an anti feature as exposing the code by default gives you more incentive to write clean understandable and explaining (self documenting) code as it's always visible. By hiding it people might be more likely to paste in big blobs ugly code that would be much better put into a reusable function then in a notebook snippet.

<comment><AUTHOR> Thats true if you sharing with someone who also understands the code - <USER2> think that the feature allows you to share the notebooks with a completely non-technical audience.

Image you had a notebook to analyse sales data and you needed to present the results to your CEO (who perhaps cannot code) - this feature lets you present the notebook as is, without needing to prepare a report in some other format

<comment><USER20> Why would you share a _notebook_ with somebody who doesn't understand the code?? What are they supposed to _do_ with them anyways? Reading? Isn't that what PDF is for? Just give them PDFs, C*O people have got piles of other things to worry about besides shared notebooks.

<comment><AUTHOR> Mostly so you don't need to convert to PDF and so that you can host the reports in a central place where everyone can read them technical or not.

Like an internal wiki for a companies data-science where the technical people can communicate their work to the non-technical people with a pretty seamless experience

<comment><USER20> This is not the experience non-technical senior leadership people are looking for, unless you are a 10-people startup.

<comment><AUTHOR> Im not sure - we have large teams using Kyso as a knowledge base for data- science work and there's also Airbnb's knowledge-repo which originally inspired us so from my point of view there is decent evidence for the need for this

<comment><USER20> Ah, that makes sense. So, basically you replaced a dashboard effort with a whole bunch of readonly notebooks, thus distributing the information delivery job among the peers outside of your DS team. Clever.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER21> We're seeing an explosion of Jupyter use as well on GitLab. GitLab already makes Jupyter easier to install on a Kubernetes cluster [https://docs.gitlab.com/ee/user/project/clusters/#installing...](https://docs.gitlab.com/ee/user/project/clusters/#installing- applications) In response to the growing demand we're doing two things:

1\. Adding better Jupyter support to GitLab 12.0 [https://gitlab.com/gitlab- org/gitlab-ce/issues/47138](https://gitlab.com/gitlab-org/gitlab- ce/issues/47138) as suggested by my co-founder.

2\. Making it easier to do the entire data lifecycle with Meltano [https://meltano.com/](https://meltano.com/) which plans to include JupyterHub

</comment><comment><USER22> Python Notebooks will the be the PERL of 2010s - write once, pretty impossible to maintain long term

</comment><comment><USER23> Very cool work here. This is a pretty epic post, so please do not take this the wrong way.

I was under the impression that FB Prophet was optimal for significantly seasonal time series data.

Honestly given the fickle nature of these kind of growth patterns beyond the very near term, an ARIMA with a flat vol or a simple eyeball extrapolation in my experience as a quant would likely generate just as reasonable/reliable results.

While I understand this is likely intended as a standalone project, it would be interesting to run a comparison of ARIMA vs FB Prophet on out of sample trending Github tools/file types, as well as the general performance of these predictions beyond a one year time frame (especially vs the reported confidence intervals in Prophet).

I am not that familiar with how Prophet works, so I am absolutely open to being humbled and corrected. I have a project myself that has a varying seasonal component and I am looking forward to diving into Prophet for a deeper understanding. I am attempting to model an Asian 2 asset spread option with a volume weighted average index price setting mechanism where the underlying exhibits seasonality in the volume traded over the trading time window. I am currently running a Monte Carlo on the valuation with a simple average settlement assumption, as opposed to a volume weighted average assumption, and I was thinking Prophet could help.

Does anyone have experience in financial time series analysis and option valuation who would care to chime in?

Also, what is everyone's thoughts on using prophet non seasonal vol clustering times series?

<comment><USER15> Hey, I posted the notebook by the OP. Thank you for your feedback! You're correct in saying that FB Prophet is for forecasting time series with strong seasonal effects. FB Prophet was the model used in the original script I found & the main point here was simply to make the notebook more readable on kyso, which has quite a few non-technical readers. I've worked a lot with ARIMAs before for financial/economic data and I like the idea of comparing the results between the two, and maybe even extend the time frame. So 1. I think that'll be my next project and 2. if your project is public I'd love to give it a read when published.

<comment><USER24> Did you try running the forecast with a log ceiling to control for the trajectory a bit? Or would that only be a concern of yours if you had to forecast past a couple of years? I find that when I use Prophet to forecast down to the day I end up creating initial forecasts with heavy log ceilings to prevent unreasonable estimates of the future and then end up removing the ceiling once enough history is established to provide a resaonable baseline.

</comment></comment></comment><comment><USER25> My two cents: We've been recently working in a FREE hosted version of Jupyter Lab mainly intended for education. Feel free to check it out.

[https://notebooks.ai/](https://notebooks.ai/)

Would love to hear some feedback.

<comment><USER26> Wondering how you are planning to keep it free. Also wondering whether you would possibly consider shifting to Sagemath/CoCalc as a service.

<comment><USER25> We got support of the local university at my city and we got a bunch of free credits at AWS. Costs are very low and we want to keep it that way so we can support the most students we can with a free access.

<comment><USER27> What happens when your funding and AWS credits run out?

<comment><USER25> We might add bigger paid tiers later if we decide to support business usage of it. For now it's only educational and we can deal with the costs, even without the credits. Containers used are small and get shut down on inactivity. So, we only need to care about concurrent users. Hope it makes sense.

<comment><USER28> > Containers used are small and get shut down on inactivity.

How do you define inactivity? If I do

$ nohup ./computational_intense_and_runs_for_100_hours.py &

Do you just kill the process (or stop the container)? In essence Jupyter is a graphical rich shell, so you providing free *nix machines - don't underestimate how this feature can be exploited (e.g. CoCalc limits at least internet access for free instances).

<comment><USER25> First, that will use 100% of the CPU quota assigned to your user, which is really small.

Second, yes. The container will be killed after 10min unless we keep detecting activity of your user in the platform. So, basically the rule is: If we don't detect user's activity after 10mins we kill all containers for that user. You could hack this by doing periodical requests to the API to simulate activity, but at some point your JWT will be expired and requests will start failing.

In any case, other students won't be affected at all by the appropriate usage and we will end up banning your account at some point when we detect it.

We also limit the amount of parallel running containers to avoid unlimited containers running at the same time.

Do you see any drawbacks on this implementation? Happy to hear about possible improvements.

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> I think it would be cool to run the same analysis on the number of R Notebooks on Github and compare the two.

<comment><USER29> Aren't Jupyter notebooks R notebooks? Jupyter stands for "Julia, Python, R" I believe

<comment><USER15> I think the file extension scraped on Github is only .ipynb, which is only python notebooks right?

<comment><USER30> I think most people save under this extension even if they are using a different kernel (i.e. they are running R, Julia, Matlab, etc. code in the notebook).

<comment><AUTHOR> Yeah so its mostly a split between Jupyter and R Studio - but Jupyter can mean different languages

</comment></comment></comment></comment></comment><comment><USER31> On the topic of Jupyter Notebooks, Is there something similar to a paid version of Google's CoLab? CoLab is so awesome for creating prototypes and even better since it's free. However, there is no paid alternative that I have seen. I do not want to have to deal with setting up my own VM or server. The way that CoLab is perfect for what I need.

<comment><AUTHOR> The OP post is a Jupyter notebook itself and if you sign up to Kyso you can actually Jupyterlab on our cloud and the post the notebooks to the web, or make them private on the paid plan - is that what your looking for?

<comment><USER31> Yeah but unfortunately there is no GPU support. I wish there was!

<comment><AUTHOR> Ah yeah ok - we are not planning GPU support soon - what if we created one of those one-click deploy a VM to aws/digital-ocean buttons and from there if you wanted to post to Kyso you could do it with git or our jupyter lab plugin.

You get most of the same experience and you can even customise various of the steps?

<comment><USER31> Interesting! That would be super awesome. Like a one-click deploy with auto- shutdown after the end of run to save money. I would definitely pay for a service like that!

</comment></comment></comment></comment></comment><comment><USER32> So I just learned they're not laptops.

<comment><USER33> This comment made my day!

</comment></comment><comment><USER34> There's also the GitHub extracts table available in BigQuery which allows analysis of the contents of the notebooks themselves: [https://bigquery.cloud.google.com/table/fh- bigquery:github_e...](https://bigquery.cloud.google.com/table/fh- bigquery:github_extracts.contents_ipynb?pli=1)

</comment><comment><USER35> We built a runnable jupyter notebook website. Would someone be able to take a look and give us some feedback?

[https://datacabinet.systems](https://datacabinet.systems)

We are VM based for now but are moving to be kubernetes based to make sharing better. Our initial market is classrooms.

<comment><USER36> Something to be aware of (and a general comment about k8s in general) is that k8s is not suitable for use in hostile multi-tenant scenarios like the one that you're describing. Once an attacker escapes from the container (see HN archives for lots of examples of this), they can p0wn the entire cluster. Jessie Frazelle has a great post on this: [https://blog.jessfraz.com/post/hard-multi-tenancy-in- kuberne...](https://blog.jessfraz.com/post/hard-multi-tenancy-in-kubernetes/)

There are expensive ways to deal with this today, e.g., running each user isolated in a separate VM. Hopefully we will have better solutions in the near future.

<comment><USER35> We were starting to work on disabling kubernetes cluster access. We will try the steps in the post.

</comment></comment></comment><comment><USER37> Especially odd because they are so unsuitable for use with git. Someone needs to find a way to fix this.

<comment><USER38> > because they are so unsuitable for use with git

Can you going into a little more depth about this statement?

<comment><USER39> Diffs primarily I'm guessing - a, it's kinda hard to parse the jsons that you see when you look at a notebook in raw text b, every time I execute a cell, it shows up in the diff as a change. That being said, there are plugins and tools that deal with these issues quite well. check out [https://nbdime.readthedocs.io/en/latest/](https://nbdime.readthedocs.io/en/latest/)

<comment><USER37> I have not had good luck with nbdiff. Minutes-long runtimes and huge memory consumption on fairly standard ipynb's.

</comment></comment></comment></comment><comment><USER40> Isn't the prediction too low? My (unsupported) prediction fitting a smooth curve in the graphic is [https://imgur.com/a/ykeIxPm](https://imgur.com/a/ykeIxPm)

</comment><comment><USER41> This seems like an incredibly complicated way to fit an exponential to data...

</comment><comment><USER42> Maybe it's time to be able to run them implicitly on Azure cloud?

</comment><comment><USER43> nice marketing, kyso.io team

</comment><comment><USER44> I hate jupyter notebooks. Joel Grus puts it perfectly: [https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUh...](https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUhkKGvjtV- dkAIsUXP-AL4ffI/edit#slide=id.g362da58057_0_1)

past hn discussion: [https://news.ycombinator.com/item?id=17856700](https://news.ycombinator.com/item?id=17856700)

<comment><USER36> What specifically do you hate about Jupyter? Is it out of order execution exacerbating the "hidden state" problem? If so, and if you already use VS Code, I encourage you to try out our Python VS Code extension. We have an "Interactive Python Window" mode [https://code.visualstudio.com/docs/python/jupyter- support](https://code.visualstudio.com/docs/python/jupyter-support) that we showed a lot of folks at pycon last week and even among the "I don't like Jupyter" crowd, it was quite well received. The key thing about our experience is that it is an editor focused interactive programming experience vs. trying to just replicate Jupyter functionality in an editor (though we are also doing work here because we believe that folks want an experience where they can move seamlessly back and forth between and editor and a notebook experience).

Disclaimer: I designed this experience in VS Code.

<comment><USER45> Please allow me to thank you for designing an experience that hits the sweet spot between maintaining a good history of work through git while allowing for interactivity and ease of exploration. I started using VS Code on seeing a video of the jupyter support within the latest release of the Python extension. This has eased and sped up my work tremendously!

<comment><USER36> Thanks for the kind words! If you find things that you would like to see improved, please do open an issue on our Github - [https://github.com/Microsoft/vscode- python/issues](https://github.com/Microsoft/vscode-python/issues).

We also need to work in the discoverability of this feature too. Lots of existing users of our extension had no idea it was there ... suggestions welcome!

</comment></comment></comment></comment><comment><USER46> exponentially?

<comment><USER47> Yeah? [https://imgur.com/a/J93nCXR](https://imgur.com/a/J93nCXR)</comment></comment>


My thoughts on MessagePack - <AUTHOR> https://gist.github.com/2908191 <comment><USER2> cheald has an excellent comment regarding MessagePack, JSON, and Protocol Buffers in the post from 15 hours ago: <http://news.ycombinator.com/item?id=4091051>

       MessagePack is not smaller than gzip'd JSON
       MessagePack is not faster than JSON when a browser is involved

Other comments from that post include:

       MessagePack is not human readable
       MessagePack has issues with UTF-8
       Small packets also have significant TCP/IP overhead

Really, anyone who hasn't read the other comments should: <http://news.ycombinator.com/item?id=4090831>

<comment><AUTHOR> Although the original blog post focuses on JavaScript and browsers, MessagePack itself doesn't mainly focus on them.

A major use case of MessagePack is to store serialized objects in memcached. A blog post written by Pinterest describes this use case (<http://engineering.pinterest.com/posts/2012/memcache-games/>). They use MessagePack with Python which is faster than one with JavaScript. They could store more objects in a server without performance declination (e.g. gzip).

It's true that MessagePack is not always faster than JSON (e.g. within browsers), and it's not always smaller than other serialization methods (e.g. with gzip compression). So we should consider that which serialization methods should I use for "my" case.

There are also general tendency which is helpful to select MessagePack or JSON:

        MessagePack is faster to serialize binary data such as thumbnail images.
        MessagePack is better to reduce overheads to exchange small objects between servers.
        JSON is better to use it with browsers.

<comment><USER3> > They could store more objects in a server without performance declination > (e.g. gzip).

The performance declination argument is bullshit. Network's a million [0] times slower than gzip.

Truth be told, once you're on the network, you're already screwed w.r.t. most serialization. The only thing efficient compression/decompression is going to buy you is lower CPU (memcached servers run at like 2% CPU util, even under heavy load [1]).

Memcache at Facebook actually uses the ascii protocol, and the memcached implementation is a braindead strtok parser (some of our other stuff uses ragel -- you'll have a hell of a time out optimizing ragel with the right compiler flags -- I've tried and failed).

Just use whichever serialization format has the best API, because I can say with near certainty that it's not going to be a perf problem for you if you're touching disk, network, etc.

[0] Obviously a made up number, but it's way slower. Especially if you're unlucky and lose a packet or something.

[1] With the exception of weird kernel spin lock contention issues, which can happen if you're not sharding your UDP packets well and trying to reply from 8+ cores on 1 UDP socket. You probably aren't.

</comment></comment></comment><comment><USER4> Zerorpc (<http://github.com/dotcloud/zerorpc-python>) uses msgpack as its primary serialization format. Among other things it is significantly more efficient for floating point data. At dotCloud we use it to move many millions of system metrics per day, so it adds up. Also worth noting: msgpack maps perfectly to json, so there's nothing keeping you from translating it to json on its way to the browser, where json is indeed a better fit. In practice this doesn't affect performance since you typically need to parse and alter the message at the browser boundary anyway, for some sort of access control.

</comment><comment><AUTHOR> We're using MessagePack in a Rails application to log user behaviors and analyze them.

Compared with other serialization libraries such as Protocol Buffers, Avro or BSON, one of the advantages of MessagePack is compatibility with JSON. (In spite of its name, BSON has special types which cause incompatibility with JSON)

It means we can exchange objects sent from browsers (in JSON format) between servers written in different languages without losing information.

I will not use MessagePack with browsers but it's still useful to use it with web applications.

<comment><USER5> If JSON compatibility is an issue, have you looked at UBJSON? <http://ubjson.org/>

May be a bit bigger than msgpack but is damn-near human readable even in its binary format and really easy to encode/decode. Also 1:1 compatibility with JSON.

Compatibility and simplicity were the core design tenantes. It may _not_ be the right choice, just throwing it out there incase it helps.

Disclaimer: I am the author of the spec.

<comment><USER6> I looked at it. It's design process is not completed. One strong negative point is that it enforces big endian integer encoding.

Another one is that it doesn't use the value space of tags as efficiently as message pack. I would use the unused space to encode small string size in the tag since objects (associative arrays) have generally many short identifier strings as keys.

I sent these as comments and requests for change but didn't receive any response yet. I don't know how open its design process is.

</comment></comment></comment><comment><USER7> As much as we would like to jump back on the MessagePack is no JSON alternative here, I would like to commend The author on taking the criticism posted earlier like a mature adult and explaining his point of view. Even admitting that some of the benchmarks might have been misleading.

</comment><comment><USER8> I've personally been using BSON[1] as a binary alternative to JSON, and it's worked out great. I've written an Objective-C wrapper around the C lib, in case anybody's interested. Every other language has a solid implementation from 10gen (the MongoDB guys). It's a solid format with a clear spec that's extensible and fully supports UTF-8.

[1]: <http://bsonspec.org/>

<comment><USER9> I am kind of partial to tnetstring passed through lzf right now.

</comment></comment><comment><USER10> I think it's hilarious that 2 of the 3 comments on the gist are about how he formatted his Markdown. _sigh_

I think MessagePack is great. Good for him for trying to make something better. He openly admits it's not better all the time. Why not help?

</comment><comment><USER11> Very interesting discussion. I work on a 2D MMORPG for Android. This is extremely relevant to me. I have a few questions though.

What if you take compression and deserialization out of the picture? For example, in my server I have a hash like data structure that gets turned into JSON for browsers and byte array for mobile clients.

For example, because the data has to be transferred at fast rates and will be going over mobile networks. The size of the packet matters because every millisecond counts.

Then to read the data, I simply read the stream of bytes and build the objects I need on the client. This has to happen mostly without allocations for example on Android to avoid the GC.

So a few questions: Does deserializing JSON cause any memory allocations? If you're not tokenizing the data and don't need to parse it, will it be a significant gain over s serialized byte protocol or JSON?

In any case, I'll experiment on my end and perhaps blog about my own findings.

<comment><AUTHOR> Disclaimer: I'm authoer of MessagePack for C++/Ruby and committer of one for Java.

As for strings, JSON has to allocate memory and copy to deserialize strings because strings are escaped.

MessagePack does't have to allocate/copy because the serialized format of strings is same as the format in memory. But it depends on the implementation whether actually it doesn't allocate/copy.

C++ and Ruby implementations try to suppress allocation and copying (zero- copy). But Java implementation doesn't support zero-copy feature so far (we have plan to do so. Here is "TODO" comment: [https://github.com/msgpack/msgpack- java/blob/master/src/main...](https://github.com/msgpack/msgpack- java/blob/master/src/main/java/org/msgpack/unpacker/ByteArrayAccept.java)).

As for the other types, C++ implementation (and new Ruby implementation which is under development) has memory pool which optimizes those memory allocations petterns. But it's hard to implement such optimizations for Java because JVM (and Dalvik VM) doesn't allow to hook object allocation.

<comment><USER11> Interesting, thanks so much for the response. I'll keep this in mind as I continue to develop my app. It's still between custom byte stream, json, thrift, etc.

But MsgPack looks interesting as well and, if anything, these blog posts have brought it into the light for me.

I looked at the java class and what _might_ help is if you can set a buffer size and use that buffer to store the data in the buffer and expand it if necessary. But that seems like a lot of work. But yeah, not sure if you can optimize based on usage patterns due to the constraint you said. In any case, great stuff and thanks for the info.

</comment></comment></comment><comment><USER12> The best way of serialization at present. I like the thought, performance and various implementations.

<comment><USER13> As nice as Message Pack might be, 'The best way of serialization' I'm not sure is a very helpful statement. It can't be 'the best' because 'the best' depends on the specifics of what you are doing. As noted in the OP: "...its pros and cons should be carefully considered, and there are many situations where it simply does not offer enough advantage...".

I, for instance, am still using the much-less-cool yaml, because I need to reference the same object at multiple points within the same serialization. JSON and (AFAIK) msgpack just dont do that, so in this case there is simply no argument. It took me far too much playing around to figure this out, because the internet is full of "JSON > yaml" and similar broad statements, and very few plain descriptions of what the actual different use cases for each type of serialization might be.

<comment><USER14> Isn't JSON a subset of YAML? So isn't it, quite literally, that YAML > JSON?

<comment><USER15> Only if you don't consider having a totally compliant implementations for pretty much every platform a feature. YAML is a big hairy beast.</comment></comment></comment></comment>


Oxford Study Shows FLAGRANT Liberal Bias Among Scientists - <AUTHOR> https://www.campusreform.org/?ID=11193 <comment><USER2> > "What an odd coincidence that 'science' always, without exception, supports > the liberal worldview," Confas observed.

I'll buy that there may be some bias, but this is just incorrect, and this article is garbage.

</comment><comment><AUTHOR> "Conservatives are right to be skeptical," he added. "Take any politicized issue that is connected to some disagreement about scientific fact. I do not believe there is a single case in the last couple decades where a major scientific organization took a position that went against the platform of the Democratic Party."

"What an odd coincidence that 'science' always, without exception, supports the liberal worldview," Confas observed.

<comment><USER3> Liberals can't handle many areas of science, including but not limited to: evolutionary psychology, any study involving standardized testing or intelligence, vaccines (in certain circles), some important parts of genetics and economics, the sociological effectiveness of interventionism, military technology. In most cases, smart liberals and smart conservatives are both right (in the areas they value) and dumb liberals and dumb conservatives are both wrong.</comment></comment>


Nobody Hates Software More Than Software Developers - <AUTHOR> http://www.codinghorror.com/blog/archives/001289.html

<comment><USER2> I completely agree that pretty much every proprietary software product that has come bundled with a device I purchased has been useless crap. There's probably one or two exceptions but I can't think of them.

I didn't find the rest of the article that interesting, but he definitely nailed his first point.

It's not so much bad programming, it's more a case of why? Why do I want to install something that by default will chew up memory by running in the background (and default to launch on startup), that I will never use, because I have so many better tools for the job.

I guess the argument is that it's useful for non power-users... Umm sure. How many of you have been asked to fix something on a relative or friends windows box only to find it grindingly slow because of all the run-time resident crap running on it? Like 10 things in the system tray? Update notifications launching one after another on boot? My first task is usually removing all of this stuff they NEVER use before even tackling the real problem they're having.

</comment><comment><USER3> You should consider your code to suck over time, if you're "doing it right".

If you keep learning new and better techniques and practices, old code starts to look immature. If you code in the real world, you make compromises to get a project launched before you're living in an alleyway.

TicketStumbler may be a steaming pile of crap in some places and my deployment strategy may have devolved to "install random packages until it stops throwing internal errors", but it was not always this way. Not only have I managed to become an even more amazing programmer in the past year (against all odds), but I've had to cut so many corners the whole thing is beginning to look like a circle. These two factors, above all others, have caused me to rather loathe my code. But that doesn't mean I always do.

Maybe this was the point Jeff was attempting to make? If I am only good by hating _all_ my code, I don't see how I could ever _be_ good.

</comment><comment><USER4> I'm a little bit nervous to say this, but this is another of Jeff's articles where I think "man, so many of this guy's problems would go away if he didn't hate Macs so much".

In my experience, Mac devs _don't_ hate software, and Mac users don't have the instinctive "oh [App X I haven't heard of]? That's going to suck".

Gruber talked more about this in his Broken Windows essay -- <http://daringfireball.net/2004/06/broken_windows> \-- and what he said about security is generally true for quality and polish as well. If I get a new app for the Mac, I know that it's going to have basic standards of fit and finish and will make a reasonable attempt at being a good Mac citizen, because otherwise the community will kill it.

When a new Mac app comes out, I'm usually fairly optimistic about what it will do. When a new Windows one appears, I'm sceptical.

This doesn't extend all the way to the quality of coding admittedly, but certainly at least until the iPhone came out Cocoa devs were a fairly select set of committed people, and the higher barrier to entry helped keep at least some of the crappiest C&P coders out of the platform.

<comment><USER5> Not even having a Mac saves you from the crapware hardware makers bundle with their crap. You have no idea how annoying it was for me to have to remove shitty Kodak photo management software from my parents' Mac (which had a perfectly good version of iPhoto) or the horrific, kernel-panic-causing HP All-In-One software. Or even the shitty Verison DSL setup CD I was supposed to use to set up my DSL modem.

</comment></comment><comment><USER6> Awareness of your own limits, and the likely quality of your work, is a good thing.

But this descends into self-flagellation. _Waaaah, I suck, you suck, we all suck. We're all the equal-worst programmers EVER._

If you're doing research into programming languages, and you're explaining your work, or asking for funding, fine.

But coming from practicing programmers, it's a common, trite, pointless and irritating refrain containing zero useful information.

</comment><comment><USER7> _In fact, I think you can tell a competent software developer from an incompetent one with a single interview question:

What's the worst code you've seen recently?

If their answer isn't immediately and without any hesitation these two words:

My own.

Then you should end the interview immediately. Sorry, pal._

It seems to me that few programmers would answer the question this way, competent or not. And it's probably not true for most competent programmers... I wonder if Jeff has actually received this answer from a interviewee.

<comment><USER8> I know I've said it in a number of interviews and have gotten offers from all but one of them. I don't know if I'd recommend it unless you can follow it up with a strong explanation of _why_ it was the worst -- and how you fixed it -- but it isn't an interview killer, IMO.

I agree with Jeff on this one. Some of my worst experiences with code has been going back over stuff that I wrote 3-5 years ago. I'm pretty hard on crappy developers, so I'm doubly hard on myself when I look back at some of the boneheaded mistakes and decisions I've made.

</comment></comment><comment><USER9> I think this article makes some good points, and I found myself nodding a lot.

On the other hand, it (along with some of the HN comments) conflates a couple of unrelated issues: (1) how well software is written, and (2) whether software forces me to do things I don't want to do.

One of the parts I nodded along with, was his story of lunging for the computer when the camera software CD was inserted. But this doesn't have anything to do with whether the source of this software was readable, variable names, etc., nor is it related to the competence of the programmers or whether they are aware of their own failings. The issue is that such software is often written with a view toward taking over the way I do things.

I want to do things _my_ _way_. That's the reason for my move from Windows to Linux, and it's why I use Firefox instead of IE when I do use Windows. It's why I avoid Flash when feasible, why I hate most children's educational software, and why I would not even glance at the software that came with a camera. Does this have anything to do with the quality of the code involved in these programs? I don't think so.

</comment><comment><USER10> I remember that scene from "War of the Roses" where the Danny DeVito character tells the horror story of the Roses to talk a new client out of engaging in an ugly and bitter courtroom battle. I think his line was (paraphrasing here) "when a man who could charge you $250 an hour wants to tell you something for free, you should listen."

I often feel that way about software. Remember the stages of a software project?

1) wild enthusiasm 2) profound disillusionment 3) search for the guilty 4) punishment of the innocent 5) rewards and accolades for the non-participants

Can't remember exactly where I read that, but software developers have experienced them all, over and over.

I don't necessarily try to talk people _out_ of a new development effort. After all, there are clearly times when we need to write software, and some projects are smashing successes. But I kind of feel like that breed of lawyer who tries to get clients to see litigation as a true last resort.

I say - is there anything already out there that could solve your problems without a new development effort? How far would a much simpler approach go toward solving your problems - could you live with it? Basically, I want clients to understand just how risky a full blown development effort truly is.

Of course, if you need business... well, nothing like an angry divorce to keep the billable hours up, and nothing like a flailing software project funded by deep pockets to keep the cash flow positive....

</comment><comment><USER11> There is a lot of value in the comments on this article, but I am afraid that the idea that you can usefully hire someone based on the answer to one question is a bit oversimplified. In any project that I have done that spans any length of time, often the first code I wrote doesn't look up to snuff with what I wrote most recently. The later code looks better by virtue of a better feel for the environment and the tools and some obscure abbreviation-finder that seems to be at work in the background.

One concern that I would have if you think the worst code that you have seen is stuff you wrote, perhaps you are not reading enough of other people's code. I bet there is a lot of that amongst us all.

In the vein of simple questions that help in an interview, particularly for someone who purports to be an expert, is "Tell me five ways that it won't work". An expert ought to have some scars, and have learned from them, no?

</comment><comment><USER12> "How do I know, incontrovertibly, beyond the shadow of a doubt, that the world is full of incompetent programmers? Because I'm one of them!"

Those who can, code. Those who can't, blog.

<comment><USER13> Pissing on Jeff's coding ability became passé when he put out the best programming Q&A site out there. You need new material.

</comment></comment><comment><USER14> Extrapolated...

Programming will make you hate software. Programming will make you hate programmers. Programming will make you hate computers.

Lisp will make you forget all of that.

<comment><USER15> Lisp is crack. Your eyes are rolling in with pleasure and you're the king of the world, but your _expressed_ sense of delight is not commensurate with people's idea of what should be enjoyable and by how much, even your programmer peers (should programming _really_ be that fun?) And to be honest, you're just getting off writing skeletal Idealized software; your toy apps lack the necessary cruft to face a stupid world, so no one ever sees it. As time passes, you're left huddled over your workstation firing dopamines like any other fiend, and the world moves on in its crisply pressed suits, doing _hard_ work and getting social validation and recognition for it. You, on the other hand, are left flailing your arms, _feeling_ the music, offering the Love and the Message to others as they avoid making eye contact.

<comment><USER14> Isn't that what I said? :p

<comment><USER15> You weren't long-winded enough.

</comment></comment></comment></comment><comment><USER16> _In fact, I think you can tell a competent software developer from an incompetent one with a single interview question:

What's the worst code you've seen recently?

If their answer isn't immediately and without any hesitation these two words:

My own.

Then you should end the interview immediately. Sorry, pal. You don't hate software enough yet. Maybe in a few more years. If you keep at it._

I have always enjoyed Jeff's columns and maybe I'm missing a subtle point (or just taking the bait), but this may be the worst "advice" I've ever read on hn.

If someone told me that the worst code they've seen recently was their own, I'd wonder why.

I feel confident in my own ability to usually write excellent code on the first try. Sure, it may need some refactoring, optimization, and flushing out of features, but it works, it generally works well, is very well documented, and can be easily enhanced by myself or someone else. If it's not right on the first try, I will fix it before I promote it. Shame on me (and anyone else for that matter) for leaving behind garbage for someone else to inherit. I've cleaned up far too many messes to allow myself to become like that.

I'm probably not alone when I say that I love developing new code and often hate maintaining someone else's mess.

Jeff is right about one thing though, almost everything I ever inherited was crap in one way or another, probably written by someone "less than senior".

I'd rephrase the whole interview question:

"What's the _second_ worst code you've seen recently?"

"My own."

"OK then, what's the worst?"

"Everyone else's."

 _That's_ the one I would hire.

<comment><AUTHOR> The irony of you saying "almost everything I ever inherited was crap in one way or another" is that somebody probably said the exact same thing about _your_ code. I bet you write amazing code, but when given a choice, programmers would rather have the joy of creating their own code than trying to interpret someone else's.

<comment><USER16> When it comes to maintaining code, I believe there are 2 kinds of crap: subjective (I don't like it) and objective (it's crap because of these 14 specific reasons).

You're probably right about people who inherit my code. I know, when they've whined about it, I confronted them. "Please show me exactly what's wrong with it. What are your specific complaints about violations?" I rarely got an objective answer. It was usually something about formatting, indenting, variable names too long, variable names too short, I'm use to it the way we did it (wrongly) at XYZ Co., something like that.

True crap can be objectively identified by a violation such as:

\- variables named so that no one except King Tut could possibly figure out what they are

\- the same variables used for different purpose _at the same time_ , usually in nested recursions

\- variables named with 1 or 2 characters

\- unassigned variables

\- variables initiated when they shouldn't be

\- division by zero

\- single entry, multiple exit (heavily maintained so that now outlying cases skip critical logic)

\- the same code in multiple places (only some of it maintained so that outlying cases skip critical logic)

\- endless recursions (for outlying cases only, naturally)

\- comments that don't agree with the code

\- data base tables with no definitions either in the schema or any code (my new favorite)

I could go on and on, but you kinda get the picture. I wonder how many readers here have posted crap like this on their "wall of shame" at one time or another. It's funny the first 2^n times. Not so much fun anymore.

<comment><USER15> Those are technical issues. The truly most nightmarish software to inherit is one that shouldn't have been writen to begin with: software made by people who either lack the programming skills, or the domain knowledge. Nothing worse than an accountant-turned-DBA's perl, or a system's programmer that just left you an steaming pile of MFC GUI. Worst of all, the person responsible for the mess is not long gone and forgotten, quite the contrary, he has been promoted through and he is now responsible for interviewing you to polish his brown magnum opus.

I breath a sigh of relief whenever Microsoft breaks and old DLL's compatibility: when you can't push through for an upgrade, pray for Redmond to force it.

[Update:

``The hardware-specific accounting package''

This one is truly the "winar": plain-jane boring software that for some goddamn reason depends on that old Hayes modem, or this matrix printer. I have been hired to do this shit so often it's not even funny; it tends to sneak past the initial discussions. You tick off "GUI" and "ODBC" and you're on your way to start hacking, making a mental note about "ability to print". What you don't know is that, not only do they want to convert the old Access package to an intranet web app, but they also want it to print receipts using the "receipt printer" -- a custom made piece of pain that uses the parallel port for data, the serial for control, and two RJ45s to whistle the Danish national anthem on error. FUCK!]

<comment><USER15> Now I can't sleep (why did we have to talk about bad code?)

``The Windows 3.1 Guru''

1) That guy who knew too much about Windows 3.1 internals and never thought this newfangled "Windows for Workgroups" or "95" would never take off. Thank you, sir. Treating obaque structures for the unsigned chars they are has never posed a problem for me. All the documentation I needed was in those 4 pages of macros you left me (completely unnecessary, given the clarity of their octal values, btw) Indeed, 10 years later, the low nibble of the AL register held an integer index into your very own " _stash_ area" just below the PSP (task_struct for linux weenies) where you kept all the open file handles. Clever of you, reusing every bit of the process memory, specially the unused parts of the MS DOS header format. (long story about this _destructive_ log- rotate elided)

Your share custody of this award with Mr. EE.

2) Mr. Electrical Engineer, how can I forget you? We all know it's all signals and gates down there, so, yeah, it was refreshing to see them again! I can't believe you rolled your own signal-driven layer on top of the Win32 message system. The ordering system couldn't have been done any other way. Looking at your code put a bar over my head, sir, you have asserted me. And to be honest, Visual C++ needed your linguistic extensions; we all know, code runs faster when it looks more and more like Fortran.

<comment><USER11> These would be richly funny if they were not so entirely true.

</comment></comment></comment></comment></comment></comment><comment><USER17> As usual Atwood manages to destroy a valid perspective (be mindful of your code quality) with unnecessary hyperbole.

I wouldn't hire anyone who said their own code was the worst they'd seen recently. Whatever points they may win for perception and awareness are totally overshadowed by their lazy acceptance of the situation.

"Which areas of your coding could you most improve on?" - now that is a question worth asking,

</comment><comment><USER18> Clearly the main issue of the post is the answer to the interview question. Although I think it's a very good answer and that to me it would probably mean I would be hiring a guy which is humble, which always views his knowledge has something that still could be better and has a view of his own code has something imperfect, and so he tries very hard to always know more and improve, and that would mean in the end he is just one of the best choices to hire, I think that in most companies that guy would never be hired.

Although it sucks, nobody gives any value to humility anymore. They want the confident, "I'm an expert" kind of guy, so giving as answer that the worst code you've seen lately is your own, without any kind of explanation of why you think that (that you view your knowledge has something which as always a long way to go) would probably be a instant KO to the poor humble interviewee.

</comment><comment><USER19> What you'll end up with is hiring either incompetent programmers who know they're incompetent, or people with no self-esteem. Great. It seems more like either Jeff is incompetent (which he admits to) or self-deprecating, and assumes everyone else is.

</comment><comment><USER20> I don't understand who votes these articles up anymore. They're just completely useless.

<comment><USER21> Not as useless as your comment.

Besides the "my code is the worst code" part, I thought he did an excellent job of explaining a software developer's natural aversion to software. I had my wife read it, and it helped her understand more about why I refused to pollute the computer with that CD that came with her new camera.

Aside: I know it's a beloved HN meme to bash Atwood, but don't let that excuse us from being lazy about it.

<comment><USER20> My comment does serve a purpose, to point out how useless this article is, since some people obviously don't notice it, and vote it up.

<comment><USER21> Your comment did not point out anything other than the fact that you personally think Atwood's articles are useless. It did not contribute anything to the discussion, and it garnered knee-jerk upvotes from people who, like you, hate Atwood's articles. I don't often downvote, and I never downvote for disagreement. I downvote comments that either add nothing or take away from the discussion, and your comment was a perfect example of that.

If you have a particular problem with this article, then follow the example of edw519 and elaborate.</comment></comment></comment></comment>


Cloudflare for Teams - <AUTHOR> https://teams.cloudflare.com/ <comment><USER2> I wonder why there is no mention of the price.

<comment><USER3> Price discrimination. What you charge a startup is going to be different than what your charge McDonalds.</comment></comment>


Rise of the Hack - <AUTHOR> http://www.tokbox.com/blog/rise-of-the-hack/

<comment><USER2> "They are a national phenomenon."

They are an international phenomenon. I see them, along with startup culture, happening in pretty much every Europe centre too.

</comment><comment><USER3> Jon Mumm used to be a professional counter striker. Read his article here: [http://eseanews.com/css/index.php?s=news&d=comments&...](http://eseanews.com/css/index.php?s=news&d=comments&id=10446)

<comment><USER4> I read this and thought "strike-breaker", was hoping to see some lurid tale of how he went from being a corporate thug to a creative hacker.

Turns out its a video game. Respectable enough hacker hobby.

<comment><USER5> It drives me nuts too but in a different way: I always read CSS pro as 'Cascading Style Sheets' and not CS:Source even in a video game context

</comment></comment></comment><comment><USER6> I attended my first Hackathon last year (in fact, the first Hackathon I attended was one that I hosted!). It was a really incredible experience. So many more people turned out than I expected, and the quality of the work was really impressive. Most people who showed up were University kids, and they really killed it. In the course of 5 or 6 hours we created 15 or more three.js based browser games (there were close to 50 people there).

My only regret has been that I've been too darn busy to host another!

</comment><comment><USER7> When Jon first told me about what his job entailed I found it interesting. He came to our Startup Weekend and even became the lead designer for our team (HackLadder). I think it would be great to branch out and see Hackathons in different areas.

Jon is spot on when it comes to our community. There has been a pretty big spike in Hackathons around here. I have noticed a lot more talk about it on the internet as well.

</comment><comment><AUTHOR> tokbox blog went down! but you can read it on venturebeat too: <http://venturebeat.com/2012/03/15/the-rise-of-the-hack/>

</comment><comment><USER8> Outstanding image! Great work.

For somebody unfamiliar, where does one go on the Internet to find the beSt list of similar events near by?

<comment><USER9> <http://hackerleague.org/>

<http://plancast.com/category/hackathons>

</comment></comment><comment><USER10> Is there a good list of upcoming hackathons online somewhere? Or places one should look?

<comment><AUTHOR> Check this out:

<http://hackerleague.org/> <http://plancast.com/category/hackathons>

</comment></comment><comment><USER11> winner!!!

</comment><comment><USER12> Great article!! Totally enjoyed the blog.</comment>


Detangle automatically separates your browser into multiple browser profiles - <AUTHOR> https://github.com/google/detangle <comment><AUTHOR> [https://twitter.com/svblxyz/status/958314262270705664](https://twitter.com/svblxyz/status/958314262270705664)

Detangle is now open-source. You can specify a list of "internal sites" to be opened in your main browser, while other sites get opened in contained / separate profiles (or Incognito mode). Great for added protection against UXSS and other threats.</comment>


Show HN: 4usxus.com – Vote on issues and compare your votes against your reps - <AUTHOR> https://4usxus.com <comment><AUTHOR> Would love to hear your feedback about the site. Thanks

<comment><USER2> > margin-left: 25%; width: 41.6667%

Real good use of space! At least you show something without JS.</comment></comment>


Tricky Twins Spoofed Trading Computers - <AUTHOR> http://www.bloombergview.com/articles/2015-12-03/tricky-twins-spoofed-trading-computers <comment><USER2> Ex options trader here. Something about the ingenuity of this scheme is familiar.

So basically they make two firms, and each quarter swap which firm is doing the business in order to get privileges that only firms that haven't traded much in the previous period would earn. I wonder how they thought the SEC would not find this out, that there's two companies outside a financial centre with owners of the same name and birthdate, alternating their trades.

I wonder if it's going to be considered a loophole that will be closed or an abuse. I suppose it could be either.

The other thing, gaming the algos, I have a good idea of how it might have happened.

You see, options people (the ones I know) are not as technologically sophisticated as stock people or futures people. Many of them rely on quite simple systems to quote their orders. Several of these are off-the-shelf systems with functions you'd expect: an autotrader than joins the BBO in some size, on every single option, not particularly fast either. This works fine most of the time because options traders are not overly concerned with the action on a specific option; they need to know their delta/gamma/vega, and those can all be thrown in a big bucket to be hedged as appropriate. The spread is normally quite wide, so as long as you're not being gamed on everything, you have ample room to lock in some profits.

People writing sophisticated algos (whom I also know) know how to do an ignore size, ie don't just blindly follow a tiny order.

<comment><USER3> Regarding the "options people not being as technically sophisticated" One of the stories from the former PHLX Options Exchange, now part of the Nasdaq umbrella is that their rapidfire price protection feature was one of the killer features that was a competitive advantage over the other options exchanges back in the day. Now a days, all the major options exchanges have this feature.

A market maker required to quote in say 80% (by self-governance rules) of options in a given underlying has a lot of quotes to update when the market moves. AAPL has 1000+ options, so a market maker may be obligated to quote at least 800 options books.

When the market moved, there would be 800 options to update. With order/ack latency around ~30-50u today, but around 400u-1ms 10 years ago, it could take almost half a second to 1 second to get those quotes updated. In that eternity, a fast trader could swoop in and execute against the old quotes to their advantage.

Rapidfire implemnted an atomic cancellation of orders if too many options executed on a single side only such as only the bids getting executed. This significantly minimized the risk of getting eaten alive on say a couple hundred contracts every single time the market moved.

<comment><USER4> I know that ISE had some sort of automatic cancellation feature from the beginning or very close to the beginning, so not sure if PHLX was the originator of this idea. Also, nearly all electronic options exchanges have a "mass quote" feature that allows changes in prices over a large number instruments at the same time. I would expect that this feature is available on every US exchange and only a few overseas exchanges lack it, and managing the generation of mass quotes vs exchange message rate limits is one of the more complicated parts of designing option quoting systems. Finally, there is also a "cancel by underlying" feature that cancels all options on a specific underlying; if it is the case that the mass quote doesn't allow quotes to be cancelled (i.e. modified to 0 size), then the usual approach is to cancel by underlying and then immediately requote at the new price level.

</comment></comment></comment><comment><USER4> I've worked in automated option market making in the past; if I were on a jury hearing this case, there's no way I would convict these guys of spoofing. They are taking advantage of a feature built in to the market for purposes similar to this: hiding your intentions. The only people who could realistically be damaged by this are professionals, retail customers will either benefit from the tiny bit of extra liquidity added by the 1 or won't be adversely affected by the hidden AON orders on the other side. The algo traders who complained to the SEC should be embarrassed at doing so; they should have been burned on a couple of trades, figured out what they were doing wrong and fixed their code to compensate. Blindly stepping up your quote or order for size on a 1 lot in order to capture rebate is just foolish and anyone who can make money off you for doing so should be entitled to it.

</comment><comment><USER5> I'm having trouble being outraged. Some of their behaviour was clearly fraudulent, and yet it seems to be a case of obeying the letter of the rules, but not the spirit.

As for the alleged "spoofing," again, I'm not sure what to say. It is normal for market makers to place orders on both sides of the market (though not at the same price). While either order could execute, they certainly don't expect both to do so. And further, market makers are regularly making and cancelling orders that they don't expect to execute. I think the court needs to find a clear distinction in what they did in order to convict them of a violation, and it is not obvious to me that the distinction exists.

<comment><USER6> The distinction between spoofing and market making is very clear. When a market maker puts an order on both sides of the market the would absolutely _love_ it if both orders traded. That is, in fact, how they make money. In general they would love every single order they put in to trade, so long as it happens in the right order.

When a spoofer puts in a spoofed order, that order is all cost. Under no scenario do they want that order to trade. The hard distinction to make in this case is not between normal market making behavior and spoofing, but between normal hedging behavior and spoofing. Hedges look like pure costs, but they are costs you are happy to pay to give up some risk. So spoofing all comes down to intent, which seems pretty easy to prove when you have chat transcripts.

Whether spoofing _should_ be illegal is another discussion, and probably one the legislature should have, but for now while it is illegal, it should be aggressively enforced as if it's not, it incentivizes the bad actors.

</comment></comment><comment><USER7> The amount of human ingenuity that goes into these stupid, pointless schemes is a tragedy.

</comment><comment><USER8> If your system has poorly thought out rules, it can be gamed. If these rules also go against market pressures, it will be gamed. I'd argue that in the case of these absurdly complicated SEC rules, it should be gamed.

If your stock exchange system (including the technical and legal aspects) needs a bunch of weird hacky proscriptions on user activity, then it deserves to fail.

<comment><USER6> Anti spoofing laws can be described in a single sentence. There is nothing complicated about them.

I'm having trouble coming up with a single rule you could make about trading that wouldn't count as a weird hacky proscription, if you view spoofing as weird and hacky.

<comment><USER4> They can be described in a single sentence, but detecting the difference between spoofing and legitimate activity can be very difficult, especially when some participants are trading cross-border and cross-asset; their orders are reacting to events which are not local to the exchange where they occur and legitimately change at a high rate because they are being affected by many different factors.

</comment></comment></comment><comment><USER9> It sounds like the real issue here is that the exchanges withhold information from the market.

Obviously the professional/customer distinction manipulation was fraud, but it is the exchange which doesn't advertise bids/offers they know about.

<comment><USER10> But that's fully disclosed. The order type is explicitly not publicized.

<comment><USER9> Right, so complaining about "hidden" orders is useless. The problem lies with the exchange, not the strategy.

<comment><USER10> The problem is the combination of two orders, which serves to mislead the market. Merely making hidden orders would be fine, making hidden orders, then making another order meant to mislead other traders is not fine.

I agree the line isn't so clear.

<comment><USER9> Any misdirection seems to be the fault of the exchanges.

Isn't it common to both buy and sell contracts in complex arrangements, where various aspects of each individual transaction "cancel out" other aspects of other transactions? Assuming you buy 1000 contracts and sell 1, you are net 999 contracts? How is that different from any other options trading strategy?

<comment><USER10> For one, the two were made from different accounts, which is suspicious by itself. If you submit from a single account, you generally can't trade against yourself.

Second, the buy and sell are the exact same price. So there's no legitimate reason to both buy and sell at the same price.

</comment></comment></comment></comment></comment></comment><comment><USER11> Another great analysis by Matt.

</comment><comment><USER12> Is there something funny about this situation that makes this footnote true?

> If Makino's one-lot trades never executed against Fineline's big trades, > they look like spoofing, which is illegal. If they did execute, though, they > look like wash trades, which are also illegal. Either way, the actually > important question is whether they were illegally tricking other market > participants, not whether they were intended to execute.

Wash sales aren't _illegal_ , you just don't get to deduct losses from taxes. My understanding is that intention to execute _is_ the main determination in spoofing.

<comment><USER6> A Wash sale & a wash trade are not the same thing. Wash trades are illegal in US markets.

</comment></comment><comment><USER13> People who do these kind of things are like heroes to me. All of this trading is just legalized gambling for rich people. I admit I trade options myself because I like educated gambling. Still seeing some smart young guys out wit the lazy algorithms made my day.</comment>


LightPaper 1.0 is released - <AUTHOR> http://lightpaper.ashokgelal.com <comment><AUTHOR> Previous discussion of it on Hacker News: [https://news.ycombinator.com/item?id=6969897](https://news.ycombinator.com/item?id=6969897)</comment>


Could We Just Lose the Adverb (Already)? - <AUTHOR> http://www.vulture.com/2016/05/could-we-just-lose-the-adverb-already.html <comment><USER2> I'm re-reading Stephen King's "On Writing" right now and he dedicates some of the initial portions of his advice section to anti-adverb rants. "The adverb is not your friend." I catch myself using them more often than I should. His take is that the context around whatever statement you've inserted an adverb into should already tell the story that adverb was inserted to tell.

</comment><comment> open-source-ux This was a difficult article to read, but I hope we don't lose the adverb.

I hope we don't lose prepositions either but I have noticed a point of divergence between British and American English relating to this.

In American English it's common to say _a couple weeks ago_. In British English you would say: _a couple of weeks ago_.

Or someone in the US might say _write me_ , whereas in the UK you'd say _write to me_. I think this is a recent change in American English? Or am I completely wrong and this has always been part of American English?

<comment><USER3> Or in Australian English where _a couple of weeks ago_ becomes _a cupala weeks ago_.

<comment><USER4> That's just eliding the 'f' sound to make it flow better. It's basically "a couple o' weeks ago".

/eɪ kʌ.pl̩ ʌ wiks ʌ.go/

And yes, "ou", "o" and "a" all render to /ʌ/ in my dialect. Yay English!

</comment></comment></comment><comment><USER5> I found this fairly awkward to read. Possibly that was the point, in which case it was well made. Otherwise, I would consider the author to be demonstrably unqualified to offer advice on writing style.

<comment><USER6> > I found this read awkward. The writing does not demonstrate the > qualification to offer advice on writing style. If that was supposed it is > good.

Fixed, because your comment on quality does not convince me with its quality.

edit: See how that slightly changed the meaning to be more clear, but perhaps not what you meant? And awkward looks arkward.

</comment></comment><comment><USER7> There's nothing inherently wrong with adverbs. They're just a tool.

Like all tools, they can certainly be misused and inappropriately applied by people who don't know how or when to use them. That doesn't mean they're a bad tool.

</comment><comment><USER8> I think he calls "yesterday" an adverb _erroneously_.</comment>


FoundationDB marries NoSQL & SQL with Akiban acquisition - <AUTHOR> http://venturebeat.com/2013/07/17/foundationdb-akiban/

<comment><USER2> At Centzy, we evaluated FoundationDB for several months. Impressive technology (horizontally scalable key-value store with transactions and failover), its fast, and it just works. We were initially concerned about using a proprietary datastore built by another startup, but after having lunch with Dave and Andrew, we decided we had to use them. One of my favorite things was how they sometimes responded to support requests with code.

</comment><comment><USER3> FoundationDB co-founder here. We're pretty psyched about this acquisition and the technical fit between our storage substrate and Akiban's open-source SQL tech.

Happy to take questions.

</comment><comment><USER4> Wow, FoundationDB is getting major traction since TechCrunch Disrupt last year</comment>


Microsoft learned from the past to design its future - <AUTHOR> https://www.theverge.com/2019/4/29/18515776/microsoft-design-open-fluent-prototypes-history <comment><USER2> I found it amusing to see pictures in the article showing Microsoft's design team using Macs instead of anything else running windows. No "eating your own dog food" here, move along.

<comment><USER3> I worked on the Windows team in the past, and it was a legitimate problem. Exposure to competitive products is great to stay aware of options and what others are doing, but the designers I worked with used Macs exclusively to the point where they didn't understand core Windows app switching/launching/task management workflows. A significant portion of my interactions was often explaining to them how proposed changes would interact with normal users, legacy apps, and everything else outside of idealized rendered screenshots.

<comment><USER4> > A significant portion of my interactions was often explaining to them how > proposed changes would interact with normal users, legacy apps, and > everything else outside of idealized rendered screenshots.

Ouch, that must be painful to do and keep doing. Did you manage to make a change in the end? (or should I take the articles' movie as an example that it did not change :-) )

</comment></comment></comment><comment><USER4> Interesting to see that Figma seems to be adopted by at least the OneDrive team. They were working on some explanatory images and copywriting there. Does anyone know if they use any other tools to collaborate on design?

</comment><comment><USER5> Did they show Macs by purpose on the pictures? ;)

<comment><USER6> Haha at least they covered up the logo with a sticker</comment></comment>


This Link Will Self Destruct - <AUTHOR> http://tlwsd.info

<comment><USER2> "You are not using HTTPS. Are you fucking stupid?"

> SSL certificate invalid

gg guys

<comment><USER3> The submitted link probably needs to be [https://tlwsd.in/](https://tlwsd.in/).

When you click on the "About" page, though, it kicks me right back into [http://tlwsd.info](http://tlwsd.info).</comment></comment>


Python 2.7.2 Unleashed - <AUTHOR> http://www.python.org/download/releases/2.7.2/

<comment><USER2> Set and dictionary comprehensions, and set literals are both very cool. I wonder though: is back-porting all the nice features of 3 into 2 going to encourage adoption of 3? I somehow doubt it.

<comment><USER3> Not doing so would feel punitive. A positive way forward would be better.</comment></comment>


It's Time For Google To Take A Stand On Paid Links - <AUTHOR> http://www.businessinsider.com/henry-blodget-conductor-ceo-besmertnik-on-paid-links-link-building-seo-2009-9

<comment><USER2> I cringe every time I hear "violating Google's terms of service." What service? The one where they index your site at your cost so they have something to serve their users?

This is starting to sound more like the RIAA complaining about a new business model undercutting its revenue. Google is not keeping up with people or technology in this case. People have figured out how to exploit (to a great enough degree) Google's ability to tell a spam site from an authority, and their response is to declare that people are in violation.

<comment><USER3> What is the tell between a spam site from an authority?

<comment><AUTHOR> In general terms:

 _Spam:_ thin site, lacks original content, few pages indexed, crawled infrequently, ranks for a small selection of keywords, weak backlink profile

 _Authority:_ many pages indexed, thick site, loads of original content, crawled frequently, ranks for a variety of keywords, diversified backlink profile

Page Rank used to be a strong indicator of this, but currently a better indicator is how often your site is crawled. That's why one authority link can do worlds more for you than hundreds (or thousands) of spammy/low quality links.

<comment><USER4> I guess it's all about how you define "thin," "few", "small," and "weak." A page on how to eradicate your home of carpenter ants would possibly look like all of these, yet may be the most "authoritative" site on the web on the small, niche subject.

<comment><USER5> If it _is_ authoritative, then you would expect it would (eventually) get linked to, no?

<comment><USER4> Perhaps, but not necessarily that much. How many people would put a link to how to build a dog house on their web page? Anybody who would probably has links to other dog house things (buy a book on Amazon, keep it smelling fresh with this goo from WalMart) etc., diminishing your algorithmically determined authority.

Not everybody blogs about everything going on in his/her life.

</comment></comment></comment></comment></comment></comment><comment><USER6> When you have things of value that can be sold or exchanged you will allways have a market. If some authority tries to ban the exchange then it will be a 'black' market. The harder Google tries to enforce their policies the more it will drive the selling or exchange of links underground. Maybe this would be a good thing but there is certainly no way they can stamp it out.

I personally think that the problem lies in the rules of Google's system. A more robust system should be able to account for and adjust for the market realities created by it's own rules. Maybe it is technically impossible but I doubt it. I'm sure the clever people at google have thought of this as well but perhaps they are starting to face 'legacy' system issues themselves.

<comment><USER7> Perhaps Google already algorithmically detects and quietly penalises such link-farming, or has tweaked their algorithm to minimise its impact. From their quality guidelines ([http://www.google.com/support/webmasters/bin/answer.py?hl=en...](http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=35769#3)):

 _Google prefers developing scalable and automated solutions to problems, so we attempt to minimize hand-to-hand spam fighting. The spam reports we receive are used to create scalable algorithms that recognize and block future spam attempts._

<comment><USER6> I'll bet they do but they can only detect the most obvious cases. It would be possible for website to sell links that even a human couldn't identify as sold, so there are limits to what their algorithms could achieve.

</comment></comment></comment><comment><USER7> I'm often wary of those complaining about Google's policies because they're based on the idea that being listed on Google is a right and not a privilege. This, to me, is faulty logic. Unless they're deliberately taking part in anticompetitive behaviour by blocking competition, I think it's perfectly within their rights to delist or demote you for pretty much any reason they can think of.

I think Google have made it pretty clear that as far as they're concerned they don't want people selling links to other sites in a way designed to manipulate search-engine rankings, and if you do so you may be delisted or have a lowered ranking on their site in the future. I think that's specific enough, personally, and think that anyone wanting a more specific policy either wants to work around them to violate the intended spirit and complain if and when Google punishes them for doing so.

</comment><comment><USER8> I used to be a bigger Google fan than I am now, and it's all because of this pagerank/link business.

It's fine to have an algorithm to prioritize sites. And it's fine for sites to manipulate that algorithm to their own ends. Both of these things are completely natural.

Trying to keep this list of who's cool or not -- secretly updated by god knows who -- and forcing the rest of the world to do as you say or suffer economic punishment by de-listing from your engine? Not fine, although Google is certainly in their rights to do so.

A better strategy is for Google to adapt, not dictate. As it is, they've created this entire system where everybody games the system but nobody talks about it, and that's counterproductive for the net as a whole.

<comment><USER9> Exactly.

Google should become better at what they are doing, work harder on identifying spam sites. Crawl more frequently to get a better grip on provenance.

Maybe set up an 'announce' service where you can dump a url before linking it to establish provenance for your original content.

Dictation is not a solution to anything it is a show of weakness.

</comment></comment><comment><USER10> With apologies to Sigmund Freud:

Google forbids us to sell links not because it wishes to do away with linkselling, but because it wishes to monopolize it.

<comment><USER11> Google has no problem with link selling--Google has a problem with PageRank selling. Sell all the links you want, just use nofollow.

If someone wants to buy a link but isn't interested in only receiving the traffic it brings (and not any pagerank), they are probably up to no good anyway.

<comment><USER12> I just checked the paid links on Google's search page. They do not use nofollow. This fact, combined with your arguments, is consistent with gojomo's hypothesis about Google wanting to monopolize paid links.

<comment><USER13> Google's search results page blocks all crawlers (including Google) using robots.txt, so it's a moot point. Nofollow's only relevant when a page might appear in a search engine.

<comment><USER10> And isn't that convenient? Google's main kind of link-selling is unaffected by their rules, but other kinds of link-selling is disadvantaged. ("Use a standard we invented to make our job easier, or risk being frozen out of 70%+ of all search traffic.")

And similarly: "We crawl your stuff; but our search results are off-limits." Of course, anyone can block Google, but given the asymmetry in business models and market power that's not much of an equalizing factor.

I like Google and think they mean well. Individually each of these policies has a reasonable basis. But Google is now so dominant -- and the positive financial rewards they get from self-serving policies so large -- that their actions deserve close scrutiny.

(The Freud-inspired quote I led with was originally about the state; Google's influence on the net is so large that they are a lot like the net's government.)

</comment></comment></comment></comment></comment><comment> onreact-com This issue has been resolved years ago. Google even has a report paid links form in the Webmasters Tools. High profile sites get their PageRank and authority diminished whenever paid links get discovered.

Most reputable SEO firms refrain from paid links nowadays and rather focus on "killer content" creation to get links naturally.</comment>


Sniff my browser: The Modernizr inadequacy - <AUTHOR> http://nefariousdesigns.co.uk/archive/2011/05/sniff-my-browser-the-modernizr-inadequacy/

<comment><USER2> I have mixed feelings on Modernizr...but this post seems to be filled with a number of weak points.

First, the Modernizr file ends in ".js" I don't think anybody is fooled about how it works and what its consequences are.

Second, the advocated method of stying your site with Modernizr is to first make it look good assuming no "advanced features" exist and then progressively enhance. This ensures that the failure of Modernizr to load will leave you with what you started with. Given that the Modernizr classes are added to the root of your document this claim seems impossible:

"However, it also means there is a significant loss in the overall structure of specificity within your stylesheets, which could easily cascade unintended effects through to other elements."

Finally his core argument is that Modernizr violates the separation of concerns in front-end web development. This is true and something that you have to be willing to accept to use it.

His proposed alternatives are strange however. He suggests server-side browser sniffing, a bad idea as the article itself explains. Alternatively he suggests using CSS cascading, but this is a bit of a tautology since this is the problem Modernizr was designed to solve.

My personal opinion is that Modernizr is a bit of a kludge, but often the least ugly option available.

<comment><USER3> > Second, the advocated method of stying your site with Modernizr is to first > make it look good assuming no "advanced features" exist and then > progressively enhance. This ensures that the failure of Modernizr to load > will leave you with what you started with.

I think the big problem and why separation of concerns matters is that people don't design a site once and then let it be. Most frontend developers maintain already existing sites, across new features, new browser versions and new end user expectations. You can't apply that "advocated method" there, or at least you can't do so equally well.

I guess that simply disabling Modernizr when adding new design elements until things are good enough, and then enabling it for the "advanced features" could be a work around. I've yet to see such a disciplined workflow somewhere, however.

Nevertheless, I agree with you that it's the least ugly option available. Maybe making it easier for a developer to temporarily disable modernizr helps styling with the "prefer good styling with no advanced features" approach.

</comment></comment><comment><USER4> There is this constant tug-of-war going on for developers. On the one hand we want to provide the best possible experience to the end-user, and on the other hand we want to include as many people as possible.

The two forces are in competition with each other though, because very often the largest group of people are running older technology.

We can broadly divide the potential user base into 3 groups. Those "ahead" with a cutting-edge browser, those with a "modern browser" and those "left behind". Today we'd consider IE6 users and Windows Mobile 6 users as "left behind". We'd consider HTML5 in a late-version of Chrome to be "ahead". Not surprisingly though the vast majority of people are in the middle.

The key question seems to be "Is JavaScript in the mainstream?". I think the answer to that is yes. Even IE6 has Ajaxian capable JavaScript. So therefore is the user-experience, of the many, improved by using JavaScript, to the point where the "few" are sacrificed? In my opinion, yes.

Certainly the user experience is improved immensely with the use of JavaScript. The question then becomes whether it is better to service say 95% ( _) of the market really well, or 100% of the market in a mediocre fashion? I would argue that the market does not reward mediocrity - and the 5% you have gained are the least likely to appreciate your effort anyway.

(_) pick whichever number here you prefer.

Take an average hotel website for example? It's certainly all-inclusive, but at the same time is horribly unusable. It has evolved since 1995, and it's still as awful as it was then. Does a hotel gain 5% more occupancy for making it all inclusive, or do they lose more than 5% for having a horrible interface?

Personally I don't believe any site is "all inclusive". Those primitive enough to include the bottom end, are too dull for the folk on the top end. So the real question becomes, would you rather have the top, or bottom demographic?

Bearing in mind that for the vast majority the decision to not have JavaScript support is completely their own?

Now of course every site has different considerations. But in my opinion the starting point for the discussion is "how much is that 5% worth?"

Will Modernizr make everyone happy all the time? Well, no. Then again, nothing will.

</comment><comment><USER5> Yeah, basically there _is_ one shortcoming of the "Modernizr approach" to front-end development -- namely that you will end up treating clients who have JS disabled as though their browsers do not support a number of CSS features.

So with the old browser sniffing way, your Firefox 4 user will still get the shiny buttons and rounded corners even if he has no JS. With Modernizr, he'll (at best) get the watered down IE6 version with flat buttons and sharp corners, etc. At worst, if you're not doing progressive enhancement right and you've based critical functionality on one of the injected classes, you might cause some real problems for this user.

I guess I just see this as a limitation of Modernizr, and not a reason to throw out the baby with the bath water. I still love it and use it in all of my projects.

<comment><USER6> (modernizr lead developer here)

Something this article brings up and that you touch on is: coupling your use of CSS3 to JavaScript being enabled.

There is an incorrect way of using Modernizr where you always surround your use of a CSS3 feature with a `.feature .box` class, which is generally unnecessary and leads to the issue the author was concerned about.

Modernizr classes are best used in the `.no-feature` case, where you do something like use background images or another clever CSS use to mimic the feature or handle the fallback situation. In general I like to code my styles optimistically though.

I left a comment on the site saying it's hard to address this subtlety of the "right way" to use Modernizr classes. We're launching a new site very soon and hope to better document this approach so people don't go down this tricky road. :)

<comment><USER7> _Modernizr classes are best used in the `.no-feature` case, where you do something like use background images or another clever CSS use to mimic the feature or handle the fallback situation._

Switching from progressive enhancement (applying styles if you detect a feature is supported) to graceful degradation (applying styles if you detect that a feature isn't supported) doesn't solve the underlying problem. In fact, it makes the situation worse. JS failure with the progressive enhancement approach means that everyone gets the fallback experience; failure with the graceful degradation approach means that you're potentially asking a browser to apply CSS that it doesn't understand and not offering it a fallback.

No matter which way you cut it, having CSS predicated on classes set by JavaScript creates a dependency where one should not exist. I realise that's the whole point of Modernizr, but in my opinion it isn't the right way to solve this problem. Others may not agree, and that's fine, but it's critically important that developers being encouraged to use Modernizr (or its underlying techniques) are made aware of the tradeoff they are making and the potential impact that may have on their users.

</comment></comment></comment><comment><USER8> Tl;dr: Entire article is nothing but 'relying on js is bad'.

Author discounts modernized entirely because it only works with javascript. Suggests you can replace it entirely by tagging your html tag with an ie class using conditional comments (thereby relying on browser sniffing, after arguing that browser sniffing is terrible and doomed to fail) and the cascade in stylesheets.

I'm not convinced this gives a fair shake to the difficulty of getting graceful degredation right with just the cascade, and any author that says 'x is bad' then relies on x is clearly presenting an incomplete argument.

<comment><USER9> A clarification: conditional comments are a feature of IE, and thus are stable and reliable, and aren't technically browser "sniffing", at least in the traditional sense.

</comment></comment><comment><USER10> This post seems ridiculous. Modernizr is "bad" because it depends on JS, but the alternative solution suggested is to go back to user agent sniffing and just do it on the server side. I'd forgive the author for not knowing why user agent sniffing is a terrible idea, but it was covered in the post itself...

</comment><comment><USER11> Modernizr does a lot of things, and creating CSS classes to mark the existence of features is just one of those things. Another thing that Modernizr does is allow you to style new HTML5 elements like <section>, <header>, etc. - I think that there is a similar issue here, and I'm really on the fence about it. Depending on the browser to execute Javascript so that CSS is able to style a new element seems risky, and for quite a marginal benefit. I guess people are just trying to make the future happen sooner, but it seems foolish to introduce an external dependency and scripting requirement just because <header> looks better than <div class="header">. What do we really gain here by being more "semantic"? Aren't we just trading reliability for fashion?

<comment><USER12> > _I guess people are just trying to make the future happen sooner_

Here's why: Encouraging people to upgrade from old IEs to newer versions by making websites generic, mundane (but still accessible) and bland for them is just about the best thing that you as a web developer can do to help move the web forward. That and experimenting with the newest technologies as much as possible and reporting bugs and other feedback to the vendors.

</comment></comment><comment><USER13> I don't think Modernizr causes harm if you _understand and act like_ the resulting CSS styles are javascript dependent (as you should with any JS-based progressive enhancement). He's right, though: there are more conditions than you'd think where javascript may not be available. Make sure to ask yourself if the effect you want is possible without javascript first (as in the button example). And always test your pages for usability with javascript turned off.

</comment><comment><USER14> There are only a handful of browsers that doesn't support javascript today [1]. There also users consciously disabling scripting on their modern browsers but they don't add up to a significant percent of browser users [2]. The second argument that an external .js file can fail to load applies to the .css file as well.

There may be arguments against the modernizr approach, but the ones in the article are quite weak.

[1]: [http://en.wikipedia.org/wiki/Comparison_of_web_browsers#Java...](http://en.wikipedia.org/wiki/Comparison_of_web_browsers#JavaScript_support)

[2]: [http://visualrevenue.com/blog/2007/08/eu-and-us- javascript-d...](http://visualrevenue.com/blog/2007/08/eu-and-us-javascript- disabled-index.html)

</comment><comment><USER15> i can't advise modernizr as _the_ answer to html/css compatibility either, that being said it's a great tool to get started using html5 css3 until a) a better tool comes along or b) browser compatibility issues go away ...</comment>