Show HN: Scratchpad.io - a real-time HTML and CSS editor - <AUTHOR> http://scratchpad.io <comment><USER2> Its a little thing, but i like how you literally don't need to click on any buttons to save / create a new one / etc. I also like how you can create your own customized path to help you remember the link... just type in a new path and start editing, and you now can share that custom URI.

I think its important to create as little friction to usage as possible, and you've done that.

<comment><USER3> Thanks!! I really appreciate that. I also like the unique URL feature :)

<comment><USER2> What's cool is that feature was completely intuitive and you didn't need to point it out. Again, a little thing, but the little bit of delight users experience when they "discover" these types of features goes a long way.

</comment></comment></comment><comment><USER4> The first thing that I loved about this, that drives me crazy on other real- time online editors, is that pressing "tab" actually creates a tabbed space, and doesn't change the active textarea. Nice job!

Any plan on open sourcing the code to do the real-time editing? I would love to use this real-time editing concept in a project I'm working on.

<comment><USER3> Thanks! It was actually really simple to make with two tools: <http://ace.ajax.org> and <http://www.firebase.com>

If you want to chat about building it I'd love to give you any tips - i'm firstname.lastname@example.com

<comment><USER5> How did you accomplish the real-time text editing? Is there any OT engine that you hacked into Firebase?

<comment><USER3> Basically on keyup I ship the whole contents of the editor and the cursor position to firebase. I'm also keeping track of all the active connections to a scratchpad document so I know who is typing. Anytime the code changes on the server, it triggers a callback function on each client that will update the contents of the textarea. If someone else is typing, the document is locked so only 1 person can type at a time. This way I can avoid all the potential merge conflicting type things that can happen.

I'm not sure what an OT engine is. What is that?

<comment><USER6> Do you know what/how Firebase is doing the realtime updates? Websockets with fallback to flash?

<comment><USER7> Hi, I'm one of the Firebase engineers working on keeping clients connected.

We use a mix of different methods to maintain a connection. We normally start with our custom long polling implementation and move to websockets when they are available. We do not currently make use of any flash sockets.

<comment><USER8> Is Firebase open source?

It states so on the sites/GH pages, but could not find pointers to repo [after 2 circle through 5 different pages poiting to each other].

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> Looks like you've made sure any recursion only goes one deep? (Awesome)

<http://scratchpad.io/5mqq98D51P>

Every change forces a reload on the iframe. I wonder if there's a way to prevent that.

Heading

<comment><USER10> That has nothing to do with Scratchpad, it's in the spec for HTML frames: "Infinite recursion is prevented. Any frame that attempts to assign as its SRC a URL used by any of its ancestors is treated as if it has no SRC URL at all (basically a blank frame)."

<http://www.w3.org/TR/WD-frames-970331>

</comment></comment><comment><USER11> Thats awesome. Also looking at the source, I discovered it's using keymaster.js and firebase.js, which are new to me. And the implementation of scratchpad.io is very simple and elegant. The design is beautiful, and the code (in scratchpad.js) is so simple that it makes for a good tutorial on using firebase and ace.js.

In fact, if anyone else is interested in forking the code and experimenting with it, feel free to fork my plunk and go to town:

<http://plnkr.co/edit/49fOC5Mq3bcBysKfdPor?p=preview>

(Hope you appreciate the irony of forking scratchpad.io via Plunker. Also, it'd be wicked if Plunker could automatically "fork" html webpages, pull in all the required resources and/or updating path references).

</comment><comment><USER12> For a first time coder, being able to see your page change live is very helpful in getting to learn HTML and CSS. This is an amazing complement to the book you are writing - nice work Nathan!

</comment><comment><USER13> Finally....they had something close to this on codecademy, but this is exactly what I need when trying out new stuff!

<comment><USER3> Thanks!! Glad you enjoy it

</comment></comment><comment><USER14> Ok, this editor rocks. Nice use of Ace.js.

One of my biggest frustrations with jsfiddle is the crappy coding area (not to mention the default selection of Mootools among others).

<comment><USER3> Thanks! All it took was the lovely tomorrow_night_eighties.css theme :)

</comment></comment><comment><USER15> This is really great, especially as a compliment to the learning tools coming to market.

For front-end designers looking for a similar effect within your own text editor and browser window, I can not live without the CodeKit app now. It does a similar real-time effect each time the template files are saved. Also supports less, js debugging/minification, and other production processes.

<http://incident57.com/codekit/>

</comment><comment><USER16> Slightly off topic: You use "enough to be dangerous" with a positive meaning, as in enough to make a real difference in the world, while Alexander Pope quote you use is probably negative: "A little learning is a dangerous thing." as in, someone who knows little can do more harm than good.

<comment><USER3> Haha this is very true. I like to think of it as an easter egg.

</comment></comment><comment><USER17> I don't wanna down a nice looking project, very well done on the interface but, another one? There are already like 7+ of these I don't really see the need to produce another one especially one that is HTML+CSS only when many of the others feature JavaScript and more.

What features do you provide that say, <http://jsfiddle.net/>, <http://cssdesk.com/>, <http://dabblet.com/>, <http://rendera.heroku.com/>, doesn't? (Just the first 4 items that came up in a Google search, the list could go on...)

<comment><USER18> I think the key extra feature is that you can share what you're working on with someone else and have both people collaborate in real-time (i.e. changes on either end show up near-instantly on the other side). Supplemented by a voice channel, this could potentially be useful.

JSFiddle et. al. don't have this, but I'm sure there are others out there that do.

<comment><USER19> plunker does have that : <http://plnkr.co>

</comment></comment></comment><comment><USER20> Have the realtime plus: <http://jsfiddle.net/> And you will rock!

Right now, you are just awesome ;).

<comment><USER5> Just so happen to have a solution for you: <http://plnkr.co/edit/?p=streamer>

DEMO: [http://plnkr.co/edit/?p=preview&s=HN4933608](http://plnkr.co/edit/?p=preview&s=HN4933608)

</comment></comment><comment><USER21> This is perfect - love the immediate feedback for my own prototyping.

Could probably use the multi-user sync for front-end technical interviews.

<comment><USER13> ^Exactly what I was thinking.

</comment></comment><comment><USER22> Oh man this is incredible - I want to find a way to embed this into Emacs, this is so useful.

<comment><USER23> You can modify Emacs so that it does something on every change in the current buffer, like for example save the change to disk if it's a file-backed buffer.

You could then have a script refreshing the browser on every file change (do it from outside Emacs, calling shell scripts from Emacs or anything 'non- Emacsy' from Emacs is too slow for the usecase here).

However you'll probably have to find a way for the 'save file on every buffer modification' to be very fast otherwise it's going to annoy you and prove impractical.

Btw I can't wait for someone to implement at least part of Emacs in JavaScript so that, eventually, users used to Emacs shall have the possibility to have a Web text editor that doesn't s*ck. Maybe using ClojureScript + the implementation of Emacs redone in Clojure (I think there's a project trying to do that right now).

</comment></comment><comment><USER24> Great work. (love things that are "clean")

        Press ⌘ + i to toggle fullscreen view

I don't have this key and suspect a lot of others don't either. ;)

<comment><USER3> I just realized that after HNing it :( Terrible oversight. However, the tooltip when you hover over the arrow button is OS-appropriate <https://gist.github.com/4321208#file-scratchpad-js-L344>

</comment></comment><comment><USER25> This is awesome! I constantly look for live editors but haven't found a good one for front-end. I tried Adobe Brackets, Firebug (for in-window editing) but all had their complexities and constraints. Scrathpad.io is by far the most intuitive and best one for HTML and CSS out there!

<comment><USER5> Have you tried Plunker (<http://plnkr.co/edit/?p=streamer>)? DEMO: [http://plnkr.co/edit/?p=preview&s=HN4933608](http://plnkr.co/edit/?p=preview&s=HN4933608)

</comment></comment><comment><USER26> Not to detract from what is a pretty cool project, and I hope OP had fun making it, but: <http://codepen.io> is this and quite a bit more. It supports HTML+JS+CSS as well as e.g. Jade, SASS and CoffeeScript- if that's what floats your boat- and some nice social features, like featured pens and saving to Gists.

Other commenters have pointed out a myriad of similar such services. I'm surprised Codepen hasn't come up already.

@brycecolquitt, you are obviously very talented, and I look forward to whatever you come up with in the future. I just hope it's not in a space as hotly contested as this.

<comment><AUTHOR> Thanks, but this was actually made by my friend Nathan Bashaw (nbashaw). I just submitted it because I think it's awesome :) I think the tool is great for a certain group of people: beginners and front-end folks who want to quickly see what something looks like or do some debugging.

I'm not sure if codepen supports the realtime google-docs-style collaboration, either, which Scratchpad is good for. It's simple product with no superfluous features.

</comment></comment><comment><USER27> Very nice. Love the responsiveness and good use of (OS-aware?) keyboard shortcuts.

Bug: (Chrome Version 23.0.1271.97 on OSX)

Expand the side panel and select a document from the RECENT list. The editor area doesn't update completely; the text loads, but there are only line numbers up to the length of the previously viewed document and it's impossible to move the cursor below the last line number (<http://i.imgur.com/h9nMh.png>).

EDIT: Correction, the line numbering is correct, the problem is with the line- height in the editor.

<comment><USER3> Whoa! Thanks for sending this in! Crazy bug.

</comment></comment><comment><USER28> Absolutely amazing. A couple things that bother me, and they might just be from Ace.js:

When typing text into a
 or an
 for example, if I type "I'm", I get the smart double quote feature, and end up typing:

I'm going to the zoo'

Likewise, if I go back into the stream of some text in a
, and want to wrap it in a span, if I click at the start of it and type , I get another span on the other side of my cursor, pushing the existing text over.

Otherwise, utterly fantastic tool. Trying to use it for real work already :)

<comment><USER3> Thanks so much, Scott! That is indeed an ace default, but I can try and figure out how to override it. Syntax completion is always tricky.

</comment></comment><comment><USER29> Pretty nice, I'll probably be using this for debugging and examples.

Bug:

Open the about panel by clicking the icon in top left (three horizontal bars). Close the panel by clicking the same icon.

Maximize the preview window by clicking the arrow in the top right of the edit panel. Click the arrow to view the code panel again.

The about panel pops out as well and the arrow button covers the show/hide about (three horizontal bars) button. Expected behavior: it should return to the view that I had before maximizing.

<comment><USER3> Crazy! I thought I encountered this once in development but couldn't repro it again. Thought I was going crazy. Thank you so much!!

</comment></comment><comment><USER30> Thank you for this! I'm in the process of teaching my brother HTML/CSS, and this is the perfect tool for when he is not sitting next to me.

</comment><comment><USER31> I would love to be able to work in realtime (a la Etherpad) with someone else - would take pairing to a whole new level. Also, style management could quickly get messy - I know you are trying to keep the UI as simple as possible, but I would love to be able to quickly toggle between HTML and CSS on different tabs (instead of scrolling).

</comment><comment><USER5> Nicely done on Scratchpad. The interface seems to be the right combination of simple and intuitive. It would be great to see the source code as many others have said.

I'm particularly curious as to how you've done the collaborative bit given that you don't have explicit control over the order of operations of edits.

</comment><comment><USER32> I would assume realtime editing will be supported by the browser developer tools fairly soon.

</comment><comment><USER33> Cool! I'm going to use it for debugging.

Quick thing: I have a page quickramen.com/test/test that has links that navigate around the page. If you click those now, it loads the code again in another mini-window and doesn't go anywhere.

JUST a bug to be aware of if you're keeping a log!

</comment><comment><USER34> I'm building a similar one (it also allows you to inspect html elements like using Firebug or Webkit's Dev tools), and it's a native Windows app (and really fast one!) <http://liveditor.com>

</comment><comment><USER35> "Press ⌘ + i to toggle fullscreen view"

What is this thing, I don't have such a key on my computer?

<comment><USER36> It's on Mac keyboard, on a not-Apple system it's Ctrl+i

Suggestion for the OP, maybe it's better to add it to your page (not everybody is using Apple computers)

</comment></comment><comment><USER37> How is this different from codemirror? Codemirror's preview utility easily handles this (and much more versatile in other ways):

      http://codemirror.net/
      http://codemirror.net/demo/preview.html

</comment><comment><USER38> Nice work and all, but what's wrong with a local .html file your favorite text editor, and a browser? I've been doing that for _years_ to play around with html, css, and js.

<comment><USER3> I wanted to reduce this:

1\. Create a file 2\. Open it in your text editor 3\. Edit code 4\. Switch to browser 5\. Open up the file 6\. Switch back to editor 7\. Edit code 8\. Hit save 9\. Switch back to browser 10\. Hit refresh 11\. GOTO 6

to this:

1\. Go to <http://scratchpad.io> 2\. Write code

</comment></comment><comment><USER39> I use Livejs for the same effect, but this is much smoother obviously since it can trigger on key press rather than polling the resource constantly from the web page.

</comment><comment><USER40> Seems very slow in firefox, can't keep up with my typing and missed the characters inbetween.

Not sure if it's supposed to just say 'Loading...' at the top the whole time either..

</comment><comment><USER41> Sinatra.io does something similar, also built on Firbase. I believe their code is on github if anyone is interested in implementation.

<comment><USER42> Sinatra.io seems to be down? Do you know their github address?

</comment></comment><comment><USER43> Really nice execution on this. So simple and just what I'd want if I was just starting to learn how to code. Bravo.

</comment><comment><USER44> Beautiful. Love the fact that out of habit I pressed Cmd+D to delete the line and it worked :) Nice work

</comment><comment><USER45> What would it take to implement javascript? Is it merely trivial, or was it left out for security?

</comment><comment><USER46> Really cool, kinda like brackets.io, but online!

Is there anyway to make the text editor pane resizeable?

<comment><USER3> I could definitely make a drag-to-resize thing, but I opted against it in V1 for simplicity's sake. We do have a fullscreen mode though

</comment></comment><comment><USER47> Will you be able to make new projects and make them private?

</comment><comment><USER13> It'd be great if the command tip wasn't only for Mac btw

<comment><USER3> Ah good call! It's control + i in mac

</comment></comment><comment><USER48> Very impressive. Love the dark color scheme.

LESS/SASS possible? :)

<comment><USER3> It's definitely a possibility! Actually, more like a probability. Just didn't want to build too much into V1

</comment></comment><comment><USER49> dear scratchpad: <http://scratchpad.io/j2la3taRYW#fullscreen>

<comment><USER3> Haha thanks so much!! That is awesome

</comment></comment><comment><USER50> Would like the option to import a url to edit.

</comment><comment><USER51> it will be great if we can pick some templates like bootstrap and start coding right away from the application.

</comment><comment><USER52> This is really cool. Nice work

</comment><comment><USER53> i wish i could adjust the width of the code section. otherwise it's pretty neat.

</comment><comment><USER54> Get this added to repl.it

</comment><comment><USER55> very cool!</comment>


Getting Your Video in Google News - <AUTHOR> http://wisnetsol.com/blog/getting-your-video-in-google-news/ <comment><AUTHOR> Videos are one of most important tool of SEO</comment>


A handy little guide to small talk in the Stone Age - <AUTHOR> http://www.timesonline.co.uk/tol/news/uk/science/article5805522.ece <comment><USER2> An excellent response by a linguistics Professor is here

<http://languagelog.ldc.upenn.edu/nll/?p=1186></comment>


Ask HN: Am I the only one lost interest applying YC? - <AUTHOR> Applying YC is time consuming and soul crashing at the end. More like a lottery now, what do you think? <comment><USER2> In my eyes, the application is free and has a lot of benefit in just doing it. It makes you think about your idea in depth and asks questions targeting different areas. For example, you may have thought about how you are going to get a user base, but have you thought about how you are going to get the _first_ user, the _second_ , and _third_? Have you thought _that far_ in the beginning? When I looked at the application this was the question that stumped me, because I had a general plan/idea as to how to go about getting users, but not something concrete that could help me get the first few. So now I'm rethinking my strategy and have some better ideas for my app.

I would still do it. The worst that will happen is that they say "no".

<comment><USER3> I don't know, i have the impression that the application is more about thinking how impressive you and your achievements are so far. And I don't see any benefits in this kind of self congratulatory or self disappointment exercise.

YC always said it was more about the founders than the idea, so applying is _nothing_ like doing a business model canvas or whatever. That said, I think it is still very worthy to apply if you think you are a YC-type founder. I just do not buy that applying is an exercise worthy for itself.

<comment><USER2> Personally, I don't really see it like that at all. I mean they obviously will ask you for past accomplishments so they can see if you have the aptitude, but I don't see it as an application where you solely brag about yourself.

> _YC always said it was more about the founders than the idea_

I've heard this before. I do believe it's true, but only after the initial idea. They have had some cases where the founders applied with an idea, got accepted, and then found out that their initial idea wasn't going to take off. Now they are really investing in _you_ , the founder, over your idea. The stories of Optimizely ([https://soundcloud.com/akharris/startup-school-radio- ep-3-pe...](https://soundcloud.com/akharris/startup-school-radio-ep-3-pete- koomen-jeremy-yamaguchi)) and Codecademy come to mind.

</comment></comment></comment><comment><USER4> The world of startups is ruthless. In my opinion, YC has positioned itself near its center of gravity by being a bit more ruthlessly efficient and a somewhat paradoxically a bit more humane. The humanity comes from being a more open about its process, more open about its thinking and providing a low barrier online application process. Sure, it's a lottery. But at least an introduction isn't necessary to buy a ticket.

I guess the ideal situation from both YC's perspective and a founder's is when the application is mostly exhaust fumes from impassioned existing execution: when the effort to produce the content of the application is something that the founder(s) are committed to independently of the YC process. Their goal is to add momentum not overcoming inertia.

YC is a popular game of chance. It's not the only one. It's targeted at a particular type of player, not everyone. Building something provides several other options.

Good luck.

<comment><AUTHOR> Could you please define the YC type founder?

<comment><USER5> As I understand it:

Graduated from or attended Stanford or an ivy league school (but preferably specifically Stanford). Proven track record of success. Under 30. Without family ties. Has impressive hacks they can mention on their application. Has a product with current traction. Has a product with unicorn upside potential ($1B possibilities). Product is very early in its lifecycle. Not a single founder (preferably two to three). Can sell the startup quickly (this is important if you make it to the interview) and address potential issues with the business plan.

But remember that at this point, they probably only interview 1 in 500 applications if that, so there's a lot of chance involved as well.

</comment></comment></comment><comment><USER6> IMO, if you've got your act together, it shouldn't be too difficult. The questions it poses related to your startup are ones you should be thinking about day in and day out anyways, and should have easy answers to. If you don't know how to acquire users or what your product's value proposition is, you're dead in the water anyways. The questions about your and your cofounder's accomplishments can be tricky, but in the same vein, if you can't sell yourself properly, you'll have trouble with conventional applications for college, jobs etc. anyways, let alone raising funds. On the flip side, if you can address the tough questions about your startup properly, you just might be able to do well without them, in which case, who cares whether you get accepted or not?

</comment><comment><USER7> The very premise that an idea will get you money is BS. I guess this is a cultural change and there are those who love it and those who hate it. I fall on the later and miss days where startups in garage could bring down huge corporation and it was all about building. Now, it seems most startups are part of a big corporation via funding and we mostly get to see a plethora of names ending with "ly" or an "xyz" clone. So, on paper they say stop thinking about money and yet everything becomes about it. A bit of a double standard I would say.

</comment><comment><USER8> I think it would be nice if YC would make their exclusion criteria explicit. I understand why they don't, but the pain it is causing young people who don't know any better is sad.

Stop worrying about YC. Work hard (for someone else), learn everything you can, save your money, and think a lot. When you have the resources then start a bootstrapped business and never look back.

<comment><AUTHOR> I was thinking YC is for what you just explained in second paragraph. I do remember the SV scene where guy says "i can not guide you unless you give me something to guide" May be this startup world is a big lie!

<comment><USER8> YC sells itself as an accelerator, not as a lottery.

One of the big changes that have happened over the last few years is that the resources you need to start a tech business is much reduced to the point that one skilled person with relatively modest savings can start and build a serious business. Get on with acquiring the skills and resources you need to start without needing outside investors.

<comment><AUTHOR> I do wish application reflects being accelerator by filtering people in the beginning who needs incubation. I wish there is a quick and easy way of bootstrapping the network and connections for a successful exit.

<comment><AUTHOR> Would love to hear more about your experience and what was the top 3 do and dont that you can share

<comment><USER8> Have a read of this post on my blog

[http://www.tillett.info/2015/06/24/why-i-kept-my-startup- in-...](http://www.tillett.info/2015/06/24/why-i-kept-my-startup-in-australia- and-why-it-was-crazy/)

</comment></comment></comment></comment></comment></comment><comment><USER9> Like all things in startup life - nothing is easy, everything is hard. To me it sounds like you've given up already.

If you don't have the fire in your belly to overcome challenges, then chances are you probably won't be very good at building a successful company.

"soul crushing", "lottery" are words that people who easily give up use. Where's the fire, where's the ambition?

That's one of the most important things I'm looking for in the founding team when I read applications. I want to see determination, fire in the soul, and an ambition to go out there and make amazing things happen.

A great founding team can take a shitty idea and still make a success out of it, a shitty team can take a great idea and make nothing out of it.

<comment><AUTHOR> I wish I can show you the fire in me about my startup! If you were knew me you were seen the determination. But 20 fellowship where min 10 will be remote out of 6500, dont you agree the lottery part. Can people say rejection email from YC is not soul crashing(), unless you knew it will not work out.

It's true that I gave up applying to YC but not my dreams!

<comment><USER9> If you are truly determined, and will stop at nothing to build a successful startup, then I strongly encourage you to apply for YC and don't worry about your chances or treating it as a lottery.

Yes, it sucks not getting into YC. I know, I didn't get in the first time I applied, it felt shitty for a day, but then I just went on with building my startup regardless. I applied again the following year and this time we did get in. I didn't give up or think about our chances once.

Determination is not something you say, it's something you do, especially when the odds are stacked up against you. So if you are determined, apply for YC, don't over think it, then keep moving forward with your world domination plans.

More here: [http://imars.posthaven.com/so-you-didnt-get-into-yc-now- what](http://imars.posthaven.com/so-you-didnt-get-into-yc-now-what)

<comment><AUTHOR> Did you applied with the same idea with more progress?

</comment></comment></comment></comment><comment><USER10> As with everything, you can become too focused on a formula as your 'winning solution' or 'silver bullet'. YC is a formula. There are many formulas to success.

YC is a leg up, not the ultimate be-all and end-all of getting your startup on it's feet.

Explore your other options. :)

<comment><AUTHOR> Search for "how startup funding works" by Anna Vital. You will realize that YC is uncle.

i think problem is wys is not wyg. Someone wants to surf and we are becoming the wave.

I would love to hear alternatives.

</comment></comment><comment><USER11> if the reason to not apply YC is "time consuming and soul crashing at the end", we should not even consider starting a startup at all.

for us, we apply because we realize how much we can learn (why do we know before even get in? i have talked to a lot of YC alum, and 90% of them that i talked to are independent and in-depth thinkers, who suggest me to look at things from different angles. and we could only imagine how much more we can grow if we get in).

<comment><AUTHOR> IMO there is no question about benefits when you get in, i am more focusing to process.

<comment><USER11> calculate your own opportunity cost and sure you will get an answer. Our time spent to perfect & submit the application is fun and valuable, just as a reference.

</comment></comment></comment><comment><AUTHOR> Solo founder = no insurance, idea needs to be easy to execute with lots of user base and growth 2 co founders = insurance, but not the best if technical guy is not that technical. 3 co founders = more insurance, best option, think about it even if you screw one or two you are good.

As as Solo founder do you have hope at all when it comes to applying to YC? Please no BS about startups are hard you need help thing!

</comment><comment><USER12> At some point you are going to have to sell at your startup. You are going to have to sell yourself. Selling is a soul crushing, time consuming lottery. But every once in a while its brilliant. If you think YC application is tough, try selling to customers. ;) You don't have to apply to YC, but my advice is that you should probably make peace with the idea.

<comment><AUTHOR> If you see selling your company as a lottery then you fouled your self long time ago about what you are doing in life. I do like my work/effort speak for itself. I do respect your opinion and have a peace with it.

</comment></comment><comment><USER13> For me, I am going to keep focusing on building my product. Its the best use of my time.

</comment><comment><AUTHOR> It seems most of the people afraid of commenting, thanks for the silent ack!

</comment><comment><USER14> I've always thought of YC as a cool message board site.

<comment><AUTHOR> You need to add comments and be part of the wave, isnt PG checks HN although he has no contribution to HN last two years even as an user, what a metric? One day if i made couple of million dollars with my stupid startup, i will do a better job. Especially to the part where you say "we will reject you most likely but send it anyways"

</comment></comment><comment><USER15> I didnt' even applied yet. I am not at all lose interest xD

</comment><comment><USER16> There's never just one way. Keep going!</comment>


Don't Disrupt Agile. Drop It - <AUTHOR> https://jeffdoolittle.com/2020/05/22/dont-disrupt-agile-drop-it/ <comment><AUTHOR> It's time to leave "Agile" behind and embrace the Post-agile era.</comment>


Scientists make telepathy breakthrough - <AUTHOR> http://www.bbc.com/news/science-environment-29093700

<comment><USER2> Am I the only one who sees the "holes" in this?

> without any contact between the two

Well, the "thoughts" were sent through the internet, so there's a link there

> "You can actually transmit information directly from one brain to another > brain without intervention of the senses."

I guess that if you are interpreting the information as light, there's intervention of sight.

Still interesting though, huge pontential here.

</comment><comment><USER3> This is a case where an article's title, rather than distorting an article's content, makes a claim not supported by the article at all.

In a telephone conversation, we link two brains by way of sound waves. A text- based chat uses light waves. A video call uses both. This experiment uses electrical stimulation of neurons, but in all these examples, well-understood physical methods are used to transmit information from place to place -- _it 's not telepathy as that term is defined_.

[http://en.wikipedia.org/wiki/Telepathy](http://en.wikipedia.org/wiki/Telepathy)

Quote: "Telepathy (from the Ancient Greek τῆλε, tele meaning "distant" and πάθος, pathos or -patheia meaning "feeling, perception, passion, affliction, experience")[3][4] is the purported transmission of information from one person to another without using any of our known sensory channels _or physical interaction_." [emphasis added]

How is electrical stimulation of neurons not "physical interaction"? Is execution by electric chair defined as telepathy?

</comment><comment><USER4> Same content free article was posted a while ago with a slightly different BBC link, bbc.com vs bbc.co.uk

[https://news.ycombinator.com/item?id=8277986](https://news.ycombinator.com/item?id=8277986)

It's no more informative the second time around.</comment>


How to Write a Lisp Interpreter In Python (2010) - <AUTHOR> https://norvig.com/lispy.html <comment><USER2> The article uses a simplified version of Scheme as a toy Lisp, for an exercise, which is an excellent choice, but a downside is that it could be misinterpreted as characterizing Scheme well.

First, there's what I think is an important distinction of idiomatic programming in Scheme, for which the first example in the article gives the wrong idea, with a noble lie: that Java code implies an immediate return, but the Scheme code does not.

Were that Scheme expression not in a tail position of a procedure (which it probably wouldn't be), it wouldn't have effect analogous to that of the Java code. And a one-armed `if` makes little sense in a tail position in modern Scheme, and looks erroneous.

Scheme is a full imperative algorithmic language, and doesn't have to be used in a pure-functional way, but idiomatic programming in Scheme tends to lean towards functional when practical. You almost never do premature returns quite like that in Scheme, for example. (One _can_ do most any memory-safe thing one wants, including the less-idiomatic and even the outrageously insane, but hopefully over time one learns the idiomatic ways, rather than overuses/misuses less-idiomatic features, like leaning on a crutch they don't need and that's actually harmful to them.)

Separate from that, another thing possibly misleading about this article is that their Scheme implementation doesn't do what's sometimes called "proper implementation of tail calls" (which would be required to be a real Scheme). Idiomatic Scheme will tend to use tail calls heavily, with impunity, which would not be the case if a Scheme implementation let such calls exhaust resources.

<comment><USER3> You realize the person whose code you are criticizing wrote one of the classic books on Lisp right?

<comment><USER4> You realize that the OP's concerns actually _intesify_ if the author in question is perceived as an authority on the topic, right?

<comment><USER5> You realise it's problematic to _emphasise_ typos, right?

<comment><USER4> You realize it's problematic to emphasize typos, right?

(Thanks, would edit if I could.)

</comment></comment></comment></comment></comment><comment><USER6> (An ((Even Better) Lisp) Interpreter (in Python)) [http://norvig.com/lispy2.html](http://norvig.com/lispy2.html)

<comment><USER4> Not to take away from the side splitting hilariousness of this novel meme, but I just noticed Lisp is actually one of the few languages where you _can 't_ add spurious parentheses. If you find that a, (a), ((a)), etc are equivalent, it's probably _not_ Lisp you're looking at.

<comment><USER7> Unless a is a function that returns itself

<comment><USER8> In Scheme or Clojure, but not in Common Lisp. Now, should we adopt this parentheses-exacticity as part of the definition what is a real Lisp?

P.S. Of course, a Lisp-2 saves us only from an unlimited proliferation of parentheses, but a simple (defmacro foo () 'foo) will allow you to get a couple whenever really needed.

</comment></comment></comment></comment><comment><USER9> Thread from 2016: [https://news.ycombinator.com/item?id=12777852](https://news.ycombinator.com/item?id=12777852)

2014: [https://news.ycombinator.com/item?id=7825054](https://news.ycombinator.com/item?id=7825054)

Two in 2010: [https://news.ycombinator.com/item?id=1746916](https://news.ycombinator.com/item?id=1746916)

and [https://news.ycombinator.com/item?id=1745322](https://news.ycombinator.com/item?id=1745322)

<comment><USER10> I've noticed you do these quite often, and I thoroughly approve. I just wonder, do you do all of them by hand, or have you automated the process by now?

<comment><USER9> Not automated. I look at the threads first, to make sure there's something relevant to bother clicking on.

Related: [https://news.ycombinator.com/item?id=20556927](https://news.ycombinator.com/item?id=20556927) :)

</comment></comment></comment><comment><USER11> And of course, evolution of the code with great explanations - [http://www.michaelnielsen.org/ddi/lisp-as-the-maxwells- equat...](http://www.michaelnielsen.org/ddi/lisp-as-the-maxwells-equations-of- software/) .

Particularly good is (canonical?) lisp-in-lisp code at the end.

</comment><comment><USER12> This is the final project of Berkeley's intro to cs class, cs61a.org.

It used to be writing a Lisp interpreter in Lisp.

</comment><comment><USER13> This is such a great post, and if you are new to Lisp or Compilers or PLT I would recommend investing time in it.

> I think it meets Alan Kay's 1972 claim that you could define the "most > powerful language in the world" in "a page of code." (However, I think Alan > would disagree, because he would count the Python compiler as part of the > code, putting me well over a page.)

If anyone is curious, I have some ideas on how you could more precisely define this problem and an objective mathematical way of evaluating it. Shoot me an email if curious.

</comment><comment><USER14> I wrote my own lisp interpreter in python and it was a very instructive experience, especially since you can get a simple working prototype in one day of hacking around.

Then I got stuck at recursively evaluating macros.

</comment><comment><USER15> I wrote a Golang version when I was trying to learn Go: [https://github.com/bediger4000/lisp-in-python-in- go](https://github.com/bediger4000/lisp-in-python-in-go)

This is a great article.

</comment><comment><USER16> There is btw a bug in the read_from_tokens

      >>> parse("(")
      ...
      IndexError: list index out of range

</comment><comment><USER17> Is there a rustlang version of this?

<comment> a-priori It's not a nice blog post and it's horribly incomplete, but I did write this JIT-compiled programming environment for a Scheme-like language in Rust.

[https://github.com/michaelmelanson/risp](https://github.com/michaelmelanson/risp)

</comment></comment><comment><USER18> this has been on before [https://news.ycombinator.com/item?id=1745322](https://news.ycombinator.com/item?id=1745322)

I used the same article to write lisp in C#.... worked out really nice

<comment><USER19> Discussed other times too:

[https://news.ycombinator.com/item?id=7825054](https://news.ycombinator.com/item?id=7825054) (2014)

[https://news.ycombinator.com/item?id=12777852](https://news.ycombinator.com/item?id=12777852) (2016)</comment></comment>


Firebase Pianobar Widget- shows current song in real-time - <AUTHOR> http://www.adamwadeharris.com/firebase-pianobar-widget/

<comment><AUTHOR> I thought it would be cool to use Firebase to add a widget to my blog to show what song I'm currently listening to with pianobar (a command line pandora client).</comment>


Introducing Project Seasponge: Quick and Easy Threat Modeling - <AUTHOR> https://blog.mozilla.org/security/2015/04/02/introducing-project-seasponge-quick-and-easy-threat-modeling/

<comment><USER2> I look forward to playing with this and have passed it to some of my security consultant colleagues for their comments.

<comment><USER3> Cool! The team will be excited for the feedback. They did great work so far.</comment></comment>


One startup's journey into SEO - <AUTHOR> http://cam.ly/blog/2011/01/one-startups-journey-into-seo/

<comment><USER2> Here are a few positive factors that influence your ranking:

\- Exact match keywords are an absolute killer when it comes to ranking

\- Local beats global if your target is geographic (e.g., mysite.ca will perform better than mysite.com in Canada, particularly if the .ca is hosted in Canada)

\- Number of backlinks to the specific page

\- Number of backlinks to the whole domain

\- Popularity of backlinks

\- A variety of backlinks. Your backlinks need to look natural to Google. You want a backlink pyramid, so don't have 100 PR5 backlinks, and 10 PR0 ones. The web doesn't work that way, and Google knows it.

\- For the same reason, have no follow backlinks as well. Whether they carry PR juice or not, it's unnatural for your site not to have a mix of no follow and do follow backlinks.

\- Backlinks from .edu/.gov sites

\- An aged domain name

\- Include the keyword in the url, title tag, description, etc.

\- Metatags are still important, even if Google doesn't care about them.

\- Make your site as fast as you can

\- Have an XML sitemap

\- PubSubHubbub is excellent to get your site noticed/indexed by Google

\- The number of indexed pages on your site

\- How frequently is your site updated (often is better than rarely)

\- Make your content unique. Duplicate content is not as much of a big deal as they say it is, but you definitely don't want to look like an autoblog in the eyes of Google (particularly in the early days of your site).

\- A public whois record

\- A domain registered for more than one year

\- Get your site indexed in DMOZ and Yahoo Directory

<comment><USER3> Why this list is no good:

[http://www.seo-theory.com/2011/01/07/why-you-cannot- reverse-...](http://www.seo-theory.com/2011/01/07/why-you-cannot-reverse- engineer-googles-algorithm/)

<comment><USER4> I think you (and the author) are wrong. It is a good list.

Sure Google's algorithm is continually evolving and uses 10,000 ranking factors. But the web is made of links, and links will always be trust factors for a search engine.

Google just wants to make sure they are 'genuine organic' links.

<comment><USER3> Let me elaborate, then. The list is dubious. There are some solid arguments against making such lists in the article I linked to. The web is made of links, people, companies, bots and more. What's a "genuine organic" link?

> Exact match keywords are an absolute killer when it comes to ranking

Where are these keywords supposed to be? I'm guessing exact-match domains. I don't think they matter much but that's up for debate.

> For the same reason, have no follow backlinks as well.

I think that's supposed to be "no nofollow". I dislike nofollow a lot. Some of your links are bound to end up in places that use nofollow, no need to worry about it.

> Backlinks from .edu/.gov sites

Spammers have been after those since at least 2005. I'm sure Google knows better than to place a lot of weight on those.

> Metatags are still important, even if Google doesn't care about them.

Meta keywords are dead for sure. Meta descriptions are the oldest standard way to provide a summary of your page, so you should use it. There are also many non-standard meta tags that can prove bad for your site (nocache,noindex,etc).

> PubSubHubbub is excellent to get your site noticed/indexed by Google

Dunno about that, is it any better than a regular RSS feed (and maybe a good old blog ping)?

> The number of indexed pages on your site

If you have enough "positive factors" you will have more pages indexed. How can that be a signal in itself?

> Make your content unique.

How do you measure the uniqueness of your content? Besides that, there's many types of content where uniqueness does not matter and is not expected, for example: lyrics, recipes, press releases, syndicated news and more.

(Duplicate) Content can be aggregated, filtered, commented on, mirrored, reorganised, reformatted and so on.

> A public whois record

Why does that matter? Whois obfuscation is offered by most mainstream registrars, lots of people use it. It's an effective tactic against all the unscrupulous people that mine whois records. Also plenty of ccTLDs do not expose anywhere near as much information as .com and the likes.

> A domain registered for more than one year > An aged domain name

That would be a great way to provide stale content to a lot of search queries. The rate of website creation is surely rising, why do we have to trust their content less? And with many people buying old websites in order to exploit this presumed signal, the noise has certainly gone up.

<comment><USER5> serious question - have you done a lot of linkbuilding? because i've seen probably 80% of these matter.

what does it matter if google makes 400 algorithm changes/year? you seem to be saying that this means there are no fundamentals to SEO and it doesn't follow.

<comment><USER3> How do you measure what and how much it matters?

I'm saying making lists of ranking factors is not a good idea. Not that all SEO advice is useless.

<comment><USER5> you didn't answer my question...

<comment><USER3> Neither did you.

I commented on half the items in the list to show why I disagree with such lists. Yes, I've done plenty of link-building but I don't think it gives me any more authority on the subject than the next guy.

What's a more important question: "How do you measure which ranking factors matter more?" or "Have you done a lot of link-building?"

I used to track SERPs for dozens of keywords daily for all my domains (I had a lot then), however I've stopped doing than now as I wasn't getting much out of it. I may try it again in the future but I'm waiting until I get some better ideas on signals to track.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> I've done my fair share of research on SEO as well during my startup and I can sum it up in one sentence.

Assuming equal on-site optimisations, he with more backlinks wins.

<comment><USER7> This is pretty much true on a per page basis, but advanced SEO becomes less of a page optimization campaign and more about strategziing such that the business has positive results regardless of the performance of individual words or pages.

Exact match domains also crush backlinks.

<comment><USER8> How do near-match domains do? I have [adjective][keyword].com and it certainly didn't jump into the SERPs.

<comment><USER7> Near match domains receive no special bonus.

Look at it this way: having an exact match means you're committing yourself to the search term, either by being first to it or by spending a lot of money to buy the domain from whomever was first. Either is likely to be a signal of quality. Near match, however, just means you had $8 to spare. (For similar reasons, cheap-domain.com or cheapdomain.biz do not receive any bonuses.)

</comment></comment></comment></comment><comment><USER9> Thanks for the article. I think it will be helpful.

Can I ask, either OP or anyone else, what sort of percent of your traffic comes from SEO? I've seen it stated on the web that 70%-80% of traffic comes from searches, but for me, it's more like 5%, with adwords and direct links being the rest.

<comment><USER10> It really depends on the type of business. Notably, a lot of smaller direct sales web companies have almost all of their traffic come from SEO.

Most sites I have been involved with see 50%-70% of traffic from search. More brand equity = less importance for search.

More importantly though, search tends to convert way _way_ better than really anything else (ppc included) when goals tracking (ie nav to contact page and thank you page).

You can probably give your traffic a lot of growth by investing more time into efforts with SEO in mind. Let me know if you need help. :)

<comment><USER11> This ^

I've managed sites with nearly 99% of traffic coming directly from search engines and some as low as 5%. It correlates directly with the type of site you are managing.

If you are targeting a very specific niche community, you would likely receive much better results injecting yourself into that community and becoming an authoritative figure and focusing on direct marketing then an all-out SEO campaign. However, if your selling cases for the iPhone 4, your much better off with focusing all of your efforts on conquering the top spots on Google.

</comment></comment></comment><comment><USER8> My own strategy right now is to set up a members' forum as part of my paid service, but to make reading the forum free for all. Hope that it will help with content generation.

</comment><comment><USER12> Interesting writeup; I also may buy one of your cameras since we're having twins in May!</comment>


Ask HN: Is it safe to publish sensitive data in public if encrypted? - <AUTHOR> Suppose you encrypt very sensitive and valuable data with the strongest available crypto. Would it be safe to store the encrypted data in public, for example in a blog post or a public github repo? <comment><USER2> It depends. Here are the questions that came to my mind in under 30 seconds:

* For how long must it be secure?

* Against whom must it be secured?

* What are your potential losses if it's broken?

* How can you be sure you've used a secure implementation?

* What do you think the "strongest available crypto" currently is?

* Can you be sure that the key will never be compromised?

* Where will you store the key?

* Do you need to send the key to someone else?

* Where will the encryption happen?

... and there's more.

<comment><AUTHOR> * For how long must it be secure? let's say for 10 years

* Against whom must it be secured? Everyone, including national agencies and organised crime.

* What are your potential losses if it's broken? A lot of money.

* How can you be sure you've used a secure implementation? No idea

* What do you think the "strongest available crypto" currently is? No idea, but I would guess someone has this answer

* Can you be sure that the key will never be compromised? No, but this is a separate problem, I assume here that the key is not compromised.

* Where will you store the key? Off-line

* Do you need to send the key to someone else? No.

* Where will the encryption happen? Locally.

<comment><USER3> Elliptic Curve Cryptography

<comment><USER2> One Time Pad!

</comment></comment></comment></comment><comment><USER4> My understanding is that NSA considers classified data that has been encrypted using a suitable cipher/key to be unclassified and suitable for transmission on unsecure systems/networks.

</comment><comment><USER5> Vernam cipher will come to rescue.</comment>


Crucial Facts About Covid-19 - <AUTHOR> https://www.justfacts.com/news_covid-19_crucial_facts <comment><USER2> A long, patient, rational and undramatic exploration of many studies, cogently presented - or, at least, that's how it struck me. I was particularly interested to see it address what I increasingly regard as the elephant in the room:

> Covid-19 death rates that are based upon reported or confirmed infections > grossly undercount the number of people with the disease. This, in turn, > makes the death rate seem substantially higher than reality.

It is to be hoped that we will achieve mass testing of significant percentages of populations to show those infected, symptomatic or not, and also those recovered but who never exhibited or suspected infection.

<comment><USER3> I'm not sure I agree. The article is very opinionated on how to trivialize and downplay the death wave created by the covid19 outbreak.

Comparing covid19 deaths with car accident fatalities does nothing to address the fact that close to a thousand patients are dying daily on countries like Italy or Spain who otherwise would not die.

Trying to water down death rates of older people by reporting deaths as "loss of potential life" is also highly questionable.

It's also mind boggling how they casually shove the infuriating "sure people die but the economy" talking point under the guise of "all facts".

I understand that people are desperate for good news, but twisting numbers and statistics to provide a false sense of relief is something that solves nothing nor helps anyone.

<comment><USER3> Just to put into perspective the article's bullshit comparison between covid19 deaths and car accident fatalities, in Spain the total car fatalities in 2018 were 1806 deaths.

Source: [https://www.statista.com/statistics/438008/number-of-road- de...](https://www.statista.com/statistics/438008/number-of-road-deaths-in- spain/)

That's currently 2 days worth of covid19 deaths in Spain.

In this last couple of weeks alone, more people died of covid19 in Spain than by car accidents in the last decade.

Justfacts.com is spreading misinformation.

<comment><USER2> I appreciate the points that you make, but would also point out that the article does not make reference to car accidents, but to accidental death. This would include overdoses (drug/alcohol), poisoning, falls, drowning, choking etc.

</comment></comment></comment></comment><comment><USER4> > The primary reason why the flu takes tens of thousands of lives every year

This article and its analysis uses one method to count excess mortality to flu, and a different method to count excess mortality due to covid-19. The method used for flu is inclusive, and will count more death as a result of flu. The method used for covid-19 is exclusive, and will reduce the numbers attributed to covid-19.

Counting death is hard, but there's no acknowledgement of that in this submission.</comment>


How Hard Could it Be? (Joel on Leadership) - <AUTHOR> http://www.inc.com/magazine/20081201/how-hard-could-it-be-my-style-of-servant-leadership.html?partner=fogcreek <comment><USER2> I'm loathe to criticize Joel, but:

I don't see how taking 2 days and leadership time demonstrates concern for employees. Getting someone in who can take care of the problem quickly and efficiently* seems like it's better 'staying out of the programmer's way' than taking two days.

I think the lesson from the toilet cleaning was, "Look, if I'm going to require perfect inspections, I'm sure as hell going to have an immaculate bathroom. I also recognize that I need to show you how to perform to the level I expect. So here's how it's done."

That hands-on, know-what-you're-doing leadership is what's respectable. Making an object lesson, on the other hand, is hardly laudable.

*This falls into the "why I hesitate": if Joel could do it faster and more cheaply than having someone else come in that knows what they're doing, my criticism is less harsh, though I still think he's taking the 'wrong' lesson from his military experience.

<comment><USER3> I think you're missing the real point of these exercises. A good hacker, and anybody else for that matter, performs best when he is secure in the knowledge that he is being trusted and supported by peers and management. putting up drapes is a subtle way of showing thet the CEO will do anything to make you comfortable and make sure you can get work done.

It's not about the most efficient way of cleaning toilets or putting up drapes, it's about psychology.

<comment><USER2> My point is that most good hackers I know see through things fairly easily, and it undermines the message of "you're supported" when that support is not functioning in the best way possible. For instance, why does it take "days" to realize the sun is shining through the window and then two more to have someone who's not that great at putting up blinds put them up? Get someone in, fix the problem, get them out, bing-bang-boom, and you have a happy hacker, given most hackers I know. Take 5 days for something that should take 2 at the most (1 day to see the sun issue, 1 more to put up blinds) doesn't make people "secure in the knowledge that he is being...supported."

<comment><USER3> Hackers laugh, cry, have girlfriends (or boyfriends) just like everyone else. And they are suspectible to the same psychology as everyone else. They just often don't realise it because they think the world revolves around logic.

I've managed hackers in different startups, and my experience is that they aren't that different from anyone else in this respect. The primary difference is that they think these rules don't apply to them, which of course makes it much easier to apply them.

<comment><USER2> I'm not saying they don't have emotions, but nice try. Nor am I saying there's no such thing as hacker psychology. I have no idea what you mean when you say something like "same psychology as everyone else", My point is that the psychological boost support provides still needs to be actual support.

</comment></comment></comment></comment></comment><comment><USER4> * When you do something humble - is it still humble when you waste a few more hours writing about it and showing it off to the world?

* When you do something humble and it's rather unnecessary waste of time and effort - is it positively humble or genuinely pointless?

* When you want to send the frugality message to the team - do you first spend a few Ks on the blinds, then tear them down and then ruin the ceiling trying to undo what you did previously? Is that the "I'll do anything for your comfort" message?

There are million other ways to show support and make your people feel good. Pointless and unnecessary stuff is not among those ways.

<comment><USER5> Keep in mind Joel might have gotten his hands dirty at other times as well and kept it to himself.

</comment></comment><comment><USER6> The prim sergeant cleaning the toilet story is getting old, but the rest of the article after that was actually pretty good. Persistence, reluctant readers!

<comment><USER7> The only part that bugs me is: what kind of leader does something like this and then writes about it to brag?

I think Joel's actions, by themselves demonstrate humility and the quality of being down to earth. But the article...I think it would it would have come off a lot better if someone else wrote it about Joel.

<comment><USER8> Speaking as one of the developers who watched Joel put up the blinds, I know that this one shared instance of "servant leadership" falls in line with plenty of others that go unmentioned.

Just off the top of my head, I remember one Fog Creek get-together when one of my guests (Joel barely knew her) mentioned to me that she was thirsty, and before I could do anything about it Joel rushed off to get her a glass of water. While not a huge deal, fetching water for his developers' guests is not something that Joel really has to do.

...and even if I were to assume the basest of motives (that Joel wants to keep his developers working at Fog Creek so they make him lots of money), it's still a compliment as a developer and the type of attitude that makes me enjoy working hard.

<comment><USER7> Thanks for chiming in - that means a lot.

Since self promotion is more rampant than ever, it's very hard to separate the marketing from the information.

<comment><USER6> How is self promotion different than marketing the company you own?

<comment><USER7> Jason Calacanis.

</comment></comment></comment></comment></comment></comment><comment><USER9> I'm all for leadership by example. I think that Joel led by example when he was still actively developing code.

Hanging blinds? Not so much. Hanging blinds strikes me more as an entertaining diversion from normal work than as a way to show leadership.

I enjoy Joel's writing and think he sounds like a good guy. I'd be curious to see what he did if some of his employees left Fog Creek to start a company. The reaction would be a good measure of leadership.

<comment><USER10> I think the right way to handle that crisis would be to support them in their decision, but make sure they aren't directly competing with Fog Creek. Maintain contact, and make a gentleman's agreement that they won't poach people from Fog Creek. Keeping things friendly would be far more beneficial in the long run.

A lot depends on how many people would be leaving, too. If it's 2 or 3 people, it's not a huge deal, but if it's half the company, and if they are going to be making bug-tracking software, I think they might have to try to take a harder line.

</comment></comment><comment><USER11> I think the article is right on the money. We've been inundated the past few months with articles about a__hat CEO's in the financial services industry who were absolutely clueless, allowed their people to take financial risks that have brought their companies to their knees, while taking tens of millions of dollars in bonuses.

Meanwhile, Joel says that sometimes as a CEO, it's helpful if you put up the blinds and not just say that you care about your employees. CEO's get stuff out of their way so their people can do the work.

Hell, a CEO just using the term, "servant leadership" to describe their philosophy of management is a pretty idea these days, considering what the financial sector is going through.

</comment><comment><USER12> did anyone else wonder why he is wasting so much time on money on stupid blinds.

so much for cash & time efficiency.

<comment><USER13> If he wasn't profitable and was blowing VC cash it'd be different. He's spending money he earned.</comment></comment>


ZeroDB – A Peek Under the Hood - <AUTHOR> http://blog.zerodb.io/a-peek-under-the-hood/

<comment><USER2> So, if I get this right, ZeroDB ends up being a service to persist (to it) a flat dictionary of encrypted buckets.

When a given bucket is requested and decrypted by the client, it will have the data as well as the references that let the client treat them as a B-tree, presumably because the clients put them there?

I'm taking that assumption from "The server doesn't know how individual objects are organized within a tree structure, or whether they even belong to a tree structure at all."

What does this mean for adding a node? Does the client have to traverse the whole B-tree (requesting log(n) entries) down to the point of addition? How about rebalancing, etc? It seems like the clients would be wholly responsible for maintaining the tree.

<comment><USER3> You're right, clients will be responsible for inserting data, re-balancing the tree etc.

<comment><USER4> Ouch. Now, any memory overrun bug or a hardware memory error in any client can kill index integrity (and possibly data itself, if data is on pages rather than individual objects). I wouldn't run an important database on a server without ECC memory - but ZeroDB doesn't seem to make that an option.

Are you addressing this in any way?

<comment><USER2> Well, if the indexes can all be updated with transactional integrity around the whole thing (which I think you need anyway, per thread below about having to do client-side rotates to self-balance the tree and keep log n) in theory this is no worse than a memory overrun/etc. in the original database server.

Most fatal errors just mean you lose an entire atomic update, and the type where you "successfully" write bad data imply the same sort of repairs as server-side maintenance would.

In practice, I think the biggest issue is that if you had to restore a backup, there'd be no way to restore only -your- tree, since the server doesn't know structure or ownership of the buckets. You'd have to roll back everyone in the bucket store.

I don't want to be too negative, because I think having something like this is a good idea. It's plain there are challenges to be solved here though.

Edit: though I do catch your point that N clients all maintaining the same index tree is potentially like having a non-redundant N-wide storage array-- failure rate is cumulative. Think you could possibly mitigate this with a single-source-of-truth per table/index client-side design, but that's an obvious potential bottleneck.

<comment><USER4> > Most fatal errors just mean you lose an entire atomic update, and the type > where you "successfully" write bad data imply the same sort of repairs as > server-side maintenance would.

That's right. Except, if clients maintain the tree, there must be some "maintenance" client, or each client must do some maintenance (find and fix things that go wrong).

My issue is, indeed, the cumulative failure rate; and I have enough experience with faulty hardware, especially memory, to know that important data needs reliable memory (e.g. ECC), and a "client does maintenance" model basically makes that impossible, unless you can mandate that all connecting clients have ECC memory - essentially, server class machines.

The problem with clients maintaining indices runs very deep: Uniqueness constraints are often implemented through index. If the index integrity is violated (easy to do - you have 1000 clients, anything that goes wrong in any of them - power surge, virus, ..., may corrupt the index), then it is possible for the entire database integrity to be violated.

I'm sure there are way to mitigate this, reducing the probability down to negligible (which is what ECC does - it reduces memory error probability to negligible, not to zero which is impossible); for example, you could have every client maintain their own index, not relying on any other client's index but only on their immutable rows. That would still let a client violate database integrity, but any other client would immediately notice and refuse to work with the database.

However, I have so far not seen any reference to these issues by the ZeroDB guys. I am waiting patiently.

<comment><USER2> Yeah. You mention ECC, and I was previously thinking some sort of integrity- checking solution modeled on parity bits could work. Problem with that, of course, is I think something needs to have complete knowledge of the data structure to make it work, plus now you have yet another thing to update over network.

I'm not sure that ZeroDB will necessarily have all the answers here prepackaged. I'm still waiting to see exactly how intelligent their client modules are. I doubt they're just delivering a bucket store, so I assume they'll have client-side code that'll encapsulate the BST handling. The question is whether it's naive or accounts for some of this. If nothing else, I assume michwill is probably taking notes!

At any rate, my hope is this ends up being an interesting enough solution to build some degree of pattern or best practice over to handle some of these aspects. These might be as simple (and limiting) as "one table, one source of truth, period," or "Use for write-seldom; read-often applications only," or some other thing like that.

Even if so, this could be quite useful for -some- subset of applications, or at the very least a useful step on the way to figuring out how to do this sort of thing.

Certainly there are other challenges here as well. Beyond integrity issues, there's still the information leakage issues. It's already been mentioned that a binary search leaks order information to begin with, but if you know what kind of tree is being used you leak a lot of order information as the rotates happen.

But again, maybe surmountable. I'm eagerly awaiting the source implementation so we can dig in.

<comment><USER3> > I assume michwill is probably taking notes!

Yes, you're correct! ;-)

> "one table, one source of truth, period," or "Use for write-seldom; read- > often applications only,"

Right. Another model - "write and read only _your_ private information". So, three use-cases here

> information leakage issues

I already think, for solving this we probably should just switch to ORAM. Very valid concern! That said, we cannot really deduce the order of objects referenced in leaf nodes (and there could be a thousand of them).

In any case. Expect our implementation to be extremely simple (but useful) first. Probably suitable for "users record their private info" application. Then we'll be addressing scalability issues, information leakage etc.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> I've heard nothing but bad things about ZODB. I question the wisdom of building a database on top of a python database.

<comment> the-dude I agree and I have worked with ZODB. It isn't a database, it is a mediocre persistence ... thingy. You store objects and now your database is tied to your code.

Have experienced nightmarrish scenarios when updating codebases.

<comment><USER3> I consider ZODB as more like a framework for quickly building databases. ZODB developers (we've talked to many of them) consider upgrading objects as no- problem (because they have very well understood patterns for that) etc.

While that's ok for Zope/ZODB developers, it's not for an average user. We need to wrap it up to make easily usable, without requiring developers to read tons of documentation and "Design patterns" in advance!

And, obviously, some universal json query language is needed

<comment> the-dude How are you going to manage querying ad-hoc fields? AFAIK every field within your ZODB-query needs to be indexed in ZODB first?

<comment><USER3> Indexes are created where you told them to. So, if you include an arbitrary field, it's probably not going to be indexed to start with. And yes, query is for indexed fields.

I guess, we can have another type of behavior where all new fields are indexed. It doesn't sound like a "clean way" to me, however some developers could desire that

</comment></comment></comment></comment></comment><comment><USER6> Interesting that it's built on ZODB. For those that aren't familiar, in essence ZODB gives your Python application a 'magic dictionary' which automatically persists any keys written to it. It was probably my ignorance, but while I thought the concept was interesting I ultimately abandoned using it since I couldn't find any out-of-band way of inspecting saved data. Eg, sometimes you want to run some ad hoc SQL to see what is stored in a relational database but the only way to access data in ZODB (AFAICT) is within the application itself.

It looks like ZeroDB abstracts away these issues, luckily. Is it intended to be a full-scale application data store, or something more specialized like a secrets store?

<comment><USER3> You're right, good that someone is familiar with ZODB!

We plan to make it a full-scale database. For that, we obviously require to have some convenient query language (similar or even compatible with Mongo's).

We've talked to original developers of ZODB and they (independently) suggested that having a language-independent query language is something they constantly thought of :-) In fact, it seems like many of them dream to take ZODB (or a db based on it ;-) where Mongo is now.

</comment></comment><comment><USER7> _" The server doesn't know how individual objects are organized within a tree structure, or whether they even belong to a tree structure at all. ... When a client performs a query, it asks the server to return buckets of the tree as it traverses it, in a just-in-time fashion"_

The server can however infer quite a lot from the order of bucket request, especially which buckets constitute a tree.

<comment><USER3> Yes, we think about it. This information is not rock-solid and changes with time as the tree re-balances. But for ultimate security, I think, we need something like a proper Oblivious RAM protocol

</comment></comment><comment><USER8> What are the advantages of ZeroDB over a database that uses homomorphic encryption like CryptDB?

Do they cater to different uses?

<comment><USER3> CryptDB doesn't use homomorfic encryption. It is doing sortable deterministic encryption. E.g. after encryption the server can determine whether a > b or a == b. They have a very good understanding of what data does it leak, when can you do that and when cannot.

In our db, random observer cannot see how data are ordered or whether elements are equal to each other.

Homomorfic encryption would be ideal of course, but it is slow and impractical for the moment

<comment><USER9> Actually, CryptDB does (or at least did originally) use homomorphic encryption (HE) for evaluating SQL arithmetic expressions. While the earliest suggestions for HE were terribly slow, CryptDB found one that was practical. If memory serves, there have since been HE improvements claimed by IBM and/or Google.

But as michwill states (and CryptDB acknowledges), their approach either leaks information or precludes classes of expressions. For example, their order- preserving encryption could be susceptible to a brute-force attack via 'a > b' ... '(a+1) > b' ... '(a+2) > b'

</comment></comment></comment><comment><USER10> How does this work without exposing the client keys in server memory? You still need to send the server your private key to query and decrypt data you specifically didn't make via other connected clients, right?

Edit: I phrased that wrong. How do you usefully query other user's data with only your keys?

<comment><USER3> Nope, query logic happens on the client, so server doesn't know the key. Client is the one who records data and indexes encrypted, and who uses indexes from remote.

<comment><USER10> Which doesn't sound very scalable at all. What is the use case for this?

<comment><USER3> It is scalable enough because you need to download log(index_size) rather than full index.

The use case - any private data which you'd like to encrypt on the server while preserving ability to search. Medical records, financial data, emails are some examples.

Easier to say what this database is _not_ for. Big data applications where you need to use a significant portion of data in the db (like 10%) in map-reduce queries is probably not what ZeroDB would be practical for.

<comment><USER10> I appreciate the answers. I'm genuinely curious because I've contemplated such a database on the side but could never see my way to the goal.

The final outstanding question I have though is: How do you deal with data you aren't the owner of? Is this strictly targeted at scenarios where you are the sole owner?

<comment><USER3> Good question. We thought about two cases. One is when your are sole owner of the data.

The other is when the owner of the data is some group (like JP Chase). In this case we can have server-side quotas for handling cases when one or several clients are compromised. So, in this case it's more about distributing trust

</comment></comment></comment></comment></comment></comment><comment><USER11> I definitely miss something, but...

How it is different from a simple mongodb where I store encrypted documents ?

<comment><USER3> With mongodb, you either pass your key along with query or lose your ability to search. In our case, we don't pass the key to the server, yet having the ability to query

<comment><USER11> Can you explain better how ?

And what you mean by "search" ?

<comment><USER3> For example, full text search over your text queries, or range queries for your numerical data.

The idea is that data in databases is organized in B-Trees, so we traverse those from a remote client. It is fast enough because usually only logarithmic fraction of a tree needs to be seen by the client.

<comment><USER12> OK but aren't you thus pushing much of the intelligence typically part of the database server to the database clients?

In this case, the "database server" is basically a dumb remote B-tree server, right? It's not running queries of any sort, or maintaining indices?

Not putting it down, just clarifying that the structure of this database system is very different. Or appears so, at least.

<comment><USER3> Yes, you are right. The server stores indexes (which are actually trees), but the query logic is on clients.

<comment><USER12> Very interesting. How much additional data, compared to a conventional fat server architecture, needs to be sent across the wire with a fat client? The blog post makes it sound not too bad.

<comment><USER3> Usually it's order of 100-200 KB per query but caching things like the tree root makes this footprint smaller (e.g. second query is always faster than first one).

We tested it with pretty ad-hoc parameters (just to check if it's practical at all!), will soon write an automatic optimizer for them, minimizing query time.

<comment><USER3> HN doesn't allow to reply too deep in the tree of comments, so I continue here.

ZODB on which we base is ACID-complaint, so it cares about simultaneous writes. You either don't use cache, or get invalidation requests if you do (so that you have up-to-date tree if you want to update).

Though, it sounds like the ideal situation is when each user has his own private data, so there are not so many simultaneous writes into the same tree.

<comment><USER2> Well, I think what makes it interesting is that the simultaneous write issues don't just apply to ZODB, but the indexes (and/or balancing data) inside the encrypted buckets being maintained by the clients to maintain their own bsts, per my question above. You can't just put a transaction around one bucket because they have to, together, form a consistent tree.

That's especially true when you consider that to maintain log n the tree will have to be a self-balancing tree, so any given modification can touch a larger number of nodes during rotation.

I might be missing something obvious, but if the client has to maintain the tree I have a hard time seeing how you wouldn't have to queue access into the tree for modifying operations. That seems like it could be a pretty significant issue for some applications.

<comment><USER3> I think, your points are largely correct. You indeed need to lock multiple buckets when you commit data which could be slow.

My point is that you can tolerate that if your application is something like gmail. Client A has his own tree saved on the server, not intersecting with the tree of client B. Client A probably is not going to write from multiple places simultaneously too often.

But if you have _groups_ of clients writing to the same tree, I think it's better to have some writing client which handles multiple commits of others.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER13> This is a very exciting project, but since the use cases are security related, I am hesitant to use a closed source project. Are there any plans to open source this?

<comment><AUTHOR> Yes, we plan on making this open source.

</comment></comment><comment><USER14> Do you have any comments on using this technology for (1) health information and (2) financial information?

<comment><AUTHOR> We think it could be very useful in verticals that need to comply with privacy regulations or store otherwise sensitive data (healthcare/HIPAA, education/FERPA, finance). We're in the process of exploring specific use cases, so we'll have more precise examples for you soon.

</comment></comment><comment><USER15> Nice follow-up ZeroDB. Looking forward to the beta!

</comment><comment><USER16> good to see that you are adding more info about how it works. please keep posting more info. sounds like an interesting technology.

</comment><comment><USER17> Closed source database? Good luck with your acquisition!</comment>


Faster Webpages with Cookieless Domains (With Keynote Testing) - <AUTHOR> http://symkat.com/105/cookieless-domains/

<comment><USER2> We were using cookieless domains at quizlet.com for our static assets (css, js, images) for awhile, but recently turned them off. Some of our users (enough to be of concern) wrote in saying the site looked all wrong, and on closer inspection, weren't receiving our assets.

Because we're used in a lot of school systems, and many school districts have weird rules (domain whitelists), our static domains were not resolving while our main quizlet.com was. We decided (for now) it wasn't worth investigating how many of our users were affected or if we could do a workaround.

We're still using a subdomain off our main domain (a.quizlet.com) which prevents most cookies from sending but not al.

Going forward, I'd prefer to start base64-embedding images in css (only for compliant browsers) and using more sprites to reduce the number of overall requests.

</comment><comment><USER3> What? a few bytes removed from the HTTP request and you double the speed..??? There is something I don't understand here.

I suspect the improvement comes exclusively from increasing the limit on the number of connections, by way of now using two sources instead of one, rather than from the removal of a few bytes in the HTTP request.

Unless I missed something...

<comment><AUTHOR> More or less: yes. It's the combination of both removing those excess bytes.

If you transmit 10 static files per page request, and have 256bytes of cookies you're sending 2560 bytes of cookies, or 2KiB per page load. That's on the end user's upload bandwidth too, which is typically dramatically less than their download.

The lessening of the connection limit also allows more entities to load at the same time, the combined effect is a good increase in speed. Two tweaks for the price of 1. =)

<comment><USER2> It doesn't quite work like that. TCP works in packets, so 256 bytes of cookies does not necessarily translate into an extra packet sent over the network. So if you have 10 static files per page, you might push a few of them over to an extra packet, but not all of them.

It's something worth doing, but it's not the biggest win you can get.

</comment></comment></comment><comment><USER4> A few extra data points about using subdomains vs different domain for static content (assuming that you serve your site exclusively off www.domain.com)...

OK: Google Analytics - You can use the function "_setDomainName('www.domain.com')" on your GA tracker to restrict the cookie domain to only the www.

NOT OK: Quantcast - The Quantcast tracking code explicitly forces the cookie domain to be ".domain.com", and the only way around it would be to alter their javascript and host it from your own server, which they do not allow (though I am not sure what the recourse would be). I emailed support about it, and they said they had no plans to let you specify a different cookie domain.

</comment><comment><USER5> I've played with this and the performance improvement is trivial with modern browsers/servers.

Way more hype than reality.

Before messing with cookies, when it takes over a second to receive the core webpage (before the other objects, stylesheets, scripts, images) in Chrome no less, you need to take serious look at how you are rendering the page on the server, ie. WordPress. You get enough cache misses with WordPress+plugins and your servers are going to be crying.

The article is also missing some important info for analysis, how large was each of the transmitted parts? How large was the base webpage? Was gzip compression used on the server?

Why not use localhost to eliminate transmission variances and prove with 10000 cacheless page fetches if cookie-less really helps or not more than a few percentage points?

</comment><comment><USER6> Something interesting to do is see the source for google.com, msn.com

Compressed really well and built for speed. Then view the source at yahoo.com

Links to look at for performance. yslow <http://developer.yahoo.com/yslow/>

page speed [http://code.google.com/speed/page- speed/docs/rules_intro.htm...](http://code.google.com/speed/page- speed/docs/rules_intro.html)

<comment><USER7> This is a total micro-optimization at best. Nearly every client is going to support compressed content. The difference between minified then compressed and unminified and compressed is tiny.

You are right though the difference probably does matter on the scale of Google, MSN, and Yahoo, but I wouldn't focus on it for my site.

<comment><USER8> 15% of clients don't do compression, usually because of "internet security" software altering the Accept-Encoding header.

From the talk you linked to in another comment... [http://en.oreilly.com/velocity2010/public/schedule/detail/11...](http://en.oreilly.com/velocity2010/public/schedule/detail/11792)

</comment></comment></comment><comment><USER9> I have been aware of domains like yimg.com (yahoo static content) forever, but it never occurred to me why you'd bother doing this until I saw this article. Thanks.

</comment><comment><USER10> <http://developer.yahoo.com/performance/rules.html></comment>


Playing with Go: Embarrassingly Parallel Scripts - <AUTHOR> http://collectiveidea.com/blog/archives/2012/12/03/playing-with-go-embarrassingly-parallel-scripts/ <comment><USER2> Yes, Go does make writing stuff like that nice, but I think the actual code given is pretty heavyweight. This does the same thing:

      package main

      import (
        "fmt"
        "net"
        "io/ioutil"
        "strings"
      )

      func main() {
        file_in, _ := ioutil.ReadFile("domains.txt")
        domain_list := string(file_in)

        done := make(chan bool)
        count := 0

        for _, domain := range strings.Split(strings.TrimSpace(domain_list), "\n") {
          go func(d string) {
            ipAddresses, _ := net.LookupIP(d)

            ip := ""
            if len(ipAddresses) > 0 {
              ip = ipAddresses[0].String()
            }

            fmt.Println("Mapping: ", d, "->", ip)
            done <- true
          }(domain)

          count++
        }

        for i := 0; i < count; i++ {
      	<- done
        }
      }

<comment><USER3> There's a library for that pattern: <http://golang.org/pkg/sync/#WaitGroup>

Instead of counting and using a done channel,

        import "sync"
        ...
        var wg sync.WaitGroup
        for ... {
            wg.Add(1)
            go dowork()
        }
        wg.Wait()

<comment><AUTHOR> Nice. Yeah I'm very new to Go so I know almost nothing about the stdlib libraries. Thanks for the pointer.

<comment><USER4> Tooting my own horn here, but I'm working on a book covering the Go Standard Library. Still in progress, and not at the sync package yet, but it's coming along. Check it out if you feel inclined.

<http://thestandardlibrary.com/go.html>

<comment><USER5> You know what'd be cool? To take arbitrary code in a language, pattern match on the implementation in that language of each std lib function, and actively recommend substitutes for duplicated code.

<comment><USER6> That would be very cool. I wonder how hard it would be though. At least in Go I know that the standard lib contains lots of duplicate code (primarily so that things that should be small don't require larger things as dependencies. I think the time package's String() functions use reimplemented fmt package functionality for example, since fmt is a much larger dependency than time should have.)

<comment><USER7> Yes, but outside the Go standard libraries, adding a dependency on a standard library isn't a big deal and won't add a cycle.

<comment><USER6> Yes. What I mean though is that if you are reimplementing, say, _fmt.Printf_ , such a suggestion system might correctly suggest you use _fmt.Printf_ instead, but also suggest you can use _func (m Month) String() string_ from _time_ , or something equally silly.

Since the standard libs in Go duplicate code, you would have to be careful that your suggestion system isn't picking up false positives. I think the idea has a lot of promise though.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> Go is great for stuff like this, especially when part of an actual system.

That said, if this was just an adhoc job (to figure out which domains point to a specific IP address) you can just use "xargs -P" or GNU parallel and it becomes a pretty basic shell script, along the lines of: cat domains.txt | xargs -P 1000 -n 1 host

<comment><USER9> Until last week I didn't know that xargs could invoke commands in parallel.

      xargs -n1 -P8 dig A <hosts.txt | grep -v ';' | grep $TARGET_IP | sed 's/\.\s.*//' >hosts-matched.txt

<comment><USER10> So what's the difference between xargs and parallel? I thought the point of parallel was that it was xargs with the addition of running things in parallel. But if xargs can do that already, is there any reason to use one over the other?

<comment><USER11> It's mostly about how they handle special characters.

[http://www.gnu.org/software/parallel/man.html#differences_be...](http://www.gnu.org/software/parallel/man.html#differences_between_xargs_and_gnu_parallel)

</comment></comment></comment></comment><comment><USER12> My attempt with Ruby and Celluloid.

<https://gist.github.com/a803d86234e8d1fc5496>

I also include a list of 100 domains in a domains.txt if anyone wants to try for themselves.

        require "socket"
        require "celluloid"

        class IPGetter
          include Celluloid

          def get(url)
            Socket.getaddrinfo(url, "http")[0][2]
          end
        end

        pool = IPGetter.pool(size: 100)
        ips = {}

        File.open("domains.txt").each_line do |line|
          line.chomp!
          ips[line] = pool.future.get(line)
        end

        ips.each do |url, ip_future|
          puts "#{url} => #{ip_future.value}"
        end

<comment><USER13> Or Erlang

[https://gist.github.com/bb01a85404e6b445dcb3#file_resolve_do...](https://gist.github.com/bb01a85404e6b445dcb3#file_resolve_domains_plist.erl)

        % -*- erlang -*-
        %%! -smp enable

        worker(Hostname) ->
            {ok, IP} = inet:getaddr(Hostname, inet),
            io:format(
              "~s => ~s~n",
              [ip_to_string(IP)]
             ).

        ip_to_string({N1,N2,N3,N4}) ->
            io_lib:format(
              "~w.~w.~w.~w",
              [N1,N2,N3,N4]
             ).

        main([DomainFile]) ->
            {ok, Bin} = file:read_file(DomainFile),
            String = binary_to_list(Bin),
            Domains = string:tokens(String, "\n"),
            plists:foreach(
              fun(Domain) -> worker(Domain) end,
              Domains
            ).

This uses <https://github.com/eveel/plists/>

</comment></comment><comment><USER14> Scala is really great for this too. The downside is the spin up time for the JVM, but the upside is that if you use SBT's script launcher it will compile and cache the script transparently for you , you can still pull in _any_ JVM dependency and you can run it just like a shell script. I needed to test for a port being open in parallel and it was a cake walk to use NIO's SocketSelector to do it reactivly.

</comment><comment><USER15> This kind of a thing is also very easy in Python if you use Gevent.

I have used Gevent a lot over the last couple years and see a lot of similarities in Go's concurrency, which I think is great.

Concurrency doesn't have to be about insane looking code!

<comment><USER16> With concurrent.futures [http://docs.python.org/3/library/concurrent.futures.html#con...](http://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor)

it could look like:

      from concurrent.futures import ThreadPoolExecutor as Pool
      from socket import getaddrinfo

      def lookup(domain):
            try:
                result = getaddrinfo(domain, 80)
            exception Exception as e:
                print("error %s -> %s" % (domain, e))
            else:
                print("done  %s -> %s" % (domain, result))

      nconcurrent = 20
      with open('domains.txt') as file, Pool(nconcurrent) as pool:
            for domain in (line.strip() for line in file):
                pool.submit(lookup, domain)

To run multiple processes instead of threads, change the import to ProcessPoolExecutor.

To support multiprocessing.Pool (for Python 2 where concurrent.futures is not in stdlib), replace pool.submit() with pool.apply_async() and use contextlib.closing() around the Pool().

</comment></comment><comment><USER17> To be fair to Ruby, it releases the GVL when a thread is blocked on IO; so for this kind of task it wouldn't be an issue, probably.

</comment><comment><USER18> To be fair to languages without such great parallelism support: you can do this using asynchronous/event-loop-based code because the parallelism will be limited by the nameserver anyway (the calling code does almost nothing, it mostly waits for the net / the nameserver).

<comment><USER19> > To be fair to languages without such great parallelism support: you can do > this using asynchronous/event-loop-based code

Well you can _do_ with callbacks anything that you can do with channels and goroutines. Go's primary appeal is that it makes concurrent[1] code easy to reason about, not that it enables you to do anything that you "couldn't do" otherwise.

Continuations are just GOTOs, and just like GOTOs, some people love them and some people hate them, but even people who like them can find them difficult in large doses. Goroutines and channels are nice, because they fit the structure of imperative code, whereas callbacks sort of resemble imperative code but "inside out".

[1] Note that I didn't say parallel!

</comment></comment><comment><USER20> in the DNS case asynchronous event handling would be super easy to do. in python asyncore with something like dpkt to construct and read DNS lookups works like a champ, as does twisted. i did a simple async DNS resolver in pure python (asyncore, dpkt) and can sustain thousands of lookups a second. GNU adns also has bindings in various languages.

you can get Go's parallelisms via CSP (e.g. python-csp, ruby-csp) and replace a lot of fragile threading/parallel code with it. i've been doing that in lieu of learning Go (i know i know .. i'm lazy) and been very pleased.

anyhow, many ways to skin cats. those are just two or three.

<comment><USER21> You wouldn't do DNS lookups asynchronously in Go to begin with. Modeling concurrency of any sort in Go the way you would with an event loop is usually a code smell.

<comment><USER22> Not sure if I understand your point, can you explain a bit more please? Are you saying event loops with select {} are considered code smell?

<comment><USER21> If I had to simultaneously generalize the idea and make it specific enough to explain it further, I'd say fiddly callback state machines are a code smell in Go.</comment></comment></comment></comment>


My DNA has been kidnapped and I've been genetically raped - <AUTHOR> http://dna-explained.com/2014/04/06/data-mining-and-screen-scraping-right-or-wrong/

<comment><USER2> Data mining (the analysis step of the "Knowledge Discovery in Databases" process, or KDD),[1] an interdisciplinary subfield of computer science,[2][3][4] is the computational process of discovering patterns in large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics, and database systems.</comment>


Scan, clean website in seconds, is the anti-malware network for websites - <AUTHOR> https://virusdie.com <comment><USER2> I'm no fan of Web Application Firewalls. I remember horrible problems because Microsoft IIS thought that certain entries to a password form were attempts at CSS or SQL injection attacks.</comment>