Better bash in 15 minutes - <AUTHOR> http://robertmuth.blogspot.com/2012/08/better-bash-scripting-in-15-minutes.html <comment><USER3> If you're using:

        set -o errexit # or set -e if you prefer

Then you probably also want:

        set -o pipefail

Otherwise, it only checks that the last command succeeds, so something like:

        ls *.ssjkfle | wc -l

will actually continue as success despite the "ls" failing.

</comment><comment><USER4> 'set -o nounset' is a must have. I Just suffered this script from Samsung Printer setting Utility. sudo ./uninstall wiped the /opt

      DEST_PATH=/opt/$VENDOR/$DEST_DIRNAME
      #remove destination
      VERSION=`cat "$DEST_PATH/bin/.version"`
      if rm -fr "$DEST_PATH"; then
      	echo "INFO: $APP_NAME (ver.$VERSION) has been uninstalled successfully."
      ...

<comment><USER5> Wow, nounset or not, doing a `rm -rf` on a variable without any check is quite irresponsible. Especially if they expect the script to be run as sudo.

</comment></comment><comment><USER6> Another useful capability I use to do cleanup is `trap`.

        function cleanup {
            ...
        }
        trap cleanup EXIT

See more here: [http://linux.die.net/Bash-Beginners- Guide/sect_12_02.html](http://linux.die.net/Bash-Beginners- Guide/sect_12_02.html)

<comment><USER7> to clean the command prompt line after CTRL-C:

        trap "{ echo; exit 1; }" INT

<comment><USER8> I don't know, if you're fearing a ctrl-c in the middle of one of those newfangled moving, colourized progress bars (hello, npm), "reset" might be more appropriate?

<comment><USER7> sure, if you do coloring/bold/underline/inverse in the script, reset would be appropriate. but I wouldn't do it by default, because it's quite slow ('time reset' takes 1s here)

<comment><USER9> "tputs reset" has much the same effect (I think; there are some cases I've noticed it not fully restoring things) without the delay.

</comment></comment></comment></comment></comment><comment><USER10> Use:

      #!/usr/bin/env bash

Instead of:

      #!/bin/bash

This makes the script more portable as you don't rely on bash (or any executable) to be in /bin.

[http://en.wikipedia.org/wiki/Shebang_(Unix)#Portability](http://en.wikipedia.org/wiki/Shebang_\(Unix\)#Portability)

<comment><USER11> Are there any situations where you wouldn't be able to find bash in /bin/bash? I haven't ever seen such a system, but I'd like to know if it's something I might run into...

The other thing is, if you're targeting obscure systems, wouldn't it be just as likely that there wouldn't be a /usr/bin/env, /usr/ might not be mounted, or that bash might not be installed at all?

I suppose what I'm asking is: for practical purposes, is the lowest common denominator /usr/bin/env or is it /bin/sh?

<comment><USER12> The lowest common denominator is /bin/sh. But bash has a lot of niceties, and the POSIX shell can get the job done, but that lowest common denominator is pretty low. You're still going to be dealing with annoying differences on the platforms anyway.

I don't use env on my shebang lines specifically because it's then PATH ordering dependent (there used to be a thing where /usr/local/bin and GNU tools were _last_ in root's path, but _first_ in non-root users' path). I'm more confident (perhaps incorrectly) that /bin/bash or /usr/local/bin/bash is what I expect it is vs the first thing found in PATH that is named bash is what I think it is (however, this applies moreso to coreutils-like things, that have different SysV vs BSD semantics/options, vs a shell such as bash, which is known everywhere to be GNU bash). Some tools, like ssh, can propagate environment settings based on local, remote or config file settings, and I'd rather not be surprised.

This used to be a bigger deal on systems that put reasonable (where "reasonable" == "what you're used to") tools in /usr/ucb or /opt/gnu, rather than system paths. If you're going to create something that's intended to be "portable", you've got bigger fish to fry than if and where bash is available, and it's wise to abstract system differences to different scripts (run.linux.sh, run.freebsd.sh, run.osx.sh, run.aix.sh, etc) than try to keep everything in one massive script anyway.

<comment><USER13> Yes, using PATH for determining bash location is quite vulnerable to all kinds of security exploits.

</comment></comment></comment></comment><comment><USER14> I'm glad this included the "Signs you should not be using a bash script" section. Bash is a very good solution for many cases, but it becomes downright unruly when dealing with a lot of string manipulation and more complex objects.

<comment><USER15>
        your script is longer than a few hundred lines of code
        you need data structures beyond simple arrays
        you have a hard time working around quoting issues
        you do a lot of string manipulation
        you do not have much need for invoking other programs or pipe-lining them
        you worry about performance

It's not a sign you shouldn't be using bash. It's a sign you shouldn't be using ONLY bash.

People who insist on rewriting an ENTIRE program in Python, Perl, or Ruby fail to understand the Unix philosophy (this is a real misunderstanding I've encountered in my work, not a straw man).

You can just write the complex part in another language, but keep the rest in bash. In other words, main() stays in bash. Python et. al. is used as just another process. bash is a language for coordinating processes.

You don't want a 2000 line bash script. But it can be worse to have a 5,000 line Python script that shells out to tons of other utilities, or awkwardly and verbosely reimplements 'xargs -P' or 'sed -i'. Often you can do the same job with 500 lines of bash and 500 lines of Python (or C), and that is the ideal solution.

Python and bash are pretty radically different languages, and they are not interchangeable (the advices "just rewrite in Python" seems to imply they are). You can use each one for the things they are good at.

<comment><USER16> Modularity is nice, but it's generally easier to debug a program in a single context.

I work with some deployment systems that chain together small scripts in a bunch of different languages. They are a nightmare to troubleshoot.

I'd much rather the spaghetti was all on one plate than follow it from table to table...

<comment><USER15> If the tools are coherently designed, it should be easier to debug, because you can just use -x to log the commands being run and paste them in to see what went wrong. It's debugging with a REPL.

The biggest mistake I see people making is to hard code paths, ports, user names, configuration, etc. inside Python/Perl scripts. All that stuff belongs in shell. All the logic and control flow goes in the scripts. If it's factored this way, then you have a very natural way of testing the scripts with test parameters (i.e. not production parameters).

I don't doubt that there are many multi language shell scripts that are spaghetti. Factoring into processes is definitely a skill that takes thought and effort, and I don't think anyone really teaches it or writes about it. The only books I can think of know of is The Art of Unix Programming by ESR and the Unix Programming Environment.

But it's definitely made me way more productive once I started thinking like this. People say talk about the "Unix philosophy" for a reason. It's a real thing :) It's not an accident that Unix is wildly popular and has unprecedented longevity.

</comment></comment></comment></comment><comment><USER17> I love the very last list »Signs that you should not be using a bash script«. That should be a required part of every language/tool introduction/tutorial.

So very often people lose track of when to use what tools. (Although admittedly, so very often people are forced into some tools by external constraints.)

</comment><comment><USER18> I found this web page, from the author of musl libc, very insightful:

[http://www.etalabs.net/sh_tricks.html](http://www.etalabs.net/sh_tricks.html)

Shell scripts are great, I use and write them every day (and quite advanced ones, too). But it's very hard to make a shell script robust.

Unfortunately it's hard to find a replacement that is stable and installed everywhere. Perl is pretty close. And python too, if you are careful about making your script compatible with all the different versions.

</comment><comment><USER19> A link on HN about improving bash and it wasn't instructions on how to install zsh. I'm pleasantly surprised :)

<comment><USER20> I like bash and all, but a well tuned zsh is amazing. I was hesitant for a while but it really improved my workflow in the shell.

<comment><USER21> Yep!

And thanks to oh-my-zsh ([https://github.com/robbyrussell/oh-my- zsh](https://github.com/robbyrussell/oh-my-zsh)) it really requires very little fiddling, maybe 10 minutes worth.

<comment><USER22> I'm a recent zsh + oh-my-zsh convert, and I have a severe case of "why didn't I do this before"s.

Here is a great gallery of oh-my-zsh themes: [http://zshthem.es/all/](http://zshthem.es/all/)

<comment><USER21> Yea, it's pretty awesome.

What's really neat is you can even use it on Windows.

My personal setup is Cygwin Zsh inside Console2 (tabbed cmd.exe shell) proxied via ansicon.exe (makes cmd.exe ansi color escape aware). It's not quite as nice as say Konsole on linux or iTerm2 on a mac, but it's the best I've found for windows.

Setup guide:

Setup console2 like this [http://www.hanselman.com/blog/Console2ABetterWindowsCommandP...](http://www.hanselman.com/blog/Console2ABetterWindowsCommandPrompt.aspx)

This post sets up ansicon: [http://www.kevwebdev.com/blog/in-search-of-a- better-windows-...](http://www.kevwebdev.com/blog/in-search-of-a-better- windows-console-using-ansicon-console2-and-git-bash.html)

<comment><USER23> I'm stuck on Windows at work, and landed on ConEmu as my terminal emulator (running bash, though - I've never made the jump to zsh):

[https://code.google.com/p/conemu-maximus5/](https://code.google.com/p/conemu- maximus5/)

I haven't tried Console2, so I can't provide a comparison, but here's Scott Hanselman deciding to switch: [http://www.hanselman.com/blog/ConEmuTheWindowsTerminalConsol...](http://www.hanselman.com/blog/ConEmuTheWindowsTerminalConsolePromptWeveBeenWaitingFor.aspx)

</comment></comment></comment></comment></comment></comment><comment><USER24>
      complete -r

disables Bash 'smart tab completion', which in theory is a great idea ( use tab to complete arguments or only list files applicable to the program ) but which never seems to work properly for me.

Disabling it saves a lot of frustrated tab-banging.

<comment><USER25> Something is terribly wrong with your setup if command line completion is not working.

<comment><USER26> A lot of installs come with over-complete smart completion configurations that make <tab> take several seconds (or even 10s of seconds) to complete in fairly common situations.

<comment><USER25> Please give me one or two examples of "fairly common situations" where it takes >=20 seconds for bash to respond to the tab.

<comment><USER27> Filename completion on remote systems using ssh/scp. It is a fairly common situation for me, although I wouldn't say it takes 20s. Maybe 2-5s?

<comment><USER9> Setting up SSH multiplexing with ControlPersist[1] can help quite a bit here, since after the first connection you don't have to go through the init/connection phase for subsequent completions.

You can even preemptively fire up a master connection to commonly accessed hosts to avoid the initial delay.

[1] [https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing](https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing)

<comment><USER27> Ah, that's interesting! I assumed that something like that was going on, as subsequent completions take much shorter (in the same command), but I did not know about preemptively connecting to common hosts.

Thank you very much, I'll try to play around with it :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER28> Nice here document feature I have found recently is heredoc with pipe, e.g.

      cat <<REQUEST_BODY |
      {
        "from" : 0,
        "size" : 40
      }
      REQUEST_BODY
      curl http://localhost -d @-

It allows to pass heredoc text to standard input of next command.

<comment><USER29> This is a prime example of useless use of cat. Heredoc already means "pass this as stdin", there's no need to pipe it. Your example without cat:

        curl http://localhost -d @- <<REQUEST_BODY
        {
          "from" : 0,
          "size" : 40
        }
        REQUEST_BODY

<comment><USER30> I like modularity:

        request_body() {
            cat <<REQUEST_BODY |
            {
                "from" : 0,
                "size" : 40
            }
            REQUEST_BODY
        }

        get_url() {
            curl http://localhost -d @-
        }

        request_body | get_url

I find it helps readability when you come back to it a year later. Of course, it's also easy to parameterize the body, if needed.

/readability sometimes trumps YAGNI

</comment></comment></comment><comment><USER31> are we going to get a better bash at one point? I've always felt like the only thing bash scripts are good at describing is I/O redirection. But conditionals, dealing with variables, pretty much everything else is frustrating and error-prone

I use fish as my main shell and its slightly better, but just testing things on variables can be a huge mess.

<comment><USER32> I enjoy using fish as my main shell, but as soon as I ran into a "curl oneliner install" that failed in fish (in my case Homebrew's at the bottom of [http://brew.sh/](http://brew.sh/)) and required me to jump into bash. I enjoy fish so much I continue to use it, but I am in a state of fear that I will at some point encounter some failing shell script that leaves my system in a broken state.

Do you have any recommendations to make fish play better with shell scripts intended for bash?

<comment><USER33> I used fish for about 6 months or so and loved it - with this exception. Initially I'd just hop into bash to do whatever I needed (as another comment suggests), but what was the "last straw" for me was not being to have some of the convenience functions that virtualenvwrapper exposes for working with python virtualenvs. My solution was to switch to zsh/oh-my-zsh - I'm pretty happy with it so far, although I do miss a few things from fish (namely the auto suggest/complete when typing previous commands).

</comment></comment></comment><comment><USER34> I'd really recommend using `set -x` or `bash -x script` to sanity check all the commands and expected output.

See [http://www.tldp.org/LDP/Bash-Beginners- Guide/html/sect_02_03...](http://www.tldp.org/LDP/Bash-Beginners- Guide/html/sect_02_03.html)

<comment><USER35> The article mentioned both -x and -v, didn't it? ;)

</comment></comment><comment><USER36> _Try moving all bash code into functions leaving only global variable /constant definitions and a call to "main" at the top-level._

One of my main complaints with bash.. the file is evaluated in order - <USER2> can't call a function on the line before it's declared.

This fails:

        #!/usr/bin/env bash
        bar='bar'
        foofunction

        foofunction(){
          echo 'foo'
          echo $bar
        }

Basically you have to write your entire script in reverse, and i'm unaware of a good way to get around it.

<comment><USER37> Well, what would you do instead? It's an interpreted language. When you type "foofunction" at a prompt, you don't want it to wait around in case you define that to have meaning later.

You probably want the interpreter to be smart: "Am I loading a script from a file, or am I receiving instructions interactively on the command-line?" But now there's two modes of execution, and code in a bash script won't work if you type it into the prompt yourself. That's a bit uncomfortable.

</comment></comment><comment><USER38> If you'd like to see a decently written piece of bash that incorporates many of these suggestions, check out pass, the standard unix password manager.

Project page: [http://www.zx2c4.com/projects/password- store/](http://www.zx2c4.com/projects/password-store/) Source: [http://git.zx2c4.com/password-store/tree/src/password- store....](http://git.zx2c4.com/password-store/tree/src/password-store.sh)

</comment><comment><USER39> Excellent, _bash the good parts_. More than 15 minutes though.

Googling _bashlint_ , _shlint_ turns up some discussion (bash -n, ksh -n, zsh -n, some github projects), but I doubt they cover this article's specifics - though most (all?) of it could be automatically checked. I think _some_ could be automatically added (e.g. _set -o nounset_ ) - perhaps a bash-subset (or coffeescript-style language) possible...

<comment><USER40> Try out shellcheck - there's an online checker at [http://www.shellcheck.net/](http://www.shellcheck.net/), and if you like it, the source for it is on github at [https://github.com/koalaman/shellcheck](https://github.com/koalaman/shellcheck).

</comment></comment><comment><USER41> The author uses ${} a lot more than I see in most code. Is it helpful to always use the ${var} syntax instead of simply writing $var?

I can see universal application of ${} being advantageous in avoiding accidental "$foo_bar where you meant ${foo}_bar" situations, and ${} makes it clearer that you're referencing a variable. The only cost would seem to be more typing.

</comment><comment><USER42> I also like [http://google- styleguide.googlecode.com/svn/trunk/shell.xml](http://google- styleguide.googlecode.com/svn/trunk/shell.xml) but of course some things that work well for Google might not work for you.

</comment><comment><USER43> >> This will take care of two very common errors: >> Referencing undefined variables (which default to "") >> Ignoring failing commands

Better is subjective... About half my scripts depend on those features. For default arguments, and fail early.

<comment><USER44> He's advocating fail early with "set -e". And as for the other, you can allow overrides with syntax like:

        COULD_BE_SET=${COULD_BE_SET:altvalue}

</comment></comment><comment><USER45> There is no need for long set flags, e.g. use

      set -e

and not:

      set -o err

etc.

<comment><USER46> They are functionally equivalent, but the longhand versions are clearly more readable/greppable/google-able. The longhand versions have the exact same benefits as calling a variable eg. `target_file` instead of `a`.

<comment><USER47> Usually true but not always. Calling a for loop iterator "index" instead of "i" just adds unnecessary noise, for example.

I think "set -e" in bash should become common on the same level, it's pretty rare that you really want a script to continue after an unguarded error return.

<comment><USER48> I set both those options in every script i write. So i do it like this:

      #! /bin/bash -eu

Because i do it in _every_ script, readability and greppability are not important to me; i just need to apply the flags and get on with the script. Taking up two whole lines for them just adds noise.

If i was more selective in my use of those flags, then i would agree that the long forms were preferable, for the reasons given.

<comment><USER49> One drawback with this is that if you, or someone, does

      % bash script.sh

to run your script, then the shebang line will never be seen, and your script will run with -e off. If the "set -e" is explicitly given, this won't happen.

As you can guess, I've done this by mistake. One case is after transferring or unarchiving files where execute flags get turned off by mistake. Or using utilities, like job schedulers, that are tricky in whether they run the script as an executable, or via a shell interpreter.

</comment></comment></comment></comment></comment><comment><USER50> One thing I've liked is throwing ${PIPESTATUS[*]} at the front of my PS1.

</comment><comment><USER51> Or you can just use Zsh, which is superior in any way to Bash. ;)

<comment><USER52> Except existing deployment spread :)</comment></comment>


Ask HN: Clean Lisp implementation for study? - <AUTHOR> I am currently learning Common Lisp and would like to study a well written Lisp Implementation for study to look inside how Lisp is implemented and also to learn the idioms of the language. What would the readers suggest? Thanks. <comment><USER2> T 3.0

<http://mumble.net/~jar/tproject/t3.0.tar.gz>

The most beautiful compiler for _any_ language.

P.S. Why not ask the specialists? the people hacking Lisp compilers for the love of it; unpaid and unsung heroes? Drop by #lisp or #scheme in Freenode and see who is there. Most competent Lisp compiler hackers wouldn't be caught dead in this neck of the woods; it's just us web guys, blowhards and joe nobodies.

<comment><USER2> Update.

I wrote various long posts here about Lisp hacking bibliography, this is the most recent one:

<http://news.ycombinator.com/item?id=835020>

</comment></comment><comment><USER3> I'm far from being an expert but "Lisp in small pieces" is frequently recommended.

There are a lot of free/opensource implementations to study. SBCL is widely used and ppl working on it are available in #lisp. It may be the same for ClozureCL.

<comment><USER4> ""Lisp in small pieces" is frequently recommended."

Lisp In Small Pieces covers everything except the garbage collector.

Truth to tell I haven't yet seen a "how to implement a lisp" tutorial that implements _all_ the bits - macros , gc etc. You might have to dig into source code. SIOD is just a little crufty but it has all the pieces in very little code.

<comment><USER2> Wilson handles GC perfectly. LiSP is written from the perspective of implementing Lisp in Lisp.

</comment></comment></comment><comment><USER5> If you just want to understand the basics, Andru Luvisi has written an interpreter in less than 400 lines: <http://www.sonoma.edu/users/l/luvisi/sl5.c> and an interpreter I threw together based off his design adding garbage collection & more ops in 500 (though the macro system is non-standard and maybe broken): <http://codepad.org/sQvOeqN9>

</comment><comment><USER6> Probably better to ask this question on lambda the ultimate ( <http://lambda- the-ultimate.org/> )

</comment><comment><USER7> If you just want to understand Lisp, and not platform-specific stuff, I recommend tackling it at a higher level.

Haskell is a great language to implement other languages in:

<http://www.defmacro.org/ramblings/lisp-in-haskell.html>

</comment><comment><USER8> i think this article is great, i read it, then downloaded the source to jscheme, and ported a portion of it to ruby. then i understood more about how lisp works, i got to a working repl that had + implemented.

[http://berlinbrowndev.blogspot.com/2008/07/simple-lisp- imple...](http://berlinbrowndev.blogspot.com/2008/07/simple-lisp- implementation-in-java-ode.html)

then half a year later I thought what the hell and sneaked in a production app written in clojure to one of our tomcats. for me learning something new is always about porting it to something familiar to grok the basics and then force myself to write real code, then there's no avoidance of those important details

</comment><comment><USER9> I wrote a version of McCarthy's original Lisp in Python if you want to study the roots of Lisp.

<http://github.com/fogus/lithp>

<comment><USER10> "I have touched on functional programming with the grace and emotion of a lover."

I feel dirty.

BTW - I just "got" the name of your project you clever project namer.

</comment></comment><comment><USER11> [http://www.softwarepreservation.org/projects/LISP/book/LISP%...](http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf)

Appendix B.

</comment><comment><USER12> LispKit lisp is a pretty good example. A good description exists in "Functional Programming: Application and Implementation" by Peter Henderson.

</comment><comment><USER13> Scheme 9 from Empty Space is both a small Scheme implementation and a book describing the source code. I read 60% of the book this summer, and wasn't disappointed.

Written in portable C, it implements bignums, garbage collection, (nonhygienic) macros, hashed environments.

<http://www.bcl.hamilton.ie/~nmh/t3x.org/zzz/>

</comment><comment><USER14> I actually like newLisp - not exactly Common Lisp, but I find the implementation easy to follow. check it out on www.newlisp.org. Compiles just about anywhere and comes with a nice testing framework so you can regression test your changes, etc.

</comment><comment><AUTHOR> Thanks for the comments. I have started looking at the provided pointers.</comment>


Experts worry that "phony numbers" are misleading investors - <AUTHOR> http://finance.yahoo.com/news/experts-worry-phony-numbers-misleading-070222254.html

<comment><AUTHOR> Companies like Tesla, Twitter and Facebook are especially guilty of this if you compare GAAP and "non-GAAP" numbers.</comment>


You don't need a password. Posterous fail.  - <AUTHOR> http://blog.dustincurtis.com/apparently-765 <comment><USER2> _It's possible to forge headers in certain circumstances. It's not easy. And this is the first time this has happened._

It's ridiculously easy to forge email headers. Headers are manually created whenever programmatically sending email messages. That's how messages can be sent from addresses that don't exist, like firstname.lastname@example.com or email@example.com. They don't even send a confirmation email that you have to approve before stuff is posted?

<comment><USER2> _Headers are manually created whenever programmatically sending email messages_

To clarify this a little, in case anyone isn't familiar, to send an email message programmatically, you basically just send a string with some headers and body content to the email server. Here are what the headers look like:

      Date: Sat, 13 Jun 2009 06:53:06 -0400
      From: Mail Delivery Subsystem <MAILER-DAEMON>
      Message-Id: <200906131053.n5DAr2Nv025105@jclinux>
      To: <root@jclinux>

To change the sender, all you'd need to do is change the from line. For example:

      From: Steve Jobs <firstname.lastname@example.com>

A default sendmail implementation will deliver that message all day. Email headers should never be used for authentication.

<comment><USER3> But most major domains use domain keys / DKIM.

<http://en.wikipedia.org/wiki/DKIM>

As far as I understand it, you can't fake being an SMTP server sending mail from such a domain because their emails get signed with a private key whose matching public key is published by DNS.

<comment><USER4> There are several ways to defeat DKIM here:

• If you can break DNS, you can get an NXDOMAIN reply, making recipients think there aren't any domainkeys

• If the domainkey private key is small, you can factor it. There's an article on HN's frontpage right now about this.

• If the server uses domainkeys, but it doesn't specifically verify the From: header, an attacker can still forge a message if they share a popular mail provider with their target. I don't know if this is still practical.

• Stupidity. DKIM is difficult to test, and as a security measure it would need to be tested.

An autoreponse confirmation would be immune to all of these attacks and would be trivial to implement correctly.

<comment><USER5> An auto-response confirmation would make posterous suck.

<comment><USER4> and yet DKIM is insecure for sender authentication.

</comment></comment></comment></comment></comment></comment><comment><USER6> Hey guys. I'm the cofounder of Posterous.

Yes, someone did figure out how to post to Dustin's site today. This security hole is now fixed.

We had a specific problem with the way we dealt with SPF records. Dustin didn't set any up, and there was a specific way that Robin Duckett's email server responded that caused us to flag it as a false negative for spoofing.

For the vast majority of users who use gmail, hotmail or other services, this was never an issue.

Since our launch on day one, we have taken email spoof detection very seriously. It's one of our core differentiators: to be able to securely post to your blog by emailing a single, easy to remember address. We don't want to do secret addresses or secret words.

Over the past 2 years, we've developed robust spoof detection ip and spend a ton of time trying to stay a step ahead of hackers. Fortunately, we've only had a few very specific, isolated cases where one of our sites was spoofed and each time we have improved our system.

Thanks for bringing this to our attention. We always need to be one step ahead of the hackers/spoofers, and we thank the Hacker News community for keeping us on our toes!

<comment><USER7> Hi,

Is it possible to publish the algorithms and technique you are using to prevent spoofing. It would really be a big help to us as well as every body else.

Thanks,

Al

</comment></comment><comment><USER8> I did it. Sorry Dustin. It really was me. I changed one field in outlook.

I realise Posterous requires you to "confirm" the post, I just wanted to see if you had defaulted that requirement to off.

<comment><USER5> You realize you broke the law and admitted to it.

<comment><USER9> Honest question, what law was broken here?

</comment></comment></comment><comment><USER10> This is definitely happening in the wild, as well. A friend of mine had some spam advertising a mobile phone site posted to her Posterous, which fed into her Facebook feed, etc..

</comment><comment><USER11> It _is_ easy.

      $ /usr/sbin/sendmail -f email@example.com
      email@example.com
      Subject: hi

      Spam spam spam

      ^D

<comment><USER12> You don't even need to know the command line, you can often do stuff like that just from Outlook.

<comment><USER13> You don't even need to know Outlook, you can often do stuff like that just from the command line. ;)

</comment></comment></comment><comment><USER14> Sure active users will notice spam posts but what about the long tail of customers who no longer update their Posterous blog? What happens when a 'creative' link marketer finds a way to index those sites and inject posts?

<comment><USER15> Running a spam filter on posts should work well.

<comment><USER9> I'd almost be more concerned with essentially getting DDOS'd with spam traffic to the posting address.

</comment></comment></comment><comment><USER16> While we're talking about Posterous, does anyone know why it adds a random number to the end of article URLs, as in <http://blog.dustincurtis.com/apparently-765> ? I know it's not a big deal, but I find that aesthetically unpleasing, as it kind of ruins an otherwise beautiful URL.

<comment><USER17> Google's crawler (especially the blog and news ones) requires a three digit or larger number in the url. That is why you should keep a year/date in a url.

Update: Not sure why I got downvoted, but here is the reference from Google News:

[http://www.google.com/support/news_pub/bin/answer.py?hl=en&#...</a>

<comment><USER18> Why do they want this?

</comment></comment></comment><comment><USER19> Why not let users use email certificates if they want? That's what I've got going on in Tgethr. Let users decide if extra trouble of setting up an email cert is worth it (it's not that bad), and now all of a sudden you have spam proof email discussion lists. We just check the message signature to make sure yep your message is signed as firstname.lastname@example.com or whatever and we'll accept the message.

</comment><comment><USER20> What I'm surprised is why posterous doesn't do more check on all the headers sent by the email software (X-Mailer, and so on) and ask for a confirmation if those other headers are different enough from a known correct configuration...

Of course someone who received an email from the blog owner could use that to fake all those headers but at least it would prevent people posting by simply guessing the email address.

<comment><USER21> Oh, we do that. This was a specific bug that is now fixed.

</comment></comment><comment><USER22> Strange that none noted that identity based encryption (IBE for the acquainted ones)solves this problem quite easily (more on <http://www.voltage.com/technology/ibe.htm>). Boneh and Franklin scheme was the first proposed one, but nowadays this is not only on crypto papers, but they are even RFCS for such schemes: <http://www.rfc- editor.org/rfc/rfc5409.txt>. There are even some non-commercial implementations around: <http://crypto.stanford.edu/ibe/>.

Of course, not using such full blown solutions will mean that posterous' heuristics techniques will be susceptible to all sorts of attacks, such as man-in-the-middle, relay attacks and so forth.

On the other hand, looking for solutions that are resilient to more sophisticated attacks, mostly considering IBE schemes, is quite convoluted (it involves provable security models, such as [http://www.google.com/#hl=en&q=provable+security+signatu...](http://www.google.com/#hl=en&q=provable+security+signature&aq=f&aqi=&aql=&oq=&gs_rfai=&fp=64f719c8669fe4b7) ). There are even variations on IBE, such as certificateless, which require you to trust even less people.

This is of course, assuming you are not willing to inconvenience users by making them reply a email you send them after they tried to poste. Such email would contain a custom made url (the secret) that would enable the post to actually be posted. On the other hand, this solution feels more inconvenient than using OAuth methods.

Nonetheless, not all users care about security/privacy (those that do, will always have the usual login scheme). If you chose to go other way, good luck to you. After all, people still use MD5 for security applications nowadays.

</comment><comment><USER23> Two solutions:

1\. Change from "Contributors can post" to "Anyone can post". Counterintuitive, but the first is based on email FROM, the second is moderated.

2\. Make a hash as your FROM address. Add it as an alias to send from in Gmail (or whatever you use). Send to posterous from the hash address. Your email address becomes your password.

<comment><USER24> To add an additional outgoing address to Gmail, don't you have to verify that you can receive messages at that account first?

<comment><USER25> You could presumably use the plus sign "hack" to make that a bit easier: [http://gmailblog.blogspot.com/2008/03/2-hidden-ways-to- get-m...](http://gmailblog.blogspot.com/2008/03/2-hidden-ways-to-get-more- from-your.html)

</comment></comment></comment><comment> mike-cardwell You should be able to PGP sign your emails to confirm that they're good. If an unsigned email suddenly appears, a confirmation email should be sent back to the sender address before it is posted.

</comment><comment><USER26> Twitter had similar issue in their Text -> Tweet system. People were using softwares to send Text messages and using anybody's phone number as they want. They fixed it by using 4 digit pin and I think Posterous should do the same.

</comment><comment><USER27> Not so big a deal IMHO. You can always set a pass if spammers start targeting your blog.

<comment><USER23> The password is for visiting, not for posting. If you set a password, nobody can visit your blog w/o the password.

<http://posterous.com/help/private_sites>

"You can set a password on your Posterous site so only the readers you want can see it. To see your site, a user must go to your site url and also enter the correct password for your site."

<comment><USER27> Right, my bad. Still there is an option to receive confirmation link for each post.

</comment></comment></comment><comment><USER28> Oh my I feel bad, I started this in the other post' comments.

</comment><comment><USER29> Posterous really does fail here. I can see why they would want to tolerate a little of this to preserve ease of use for their users (just like Amazon with their Kindle email address). However, there are a number of steps that Posterous can take to combat forged headers in ways that should not impact users at all. Enabling SPF, for example, would be a good start.

Technically, it's the same problem as email spam, and most of the same tools can be used to combat it. Posterous should flag posts that they aren't sure of and make users confirm them before putting them up, etc.

EDIT:

The other fix would be to use an email address that can't be guessed from the blog address. In other words, the email address is the password.

<comment><USER30> > "The other fix would be to use an email address that can't be guessed from > the blog address. In other words, the email address is the password."

You'd still be sending your password in the clear, possibly through other peoples mail servers. Not great security.

<comment><USER29> The perfect is the enemy of the good.

There is a trade-off here between security and usability. 99% security is good enough for a lot of purposes and has its place.

<comment><USER31> Except that's more like 10% or even 1% security.

<comment><USER29> Oh really? I don't think you know what you mean.

In point of fact, I just sent myself a very important password in clear text. Hack me.

<comment><USER30> The task for a spammer isn't to hack <USERS> account. It's to hack _ANY_ account.

Being able to hack any posterous account is going to be far far easier than trying to hack a particular account.</comment></comment></comment></comment></comment></comment>


New DDoS provider to Daily Stormer, 8chan taken offline by provider - <AUTHOR> https://twitter.com/alexstamos/status/1158406276570734592 <comment><USER2> I'm still waiting for sites like 8chan to be permanently housed in a darknet. I know they have an onion, but we get lost when we offer both a clearnet _and_ an onion service. It's contradictory and hypocritical. Just offer the onion service under a really well provisioned VPS (that can handle absurd amounts of traffic) and be done with trying to find a WAF[0]/CDN every time shit hits the fan

[0] [https://en.wikipedia.org/wiki/Web_application_firewall](https://en.wikipedia.org/wiki/Web_application_firewall)

<comment><USER3> 8chan and similar make no money. The reason they are able to survive is the cost to run a site is so low (with DDoS protection being critical to lowering resource usage).

Additionally, being on the clearnet is much more accessible to people than having to install Tor, find the onion address and navigate to it. Every barrier added to accessibility bottlenecks the number of people that end up on your site.</comment></comment>


WHY IS OUR SOLAR SYSTEM FLAT? - <AUTHOR> http://degreed.com/blog/solar-system-flat/

<comment><USER2> flat relative to what? the median of the sun? from what angle?

<comment><USER3> If you watch the video, which I'm guessing you didn't, because it answered your question.

Basically the question boils down to, when a lot of things orbit the same center of mass, why do their orbits all roughly align to the same plane?

<comment><USER2> I understand that the conservation of angular momentum amplifies any initial tiny spin... however... I was commenting on the fact that the sun also demonstrates precession and itself traveling thru the milky way... so the point of anything being flat is relative to what? when?</comment></comment></comment>


How did you meet your cofounder? - <AUTHOR>

<comment><USER2> Advertised for one on campus - used facebook and posted flyers all over the CS dept.

Got 15 interested guys, picked two and spent the summer with them hacking on a project. We lived and worked in a tiny apartment, all sleeping in the same room. Best summer of my life.

<comment><USER3> All sleeping in the same room?

Wow. That's "hardcore".

<comment><USER2> Yup, well there were two bedrooms, and our servers got the larger.

The room we slept in could barely fit a bunk bed and twin bed (which is what we did). Regardless, we had such different schedules, it didn't really matter. Usually when one person was asleep - the other two would be working.

<comment><USER3> How did it work out?

Can you share any links?

</comment></comment></comment></comment><comment><USER4> Our university's ACM Intercollegiate Programming Competition Team

<comment><USER5> Me too! ;-)

<comment><USER4> Really? What a coincidence! :P

</comment></comment></comment><comment><USER6> Hardest programming class I could take during undergrad. Spent weeks worth of sleepless nights hacking in a lounge working on a final projects.

I dropped out first, started a company, then came back just in time for him to drop out and start the company we're currently working on.

<comment><USER7> What class would that be?

</comment></comment><comment><USER8> Got drunk in the same pub on our first day of university

</comment><comment><USER9> He was an undergrad at the university where I went to grad school. Though he's only 1 year younger, there was a 2 year overlap because he got kicked out of college for a year for not doing his classwork.

</comment><comment><USER10> Tried a project with one and saw we worked great together. Anyone you are not sure of, try working with them on something and see what happens. If you are still not sure you haven't lost out. Plus, working on stuff is quite fun.

You really need to test all aspects of the relationship with cofounders. So deliberately acting angry to see what they do or needing help and seeing how they respond are examples.

Spark is one thing you notice if things work out. Good luck!

</comment><comment><USER11> I met one in the post office by overhearing his conversation and recognizing a fit. Another I met through linkedin by searching for a specific skillset and a third I met by asking a friend if he knew any good matches for my project.

</comment><comment><USER12> One at startup school '05; one was a friend of a friend I met at MIT (who then co-founded a YC-funded company.)

</comment><comment><USER13> I am actively looking for YC cofounders.

Seriously, please contact me if you are interested (in our project, what we're doing, etc):

nathanatlitepostdotcom

[I should clarify: by "YC cofounders" I do NOT mean co-founders to apply with me to YC. I want "YC people": cofounders who are otherwise involved, inspired by or motivated by YC.]

</comment><comment><USER14> Still looking for the cofounder..

<comment><USER13> What are you building?

</comment></comment><comment><USER15> High school math team.

<comment><USER16> Wow. I first read that as "high school math exam."

</comment></comment><comment><USER17> Lived across the hall from him in college, sophomore year.

</comment><comment><USER18> Facebook, talking about a Facebook API Ruby adapter

</comment><comment><USER19> I know him since we were about 4 years old. ;)

</comment><comment><USER20> Dinner party at a friend's apt.

</comment><comment><USER21> he hired me for my first job

</comment><comment><USER22> In 7th grade Algebra class.

</comment><comment><USER23> I haven't, yet...</comment>


Billionaire Buffett says Bitcoin almost certainly will come to bad ending - <AUTHOR> http://www.theage.com.au/business/markets/warren-buffett-says-cryptocurrencies-will-almost-certainly-come-to-a-bad-ending-20180110-h0gjhs.html <comment><USER2> > "Now when it happens, or how or anything else, I don't know. We don't own > any, we're not short any, we'll never have a position in them," he said.

"I get into enough trouble with things I think I know something about. Why in the world should I take a long or short position in something I don't know anything about?"

-

I'd believe him a little more if he had some stake in it, one way or the other.</comment>


Ask HN: why are there no angular lines in online UI/UX - <AUTHOR>

Thinking about it recently - why are there no angular lines in the UI of everything online...
There appear to be only grid-based tables.
While there could be a UI/UX where the screen real estate is cut in an angular way.
It would be interesting to experiment with this as I think that certain types of data could be central - but others could be trimmed to the corners of a screen and do not need to be presented in a square and flat table.
Any thoughts? <comment><USER2> Angular as in circles, rounded corners, spheres and generally not grid based designs?

Originally the idea came out of necessity. Computers cannot do circles. They can just do many-MANY-edged 'things' that appear as circles to us.

If you draw a simple circle in your favourite graphics program (not a vector image), save and it and zoom in; you'll notice pixels right? Well, those are the edges/lines that I was talking about.

Also, put three blocks in a row. They will be more space efficient for putting stuff into than three spheres/circles in that same area. That is why a circle of Diameter D can fit into a cube whose dimensions are DxDxD and still have room to spare.

Squares (which is what pixels on your monitor are anyway ... usually at least) are simply more space effiecient.

Though many designs do use circles to enhance or bring to attention a certain topic, message or idea.

You can though create a 'angular' design. But it'll be very much so on the creative side of things and won't fit most standard uses.

</comment><comment><USER3> Because digital screens are rectangles and the native DOM elements are rectangles. And raw photos are rectangles. And the books and posters and signs and paintings that web designers emulate are rectangles.</comment>


Show HN: Built this using Foundation 3 in something like 4 months. - <AUTHOR> http://cloudiff.com/demo It's free servers monitoring and centralized cloud management app. We're hoping to lunch anytime soon. What do you think? Thanks. <comment><USER2> In FF 16 there is a CSS issue with the logout button when you hover on the border. Not sure if that's Foundation or your implementation but it doesn't seem to happen on the new server button.

It looks nice but I really have no idea what it does. The logo links to # so I can't get to any sort of production description, value proposition or CTA.

</comment><comment><USER3> Finally an honest "I built this in X time." Gratz. What does it do? I waited a bit for it to finish loading on my phone then gave up. Unless you hit me with an immediate "this is the problem solved" I generally don't spend much time on pages.

</comment><comment><AUTHOR> We're running a bit slow, servers aren't quite coping with the amount of traffic..</comment>


Slack's direct listing: what went down with its Wall Street advisers - <AUTHOR> https://www.businessinsider.com/slack-direct-listing-what-goldman-sachs-morgan-stanley-allen-did-2019-6 <comment><AUTHOR> [https://outline.com/jzuLBD](https://outline.com/jzuLBD)</comment>


Amazon Knows What You Buy. And It's Building a Big Ad Business from It - <AUTHOR> https://www.nytimes.com/2019/01/20/technology/amazon-ads-advertising.html <comment><USER2> This is really just a way to tax their 3rd party vendors, having cloned 10,000s of their products already to compete with them.

Now Amazon will either get the sale from dominating results with their massive in-house product range, or the 3rd party vendor has to buy the sale back via advertising.

They are being investigated in Europe because of the likelihood they are selecting which products to clone based on the 3rd party vendors' sales data.

[https://www.reuters.com/article/us-eu-amazon-com- antitrust/e...](https://www.reuters.com/article/us-eu-amazon-com-antitrust/eu- regulators-want-to-know-if-merchants-hurt-by-amazon-copies-idUSKCN1M82IU)

They probably copy online services too, so you can imagine how much growth opportunity they can measure hosted on AWS by 3rd party developers.

When you think about the risks of building on a platform like Twitter who might copy your work if you're popular, with Amazon you can see this scaling from software on AWS all the way to whatever fraction of a penny profit is in shower curtain hooks.

[https://www.theregister.co.uk/2013/03/08/amazon_copies_partn...](https://www.theregister.co.uk/2013/03/08/amazon_copies_partner_products/)</comment>


Patreon scraps new service fee and apologizes to users - <AUTHOR> https://www.theverge.com/2017/12/13/16772556/patreon-drops-service-fee-plan-apology-update <comment><USER2> [https://blog.patreon.com/updating-patreons-fee- structure/](https://blog.patreon.com/updating-patreons-fee-structure/) is the original source, rather than a news site just reporting on that source.

(Submissions of the original seem to be auto-killed, probably as part of blocking "patreon.com" links. "blog.patreon.com" ought to be allowed.)

(EDIT: that seems to be fixed now; thanks to whoever fixed it.)</comment>


Service to determine site platform - <AUTHOR>

Does anyone know of a website which will accept a URI and return a description of the platform and applications used to host, deliver and build the site.  I thought I had one in my del.icio.us saves...but seems to be mia.<p>Thanks, Tom <comment><USER2> <http://www.netcraft.com/> can be used to determine platform.

</comment><comment><USER3> <http://builtwith.com/></comment>


Most frequently visited websites - not what you'd expect   - <AUTHOR> http://lsvp.wordpress.com/2007/04/20/most-frequently-visited-websites-not-what-youd-expect/ <comment><USER2> Webkinz is incredibly popular w/ kids. My cousins (12 and 20) are basically obsessed w/ them.... It's a great model: kids are hard to advertise to, so make them buy a plush toy to get on the site.

<comment><AUTHOR> Someone with kids has metioned the Lego site to me too. I guess the young childrens market is probably just as big as the Myspace Market

</comment></comment><comment><AUTHOR> Not what I expected at all.</comment>


Falldown Game in less than 1K of Javascript - <AUTHOR> http://js1k.com/demo/757

<comment><USER2> The falling down one didn't respond to the keyboard for me :(

This one is very nice:

<http://js1k.com/demo/434>

I love these little demos, they're going to be an amazing resource for anybody that wants to learn how to program in a fun way. 1K of code, a bit of time to figure out what makes it tick.

Food for lots of code hungry children worldwide!

<comment><AUTHOR> Thanks for the feedback! What browser/OS did you play it on?

<comment><USER2> Ubuntu 9.10, firefox 3.5.6

</comment></comment></comment><comment><USER3> Looks good, quite addictive! As long as we are showing off js1k games here is my submission: <http://js1k.com/demo/688>

It runs nicely on my machine (especially with chrome/safari), anyone care to give some feedback? cheers.

<comment><USER4> They hang out along the edges, otherwise cool concept. Maybe have some more colors?

</comment></comment><comment><USER5> There's about a day left on the js1k competition deadline, so there's still time to knock an entry together :D

</comment><comment><USER6> I like that -- one of my favorite TI-83 games. Here's my js1k entry (-:

<http://js1k.com/demo/283>

</comment><comment><USER7> Nice work. Runs quite well in a small FF window which suprised me with the glow you have going on.

<comment><AUTHOR> Thanks ! It's actually really smooth in Chrome, Opera and Safari even in larger windows.

It's as if Firefox was rescaling the canvas without hardware acceleration.

</comment></comment><comment><USER8> Still playable even when the ball goes offscreen!</comment>


3 Web 2.0 business models – where do you fit? - <AUTHOR> http://briancray.com/2009/04/17/3-web-20-business-models/

<comment><USER2> I thought that the Web 2.0 revolution brought about a huge wave of _user- generated content_. A Web 2.0 app doesn't necessarily have to create content or aggregate content. It just needs to make it easy for users to create the content and share it with their friends.

It seems like there are a lot of gaps in these categories.

And I still hate the buzzword "Web 2.0".

<comment><AUTHOR> Good point about user-generated content, and I'd say that this still fits under content aggregators because the content sharing is _usually_ topic- specific and driven by information. The exception to this rule are things like Facebook and Twitter where there is peer-to-peer communication only.

It's true that Web 2.0 as a buzzword can get tiring, but can we agree that there is a shift to conversational media vs. one-way media? There has to be some name to identify that shift, and Web 2.0 seems to be the word of choice for now.

</comment></comment><comment><USER3> Is there an option for "actually charges people money" if we happen to use AJAX?

<comment><AUTHOR> Not sure what you mean?

<comment><USER3> You seem to have a lot of Content This and Content That and Content The Other and I'm sort of missing the "sell things to people for money" option. (I suppose, on a reread, that I'm a Content Advertiser under your taxonomy... which seems to suggest that my business model is to be parasited off of by the Web 2.0 economy?)

<comment><USER4> Thank you, I was going to make something similar. IANABS (I am not a Business Student) but I believe that a business model involves revenue streams.

This kind of article _might_ be a useful way of categorizing Web apps, but it's name suggests it would be some kind of starting point for people thinking about monetizing their product, which it doesn't.

</comment></comment></comment></comment><comment><USER5> briancray, where do you fit in?

<comment><AUTHOR> Well, one of my apps, Nearby Tweets, fits into the Content Aggregator model</comment></comment>


How to Learn Solidity: The Ultimate Ethereum Coding Guide - <AUTHOR> https://blockgeeks.com/guides/how-to-learn-solidity/ <comment><USER2> And remember, that there's a special function called VIP() .

This is an implied function that wraps around your full code. This is instantiated if/when core founders are losing money on a function that possibly had a bug in it. This function returns the wrapped function, under the guise of a new blockchain, with invalidations applied to all 'unintentional' side effects of bad or buggy code.

It happened with the DAO, because founders lost too much money. Just be aware, that VIP() can happen to your code, regardless if your implementation is a bug or intentional. Wrong people losing money means VIP() returns successful.

<comment><USER3> To me, this turns me off to the entire tech, no matter how stupid and kneejerk that sounds. If the biggest draw to get me using your technology (immutable contracts) is actually not true, then I'm less inclined to give it a shot outside of novelty value.

<comment><USER4> Unfortunately it's not an honest portrayal of events.

Disclaimer: I was a DAO investor. The DAO held 5% of my Ether at the time of the attack.

Having said that, it's not as simple as saying: I support the DAO refund because it was in my best interest. That doesn't necessarily follow, and it didn't necessarily follow for most of the other DAO investors.

I still held 95% of my crypto $ in Ether. I would have been happy to eat the 5% loss if it would have been the right thing to do and instilled confidence in Ethereum and its leadership. There were many debates at that time about what the right course of action was, and it was a very hard decision. If saving 5% of my investment meant I ended up with a worthless 100%, it would be foolish to save that 5%. And everyone else knew this as well.

I think the decision the Ethereum team made was the best moral and legally advisable choice. They had the opportunity to stop the theft of 14% of all Ether, which would have gone to a criminal actor. It required a very risky solution (a hard fork) which came with big consequences (an alternative chain which remains to this day), so the solution wouldn't be feasible for smaller attacks (as some people like the poster before you try to allege). But for an attack which threatened 14% of the available supply, a hard fork to retrieve the funds was a viable solution.

Most of the market agrees with this decision. In the time since the Ether market cap has grown to the second largest of any cryptocurrency at 1/2 the size of Bitcoin. The alternative chain that didn't fork is still alive but only has less than 10% of the Ether market cap. The vast majority of dApps are running on the Ether chain. And it's the chain that is being taken seriously by enterprise (look up "Enterprise Ethereum Alliance"). Large businesses know the founders can't simply hard fork whenever they want to better themselves. The model works around consensus, consensus that is clearly shown by the considerations listed above, and I am glad that the consensus model is strong enough withstand an attack on 14% of its supply and respond with a difficult solution to rescue the chain from a catastrophe. It makes me feel more confidence in the technology, the community, and the protocol.

<comment><USER5> Your whole argument is predicated on the idea that what happened with the DAO was "theft".

All the publicity for the DAO explicitly said that the code was the full specification of the contract even if it disagreed with any other statements, and the code allowed someone to transfer the funds to their own child DAO. What crime, exactly, are you accusing them of committing when everyone agreed to a contract that allowed them to do what they did? How do I know using another contract won't be labeled theft and rolled back?

> Most of the market agrees with this decision.

This is exactly the problem. Ethereum would have been interesting if it could enforce unpopular decisions because they're specified in the code and them's the rules. Instead it will enforce them as long as there's not too much public outcry. No thanks.

<comment><USER2> > Your whole argument is predicated on the idea that what happened with the > DAO was "theft".

So did the Ethereum Devs, who owned a bulk of coin from the initial auction. And they just so happened to also control development of the protocol _AND_ the client programs. This was the message they "asked" users... [https://upload.wikimedia.org/wikipedia/commons/thumb/d/d7/Et...](https://upload.wikimedia.org/wikipedia/commons/thumb/d/d7/Ethereum- Wallet_and_Mist_Beta_with_Hard_Fork_choice_0.8.1.png/1113px-Ethereum- Wallet_and_Mist_Beta_with_Hard_Fork_choice_0.8.1.png)

"....IN WHICH FUNDS RELATED TO THE EXPLOIT ARE RESTORED...."

Consensus is nice, especially when you can manufacture it. Noam Chomsky talks about this in great detail. [https://www.youtube.com/watch?v=tTBWfkE7BXU](https://www.youtube.com/watch?v=tTBWfkE7BXU)

Chomsky: "The first place to look is, who's in a position to make the decisions that determines the way a society functions."

</comment></comment></comment></comment></comment><comment><USER6> One of the main Ethereum co-founders is working on a new experimental contract language targeting the Ethereum Virtual Machine called Viper which still seems to be under relatively active development[1].

Why pick Solidity if Viper and other languages like Low-level Lisp-like Language (LLL) by Consensys[2] are around the corner? Or is this just a matter of language flavour similar to how anybody can pick from a plethora of languages targeting the JVM (Clojure, Scala, Java, etc.)?

[1] [https://github.com/ethereum/viper](https://github.com/ethereum/viper)

[2] [https://media.consensys.net/an-introduction-to-lll-for- ether...](https://media.consensys.net/an-introduction-to-lll-for-ethereum- smart-contract-development-e26e38ea6c23)

<comment><AUTHOR> Viper is not completed, and it might never be completed.

Solidity is here now, and the demand for Solidity developers is growing by the hour.

<comment><USER6> _> >[...] it might never be completed._*

"Complete" is certainly in the eyes of the beholder, but Viper's author seems to think the language is "complete enough" to have started writing Ethereum's new Proof-of-Work/Proof-Of-Stake hybrid system (aka Casper contracts[1]) in Viper[2][3].

[1] [https://www.ethnews.com/proof-of-stake-vitalik-buterin- share...](https://www.ethnews.com/proof-of-stake-vitalik-buterin-shares- casper-contract-code)

[2] [https://ethereumclassic.github.io/blog/2017-03-13-viper/](https://ethereumclassic.github.io/blog/2017-03-13-viper/)

[3] [https://github.com/ethereum/casper/blob/master/README.md](https://github.com/ethereum/casper/blob/master/README.md)

</comment></comment></comment><comment><USER7> I've been using Solidity for a bit. I wouldn't say it is my favorite language, it's a bit too javascripty for my personal preference, and its genuinely hard not to introduce subtle security flaws into the code. As a smart contract language it leaves a lot to be desired. That being said, it is really the only game in town on Ethereum right now. The other language implementations on the EVM are in various states of disuse. If you are interested in smart contracts on Ethereum you have to learn Solidity unless you want to reinvent the wheel with your own language implementation.

<comment><USER8> well said, agree with you!

</comment></comment><comment><USER9> Last year I was pretty excited about Ethereum for a few months. But then I got to thinking about the social problems I care about, like how factory and middle class jobs are being wiped out by robots and AI, and I couldn't think of how DAPS could fix those sorts of problems on a large scale.

<comment><USER10> People are not meant to work in factories. They're meant to own parts of factories and other productive enterprises. Ethereum reduces the barriers to entry to becoming a stakeholder in value created by enterprises. The cost to exchange a token is 1000X less than the cost to exchange a security. This is an orders of magnitude reduction in barriers to entry and expansion of liquidity. It means the economy can grow in all directions and become more inclusive in its ownership structure.

We're still very early days, but this is what distributed smart contracts make possible. The second and third order consequences of this will be enormous.

<comment><USER9> Do you mean people won't work in factories but instead sit around and collect profits, or do you mean people will work in factories but own them cooperatively, or that people will work but not in factories but in some other type of economic enterprises, or what?

And could you give me a link where this is described in more detail, including addressing various possible objections?

<comment><USER10> I mean people will sit around and collect profits.

I don't have any links about this. This is my own view on an ideal future. I haven't come across any objections to it.

<comment><USER9> >I don't have any links about this. This is my own view on an ideal future. I haven't come across any objections to it.

This is what I am talking about. Ethereum supporters think it is going to make the economy into an anarchist-libertarian techno-paradise, but they don't present specific ideas and write them up so they can be subjected to critical scrutiny. You know, lots of ideas seem great until you try to work them out.

My own thought is that economics starts with basic needs and desires, like for food, shelter, health care, internet access, and so on. Today these are provided largely by small and medium enterprises, and people earn their money working for them. I don't see how most of these things could be provided by DAPS, and so I conclude Ethereum isn't going to make much difference to the average person.

<comment><USER10> Here's a more detailed write-up on it:

[https://www.reddit.com/r/Futurology/comments/61lg8g/the_robo...](https://www.reddit.com/r/Futurology/comments/61lg8g/the_robot_invasion_is_coming_and_its_gunning_for/dfgdoka/)

You're right that there are no complete road maps written out. But proponents of this vision can hardly be blamed for not being able to create a detailed plan for things they believe will be invented. It will necessarily be impossible to predict exactly what form the future will take. Yet we can make reasonable extrapolations based on the nature of knowledge, and the role knowledge plays in investment.

</comment></comment></comment></comment></comment></comment><comment><USER11> FWIW: The current Humble Book Bundle does contains the book Introducing Ethereum and Solidity[1].

[1]: [https://www.humblebundle.com/books/linux-book- bundle](https://www.humblebundle.com/books/linux-book-bundle)

</comment><comment><USER12> Are there any resources more informed readers might suggest for building a "toy" EVM language? Say based on racket? Thought it might be a good way to understand things deeply.

</comment><comment><USER13> Now this truly is a job for the future..

</comment><comment><USER14> bullish. whenever the average HN comment is negative for new tech you know it's going to be huge.

great guide here too.</comment>


Mozilla announces WebAssembly System Interface, what JVM should have been - <AUTHOR> https://www.theregister.co.uk/2019/03/29/mozilla_wasi_spec/ <comment><USER4> No slight against WASM (which I've promoted), but early Java people would be well-advised to sit down somewhere they can sob in privacy, before reading this article:

> ...you can't run Java code in a browser without a plugin...

> ...WASM, being memory safe and tuned for validation, also has security > advantages over Java applets...

> ...explained the difference between WebAssembly and Java thus: "WebAssembly > has been designed to scale from tiny devices to large server farms or > CDNs...

> ..."That's how important it is. WebAssembly on the server is the future of > computing. A standardized system interface was the missing link...

> ...But a write-once, run anywhere binary represents a worthwhile effort...

<comment><USER5> I am not surprised. As someone who cut his programming teeth on Java and .Net, modern web development feels really out of touch with reality.

I really don't get why the community is hell bent on reinventing the wheel, especially when the main defense of npm is that you don't have to reinvent the wheel.

It's really disheartening to see that the best minds of the generation are busy spending time to reinvent the same old things instead of trying to improve upon the existing systems.

<comment><USER6> .NET ecosystem is fantastic right now. .NET Core is fast and efficient, asp.net is better, faster and more full-featured than ever, and MS is ahead of everyone with Blazor Components that finally provide a real alternative to JS frontend frameworks and can run either server-side or as WASM modules.

<comment><USER7> I agree largely from a development perspective, but the client-side javascript frameworks are ultimately more performant, particularly on mobile, and are not as bad as they used to be. Neither server-side nor WASM are ideal for mobile devices. Downloading many megabytes of .NET dependencies for a WASM app is obscene compared to like 30 kilobytes for React.

<comment><USER6> It won't be megabytes, that's only in the experimental phase. Binaries can shrink dramatically once run through a linker and trimmed down to only used APIs.

As far as performant, I agree it wont matter in most trivial interactions but large datasets with computations already see an impact (for example we have million row datatables to show sometimes).

<comment><USER8> > Binaries can shrink dramatically once run through a linker and trimmed down > to only used APIs.

It's worth pointing out that .NET Core has an assembly/executable generator that can package only the .NET libraries that are actually used.

</comment></comment></comment></comment></comment></comment><comment><USER9> Regardless of any technical merit or reasoning, one big advantage is that Javascript/WASM is not controlled by any one large company or organization.

This provides substantial differentiation from .Net/Java/Flash/etc.

That said, this advantage risks diminishing if Google's influence continues to grow.

<comment><USER10> > Javascript/WASM is not controlled by any one large company or organization

It's disproportionally affected by Google.

<comment><USER11> Given that Mozilla has done tons of WASM work, and that one of the earliest implementors was Rust, comparing WASM to Oracle's iron fist over the JVM doesn't seem to jive with reality...

<comment><USER10> I didn't say Mozilla, I said Google.

I didn't say iron fist, I said disproportionately affected.

<comment><USER12> And it isn't true, because Mozilla is very involved, and every browser engine will have to get on board.

<comment><USER10> First and foremost Chrome has to get onboard. Hardly anyone cares if Mozilla implements anything.

A few people will care if Safari does.

Edge is out if the picture and is being replaced by Chromium (aka Chrome).

On WebAssembly specifically. Here's WebAssembly working group: [https://www.w3.org/2000/09/dbwg/details?group=101196&order=o...](https://www.w3.org/2000/09/dbwg/details?group=101196&order=org&public=1)

I count 18 people from Google, and only one from Mozilla. The second largest representation is from Microsoft (8 people), but their input is now Chrome/Chromium input.

</comment></comment></comment></comment></comment></comment><comment><USER13> It's kind of true -- it's designed with a recognition that computers are real things, and shouldn't be abstracted out in terms of a specific programming model. The JVM approach (somewhat by design) limits the easer of interaction with other systems.

The biggest drawback in WASM to me has alway been the memory model. While it is memory safe, it's currently opaque to the host environment which means that the general implementation is to over allocate address space and essentially implement pointers as integers. That results in pointers that can't be trivially shared between environments.

That said, I think that if I /were/ to be making an app that had plugins, or an OS from scratch, etc I would define an interface via WASM. Screw loading unbound untrusted code into my process. My code is 100% bug free _, I don 't want to other people to break it.

_ This is not remotely true :D

<comment><USER14> How do you feel about the app extension model, i.e. loading bound semi-trusted code into a sandboxed process and communicating via XPC?

<comment><USER13> App Extensions are a really good solution* to the same problem domain, with the additional constraint that you have to support arbitrary compiled code, with all the intentional or unintentional memory safety violations.

* I am biased here.

<comment><USER14> > I am biased here.

Haha, I'll defer to you then. But I'm still curious about this:

> you have to support arbitrary compiled code, with all the intentional or > unintentional memory safety violations

But there are multiple barriers here, are there not? First, your code needs to be signed (and notarized, at some point), and then at runtime the app sandbox/entitlements/process separation ensures there's not much you _can_ do even if you're running arbitrary code. Sure, the first "barrier" isn't necessarily all that restrictive or thorough, but the other one should ideally stop anything horrible?

<comment><USER13> It's not so much a problem of signed vs. unsigned code.

There's the "are you malware that's got through whatever rules exist in the target App Store", but there's also the "is the extension code buggy".

In my experience the latter is the real killer - there are all sorts of things an App Store review process can be used to prevent malware, but they're all dependent on the code doing what static analysis claims it's doing - <USER2> can't protect against terrible code -- my experience writing software has taught me that no one rights 100% correct code, 100% time.

Historically GPU drivers have been a terrible source of bugs (it's why WebGL has such tight restrictions on features and shader syntax) not because of intentional malfeasance, but because you're processing arbitrary untrusted content in a trusted environment (my understanding/belief is that the windows driver model has pushed this out of the kernel? please confirm/deny), so functionally forcing that into a restricted execution environment is a win.

Obviously the memory model of WASM still allows for things like use after free, out of bounds access, etc; but at least it's not unbound into the host address space.

<comment><USER14> I guess you're assuming that the platform/runtime that's doing the isolation is close to being correct, which we both know isn't quite true ;) The only benefit I see is that you're adding an additional level of abstraction (namely, you're executing a vetted selection of native code rather than arbitrary native code), which makes reaching the point where you can actually break things harder. Was this the point you were trying to make?

<comment><USER13> The idea is to reduce the ability of the untrusted code to go wrong and compromise the host.

Going wrong can mean "exploited by malware" through to "extension code trawls the host process address space to provide 'features'"

The latter of these two used to happen all the time with "haxies" on OS X.

The run on security benefit of providing a semi-virtualised environment for third party/untrusted code is that if the VM is exploited _you_ are able to fix and ship a fix for the VM. You can't fix the untrusted code.

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> The quotes from the Web Assembly team at Mozilla and Solomon Hykes read like these people are suffering from some kind of late 1990s amnesia. Till Schneidereit is saying exactly the same thing Sun said when Java was announced. Acting like they are breaking new ground strikes me as unhelpful and it's not clear to me how it will help the cause. Maybe WebAssembly will enjoy more success than Java, I truly hope that it does.

IMHO, the success of these products has very little to do with how good they might be technically. It's all about sales and marketing and, eventually, politics. Perhaps that's where the motivation to re-write recent history is coming from.

<comment><USER16> > It's all about sales and marketing and, eventually, politics.

Not in this case. Java failed because it sucked from a technical point of view. It traded better 'OO purity' in exchange for worse security, portability and performance.

The end result is that users got something that was slower and buggier for no gain.

<comment><USER17> Where did you get this "Java failed" idea? Java is one of the most widely used languages in the whole world...

<comment><USER16> Java failed as a portable language-agnostic VM for user-facing apps.

You know, the use-case it was originally designed for. (I read the hype for Java 1.0, I was there.)

Nobody at the time could imagine that Java would eventually become the enterprise COBOL replacement.

<comment><USER15> I'm not so sure it failed. I use IntelliJ IDEA daily, people in my office use WebStorm for Javascript development. There are Swing applications out there that people use, admittedly the ones I've seen are mostly business software. And of course all Android applications are leveraging Java.

It's argued that only IDEs use Java, but Java + Swing strikes me as the most popular cross-platform language and toolkit currently in use.

<comment><USER18> that's only if you don't consider "web/browser" to be "cross-platform"

</comment></comment></comment></comment></comment></comment><comment><USER6> I wonder what new standard we'll come up with 10-15 years from now that'll be titled "What WASM should have been".

<comment><USER19> Isn't that what the goal of software should be? Solving problems with older implementations?

Do people think WASM is not better than the JVM for streaming browser based usage? Did it learn nothing from the JVM? The sarcasm in these threads always confuses me.

<comment><USER20> Yes, because WASM is basically politics due to the refusal of other browser vendors to adopt PNaCL, and it doesn't provide any security for memory bound and null pointer exploits inside the sandbox.

<comment><USER19> So where were JVM targets for all my favorite languages, or any browser support for that matter?

I'm not going to debate any points against the JVM, but all I know is that I can compile languages I want to all browsers _today_. Why can't I do that for the JVM?

Something must be a massive, gory failure of the JVM and/or the companies involved with the JVM for it to be an equal to that of WASM, and yet have zero traction for modern languages and browser targets.

<comment><USER20> Depends on the language I guess, I can compile Java, Kotlin, Scala, Clojure, TCL, Python, Ruby, Common Lisp and any other one able to produce LLVM bitcode on the JVM.

Also 86% of the mobile OS market runs on Java.

Additionally smartcards, a big portion of electricity meters, factory automation, smart copiers, M2M gateways, Bluray Players,... run on Java.

Alongside JavaScript and .NET, it rules software development across the corporations of the world.

Hardly a failure, only for naysayers.

</comment></comment></comment></comment></comment><comment><USER21> Kind of funny. WebAesembly is basically like a pre-installed Java Browser Plugin.

<comment><USER22> Without all the bugs and security holes.

<comment><USER23> Everything was intended to be secure but no one writes perfectly secure code, same goes for WASM. Java exposed a lot of system access (files, devices, I/O) that you just don't have in WASM. In that aspect it's more secure, but it also does less. If you're going to include the browser APIs, you can argue they're just as insecure as Java given their less than stellar history.

<comment><USER22> Name any technology that loads and executed remote content with a better track record than browsers and JavaScript. At least anything with even 1% the capability.

</comment></comment></comment></comment><comment><USER24> I wish JS made it easier to sandbox itself with existing mechanics - <USER3> simply unreferencing a system API. I am looking forward to this.

</comment><comment><USER25> Java is a language backed by the JVM it tried to do two things:

Be _the_ language (Java)

Be _the_ platform (JVM)

Wasm is a compilation target, it's better to compare it to the JVM than Java. The thing that has me excited is that _every_ language will be write once run everywhere not just some new and unproven language (like Java was at the time).

</comment><comment><USER26> The JVM in the browser was supposed to be safe, too.

<comment><USER13> The underlying problem is what the threat model was - in Java/JVM a significant amount of effort was put into addressing "what if the byte code itself is malicious", with the core applet VM having quite a lot of power over the system - e.g. file I/O, GPU access, etc.

<comment><USER20> Actually only when one did not bothered to spend any effort with Security Managers, Classloaders separation and JAAS.

All the tools were there.

WebAssembly is still not safe from internal memory corruption, due to lack of memory tagging and bounds checking.

<comment><USER27> In general the groups doing standardization for browser APIs and runtimes don't seem to care much about whether web applications are compromised, only whether the browser or host platform are compromised. It's reasonable for the latter two to be priorities, but when we're talking about huge gmail-tier applications running unsafe C in a sandbox that have access to All Your Important Data, we're going to massively regret letting type safety and other features slide.

<comment><USER16> > ...we're going to massively regret letting type safety and other features > slide.

PHP is memory safe, and yet is a larger source of data breaches and security bugs than C by (rough guess) an order of magnitude.

C is not the bogeyman you're looking for.

<comment><USER28> PHP may be memory safe but the shitload of poorly written C extensions enabled by default on all hosts that shipped with the distribution from 1995 to 2010 sure as hell weren't.

<comment><USER16> C extensions wasn't the part of PHP that broke the amateur Internet, SQL injections were.

</comment></comment></comment></comment></comment></comment></comment><comment><USER29> Is there a link to the WASI spec?

<comment><USER30> This seems to be the current documentation: [https://github.com/CraneStation/wasmtime/blob/master/docs/WA...](https://github.com/CraneStation/wasmtime/blob/master/docs/WASI- documents.md)

</comment></comment><comment><USER31> Why do they call it webassembly if it doesn't even have goto.

</comment><comment><USER32> This previous thread covers differences between the JVM and WebAssembly:

[https://news.ycombinator.com/item?id=19502702](https://news.ycombinator.com/item?id=19502702)

tl;dr

\- JVM doesn't do enough to separate computation and I/O

\- JVM doesn't run C code very well, or it requires research-level technology to do so (Graal). This applies to both computation and I/O -- it has a completely different I/O interface than C programs rely on.

\- Photoshop / Word / Excel / etc. were never ported to the JVM. The browser actually has better equivalents of them.

<comment><USER33> That last one is really stupid.

The reason they weren't ported is because they were legacy codebases with some code going back to the original versions including lots of assembler. It was always going to be a monumental task to rebuild them from scratch.

<comment><USER32> That you don't have to rebuild such apps from scratch on WASM is exactly the point.

You can compile legacy codebases in C and C++ to WASM. It was designed for that. Doing that on the JVM requires "research-level" techniques because the bytecode has a completely different design.

</comment></comment></comment><comment><USER34> I see nobody's mentioned that Kotlin can compile to WebAssembly (in addition to Java bytecode, native binaries and JS).

Kotlin is a beautiful language and hopefully WebAssembly will help it to gain traction outside the Android world.

<comment><USER35> What Kotlin is now has no bearing on what JVM was then or even what JVM is now

</comment></comment><comment><USER36> Isn't most of the work done already with LLVM intermediate representation? Stick to POSIX (and Qt if necessary) in the code, distribute the IR with a compiler which compiles and installs it, and job done.

<comment><USER37> There's no such thing as a single LLVM IR. It's actually a family of IR's where the actual representation is always dependent on target architecture.

</comment></comment><comment><USER38> If you want to pull the chain on this diarrhea:

[https://github.com/stevespringett/disable- webassembly](https://github.com/stevespringett/disable-webassembly)

<comment><USER39> Please don't post name-calling dismissals of other people's work to HN. This is in the guidelines: [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html).

The article you've linked to is much better than this and deserves a better representation than that.

</comment></comment><comment><USER40> Speaking as someone who has made their career with Java, "what the JVM ought to have been" is marketed for server-side use from the beginning.

It was poorly suited for the browser from the earliest days, and even now the language and runtime have evolved to make it a round peg for that square hole no matter how WASM develops.

Yet because students and younger webdevs see it through the prism of trying to be a browser-side solution, it taints the brand for people who lack business application experience. Java is thought of as "insecure", even though the OVERWHELMING majority of security patches are for the browser applet plugin, which hasn't been widely used in twenty years and now being retired. Instead of being " _a thing that makes large-scale business server applications more performant, and more tenable for large teams to work on_ ", it is often seen by newbies as " _a failed React.js alternative_ " where it comes up short.

Actual Java developers stopped thinking of it that way decades ago, but the history unfortunately persists.

<comment><USER41> Java in the beginning was standaline on the client side too. The web browser plugin came post 1.0 and the pre 1.0 (Oak) target applications were more IoT, not web</comment></comment>


Sqlite and couchdb creators announce unQL. SQL for NoSQL  - <AUTHOR> http://www.unqlspec.org/

<comment><USER2> As I said multiple times, in my humble opinion this is completely a useless effort. There are two cases:

1) Two databases are semantically very similar. In such a case even if the query language is different porting your application from one to another is trivial, especially if you have a minimal layer of abstraction between your program and you data, things like fetchUser(ID) or alike.

2) If the data model is different, using the same query language does not help at all.

So the question is WHY? :)

<comment><USER3> A gazillion already-made programs (for example data visualization tools, analytics software, etc.) speak only SQL. If the choice is between reinventing the wheel for each NoSQL DB vs. using a SQL DB (such as <http://luciddb.org/> [1] and soon postgres with their recent introduction of foreign data wrappers[2]) as an intermediate layer or even a full data mart that does what you want query-wise, a lot of people want to save time and just use the SQL DB. (Why do you think Hive exists?) There's more to DBs than storing user data for cat picture sharing sites...

[1] Disclaimer: I wrote a preliminary CouchDB connector for Lucid that'll get released separately with the upcoming 0.9.4 release.

[2] I believe one of the first postgres foreign data wrappers was around Twitter. So you move some API accessing code from an application layer that realistically only your application can talk to, to the DB layer, and now everything that can talk to the DB layer can access the data. (Unless they want it raw, then they'll get it from Twitter.)

<comment><USER4> In the same vein, I have been thinking about a mysql wrapper around various nosql products.

Same issue, installed base and connectivity.

Composition, interfaces, abstraction.

</comment></comment></comment><comment><USER5> What's the point in adding an abstraction layer above NoSQL datastores when most developers are already interacting with the datastore using another level of abstraction (e.g., an ORM in their language/framework of choice)?

</comment><comment><USER6> I'm at the CouchConf -- the demo of UnQL was fantastic.

The idea is that you shouldn't be locked into a NoSQL database, and that one language that would work across all NoSQL databases will grow the community. How each database manages their implementation of the spec is up to them.

INSERT into c1 value {"foo":1, "bar", ["a","b","c"]};

Now ignore the datastore behind that-- it could be Mongo, Couch, etc etc, but you know that your query won't need to be rewritten in case you decide to switch databases. This can only be a good thing.

</comment><comment><USER7> In 2009 I wrote in "The Dark Side of NoSQL" (1) about problems with NoSQL:

"1. ad hoc data fixing – either no query language available or no skills 2\. ad hoc reporting – either no query language available or no in-house skills"

(1) <http://codemonkeyism.com/dark-side-nosql/>

</comment><comment><USER8> I don't know if I like it or not. If you know SQL well and want to switch to a NoSQL database, what's easier learn? The "proprietary" API of the dababase (like Redis, or MongoDB) or the limitations of unSQL?

I can't speak of other NoSQL databases, but unSQL doesn't seem to expose most Redis' features, like lists & sets.

<comment><USER9> _what's easier learn? The "proprietary" API of the dababase (like Redis, or MongoDB) or the limitations of unSQL?_

That's what bothers me about this, too.

While I can understand the goal of making noSQL databases more accessible to people who already know SQL, as well as the need to unify the commands across all the different flavor of noSQL databases, there's something fundamentally flawed about accessing unstructured data via logic intended for structured data.

<comment><USER10> The data is not necessarily unstructured, it just doesn't have a strictly inforced schema. So I'm very intrigued by this if they're adding it as a layer on top of CouchDB views. If they are, you could use your views to selectively filter on documents with a known structure, and then safely operate across a subset of your docs with unql. We'll see where they go with this though.

<comment><USER10> To clarify for the downvoters, I'm talking about this in the context of CouchDB, which I believe was a fair assumption to make seeing as Damien Katz is one of the principle people involved and it was announced today at CouchConf.

In CouchDB, running a filter on top of view results is something you can only do in list functions or client side, so I am very curious to see how they incorporate this into CouchDB.

</comment></comment></comment></comment><comment><USER11> I'm kind of wondering, what next? An optimizer?

I think there's a lack of clarity on what's being reinvented here, why, and where this is all headed. And when we get there, is it really something fundamentally new, or is it something that existing SQL products can absorb along the way?

</comment><comment><USER12> I know SQL moderately well, but does anyone share my view that moving AWAY from using SQL syntax is a GOOD thing? I personally really dislike writing SQL, i'm sure lots of other people do too, thats why people write query builders etc.

<comment><AUTHOR> Actually i find SQL a great thing, and one thing that bothers me with all those "other" databases is that you have to start from scratch with everyone of them. A common Query language would be awesome!

</comment></comment><comment><USER13> Title is missleading. Nosql is an umbrella term for keyvalue-, document-, object-, graph-, column- etcetera- databases. The API's these provide are probably using a paradigm most natural to the actual habitat the data lives in already. Is this "no sql for nosql" really a problem that needs solving or merely a flakey abstraction thats super interesting to implement?

Unql seems to only focus on json based document stores so it might actually make a decent abstraction.

Too easy to dismiss as impractical and i hope these guys pull off something amqazing if only for json document stores.

<comment><AUTHOR> The title doesn't state that it's for every NoSQL database.

</comment></comment><comment><USER14> We've had great success with using ElasticSearch to index our couch db databases in near realtime into it's lucene indexes, and then querying those.

It allows each tool to focus on what it does well.

<comment><USER15> Looking into something very similar right now. What's your tradeoff been in terms of, for example, disk taken up by Elastic for indexing? How do you find Elastic in terms of how well it keeps up with indexing the data recently inserted into couch?

</comment></comment><comment><USER16> It can't get much simpler than this can it?

Redis: redis.set "foo", "bar"; redis.get "foo"

Mongo: db.stuff.save({json: "object"}); db.stuff.find({json: "object"})

<comment><USER17> it will get simpler when your statement would look like :

Redis, Mongo, <insert your own zoo> ... : db.set(), db.find()

</comment></comment><comment><USER18> Uncle or Uncool. The hipster query language.

</comment><comment><USER9> It looks like a SQL syntax to access noSQL databases.

I'm not sure if that's a step forward or backward.

<comment><USER19> SQL is a fantastic set based language. Using SQL to query MongoDB would be about 7 steps forward for just about everything I do, particularly if "SQL" magically included joins.

<comment><USER3> Also would be cool if everything magically supported the 1-Bucket-Theta algorithm. :) [http://www.ccs.neu.edu/home/mirek/papers/2011-SIGMOD- Paralle...](http://www.ccs.neu.edu/home/mirek/papers/2011-SIGMOD- ParallelJoins.pdf)

</comment></comment></comment><comment><USER20> It'll be interesting to see how much this gets adopted. Ad-hoc query languages like SQL are great for operations and reporting, but if your NoSQL store needs to conform to the limitations of the unQL language, aren't you losing that "polyglot persistence" quality of using NoSQL?

</comment><comment><USER21> I think the whole SQL vs NoSQL stuff is a bit mislead.

Relational databases are not dinosaurs. They solve lots of problems. It's the database schema which is a PITA, so let's just have relational engine (with reasoning, like Prolog) without the schema and move on.

Most people don't need NoSQL for scalability, they choose it because they dislike SQL. So maybe it's SQL (as a syntax) we should ditch, together with the schema thing, not the relational model as it is. In this context, the effort with unQL, while being a remedy for some short-term situation, is actually not so cool in the long term, because it will keep the legacy undying.

</comment><comment><USER22> In my ignorance I see "...an open query langauge for JSON..." and I think "Cool. Now I'll be able to store complex database in plaintext." Am I wrong or just missing the point?

I like the idea of SQLite but it would be nice to not have to jump through hoops to access the databases of programs (I recall having some trouble with Pidgin in the past that I gave up on because I was too much of a newb to know how to interact with SQLite databases).

</comment><comment><USER23> Don't repeat the same mistakes as OQL[1]?

[1] <http://www.odbms.org/ODMG/OG/>

</comment><comment><USER24> While my first thought is that I would rather use native APIs for each datastore, I do use SPARQL and SQL a lot so there is some sense to a standard query language.

The problem is having too abstract of an interface that does not allow easy access to functionality of different datastores.

</comment><comment><USER25> Is that pronounced "uncle", or "uncool"?

<comment><USER26> SQL is pronounced as _sequel_ , so I'd definitely go with _uncool_. Even though this actually can be quite cool.

Looking forward to explaining to a client that _yes, we just need to run this one uncool query to get the report_ ;-)

<comment><USER27> SQL is not pronounced as "sequel". The ANSI standard declared that it is pronounced S-Q-L.

<http://en.wikipedia.org/wiki/SQL#Standardization>

</comment></comment></comment><comment><USER26> You could also claim that Sparql is already "a standard query language for NoSQL"

<http://decentralyze.com/2010/03/09/rdf-meets-nosql/>

</comment><comment><USER28> Anything that makes Couch more powerful is great! I'm interested to see how they integrate this into the viewserver, it seems like it allows a bit more expressibility than the current JS system does.

</comment><comment><USER29> unrelated to databases, but does anyone know how they made those charts for the grammar? is there a tool for that (ie something that would generate those charts automatically given a grammar).

<comment><USER11> Those are called "railroad diagrams". I don't have a specific recommendation, but this should get you started:

[http://stackoverflow.com/questions/773371/what-is-a-good- too...](http://stackoverflow.com/questions/773371/what-is-a-good-tool-for- creating-railroad-diagrams/)

</comment></comment><comment><USER30> It seems all of the examples are coming soon. Where is my friend SCOTT/TIGER?

</comment><comment><USER31> This deserves a XZibit meme.

<comment><USER32> Yo dawg, I heard you like NoSQL but missed yo SQL so I made UnQL for yo NoSQL.

<comment><USER33> Eminem, is that you?

</comment></comment></comment><comment><USER34> Ohh, the irony: NoSql - because SQL is bad and we thought of fixing it! Fast forward some years into the future, current times - NoSql gets its, you guessed it, SQL back.

Can someone help me stop laughing at this, please?

Nevermind, it looks like a waste of time.

<comment><USER35> Non-relational databases have nothing to do with SQL, they have to do with trade-offs in data structures, consistency, availability, fault-tolerance, and distribution.

You need to understand how ignorant you are, because until you do you're going to have a calcified and incomplete understanding of databases.

<comment><USER34> Quite the opposite, I clearly recall some people from the NoSQL camp (read non-relational data stores, non SQL enabled) saying that SQL itself is bad and something else is needed.

I should have mentioned this before the zealots kicked in.

<comment><USER35> > I clearly recall some people from the NoSQL camp (read non-relational data > stores, non SQL enabled) saying that SQL itself is bad and something else is > needed

No, wrong, irrelevant, who said this?

<comment><USER34> I know it's wrong, you'll have to kill me to make me stop using SQL.

I wish I still had the article from 2009 - 2010 which said that SQL is too difficult and other things like that.

Anyway, this is what I was talking about when I originally replied.

<comment><USER35> I'm fond of SQL, you're either intentionally misinterpreting me or I'm not being clear. Non-relational databases are not about not using SQL, to believe so is dense and willfully ignorant.

I asked for specifics, you didn't provide them. I have to assume you're a troll at this point. I'm done here.

<comment><USER34> Ok, maybe I didn't use the right words or something: There was an article from someone in the NoSQL camp which said that SQL is bad.

I DO and DID understand that the querying language isn't restricted to relational databases.

What I was saying I REMEMBER reading something a while ago on the website of a NoSQL store or in an article which said something along the lines: "SQL is old and should be deprecated anyway, it was time for something new"

THAT was why I was amused.

If you think I was trolling or anything, ok, I'll just stop commenting on this site. It's not worth the trouble at all.

This is my last comment.

edit: I think this was it: [http://gigaom.com/cloud/facebook-trapped-in-mysql- fate-worse...](http://gigaom.com/cloud/facebook-trapped-in-mysql-fate-worse- than-death/)</comment></comment></comment></comment></comment></comment></comment>