TextQL: Execute SQL Against CSV or TSV - <AUTHOR> https://github.com/dinedal/textql <comment><USER2> If you work with CSV a lot, then also check out xsv, "a command line program for indexing, slicing, analyzing, splitting and joining CSV files", from Andrew Gallant (burntsushi of Go and Rust fame): [https://github.com/BurntSushi/xsv](https://github.com/BurntSushi/xsv)

<comment><USER3> Another great tool to wrangle csv file is miller

[http://johnkerl.org/miller/doc/](http://johnkerl.org/miller/doc/)

<comment><USER4> You can also use log parser lizard to parse CSV files.

</comment></comment></comment><comment><USER5> Seem very similar to q

q is out there for years, with a very large community.

q is a command line tool that allows direct execution of SQL-like queries on CSVs/TSVs (and any other tabular text files).

[http://harelba.github.io/q/](http://harelba.github.io/q/)

<comment><USER6> Yes, I use q all the time for slicing and dicing delimited files. The only problem I have with it is that the name can make it a littler harder to find if you don't remember the repo.

Since q will read stdin and write CVS to stdout you can chain several queries on the command line, or use it in series with other with other commands such as cat, grep, sed, etc.

Highly recommended if you like SQL and deal with delimited files.

</comment></comment><comment><USER7> This looks very simple and easy to use.

Are there any differences with using this instead of CSVKit?

[https://csvkit.readthedocs.io/en/1.0.3/](https://csvkit.readthedocs.io/en/1.0.3/)

It includes a tool called csvsql.

Example usage -

      csvsql --query "select name from data where age > 30" data.csv > new.csv

<comment><USER8> Was wondering the same. I've been using csvkit a lot lately. This looks like a subset of functionality using more or less the same approach. Csvsql uses sqlite under neath and you can do some nice things with this like joins, using sql functions, etc. There is also stuff like amazon athena that allows you to do similar things in s3 at scale.

Csvkit is great with pipes and you can also easily convert between csv tsv and even stuff like json.

</comment></comment><comment><USER9> I've worked on a similar tool in the past called Charlatan [1] (no I didn't choose that name), but it implemented its own SQL-like language and was extensible to support any file type.

Its key difference with TextQL and similar alternatives is it works on streams instead of importing everything in a SQLite then querying it. It seems strange to read a whole file in memory then perform a `SELECT` query on it when you could just run that query _while reading the file_. That means a much lower memory footprint and faster execution, but on the other hand you can only use the subset of SQL that's implemented.

[1]: [https://github.com/BatchLabs/charlatan#charlatan](https://github.com/BatchLabs/charlatan#charlatan)

</comment><comment><USER10> If you are on Windows you may consider using Log Parser Studio [1]. It is a GUI over logparser.exe which understands a lot of various file formats and uses SQL to query them [2].

[1] [https://gallery.technet.microsoft.com/office/Log-Parser- Stud...](https://gallery.technet.microsoft.com/office/Log-Parser-Studio- cd458765)

[2] [https://en.m.wikipedia.org/wiki/Logparser](https://en.m.wikipedia.org/wiki/Logparser)

<comment><USER11> Came here to say this. Log parser is amazing. Check out Log Parser Lizard for a good IDE.

</comment></comment><comment><USER12> This is one of those problems that is just hard enough that everyone feels it is useful to solve and just easy enough that everyone can solve it. Hence the dozens or even hundreds of independent solutions. For some reason I never end up using any of them!

</comment><comment><USER13> Apache Drill ([https://drill.apache.org/](https://drill.apache.org/)) is also a great alternative. Although advertised as a query engine for Hadoop, it works perfectly fine on local CSV and JSON files.

</comment><comment><USER14> How is this different from [https://github.com/harelba/q](https://github.com/harelba/q)

</comment><comment><USER15> For anyone looking to do this with plain SQLite, one can import a CSV by running (in the REPL)

        .mode csv
        .headers on
        .import my.csv tablename

This does look extremely convenient, though; being able to use UNIX pipes will be a huge improvement to my workflow.

<comment><USER16> To have an idea of what can be done with sqlite look at this solution to the Hashcode 2018 [https://github.com/mingodad/sqlite3-hashcode-2018](https://github.com/mingodad/sqlite3-hashcode-2018) using only sqlite3 command line.

</comment></comment><comment><USER17> I usually appreciate the animated gif screenshot (especially for curses based programs), but this particular animated gif would have been better off as just plain text—there's too much interesting data that can't be copied, it's real hard to go back to look at something that you missed and it's frustrating to have to wait while fake typing happens. I really wish there was just a transcript of the gif that I could read at my own pace.

</comment><comment><USER18> There is also BigBash [0] that converts an Sql statement to a bash one-liner (using sed, grep, awk,...) which can then run to execute the query. The advantage is that you can let it run on very large file(s) because of the streaming nature of linus bintools.

[0] [http://bigbash.it](http://bigbash.it) or the corresponding Github repo.

</comment><comment><USER19> Sqawk [1] is an Awk-like program that uses SQL and can combine data from multiple files. It is powered by SQLite.

[1] [https://github.com/dbohdan/sqawk](https://github.com/dbohdan/sqawk)

</comment><comment><USER20> I like VisiData[1][2] for CSV/TSV data exploration. textql is a good command- line follow-up to run quick direct queries against the data without needing to see visual/spatial structure.

[1] [https://jsvine.github.io/intro-to- visidata/](https://jsvine.github.io/intro-to-visidata/)

[2] Previous HN VisiData thread [https://news.ycombinator.com/item?id=16515299](https://news.ycombinator.com/item?id=16515299)

</comment><comment><USER21> Yawn. We've had this in Perl for _decades_ using the DBI Shell (dibsh). DBI is a common database interface. To connect to a data source, one uses an appropriate driver (by convention, in the DBD name space, such as DBD::Pg or DBD::MySQL) to get a db handle that can run SQL commands. There are a lot of drivers for things like CSV files, in memory database, etc.

Plus you can access all this programmatically and not just through the dbish command. Since this is a central part of Perl infrastructure, there are a ton of tools that extend the DBI, from a variety of ORMs like DBIx::Class, to data export and even DDL delta management.

Generalized Database Interface: [https://metacpan.org/pod/DBI](https://metacpan.org/pod/DBI) DBI Shell program: [https://metacpan.org/pod/DBI::Shell](https://metacpan.org/pod/DBI::Shell) CSV driver: [https://metacpan.org/pod/DBD::CSV](https://metacpan.org/pod/DBD::CSV)

</comment><comment><USER22> Ha. Shameless plug, but just about a week ago I started to work on a somewhat analogous thing. Executing SQL on XML [1]. I came up with this idea after trying to use stackoverflow data dump and finding out it was stored as XML. I wanted to run queries with LIKE operator on it.

[1]: [https://github.com/kamac/AskXML](https://github.com/kamac/AskXML)

<comment><USER23> This is a great idea. There are so many APIs and text dumps from websites and databases that are XML, and it would be nice to be able to query them with regular relational operations. I'll be following your project! I will say that I believe a streaming backend would be more suitable, since most of the XML dumps I'm thinking of are humongous and won't always fit in memory.

<comment><USER22> Thanks! About fitting into memory; the way the library currently works should technically work even on very big files, as the XML is iteratively converted into a SQL database. You can choose whether you want that database to be in a file or in memory. The only thing that could break right now is synchronizing back to XML after having made changes with UPDATE or INSERT statements, because I'm not paging results.

</comment></comment></comment><comment><USER24> ClickHouse ships with a command line tool which does this (without the actual database server):

        ps aux | tail -n +2 | awk '{ printf("%s\t%s\n", $1, $4) }' | \
            clickhouse-local -S "user String, mem Float64" \
                -q "SELECT user, round(sum(mem), 2) as memTotal FROM table GROUP BY user ORDER BY memTotal DESC FORMAT Pretty"

        ┏━━━━━━━━━━┳━━━━━━━━━━┓
        ┃ user     ┃ memTotal ┃
        ┡━━━━━━━━━━╇━━━━━━━━━━┩
        │ clickho+ │      0.7 │
        ├──────────┼──────────┤
        │ root     │      0.2 │
        ├──────────┼──────────┤
        │ netdata  │      0.1 │
        ├──────────┼──────────┤
        │ ntp      │        0 │
        ├──────────┼──────────┤
        │ dbus     │        0 │
        ├──────────┼──────────┤
        │ nginx    │        0 │
        ├──────────┼──────────┤
        │ polkitd  │        0 │
        ├──────────┼──────────┤
        │ nscd     │        0 │
        ├──────────┼──────────┤
        │ postfix  │        0 │
        └──────────┴──────────┘

Has the advantage of being really fast.

<comment><USER25> the additional requirement to set up the schema is kind of onerous.

<comment><USER26> Shameless plug: Sqawk can do nearly the same without you defining a schema.

      $ ps aux | sqawk -output table \
                       'select user, round(sum("%mem"), 2) as memtotal
                        from a
                        group by user
                        order by memtotal desc' \
                       header=1
      ┌────────┬────┐
      │dbohdan │67.1│
      ├────────┼────┤
      │  root  │3.5 │
      ├────────┼────┤
      │ avahi  │0.0 │
      ├────────┼────┤
      │ daemon │0.0 │
      ├────────┼────┤
      │message+│0.0 │
      ├────────┼────┤
      │ nobody │0.0 │
      ├────────┼────┤
      │  ntp   │0.0 │
      ├────────┼────┤
      │ rtkit  │0.0 │
      ├────────┼────┤
      │ syslog │0.0 │
      ├────────┼────┤
      │ uuidd  │0.0 │
      ├────────┼────┤
      │whoopsie│0.0 │
      └────────┴────┘

Link: [https://github.com/dbohdan/sqawk](https://github.com/dbohdan/sqawk)

</comment></comment></comment><comment><USER27> Looks pretty good!

Just for historical interest, similar things have been done before.

e.g.[http://quisp.sourceforge.net/shsqlhome.html](http://quisp.sourceforge.net/shsqlhome.html)

See also Facebook's osquery of course. Turns even complex structures into sql tables -- helped by Augeas.net.

</comment><comment><USER28> Does this have any benefits over lnav[1]?

[1] [https://www.lnav.org](https://www.lnav.org) \-- a feature-rich, powerful, flexible "mini-ETL" with an embedded sqlite engine -- is one of my all-time favorite CLI tools that somehow remains under the radar

<comment><USER29> [http://innolitics.com/10x/graduate-from-sed-and-sort-to- lnav...](http://innolitics.com/10x/graduate-from-sed-and-sort-to-lnav-the- logfile-navigator/)

</comment></comment><comment><USER30> PostgreSQL does this out of the box with :

      CREATE EXTENSION file_fdw;
      CREATE SERVER import FOREIGN DATA WRAPPER file_fdw;
      CREATE FOREIGN TABLE foo (
        col1 text,
        col2 text,
        ...
      ) SERVER import
      OPTIONS ( filename '/path/to/foo.csv', format 'csv' );
      SELECT col1 FROM foo WHERE col2='x';

<comment><USER31> Having to define the schema is the major hangup for me in my workflow. I would prefer a wrapper that could run within unix pipelines around arbitrary text data files, but they all use SQLite.

I personally have been using harelba's q (the un-googleable utility), which is just fine.

It would be great not to shift gears into SQLite syntax and date formatting all the time. Does anyone know of a similar tool that runs over postgres?

<comment><USER28> > "prefer a wrapper that could run within unix pipelines around arbitrary text > data files"

That sounds like [lnav]([https://www.lnav.org](https://www.lnav.org)).

For queries, it does use SQLite under the hood, but before querying your "arbitrary text data" files, you can use regex to define a custom format (eg with named groups and back references), providing structure against which standard SQL is an ideal tool to query.

It's simpler than I'm probably making it sound. Highly recommended.

</comment></comment></comment><comment><USER32> These responses are so HackerNews. Everyone asking 'why not use x project?', and each x is different...

</comment><comment><USER33> In the past, I've used fsql[1] when I needed this. This looks interesting though.

1: [https://metacpan.org/pod/distribution/App- fsql/bin/fsql](https://metacpan.org/pod/distribution/App-fsql/bin/fsql)

</comment><comment><USER34> I'm a big fan of SQL Notebook [1]. It is open source and has a great UI to quickly work with a number of importable formats (CSV, Excel, JSON) as well as server-based DBs.

[1] [https://sqlnotebook.com/](https://sqlnotebook.com/)

</comment><comment><USER35> One thing that I would like to add to my projects is the animated GIF shell. I've looked at several ways to create them but they all seem really clunky and force me to type correctly the first time. What do people use to create these helpful animations?

</comment><comment><USER36> I'm pretty sure many people have written or attempted to write similar tools at least once before. It's as simple as programmatically importing to an in- memory sqlite instance and then executing SQL queries.

</comment><comment><USER37> Apache Spark can do this too, similar to the Apache Drill comment. This thread is starting to feel like Stack Overflow a bit.

<comment><USER38> If only when googling "sql on csv" this comments page would show up first.

</comment></comment><comment><USER39> This is cool. How does it work? I'd guess cleaning the data, and copying to SQL Lite?

<comment><USER9> It imports everything in an in-memory SQLite database.

</comment></comment><comment><USER40> Apache MetaModel is great for this in Java.

</comment><comment><USER41> While dealing with CSV files using SQL statements is cool and all, SQL queries are still SQL queries.

<comment><USER42> [https://en.wikipedia.org/wiki/Truism](https://en.wikipedia.org/wiki/Truism)

</comment></comment><comment><USER43> I use R for this.

<comment><USER44> Can you show an example of how to use R to read tabular data on stdin (e.g. in a UNIX pipe line) and perform SQL queries?

<comment><USER45> Why would you be running SQL queries in R, a language with built-in support for handling tabular data?

<comment><USER44> I was asking because the topic of this discussion is specifically running SQL queries (that's what TextQL does).

But, there certainly are good reasons for using SQL in R. Packages like sqldf exist, so their authors would probably be able to give the best answer to your question.

Some reasons:

\- SQL is a very widely-known DSL for working with tabular data, so it may make sense to make that interface available within R.

\- For certain operations (e.g. specific types of joins) it may be more natural / easier to express the operation in SQL.

\- For large data sets, some operations in R have large memory footprints, and doing the operations in a database may have lower memory requirements.

</comment></comment></comment></comment><comment><USER46> In SQLite you can just do

        sqlite> .import myfile.csv mytable
        sqlite> select ...

What does this tool give you that SQLite doesn't do out of the box?

<comment><USER47> As a command-line tool it could serve as a more powerful replacement for unix utilities that I often use for simple analysis jobs: sort, uniq, wc, awk, sometimes cut and sed. I'm going to give it a try.

It looks like textql loads everything into an in-memory SQLite instance, whereas I'd really like to see an approach that uses the SQLite's virtual table mechanism ([https://sqlite.org/vtab.html](https://sqlite.org/vtab.html)), which would avoid the loading step and perhaps make streaming processing possible.

<comment><USER48> What you describe would be a super nice utility. Great for prototyping and development, as the code could be copy & pasted from such a tool into application code.

The streaming would make it memory efficient, and possibly able to handle some big data - maybe not true "Big Data", but certainly 10s of gigabytes.

Anyone want to take this idea into a GoFundMe site?

<comment><USER6> SQL is set-oriented. How would that work on a potentially indefinite stream, other than as a simple filter which you could just do with a tool such as awk.

<comment><USER49> Many relational operations don't require a whole stream to compute, and many of those that do don't need it all at once.

Projection (mapping), a join against a fully loaded other side as well as filtering work.

Aggregation can consume an indefinite stream with limited working set if the cardinality of the grouping key isn't large.

And of course you can combine these in nested and unioned operations, computing across multiple indefinite streams concurrently and with limited working set.

It would be tricky to make work effecively without hinting for things like joins, for sure; join order is one of the hardest bits a query engine optimizes.

</comment></comment></comment></comment></comment><comment><USER50> we should be aiming to kill SQL as a language

<comment><USER4> Why? Not surprised you got down voted but I am curious?

<comment><USER51> It is a negative comment. We should make alternatives not kill off technologies. Even the Commodore 64 has use with the right modern add ons, now that is hacking.</comment></comment></comment>


Feds Seize Silk Road 2 in Major Dark Web Drug Bust - <AUTHOR> http://www.wired.com/2014/11/feds-seize-silk-road-2/ <comment><USER2> "like its predecessor, was initially run by a pseudonymous figure known as the Dread Pirate Roberts"

""It has been over 24 hours since we last heard from our captain," Benthall is accused of writing under his Defcon handle on December 22nd of last year. "As his second in command, I have very clear instructions as to what to do in this worst case scenario…I cannot elaborate on specifics, but the marketplace is safe in my hands until the Captain returns or his successor appears." "

An so DPR legend continues.

</comment><comment><USER3> I think you're gonna have to create your own communications infrastructure if you want to work against the people who control the current one.

Look at the cartels. They had to create their own cell networks and radio systems for communications.

<comment><AUTHOR> Wow, thats really interesting. I had not heard that before.

Just looked it up... [http://www.wired.com/2012/11/zeta- radio/](http://www.wired.com/2012/11/zeta-radio/)</comment></comment>


Microsoft is currently developping 10 different applications to do the same damn thing - <AUTHOR> http://www.secretgeek.net/sync_live.asp?

<comment><USER2> Saying that all of those apps are the same is a naive interpretation of the services. There's overlap, but that's because a lot of those products started out with disparate purposes and are now being consolidated (Groove+FolderShare is a large part of the current version of Live Mesh). They're iterating on the same idea and realizing a grander solution, albeit very slowly.

</comment><comment><USER3> To be fair, most of them don't work very well.

</comment><comment><USER4> ... no! Then instead of 10 committees of 20 people, they'll have 1 godawful app designed by 1 ginormous committee of 200 people!

</comment><comment><USER5> Reminds me of joelonsoftware's recent Architecture Astronauts post.

A relevant quote:

"The hallmark of an architecture astronaut is that they don't solve an actual problem... they solve something that appears to be the template of a lot of problems. Or at least, they try. Since 1988 many prominent architecture astronauts have been convinced that the biggest problem to solve is synchronization."

<http://www.joelonsoftware.com/items/2008/05/01.html>

</comment><comment><USER6> Misleading article.

To say sychronizing files from your PDA to your Ford is the same as sharing folders on the web? That's a bit of a stretch.

Could there be some kind of uber- sharing-synchronization program? Beats me. Seems like there are a lot of market segments and problems. Why would we want one, big, bulky product that does a gazillion different types of synchronization?

You see, Microsoft gets it both ways. They write the product that does everything, people bitch about enterprise apps. If they write a dozen niche products that are somewhat similar, people bitch about duplication.

For me? I'm a big Groove fan, even if they did cut the crap out of the features to make it less competitive with their other stuff.</comment>


Ask HN: Lead asked about corporate accounts. - <AUTHOR>

I started a "startup" 3 months ago and I've been plugging away at coding and marketing it on the side. I've got a couple of paying customers but things are still pretty raw.
Today a person reached out to me to ask who to contact about corporate accounts. I feel like I need to take advantage of this small opportunity but I'm not really sure how.
It would take me 2 to 4 weeks to have the app ready for enterprise use (as I imagine it) at even the most basic level.
What's my play? <comment><USER2> Setup it up at the most basic level. Use this customer as a beta for it. Make it awesome for them, expand, plan to release for more corporate accounts after you feel like you are ready.

</comment><comment><USER3> Probably depends on the app/service</comment>


Softonic fires 66% of its employees - <AUTHOR> http://novobrief.com/softonic-layoffs/

<comment><USER2> Author of the article here. Just wanted to share a small update to this story.

Basically, on the day of the announcement of the layoffs, Softonic also announced that it was acquiring a SF-based company: [http://novobrief.com/softonic-employees- layoffs/](http://novobrief.com/softonic-employees-layoffs/)

<comment><USER3> Is this linkbait? Seems strange to make a comment like that without mentioning the name of the other company.

<comment><USER2> I don't think it is and I wouldn't mind editing the comment and getting rid of the link. Sorry.

Name of the acquired company hasn't been published yet because it hasn't been disclosed.

Sorry for the confusion.

</comment></comment></comment><comment><USER4> Sometimes the children I teach want to install the software we use at home. For some pieces of software, Softonic is on the first page of search results for the name of the software. It's usually half-way down (used to be more near the top, thankfully this changed).

I always warn them to never ever get their software from Softonic, but instead always make sure you end up on the official site for the computer program. Because on numerous occasions, downloads from Softonic came with "extras", a toolbar or changing your default search engine / homepage, or pop-ups telling that their computer is "at risk"[0].

I'm always very careful explaining this because the last thing I want to happen is some computer-clueless parent installing software for their super- smart kid, getting all sorts of ad- and spyware along the way.

So my point is, I'm a bit ambivalent? On the one hand, yes I absolutely think it's terrible that all these people will be losing their jobs. It's a pretty big hit (though not as big as what Opera did...). On the other hand, what this company was doing ... was it creating value for anyone?

[0] well, I won't lie to them and tell them that indeed it is, unfortunately there's absolutely nothing they can do about it (I usually tell them to at least get MSE, an adblocker and maybe Ghostery, but that does nothing to protect you from scammers, phishing, or governments).

</comment><comment><USER5> I remember the days when softonic was "the" place to go to discover new software. Then the whole site started to become shady with SMS for fast downloads and whatnot. At that point I stopped trusting the site completely, I guess I was not the only one.

</comment><comment><USER6> 450 employees!!?? At least ten times what I would have ever guessed. I'd be surprised if mac update had more than 4.5 employees.

</comment><comment><USER7> Location: Spain. Is anyone really surprised?

<comment><AUTHOR> Things are changing here. The funny thing is that this company was an example of success for newspapers. We knew this was going to happen, this company has been installing garbage in computers for years, that's not a solid bussiness plan</comment></comment>


Ask HN: Why do so many companies hire only local? - <AUTHOR>

I've noticed that most tech companies, when they do post for new hires, are looking for someone in X area.
I've been a contractor for several companies, and I can say that my performance has always been praised. In all cases, I have not worked on site, or even locally for some. This does not impact my ability to perform, and in fact may sometimes enhance it, since I don't have people wandering by to disturb me.
For those of you who deal with hiring, what influences you to only look locally, and not for remote workers? <comment><USER2> I've managed workers in different countries. I can confirm that it takes much, much more communication to keep remote workers "in the loop" and energized. It wouldn't appear that way at first glance (the distraction and needless meetings skipped).

Tech companies are run by people. They want to be understood and have an ongoing dialogue, as things develop very rapidly. Many decisions are reached at lunch/bathroom breaks/bar which is naturally inclusive of local talent (meaning you don't have to really explain much, as others will carry the message for you vs remote)

One missed point, cities have personalities or rather a group of people predisposed to certain ideas. You have just a little more certainty understanding who you are hiring when you have a vague sense of which (cities/communities) they are chosen. It's small, but makes a huge difference when trying to create momentum in a company.

Tasks will be outsourced...team members won't. As the tech scene continues and the high transitional cost of tech talent continues...i think more employers will want a little more 'buy in".....we know can do the job as well....that's not really the question.

</comment><comment><USER3> In addition to the general workday communication issues, there are individual income tax issues, as well as benefits issues. The company now has to do payroll tax in another state, and find a health care plan in the employee's state. This is often not worth the expense for a small company.

Furthermore, location of an employee in another state may establish a tax nexus in that state (depends on the state), causing the company to have to pay income taxes in that state. This is generally not worth it for a company of any size.

If you're a telecommuter, you are generally best off by trying to work for an enlightened huge company that already has offices everywhere, or doing corp- to-corp consulting.

<comment><AUTHOR> As an employee yes, but as a contractor no. I'm Canadian, and working for an American corp. It was actually easier for them to get me working, rather than an employee.

<comment><USER3> I think it's actually easier to hire someone overseas because there are no interstate tax or benefits issues.

</comment></comment></comment><comment><USER4> There are a few reasons as to why most companies are only looking to hire local candidates. The first one is that companies are familiar with the area they're in, they know which colleges have better cs programs, they know which smaller companies had an impact in the area, and which ones they should avoid on resumes.

Another is simply that when you extend an offer to a candidate the company has already invested massive amounts of time and money in the interview process. Candidates are more likely to balk at accepting an offer out of state once they really start thinking about leaving whatever area they've called home for the last 3+ years.

Also in regards to telecommuting, you may be able to do that well but I can assure you that there are more remote workers who do it poorly.

All of that being said, some folks are able to make a good living moving around and doing contracts and remote work. You should be aware though, that if you hope to one day land a permanent position at a reputable company after years and years of contracting you're going to have a much harder time than someone who has worked multiple permanent roles.

</comment><comment><USER5> I find it really dumb, considering these are tech companies. Yes the bandwidth is higher in person but it's a professional arrangement, not a marriage. It's just 19th century thinking honestly.

I think sooner than later companies who learn to accept distributed teams will have an edge over those which don't.

You just need to trust your employees more and have an internet connection.

</comment><comment><USER6> I've never hired an employee, but I'd imagine that they're looking for someone to be a member of the team so to speak.

Contract work is when you need a certain task or objective done. But being a team member may involve driving new ideas, working with others, etc. That sort of stuff requires a lot of communication that can't be done by Skype or email.

<comment><AUTHOR> Being on contract doesn't prevent any of that. I've pushed a few new ideas, and built out an entire new system. In fact, my current contract has considered having me on board such a success that they've hired two new contractors (I was their first experience with this).

</comment></comment><comment><USER7> I feel like a lot of the time a remote worker may come on strong at the beginning of an assignment and slowly drift away, as the lack of constant direction and team involvement causes them to become bored and distracted. This is something I'm actively trying to fix because finding good local workers can be very hard if you aren't in SF or NYC.

</comment><comment><USER8> I think it's a matter of personality and also the ability of the company to identify which team members can work remotely. While it feels "safe" to keep somebody under constant supervision, it doesn't necessarily make them more productive. Less experienced people tend to need more hand-holding, which can be a factor in adopting a remote employee model.</comment>


Scottish wind turbine explodes in storm - <AUTHOR> http://www.youtube.com/watch?v=eQrtXx3aJdM

<comment><USER2> See also "Hurricane Bawbag". We don't get winds gusting to 165mph here very often ...!

<http://en.wikipedia.org/wiki/Hurricane_Bawbag>

<comment><USER3> You are Edinburgh based right? Weirdly enough, I'm further north yet it was not as bad here. (Inverness)</comment></comment>


HTML Live Video Streaming Via WebSockets - <AUTHOR> http://phoboslab.org/log/2013/09/html5-live-video-streaming-via-websockets

<comment><USER2> I _just_ implemented something similar. I have an IP webcam on my network, but I noticed that the built-in viewer uses 30% cpu time on my laptop when I view it in Chrome. (20% on Firefox, about the same on IE using an ActiveX version of the client.) The built-in viewer supports a 'cell phone' mode that sends a stream of jpegs, but in an awful way that uses even more cpu time. I decided to write my own wrapper around the url it uses to get a jpeg of the current camera view.

This uses requestAnimationFrame, and lets me set the max framerate I want. Since I'm just watching my dog Mischa and she sleeps most of the time, I set it to 0.5fps which keeps my cpu usage around 1%.

<comment><USER3> $("#mischacam").attr("src", camURL + "/snapshot.cgi?_=" \+ Math.random());

Out of curiosity, why do you append a random number between 0 and 1 to the URL?

<comment><USER2> mannix is right; to get the browser to reload the image I have to change the url every time. You'd think in a well-designed webcam API the image would come back with appropriate headers to prevent caching, but I didn't even bother to check. The webcam software is not well-designed.

</comment></comment></comment><comment><USER4> It's not very clear from the article why this solution is superior to using a x-mixed-replace/multipart stream that contains image/jpeg bodies. If you don't have any audio just stream your jpegs using the x-mixed-replace HTTP response type. All browsers support it (even IE6).

<comment><AUTHOR> Better quality and framerate with lower bandwidth usage. At peak, the example in the article served 250 clients with smooth 30fps video while using about 120mbit/s.

I agree that this is a huge hack and just serving JPGs would be much easier, but this hack turned out to work pretty well.

</comment></comment><comment><USER5> It's awkward that in 2013 we have to complexify design to this point :

"Streaming video through WebSocket (thus over http upgrade) through a plain- tcp-to-websocket-proxy on top of...".

It feels over-engineered and like TCP-Sockets are a revolution. I know, we can't easily expose plain-socket because of security but still... What's next? UDP for 2018? Socket-binding for 2022?

(Nice work though)

<comment><USER6> Well, traditionally this would be done using something like Flash and RTMP, but Flash is dead on mobile - Apple considered it too CPU and battery hungry and a poor user experience, so they banned it. Hence MPEG decoders implemented in JavaScript.

<comment><USER5> I think you missed my point. I'm not criticising what phobolabs (who is very talented BTW) has done or what any end-developer is doing.

It's just crazy how the standard is slow to come up with new basic APIs.

I mean, you can make a P2P video streaming application since 1995 using straightforward architecture in a desktop App using less cpu, less middleware and less engineering.

</comment></comment></comment><comment><USER7> Super cool. I just submitted a pull request to jsmpeg to move some of the RGB conversion stuff to the GPU using WebGL. Reduces CPU usage by 5-10%. [https://github.com/phoboslab/jsmpeg/pull/1](https://github.com/phoboslab/jsmpeg/pull/1)

</comment><comment><USER8> I just tried your instant webcam app, and omg it is so cool. I've been looking for something like that. Very impressive low latency.

It's free too, how are you making money? Thanks for making it! (Now I just need something for streaming video between laptops, between laptop and mobile device. I find it very annoying how FaceTime doesn't let you call yourself from one device to another.)

</comment><comment><USER9> The Raspberry Pi has hardware video encoding support via the GPU. I dug up this blog post about it:

[http://theiopage.blogspot.co.uk/2013/04/enabling- hardware-h2...](http://theiopage.blogspot.co.uk/2013/04/enabling- hardware-h264-encoding-with.html)

Would be great to integrate this with the node app.

<comment><USER10> I was just wondering the same thing. Is there some other reason why he would not use the GPU to do h264 encode/decode? It appears that ffmpeg does support it.

</comment></comment><comment><USER11> This looks awesome, but I'd love to hear some comparisons between this hack (it is definitely far simpler) and WebRTC or multipart JPG.

Also, great hack, it seems so obvious, but is actually pretty great.

</comment><comment><USER12> As a side note, the page completely hangs Chrome for iOS.

<comment><AUTHOR> Chrome for iOS is not allowed to use the JavaScript JIT, hence decoding the MPG is extremely slow. Try Mobile Safari and complain to Apple :)

</comment></comment><comment><USER13> Just released something that streams video like this using MJPEG: [https://igorpartola.com/projects/hawkeye/](https://igorpartola.com/projects/hawkeye/). I thought about using WebSockets but MJPEG seemed to be reasonably well supported by all the browsers I cared about do I chose to use it instead.

</comment><comment><USER14> why is that better than any webcam using multipart jpg?

</comment><comment><USER15> I wonder if you could just encode each frame as a gif and send it to the browser as part of an animated gif a la gifsockets ([https://github.com/videlalvaro/gifsockets](https://github.com/videlalvaro/gifsockets))?

<comment><USER16> I did that some time ago: [https://github.com/jbochi/gifstreaming](https://github.com/jbochi/gifstreaming)

</comment></comment><comment><USER17> Very cool! I like the simple approach of foregoing all the fancy new tech and doing the simplest possible thing (though the quality/bitrate does suffer)</comment>


Japan invents speech-jamming gun - <AUTHOR> http://www.foxnews.com/scitech/2012/03/02/japan-invents-speech-jamming-gun-that-silences-people-mid-sentence/?intcmp=features

<comment><USER2> This is interesting as I've noticed this very effect.

Sometimes I talk with people over Skype. At times, there can be an echo if they are using a laptop speaker and mic. When I talk, after a slight delay, I get an echo of what I just said. It stops me cold from saying anything else...

</comment><comment><USER3> I want to see a video demo of it!</comment>


Female Founder Successes of 2009 - <AUTHOR> http://www.women2.org/2009-founder-successes/ <comment><USER2> The creation of this list is evidence that women are not accepted into entrepreneurial enterprises as well as men are yet. There shouldn't need to be a list of female founders to single them out.

<comment><USER3> It is a site for/about women founders. The fact that they made a list of successful women isn't really evidence of anything. If it was a site about founders from the east coast we could just as likely see such a list without assuming people from the east coast are not accepted into entrepreneurial enterprises.

<comment><USER4> I think he means the fact that we don't go around making lists of "Male founder successes," but we do for female founders is indicative of the imbalance between male and female startup founders, because of the emphasis it places on females.

Perhaps the list itself isn't indicative of anything, but the site and the list combined might be. There has to be a reason why it's a site for women, especially since we don't have sites for startups explicitly for men. That reason is probably because of our patriarchal society more than anything, but his implication that perhaps it indicates that the male to female founder ratio is a little lopsided is not invalid.

I don't think it necessarily means women are not accepted into entrepreneurial enterprises, because it could also mean that women simply are less interested.

</comment></comment></comment><comment><USER5> What about a list of the successes of people with brown hair?

Also, this could be a case study in how overuse of exclamation points makes them meaningless.

</comment><comment><USER6> I'd rather see more efforts to bring women into the top of the startup funnel, vs. singling them out later on if they become successful.

There is some value in having good examples of successful women in entrepreneurship in convincing more women that they should consider it, but I think "this person in my graduate department launched a startup" is probably more significant (regardless of gender) than "a famous woman startup founder and CEO exits, although I've never met her".

women 2.0 actually does a pretty good job with other programs to try to bring qualified women into the startup world (partnership with Founder Institute, various meetups, etc.). I think the most valuable thing would be attracting more women into college entrepreneurship clubs, entrepreneurial classes in undergraduate and MBA programs, and in non-startups, encouraging them to take responsibility to project/product management on products which may become spinouts or startups.

</comment><comment><USER7> Since we're splitting into demographics - what's with the low number of classic ango-saxon women there, and more asian, indians, jewish, etc?

<comment><USER8> Given that the majority of them are coming from technical/web startups, you're going to see a huge number of technical founders. Five years ago, when I graduated in CS at a major American research institution, you could throw a dart at a map of Asia and be guaranteed to hit a country that had contributed more female engineers to my CS department than there were female white engineers in the class. Five years later, guess who went on to become technical founders?

If you want to phrase things in the positive sense, you might phrase it not as "What does CS do wrong?" but "What does environmental engineering or biomedical engineering do right?", since they got most of the women in the engineering school. I have the heretical impulse to say "Offer obvious opportunities to have a career primarily revolving around interacting with human beings" but I have learned that "Maybe they just don't _want_ to spent 14 hours a day reading XML files" costs me friends.

</comment></comment><comment><USER9> Sexism exists so long as people keep tallies -- same with racism. They have new life breathed into them whenever anyone interposes the racial and gender lens between themselves and all they observe. Sigh. Here's to another year of counting (and then of course judging) based on race and gender. Ah, such progress.

</comment><comment><USER10> it seems to be primarily men who think that this kind of thing is meaningless, fellas.</comment>


History of the Text Editor - <AUTHOR> http://vanstee.me/history-of-the-text-editor.html

<comment><USER2> more like "a history of vi" (via ed and on to vim) with a side of emacs. :-/</comment>


Show HN: Qualified, unit test driven code assessments for better tech interviews - <AUTHOR> http://qualified.io/?utm_source=hackernews&utm_campaign=r_showhnqualified <comment><USER2> Looks awesome. Trying it out now :)

</comment><comment><USER3> Looks dope, will check it out.</comment>


QuickCheck by example: Number theory and Okasaki's red-black trees - <AUTHOR> http://matt.might.net/articles/quick-quickcheck/

<comment><USER2> Have used open-sourced QuickCheck equivalent for Erlang, called PropEr. Great exercise in abstract thinking. Although testing stateful machines in scenarios where time matters turned out to be very hard.

<comment><USER3> Time is fun. You have to essentially mock it. Write a system in which you can control how time advances and then define your postconditions in the stateful system such that time is not violated. I have an approach here, taken for the 'fuse' project in Erlang (Using the commercial version of Quviq QuickCheck):

[https://github.com/jlouis/fuse/tree/master/eqc_test](https://github.com/jlouis/fuse/tree/master/eqc_test)</comment></comment>


Facebook Status Messages To Become Publicly Searchable - <AUTHOR> http://www.nytimes.com/external/readwriteweb/2009/06/24/24readwriteweb-the-day-facebook-changed-messages-to-become-18772.html

<comment><USER2> "In time, though, people may very well decide they are comfortable with their social networking being public by default. That will be a different world, and today will have been one of the most important days in that new world's unfolding."

Isn't that world here already through myspace and xyz social networks? FB's value proposition was that it was a safe place to keep in touch with people in your life. Now it's changing its entire strategy and value proposition because Twitter took one feature and implemented it really well. Is the whole data and search business model that much lucrative to risk the trust and stickiness of its userbase?

In my opinion, I think facebook should improve on its strategy for encouraging real online identities. they already started this with fb connect. I believe there's value in knowing who people really are on the web.

Fb == extension of real life on web. Only makes sense for them to improve that identity platform.

<comment><USER3> I agree with you. Perhaps Facebook does too - we never know. Facebook's decided over the years that it ought to try out a new feature first, get feedback later.

I think this move is a big mistake on their part, but I also think Facebook will revert to their old idea the minute users get pissy. I almost don't mind, since it's fun seeing them mess with things and because they never make it impossible to avoid their changes.

</comment></comment><comment><USER4> For an article syndicated from a blog, wouldn't it make more sense to use the link from the original article? For one, the domain will show up properly, and for two, you can't comment on the New York Times syndicated version. Last, I didn't notice until looking at the readwriteweb.com site--there was a significant update to the original post that includes a direct response from Facebook:

[http://www.readwriteweb.com/archives/the_day_facebook_change...](http://www.readwriteweb.com/archives/the_day_facebook_changed_messages_to_become_pulic.php)

<comment><USER5> But in the update they also describe problems with the setting; it doesn't update properly when the user changes it. Eventually FB will get this ironed out. But where is the QA on this effort?

</comment></comment><comment><USER6> I had a friend offering a blowjob to the person that could help her with her term paper. I can see this coming back to haunt her in a few years.

</comment><comment><USER7> Title is a bit misleading -- I think an edited original, "Facebook Status Messages to Become Public by Default," is more appropriate.

</comment><comment><USER8> Each day that goes by my Facebook account dies a little bit and I tweet a bit more.

<comment><USER3> You're one of a few hundred thousand people who think that in the midst of a sea of a hundred million who are just getting used to Facebook.

Luckily, the Internet is free so everybody can use whatever they please. You can even Tweet your Facebook status messages if you don't want to disappoint anybody.

<comment><USER4> Although Twitter is minuscule compared to Facebook, because it's always been public by default, its influence may be greater in some ways than Facebook. That's the whole point underlying this article: clearly Facebook has been having some Twitter envy for awhile, and they've decided that if they can't join them, they'll try and beat them.

<comment><USER3> I get that argument a lot here, and I've written lengthy counterarguments before, but that's not at all the case. Look at the features that Facebook introduced to the Internet. They introduced the public feed in the first place, they introduced status updates, peopletagging, granular privacy settings, friend metadata, a whole slew of features I'm certain I missed or am hesitant to credit Facebook. Without Facebook, you don't have Flickr's photo privacy, you don't have FriendFeed, you don't even have Twitter.

The idea that Facebook thinks it has to "beat" Twitter is ludicrous.</comment></comment></comment></comment>


Show HN: my weekend project - Scrapious, to export bookmarks from Delicious - <AUTHOR> https://chrome.google.com/webstore/detail/nbahmnpelbdcmkpllmmadklmmienpggd

<comment><USER2> Looks like an interesting project. Any reason why you chose Chrome over anything else?

<comment><AUTHOR> It's a webapp; it could be in the form of a simple website. But it scrapes with Ajax, and you can't do that from an ordinary website (same-origin policy).

HTML5 should let the app ask for permissions, but this is seldom supported.

It would be possible to scrape from a server, but that would expose the same IP over and over again, which would be very easy for the target to block; a client-side scraping tool is run from many IPs, making detection and blocking a lot harder (I hope).

Finally, while coding the core of the app is enjoyable, cross-testing css rendering differences is a big pain. Publishing this as a Chrome webapp is a way to make sure every user (all three of them) uses the same browser.

<comment><USER2> Make that all four of them :)</comment></comment></comment>


Understanding the hot hand, the myth of the hot hand, and the myth of the myth - <AUTHOR> http://andrewgelman.com/2014/08/12/understanding-hot-hand-myth-hot-hand-time/

<comment><USER2> Here's some context:

[http://en.m.wikipedia.org/wiki/Hot- hand_fallacy](http://en.m.wikipedia.org/wiki/Hot-hand_fallacy)

<comment><USER3> Except the argument in sports is exactly this:

       > Athletes are not machines, and anything that can
       > affect their expectations (for example, success in
       > previous tries) should affect their performance—one
       > way or another. To put it another way, there is little
       > debate that a "cold hand" can exist: It is no surprise
       > that a player will be less successful if he or she is
       > sick, or injured, or playing against excellent
       > defense. Occasional periods of poor performance will
       > manifest themselves as a small positive time
       > correlation when data are aggregated.

Basically the question of what factors could be present that would increase the success likely hood of a basketball scoring attempt, and then whether or not those factors can persist over time or if they are always just noise. And the myth part being that someone who believes they will be successful is more likely to succeed than someone who has no expectation of success.

The science then is can the expectation help, and if so can it bias success in a statistically significant way.

</comment></comment><comment><USER4> > (3) not every streak is a hot streak, so the real infrequent, but persistent > hot hands get diluted;

no true scotsman would have a not-hot streak.

<comment><USER5> On a purely mathematical level, though, you can see how this could be true.

Like, take basketball out of the equation. Let's say I code up a button. When you press it, it shows a screen that says "You scored!" or "You failed to score. :(" Now, I code the button to generally say "you scored" 1/2 of the time, and "you failed to score :(" 1/2 of the time. But I also code it to say that on 1% of all scores, it enters a "hot streak" mode where for the next ten button presses, it's 80% likely to score.

We would agree, yes, that this button has hot streaks?

But now suppose that you look across a large dataset for periods of ten button pushes in which more than 7 of them are scores. Those are streaks of scores. Some of those are going to be because the button is in hot streak mode. But some of them are just going to be because the normal 50% chance clustered.

Now, is my button anything like a good model for how basketball works? I have no idea. But the sentence "not all streaks are hot streaks" is not self- evidently crazy.

<comment><USER6> Or to pick at the semantics a bit: Not all streaks indicate causality. Some are random artifacts.

Easy enough to prove by flipping a coin. Streaks naturally happen and should be _expected_ from randomness.

</comment></comment></comment><comment><USER7> I spent a long part of this article thinking it was about poker.

</comment><comment><USER8> as with most real world situations, the problem (in my estimation) is that statistics and the experimental model are not yet sophisticated enough to sort out the effect, so it's hard to distinguish from random. the statistical model would likely need many more variables, be non-linear, and measure things (like emotional positivity) more dynamically and/or accurately than possible today.

for example, my shooting shoulder is chronically injured. if i warm up properly, i seem to shoot better. is this a psychological effect? a physiological effect? something else? in game situations, it's hard to isolate the effect from all the confounding variables. even if you could, you'd need to measure all the confounding variables to negate them in the analysis.

i suspect a non-logic based "math" will have to be developed (quantum computing?) before we can understand these kinds of situations.

</comment><comment><USER9> It appears academics argue much like folks on the Internet argue, just with different vocabulary and maybe a touch nicer.</comment>


Twitter acquires Clutch.io - <AUTHOR> http://blog.clutch.io/post/29340796276/clutch-joins-the-flock <comment><USER2> I don't want to moan, but I wish we could get out of this mindset of reporting "company X acquires company Y" when in fact it's a case of company Y shutting down and the founders going to work at company X.

Congrats anyway.

<comment><USER3> We took good care of all our investors

<comment><USER4> I think mbitter's point might be better explained by an example from outside the world of computers. Audi (high end cars), for example, recently acquired Ducati (high end motorcycles). No-one expects Ducati to be shuttered. Instead, people are expecting both companies to remain and to share their IP and improve both the cars and and motorcycles as knowledge flows from both companies.

In the land of computers being acquired tends to mean the acquired co. shutters their business, and everything they worked on ceases to be available to the customers who made them a company worth acquiring. I seriously applaud your allowing customers to set up their own servers. It's quite atypical... but it's kind of a dead-end thing unless you open-source the code. "Hooray... We can continue... but things will never improve, bugs will never be fixed, and our needs will eventually exceed the capabilities of the software that no- one is working on anymore."

Wouldn't it be nicer if the acquiring company valued the company being acquired enough to keep it going, or roll its functionality into its own offerings so that customers can continue being served?

<comment><USER5> The difference here is that the greatest asset of Ducati is it's brand as a high-end motorcycle producer. Thus getting a return on this acquisition from Audi's perspective means continuing to deliver on that brand's promise of awesome, pricey motorcycles. (Which incidentally makes good money for Audi as well)

The greatest asset of Clutch.io is the technical team and their ability to produce well-crafted code which means getting a return on the acquisition means deploying that team toward important projects at Twitter which could help them grow their userbase, engage existing users and monetize those users.

(Disclaimer - I have not spoken to the team about why they were acquired and what they'll be working on, but my statements are representative of what the majority of these types of acquisitions are like)

</comment></comment></comment></comment><comment><USER6> When this happens, I personally would love to see a blog post by the founders on how the soon-to-be-shut business was actually doing, so that current competitors and other startup-watchers can learn. You're anyways leaving the space open, so better to leave the lessons behind.

</comment><comment><USER7> Congrats to the two Erics - and I love the way you're handling the transition and existing customers, seems very classy.

<comment><USER8> It's inline with how those guys are. Classy hackers.

</comment></comment><comment><USER9> Is it just me, or does there seem to be a plethora of companies named in the form something.io nowadays? Feels like it's the next version of Flickr and all the diminutives missing that 'e'.

<comment><USER10> .io seems solidified as branding indicating that the business is oriented to developers, being a niche of "B2B." It may feel like a fad but the TLD is valuable for reasons other than availability, unlike deliberate misspellings you cite such as Flickr.

</comment></comment><comment><USER11> We're a new TechStars company who've built a super simple mobile A/B testing tool. Could be a great alternative to Clutch. If you'd like to try it out, or give us feedback or ideas, ping us at founders <at> leanplum <dot> com.

</comment><comment> k-mcgrady Congratulations to the team! They had a really cool product and the way they are handling the acquisition is great. Glad they aren't just killing it.

</comment><comment><USER12> While we can't blame any individual team taking an aquihire deal, I have to wonder if their prevalence these days is undermining future startups and sending potential customers back to an old school mindset of going with large vendors who are less likely to shut down.

After all, why work with a company made up of a few 25 year olds testing out a market, who may "pivot" any time, and will shut down their product when some big company wants to hire them? Seems like a justifiable choice to go with an inferior product from a profitable established company which is much less likely to disappear in two years.

Even as a startup guy, I'm hesitant to build on startup products. YC Startups may be even worse in this context as they are frequently picked off and shut down, and the founders are young and not necessarily in the product for the long haul.

</comment><comment><USER13> Are there any alternatives to Clutch.io?

<comment><USER14> Take a look at our side project Switchboard: easy A/B testing for mobile

<https://github.com/KeepSafe/Switchboard> It's open source

My co-founder Philipp is talking about this project at next week's Silicon Valley iOS Dev Meetup <http://www.meetup.com/sviphone/events/38752252/>

</comment></comment><comment><USER15> Congrats, this is interesting because Eric Florenzano one of the co-founders co-founded Convore.

<comment><USER16> actually, both erics

</comment></comment><comment><USER17> If I may ask, what issues were the Cluch.io guys having building a sustainable business around A/B testing for mobile?

</comment><comment><USER18> Congrats to the Team at Clutch.io :)

</comment><comment><USER19> Congrats and thanks for keeping the idea up; a noble gesture.

</comment><comment><USER20> Whoa! Congrats to the Erics, awesome news, guys!

</comment><comment><USER21> Congratulations to the two Erics!

</comment><comment><USER22> >We're happy to announce that over the coming weeks we will make available everything you need to run Clutch.io on your own servers, so that even after our hosted service is no longer running, you can continue to operate it on your own.

Everyone take note. This is how you handle an aquihire correctly

<comment><USER23> Agreed. Seems like everyone is a winner given how they are handling things.</comment></comment>


The "overlearning the game" problem - <AUTHOR> http://andrewoneverything.com/the-overlearning-the-game-problem <comment><USER2> It's related to Goodhart's Law:

 _Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes._

This is often the result of attempting to overoptimize a system. You can optimize a race car to a huge degree, because you know _exactly_ what you want it to do.

You can't optimize a schooling system, because you _don't_ know exactly what you want it to do. A little noise is a good thing, because the you want a little wiggle room for teachers to sidestep the dictums of education czars, and students to sidestep the dictums of teachers.

The Greeks solved this quite a few years ago, with sortition. Under sortition (injecting noise into elections - <http://en.wikipedia.org/wiki/Sortition>), Bush and Gore would have been forced to pay "paper, scissors, rock" for the presidency. Under the US's more pure democracy, they would have been tempted to make all kind of Faustian bargains with sordid players to nail down the last 0.01% of votes.

Randomization means that the last percent is just not worth chasing, so players in a competition won't be tempted to bend the rules for a tiny advantage.

The same process could be used for tests. If you allocate places in desirable courses (say medicine) randomly to anyone above a certain score, the top students won't bother drilling as hard just to get the top score.

Stocks are the same - quants wouldn't sweat timing as much if their placement in order books was randomized. It would be more efficient to pay attention to fundamental value than momentary fluctuations if they weren't guaranteed to make large profits on the momentary fluctuations. Some would still work on timing, but not as many.

Patents are just bad policy badly implemented at the moment, not over- optimized.

<comment><USER3> I don't think it is a good idea at all. If the political power boiled down to picking paper over scissors, the loosing side may very well ascribed no legitimacy to the outcome. That is almost certainly going to end far worse than actually counting all the votes (i am looking at you, supreme court).

The same problem happens with your exam example. Why is it fair that someone less competent than you got in? A better solution would be to let everyone over the cut off point in.

I can't disagree with your example of the stock market, so long as the trades were still executed with a few seconds.

<comment><USER4> It would just be recognizing the jitter that exists _inherently_ in the system. You will always get a different number when counting the votes, so you can end up with a "50% voted for both" situation.

Why is flipping a coin worse than pretending the next count will be more correct? You kind of contradict your own point by implying the supreme court didn't supply legitimacy to the election. Nobody can blame a quarter for being stiffed with neocons on the heads side.

<comment><USER5> The trouble is that getting people to recognize that is really hard, and just changing how the system works probably won't do it. If you want to politically antagonize nearly any American with a strong political affiliation, regardless of what that affiliation is, just tell them that the margin of error in the 2000 presidential elections was far larger than the difference in votes, and therefore both candidates were about equally legitimate winners.

</comment></comment></comment></comment><comment><USER6> If you pay a man by the hour, he'll work a lot of hours. If you pay him by the brick, he'll lay a lot of bricks.

These "games" are basically the equivalent of counting lines of code or checkins. We're measuring poor proxies instead of the things we're actually interested in. The solution isn't an arms race to build bigger and better proxies, the solution is to measure real things instead of artificial ones.

Here's just one example of what I mean by "measure real things". Electing representatives every X years to decide the laws of the land was once upon a time the fairest and best way to have the voices of the masses heard. Today it is feasible to directly poll everybody about every issue, so we no longer need the proxy. If you say everyone cannot be educated about every issue, fine, I can "follow" PG's votes on wall street reform and grellas's votes on IP tort reform and Schneier's votes on TSA etc just by copying their votes on those issues into my ballot, a permission which I can revoke at any time or on a vote-by-vote basis, as easy as unfollowing them on VoteTwitter. This is better than the proxy of professional politicians deciding every issue with fixed terms.

<comment><USER7> As you allude to, the problem with direct democracy is that people are stupid. Not all people about all things, but most people about most things. The realm of human knowledge is simply too vast for anyone to have a broad command of, so we specialize. This means that, for any given issue, the vast majority of people are ignorant about its specifics. Thus we cannot expect the masses to make wise decisions about most issues. The obvious counter-argument to this is education, that educating the public on the issues at hand would lead to better decisions. However, that would fall victim to the same demagoguery and corruption that we see infecting politics today.

By way of illustration, think of an issue that you care deeply about and are well-informed on. Now ask yourself if you would trust the public at large to make policy on it. For every pg, grellas, or Schneier, you'll have far more popular Becks, Palins, and Bachmanns. Direct democracy would be inherently susceptible to such demagogues, and as such cannot be seen to be any more reliable, or likely to produce wise policy, than the current system.

<comment><USER8> >> The realm of human knowledge is simply too vast for anyone to have a broad command of, so we specialize.

This statement could actually be used just as easily against voting for representatives as we do today. The real question is do you trust public at large to make a better or worse opinion than <name your representative> on some issue you are deeply informed on? In other words the comparison should be versus the existing system as a whole, not versus an expert on a single topic.

My perception of the demagogue is that their power comes from railing against the current authority. So while I can agree that it seems likely that a 'true democracy' would instantly prop up the Becks and Palins of the country, who do they blame when a majority of people vote that they are wrong? Or maybe even more extreme who do they blame when their views and proposals are voted for but still fail?

There is no longer a 'bad guy' to blame and arguments put forth by a demagogue will have to deal with two new realities-

1- They can no longer claim they are part of the majority opinion in an effort to gain further support. It'd be hard for Sarah Palin to keep up her 'regular joe' persona and reconcile it with being in major disagreement with a majority of the country

2- They can no longer claim obstruction by some authority steam rolling them in legislature. It would become pointless to argue about which pundit supported which policy or law in the past, because every individual would have their personal voting record.

<comment><USER7> >> This statement could actually be used just as easily against voting for representatives as we do today.

True. I'm no big fan of representative democracy, it just happens to be the least bad system we've tried.

>> The real question is do you trust public at large to make a better or worse opinion than <name your representative> on some issue you are deeply informed on?

Well, yes, because they are at least (usually) experts in business or law, which are the domains that are most important to government. Some will be military or foreign policy experts, and they will (hopefully) be appointed to the relevant subcommittees. It's a fairly broken system, open to all kinds of abuses and with little assurance of wise leadership, but I still think the competence of the average politician in crafting policy is higher than that of the average citizen.

>> My perception of the demagogue is that their power comes from railing against the current authority.

I think there's quite a bit of truth in that. I don't, however, think it's a necessary truth. Hitler was a classic demagogue, even and especially after he gained power. He appealed to antisemitism and a fierce nationalism, rather than to the incompetence of politicians. While the current crop in the US gain power by railing against the established political class in DC, that's not the only possible source of power for them.

>> It'd be hard for Sarah Palin to keep up her 'regular joe' persona and reconcile it with being in major disagreement with a majority of the country.

Reconciling her views with reality has never been particularly important to her, and I don't see how that would change in a direct democracy. She could just as easily rail against the liberals in the MSM who poison the electorate with their propaganda.

Ultimately, I picture direct democracy working quite similarly to the reddit hivemind, and I'm not convinced that's a good thing. It would likely be passionate and dynamic, but liable to go off the rails from time to time.

</comment></comment></comment></comment><comment><USER9> Systems theorists say that "structure predicts behaviour". It's a bit trite, but also deep. Here's an example.

The US political system pretty much guarantees bad budgeting. Members of Congress are elected fairly independently. There is no party discipline, so each member will operate independently to maximise pork. This encourages horse-trading within and across party lines; nobody can be forced to give up something for a general good.

There's more: there's no incentive to balance the budget. The Executive's separation means that Congress does not need to concern itself with proper administration; it only doles out the cash. It has every incentive to ... maximise pork.

The dynamic behaviour of American government arises from the static structure. The drafters of the constitution drew on their knowledge of history and current affairs to try and avoid certain pitfalls. The US Constitution was state-of-the-art when it was written. It's less so now.

Countries where the Executive is formed out of the Legislative -- the Westminster system -- tend to have much stronger party discipline, because that discipline is required to pass budgets, enact legislation and to form the Executive. This tends to almost eliminate horse-trading, except between parties and independents. It's not perfect -- whole parties can engage in pork too -- but when policy emerges that benefits the many at the cost of a few, countries with party discipline will find it easier to adopt than those without.

Australia, which has the amongst the toughest party discipline in the democratic world, is also a reform leader. And I think a lot of that is explained by our constitutional arrangements.

</comment><comment> loup-vaillant I think "overlearning" is not a good term (edit: it is quite accurate, though), because it tend to suggest the worst solution of all: that we _refrain_ from learning. I prefer "lost purposes"[1].

The primary purpose of a game is generally to Have Fun. This purpose is lost if you have "solved" the game. The stated purpose of patents is to foster innovation. However it doesn't work[2]. The purpose of schools is learning. However its methods are flawed [3,4]. And so on.

The trick is to know your goals, and then find out means to best achieve them. The author said:

> _But, in real life, we need to keep "playing the game": we need to have > elections, and protections for inventors, and laws that govern society, and > a market where companies can raise money._

But the _actual_ goals are different: We don't need election, we need a working democratic system (which may, or may not, mandate elections). We don't need protections for inventors, we need innovations. We don't need laws, we need a fair and working society. We don't need a market where companies can raise money, we need a working economy.

Well, I could attempt recurse further up until pure morality, but that would be intractable. But at least you get the idea. If something looks broken, think about its ultimate purpose before you try to fix it.

[1]: <http://lesswrong.com/lw/le/lost_purposes/>

[2]: <http://www.dklevine.com/general/intellectual/against.htm>

[3]: <http://vimeo.com/5513063> (Dr Tae)

[4]: <http://www.khanacademy.org/>

<comment><USER10> Overlearning/Overfitting is a term from machine learning/statistics that means that a learning system fits its learning data so well that it does not generalize to new data. The examples given in the article seem like perfect examples of this, e.g. elected officials fit the election process very well, but do not "generalize" to leading a country.

<comment> loup-vaillant Ah. I didn't see it as jargon. I recognize the term is accurate, but my main point still stands: the _connotation_ is still there, and still does suggest we should learn _less_ about those systems if we want to preserve their original purpose.

Worse, the end of the post suggest we _have_ to preserve things like schools, markets and elections, even though we learned enough to spoil them. Therefore, the only thing left to sacrifice is learning itself.

That would really really sucks: I'm curious, and I always want to learn more. There are many things I would sacrifice before I accept to put hard limits on what I am allowed to learn.

</comment></comment></comment><comment><USER11> Hey USA, your first past the post system actually does tend towards dual party governments [0]. Compare it to preferential system where voters can pick several parties or vote for one party for the preferences listed.

For example, look at this: <http://www.abc.net.au/elections/federal/2010/guide/deni.htm> With USA's system, Australia Labor Party would have got re-elected.

There's still improvements to be made for democracy that doesn't need to involve such fancy technology yet [1].

[0] <http://en.wikipedia.org/wiki/Duvergers_law>

[1] <http://techpp.com/2011/06/08/homomorphic-encryption/>

</comment><comment><USER12> I'm reminded of Mechanism Design: a Nobel prize winning theory of economics that starts with the supposition that agents in any system will exploit its rules to maximize their personal gain. The corollary, which I've been quite taken by, is that where we can influence the rules of the game, we should design them such that exploitation serves a social good.

</comment><comment><USER13> Many people see this from a macroeconomic or political point of view. So I think adding a more philosophical point of view might also be of value. Of course, because it is philosophy, I can only present my own point of view. There is no right or wrong.

For me, as a Zen student, the solution is to just accept that that it is, what it is. No system (shape, for Zen students) can be perfect. It is created to solve a problem and with time starts to fail badly at doing anything about the problem. Then another system is created by someone else and the cycle restarts. You might think for example that the stock market or democracy is a thing that doesn't change, because it exists longer then you live. But in the end it will change. We had different political systems before and we will develop different systems in the future.

So in the end the system is one of the most important things we have, because it gives us something to base our decisions and actions on and goals to strive for. But also the system is nothing, just an illusion we create for us, maybe based on how we understand the illusions other people created for themself.

The thing that is interesting to me personally is that every system itself is instable and will change or die in the end. But the life cycle of a system, what it does for us and doesn't, that all will always stay the same and even though we try to change that, we will never succeed. So while it changes a lot in one way it is totally unchangable in another. But that just as a side note.

</comment><comment><USER14> The cynic inside me would extend this problem to the entire economy or even human society itself. Any economic transaction is supposed to benefit both parties: the buyer offers an amount he is willing to pay for what he wants, the seller provides his goods or services at a price he is willing to accept. As PG might say, wealth is created in tandem with creating stuff that people want.

The problem is that some elements in society have become extremely effective at creating a perversion of "stuff people want" for their own benefit. They exploit loopholes in the system, whether by preying upon the poorly-informed, shifting costs onto externalities that we can't price properly, engaging in corruption, or producing items of questionable actual value but very attractive perceived value. They take advantage of our desire for easy answers to our problems with minimal effort expended.

Case in point: Ponzi schemes, coal power plants, Halliburton, the tobacco industry, Zynga, and even to an extent, religion

We've tried to control this with laws, education, and social norms, but ultimately it seems the invisible hand reigns supreme.

</comment><comment><USER15> This article seems to be addressing 'the spirit' of a document or system.

Perhaps another way to look at the problem is to imagine creating an A.I. that you want to succeed at whatever system you present. In most cases, an A.I. will take the literal interpretation of the system and become a test-taker, an electable 'gotcha-game' politician or even an entity that finds it can maximize game theory to its own ends by complicating the rules of an existing system to the point of absurdity once it becomes powerful enough to modify and create rules.

So then how do we create systems resistant to beings that take everything literally? I suppose the only way is to reward certain outcomes as opposed to rewarding the direct product of the system itself.

Examples: After an election, have we elected someone who has met with a high degree of favorability in the electorate by the end of his term?

After having students become proficient test-takers, do they then become excellent doctors, lawyers, engineers, etc.. ?

In a game invented to be fun or fair, once overlearned, do they produce fun or fairness?

If not, then a new game needs to be created or the existing one might need to be extensively modified to produced the desired product. This is where the internet shines, where everyone is welcome to take an existing system and modify it to something better. The problem with politics, law, stock market, etc... is that they have become the only method adopted in real society (there is only one game in town).

If reality were allowed to adopt, incorporate or evolve from systems/games from virtual reality (i.e. internet) there might be some productive change. But first we would need to see the first step taking place, that being even flawed virtual systems are allowed to manifest in significant proportion within real life society.

Don't hate the game, hate the player. Every game that profits a winner will have its cheaters.

</comment><comment><USER16> We run into this situation in programming quite a bit. The answer is refactoring: taking the principles and patterns that work, streamlining them, ditching the cruft that works for a few edge cases but mostly gets in the way, simplifying the underlying metaphor of the system, and re-asssembling.

The only way to do this with much larger systems, such as systems of governance, is revolution or exploring new lands. Personally I'm a bit concerned about revolution -- the assumption with refactoring is that the people refactoring understand what the "good" parts are and what the "bad" parts are. They also need to be able to generalize and simplify in order to keep the system cognitively approachable. In my experience, it's very easy to be angry-tear-down-the-system guy, very difficult to actually refactor. As the author points out, it's not that these systems are entirely useless. The hell of the thing is that the reasons for creating these systems are still very valid.

I remain convinced that programmers (hackers) have a lot to add to the discussion when talking about complex, brittle systems. After all, we spend a lot of time both working with them and fixing them. To me, programming and systems architecture is applied philosophy. Very cool stuff.

<comment><USER9> Refactoring is what political types call 'reform'. Reformations are safer than Revolutions, because revolutions dissolve all civil structure. They allow the ruthless sociopath to prosper.

What programming has taught _me_ is humility in the face of complexity. Refactoring only works because we can, at a stroke, reverse our changes; because we can place those changes in isolation and test the heck out of them.

But society, constitutions, laws: they are not software systems. A revolution can't be rolled back. A constitutional change is a hotfix applied to a running system. It's important to be careful.

You might find reading Burke on the French Revolution interesting; also Oakeshott on conservatism (the intellectual kind, not the god-n-guns kind).

<comment><USER16> I think we completely agree. I did not mean to imply anything but humility was indicated here.

"Reform" is a big, fuzzy word. A lot more brittleness and complexity has been added under that rubric. Systems of governance eventually get "over learned", that is, people adapt to the system and use it in more ways than it was intended. The ability of hundreds of millions of people to adapt is much greater than any amount of complexity that we can create. Yet we persist in the idea that by continuing to "tweak" the system, somehow magic will come from it. To me this is where humility comes in. We can never create a perfect government. The best we can do is have one that balances stability and refactoring.

Election cycles and constitutional amendment processes were supposed to do this. What we've found, though, is that legal precedent, statutory code, and an expansion of pure democracy and the role of career politicians, given enough time, puts us exactly where we didn't want to go.

Interesting side note: overly complex and brittle systems that people execute actually give dictatorial powers to those responsible for executing them. That is, in a computer system, the complexity creates program crashes and dysfunction, because the computer has to treat each instruction cycle and piece of data the same. In a system executed by people, a subtle social goal takes over: things still appear to happen. The system still superficially looks like it is working somewhat -- those in authority just selectively apply and bend the rules depending on whatever their whims are. If you've ever worked in a large organization with too much process, you've easily observed this: most struggle under burdensome rules, never being able to get anything done. A few, however, work completely outside the system, given special permission. These folks are usually kept off the radar so as not to upset the troops. Complex people systems don't ever _stop_ working, they just make things miserable on most everybody and give a few permission to do whatever they want.

You'd think that modern tyrannies would look like old Libya: one guy in a funny hat ruling totally. But that's not the way it works any more. It's not one guy, it's a distributed oligarchy, and it's not out in the open, it's obfuscated.

</comment></comment></comment><comment><USER17> Less Wrong talks about this a lot; their term for one variant of it is "superstimuli".

What's the difference between your "overlearning" and "hacking"? They sound like the same thing.

This phenomenon is the reason for the Wikipedia rule, "Ignore All Rules".

<comment><AUTHOR> > What's the difference between your "overlearning" and "hacking"? They sound > like the same thing.

They are similar, but I think one difference between "hacking" and "overlearning" is that a hacker can learn to subvert/circumvent a system quite quickly, and usually does so in a way that breaks the rules. Overlearning usually takes longer to achieve, and is a more insidious problem because it achieves its goals without actually breaking any rules.

<comment><USER17> It depends on what you think "the rules" are. Generally speaking, hacking happens in a framework that has unbreakable rules, whether those are rules of a game, of a CPU, or of an OS. A solution becomes a hack when it exploits those rules in a way that was previously thought impossible.

Of course, there may be _other_ , breakable, rules that are broken — "You can't render 3-D in real time on a PC!" — but those aren't the relevant rules.

Many hacks are not quick at all to achieve.

<comment><AUTHOR> I guess an example of hacking would be someone hacking the phone system to make free long distance calls. An example of overlearning would be someone learning the rules of society to such a degree that he gets free long distance calls (e.g. using lawsuits against phone companies).

The former would get fined or imprisoned if caught, but the latter is fine under our legal system.

<comment><USER17> It's pretty bizarre that you're using the derogatory definition of "hack" given that you're posting on _Hacker News_. And it doesn't incline me to want to talk to you further.

<comment><AUTHOR> Sorry, I was just using the colloquial meaning of hack. Did not at all mean to appear derogatory for the type of hacking exemplified by HN, of which I take part in.

In any case, I do see your point about similarities between hacking and overlearning, but I think there are some differences. If I can articulate those difference better, I'll reply a little later.

<comment><USER18> I think overlearning can be considered to be a subset of hacking. In both cases laws (as in physics) are exploited however while hacking may or may not break rules as imposed by society overlearning does not.

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> Very interesting post!

However, I got hooked on thinking about the childhood game that he mentions briefly in the introduction. Asking more and more questions about an object and wanting all of them to be consistent is a good description of mathematics. According to Godel, both teams are _bound_ to lose, because you cannot create a system of descriptions/properties about a system that are self-consistent (as their number increases), there will always be questions to a team whose answer will be inconsistent with the previous set of questions. The game is then to see which team can push the inevitable further.

IDEABOLT: It would be interesting to develop a program that plays this game. Each answer could be stored as an RDF statement in database.

</comment><comment><USER20> There was a book written in the 80's that covered the overarching philosophical discussion on this kind of thinking about games. The author broke games into two categories: finite and infinite.

Finite games like the one invented by the children described by the OP have end states, they have winners, and the goal within them are always framed from that perspective. While infinite games have no winners, have no end, and the goals are often framed around ensuring that the game never ends.

It's easy to think of finite games in our lives, they are everywhere in our society, and the OP does a good job of pointing out some of the less obvious cases. Infinite games are less obvious, the one that I found most illuminating was "language", a game where people actively collude to extend the game to the end of time, and has no winner.

Anyay, it's an interesting read, but a little bit mumbo-jumbo by the middle of the book. I would recommend it.

[http://www.amazon.com/Finite-Infinite-Games-Vision- Possibili...](http://www.amazon.com/Finite-Infinite-Games-Vision- Possibility/dp/0345341848/ref=sr_1_1?ie=UTF8&qid=1314621221&sr=8-1)

</comment><comment><USER21> I think it is time we recognize that it is not the systems that shape our world but the individuals.

There is no optimal system, be it political, economic or whatever.

You can always find loopholes.

I think it is time we take responsibility for what we do - then there is no need for a better system.

The patent system is not responsible for people attacking each other - its the people.

</comment><comment><USER22> Great post. I face this everyday in my job at an "old media" company. There are policies, rules and constraints in place which served some purpose at some point in time, and haven't been challenged for years. Unlearning the rules of established games is almost as important as evolving them. Everything should be questioned and challenged to find the root of the point of the rule/law/restriction and if it doesn't make sense any longer, throw it out.

</comment><comment><USER23> Doesn't "Exploiting the System" capture the same idea? I think basically what people are doing here is understanding the core weakness that is intrinsic to the system(which probably is not perfect) and using(abusing) it to their advantage.

</comment><comment><USER24> Regarding politics, it is interesting to note that the people have not, for their part, also overlearned the game of electing the best man for the job. I wonder why this asymmetry exists?

</comment><comment><USER25> Have you noticed that yesterday every comment was concise, and now that applications are open, every comment is elaborate, long-winded, and footnoted?

</comment><comment><USER26> Seems most of the things mentioned are an inevitable result of centralizing and expanding government power. Someone is being gamed alright.

</comment><comment><USER27> People are predators and that's what predators do, never stop searching for what they need. Predatory behavior is the problem, and there is little will in successful predators to outlaw there own behavior.</comment>


Colonia Dignidad: The Curious History of a German Cult in Chile - <AUTHOR> http://www.theamericanscholar.org/au08/torture-falconer.html

<comment><USER2> This is the sort of thing you'd see on early reddit, and I wish there was more of it. I'd guess that half of you think this is "not hacker news", or not appropriate, or whatever--so I wonder where this _would_ be appropriate. Anyone know?

<comment><USER3> this may be a stretch, but I would say its appropriateness could be tied to the social engineering involved, and how far astray and damaged these people became as a result. In the building of our social web we need to be aware of such cases and make sure that our networking is truth-friendly. I guess this is a huge discussion, but I find the article very relevant to say the least. In any case, the 'virality' concept that is so web 2.0 was discovered by religions millenia ago.</comment></comment>


Share your Mouse and Keyboard - <AUTHOR> http://www.keyboard-and-mouse-sharing.com <comment><USER2> I'm actually more interested in seeing a system with two mouse/keyboards on one huge screen that two people can share. Then two people can browse together.</comment>


StartHQ Capitalizes on Chrome's Change of the New Tab Page - <AUTHOR> http://www.arcticstartup.com/2013/09/30/starthq-capitalizes-on-googles-change-of-the-new-tab-page

<comment><AUTHOR> All this wouldn't have happened without Finn ([https://twitter.com/finnpauls](https://twitter.com/finnpauls)) so big thanks to him!

For the growth hackers amongst you - the obligatory hockey stick curve is here: [https://www.dropbox.com/s/g8c15ks93mo519e/hockey- stick.png](https://www.dropbox.com/s/g8c15ks93mo519e/hockey-stick.png)

Keep in mind that the visitors to [https://starthq.com](https://starthq.com) arrived via an interim Chrome extension page - that has been getting 3K uniques a day for two days, so just so you know what kind of traffic to expect on a slow Saturday with 40 odd points on HN.

Original submission here: [https://news.ycombinator.com/item?id=6462056](https://news.ycombinator.com/item?id=6462056)

Also a bit of a technical detail, but the amount of traffic we've been getting as a result exposed a memory leak in our server running Node, which we're now patching up. Fun times.

</comment><comment><USER2> This article seems to miss the fact that the old "new tab page" also showed frequently visited pages, but _also_ showed apps.

It took me a very, very long time to work out where my apps were when I went to use one because I had hidden the apps link on my bookmark bar a few days previously when I had first seen it. (My bookmark bar lacks space as it is!)

<comment><AUTHOR> So did I understand you correctly that you prefer to see the frequently visited sites, rather than the apps on your new tab page?

Also, what does everyone think about bookmarks on the new tab page - is that something everyone uses, or would just having a tile that links to chrome://bookmarks/ be sufficient?</comment></comment>


Ask HN: Where to start learning web development? - <AUTHOR> <comment><USER2> I'd definitely start with codeacademy. There's two ways I look at it, if you want to be front end and start easy then go into html and javascript front end design then work to the back end. If you want to be a software developer or engineer start with back end development aka C, C# etc, but it will take more math. Up to your own path. I'd also suggest a small project idea in your head you want to accomplish. I just wanted to create a website, did it and then lost my passion, but in the process of that project learned a lot. Lastly, paying someone on upwork.com a dev for example to keep you accountable and teach you javascript for $5 an hour is actually a great hack for me after I finished codeacademy.

</comment><comment><USER3> This [http://eloquentjavascript.net/index.html](http://eloquentjavascript.net/index.html)

</comment><comment><USER4> I used codecademy for HTML, CSS, JS, and SQL. I found their material to be simple and easy to follow. Their practice problems were quite helpful too. [https://www.codecademy.com/catalog/subject/web- development](https://www.codecademy.com/catalog/subject/web-development)

</comment><comment><USER5> Definitely recommend Udacity! They offer various free courses and nanodegrees in topics like full stack, mobile, and react. I've taken a few of their nanodegrees and supplemented with other online courses from Wes Bos and React Training to good effect.

</comment><comment><USER6> MDN has some great, up-to-date docs on the underlying techs of HTML, CSS, and JS.

[https://developer.mozilla.org/en- US/docs/Learn](https://developer.mozilla.org/en-US/docs/Learn)

</comment><comment><USER7> [https://www.freecodecamp.org/map](https://www.freecodecamp.org/map)</comment>