Timescale raises $16M from Benchmark, NEA, Two Sigma Ventures - <AUTHOR> https://blog.timescale.com/raised-16m-from-benchmark-nea-to-understand-machines-time-series-data-iot-9eff8213c15c <comment><USER2> The value proposition appears to be a much better time-series solution for the PostgreSQL environment, and in that regard it looks like it is well-executed. But I don't follow the assertion that it is targeted at machine-generated sensor data per se for a couple reasons. I'm not sure it is possible to hit the operation metrics/characteristics typically required for these workloads from within the PostgreSQL ecosystem. Interesting database implementation but I am not clear on the core technical differentiation and value proposition compared to other databases. Maybe someone from TimescaleDB here can fill in?

For example, what is the online insertion rate story? As a rough rule of thumb, I expect an inexpensive Linux server to drive a million records/second through parsing, processing, indexing, and disk storage because you need that kind of throughput for efficiency when dealing with sensor data. But I would not expect that from PostgreSQL -- it is optimized for a different workload. Does this bypass the normal PostgreSQL I/O path some how?

What is the scale-out story? One of the most difficult parts of dealing with sensor data models is that you need to continuously shift data and load between machines to get consistent performance. TimescaleDB's sharding model looks like it might actually be able to support this kind of load shifting but PostgreSQL has no such architectural concept and I don't see anything written about it in a cursory read.

What is the geospatial analytics story? This is a huge part of analyzing sensor data for many applications. PostGIS uses GEOS, which is the only high- quality geospatial geometry engine in open source, but is that integrated into TimescaleDB? Is there proper support for high-performance geospatial sharding?

Looks like a solid architecture and implementation though.

<comment><AUTHOR> Thanks for the detailed comments. Let me try to answer a couple of them.

\- We don't bypass the Postgres I/O in general, but have rewritten a lot of the insert path as it relates to managing inserts across many chunks (child tables in Postgres). You can see a detailed explanation and performance comparison of PG10 declarative partitioning vs. TimescaleDB here: [https://blog.timescale.com/time-series-data- postgresql-10-vs...](https://blog.timescale.com/time-series-data- postgresql-10-vs-timescaledb-816ee808bac5)

\- What we find is that in many sensor settings, at least when deployed at centralized collection points, you'll actually place your DB/storage infrastructure behind a front-end anyway (web interface, MQTT, SSL termination point, etc.), and then batch data through a data pipeline like Kafka. This means that when the data hits TimescaleDB, you can perform larger batch inserts (even 10K rows / batch), which is how a single-node Timescale can support 100,000s of rows/sec (millions of metrics/sec). More benchmarks: [https://blog.timescale.com/timescaledb- vs-6a696248104e](https://blog.timescale.com/timescaledb-vs-6a696248104e)

\- So while this isn't the native I/O speed of Linux, it's not just about raw write rate; you also get those other things that are critical for a robust data product: reliability, flexible and optimized query performance, etc.

\- Scaling _out_ OLTP workloads has in general been challenging, but there are some nice characteristics of time-series workloads that make this better. We're working on scale-out clustering right now, and hope to be able to announce some things later this year.

\- We fully integrate with PostGIS and other Postgres extensions. Some of our users customers are very much employing TimescaleDB + PostGIS for temporal+spatial analysis: doing things like asset tracking or logistics over time.

</comment></comment><comment><USER3> We recently integrated TimescaleDB extension into our PostgreSQL database at [https://nightwatch.io](https://nightwatch.io). For performance reasons, we decided to implement time based table partitioning. We were evaluating native PG 10 partitioning vs. TimescaleDB, and concluded the latter will spare us a lot of work, maintenance and headaches. We were right - we got table partitioning basically "for free", query performance improved, and there is no partitioning maintenance involved, everything is automatic behind the curtains with their concept of hypertables. Support is really good too, they are very helpful on Timescale Slack channel.

<comment><USER4> I had the same experience. I manually partitioned a pg10 db and it was quite a lot of work to make sure I had all the partitions I need. Then I switched to timescale and got it was just a breeze. I could partition a dataset both in time and in foreign_key with a single command, and performance (especially huge inserts) jumped up. Or to be more accurate, the performance degradation of my table which made each subsequent insert a bit slower than the previous was completely gone. All in all, very happy with timescaledb.

Congrats with the funding!

<comment><USER5> Thank you!

</comment></comment></comment><comment><USER6> I'm wondering if there is a way to use Timescale in combination with Citus.

We are well adjusted to Citus as we're using now for almost 2 years. For most data it's great but we do have some time series data where we have to do a lot manually to achieve desirable results. I'm wondering if there is a way to take advantage of both, but not sure how would the query distribution work in combination of the two.

<comment><USER7> Craig from Citus here. Its a question we get quite often. Unfortunately, I'm actually not sure. There is a chance they would play just fine together, and from a quick glance they likely would but we've not explored this ourselves yet. Since Citus would be running on the coordinator you'd place timescale on each distributed node and at that point what Citus passes down is standard SQL which timescale can act on. All that said we've not heavily tested it ourselves and there would be some open questions on designing the right experience.

We do have a number of users that are using Citus alongside the native time partitioning that was released in Postgres 10 with good results.

Edit: We also had this blog post in our pipeline that speaks to time partitioning with Citus which just went live today - [https://www.citusdata.com/blog/2018/01/24/citus-and-pg- partm...](https://www.citusdata.com/blog/2018/01/24/citus-and-pg-partman- creating-a-scalable-time-series-database-on-PostgreSQL/)

</comment></comment><comment><USER8> I set up a time-series Postgres 9.6 database last year with traditional table partitioning (based on entity id, not time). I looked at TimescaleDB but it was about a month old at the time so I wasn't comfortable recommending it. Also I got the impression that it was little more than a wrapper around traditional partitioning, just to make it easier to use. Looking at their Github now it seems to be more than that though. I will have to give it a closer look! But also you might want to clarify your literature about what's going on. Is it something like cstore_fdw with its own files (but not built on FDW)?

I have a Postgres extension of my own for fast timeseries inserts that gives you back the data as an array ([https://github.com/pjungwir/floatfile](https://github.com/pjungwir/floatfile)), but it is much less developed and comes with a lot of caveats. My own approach is very simple: just make a file of 8-byte floats and keep appending to it. Is timescaledb something like that (but better)? One of the drawbacks with my approach is that you have to re-implement a lot of things (like histograms). Even though you say you support "all of SQL" it looks like you are similarly re-implementing things that would be slow otherwise (histograms).

Also, I see a lot about insert performance, but I keep getting projects where inserts aren't challenging but getting fast analytic queries is. I'd love to see some examples/numbers there.

I really like the idea of keeping timeseries data quasi-connected to your main RDBMS, so I'm thrilled to see successful projects in that space. It can definitely be challenging. I wonder if something like influx_fdw is possible and would make sense? (I've never used InfluxDB.)

<comment><AUTHOR> Thanks for the comments, and it sounds like you want a deeper dive as to some of the design decisions around TimescaleDB.

What might be useful to start is two of our technical blog posts around how we design internal partitioning [1], as well as how this compares against declarative partitioning in PG10 [2].

We aren't rewriting the lowest storage layers like cstore_fdw, so we can continue to use Postgres' heavily-tested storage layers....as well as work with its existing streaming replication and recovery mechanisms. We also aren't packing individual writes into files (like it sounds you are doing with floatfile), arrays (like in tgres), or other TOASTED data structures, although we might be compatible. We certainly do support the range of types (arrays, JSON/B, etc.) in our data rows.

Fully agree that projects differ on their needs of inserts vs. queries time, but in addition to much higher insert rates, our general data model and aggressive constraint exclusion across chunks provides meaningful time-based query improvements. We also have all sorts of planner optimizations, e.g., we taught the planner that querying for the data by time rounded-to-some-time- bucket preserves the sort on time --- which is useful for all kinds of ORDER BY/GROUP BY queries. We've published various benchmarks around this [3].

Finally, there are a lot of changes needed to "just make things work", which also includes support for constraints, foreign keys, upserts, triggers, etc., which enable capabilities like data retention and continuous aggregation policies.

[1] [https://blog.timescale.com/time-series-data-why-and-how- to-u...](https://blog.timescale.com/time-series-data-why-and-how-to-use-a- relational-database-instead-of-nosql-d0cd6975e87c)

[2] [https://blog.timescale.com/time-series-data- postgresql-10-vs...](https://blog.timescale.com/time-series-data- postgresql-10-vs-timescaledb-816ee808bac5#88f3)

[3] [https://blog.timescale.com/timescaledb- vs-6a696248104e](https://blog.timescale.com/timescaledb-vs-6a696248104e)

<comment><USER9> To confirm, Timescale does not currently support upserts? INSERT ON CONFLICT ... ?

<comment><USER10> We do support UPSERTs: [http://docs.timescale.com/v0.8/using- timescaledb/writing-dat...](http://docs.timescale.com/v0.8/using- timescaledb/writing-data#upsert)

There is one limitation mentioned there ("TimescaleDB does not yet support using ON CONFLICT ON CONSTRAINT with a named key") that doesn't work, and a tip that mentions requirements for the UNIQUE constraint, but these are not usually dealbreakers.

</comment></comment></comment></comment><comment><USER11> How does it compare to other time series db? I've used influxdb, prometheus, graphite, Elasticsearch and even Splunk.

I mean in terms of complexity, manageability, performance, scalability...

<comment><USER12> Also curious about this, and would add OpenTSDB to that list.

<comment><USER5> At a high-level, we've taken a unique approach to the time-series data problem. While every other time-series db has taken a non-relational approach (which also means no or poor SQL support), we've built ours on Postgres. As a result, not only do we support full SQL (JOINs, complex WHEREs/GROUPBYs, window functions, etc), but we also inherit the reliability and vast ecosystem of Postgres.

At a more detailed level, comparisons between TimescaleDB and others varies depending on the database. Typically though, if you are most concerned with data compression or simple column scans, some of the other options might be more suitable than ours. But otherwise you should see similar (if not better) insert and query performance, while maintaining all the other benefits of a relational DB / Postgres.

And with your relational data living alongside time-series data, many people have been able to simplify their stack (from multiple dbs into one db), which makes things easier to manage operationally and at the application level.

Re: OpenTSDB, we hear that managing an HDFS cluster / Zookeeper / etc can be a headache. You obviously don't have that problem with TimescaleDB.

Re: Prometheus, I should add that we've open-sourced `pg_prometheus` [1], which allows you to write Prometheus data into Postgres / TimescaleDB. So with regards to Prometheus, we are more of a complement than a replacement.

[1] [https://github.com/timescale/pg_prometheus](https://github.com/timescale/pg_prometheus)

</comment></comment></comment><comment><USER13> "We realized that this problem needed a new kind of database, and so we built TimescaleDB: the first open-source time-series database designed for scalability and complex analysis."

I'm not a DB expert, but what have financial firms been using for decades? Likely dealing with much larger securities datasets...

Or, is it just that TimescaleDB is 'opensource'...

<comment><USER14> Yes, they have. kdb+ has been the standard bearer within finance for more real-time trading applications. Still, q as a query language tends to marginalize more of the organization from having access to the data.

TimescaleDB offers full-SQL at scale, opening the database up to more people. It's used to power longer-term trading platforms (for e.g., commodities trading) and interactive analytical dashboards (for e.g., research analysts). We also see applications in compliance and risk management.

And yes, TimescaleDB is open-source. So we see TimescaleDB as complementary to kdb+, which has been around for much longer as a proprietary system, is certainly full-featured and powerful, but is used more in real-time trading applications.

</comment></comment><comment><USER15> Oh how I wish AWS RDS would add TimescaleDB as an extension. Not sure if we could migrate at this point (we are using InfluxDB for time series and metrics and Postgres for related relational data), but if TimescaleDB was available on RDS I'd seriously consider it as it would simplify our approach by a tenfold. Congrats on the funding!

<comment><USER10> Thanks! We're continuing to try and get that accomplished ourselves. It always helps for them to hear from potential users to let them know there is demand for it.

We have an issue with instructions tracking this: [https://github.com/timescale/timescaledb/issues/65](https://github.com/timescale/timescaledb/issues/65)

<comment><USER15> I already sent them a message back in May, I guess it wouldn't hurt to send another one too :)

</comment></comment></comment><comment><USER5> Hi all - Very excited to announce the news, and are here in case there are any questions. Thanks!

<comment><USER16> Curious to know who is currently using timescaleDB? I can't find anything on the website.

Congrats on the funding.

<comment><USER14> Thanks for asking. For some of the larger companies, takes a little bit of time to work with their marketing / legal teams :)

We do list some of them within the blog post:

\- STE Energy, to back operational dashboards used to monitor 47 power plants in Italy, Albania, Colombia, Perú, Honduras, Costa Rica, and Chile, replacing Redis

\- InTraffic, which monitors all of the roughly 5,000 remote controllable railroad switches of the Dutch railways, for use with Tableau as a Business Intelligence solution

\- One of the largest American telecom companies, for storing server health data

\- A top financial services firm, to replace Oracle for powering interactive analytics of market data

\- A public lighting and semiconductor company, for deployment in a small- footprint IoT edge device collecting and analyzing sensor readings

</comment></comment></comment><comment><USER17> Thanks! I didn't know time-series databases existed. I had to rummage through examples to understand what it even means because it wasn't clearly explained in the FAQ. This makes things so much easier.

<comment><USER14> Hi, thanks for the feedback on our FAQ. We did write a blog post that goes into what time-series data is and why purpose-built time-series databases should exist / are necessary.

Please read it here: [https://blog.timescale.com/what-the-heck-is-time-series- data...](https://blog.timescale.com/what-the-heck-is-time-series-data-and-why- do-i-need-a-time-series-database-dcf3b1b18563)

</comment></comment><comment><USER18> Have you guys had any interest from the existing SCADA process historian vendors (Wonderware Historian, OSISoft PI, etc) or historian users about using Timescale?

<comment><USER19> I think the scada vendors have already solved the same set of problems that timescale solves.

Edit: pretty sure the scada vendors partition the data in to tables so each table is a day or week of data, they do rollups on the data so the detail is available if you do a query with a 1 minute timespan but the data is already aggregated in to representative values for a 1 minute interval if the query timespan is 1 day.

Sounds like timescales clustering story might be better, but with OSI or wonderware your clustering story is ms SQL or oracle.

<comment><USER20> Except that most process historians store data in a non-relational way which makes it difficult to integrate with other types of data.

Process historians, like most NoSQL time-series offerings, are traditionally optimized for fast data retrieval (for HMIs, real-time monitoring dashboards, etc.) but are really weak when it comes to analytics. There are new products like Seeq (a kind of a Tableau for process historians) that attempt to address this, but somehow miss the fact that for analytics, you really need your data to exist in a dataframe-like shape.

Today, if you want to do any kind of non-trivial analytics, you typically have to first ETL data from the historian to a relational database via REST or OLE DB API. TimescaleDB on the other hand already stores data in a native relational format and lets you run full SQL on it.

</comment></comment></comment><comment><USER21> One more thing: If this is the kind of thing that excites you, please come join us: [http://www.timescale.com/careers](http://www.timescale.com/careers). We have open positions in engineering (core database, R&D, customer success and Frontend) as well as content marketing, evangelism, recruitment, and partnership.

</comment><comment><USER22> Cool. So where's the value here? How are you making money? Didn't understand that from the first 2 or 3 paragraphs. Seems like you have a lot of GitHub stars, but how about revenue?

<comment><USER5> We're following the classic "open core" model (similar to MongoDB, Elastic, Confluent), and we already have customers paying for features related to enterprise-grade reliability, performance, and support. We'll have more to share in the upcoming months.

<comment><USER22> Nice thanks for the response!

</comment></comment></comment><comment><USER23> What is the difference between TimescaleDB and a conventional column-store data warehouse with a time partition key?

<comment><USER10> We touched on this a bit elsewhere when someone asked about InfluxDB. One of the biggest advantages is that by being built on Postgres, we are able to support the full range of SQL/RDBMS features including secondary indices, complex WHERE predicates, CTEs, and more. Time-series data can also be easily stored alongside relational data, which makes it easier to simplify your stack.

<comment><USER23> Column store data warehouses have all those features, except indices which don't make sense in the context of a columnar architecture. It seems to me that if you create a table in a conventional data warehouse with a partition key on a time column, you have all the advantages of TimescaleDB, plus all the advantages of an MPP columnar data warehouse. Am I missing something? Why should anyone use TimescaleDB instead of just a specific configuration of a data warehouse?

<comment><USER21> A lot of time-series data has complex relations and cross-correlations that make it unsuitable for column stores. Take for example a device that reports temperature and humidity. You may want to enforce foreign key relations between devices and readings, or you may want to do analysis for device in a particular location (which may be part of readings or devices table) secondary indexes help here. Or you may want to easy access to all data where temperature is above 90 degrees (partial indexes help here). You often want this in a real-time dashboard where the kind of latencies you see with MPP are problematic. Pretty much the columnar/MPP stuff is good for simple aggregates for ad-hoc non-real-time queries but TimescaleDB is geared to more complex analysis to power dashboards.

Plus, when you know which particular aggregates you want to regularly query, TimescaleDB makes it easily to perform continuous roll-ups so you can store in a separate (hyper)table.

<comment><USER24> See Clickhouse for `oltp` columnar store (built for powering dashboards).

<comment><USER21> Clickhouse is a very cool project but is more analytical than OLTP. Percona had a nice writeup[1] (the comparison to mysql is especially apt -- note the lack of real-time updates and deletes in Clickhouse).

[1] [https://www.percona.com/blog/2017/02/13/clickhouse-new- opens...](https://www.percona.com/blog/2017/02/13/clickhouse-new-opensource- columnar-database/)

</comment></comment></comment></comment></comment></comment><comment><USER11> How does it compares to other time series database? I mean, things like InfluxDB, Prometheus, Graphite, etc.</comment>


New longevity drug mimics caloric restriction in mice - <AUTHOR> http://blog.wired.com/wiredscience/2008/07/caloric-restric.html <comment><USER2> This is about resveratrol, which isn't a new drug. It's been in and out of the news for quite a few years now. The article is a little confusing, but it seems that the current research has linked it to a mechanism which is also involved in the "starving mice live longer" phenomenon. That's definitely good news, scientifically.

But the "wonder drug" theme is a little overstated. There have been _lots_ of teasingly good results with this drug in mice over the last decade or so, but we're still waiting for a pharmaceutical to result.

<comment><USER3> This article is confused about resveratrol. The important thing is that there is a super-resveratrol undergoing clinical trials, which they mention at the end. It's something like 10,000 times stronger than resveratrol.

Resveratrol itself is not economical to take in such high, effective doses.

<comment><USER4> This is not true - I have been taking 500mg for a year. 500mg is about equivalent to the dosage that they have given to mice in similar studies (per kg).

</comment></comment></comment><comment><USER5> Aubrey de Grey has argued that caloric restriction can probably only extend life in humans by ~2 years. The idea is that it's not a percentage of lifespan that you can gain, but rather a fixed amount of time. He says the data supports his interpretation, and that it makes sense evolutionarily because extending lifespan in response to low calories is mostly useful if you can get through a short famine and get a chance to mate that would have otherwise been lost. If it's a long famine that's a much harder problem for evolution to solve and there's no evidence it is solved.

Therefore caloric restriction probably isn't very important. Does anyone know otherwise?

<comment><USER6> Be wary of evolutionary arguments. It is possible to rationalize any current behavior/biological trait based on a made up evolutionary theory. It is also easy to forget that the human body is a very imperfect system (and therefore should be open to hacking).

If he has data, that is another thing, but so far it seems the research really isn't good enough (on humans) to do anything more than conjecture.

<comment><USER5> Evolution does not create features by accident. I'm not proposing a trait. I'm proposing a _lack of a trait_ (built-in long term life extension feature in humans), which is very different than proposing a trait, and wins by the default unless someone has an argument to the contrary.

<comment><USER6> Traditional evolutionary theory says that features are created at random, but they are not kept by accident.

I thought you said this was Aubrey's theory, not yours. Perhaps you should quote him or rephrase. Can you also site how lack of default traits also 'win by default' in evolutionary theory?

<comment><USER5> It was Aubrey's theory first. It's also my theory, currently.

You are mistaken about evolution. There are random mutations, but those are not complex _features_ , they are tiny changes. It takes many mutations to build up to a major feature.

Example of "lack of trait" winning by default: might humans have the ability to breath the air on Mars? no. why? there's no reason that feature would have been created. you don't have to be unsure whether they might or might not; unless there's an argument on the other side, the "no breathing on mars trait" wins by default.</comment></comment></comment></comment></comment>


How Ethernet Went From 3 Mbps to 100 Gbps and Beyond - <AUTHOR> http://www.wired.com/epicenter/2011/07/speed-matters/

<comment><USER2> Duplicate story, many comments already.

<http://news.ycombinator.com/item?id=2767428></comment>


Ask HN: Best places to post resume for recruiters.  - <AUTHOR>

If you were to take a lazy approach to job hunting by letting the jobs hunt you, where would you post your resume? Other than Linkedin that is. <comment><USER2> I heard back from lots of people on angel list (<https://angel.co/>). Make an account and go through who is hiring. Usually you have to message them, but they can also browse your profile.</comment>


Chrome Web Store – Share Enhancer for SoundCloud - <AUTHOR> https://chrome.google.com/webstore/detail/share-enhancer-for-soundc/gelmdamogiijgobmoofjoekkcilnmnkb <comment><AUTHOR> Iv'e always wondered why Twitter and Facebook weren't more connected so I built a chrome extension that connects them through a short link at [http://2fb.me](http://2fb.me)

This is an enhancement for Sound Cloud which offers up a way to share sound cloud's more easily between them.</comment>


Higher wages and employee turnover - <AUTHOR> https://operationsroom.wordpress.com/2015/03/19/higher-wages-and-turnover/ <comment><USER2> Summary, "Turnover costs money, raising wages reduces turnover, solve for the local minimum with respect to costs."

It is useful to understand that this is a 'natural' floor on wages.

<comment><USER3> My brother started and runs 3 small pizza restaurants. He starts employees at double minimum wage and gives his employees ok health insurance. It's pure capitalism: you get an entirely different class of workers when you pay more. They have cars that work and tend to show up when scheduled able to work, ie not so hungover, drunk, or stoned they can't do shit. I'd like to think he'd do it because it's the right thing to do, but it's nice when morality and financial incentives align.

<comment><USER4> How much does he charge for a pie and what is his margins?

<comment><USER5> "How much does he charge for a pie"

It's a pizza business, not a pie business. Pizzas aren't pies.

<comment><USER6> "Pizza" is just Italian for "pie".

[http://www.etymonline.com/index.php?term=pizza](http://www.etymonline.com/index.php?term=pizza) [http://dictionary.reference.com/help/faq/language/e02.html](http://dictionary.reference.com/help/faq/language/e02.html)

</comment></comment></comment></comment></comment><comment><USER7> Costco has known about this for years. They've consistently resisted shareholder demands to reduce payroll spend while paying nearly double industry average.

The result? A 5% turnover rate after a year, not to mention happier and better employees. Pay attention to their name tags next time you visit, they list their starting year.

This is the first article I could find on Google, there's a lot more out there.

[http://www.businessinsider.com/costco-pays-retail- employees-...](http://www.businessinsider.com/costco-pays-retail- employees-20-an-hour-2014-10)

<comment><USER8> For point of comparison, from the article:

 _Turnover in the retail sector has been steadily rising and now stands 5 percent a month._

<comment><USER9> I think the costco article is saying 5%/yr, and the original article is saying saying that walmart's 60% turnover is consistent with retail average of 5%/mo

<comment><USER10> Here's an interesting article on the right way to measure turnover: [http://www.ere.net/2003/10/06/the-right-way-to-measure- turno...](http://www.ere.net/2003/10/06/the-right-way-to-measure-turnover/)

Tl;dr: provide context.

</comment></comment></comment></comment><comment><USER11> The Walmart in my town seems to be irregularly staffed. On some days the store looks like a dump, nothing is clean, nothing is fronted and faced, I don't like to be there because it feels like the kind of place where you'd be the victim of a crime. On other days the store is in pretty good condition.

The Target, on the other hand, seems crowded with workers around 9 in the morning. It's a much more consistently pleasant to go there.

<comment><USER12> Here in Canada, Walmarts are okay. It's the American experience anyway, so we take it for granted.

Target, on the other hand, seems very empty these day. My local Target look like an abandoned building and I can't seem to find any products.

<comment><USER13> Target is closing all of its stores in Canada.

[http://www.cbsnews.com/news/target-pulling-out-of- canada/](http://www.cbsnews.com/news/target-pulling-out-of-canada/)

<comment><USER12> That was the joke, sorry.

</comment></comment></comment></comment><comment><USER14> I wonder how much this relationship also holds in the tech industry. I'd love to see more companies emphasizing things like work life balance instead of free food or gym memberships.

I don't even really feel as though working more hours necessarily makes me more productive.

<comment><USER15> I usually don't even work 8 hours, I just stay in the office.

</comment></comment><comment><USER16> I worked for an electronics/computer retailer for 18 months, making about $8 per hour, in California, in 2001. Miserable job: two days opening at 7 am; two days closing at 11 pm; one day starting at 9am. I was working Sat-Mon-Tues- Thurs-Fri. I was off on Sunday and Wed. Look how miserable was it: low pay, odd timings, odd days off.

Then I found another job that was paying $13 per hour in 2002. It was a security guard job. I could read books at work, and walk around. Sure, it is a miserable job compared to a programmar gig; but better than my retail job.

If you are a student, working part time in retail or in any service job, your experience is not miserable: you are getting pocket money, after all. But service sector employs more full time adults. If you are an adult working in these places, you see people without health insurance, working two jobs, lack of sleep, unhealthy people.

Psychological impact is tremendous. And journalists, researchers don't compute the cost of psychological impact, as individuals bear the cost.

When I was working as a security guard, one day a colleague did not show up for work. A week later, we found that he died of cardiac arrest. This guy was a weightlifting champion in his younger days. He was paying child support, working two jobs: working 80 hours a week, in odd time schedules.

This working model is exported to other countries, to other industries from the service sector. Welcome to the new World!!

</comment><comment><USER17> My best argument so far for the (mostly symbolic) raising of the minimum wage had been that going from ~$7 to ~$10-15 is approximately _doubling_ the purchasing power of individuals who need it most (and will use it), at a low cost to everyone involved. It's an amazingly good return on a relatively tiny social investment. The best argument _against_ it that I've seen is that black-teen unemployment goes up dramatically every time the minimum wage increases. However, it turns out that those scary data points (promoted usually by libertarian thinktanks) exactly correlate with general economic recession periods.

Never once did I imagine that employee turn-over would have such a huge economic impact, but it makes total sense. I've definitely known that happy employees prefer to keep working, but I never connected the dots to dollars spent on boarding new employees. And now I have a talking point that the hardest "show me the money" opponent can wrap her head around without accusing me of appealing to emotion or morality.

I welcome more talking points (pro and con) since I'm not formally educated in economics.

<comment><USER18> Some thoughts:

1) McDonalds and other large minimum wage employers know what a minimum- skilled employee is worth to their bottom line, and they set prices accordingly. They know that after a wage bump they'll have reduced turnover of their better employees, but they also know that when they do have turnover, that new unskilled employee is worth 1.6 big macs per hour, so they'll raise prices as quickly as they can to compensate. The net result is inflation, and _inflation destroys any purchasing-power increase_ the minimum wage creates. (My own research suggests this takes about 2 years to fully play out.)

2) Increased unemployment right after minimum wage increases might be one of the sources of "general economic recession", rather than (as you imply) one of the consequences. Based on my own research, the unemployment bump actually tends to start about 3 months prior to a minimum-wage bump (as companies slow new hiring), peaks 3-6 months after, and then decreases inverse to price increases/inflation.

3) when people are unemployed, taxpayers pay 100% of their expenses. When they work for the minimum wage, they earn N% and taxpayers pay 100-NxS% (where S is the scaling factor for how quickly benefits are lost in relation to outside income; S should be less than 1.) But in light of points 1 and 2, someone can only get a job if N < their value to some company. So if the minimum wage N is set high enough, it forces people out of the economy _long term_ (because if they're unemployed, they're not gaining work experience, and therefore they're not increasing their value.)

I'm of the opinion that setting the minimum too low is self-correcting (because of turnover and the existence of systems like welfare), while setting the minimum too high is the cause of significant unemployment. So I'd rather err on the low side, pairing a low minimum wage with strong welfare and related benefits.

</comment></comment><comment><USER19> Maybe turnover partially explains Walmart's willingness to raise wages. But above all, I think the modest wage hike is a response to worker's demands.

There's a growing nationwide movement for $15/hour and a union in the service economy. Walmart is making a meager concession today to try to stave off bigger demands in the future.

<comment><USER20> the money behind that push is the unions because government contracts are tied to minimum wage, hence if they can push the idea that 15 is fair; I think its excessive in the extreme for unskilled labor; then that in turn boosts what government contract pay out.

32k for unskilled???

</comment></comment><comment><USER21> I can't but help think the high turnover is caused by more than just the low wage. If the workplace is miserable, doesn't challenge you and doesn't offer much opportunity for growth, the turnover is going to be high regardless if you increase rates by a couple of bucks.

We have turned into a service force where it is easy enough to jump from one service job to the next. There isn't a lot of loyalty from employees in these jobs because they are not rewarding.

What can we do to change that? Honest question. I am not concerned about finding ways to lower the turnover and keep people in dead end jobs. I would much rather see more jobs being offered that give people opportunities.

Just my 2 cents.

</comment><comment><USER22> This sounds like great news for low-wage workers, but the article seems to punt on the Why. Since so many people have been _under_ -employed the last 8 years, I wonder if the increased turnover is due to over-qualified employees who were just working a Walmart gig until they could find something better, e.g. college graduates or folks laid off from a higher-paying job. I wouldn't be surprised if this were a large part of the effect, and in that case I'd figure that:

\- raising wages would not help retain these people.

\- the increased turnover will subside as the underemployment problem fixes itself.

I'm curious what other folks think about that theory?

<comment><USER23> I would naively assume that the market forces of high unemployment would be more powerful than over-qualified employees would account for unless they sought them specifically.

Given all the mechanisms they already have in place to control the impact of turn-over it doesn't surprise me that they realized it would be cheaper just to pay people a little more.

</comment></comment><comment><USER24> The bosses of Indian IT services companies need to read this.

They have ridiculously high attrition rates of upto 30 - 40 %.

75 % of the people hired leave within 3-4 years.

So basically they're the tech equivalent of Walmart.

</comment><comment><USER25> Would raising the minimum wage reduce turnover too?

<comment><USER26> If you raise the minimum wage, people that are earning more will also have a increase in salary in medium term. This solution only move the bell graph of salary to the right.

<comment><USER27> It can move things to the right, but more so for the rich than the poor. That is, unless you are assuming minimum wage services are primarily consumed by minimum wage workers.</comment></comment></comment>


Nearley – Parser toolkit for JavaScript - <AUTHOR> http://nearley.js.org/ <comment><USER2> I love the Earley algorithm! It's relatively small, online and works with any context-free grammar. It seems to be the basis of many other algorithms I've read about in the literature, suggesting it is easy to extend. The grammar is pluggable, allowing the programmer to easily create an API around it. I wonder why it's not widely used... Whenever I see a general algorithm being employed in parser tools it's GLR. Perhaps a good Javascript implementation of Earley could change that.

Nearley implements Joop Leo's improved handling of right-recursion¹ but perhaps it can be optimized further by using the method of precomputation² described in Aycock & Horspool's 2002 paper.

¹ [https://dx.doi.org/10.1016/0304-3975(91)90180-A](https://dx.doi.org/10.1016/0304-3975\(91\)90180-A)

² [https://dx.doi.org/10.1093/comjnl/45.6.620](https://dx.doi.org/10.1093/comjnl/45.6.620)

<comment><USER3> While a GLR parser has the same worst-time complexity O(n^3), it usually performs better on nearly deterministic or deterministic grammars (though Earley can be modified to work better on deterministic grammars). Apart from that the algorithms give quite similar results, and can both handle any context-free grammar. GLR might have the advantage that it doesn't require any "tricks" to handle right-recursion, and that the algorithm itself is very compact.

I haven't implemented an Earley parser yet (should probably give that a try), but I suspect it's not more difficult than implementing a GLR parser (for a great GLR reference implementation, check out the Elkhound paper: [http://scottmcpeak.com/elkhound/sources/elkhound/algorithm.h...](http://scottmcpeak.com/elkhound/sources/elkhound/algorithm.html)).

In any case time complexity is not always the best performance measure in the real world, as it's usually the constant overhead that makes up for the largest performance differences, at least when parsing programming languages (which often are fully or almost fully mostly deterministic and can be handled in linear time even by recursive-descent parsers given the right grammar). Here, simple shift-reduce parsers really shine, as they do not do any backtracking and work with a simple rule table and a heap/stack for the tokens they emit. Also, the (optional but often useful) lexing phase of parsing should not be underestimated, as it can be as tricky as the token-based parsing that follows. Python, for example, cannot be lexed with a context-free grammar as the indentation is stateful (and newlines are treated differently depending whether they occur inside a bracket/parens expression or not, which requires a grammar to keep track of the nesting)

The main advantage of "traditional" tools like yacc or bison is that they are highly optimized, and produce parsers that can process > 100 kloc / second, which is hard to achieve with most other frameworks (I couldn't find any benchmarks on the JS parser).

<comment><USER3> Correction: If you want the parse tree, GLR and Earley can have O(n^4) time complexity it seems:

[http://cstheory.stackexchange.com/questions/22621/complexity...](http://cstheory.stackexchange.com/questions/22621/complexity- of-a-glr-parser-on-an-ambiguous-grammar)

</comment></comment></comment><comment><USER4> Looks like a great toolkit. I'm going to convert some of my PEG.js grammars over to test it out. It would be good to have an online page like [https://pegjs.org/online](https://pegjs.org/online). It allows people to start playing without all the setup.

</comment><comment><USER5> feature set looks very similar to [https://github.com/SAP/chevrotain](https://github.com/SAP/chevrotain)

would be good to have a comparison

EDIT: actually there is one [1], and it includes Nearley which does quite poorly.

[1] [http://sap.github.io/chevrotain/performance/](http://sap.github.io/chevrotain/performance/)

<comment><USER2> The difference in performance is explained by the capabilities of the algorithms used by each parser library.

The Earley algorithm can parse any context-free grammar while Chevrotain appears to be restricted to the LL(k) class of grammars. Earley is O(n³) in the worst case but it performs better with more restricted classes of context- free grammars. Even if given the same grammar, Earley will still incur the cost of generality, a cost which LL(k) parsers don't have to pay.

<comment><USER5> Okay, do you have a non-contrived, practical example that can be handled by Nearley but not LL(k)?

It also doesn't help that linear LL(k) performance/optimizations is the first feature listed.

</comment></comment></comment><comment><USER6> How good is for error reporting? That is my #1 thing that drive me away for parser generators...

</comment><comment><USER7> Title should probably be "Nearley - parser toolkit for JavaScript".</comment>


Q: Why can I access an out-of-scope C++ var? A: So you rent a hotel room... - <AUTHOR> http://stackoverflow.com/questions/6441218/c-local-variable-can-be-accessed-outside-its-scope/6445794#6445794 <comment><USER2> I say this as a C++ programmer (C++ is essentially the only choice in the domain in which I currently work), but this really underscores how unreasonable C++ is for writing secure/high reliability applications. On the one hand, the compiler will curse you out for trying to use a non-const iterator in a const function (sheesh, what kind of an idiot are you, anyway)? On the other hand, you can read and write to every bit of memory allocated to your application, and it will stand by and do nothing. I think the number of applications which had buffer overflows at least at some point is statistically indistinguishable from 100%.

Try this hotel analogy: you go to a hotel in which you once stayed, and tell them that you're going to dump every bit of possessions and furnishings in all the rooms outside on the street, rifle through them, set them on fire, then photograph all their guests in the nude and distribute the pictures. Most hotels will object. C++ won't.

<comment><USER2> It seems from some of the comments down the thread that the idea of writing over memory in C++ is new to many people. Let me introduce another couple of concepts.

You don't have to stop with writing over the variables. You can write over the return address of a function, which is stored on the same stack as the variables. If you write some user-supplied buffer to memory, and you didn't carefully make sure that the memory is allocated for this purpose, the (malicious) user can supply you with data that will set the function's return address to something in his buffer, and the function will go on executing his code. That's called a buffer overflow vulnerability, and it's been well known and exploited for decades; half the patches you see coming down the pipeline used to be for this.

Now, there are good ways to protect against buffer overflow vulnerabilities. If you make the stack and data non-executable, and the pages containing code non-modifiable, then seemingly there is no place for the attacker to place his code and get it to run. Modern processors support this in hardware.

Except that some people came up with something called return-oriented programming. This works by finding little pieces in your code that do something simple like increment a register or write a byte of data, followed by a "return" instruction. These tiny pieces of code are called "gadgets". You can usually find enough gadgets to make a Turing-complete language. Then you write a compiler which transforms your code into a stream of calls to the gadgets. Now all you need is one overwrite of the return address on the stack, and you can start playing the application like a piano, without ever executing data or modifying code. There are now tools to automate all this.

I don't want to reveal anything about where I work, but we write applications which need to be reliable and secure. We have regular meetings to discuss vulnerabilities and security techniques. At the meeting where we discussed return-oriented programming, when we asked what we could do to protect from it, the consensus answer was "You can't. Don't make any buffer overruns." The problem is it's not clear that anyone ever wrote a program free from those.

Since hotel analogies seem to be so popular, here you go: <http://en.wikipedia.org/wiki/Psycho_(film)>

<comment><USER2> One last addition - I hope I'm not overstaying my welcome with all these little lectures on security. It used to be that a buffer overrun typically resulted from carelessness, like

       char name[100]; //Who has a name longer than 100 characters?
       printf("What is your name?");
       scanf(name, "%s");

It turns out that the same kind of people who have names like "Robert'); DROP TABLE Students; --" sometimes have 150-character names with a return address and a few bytes of malicious code. That's something that you handle kind of like you sanitize your inputs. You don't pour user input into a limited space. Scanf is deprecated now, and this kind of a vuln is becoming rare.

But as I explain a couple of comments down, a buffer overflow doesn't have to be this simple. You can get a bad pointer in a structure anywhere in your program, and when you use the structure, you will overwrite the stack. It's very hard to catch that kind of vuln - basically, it's a race with you (and white hats) vs. black hats. This is why I welcome the switch from C/C++ to managed code which is happening now, but very, very slowly.

<comment><USER3> I don't think you are overstaying your welcome. Actually, I learned alot from your posts. Thanks.

<comment><USER4> Off topic/meta: In the days when comment scores used to be shown, I could do the HN-equivalent of saying "hear, hear!" [or as the Internet says it: "here, here" :)] by upvoting a "thank you" note like the above. Now, if I _really_ want to let you know that I too liked your comment, I have to post a "me too!" comment like this one. Anyway, thanks. TIL about "return oriented programming".

</comment></comment></comment></comment></comment><comment><USER5> Eric Lippert is a prolific writer and is amazing at explaining anything related to computer programming. His blog is a MUST read for every .NET developer: <http://blogs.msdn.com/b/ericlippert/>

I also follow Eric's activity on SO here:

[http://stackoverflow.com/users/88656/eric- lippert?tab=activi...](http://stackoverflow.com/users/88656/eric- lippert?tab=activity)

Some of his answers which I liked:

[http://stackoverflow.com/questions/2704652/monad-in-plain- en...](http://stackoverflow.com/questions/2704652/monad-in-plain-english-for- the-oop-programmer-with-no-fp-background/2704795#2704795)

[http://stackoverflow.com/questions/921180/how-can-i- ensure-t...](http://stackoverflow.com/questions/921180/how-can-i-ensure-that- a-division-of-integers-is-always-rounded-up/926806#926806)

[http://stackoverflow.com/questions/5032081/coding-style- assi...](http://stackoverflow.com/questions/5032081/coding-style-assignments- inside-expressions/5032287#5032287)

He is on HN as well:

<http://news.ycombinator.com/threads?id=ericlippert>

<comment><USER6> Heads up: the two links at the top go to the same source:

 _Eric Lippert is a prolific writer and is amazing at explaining anything related to computer programming. His blog is a MUST read for every .NET developer:<http://blogs.msdn.com/b/ericlippert/>

I also follow Eric's activity on SO here:

<http://blogs.msdn.com/b/ericlippert/*>

<comment><USER5> I don't know who downvoted you, you are correct. I fixed up the links.

</comment></comment></comment><comment><USER7> I'll probably get downvoted, but I think elaborate real-life analogies like this are more likely to confuse novices than help them.

A better explanation in my opinion is to draw a diagram of a stack and showing that returning from a function just decreases a pointer. C++ programs don't scrub the top of the stack once you finish a function because it is a waste of time.

If another function was called before the 2nd call to foo(), then the variable on the stack would be overwritten.

<comment><USER8> The analogy is supposed to explain the concept of _undefined behavior_ , not the details of memory management.

If you are used to safe languages, the concept of undefined behavior might be confusing. In those languages, an operation is either allowed or not allowed, and if it is allowed it has well-defined behavior. In C a number of operations does not have well-defined results, but is still technically possible to perform, or might be possible depending on circumstances. But you shouldn't use them. The analogy is supposed to explain that.

<comment><USER9> undefined behavior is fun. All this stuff gets a lot more interesting in microcontrollers, where security is not an issue and you sometimes have to manage memory directly and store data between power cycles.

</comment></comment></comment><comment><USER10> I had a roommate in college who would write code like this. I (and several other people) tried to explain to him that he couldn't necessarily count on the pattern working all the time. His response was to say, "But it works here!" and continue abusing C++'s undefined behavior.

I stopped giving him help with his CS coursework in fairly short order.

</comment><comment><USER11> Eric Lippert was one of the highlights of working at Microsoft. He was (and I'm sure, still is) very active on the internal C# mailing list. His answers are always very entertaining to read. He has a smugness that is deserved and not irritating (mostly :)), and inside the walls of MS he turns that smugness up quite a bit higher.

<comment><USER12> I'm glad you enjoyed my answers; however, I am never intending to come across as smug. It is difficult to get across subtleties in an impoverished text-only medium; it has certainly been my experience that _instructive and constructive criticism_ can easily be read as smugness. It's also been my experience that trying to deliberately make it sound less smug often just turns it into sounding like condescention. Text-based communication of technical issues is a hard problem and I do struggle with finding the right tone. Thanks for the feedback.

<comment><USER11> I don't think "smug" is quite the correct term. I meant nothing negative with my comment (and was teasing with the "mostly"). You do have an "air of authority" about your answers, but that authority is clearly earned. I enjoyed your answers on the C# mailing list quite a bit.

</comment></comment></comment><comment><USER13> Great analogy and well written. We often see attempts at this style of explanation, but we usually fail at one or both of those qualities.

While I wish I saw more of this style of teaching in technical publications, it's admittedly rarely done well enough to convey the point.

</comment><comment><USER14> I'm not sure I agree _entirely_ with this explanation. He emphasises that this is unsafe behaviour that is protected against in safer languages - I've never heard it said that the use of pointers should be considered unsafe and there are also pointers in other languages that exclude things like multiple- inheritance and have garbage collection (which could be considered safer languages).

I like the room key analogy (almost) but I don't think it needs to be stolen and I don't think all the talk about policies and contracts is necessary at all.

It could simply say the thing you're returning is not the local variable. It is a pointer.

I have a box and instead of giving you the box I give you a key to the box. Anytime you want to use the contents of the box you must come over to the box and open it, but you can't take the box with you. If you want to put something in the box you just walk over and put it in - but here's the rub: you have no guarantees that you're the only one with a key. I could give out the key to a bunch of people so you don't know what will be in the box and who owns it.

Use with caution!

<comment><USER15> I think he means that pointers are unsafe in the same way that dynamite is unsafe. They both have legitimate uses, but you need to be very careful when you are using them. (Rather than unsafe in the sense of don't use them at all).

I like your box analogy, but I don't think it goes far enough - with C++ pointers it is not always ok to put something in the box. If the pointer is out of date or invalid there is a chance that writing to it will make the OS kill the entire process.

Languages with garbage collection don't tend to have that kind of problem, since whatever is at the memory location will stay at that location until everyone surrenders their pointers.

<comment><USER16> I agree with you for the most part. C/C++ lets you do what you want, presumably because you know what you're doing. But I don't agree that pointers are evil, and should be avoided. If you want to do any practical programming with C/C++ at all, you have to learn how to dynamically allocate memory and use pointers. Generally, pointers work as advertised. It's the cases where they work when they shouldn't that is the problem. In these cases a compiler usually warns you, so you are still covered. So in C/C++, just because it works, doesn't mean your code is right.

<comment><USER17> _But I don't agree that pointers are evil, and should be avoided._

No one is saying that at all. Pointers aren't evil, but they are dangerous.

Perhaps a better analogy is a sharp chef's knife. In the right hands it's an effective and efficient tool that lets you do things quicker and just as safely as any other tool.

In the wrong hands it is dangerous to the person using it and to those around them.

There are numerous other examples: welding torches, motorbikes, explosives etc etc.

<comment><USER15> Exactly. I didn't mean to imply that pointers are evil or should be avoided. That was supposed to be the point of my dynamite analogy, but I guess the comment made elsewhere on this topic about the inadequacies of analogies holds true here.

So, for the avoidance of doubt, I believe: pointers are awesome, powerful tools and you can do some great things in C/C++ using them and I sometimes miss them (a little bit) when using other languages. But you can also do some terrible things with them - and I have done some spectacularly bad things with them in the past. But that doesn't mean they are bad - it just means that I am reckless.

<comment><USER16> ^ Ditto. I thought pointers were being shown in a negative light at first, but now I see the point. In my opinion too, manual memory management is a very important part of developing highly scalable applications, but should only be done if absolutely needed - which is fortunately rarely the case nowadays as most people just develop for the Web, and can afford to throw money at the problem. But for embedded systems where resources are limited and inputs are very limited, it is still very useful. In the early years, it was C + inline assembly for further optimization - where you try your best to avoid assembly. I guess now, it's a dynamic/interpreted language, plus C/C++ for further optimization, and avoid C/C++ like the plague as much as possible.

</comment></comment></comment></comment></comment></comment><comment><USER18> Local variables in C and C++ are just put on a stack (i.e. _the stack_ ) and its behavior is pretty predictable. For example:

      #include <stdio.h>

      static int *foo(void)
      {
          int i = 42;
          return &i;
      }

      static int *bar(void)
      {
          int i = 43;
          return &i;
      }

      int main(void)
      {
          int *f = foo();
          /* with next line commented out prints 42, with it, 43 */
          bar();
          int i = *f;
          printf("%i\n", i);
          return 0;
      }

Until you've made another function call that reuses that space on the stack you'll almost certainly still have valid values when accessing those local variables by address. Things are a bit vaguer in C++ where those variables may be objects which have destructors which have already been called.

Edit: Made it predictable, per caf's (correct) comment by adding:

      int i = *f;

<comment><USER19> In embedded systems where interrupts are handled from user space (or the code you show is in the kernel), then it is guaranteed that your stack will be obliterated if an interrupt occurs at the exact right time, so your value will not be there.

This is not academic: I have debugged code that relied on that behavior, and would fail once in a million runs. Very frustrating to figure out.

<comment><USER20> Same in userland if you're using POSIX signals (and you haven't explicitly configured a separate sigstack)

All it takes is the user resizing their xterm and JUST the right moment and the SIGWINCH handler will happily step on your out-of-scope stack objects.

</comment></comment></comment><comment><USER21> A good analogy no doubt! But you should be careful when using analogies. Every behavior of the analogy might not be applicable to the original problem.

Analogies can only help make people "understand" something. Normal brains try to relate things to what it has learnt so far. So analogies seem to help. Analogies in themselves are not scientific explanations obviously.

A scientific/rational explanation to the question posed by the user would be - C++ makes no promise about the behavior of the program when out-of scope C++ variable is used. Obviously this explanation -a simple one is quite less dramatic.

</comment><comment><USER22> Title is a bit misleading... but still a great answer.

</comment><comment><USER23> 200 upvotes in nine hours?! Is that a Stack Overflow record?

<comment><USER24> no, I guess many people started to understand this behavior just now. (something you don't meet when all you do is javascript and ruby/python)

<comment><USER25> Maybe I'm an elitist but I find it rather scary that people write code for a living without understanding concepts such as this. It's easy to forget what a huge service StackOverflow provides to the developer community by spreading knowledge in an easily accessible and entertaining fashion.

<comment><USER26> If those people who write code for a living never program in C++ or other languages with pointers, instead using languages like Java and Python, I wouldn't expect them to know what happens when you mess around with pointers in C++. If you don't use pointers, and your language doesn't even support pointers, why care about the behavior of pointers? Understanding the simpler concept of "references" is good enough.

</comment></comment></comment></comment><comment><USER27> if you can always legally access the book, which will for sure be available to you in that drawer, then the hotel key you stole must be a closure.</comment>


WebKit for Developers - <AUTHOR> http://paulirish.com/2013/webkit-for-developers/

<comment><USER2> Very interesting article. I never really understood the nuances of the different WebKit ports (and once naively assumed that WebKit === WebKit). I also didn't have an appreciation for the amount of work going in behind the scenes. The link referenced in the article (<http://trac.webkit.org/browser/trunk/LayoutTests>) is a fascinating line of sight into the testing that goes on for the core.

</comment><comment><USER3> This is why the webkit- prefix is nearly worthless. I've had a lot of problems where chrome and safari were rendering prefixed things differently and I couldn't target them individually, because they both use the same prefix. This is the whole point of having the prefixes!

<comment> eric-hu "CSS parsing is the same, though. Slurping up your CSS and turning it into CSSOM's pretty standard. Yeah, though Chrome accepts just the -webkit- prefix whereas Apple and other ports accept legacy prefixes like -khtml- and -apple-."

Using this information, can you target Chrome with the webkit- prefix and Safari with the apple- prefix? Specifying the apple- prefix after webkit- will ensure that Safari uses that one, right?

</comment></comment><comment><USER4> > So first, WebKit parses HTML the same way. _Well, except Chromium is the > only port so far to enable threaded HTML parsing support._

Nope. Recent WebKit Nightlies have threaded HTML parsing as well.

[https://www.webkit.org/blog/2259/last-week-in-webkit- threade...](https://www.webkit.org/blog/2259/last-week-in-webkit-threaded- html-parser-and-background-blending/)

<comment><USER5> I'm one of the developers of threaded HTML parsing in WebKit. Threaded HTML parsing is only enabled in Chromium (for now). We're still working through the kinks. :)

<comment><USER4> Oh? So the "Last week in WebKit" isn't actually in the nightlies yet? Interesting.

<comment><AUTHOR> It'd be better to think of it as "Last week in the WebKit project". As as the post points out, the WebKit nightly is only representative of the mac port of WebKit, which Apple mostly calls the shots on. In many cases Apple takes a conservative approach when enabling features that other ports may be implementing or experimenting with.

</comment></comment></comment></comment><comment><USER6> I don't really get the initial point. The browser is, for most developers, a black box. And it should probably stay that way. Ideally, it is an exchangable black box.

Still a neat article. I just find that point odd.

<comment><USER7> For context, check out the video of the presentation where I made this point: <http://www.youtube.com/watch?v=kiPe7DPmEgE>.

TL;DR: to build efficient applications in the browser, you _do_ need to understand how the browser works. Just like understanding how an operating system works will help you design good applications for the OS (especially when it comes to performance).

<comment><USER6> I think my concern is that more of us should just follow the standards and build a minimal document for the users. Obviously, those of us that need efficiency should know what we are doing. But, just as the vast majority of programmers today have high zero understanding of cache impact on algorithm performance, the vast majority have little understanding how browsers do their thing.

Is it worth knowing for other purposes? Well, I think so. It is worth knowing just to know, if no other reason. And it is also worth knowing in the cases where you are building something highly specialized. However, when I see things like the template tag making it in to webkit, we are no longer treating it as a blackbox where we focus on inputs and outputs. Instead, a growing number are getting to muck around in the internals.

Is this bad? I honestly can not say. However, I do know that from my perspective web apps have not been getting more impressive. Nor do I feel I am missing something from them. Seems if folks want items that are as efficient as you are alluding to, you should be doing a native application.

</comment></comment></comment><comment><USER8> I'm interested to see whether Opera's move to WebKit means the end of the Dragonfly suite of tools (which are fantastic, by the way).

<comment><USER9>
       *sad panda* @hzr The switch to WebKit unfortunately means
       that Opera Dragonfly is no more.

<https://twitter.com/runeh/status/301616059729969152>

And yes, it is/was an amazing tool. My favourite web inspector by a mile.

</comment></comment><comment><USER10> I'm a little surprised by this line, in reference to Chrome on iOS:

        "Also, for what it's worth, JavaScript is so rarely the
        bottleneck on mobile that the lack of JITting compiler
        has minimal impact."

But then I'm making HTML5 games so JavaScript performance is pretty much the only thing I care about.

</comment><comment><USER11> Levi Weintraub has a really great tech talk hosted at airbnb, where he talks abouts all the different aspects of WebKit and it's components: [http://www.youtube.com/watch?v=GGzmST5nNSM&playnext=1...](http://www.youtube.com/watch?v=GGzmST5nNSM&playnext=1&list=PL_gU0T3qTDt-1nyGuNoqKW3N5BYDNlZ3y&feature=results_main)

Some more really good tech talks @ <https://www.airbnb.com/techtalks>

</comment><comment><USER12> Does anyone know why there are two different font paths and how WebCore chooses which one to use? I assume the difference between fast and complex would be very platform dependent.

<comment><USER13> The fast path is for most languages where the text glyphs are placed in order. The complex path is for languages where the shape and size of a glyph depends on the characters around it. For example, Arabic text or Thai text would go through the complex code path. There are some other cases where the complex text path is used (I think "text-rendering: optimizeLegibility" forces the complex text path).

In Safari and most of the time on Chromium Mac, the complex code path is handled by CoreText. In other Chromium ports and on Blackberry, complex text is handled by Harfbuzz-NG, which is also used by Firefox.

</comment></comment><comment><USER14> Is anyone aware of any resources that shows you how to setup your machine to see how an HTML page is rendered in Webkit, step by step, like remote debugging?

<comment><AUTHOR> The Chrome DevTools timeline gives a solid overview of the construction of each page. You can see the progression of grabbing network assets, Parsing HTML, Recalculating styles, Layout (aka reflow), Paint, Compositing

Here's a trace of this page, for example: <http://paulirish.com/i/dd8f40.png>

A far more low-level view is available in about:tracing - More on that here <http://www.html5rocks.com/en/tutorials/games/abouttracing/>

<comment><USER15> +1 the timeline tool is great. Whenever you have performance issues in a web app and javascript is not the culprit, that's the tool to turn to.

</comment></comment></comment><comment><USER16> in FF, on every page load the page scroll jumps to around the "Opera just moved to WebKit. How does that play out?" section. is this intentional?

<comment><USER17> I can confirm this too. Can only be intentional

<comment><AUTHOR> Not intentional. I believe it's the embedded slide deck afterwards. I'll de- embed it for the time being. Weird bug, not sure whose it is, yet.

<comment><AUTHOR> Addressed. I bet it was a focus() coming from inside the iframe slidedeck embed.

<comment><USER18> Still an issue.

</comment></comment></comment></comment></comment><comment><USER19> Reading the article, it sounds like "its cool because now you only have to 'dev for webkit' and forget the rest".

Makes me uncomfortable.

<comment><USER20> why?

</comment></comment><comment><USER21> I'm working on a project to embed WebKit in vehicles. One of the things I'm interested in doing is running a suite of tests to ascertain things like compliance with various specs and performance. I see the 28k layout tests in the WebKit repo, but would be interested in any other tests -- especially those that can be easily loaded into an iframe.

</comment><comment><USER22> @paulirish FYI the link to CoreFoundation is invalid

<http://developer.apple.com/corefoundation/%3Cbr%20/%3E>

</comment><comment><USER23> Just curious - How close is WebKit in terms of complexity, to the Linux Kernel?</comment>


Ask HN: Where can I learn about the latest in IT infrastructure? - <AUTHOR> Stuff like this probably isn't the most common, but I'd love to start digging around topics like storage, networking, security, etc. enough to understand why the work of the startups in the space is important.<p>Are there any recommended readings, sites, books, communities that are recommended? <comment><USER2> It's not really clear what you're looking for. I mean the scale at which a startup will be working will be more technical than what a lay person will usually be exposed to. See Perlis's Epigram 48.

'Real time' discussion of the topics you mention is occurring in various StackExchange sites and artifacts are being produced on Github and similar repositories because most of what is happening in those spaces is happening as open source. And most of it is based on things that have been happening in open source since the days of 386BSD or so.

Now if you really want a visceral understanding of what a startup might be able to sell, buy four Raspberry Pi's and a switch and a book on Linux and try to make it do something interesting that uses all four nodes of the mini network. You'll find, that like a Facebook relationship, it can be complicated and like a real relationship it's a multi-year project.

Good luck.</comment>