SLING: A Natural Language Frame Semantic Parser - <AUTHOR> https://research.googleblog.com/2017/11/sling-natural-language-frame-semantic.html <comment><USER2> Disappointed to see zero mention of related work, and no comparison against the many other papers that do transition-based SRL, many of which can produce arbitrary output graphs.

The authors use a transformation of the PropBank data, so their numbers aren't directly comparable against the literature. That's okay --- the PropBank format itself isn't that great, and we'd like a better balance of usability and learnability. However there have been dozens of systems published against a different graph transformation of the PropBank data, e.g. the CoNLL 2008 and 2009 shared task: [http://ufal.mff.cuni.cz/conll2009-st/#task](http://ufal.mff.cuni.cz/conll2009-st/#task)

There should be no problem to compare these systems against the system in this paper. The unfair comparison would be to take a system that produces PropBank output, and transform its output into this format. This comparison would be unfair because the system would be optimising a different measure. A fairer comparison would be to take a system that can produce arbitrary graphs, and train it on the authors' format.

The transition system looks quite interesting, and pretty easy to try -- so I'm actually tempted to implement this and give it a go. However with no meaningful evaluation we have no way of knowing whether it outperforms the many alternatives that have been proposed, e.g. [https://www.semanticscholar.org/search?year%5B%5D=2017&year%...](https://www.semanticscholar.org/search?year%5B%5D=2017&year%5B%5D=2017&q=incremental%20semantic%20parser&sort=relevance)

The accompanying paper is described as a tech report, so arguably it shouldn't be held to the same standard. But still. I'd have hoped to see more leadership from the anchor author Fernando Perreira -- who has written plenty of relevant literature on this, starting in the 80s.

The field is already very difficult to follow, even if papers are more forthcoming about how they fit into previous work. Without that, the claims in each paper become very difficult to understand. I would therefore suggest rejection of this paper if I received it for blind review.

<comment><USER3> I think this comes across as a little too critical.

I agree with everything you have said. But to me it looks like this came from an industrial implementation which they decided to open source.

That means they weren't aiming for SOTA on standard datasets, but for what worked best on their internal benchmarks. The evaluation procedures for the published report looks like it was derived from how they train it internally.

That means they probably had to justify all the time they were spending on open sourcing it, and writing a tech report.

So while it would be great to see standard datasets tried, I much prefer seeing the code open sourced.

Also, nothing stopping someone else publishing a nice short paper on the CoNLL datasets using this software.

<comment><USER2> I'm not sure why you're thinking this was an industrial implementation? It's built on DRAGNN, which is pretty new, so that doesn't give much time for it to be in production. Also, if it were in production the blog post would surely say that -- instead it's quite clearly marked as an experimental system, for research. The lead author seems to be a researcher, too.

<comment><USER3> The whole thing feels like it is bits of code extracted or rewritten out of an existing system.

Take the whole Myelin JIT compiler. That's a pretty weird thing to build just for research into frame parsing.

</comment></comment></comment></comment><comment><USER4> I know this is a blog post and not a research paper, but I see two big flaws in it that may mislead the reader:

 _(i)_ failing to acknowledge that this same task (semantic role labelling / semantic graph generation) is performed by many existing NLP tools and not even hinting at some comparison results. One would think Semantic Graphs (c) are a novel thing invented at Google (TM).

 _(ii)_ Presenting the fact that all the intermediate features are magically computed inside a black box as an improvement. This is, to me, a huge loss. There is great value in this classical NLP pipeline they so much criticise, especially in terms of explainability: "Why is _mail_ an agent in the semantic graph? Because the tagger wrongly tagged it as a _noun_ ". I can (easily enough) correct the above in a classical pipeline, while I'm wondering how should I stir those neurons in Google's implementation to fix it.

Anyway, this ended up a bit more ranty than I expected. I actually am looking forward to try this out and see how well it works. I think it's very positive that google open sources their research projects, as I find this is surprisingly uncommon in academia.

</comment><comment><USER3> If you are interested in this, the AllenAI group has an implementation of their model for a very closely related task (semantic role labeling) online: [http://demo.allennlp.org/semantic-role- labeling/](http://demo.allennlp.org/semantic-role-labeling/)

</comment><comment><USER5> In contrast to most of the other NLP tasks people train neural networks to do, this one is actually useful. (ex. the frame corresponds to some action that can be passed onto conventional software)

<comment><USER6> > In contrast to most of the other NLP tasks people train neural networks to > do

Could you give some examples of these types of tasks (and maybe associated projects/papers) that you think are not actually useful?

<comment><USER5> sentiment analysis over broad domains, for one. predicting the next token for Penn Treebank.

</comment></comment></comment><comment><USER7> First time I heard of DRAGNN, my mind is blown: [https://github.com/tensorflow/models/blob/master/research/sy...](https://github.com/tensorflow/models/blob/master/research/syntaxnet/g3doc/DRAGNN.md)

<comment><USER2> Mostly DRAGNN is a lot of hard work to compensate for how difficult Tensorflow makes your life if you're trying to write this type of model. If you use a library like Chainer, PyTorch or DyNet, the problem this solves simply never occurs.

<comment><USER8> This may be true for training, but I have not heard of a real deployment story for low latency NLP models in any of these define by run frameworks, which this attempts to solve.

<comment><USER2> I haven't benchmarked, but I think you'll find DyNet to be significantly faster than DRAGNN in that setting.

I'm also not sure DRAGNN is a good direction to head in, if that's the problem. You'll never ever be able to change anything in it, so if performance isn't good you're stuck. Writing the forward pass isn't very difficult, so I'd much rather be able to replace parts of a network with optimised code if necessary.

<comment><USER8> Would be good to see some non-batch inference benchmarks for this. I couldn't find any numbers online.</comment></comment></comment></comment></comment>


Addressbook History goes into the cloud with App Engine - <AUTHOR> http://ajaxian.com/archives/addressbook-history-goes-into-the-cloud-with-app-engine

<comment><USER2> I saw the presentation of this at the Google IO conference. There was a marked audience reaction when they showed this on the screens, along with calls for "can we get the source code." We're toying with an interface like this that's user-selectable between timeline-slider view and wikipedia-like history view. I'm guessing that temporal completeness in data structures and UIs with respect to any atomic data structure might be rather more significant in the future.</comment>


Largest CTF in the World: NYU-Poly CSAW CTF - <AUTHOR> https://csawctf.poly.edu/scoreboard/

<comment><AUTHOR> DEFCON CTF teams with points: 303 (<http://ddtek.biz/qualsDC20scoreboard.html>)

CSAW CTF teams with points: 639

STRIPE CTF competitors: ~7,000

CSAW CTF competitors: ~9,100

Writeups are being posted in reddit.com/r/netsec:

[http://raidersec.blogspot.com/2012/09/csaw-ctf- quals-2012-re...](http://raidersec.blogspot.com/2012/09/csaw-ctf- quals-2012-recon-1-3-writeup.html)

[http://blog.lse.epita.fr/articles/31-csaw- ctf-2012-exploitat...](http://blog.lse.epita.fr/articles/31-csaw- ctf-2012-exploitation-200300400500-writeups.html)</comment>


HashiCorp and Google: easing secret and infrastructure management - <AUTHOR> https://cloudplatform.googleblog.com/2017/09/HashiCorp-and-Google-expand-collaboration-easing-secret-and-infrastructure-management.html <comment><USER2> What do people here use to store and source-control secrets/almost-secrets and make them available to (pick n) terraform/ansible/salt/chef/...?

I've heard a lot of good things of Hashicorp Vault ([https://www.vaultproject.io](https://www.vaultproject.io)) but been hesitant to go with it.

<comment><USER3> I almost never heard anything negative about vault until I did a presentation on all the troubles I had with it. Here are the slides about the issues I had:

[https://docs.google.com/presentation/d/1ipP2eB9pW5j3WDvzCGz9...](https://docs.google.com/presentation/d/1ipP2eB9pW5j3WDvzCGz9Wy4MBoK2SMvOTZtPdf5kSNs/edit#slide=id.g1fa55b980e_1_164)

At the end of the day, Vault gives you secret keys to manage which makes automating the unsealing process not fun if you want full automation.

Talking with Armon from Hashicorp they planned to work on some much improved docs around vault which should help with a lot of the issues of making vault usable, because quite frankly they are very challenging to understand right now.

<comment><USER4> I unseal using ansible (with the unseal keys in ansible-vault) and automate the configuration fully through ansible. For example you can use the ansible expect module:

      - name: unseal 1
        expect:
          command: '/usr/bin/vault unseal'
          responses:
            'Key \(will be hidden\): ': "{{vault_seal_key_1}}"
          echo: yes
        when: vault_sealed_result.rc == 2 and vault_seal_key_1 is defined
        tags:
          unseal

<comment><USER5> I'm interested in your solution, you are using ansible-vault to store the Hashicorp Vault unseal key(s)? Isn't this just pushing the problem out another level or am I missing something? Thanks.

</comment></comment></comment></comment><comment><USER6> All of the major clouds already have good secrets management built in. We have a simple library that uses Google's Key Management Service in a standalone project to encrypt/decrypt files held in a private storage bucket. Access to keys and files are controlled by service account roles. Seamless, efficient, no-ops model with built-in auditing and fine-grained control that works everywhere.

<comment><USER7> This sounds way simpler and 10x better than what most organizations do (secrets in virsion control, secrets on local file system or environment variables). Do you mind doing a quick how-to? It could probably help 90% of organizations take a step towards better security.

<comment><USER6> Google actually has a technical how-to here: [https://cloud.google.com/kms/docs/store- secrets](https://cloud.google.com/kms/docs/store-secrets) and in-depth solution architecture here: [https://cloud.google.com/kms/docs/secret- management](https://cloud.google.com/kms/docs/secret-management)

Key Management Service creates and maintains private keys for you and provides an API to easily encrypt or decrypt some data. Basically call method _KMS.Decrypt( "name_of_key_to_use", <bytes>)_ and get back decrypted content. Secrets are simple text files encrypted with KMS and stored in a private storage bucket. For example we have something like "database- secrets.dev.json.encrypted".

We have a small library that took a day to write, used in all of our projects that does the following on startup: open private storage bucket, download encrypted file, call the KMS API, decrypt the file, and parse the raw contents as json. Now the app has the secrets in-memory to be used anywhere. No infrastructure required, nothing on disk and this is universally accessible whether inside Google cloud or on local machine. Takes under 1 second when running in the cloud.

I dont think I can do better than the documentation but let me know if you have any questions.

<comment><USER8> AWS offers a similar service for anyone interested.

[https://aws.amazon.com/kms/](https://aws.amazon.com/kms/)

You can limit access to the keys based on IAM instance profiles. So that only certain instances can access specific credentials.

</comment></comment></comment></comment><comment><USER9> Anything secret involves a master key. If you don't trust AWS, then you need to supply your own master key. But for most setup, IMO, you should just let AWS handle the key management, and you use role to decrypt. Rotation is a big deal though. For server, SSH key can be encrypted in KMS and we either completely replace the box, or we rotate one box at a time. For DB servers, it's important to choose a DB that can stream data to a new box with as little impact as possible (or allows replication). But these takes time to develop (I can't use container to host DB or critical applications because the network performance, at least a year ago).

BTW Mozilla's sops [1] is quite interesting. I've been testing this for a while now.

[1]: [https://github.com/mozilla/sops](https://github.com/mozilla/sops)

</comment><comment><USER10> The AWS EC2 Systems Manager Service and the Parameter store: [https://aws.amazon.com/ec2/systems-manager/parameter- store/](https://aws.amazon.com/ec2/systems-manager/parameter-store/) is a great way to store secrets with integrated encryption provided by KMS.

</comment><comment><USER11> It's not really clear for me from the docs. But can you now use kubernetes secrets to not be stored in etcd but in vault? Or is just the token retrieval part fixed? The docs are a bit terse and don't mention much stuff on how you'd actually use it.

If I create a kubernetes secret will it be stored in vault if I set some magic switch? Or are we not there yet?

<comment><USER12> Not there yet. You can store secrets in Vault, and now a kubernetes pod can authenticate against Vault which will allow it to retrieve secrets. If you're running your app in k8s, your app will be able to use the configured token to get to vault.

<comment><USER11> Thanks. But it seems like a good first step into the right direction!

</comment></comment></comment><comment><USER13> I worry a lot about how these megacorps will treat "collaborators" vs "non collaborators" in the coming years. Obviously you can't just outright buy everyone, but they seem to be increasingly abusive towards technologies and teams that aren't on board with their interests and ideology.

Actually I'm more worried about how Facebook and Amazon treat non compliance, but Google sure seems to be getting shadier every day.

This combined with the W3C evolving into a corrupt entity just makes me want to get out of tech completely. Maybe if I could get some awesome dev job at the EFF?

<comment><USER14> From what I can tell, this is all opensource using their publicly documented API. That is, you could implement the same support for GCP in your own auth backend product, and you could implement the same support for your own cloud platform in Vault. So… I don't really get what you're talking about in this context.

<comment><USER15> _" We're working to enhance the integration between HashiCorp Vault and GCP, including Vault authentication backends for IAM and signed VM metadata."_

There's not much detail in that. But, you could certainly read it in a way that using Hashicorp products might be lower friction than using other products on GCP.</comment></comment></comment>


Google's Quantum Computer in Limbo After Government Shutdown - <AUTHOR> http://www.wired.com/wiredenterprise/2013/10/dwave-nasa/ <comment><USER2> I have the perfect solution... The D-Wave Two should be used to resolve the argument between the Republicans and the Democrats.</comment>


The iPhone 11 and iPhones 11 Pro - <AUTHOR> https://daringfireball.net/2019/09/the_iphone_11_and_iphones_11_pro <comment><USER2> Except that I just bought a XS last year, because _that_ camera was supposed be such a big improvement (and it was). Gruber is of the opinion that the 11 has two big things going for it: battery and camera. I might not care much about extra battery life, but I _do_ care about the camera. Not to the tune of another $500 for the upgrade, though. That's okay, the leather case on the XS is just now starting to look worn in.

</comment><comment><USER3> I'm going from an XR to an 11 Pro because the 11 Pro is smaller, 5.8" vs 6.1"</comment>


The Google Chrome Comic (2008) - <AUTHOR> http://www.scottmccloud.com/googlechrome/ <comment><USER2> Took me a little while to realize the page you see when you load that url is just an excerpt from the comic. The whole thing is here: [https://www.google.com/googlebooks/chrome/index.html](https://www.google.com/googlebooks/chrome/index.html).

<comment><AUTHOR> Yeah, I wasn't sure whether to link that instead. The comic is delightful, but not too discoverable.

Anyone remember what it was like to open Chrome for the first time in '08 or so? It was incredible. The design was the embodiment of fresh, wide-eyed optimism.

<comment><USER3> It was pretty amazing. That was the day I finally ditched IE for good.

<comment><USER4> Yeah, it also felt blazingly fast. Chrome is still fast now, but sites and web apps have added so much bloat that they cancel each other out. Back then, on the other hand, Chrome was lightning compared to the standard experience with any other browser.

<comment><USER5> May be the browser should institute a 'lite' mode which kills js execution after some 60ms. If the content doesn't show up by then, an error is shown (rather than a broken page).

<comment><USER6> Then user could just disable Javascript.

</comment></comment></comment></comment></comment></comment><comment><USER7> > A multi-process design means using a bit more memory up front. Each process > has a fixed additional cost. But over time it will mean LESS MEMORY BLOAT.

Unfortunately, it hasn't quite worked out that way.

<comment><AUTHOR> Someone pointed out that websites have grown by ~10x since 2008, so a lot of the memory usage may have been unavoidable. It'd be interesting to know what the hypothetical lower-bound is.

<comment><USER8> I think, the hypothetical lower-bound would put webpages onto the hard drive whenever possible, therefore not really being dependent on webpage size...

<comment><AUTHOR> "Never underestimate the bandwidth of a station wagon filled with backup tapes hurtling down the highway."

</comment></comment></comment></comment><comment><USER9> We who were Opera fans got really upset over this because Chrome borrowed a lot of ideas from it without giving it any credit.

<comment><USER10> As an IE fan I was annoyed that they stole porn mode (IE announcd InPrivate before chrome existed). I still know quite a few people who think Chrome invented incognito and that IE ripped it off.

Now that I look it up, apparently the idea started with Safari, but it didn't get much press.

I think the biggest innovation in chrome was how rapidly they shipped. Chrome copy-catted on hardware accelerated rendering and porn mode, but shipped first in both cases.

</comment></comment><comment><USER11> Yay! That's the one that arrived in Germany a day early ([http://blogoscoped.com/archive/2008-09-01-n47.html](http://blogoscoped.com/archive/2008-09-01-n47.html))

P.S - I wonder is Sundar figured out how "shipping works at Google" yet.. ([https://youtu.be/LRmrMiOWdfc](https://youtu.be/LRmrMiOWdfc))

</comment><comment><USER12> I remember having a discussion with a co-worker the day they announced, and he swore Chrome would go nowhere, and while I definitely thought it would be an uphill battle, I brought up the fact that their home page is their greatest advertising opportunity (now, they've very smartly moved that role to the less valuable, but still insanely high traffic, properties). So I thought they definitely _could_ pull it off, but a year before chrome, I stupidly thought WebKit should just fold up shop and adopt Gecko. So obviously I can't really predict much.

But then I actually tried it as a secondary browser for about a week, and fell in love.

Not long after, I finally realized two things: 1\. Speed is the killer feature, not just of chrome, but of any software. 2\. The auto update feature, on every platform (such as iOS' app store) and browser, was the most ingenious of the two. As a web dev, my biggest worry was the potential chaos, some of which has happened, but waaaay less than I could have ever hoped. They truly did make the updates a first class experience from the developer to the different cases of users.

</comment><comment><USER10> Wow, I remember the first time I used chrome all the way back in 2008. It was a little buggy (it didn't support flash properly at first iirc) but it sure was fast.

Now it has all the features, is still fast, and just generally is more awesome than the other windows browsers, and the other mac browsers, and from most of the Linux desktops I've seen, the other Linux browsers.

I know it's a privacy issue and I don't particularly like boosting Google's numbers, but I don't see myself switching away anytime soon.

</comment><comment><USER13> > but as time goes on, fragmentation results -- little bits of memory still > get used even when a tab gets closed

Don't you still have the same problem with multiple processes, just moved to the kernel for them to deal with? It seems like if you kept everything in- process, you could use custom allocators and apply a sufficiently smart architecture so nothing stayed longer than necessary.

<comment><USER14> I think it's considered a pretty bad leak in the kernel if terminated processes leave unreclaimable memory behind.</comment></comment>


Quick test to see how well you know popular digital products - <AUTHOR> https://www.producthunt.com/games/frozen-products <comment><USER2> I got 80%. Very nice.

<comment><AUTHOR> Congrats!:)</comment></comment>


Cassandra - Trial by Frustration - <AUTHOR> http://blog.meltingice.net/work/cassandra-trial-frustration/

<comment><USER2> To save some frustration, I'd recommend starting with the DataStax Cassandra documentation: <http://www.datastax.com/docs/1.1/index>

For "timeline" type data like this, I'd recommend reading these two explanations:

[http://rubyscale.com/blog/2011/03/06/basic-time-series- with-...](http://rubyscale.com/blog/2011/03/06/basic-time-series-with- cassandra/)

[http://www.datastax.com/dev/blog/advanced-time-series- with-c...](http://www.datastax.com/dev/blog/advanced-time-series-with- cassandra)

P.S. comments on your blog are 404ing</comment>


Show HN: Drawingbots.net – Website for Plotter Enthusiasts - <AUTHOR> https://drawingbots.net/ <comment><USER2> Plotters, turtles, And even printers are great ways to get people interested in tech and math. Software may eat the world but it's ephemeral and you can't put it on the fridge.</comment>


How times have changed for PostgreSQL - <AUTHOR> http://opensource.com/business/14/3/postgresql-ugly-no-more <comment><USER2> I was 76% expecting a treatise on how various time and date fields and functions have evolved over PostgreSQL versions.

</comment><comment><USER3> I have been using Postgres since I was a student, but I must recognize that being a better programmer than sysadmin, I always used it after some kind of abstraction layer. Today, this layer is Rails with ActiveRecord.

I want to convince my boss to use Postgres instead of MySQL in a new project we've just started. Does anybody know about a good comparison of both databases, or a nice list of unique Postgres features? I have been googling a little bit but the findings weren't that good.

<comment><USER4> Reasons I like PostgreSQL compared to MySQL:

\- Timestamp with time zone

\- More robust, fewer crashes, less corruption of data

\- More features (JSON data type, partial indexes, function/expression indexes, window functions, CTEs, hstore, ranges/sequences/sets, materialized views, too many to list)

\- More disciplined (doesn't do things like auto-truncate input to get it to fit into a column)

\- Not owned by Oracle, it's actively developed, regular major release schedule, developers/maintainers are talented and trustworthy, etc.

\- Better Python driver (don't know about other languages)

\- Choice of languages for database functions/procedures (Python, JS, etc.)

\- Better partitioning support

\- Better explain output, explain analyze, buffers

\- Multiple indexes allowed per table in a query (I hear MySQL has made a little progress here since I last used it)

<comment><USER5> Add "built-in fulltext search" and the fact that you've just replaced Redis, Sphinx/Solr/Lucene, and Mongo with a single application.

THAT, I think, is the biggest sell, at least to me. The fact that PgSQL does so much and so well.

I used Pg for full-text search in the past and the fact that I did not have to bother with setting up Solr or an interface was a wonderful feature. The search index lived right alongside my data.

<comment><USER6> Technically mysql has "built-in fulltext search" as well, but the implementation leaves something to be desired.

<comment><USER5> I couldn't find anything nearly as powerful as Pg so I would discount it fully.

</comment></comment></comment></comment></comment><comment><USER7> If you want job security and fat paychecks for the next decade, this is a very good time to learn PostgreSQL well and specialise as a PostgreSQL consultant.

<comment><USER3> The question I always ask myself is: how does one learn specialized stuff like managing a DB without working with it professionally? This also applies to Big Data, scaling, etc.

I would love to work building highly scalable systems, but I don't get to do it at my current position, and all the job offers out there require having experience doing it. Looks like a chicken-and-egg problem to me.

<comment><USER7> Contribute to open source software, attend meetups, talk at meetups.

<comment><USER3> Can you recommend some interesting open source projects for an experienced Ruby / Rails developer, but with very little open source experience?

</comment></comment></comment></comment><comment><USER8> It is worth pointing out that AsterData and Redshift are built on pgSQL. From what I have read, this is due to licensing issues. MySQL is ok but I'm unconvinced that MySQL is a good option for single instance terabyte size databases. I'm not sure if pgSQL is any better but you hardly see large unwieldy pgSQL instances. Large slow MySQL instances are everywhere. And it's frustrating.

</comment><comment><USER9> What are the best resources for one to learn PostgreSQL?

<comment><USER10> I'd start here: [http://www.postgresql.org/docs/9.3/static/index.html](http://www.postgresql.org/docs/9.3/static/index.html)

I've always found this documentation to be great.

If you've worked with RDBMS' before, you can skip around the chapters. If not, I'd read up through chapter 14 and go from there.

You should be able to easily install it on whatever OS you're running.

</comment></comment><comment><USER11> Since we are on the subject of PostgreSQL, does anyone know of a good resource that would teach me how to create webapps where each "customer" would get their own "database"? What are the best practices for that? Also, what are the best practices for ALTER'ing tables in production (e.g. adding/deleting columns).

Here's a concrete (made-up) example: I want to write a webapp where people would sign up to do some personal tracking. They create variables they are interested in (weight, mood, calory intake, etc.) And then enter their data daily. So for each customer I need to have a different database with different columns, and I want them to be able to add/delete variables "on the fly". Is it very straightforward, and hard to get wrong? Or are there "best practices" for this sort of thing? Thanks.

<comment><USER12> Look up the following terms

1\. SaaS tenancy models. This is the data separation.

2\. Custom fields would be usually represented as an EAV model (Entity- Attribute-Value).

3\. Don't ALTER in production if you can help it. If you're going to do it, use migrations which are scripts which first add columns, then transform data, then reapply constraints.

<comment><USER13> I've also seen applications basically just assign a fixed number of additional columns on the relevant tables for custom fields - although this approach doesn't win any awards for elegance it can be pretty straightforward.

From what I've seen, applications that use EAV tend to evolve to suffer from bad cases of the "inner platform effect":

[http://en.wikipedia.org/wiki/Inner- platform_effect](http://en.wikipedia.org/wiki/Inner-platform_effect)

NB There is nothing "wrong" with using EAV - just that it seems prone to misuse (a bit like XML).

<comment><USER12> The only problem with a limited number of custom fields attached to the table is that they aren't necessarily optimised for sorting, might extend past the row size limit of database tables (this is a pain!), can't be reliably typed and a client will always want one more.

EAV is a sort of inner platform thing I agree, but the correct solution i.e a document store with full field level indexing that works with enterprise loads doesn't exist (yet). CouchDB was promising on that front but didn't go all the way.

XML is fine. Just don't stick it in database columns (my favourite chunk of pain!)

<comment><USER14> I like HStore a lot better than XML for postgres. And it's Waaaaay better than the wide table model.

Way back when, I did a data model that added columns for arbitrary data fields that the users wanted in PG, and wound up with a wide table model. PG can store a surprisingly large number of fields, I think it got up to the mid to high hundreds after years of this. At the time, hstore wasn't there, xml was either not there yet, or just recently added. And the queries for EAV looked surprisingly awful, especially when added to the not exactly straightforward queries we were doing on the events.

It wound up being an extremely large, extremely sparse table, with some fields having 100% usage, most having >>.01%, and a few getting used in the 1% range. On the plus side, it was possible to index any of the fields, which was especially useful with functional or 2 column indexes.

If I had to do it again, I'd be on hstore, or maybe hstore/json. It wasn't pretty but it wasn't the fatal flaw in that startup.

</comment></comment></comment></comment></comment><comment><USER15> Yes PostgreSQL is awesome. From my experience with a large installation, we had many more issues with the actual hardware on EC2, than we had with postgres which ran on said hardware. However, I was pleasantly surprised Amazon's RDS now supports PostgreSQL too, so you can abstract away most of the hardware considerations, assuming they've optimized their stack: [https://aws.amazon.com/rds/postgresql/](https://aws.amazon.com/rds/postgresql/)

</comment><comment><USER16> If I have a bunch of records that need to be fetched together often and can't be dumped in a hstore of JSON column, how can I ensure data locality on disk in postgres?

Since primary key is clustered in MySQL, I choose an appropriate key. Is there an equivalent mechanism or is periodically running the cluster command the only option?

<comment><USER17> You cannot do this other than relying on covering indexes and index only scans. PostgreSQL does not support index organized tables.

</comment></comment><comment><USER18> " I see experienced enterprise software professionals who are versed in operating systems and Windows, for example, taking the training to advance their careers. "

That had me laughing.</comment>


Learn Shen - <AUTHOR> http://www.shenlanguage.org/learn-shen/index.html# <comment><USER2> It has an absolutely myopic and absurd license.

      http://www.shenlanguage.org/license.html

The strange part is "no derivatives that don't comply with the spec, even if you rename it." It is not free software.

Unfortunately this is something I have to mention every time Shen is mentioned, because I'm irate that, in spite of my interest in it, that both it (and its predecessor Qi) insist on doing something so strange with no apparent legitimate purpose (some kind of bogeyman of fragmentation killing the project).

<comment><USER3> I think the license is worded rather poorly and that's unfortunate. I think it would be better worded as this: if you create a port/version/mod of Shen, it must be able to execute spec-compliant Shen code. That's it. You can do all the changes, experimentation, etc. so long as it is somehow capable of executing stock Shen code, it's allowed. I think that means that you can even have a flag that toggles between "standard-compliant" and "everything-works- different" because anyone with Shen code can set the flag to "standard- compliant" and run their code. So that situation would be fully compliant with that license.

<comment><USER4> _I think it would be better worded as this: if you create a port/version/mod of Shen, it must be able to execute spec-compliant Shen code._

Who decides whether a language I come up with is a port/version/mod of Shen? How much of Shen's features can I include without crossing that threshold? Does that threshold move if I can point to other non-Shen languages as the source of those ideas?

<comment><USER3> The license is for the software--the Shen implementation. If you don't use the Shen implementation provided, then you're not subject to the license. If you create a port/version/mod of Shen then you are using the software. The way ports of Shen are created is by implementing Klambda. The provided Shen implementation compiles Shen to Klambda, which your port then compiles to a host language. So, unless you really want to clean-room reimplement the entire Shen language (which is NOT how porting is intended to be done) then you're using the software that is subject to the license.

</comment></comment></comment></comment><comment><USER5> I don't understand what Shen brings to the table that other functional languages don't. Is there a concise imperative of why to learn Shen?

<comment><USER6> It seems like Shen is a successor to Qi, which if I remember right, had a type system that incorporated full-blown classical logic. If Shen follows in this tradition, then its type system would be significantly more expressive than standard functional type systems. It would allow not only arbitrary predicates on input and output, but also relationships between them. Needless to say, such a type system is Turing-complete, and therefore not guaranteed to terminate.

Systems like this are usually used as theorem-proving systems, not actual production languages. I think Qi was intended to be a practical language that allowed arbitrary proofs about the properties of its programs.

I have not been able to find any reference to this in a quick perusal of Shen's documentation.

<comment><USER7> Standard type systems are Turing equivalent. This includes the type systems of functional languages like ML and Haskell. In fact, there's a neat correspondence [0] between algebraic data types and propositional logic (examples in Haskell):

      union types (data T = mkTA A | mkTB B) <-> logical disjunction (A ∨ B)

            product types (data T = mkT A B) <-> logical conjunction (A ∧ B)

            function types (type T = A -> B) <-> logical implication (A → B)

                          the unit type (()) <-> true (⊤)

                                  bottom (⊥) <-> false (⊥)

(And, if you were wondering, that's where the symbol for "bottom" comes from!)

Inference on richer type systems, e.g., systems that support dependent types (types built up from expressions about the values they represent) is in general undecidable, which is why some dependently typed languages (e.g., Coq) have type systems that are not Turing equivalent—the language designers have consciously pared down parts of the type system in order to guarantee that type-checking terminates.

0\. This is the Curry-Howard correspondence: [http://en.wikipedia.org/wiki/Curry–Howard_correspondence#Cor...](http://en.wikipedia.org/wiki/Curry–Howard_correspondence#Correspondence_between_classical_logic_and_control_operators)

<comment><USER8> Shen is an example of a language with a richer type system. Dependent types can be implementated in Shen.

<comment><USER7> Yes. My response was of course to clarify that "ordinary" type systems are Turing equivalent and incorporate classical logic.

Dependent types correspond to first-order logic in the Curry-Howard interpretation. I don't know much of anything about Shen, so I can't comment further on it specifically.

By the way, limited dependent types can also be implemented in Haskell through type-level programming. The paper that introduced this idea [0] is a good read, and doing some type-level programming is a great way to get to grips with the ins and outs of Haskell's type system. If you're interested in type- level hackery, also be sure to read the HList paper [1].

0\. [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.22.2...](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.22.2636)

1\. <http://homepages.cwi.nl/~ralf/HList/paper.pdf>

<comment><USER9> The argument is that agda is relatively straightforward for haskellers and similarly, coq for ocaml devs to adopt. This is a good succinct example of former. Coq has lots of tutorials

[http://www.cs.uu.nl/education/scripties/pdf.php?SID=INF/SCR-...](http://www.cs.uu.nl/education/scripties/pdf.php?SID=INF/SCR-2011-062)

There's also typed racket, which i don't know anything, it's been mentioned to me as "theorems for free"

<http://docs.racket-lang.org/ts-guide/>

</comment></comment></comment></comment></comment></comment><comment><USER10> I think for macro enthusiasts, being able to program the reader directly(with pattern matching) is a good selling point. It also has an inbuilt compiler compiler.

</comment><comment><USER11> I've bought the Qi book and have been enlighted by the language it describes. I'm looking forward to play a bit with Shen and see how it compares to other lisps.

</comment><comment><USER12> Interesting logo! lol.

</comment><comment><USER13> Anyone else thought of Shen in League of Legends when this was posted? :)

<comment><USER14> Yep. Sharing a name with one of the most played characters in the most popular multiplayer game in the world might not be very good for SEO, I assume.

A quick google for 'Shen' proves as much.

<comment><USER15> In my google search for 'shen', the language homepage is on 4. position. That's pretty well for an obscure programming language imho. LoL character has 2 results and one video in the first page, so it's not really overpowering other Shens.

<comment><USER16> Errr, in the search I just did it's #3. But I guarantee you that has a lot more to do with Google's knowledge of what I search for and am (potentially) interested in than anything else.</comment></comment></comment></comment>


Show HN: Colourising video with machine learning and serverless functions - <AUTHOR> https://dev.to/developius/colourising-video-with-serverless-machine-learning-c8a <comment><USER2> OpenFaaS looks pretty cool. Have you also checked out Phission and Kubeless (both of which are also FaaS solutions built on top of Kubernetes)? I would be curious to hear your thoughts on those.

<comment><AUTHOR> I've not personally tried either of those, but OpenFaaS also supports Kubernetes out of the box with faas-netes ([https://github.com/openfaas/faas- netes](https://github.com/openfaas/faas-netes)) :)</comment></comment>


How To: Hack Boxee To Bring Hulu Back To It - <AUTHOR> http://lifehacker.com/5157615/how-to-reinstall-a-working-hulu-in-boxee <comment><USER2> It should never have been removed.

Tsk Tsk NBC & News Corp.</comment>