Ask HN: What are the cons of PostgreSQL? - <AUTHOR> Everywhere I go, people are singing the praises of PGSQL. I never hear about the disadvantages. Maybe it's just that good and there's nothing to complain about. <comment><USER2> I've worked with Postgres, MySQL, Oracle, and SQL Server, and Postgres is my go-to choice. I can think of a few cons though:

\- Replication is harder to set up than MySQL. And there is no master-master replication like in MySQL. (Of course it also doesn't lose your data.... :-)

\- I wish there were support for SQL:2011's bitemporal stuff. (There are some tools/extensions/published patterns to add "transaction-time" aka "system- time" support, but none I know of for "valid-time" aka "application-time", let alone both at once.) But this is very niche and not really fair to expect so quickly from a free product.

\- I wish I could share a database between separate users, give them each their own schema, and not let them do `\dn` to see what other schemas exist. Again, very niche.

. . . Okay I thought I would have a longer list but I've run out of ideas. :-)

\- EDIT: One more. You can't do `UPDATE ... ORDER BY ...`. This mattered to me once when I had multiple multi-row UPDATEs running at the same time, and if they touched the same rows but in different order, they would deadlock. You could solve that with `ORDER BY id` so that everyone obtained locks in the same order. Apparently this is not a big enough problem for anyone else in the world to care though.

I think the lesson here is that there are no real gotchas, just small annoyances that are very unique to your own project. Unless you have quite unusual requirements, I doubt there is going to be a deal-breaker issue.

<comment><USER3> PgCluster, BDR, rubyrep, and Postgres-XL all provide master-master (or analogous) replication schemes. I cannot attest to the robustness / production-readiness of any of them. And yeah, it really is a PITA to set it up with Postgres. I will note that I once[1] lost a whole cluster's data due to a combination of Slony's sharp corners and operational difficulty of Postgres.

The schema visibility thing seems like it might be solvable with row-level security in 9.5? (I.E. apply constraints to the system catalogs, perhaps?)

The UPDATE one is good to know. Did you work around it by doing `SELECT ... ORDER BY id FOR UPDATE`?

[1] - Slony had a problem where global transaction IDs rolling over caused it to do Very Bad Things and we had had to turn vacuuming off because performance during a peak period -- but we forgot to turn it back ON and... So yeah, a combination of sharp corners in Slony and Postgres + user error == our cluster systematically ate itself.

<comment><USER2> Ha, I thought about RLS on system catalogs, but it's not supported. Here is a long thread about it: [http://postgresql.nabble.com/Multi-tenancy-with-RLS- td586211...](http://postgresql.nabble.com/Multi-tenancy-with-RLS- td5862118.html)

It is also possible to revoke privileges on pg_namespace, but that breaks too many things for my taste (\dt e.g.).

I think I worked around the deadlock issue by detecting the failure in application code and trying again. It was for background workers in a side- project SaaS I abandoned after a few months, so a sloppy fix was very tolerable. I like your SELECT FOR UPDATE idea though!

</comment></comment></comment><comment><USER4> In my experience, PostgreSQL misses just a few things compared to MySQL:

\- Tables cannot be organized as a clustered index (this can be useful to control rows ordering at the storage layer).

\- No builtin query cache (but I'm not sure it's very useful).

\- It's easier to find hosted MySQL services like Amazon RDS Aurora and Google Cloud SQL.

\- YouTube Vitess for MySQL is really great. I don't know if there is something similar for PostgreSQL.

I won't write about the endless list of PostgreSQL's advantages since it wasn't the question ;-)

<comment><USER5> FYI Amazon RDS now supports Postgresql: [https://aws.amazon.com/rds/postgresql/](https://aws.amazon.com/rds/postgresql/)

<comment><USER4> You're right. My comment was misleading. Amazon RDS supports PostgreSQL. What I meant is that it would be great to have Aurora' scalability and performance with PostgreSQL instead of MySQL.

</comment></comment></comment><comment><USER6> First, postgres really is great. The few things I'll list here do not make me willing to replace it with something else, however:

1\. Tooling is not on par with other databases. If you want nice GUI that holds your hand and does everything with one click - you may find offer for postgres lacking here. I don't care much about that, but some people do.

2\. There is no "in memory" storage (but you can use in memory fs to get the same effect).

3\. Extensions often define their own operators (that's good), which are made of some combination of +-*/>=~!@#%^&|`?. Yes. Your code may look like its perl. You can make "select foo @@>&!|~ bar from baz" to be valid query.

4\. Finding hosting for postgres in not a problem, but its not something as universal as mysql.

Overall, it really is solid database. I've never been disappointed by postgres, while mysql is always a walk through a forest of wtf for me.

<comment><USER7> __2\. There is no "in memory" storage (but you can use in memory fs to get the same effect). __

    # Wouldn't this work?
        fsync=off
        synchronous_commit=off
        full_page_writes=off
        bgwriter_lru_maxpages=0

<comment><USER8> I believe data would still be written out to disk (unreliably) during checkpoints with this config.

<comment><USER9> Unless you define the tables as unlogged. In that case they'll only be written to disk on a regular shutdown and/or if there's not enough space in shared buffers.

</comment></comment></comment></comment><comment><USER10> I have found it unnecessarily distracting to evaluate reputable databases on incidental features. If an important feature is missing it will be added eventually, if you need a very specific feature then you already have your answer as to which DB to use. Unless you have a very specific reason not to use PostgreSQL as your SQL database then you should just use it.

I simplify the selection to the more general architectural value. Do I need SQL, NoSQL, or graph? What about deep JSON queries, embeddability, specific performance requirements, scalability?

Given that, I use PostgreSQL as the default choice for SQL/JSON db in all cases that don't require imbedding. SQLite for imbedding.

There is one big disadvantage, the same for all fixed schema index based databases. Scalability. Engineering effort rapidly goes to infinity as we asymptotically approach the throughput event horizon.

</comment><comment><USER11> No support for temporal data as in SQL:2011.

There were some attempts but none made it so far. See also: [https://wiki.postgresql.org/images/6/64/Fosdem20150130Postgr...](https://wiki.postgresql.org/images/6/64/Fosdem20150130PostgresqlTemporal.pdf)

<comment><USER12> > No support for temporal data as in SQL:2011.

Clearly, it doesn't have support for SQL:2011 temporal _syntax_ , but with range types and exclusion constraints, it seems to have fairly good support for doing the things you can do with SQL:2011's temporal features.

(And, having dealt with business data where _multiple_ business [application] date ranges of relevance applied, I think that SQL:2011's "bitemporal" approach which provides special syntax for a _single_ each system of system and application date/time range is in some respects creating an overly- specialized solution to a too-narrow subset of the temporal data problem.)

</comment></comment><comment><USER13> For me the only con is just upgrading when there are patches etc.

There are so many upsides to PostgreSQL. I build my food app on redis as I did not know what my schema was going to be. That worked great, but simulating the power of what you get with PostgreSQL is that much harder.

I am in the process of converting things back to PostgreSQL now that my schema has settled. The one thing I will probably keep in Redis is the geo location code as this feature works super well.

<comment><USER14> Have you seen PostGIS?

</comment></comment><comment><USER15> The only thing I missed in Postgres after switching from MySQL was that text was not searched case insensitively, i.e. searching for "alex", would not match "Alex". However, this was easily solved by using the CITEXT type, which is exactly for that purpose, but you have to enable it as an extension [1].

Except for that the only thing that was a problem after switching is that Postgres is a bit more strict about some stuff, such as dividing by zore, which throws an error in Postgres but results in NULL in MySQL.

The main reason I love Postgres, and which is also why I switched, are the Database Definition language statements that support transactions (ALTER TABLE). This allows for much better migrations, if for some reason something fails you will not be in a state that is undefined.

[1]: [https://www.postgresql.org/docs/9.1/static/citext.html](https://www.postgresql.org/docs/9.1/static/citext.html)

</comment><comment><USER16> It's not as widely supported as MySQL (though that is changing over time). Until very recently it lacked UPSERT functionality.

</comment><comment><USER17> I've used it at work for 11 years. No real complaints. Love it.

Some people making just one app might look at SQLite, simply because it's simpler to set up. But those are my only two suggestions: PostgreSQL or SQLite.

</comment><comment><USER18> Lack of query plan locking. I've had the query heuristics go bad and queries that used to take under 5 ms balloon to several seconds. It would be nice to be able to lock query plans that deliver satisfactory performance for very high volume queries.

It's a great db though, with very few bugs, high quality code, responsive devs, and relatively cheap support plans (like maybe $25k/year to get 24x7 support? I don't remember the numbers exactly) available from enterprisedb.com who have a bunch of core devs. In my one experience they knew their stuff.

</comment><comment><USER19> No built in compression. Really big deal for reducing IO and DB size.

</comment><comment><USER20> Select count(*) takes a surprisingly long time, even on an indexed table. Apparently thus is because of the MVCC model and row count estimates are available to mitigate.

<comment><USER8> My understanding is that this problem was alleviated with index-only scans in Postgres 9.2. Using these, each tuple no longer needs to be visited to check for visibility.

[https://wiki.postgresql.org/wiki/What's_new_in_PostgreSQL_9....](https://wiki.postgresql.org/wiki/What's_new_in_PostgreSQL_9.2#Index- only_scans)

<comment><USER21> Index only scans work in some cases but not all. In order for an index only scan to be more efficient than a regular index scan, the visibility map needs to be up to date. This is a problem because (currently) the only thing that updates the visibility map is the vacuum and the autovacuum only runs after a large number of updates/deletes happen on a table. If your table is largely read/insert only, the only way to keep the visibility map up to date is to manually call VACUUM on your tables periodically which is a huge pain.

</comment></comment></comment><comment><USER22> Something relatively minor I've noticed coming from MySQL, you can't change the column order of a table without recreating it.

</comment><comment><USER23> Compared to the big commercial engines, the one thing I miss is query parallelisim, but the initual stages are coming in v9.6: [http://rhaas.blogspot.com/2015/11/parallel-sequential- scan-i...](http://rhaas.blogspot.com/2015/11/parallel-sequential-scan-is- committed.html)

</comment><comment><USER24> Sometimes I wish PostgreSQL supported a shortcut syntax like Oracle's CONNECT BY. It's so much more compact than the equivalent CTE.

Oracle's installed base is the main reason my current gig is highly dependent on it. In so many cases, we have to integrate using Oracle "datapump" configuration. We'd look silly not running Oracle.

<comment><USER12> > Sometimes I wish PostgreSQL supported a shortcut syntax like Oracle's > CONNECT BY.

The commercial PostrgreSQL derivative from EnterpriseDB does support CONNECT BY, but since Oracle compatibility seems to be a big part of what they sell and SQL standards compliance is a big focus of the core Postgres product, I don't see it likely that that would get contributed back to the core. (And CONNECT BY is, IIRC, substantially more limited than standard recursive CTEs, so not having it, while it does hurt migration from Oracle's proprietary dialect, avoids people heading down the CONNECT BY path and then being forced to burn it down and start over with standard CTE syntax when they run into a wall. So there's a case that its better _just_ to have standard CTEs.)

</comment></comment><comment><USER25> The stored data format isn't compatible between major versions.

Because of that, upgrades are either stressfull and cumbersome (ie. slony + switchover to a promoted and upgraded slave), or imply a large downtime (ie. pg_upgrade).

Also, because of that (WAL format), you can't use native replication between different major versions of PostgreSQL.

</comment><comment><USER26> It is not easy to bundle it. You can statically link to SQLite (afair, but you can ship the .so/.dll file to get the same result) and put the database in a "mydb.db" file which makes your program easy to distribute. Postgres requires a server to be running.

Other than that, Postgres is (IMHO, as always) the perfect RDBMS.

</comment><comment><USER27> Can't upgrade across major releases without downtime. (Supposedly is on the roadmap for next release or two)

Doesn't allow you to force the use of an index in querying.

Materialized views always do a full rebuild, instead of applying deltas.

No estimates for when a long-running query will complete.

</comment><comment><USER28> It annoys me that I can't use uppercase letters in table or column names without wrapping them in nasssty quotation marks.

</comment><comment><USER29> Doesn't have a bitcount function unlike MySQL, which I ended up needing for one problem.

<comment><USER3> Not built in, but you can do this:

      CREATE OR REPLACE FUNCTION bitcount(i integer) RETURNS integer AS $$
      DECLARE n integer;
      DECLARE amount integer;
        BEGIN
          amount := 0;
          FOR n IN 1..16 LOOP
            amount := amount + ((i >> (n-1)) & 1);
          END LOOP;
          RETURN amount;
        END
      $$ LANGUAGE plpgsql;

<comment><USER29> The reason for a bitcount was to check DNA sequences very quickly. Using a loop rather than a single CUP operation slowed it down a fair bit.

</comment></comment></comment><comment><USER14> Support for distribution (multi-master, etc.)

But this is for all of the right reasons.</comment>


Data Mining is a new weapon in the fight against Medicaid fraud - <AUTHOR> http://www.dataminingblog.com/guest-post-data-mining-a-new-weapon-in-the-fight-against-medicaid-fraud/

<comment><AUTHOR> Possibly the start of an arms race between fraudsters and data miners?</comment>


Q: addition, multiplication, exponentiation, then?? - <AUTHOR> http://answers.google.com/answers/threadview?id=743129

<comment><AUTHOR> I send this in because I've had a hunch about this my entire life. Math is fascinating.

</comment><comment><USER2> Thanks, I had no idea!</comment>


Huawei risk can be managed, say UK cyber-security chiefs - <AUTHOR> https://www.bbc.co.uk/news/business-47274643 <comment><USER2> This is interesting. What this says to me is that the UK is essentially saying to China "Yeah we'll play your game". They're expecting some sort of attack, but they reckon that they've got the capability (at GCHQ or wherever) to counter that, and possibly turn the tables by feeding back counterintelligence.

It could potentially be a good book in 1-200 years' time once all this is declassified.

<comment><USER3> Or, could it be possible that they audited the code and found no backdoors or security issues? Now that would be crazy! /s

<comment><USER4> On that note, backdoors would either be hidden in proprietary apps, or delivered through OTA updates. Maybe a mix of both. I'd be interested in a diff of stock binaries vs recompiled "source" binaries to see if they differ significantly enough.

</comment></comment></comment><comment><USER5> Some context for those unfamiliar; Huawei and the UK government setup the "Huawei Cyber Security Evaluation Centre" to review Huawei device inner workings and updates before deploying in the UK.

Despite this, it sounds like the UK is still removing Huawei devices from the most critical areas of their networks.

</comment><comment><USER6> The way I heard it is that the U.S. could not domestically provide an off the shelf end-to-end 5G solution. In addition, Huawei had at some point declined to install NSA backdoors. This was essentially the backdrop that led to Huawei being tarred and feathered.

<comment><USER7> > The way I heard it

> In addition, Huawei had at some point declined to install NSA backdoors

I think a statement like this needs some corroboration beyond word of mouth

<comment><USER8> > I think a statement like this needs some corroboration beyond word of mouth

I'd like to see that same standard applied to the initial allegations against Huawei. Still no proof that they've done anything wrong.

<comment><USER9> The question is not whether they've done anything wrong yet. The question is what they will do if Xi Jinping orders them to deploy a backdoor.

</comment></comment></comment></comment><comment><USER10> Aren't most of the alternatives to Huawei EU companies? e.g. Ericsson, Nokia? If they are acknowledging there is a risk, why not buy from them instead?

<comment><USER11> Two things I have seen mentioned from industry people: Huawei is a lot better in R&D (use more money than Apple) and getting some feature you want deployed is easier and faster. Secondly Huawei is not only cheaper but also higher quality.

I know this doesn't fit with the witch hunt on Chinese companies the US is running but there you have it.

<comment><USER12> Not only that, they also heavily benefit from the R&D done at other companies. Add in generous subsidies from their parent government and you have a very attractive commercial offering, certainly.

Indeed, this kind of money is rarely invested without a known, outsized, expected return and, in this, I seriously doubt Huawei and their owners are lacking in intelligence. I mean, who do we think they are - Winnie-the-Pooh?

<comment><USER13> Not only that, Boeing and Airbus also heavily benefit from the R&D done at other companies. Add in generous subsidies from their parent government and you have a very attractive commercial offering, certainly.

Indeed, this kind of money is rarely invested without a known, outsized, expected return and, in this, I seriously doubt Boeing and Airbus and their owners are lacking in intelligence. I mean, who do we think they are - Winnie- the-Pooh?

<comment><USER12> Power is as power does, or so I've been told.

</comment></comment></comment></comment></comment><comment><USER14> how does one manage a closed source binary with a backdoor in it in one's network equipment

<comment><USER3> CSEC has access to the code that is runs on Huawei's kit. Huawei is required to have reproducable builds to show that the code running in the network is the exactly same as what was vetted by CSEC.

<comment><USER9> Which provides exactly no protection against hardware backdoors.

HCSEC had 30 staff in 2015. Are they claiming to have done a line-by-line audit of all Huawei's code by now?

<comment><USER15> Obviously software build reproducibility

"Huawei is required to have reproducable builds"

doesn't protect from hardware backdoors. That solution is the proper solution to the software backdoors, again:

"the code running in the network is the exactly same as what was vetted by CSEC."

<comment><USER9> I know it's obvious. The point is that some tens of HCSEC staff (mostly Huawei employees) thoroughly "vetting" tens of millions of lines of Huawei code for backdoors would prove nothing, even if it wasn't ludicrous, which it is.

<comment><USER15> > would prove nothing

 _It could surely prove something_ : those are reproducible builds. That means you can prove _after some breach is detected_ that it originates from the given sources, if it is so. That in turn means that if something happens it won't be Huawei employees who would investigate these sources. I'm sure that once a company offers the sources like this the company itself won't plan to mess with the sources. Because then the unwanten intervention can be proved.

</comment></comment></comment></comment></comment></comment><comment><USER16> The UK is in the midst of Brexit and is trying to increase trade with China. In addition, the Chinese seem the only ones able or willing to supply nuclear plants to the country...

They cannot afford to fall for the US-led hysteria.

</comment><comment><USER17> Did they also consider DOS attacks of various kinds?

<comment><AUTHOR> Yeah. I mean one attack is eavesdropping on conversations, but almost as bad a threat would be to just remotely disable the equipment.

</comment></comment><comment><USER18> Looking forward to seeing Huawei OS being announced, and the backlash that is going to trigger across the whole world.

<comment><USER3> Huawei already has an OS for IoT applications: [https://en.wikipedia.org/wiki/LiteOS](https://en.wikipedia.org/wiki/LiteOS)

<comment><USER19> Source code here: [https://github.com/LiteOS/LiteOS](https://github.com/LiteOS/LiteOS)

</comment></comment></comment><comment><USER20> "Managed" by who exactly?</comment>


Ask HN: How to prepare for Tech Interviews? - <AUTHOR>

Hello HN, I'm a CS Major. TBH I'm not comfortable with Algorithms and Data Structures, I looking to the refresh my knowledge. Any recommendations for Books or Websites where I can refresh CS concepts? <comment><USER2> I am making a post: and that post, is that you should check out this github repository ([https://github.com/cassidoo/getting-a- gig](https://github.com/cassidoo/getting-a-gig)). This is a repository and it was made by a developer, and that developer is a woman who documents how she received 10 offers.

</comment><comment><USER3> Go on informational, throw-away interviews. That and do something like [https://leetcode.com/](https://leetcode.com/)

No secret sauce here. Just need practice.

</comment><comment><USER4> [http://kelukelu.me/interview/prep.html](http://kelukelu.me/interview/prep.html)</comment>


Is It Time to Talk About Post-Capitalism in America? - <AUTHOR> https://forwardblog.substack.com/p/is-it-time-to-talk-about-post-capitalism?r=5iwv9&utm_campaign=post&utm_medium=web&utm_source=hackernews <comment><USER2> Change would had an opportunity to happen if by far most people could think by themselves and not be easily manipulated. But both cognitive bias and bad education systems (sometimes intended that way) goes against that possibility.

The system is fragile, but shaking it won't lead to an stable state (not in the same conditions/number of participants that we are having now) but to an unstable and/or fragile one.

And we are running out of time to make the big changes in global culture needed to derive this into an antifragile state. The ball is already rolling toward climate change, mass extinctions, ecosystems destruction and the world becoming unsustainable for most of us, and we will not be able to stop it.

</comment><comment><USER3> No, it's not time.

Before coronavirus, we had a roaring economy, record low unemployment, rising wages, and a soaring stock market.

I think we can return to that state, let's not radically change things. Coronavirus was a 'black swan'.

<comment><USER4> In addition, the US economy is rebounding faster than anyone predicted, indicating a strong foundation. Capitalism did that.

</comment></comment><comment><USER5> Good idea, you won't be allowed to afterwards.

<comment><USER6> Hopefully that will be a short period.

</comment></comment><comment><USER7> "Millennials are the most informed..."

Millennials have least overlap with a living example of Communism. A turning point in Orwell's life came when he visited USSR. There's no USSR for a millennial to visit and validate his/her theories, College campus politics doesn't challenge them enough.

<comment><USER6> Orwell was opposed to the USSR when he realized their imperialism, not when he realized that socialism was a bad idea.

In any case, there are living examples of socialism right now. They are small in scale, but seem to be better than capitalism. Millennial socialism is also much closer to the socialism of George Orwell in any case, because it is much more libertarian than that of the 1900s.

<comment><USER8> Where are you talking about? Cuba, Venezuela?

<comment><USER6> Modern experiments would be Rojava, the Zapatistas, and so on.

Venezuela never got even close to socialism due to frankly stupid economic policy, their share of private enterprise was even higher than France, and Cuba while much better than comparable South American states is not modern, nor an experiment.

</comment></comment></comment></comment><comment><USER9> If you find this interesting, the person you should be following is Yanis Varouvakis.

[https://en.m.wikipedia.org/wiki/Yanis_Varoufakis](https://en.m.wikipedia.org/wiki/Yanis_Varoufakis)

</comment><comment><USER10> It's not Capitalism that is failing. It's Democracy and Government. We can't effectively identify problems and pass laws that fix them.

Our society is like a runaway train and we can try and apply the brakes here and there, but everyday people don't have the power to say, "This is wrong, lets do something different."

If it was just a case of rich getting richer and poor getting poorer then, yes perhaps the problem is Capitalism, but there are a lot of other problems in society that we seem unable to fix.

Police brutality, Medical Insurance, College Loans. Guns. The worlds on fire and we are not moving quick enough to reduce CO2 emissions.

update: To clarify, what I was trying to say is that, if we had a functioning Democracy, we would identify that there is a problem where some people are very rich, and some very poor, and we would be able to make new laws that solve the problem. (Like taxing rich people for example.)

<comment><USER11> You had me at 100% agreement until you said "mussel" :). But you hit the nail on the head when you said "runaway train", and no one is in control. These problems definitely exist in other western counties, but there is something about US society that seems to amplify the process massively, where European counties for example seem to apply a dampening effect on the worst extremes.

<comment><USER10> haha, sorry. I deleted that line. Now it sounds like there was some seafood angle in my first draft.

</comment></comment></comment><comment><USER12> Capitalism seems like it could do an OK job if we priced in more externalities like resource extraction (e.g. fee for taking oil out of the ground) and pollution.

<comment><USER6> That would not fix the issues of the tendency for the rate of profit to fall, exploitation, and so on. But it would be an improvement.

</comment></comment><comment><USER13> It's never _not been time_ to talk about post-Capitalism. (America is not a political state like the former USSR). It's just always been a _short discussion_.

</comment><comment><USER14> What about this alternative:

Capitalism where you can't amass more personal wealth than $100M.

The only reason I would think you would need more money than that is if you wanted to push ideas that the population would not vote for.

I have much more faith in people closer to issues solving them with more good faith and efficiency than unelected billionaires being effectively philanthropic.

I would rather see more rich people relaxing with their wealth than trying to outmaneuver every last civil liberty we have for an extra dollar. Even better if they use their wealth to make their local community better. That way, there are actual human interactions between those making suggestions and voters.

Makes a good environment for natural competition as well.

Would we have a worse quality of life because of this? Important question, but my gut says no.

I'm all for civil liberties, but uber rich people can get away with weird things because they can ignore all the "no's" and essentially bribe decision makers.

<comment><USER15> What if you started a company, grew it and went public and in the ensuing years the markets pushed its value up such that your share was suddenly, and temporarily, worth more than $100m? Are you forced to sell the excess and be forced to pay a capital gains tax and reduce your control over your company?

<comment><USER14> I think the increased competition would limit huge valuations, but maybe any overflow could go to other employees. Beyond that though, it's hard to see what would make the most sense. My guess is that we would have a system different than owning stock in a random company. There's probably no reason we need to be tied to what we currently do.</comment></comment></comment>


Think Crosswords meets Scrabble meets Angry Birds - <AUTHOR> http://www.uspeaklanguages.com/signup/

<comment><AUTHOR> Play and discover your way to fluency. Our unique language learning applications have had a lot of love and thought put into them - making the product something really rather special.</comment>


GCHQ: the web is a terrrorist's command-and-control network - <AUTHOR> http://www.ft.com/cms/s/2/c89b6c58-6342-11e4-8a63-00144feabdc0.html

<comment><USER2> Ironically, so is GCHQ's, if not more so. It's interesting to watch all of these statements from those empowering the surveillance states. It's almost like they use Newspeak on purpose. Truth is the enemy, the Internet is a terrorist tool!</comment>


Updating our approach to misleading information - <AUTHOR> https://blog.twitter.com/en_us/topics/product/2020/updating-our-approach-to-misleading-information.html <comment><USER3> We are one step closer to implementing the long awaited social credit system in the USA. Yay I can't wait!

</comment><comment><USER4> Thank you Nick Pickles and Yoel Roth for helping Twitter join the scientific research process. Where will your findings on what is/isn't true be published?

</comment><comment><USER5> This gets funnier everyday, the hamfisted editorializing. If you want to capitalize on user generated content, while also being shielded from the legal consequences of user generated content, you can't insert editor's notes.

<comment><USER6> That is utterly wrong. The dichotomy doesn't exist and is not anything "new" but the exact intended function of Section 230.

<comment><USER5> > ..exact intended function of Section 230.

Interesting, the deferral to the intent of the law's author - it kind of makes the whole argument about the second amendment seem silly. That would be a dangerous game to play in this case though, given the fact that 230 is a small part of the much larger product of the mid 90's moral panic - <USER2> Communications Decency Act. Courts have ruled pretty narrowly on the issue of editorializing and 230 protections, but in cases of defamation they've made it clear that so long as the edits don't alter the meaning - they're protected. At the same time, courts have made it clear that simply changing the context on another's work can be enough to constitute fair-use and effectively create a new work. Those aren't compatible ideas.

The purpose of these tags is to alter the underlying message, they even admit it - they wouldn't function as intended otherwise. I think the funniest example I've seen of this was a 1970s documentary about our imminent destruction by way of global cooling, narrated by Leonard Nimoy. Youtube's little wrong-thought tag helpfully informed me of our imminent destruction by way of global warming.

</comment></comment></comment><comment><USER7> Next logical step will be only accepting links to whitelisted domains.

</comment><comment><USER8> Okay, now how about not having your design be terrible?

<comment><USER9> Please don't do this here.

<comment><USER8> I'm just mad at Twitter, because they should know better _especially_ with their interface and how simple it was.

<comment><USER9> (I agree and use the Minimal Twitter extension.)</comment></comment></comment></comment>


How Artificial Intelligence Is Changing Science - <AUTHOR> https://www.quantamagazine.org/how-artificial-intelligence-is-changing-science-20190311/ <comment><USER2> The article states positive impacts on science, but there are also negative impacts on science. For instance, the hype of AI has caused a brain-drain on related fields (such as cognitive science or applied mathematics). AI research itself suffers from companies buying up the academic talent. And researchers slap AI (which is usually deep learning) on a decade-old problem, without any care for complexity/benchmarks, implementation/usage, and proper validation methods, just to get published or receive funding.

<comment><USER3> I'm really afraid that ML is mainly just going to become automated p-hacking, and bring about a dark age to much of science. In a publish or perish world, how can you compete with someone with enough budget to set a bunch of models looking for any specious correlations in data sets and publishing what comes out the other end? Like we'll still have great breakthroughs from the top of the field, but a lot of grunt work style studies are going to lose their ability to be trusted.

<comment><USER4> >ML is mainly just going to become automated p-hacking, and bring about a dark age to much of science

It is already happening. A lot of people who can contribute to actual Science are moving into AI/ML field for the money and the industry/media hype are reinforcing this. Everything is "Deep${NONSENSE}" nowadays whether it is relevant or not. As a beginner, when i started to learn NNs, i couldn't get past my initial hurdle on how to validate the results on actual real-world data. What Statistical metrics do i use to "know" that the blackbox is working correctly? What are the assumptions and limitations that i need to be aware of to understand and have faith in the output? Most people don't seem to know or care; it is "magic" to them. In a world awash with data, reckless application of NN models to any and every problem is only going to drown us in spurious results and muddying all Scientific endeavours.

</comment></comment></comment><comment><USER5> Here's my crackpot idea, in case anyone out there is willing and qualified to put in the hard work:

Start with a detailed model of the solar system. Make a million copies of it. In each copy, insert a planet in a random orbit, with random mass. Measure the orbits of everything, perturbed by the new planet.

Feed the measurements of everything, except the new planet, to an A.I., and have it estimate the position of the new planet. Give it feedback on how accurate is was. Repeat a million times. It should learn to pinpoint ninth planets in solar systems like ours, from the perturbations on orbits of known bodies.

Then feed it the real measurement history from the real Solar System. It should output the location of Planet Nine.

[https://en.wikipedia.org/wiki/Planet_Nine](https://en.wikipedia.org/wiki/Planet_Nine)

<comment><USER6> Orbital equations are straightforward and deterministic, I am wondering why an AI would be needed for this? You could solve explicitly.

<comment><USER7> Well, an unperturbed orbit is straightforward and deterministic. But three+ body problems (which I believe the original comment is describing) do not have closed form solutions and are general simulated: [https://en.wikipedia.org/wiki/Three- body_problem](https://en.wikipedia.org/wiki/Three-body_problem).

The reason planet nine is suspected to exist is due to the commonalities in the orbits of trans-neptunian objects. That is, there appears to be a large gravitational influence on TNOs that causes the distribution of their orbits to exhibit irregularities that don't make sense with only two factors influencing their orbits.

<comment><USER8> Indeed, and it's worth generalizing that the vast majority of contemporary problems do not yield themselves to closed form solutions. We are constantly finding numerical solutions.

If you went to college when I did, first of all, you're eligible to join the AARP, and second, the problems that you studied were overwhelmingly solved in closed form. This was true in math as well, both in college and at the K-12 level.

Now, are ML algorithms worth adding to our tool belt of numerical methods? Oh, probably. When the fad is over, some useful applications will remain. We already use regression a lot, and that's a primitive form of ML.

</comment></comment></comment></comment><comment><USER9> The negative comments here are disappointing. ML is a fantastic tool for science, where it can propose a model that works as a starting point for getting to a model that works AND that you can understand.

This is quite common in physics, for example, where people are happy to build elaborate experiments just to poke at the universe in weird ways. An ML algorithm is a theorist's particle accelerator where they can treat it as something to be explored to gain insight.

The reason people are pissed about this is that we're doing this breadth- first, because the incentives make it that way. People are right to be concerned if we never get back to deeper analyses, but I'm not at all concerned.

At some point the low hanging fruit will be gone and every scientific community will be better off having these new results. As we get better at probing the black box, and we will because there's a lot of value behind doing so, we will start to shift back to the deeper questions.

<comment><USER10> It can't really propose a model that is understandable by most human means but I agree that it can find new relations that we can explore.

</comment></comment><comment><USER11> Very odd and may I say wrong article. The basic about AI provide a breakthrough is ok. But I would not call newton law as simulation.

The real development is verbal non-maths theory. Not simulation. In fact this kind of theory go first. If Aristotle etc. said ... sun must be revolving about earth. Some basic maths (geometry and algebra).

Observation is the second approach. And a breakthrough. Kepler and later Galilei watching juipter's Moon.

Then maths as a tool. Not just simple verbal theory. Calculus, non-Euclid geometry, wave mechanics, ... to these days physics is nothing but maths like. The sad thing about social science is only data. Only economic has some maths. Still data and verbal theory.

Then computer provide data analysis tool as well as simulation and visualisation. This is an aid more.

Then data as a tool. And the new breakthrough is AI helping to suggest models.

Theory, Mathematical model, Data observation (to disprove, to hint, to post question and to generate theory based on pattern)

then Various tool to assist above including AI.

</comment><comment><USER12> Ever more algorithms and models and data, ever less understanding and scientific theories.

Soon, instead of "theory of gravity" we'll have "generative DNN of science papers and grant writing" that no one will understand, but can generate papers that pass peer review and earn grants and pull in all the monies, effectively monopolizing and halting all government funded scientific progress.

Meanwhile, actual science will continue on in the amateur ranks, from which has always come the true breakthroughs.

<comment><USER13> [[citation needed]]

What has been an amateur science breakthrough in the last century or two which didn't have at it's base some billions of dollars of government funding.

<comment><USER14> The Special Theory of Relativity, from some moonlighting patent clerk.

<comment><USER15> That was over a century ago (although within the "century or two" limit specified). But it's not at all characteristic of the majority of scientific discoveries made in the last century, which were largely made by professional scientists and grad students on their way to becoming professionals. The list of Nobel Prizes in various sciences over the past century I think demonstrates this.

<comment><USER10> Maybe government money has squeezed out innovation a la Einstein.

</comment></comment></comment></comment></comment><comment><USER16> In my field of science the only thing that changed is that people started calling algorithms ai. For marketing purposes.

</comment><comment><USER17> Can we use this to make an app and figure out the optimal diets for everyone? Or a GAN for generating the funniest jokes?

<comment><USER18> > Or a GAN for generating the funniest jokes?

You _really_ want to be careful with that, Monty Python made an excellent documentary about the weaponization of such high grades of humor and the results, to put it mildly, weren't funny.

<comment><USER17> Sorry what? Can you offer us a link?

<comment> D-Coder More than you want to know: [https://en.wikipedia.org/wiki/The_Funniest_Joke_in_the_World](https://en.wikipedia.org/wiki/The_Funniest_Joke_in_the_World)

Just the right amount: [https://www.youtube.com/watch?v=_yo9WHrTvks](https://www.youtube.com/watch?v=_yo9WHrTvks)

</comment></comment></comment></comment><comment><USER19> AI is only as good as the programmer who created it. We are really in the stone age when it comes to AI.</comment>


Harvard Scientist Proposes Refreezing Arctic to Prevent Global-Warming Disaster - <AUTHOR> http://mashable.com/2012/12/22/arctic-global-warming/

<comment><USER2> As distasteful as geo-engineering might be (and I see many comments here to that effect), it is probably the only politically palatable way to actually do something about the problem. The country with the world's largest CO2 emissions, China [1], has a political class that owes its legitimacy to continued economic growth. Other countries, hard hit by the recent economic crisis, are finding it difficult to confront the issue as well (e.g. witness Barack Obama's four years of near-silence about it). Yes, these proposals are terribly risky, but they deserve well-thought replies that highlight the actual risks and difficulties instead of high-handed dismissal on grounds of "geo-engineering". There are other fields of engineering where safety is paramount, and perhaps the concepts can be brought over. Having no options is certainly no guarantee of safety.

[1] [http://en.wikipedia.org/wiki/List_of_countries_by_carbon_dio...](http://en.wikipedia.org/wiki/List_of_countries_by_carbon_dioxide_emissions)

</comment><comment><USER3> I am afraid that one of these days geo-engineering ideas are going to be implemented. Wasn't there some billionaire who dumped on a massive scale iron in the Ocean just because he believes this is going to help climate change?

In my opinion it is absolutely crazy to implement "stuff" in a super complex system (in which humans fail over and over again to predict relatively "simple" occurrences such as earthquakes or the weather) and present a "solution" by pretending to have the slightest understanding of all the potential long and short term side effects.

<comment><USER4> But the climate models which we consider 95% accurate (and on which the scientific consensus surrounding AGW is based) say it will work.

If the models are inaccurate for predicting what might happen under a Pinatubo-like forcing, why do we believe they are accurate for predicting what would have happened absent anthropogenic carbon emissions? (Note that scenario also involved Pinatubo-like forcing.)

<comment><USER5> Perhaps different levels of confidence are desired for different sorts of actions.

</comment></comment></comment><comment><USER6> This reminds me of when we have deliberately or accidentally introduced a new species into an environment with very negative results. We then follow up by introducing a second species to attack the first species. We know how that has usually worked out. Just one example here: <http://en.wikipedia.org/wiki/Cane_toads_in_Australia>

</comment><comment><USER7> I don't understand how all the excitement about global warming fits in with basic economics.

If the Earth's sea level rose several meters in an instant, yes, it would be a worldwide catastrophe with massive loss of life. But I fail to understand how that's physically possible; such change would take years or decades.

This would give people time to adapt. If the ocean's risen and there's seawater in your house up to your ankles, you move; you don't stick around for three more years until it's over your head and you drown.

Agriculture would certainly be affected by climate patterns. But how do we know that the outflow of farmland that stops being usable for farming due to climate change won't be balanced by an inflow of farmland that becomes suitable for farming due to climate change?

And even if agricultural output goes down: (1) New technology is invented all the time which improves agricultural output. Maybe by the time significant amounts of farmland become unfarmable, we'll be able to make due with less.

(2) Livestock takes a lot of land, because fundamentally, growing plants and eating them is much more efficient than growing plants, feeding them to animals, and then eating the animals. Vegetarians have been around a long time, and we know that it's possible for humans to live on plants alone. So if there's less farmland, why won't we just eat less meat as food becomes more expensive?

(3) Everyone needs to eat. iPod's are a luxury; food is a necessity. As food gets more expensive due to a scarcity of land and people start spending more of their income on food and less on iPod's, Apple will just close a couple factories and their land will be converted to farmland (funding for this will be available, because, again, if food is expensive, owning farmland will be very profitable.)

I fundamentally fail to understand why global warming deserves the same level of threat as (say) a bioengineered plague, a large nuclear war or an enormous asteroid on a collision course with Earth would.

<comment><USER8> As far as I can tell, the problem is that we really don't know about the long reaching effects of global warming, and you're not accounting for the extreme possibilities. There have been periods of time where the earth was extremely inhospitable to life. I'm not saying that will happen but to simply believe that there will be enough food without actually modeling possible outcomes is foolish.

</comment></comment><comment><USER9> I suppose that as good as this sounds, and I have heard about it many times [1][2][3], it boils down to opportunity cost. To me it will allow for a powerful counter argument when it comes to hitting emissions targets.

[1] [http://www2.eng.cam.ac.uk/~hemh/climate/Geoengineering_RoySo...](http://www2.eng.cam.ac.uk/~hemh/climate/Geoengineering_RoySoc.htm) [2] [http://aircrap.org/uk-climate-fix-technical-test-put-on- hold...](http://aircrap.org/uk-climate-fix-technical-test-put-on- hold-2/332429/) [3] <http://www.bbc.co.uk/news/science-environment-14936184>

</comment><comment><USER10> It is clear to me that crazy geo-engineering 'solutions' will pop up more often as we are passing one climate tipping point after the other. However, it is not like we are doing too less geo-engineering, we are doing too much for thousands of years. Agriculture, deforestation and industrialization have already left marks on the planet. So, why adding some more instead of substituting fossil energy with solar power? Just because of lack of political will and failure of leadership?

<comment><USER11> There is the interesting side effect that we'd have some measure of control over how much global warming we can handle. Instead of _stopping_ some action X, we could instead _amplify_ some geo-engineering Y.

It has a certain appeal. Not advocating for this, just pointing out the perspective.

<comment><USER10> Sure, pumping enough aerosols into the atmosphere will stop nearly instantly global warming. And it opens a path to burn even more carbon. But whom would you trust to run this solution over the next millions of years until all carbon is back in Earth's crust? The point is, if it stops working all hidden and accumulated warming will kick in virtually over night. And nobody wants the planet to warm by 4°C within days.

<comment><USER12> While I do not advocate to messing up with the atmosphere, your statement shows a clear lack of understanding. Carbon being trapped is NOT the same as heat being trapped in the earth's crust. The Earth could NOT warm by 4°C within days - that is ludicrous.

Here's the deal: the Earth is warmed by the Sun; it cools off by emitting back (glowing in the infra-red region if you wish) some of the heat. An equilibrium is reached when the amount of energy emitted is equal to the amount it receives. (Thus, Mars is cooler than the Earth which is cooler than Venus...). The amount of energy that can be emitted back into space depends on the property of the atmosphere. Some gas (so-called greenhouse gas) will prevent energy (in the form of infra-red radiation) from escaping, thus contributing to the rise of the temperature, until a new equilibrium is reached. (The proposed solution is to change the atmosphere properties so that less energy reaches the Earth in the first place.) We are currently seeing such a heating taking place, and it is not happening "within days".

<comment><USER10> You did not get the point. Currently we are close to 400ppm/+0.8°C without geo-engineering. If we use geo-engineering, keep temperature at current level and continue emitting CO2 to let's say 1000ppm, we can't stop because the greenhouse effect of these extra 600ppm will kick in as fast as the reducing effect of the engineering slows down.

But you are right: This is not comparable to today's situation where concentration rises by 1-2 ppm y⁻¹, but it is comparable to injecting instantly the equivalent of 600ppm into the atmosphere. It's like the opposite of a huge volcano eruption.

So to make a long story short, any geo-engineering solution which does not remove greenhouse gases from the atmosphere and turn them into a stable chemical compound is nothing but a time bomb - exploding at failure.

<comment><USER12> Yes, I did get the point, but I don't believe that you get the Physics. It takes a lot of energy to raise the average temperature of the earth by 4 C - much more than we are getting from the Sun "within a few days". Stopping (most) of the Sun's light to reach the Earth would be an effective (if not crazy...) way of cooling down the Earth. Adding a huge quantity of greenhouse gases is not going to turn the atmosphere into a one-way trap for radiation. The physics involved in light being blocked by air-borne aerosol particles and that of the trapping of heat by greenhouse gases are different, and their effect, while going in opposite direction, are not the exact inverse one of another.

<comment><USER10> Would you prefer to read 'weeks'?

<comment><USER12> No, I would prefer at least "years", but even better "decades". A few days (or weeks depending on your background) is the time required to read a few papers on the Physics of the atmosphere.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER13> I know this post is specifically about global warming, but I thought I'd talk about climate change and conservation in general.

We are at the point of no return for climate change in my opinion. I just don't see a global will to enact the kind of change which would be required to reverse this.

Tiger populations have decreased from an estimated 100,000 at the start of the 20th centry to around 3000. Lion populations have decreased 30-50% in the last few decades alone. ([1] and [2]). I fear that we will see these, and many more iconic species go extinct in our lifetime.

I'm sure this has been discussed before, but the ice-melt could precipitate a "Day after Tomorrow" kind of event, with the warm currents changing course [3].

[1] <http://en.wikipedia.org/wiki/Tiger#Conservation_efforts> [2] [http://en.wikipedia.org/wiki/Lion#Population_and_conservatio...](http://en.wikipedia.org/wiki/Lion#Population_and_conservation_status) [3] [http://science.nasa.gov/science-news/science-at- nasa/2004/05...](http://science.nasa.gov/science-news/science-at- nasa/2004/05mar_arctic/)

<comment><USER4> You do realize that Lion/Tiger populations have basically nothing to do with AGW, and have dropped due to hunting by humans and habitat destruction, right?

<comment><USER13> Yes, I do realize that, which is why I prefaced my comments with the disclaimer. The point I was trying to make was my opinion that environmental changes due to human activity is now at an irreversible point. The examples of the Lion and Tiger populations highlight that even with much increased awareness, the decline of these species is continuing. This is happening with global warming as well, and I don't know if there is a point at which the awareness actually translates into action.

</comment></comment></comment><comment><USER14> Expect to soon be reading about the loss of jobs for such proposals. Iceland plans to expand their port to handle Northwest Passage traffic. Governments and industry are planning to drill the Arctic seabed. Big business will fight attempts to reverse global warming.

</comment><comment><USER15> And how much will this "solution" (extraordinary evidence required) --cost-- ?? (On the face, apart from economic & natural costs.) Compared to (over- simplified outline) 1. TERMINATE beginning a relentless closing down of the biggest CO2 emitters? And 2. CONSERVE retrofitting human dwellings for far great energy efficiency? And 3. REPLACE building the offshore wind resources to offset terminated emitters?

Who will pay for the cost of such a program? Who will control it (and thus shoulder the responsibility)? What measures will additionally be needed to protect against the failure of the "solution"?

</comment><comment><USER16> Averting global warming by instigating a mild nuclear winter does not seem a reasonable course of action.

</comment><comment><USER17> Be cheaper to have a largish silverish baloon aligned to block the sun at altitude, also knowing how we do freezing currently it will on balance not add more warm air.

<comment><USER18> I did not do the math, but I think a huge, huge object would be needed in order to shadow even 1% of the Earth.

<comment><USER17> we are only talking about the artic here, and with altitude the effect increases as relative size of objects etc and shadows.

But it is a more saner and plausable idea than the article moots. Not that anybody seems to care.

</comment></comment></comment><comment><USER19> Suppose you block a percentage of sunlight from reaching the earth. The main reason this is a bad idea is that plantlife on this planet has evolved for a certain amount of light. Yeah, it'll cut down on melting. But what that do to ecologies all over the world? Yes, volcanoes can have this effect. But volcanoes of that scale don't happen very often.

</comment><comment><USER20> These kinds of solutions scare me because of their short-sightedness. I haven't seen any comforting explanations of how these reflective particles _leave_ the atmosphere and where they end up such that their effects can be stopped. Solar energy has been and will continue to be the only actual _source_ of energy on our planet. All other resources are finite and will eventually be depleted, many within our lifetimes or those of our children if reserve estimates are accurate. If our strategy for using finite energy resources is to block out the energy from our only infinite (at least millions of years worth) energy source, it would seem that we are shooting our grand children in order to have a bigger party today. Once the prices of oil and fissile materials get high enough, we'll be facepalming at a global scale when we have to rely on solar-derived power (biomass,PV,wind,etc) to fill the gap. Even if there are no unintended consequences (which I do NOT believe is the case), the intended consequence of reducing the total solar energy input to the planet seems like a simple case of short-term fix, long term problem.

<comment><USER4> _I haven't seen any comforting explanations of how these reflective particles leave the atmosphere and where they end up such that their effects can be stopped._

That's only because you haven't bothered to educate yourself. The answer is gravity brings the particles back down to the land/sea over the span of a few years.

We've actually tried this experiment a number of times (for example, Pinatubo in 1991, Krakatoa in 1883, various smaller eruptions as well) and on a scale more massive than most geoengineering proposals. It has worked as the models predicted and the long term effects have been negligable.</comment></comment>


Vote for We Carpool today to win Crowdfunder OC - <AUTHOR>

traffic getting worse? Irvine startup We Carpool helps neighbors to carpool to fun local events, work and school. We need your votes to win Crowdfunder OC competition in just 2 days! \n\n
 We are currently among the top 10 in the Crowdfunder OC competition, but we need your help to win the contest.<p>Supporting us is easy:
 1) Login to CrowdFunder, Signup in case you do not already have an account. Please make sure you are logged in as sometimes votes fail to register in their system.
 2) Click http://www.crowdfunder.com/crowdfundx/cfxoc/companies#company-948
 and vote for WeCarpool. We have a dark green logo.
 3) Confirm that the vote button changed to "Vote Tallied". Please click it again if it did not.
 Help us win this competition and share this message with your friends and on Facebook. <comment><AUTHOR> [http://www.crowdfunder.com/crowdfundx/cfxoc/companies#compan...](http://www.crowdfunder.com/crowdfundx/cfxoc/companies#company-948)</comment>


A message to my doomed colleagues in the American media - <AUTHOR> https://medium.com/@alexey__kovalev/message-to-american-media-from-russia-6e2e76eeae77 <comment><USER2> Well, at least for now, they're going to try the opposite approach and see what happens. An open letter to Trump from the US Press Corps:

[http://www.cjr.org/covering_trump/trump_white_house_press_co...](http://www.cjr.org/covering_trump/trump_white_house_press_corps.php)

 _" We're going to work together. You have tried to divide us and use reporters' deep competitive streaks to cause family fights. Those days are ending. We now recognize that the challenge of covering you requires that we cooperate and help one another whenever possible. So, when you shout down or ignore a reporter at a press conference who has said something you don't like, you're going to face a unified front."_

</comment><comment><USER3> I remember W. played a similar game. He would limit access to the press, which would cost the news agencies money because they couldn't cover the man. To get access, they had to have a history of printing favorable articles about the administration.

The press will always complain about this, but apparently in W's case, it was more egregious.

[http://www.nytimes.com/2002/10/14/business/white-house- keeps...](http://www.nytimes.com/2002/10/14/business/white-house-keeps-a-grip- on-its-news.html)

It was pretty obvious Trump is even less tactical about it than W though when he would refuse to answer the yelling guy's question because he disliked that they pushed the Trump dossier.

<comment><USER4> Who's W? George W Bush?

<comment><USER3> Yes.</comment></comment></comment>