Hollywood gets its own open-source foundation - <AUTHOR> https://techcrunch.com/2018/08/10/hollywood-gets-its-own-open-source-foundation/ <comment><AUTHOR> Having interned at DreamWorks, it was amazing to see how much the studio depends on Open Source software... Right from having everyone use Redhat systems across the board to massive use of Python for a lot of the pipelining and processes and internal tools, DW is a strong company when it comes to Open source usage (contributions as with most companies can be improved quite a bit).

Strongly welcome this venture and hope the industry as a whole can move forward because of it.:)

<comment><USER2> Very cool.

Were there any projects outside of the VFX Platform (vfxplatform.com) you could point out?

As mentioned in this article, it seems that the VFX studios, even the small ones, are very siloed and protective. Of the studios with Github accounts, the software they release is usually stuff they don't use any longer.

<comment><USER3> [https://research.dreamworks.com/open-source- projects/](https://research.dreamworks.com/open-source-projects/)

In particular look at OpenVDB (volume database). It is a wide and shallow volume format designed for performance around certain use cases. It also includes good lib support for the common simulation functions.

A lot of the other open source offerings from the animation/vfx studios are about common formats and pipelines and may be of less interest outside of the industry.

<comment><USER2> I think you're right. I should have specified formats and pipeline components. There are business reasons to not open source these things, but it couldn't hurt to ask.

OpenVDB was a crucial step. It's hard to live without it these days.

</comment></comment></comment></comment><comment><USER4> Hollywood wants to make sure they don't get stung by the draconian copyright laws they create, lobby, and pass.

Oppose them by

* boycotting hollywood movies

* making copyright liberalization a topic for your elected officials

* creating and distributing art under open licenses

<comment><USER5> How would you finance movies made under an open license? Movies are expensive to produce.

<comment><USER6> Crowd funding is one example. You're not going to crowd fund a $200m blockbuster but I don't see why we need to have those kinds of movies. Copyright and intellectual property laws benefit a small minority of people at the expense of the masses, and strike me as unfair. If a $200m movie is only feasible with draconian copyright laws, perhaps such a movie is unnecessary? I'd rather have the freedom to remix art and benefit from others doing so than live subject to surveillance just so the state can be sure I'm not copying movies that can be copied at no loss to others.

I don't make movies but I have been working nights and weekends for nine months on a robot design [1] that is CC0 licensed, which means no copyright public domain.

The work can be a reward on its own, and work done by this kind of passion has a character very different from a massively for profit movie. Personally I'd enjoy the art more if we had no copyright. Then only those passionate about the art would make it, and people would fund what they enjoy. I'd vastly prefer that system.

[1] [https://youtu.be/DXPmqCd0r04](https://youtu.be/DXPmqCd0r04)

<comment><USER5> May I ask how you pay your rent and buy food?

<comment><USER6> I work a full time job.

<comment><USER5> Making professional movies _is_ a full time job.

<comment><USER6> Sure, and one whose economics could be supported without made up rules about how to exchange information that can be copied for free. That is what I claim.

In other words, I believe artists and drug manufacturers alike would still find ways to fund important work even without intellectual property law or secrecy of information.

<comment><USER5> Yeah movies would be reduced to commercials. Commercials are free after all, so that proves it is possible. So we would probably still have movies like Transformers and Star Wars, so I guess geeks would be happy.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> Based on having attended a presentation on this at FMX this year, the main practical goal at the moment seems to be to help maintain low-level open source tools everyone relies on, like OpenEXR. It didn't sound like disrupting the software ecosystem by funding open-source alternatives to current "industry standard" tools was on the agenda, just making sure what is already in use continues to work.

Overall, I got the impression this was a serious undertaking with a pragmatic approach to solving problems for the VFX industry, so here's hoping it works out.

</comment><comment><USER8> >The founding members include a number of high-powered media and tech companies, including Animal Logic, Blue Sky Studios, Cisco, DreamWorks, Epic Games, Google, Intel, SideFX, Walt Disney Studios and Weta Digital.

Having worked for one of these companies I'm a bit surprised to see ILM's absence from the list. We were pretty big users of their OpenEXR image format.

<comment><USER9> ILM created OpenEXR.

<comment><USER10> Yes, that's what GP is saying.

</comment></comment></comment><comment> a-b ... with DRM & Hookers (c) Futurama

</comment><comment><USER11> Interesting. However, it was very confusing when reading a section which attempted to link Nick Cannon - CTO of Disney, but instead linked towards Nick (Scott) Cannon the enternainer.

<comment><USER12> TBH I thought they were one and the same when I read the article, but yeah they're different people.

</comment></comment><comment><USER13> Looking at the founding members, I am surprised there are no GPU vendors; which Hollywood must make an extensive use of.

</comment><comment><USER14> Good chance for GIMP to finish GTK+3 transition.

</comment><comment><USER15> Should we expect movies released with open source av codecs anytime soon?

<comment><USER16> It sounds like the emerging standard will be to streame/distribute via the open-source AV1 codec... and then cover it in Widevine DRM.

Probably handled by the TPM they upsold you for security.

</comment></comment><comment><USER17> This looks like a pr stunt to try and put the mpaa in a better light after thy sued everyone.

<comment><USER9> Is the MPAA a part of this? It seems like visual effects and animation companies to me, who already rely heavily on the entire linux eco system.

<comment> matt-attack No, this was an effort by the Aacademy of Motion Pictures Arts and Sciences. Specifically, their Science and Technology Council.</comment></comment></comment>


Ask HN: Looking for beta testers. Can you help? - <AUTHOR> I've just launched a new business intelligence service. I need people from the target audience to provide feedback on our sales process and messaging (you won't be asked to buy or anything). We just want to be able to iron out the kinks in our sales process by getting feedback from people, as not everyone is vocal about how they feel.
Can you help?
Business Intelligence is what helps companies create really smart data driven marketing strategies. It's what companies like Amazon and American express have used for years but this is not accessible to the small business.
Dropbox used business intelligence to develop their viral marketing strategy that helped them grow past $90m.
I'll show you a live case study that we're working on right now. A start-up that was recently mentioned in Forbes magazine; is a prestigious Sales Expo award winner and currently attracting high profile investors.
You need to be the founder of a business or start-up with a product or service that serves a minimum of 50 customers a month either as new customers, existing customers or a combination of new and existing though.
If you're interested please contact me by arranging a quick 10-min demo using the link below. Thanks in advance.
https://calendly.com/khurammalik-me/quick-demo <comment><USER2> Curious as to what audience is the target audience.

<comment><AUTHOR> Any start-up or small to medium size business that has been operating for at least a year.</comment></comment>


Here's the Awful 146-Word "Essay" That Earned an A- for a UNC Jock - <AUTHOR> http://www.slate.com/blogs/moneybox/2014/03/27/the_unc_fake_class_scandal_athlete_got_an_a_for_a_one_paragraph_paper.html <comment><USER2> (2014)</comment>


Firefox phased deprecation of browser FTP from v77 - <AUTHOR> https://www.theregister.co.uk/2020/03/20/firefox_deprecates_ftp/ <comment><USER2> > is way behind Chrome

Idk man, I don't see this as a great thing. I like having access to FTP. I still use it sometimes.

They claim they're dropping support because its insecure but to me it seems weird that they continue to support insecure HTTP but not FTP?

Also, secure FTP is a thing. I wish they'd keep supporting FTP but I guess it's additional code to maintain that isn't used often...</comment>


Replica's AI voice demo using replicas of the world's most famous voices - <AUTHOR> https://www.youtube.com/watch?v=f4DgHI9J3U8 <comment><USER2> You can find out more by reading their Medium post: [https://medium.com/@replicastudios/introducing-replica- studi...](https://medium.com/@replicastudios/introducing-replica- studios-5afbb4455818)</comment>


Why your Referral System Won't Work - <AUTHOR> http://georgesaines.com/2010/07/21/why-your-referral-system-wont-work/ <comment><USER2> I think that there are a couple of factors you are omitting from your analysis.

First, if you take a look at Dave McClure's AARRR model (and if you haven't, you really should [1]), you realize that Referral is something that you only want to attempt after you've got ridiculously satisfied customers. You can't expect people to spontaneously recommend your product on to others until it is absolutely friction-free, and serving that sweet spot so sweetly that they almost can't help it.

Second: not all niches are created equal. Are folks whose itch is scratched by your particular product significantly more likely to be in touch with others having the same itch? For some niches, yes. For others, no.

Finally, in the absence of a Freemium model, you definitely need to have a well-organized Activation process; some way for people to get engaged with the product prior to a commitment.

[1] [http://500hats.typepad.com/500blogs/2007/09/startup- metrics....](http://500hats.typepad.com/500blogs/2007/09/startup- metrics.html), for example.

<comment><AUTHOR> Hey Michael, I've actually not heard of McClure's AARRR model, I'll check that out. I've read read some of McClure's other articles, but not that one!

Your first comment about the referral only working in the sweet spot probably did kill it for us. We still aren't at the sweet spot, and the process of engaging with the product definitely isn't frictionless like it is for dropbox.

</comment></comment><comment><USER3> My good friend recently made me signup w/dropbox so he can get more space, and I'll probably do the same with my friends.

The main reason why it works is because we're helping each other achieve something; we're making an existing process easier/more convenient. It's a MUTUALLY BENEFICIAL activity (ie, it's easy for us to share files with each other).

Dropbox is a platform product like Facebook (ie, join this site, and we can keep in touch together easier). This strategy would work for all platform products.

As for your startup, you have to ask why would I would recommend someone a product that eases remembering japanese/chinese chars, there's no process after that at all. There's nothing MUTUAL BENEFICIAL here at all. If my friend is using your tool, why do I care? It doesn't help me at all.

<comment><AUTHOR> Very true, jscore, referring people to Skritter has no network effect at all. We do offer extra subscription time to both the recommending party and the new referral, and we spent a long time trying to figure out the right mix of benefits for both, but I think you're right that without the networking effect there's a lot less interest to recommend people.

As you say, it doesn't help you to recommend anyone to a site where the primary activity is independent learning. Now, we could alter that and some social features, but my best guess is that it's not just "social features," because tacking that stuff onto Skritter probably wouldn't help too much! To get maximum gain, we would have needed Skritter to be more easily spreadable from the get-go. :)

</comment></comment><comment><USER4> Just an FYI -- your page hung on loading "Waiting for dev.visualwebsiteoptimizer.com..." for at least a good 60-75 seconds.

<comment><AUTHOR> Hmmm, that's no good. I'll have to check that. Thanks for the tip.

</comment></comment><comment><USER5> Just to chime in, since I discussed referral systems with these guys and on my blog: mine has similarly been an utter failure, netting twenty trials and no sales. Five bucks in AdWords would work better. YMMobviouslyV

<comment><AUTHOR> Thanks for the comment patio! Our conversation about referral programs is what actually prompted me to write this. If it had just been us, I would have assumed that we were just doing it wrong, but when someone with a lot more experience and skill tried it with similar results I thought "huh, I wonder if there isn't a pattern there."

</comment></comment><comment><USER6> I think the analysis here is close but perhaps _slightly_ off.

1\. Your company has to have freemium pricing. - This is a factor that affects conversion rate. It's not so much about freemium as it is about friction. I think the idea here is that you have to have a more friction free offering for the referral, specifically the offer that the person being referred sees. If they get offered something a little extra than what they could get on their own it should be more appealing.

2\. Your product has to have wide appeal. - I think what you really mean here is that your customers need to have people they can make referrals to. This is obviously a big factor in the number of referrals made. Who are your customers? A businessman learning Japanese on his own might just not know anyone that would use the product other than himself. Students might know lots of other students who might be interested - which are you dealing with?

I also like the idea tomjen3 mentioned about incentives for both parties. This helps encourage people to make the referral, do it through the system , and helps the person being referred feel some form of obligation to the referrer and see that they can get something out of it.

I'm not saying that "you're doing it wrong" or that if you make some changes it will work, just that maybe there are some deeper reasons as to why it won't work in your situation.

<comment><AUTHOR> These are good points ryanelkins, especially about the distinction between "having wide appeal" and having customers that know other customers in the market. This was definitely a problem we faced early on. Essentially we thought we'd be selling to students and instead we've done most of our business with non-students.

Regarding the rewards, we did offer both the customer and the person being referred free time on the site as a reward, but either we didn't do a good job making that clear (very possible) or it wasn't compelling enough. To be honest, we're still not sure which it was.

</comment></comment><comment><USER7> I definitely agree on #2, that your product needs wide appeal, but #1 (fremium) seems not to be the case.

You can find plenty of places out there that do a refer-a-friend in exchange for the referrer receiving something in return. My gym (24 hour fitness) does this and I have referred people to it before.

I suspect in this case the problem mostly lies in the product being too niche, not that it costs money.

<comment><USER3> I don't buy the theory that the product needs wide appeal at all (#2).

Gym referrals work because it's mutually beneficial, "hey, come join 24h fitness, so we can workout together". I win, and my friend wins.

It's much easier to go the gym with a partner for motivation, etc. than to go by yourself.

Now, let's take travel, which is also not niche. If I'm lonely planet trying to sell travel books, referral won't work because it's not mutually beneficial for both of us to have the same book, etc.

Good discussion going on. I'm actually brainstorming on a referral strategy for one of my sites (travel based).

<comment><USER7> Mutual benefit certainly helps. I just wonder if you have to have it.

The people I referred I actually never work out with. It was just a trivial way to get $25 or whatever the bonus was at the time.

I believe Zecco (brokerage) also has a referral system going. Again, using it is beneficial monetarily - I could care less though if my friends use it.

One thing that is probably true: If there is no 'network effect', you probably need a much larger reward to get referrals.

</comment></comment></comment><comment><USER8> I also think that your main page is a lil visually busy, which if we're contrasting against Dropbox, is a major difference that made me less inclined to proceed with the free tour. Also, at the bottom of the page you introduce the founders, which I think is better left til after you've proved the product worth the attention in the first place.

<comment><AUTHOR> Yeah, Dropbox has a wicked simple homepage, I love it. I'm actually multivariate testing almost everything on that homepage, so it's possible you got the variant with the animation (would make it a lot more busy).

Substituting out the founders bit is something we've played with, but the previous test didn't get conclusive results one way or the other, so I need to keep trying different things with that space.

</comment></comment><comment><USER9> I think affiliate marketing that you mentioned can be really undervalued to. Sure the business has a shady underside but on the good side of affiliate marketing your paying for results only and hopefully attracting people who spend most of their time marketing to see if they can sell your product.

</comment><comment><USER10> Also you forget point 3:

It has to have incentives for _both_ parties. I specifically didn't send my mother an invite to dropbox because I didn't want to seem to do it so I could gain something at her expense _.

_ Dropbox actually does give something to both sides, but if I miss that how many others do you think does too?

<comment><USER11> I've recommended Dropbox to several people, but have only sent invites to those I think wouldn't mind me getting something in return for the referral (and even to those I've told about the extra space we'd get and that they could of course sign up without my link).

</comment></comment><comment><USER12> The reason Dropbox works is simple: it's a carrot on a stick. Most invites don't have that and provide no incentive, either for the inviter or the invitee.</comment>


Ask HN: Moving to the Bay Area as a Student - <AUTHOR>

I've wanted to move to the Bay Area for some time now and I think I'm finally at a point where moving is a viable option. I'm planning on transferring to a CCC for Fall 2012 to finish my basic studies and then transfer to a UC school. My reason for wanting to finish my basic studies at a CCC is that, while at the CCC, I will meet the requirements to reclassify as an in-state student for tuition purposes, drastically lowering the cost of attending any of the UC schools.
The biggest problem I face is money. One of the requirements for reclassifying as a California resident is that I have to prove financial independence. This means that I pretty much have to make it on my own for the next two years.
California isn't a cheap place to live, especially when I could end up working at Burger King making minimum wage. I was hoping there would be someone here who could give me some insight into whether it is possible and what it would require for me to support myself. I should qualify for a Pell Grant which will pay most of my tuition and I can use a series of scholarships/loans to pay the rest of my fees. I am already $30,000 in debt and I'm still looking at another 4 years of school because a lot of my credits won't transfer, so I don't want any more loans if I can avoid them.
Where would be the cheapest place to live? I'd like to be within an hour's drive of downtown San Francisco so I can throw myself into the startup scene and, maybe, find a good job.
How many hours a week would I need to work to support myself, assuming I was making minimum wage or slightly above? I'm willing to work full time if that's what it takes, but I'll be in school full time as well.
Any related advice/comments are definitely appreciated. If moving out West on my own is a silly idea, please, do tell me. I'm just a kid, what do I know? :) <comment><USER2> I think you'll find it difficult working at Burger King. I recommend learning something that is employable immediately in the start up scene. That could be as non specific as marketing/sales or as specific as let's say Ruby on Rails or Python.

Take out large school loans if you have to (to avoid doing work that has no learning curve: Burger King) and subsidize your learning for real employable skills.

HOpe that helps friend

<comment><AUTHOR> The Burger King comment was a joke. I hope I wouldn't have to settle for that. (Not that there's anything wrong with working in fast food)

I'm working on a project now (stack includes Python, Bootstrap, postgresql eventually) that I hope will help me get an internship for the summer to help pay for next year.

</comment></comment><comment><USER3> At minimum wage or slightly above, you would need to work about 60 hours a week to get by. You should make it a priority to find roommates to split costs.

California community colleges charge upwards of $500 a unit for out-of-state students. Find out what your preferred school charges and factor that into your plans.

The cheapest place to live in the Bay Area might be Vallejo, or possibly some run-down areas of the East Bay. However, moving there would put you out of daily driving range of most of the Bay Area's software companies which are on the south end of the peninsula in the Menlo Park area, or in San Jose.

<comment><AUTHOR> I figured I'd need roommates to make things work. What I didn't think of was how far Berkeley is from Menlo Park/San Jose. Seeing as I may not have a car for a while, I might have to consider moving to San Jose.</comment></comment>


An HTTP reverse proxy for realtime - <AUTHOR> http://blog.fanout.io/2013/04/09/an-http-reverse-proxy-for-realtime/ <comment><USER2> This is awesome! I released something exactly like it a few years ago, hookbox (MIT licensed): [https://github.com/hookbox/hookbox/blob/master/docs/source/i...](https://github.com/hookbox/hookbox/blob/master/docs/source/intro.rst)

Basic idea is this: You put all your real-time stuff in a message queue (MQ) which communicates directly with the browser. For authentication / authorization and various other forms of permission / logging, you have the MQ communicate with the web framework via http callbacks (Webhooks) and a standard REST API. So the architecture is:

User <\--Websocket--> MQ :: publish/subscribe

MQ --Webhooks--> PHP/Django/Servlets/etc. :: user signed on, user joined a channel, etc.

PHP --REST--> MQ :: publish(msg), remove(user, channel), etc.

The key is to include cookie information in the callbacks from MQ -> PHP so the callback happens in the context of the user session. Suddenly you can do things like write a chat app in 30 lines of php + js, or a persistent time series in 20, and it really feels magical.

I actually started Hookbox almost as a statement of irony, because I was really frustrated about the major pushback I was getting to sockets in web browsers at the time. I'd just finished writing/submitting the initial proposal for Websocket, and I wrote this tongue-incheeck piece about the mismatch between typical web development and network server programming: [http://svwebbuilder.wordpress.com/2008/10/20/html5-websocket...](http://svwebbuilder.wordpress.com/2008/10/20/html5-websocket- and-webjneering/)

So Hookbox started as a 2-3 day project that took on a life of its own for a while and ended up being really useful. This project was one of my smaller open source codebases and to this day I receive tons of interest and requests for maintenance, though I've abandoned it for years due to time.

I'm sure there's a huge market for this sort of thing. It's great to see Pushpin, I'll definitely check it out!

</comment><comment><USER3> This is a great product, thanks for building it.

In a previous project that I had worked on, we were spawning up complex pieces of infrastructure using Chef integrated with fronting Rails app. Realtime updates were always tough to orchestrate with custom Rabbit MQ feedback from the Chef clients pushing out to Rails clients using JS.

I believe a solution like this one would come in very handy for pushing out realtime updates for long running infrastructure requests from a distributed system. Kudos!

</comment><comment><USER4> Nice! We're working on something similar in Ruby EM at <http://firehose.io/> if anybody is interested.

Glad to see more streaming REST implementations coming alive!

<comment><USER5> Brad, firehose looks cool too. Are you guys using it in production? (Should I be worried that the Travis build is failing?)

<comment><USER4> Yep! We pump a lot of messages through this per day. Not sure why Travis is failing, probably a recent em-hiredis bump. I'll check it out.

</comment></comment></comment><comment><USER6> I started to feel like i am falling behind with all these interesting libs, servers, frameworks, protocols, languages, platforms recently.

Especially, if you are a student and trying to keep up with all these new stuff. It is becoming really hard to decide what to learn next or what to focus on.

<comment><USER7> Keep an eye on what's out there, surface level among all the things that are useful (or just interesting to you). Dive in when you have an actual problem to solve. Don't decide before you dive in, use that surface knowledge to know what your options are and where to start but don't make decisions with it (can be tricky).

And as others are saying, know your fundamentals, and know your theory. Get experience doing something real start to finish as often as possible and in order to do that you will have to dive into things. That's how to decide what to learn next.

<comment><USER8> Upvote.

At the startups that I've worked, things move quickly and unless you have a personal motivation to learn something, you can't keep up.

It's best to get a cursory view of your available options (star them, bookmark them, commit them to memory) and when the situation arises for a specific problem to solve, you'll know of a handful of options to further investigate.

</comment></comment></comment><comment><USER9> I've written memqueue (<https://github.com/halayli/memqueue/>) for similar purposes.

Memqueue is a revision-based queue server with a REST API. Multiple consumers can poll the same queue at a different pace by using revisions. A revision is sort of a cursor that allows a consumer to specify where to poll from in the queue. If a connection drops it's not a problem, you continue from the revision you stopped at after you reestablish a connection. Each time a new message arrives to the queue, the revision is bumped by one and consumers are expected to poll from the new revision.

It also allows you to specify message & queue expiries so you don't have to manage memory growth.

</comment><comment><USER10> I like seeing mongrel2 as part of this stack. It seems like the perfect fit for an architecture like this and I get the feeling that's it's heavily underused.

<comment><AUTHOR> Mongrel2's usage of ZeroMQ messages to manage low-level protocol was a great influence to Pushpin's design. In fact, one of the other components in the stack, Zurl, is basically the inverse of Mongrel2 (doing outbound HTTP instead of inbound). Love all these little worker components.

</comment></comment><comment><USER11> This looks really interesting. We've been using the nginx-push-stream-module to achieve something similar, but the public facing HTTP API design isn't very flexible, so something that lets you configure this explicitly would be great.

<https://github.com/wandenberg/nginx-push-stream-module>

<comment><AUTHOR> Thanks for the comment. At my last job, we were unable to migrate an existing API over to a certain realtime solution without breakage, and that was motivation to create something more versatile.

</comment></comment><comment><USER12> Personal peeve: There is still a name collision beetween the popular use of "real-time" to describe "live" updates (usually as the defining aspect being "without polling") versus the definition of "real-time", implying that events are delivered to the customer in a guaranteed period of time. Which is obviously not true.

<comment><USER13> Can you give an example of any 'true "real-time"' projects implementing any aspect of HTTP? I think this ship has sailed.

<comment><USER14> Yeah, for example, there are web services whose design goal is <30ms completed request so they can guarantee their client timely data based on a SLA. If this "real-time" project can't guarantee time of transactions, it's not "real- time".

<comment><USER15> How is this even possible? All it takes is a clogged router buffer 'somewhere' along your network path and that <30ms web request is gonna get blown away. Am I missing something here?

I'm probably just being ignorant lol. Can you point me to an example company offering a service like this?

Thanks.

<comment><USER14> I can't, but you could for example look at the financial services industry or providers of real-time data feeds.

In general these services are not available through the public internet so routers with clogged buffers are not usually an issue. But it depends on the SLA.

</comment></comment></comment></comment></comment><comment><USER16> I realize http is extremely well known and documented so it's relatively easy for backend communications but I've always wondered how efficient it actually is considering how old the standard is.

<comment><USER17> It's actually fairly good now that most of the improvements in 1.1 can be used with browser support like pipelining, compression, keep-alives and so on. However it isn't perfect SPDY is better still as it allows for things like muxing and pre-emptive resource downloading.

</comment></comment><comment><USER18> I'm glad you built this Justin. I have wanted this functionality many, many times in the social gaming world and never had the time/budget to build it and instead hacked something together. I love seeing how XMPP has influenced the design of very useful and much more pragmatic solutions to near-real-time communications.

</comment><comment><USER19> That's nice, but I have a question about the mechanism.

In the document I saw that Pushpin send response to client while it's waiting for the response from web application, right? How is it possible? I mean if you send a response to the client, you can't send anymore responses after that.

<comment><AUTHOR> If Pushpin is told by the web application to hold a request open, then it does not send anything to the client (at least not in the long-polling case. streaming works different but I've yet to write about it). So when the application publishes data at a later time, it is the first payload the client sees.

</comment></comment><comment><USER20> This is really cool. To make sure I understand, using it would replace the need to use services like pusher.com? Also, if I were using, say, Sinatra with nginx and unicorn, where would this sit in the pipeline?

<comment><AUTHOR> To be clear, this is software, so comparing it to Pusher.com is a bit of an apples to oranges comparison. Pushpin is more comparable to Socket.io, Juggernaut, Faye, etc.

What makes Pushpin special is that you can control what the outside-facing HTTP exchanges look like, which makes it good for implementing APIs, and may also be useful if you're just really anal about how your client/server interactions work. :)

The cost is that you need to design your protocol and write client code (the other solutions already have their own special protocols and come with corresponding JavaScript libraries ready to go). So whether or not Pushpin is good for you depends on what level of control you're after.

In the pipeline, Pushpin goes in the very front, just behind a load balancer (if any). The reason for this is you could put instances of Pushpin in different geographic locations, all fronting an application in a single location. So you want it the furthest out, closest to any users that might be connected to it.

<comment><USER20> Thanks!

</comment></comment></comment><comment><USER5> Cool! This sort of architecture looks great for our Rails API we're developing. I'd be curious to see benchmarks to know how many connections I can expect to be maintained by a single instance of pushpin.

</comment><comment><USER21> AFAICT, you can do the same with newer versions of HAProxy and nginx. Is there any advantage of using this over those well tested and proven alternatives?

<comment><AUTHOR> Nginx has the Push Stream Module, which is similar but not quite as versatile. I don't think you could implement the "incremental counter" API described in the Pushpin article, for example. Whether or not this matters to you comes down to how much control you need.

I'm not aware of such functionality in HAProxy but would love to hear about it (I'm an HAProxy fan :)).

<comment><USER21> While "incremental counter" is a cool idea, I fail to see how it really improves anything. You're just moving the same problem from one layer to another. Node.js does all Pushpin does and you can write "simple chat server in 30 lines of code" as well, without the need to set up another piece of software.

This look like band-aid for developers still stuck with Python, PHP, whatever... technology from 2000's. I see I've got a down-vote from fanboys already, but that's just the way HN works: it's hard to have a constructive coversation, but it's easy to hate.

P.S. @jkarneges, this reply was not directed at you, but whoever down-voted my simple question. I mean, how can you down-vote a question? Let's not question anything and spread love, a la Facebook "like-button-only" style. :(

<comment><AUTHOR> I've upvoted your original post to see if that helps.

The article does play up the compatibility with legacy frameworks. However, the proxy approach itself was designed independently of this, and the versatility turned out to be a bonus. Some background: [http://blog.fanout.io/2013/02/10/http-grip-proxy-hold- techni...](http://blog.fanout.io/2013/02/10/http-grip-proxy-hold-technique/)

Basically, I'm positing that as a system gets larger, then moving the problem to an outer layer is good design, even if all of your backend code is event- driven. You can use Pushpin and Node together with a straight face. :)

</comment></comment></comment></comment><comment><USER22> Awesome Justin,

finally something that every web developer understands immediately and really takes the pain out of realtime for a lot of people.

</comment><comment><USER16> mirror: [http://google.com/search?q=cache:http://blog.fanout.io/2013/...](http://google.com/search?q=cache:http://blog.fanout.io/2013/04/09/an- http-reverse-proxy-for-realtime/)

If you are using wordpress and not wp-super-cache, when a page becomes popular your server is going to have a bad time.

 _update: looks like they fixed it_

<comment><AUTHOR> Thanks for the tip! I've enabled wp-super-cache now, and the page seems to be working again.

</comment></comment><comment><USER23> Love the idea and clean API! Is this production quality?

Also, just curious - how come qt is required?

<comment><AUTHOR> Current state of code is "it probably works". We will be hardening it over the next few weeks though to get it production ready (will be deploying it on fanout.io, to replace our older code).

Qt, because... it's a nice C++ event-driven lib. :)

<comment><USER23> Thanks, can't wait to try it out.

Must look into Qt then sometime. :)

</comment></comment></comment><comment><USER24> Does each publish_http_response_async release all pending requests or just one?

<comment><AUTHOR> It releases all requests held on the specified channel.

</comment></comment><comment><USER25> Impressive!</comment>


SPRINT declaring war on Adderall - <AUTHOR> https://www.indiegogo.com/projects/sprint-declaring-war-on-adderall

<comment><USER2> So because they studied CS they're qualified to design drugs? I didn't see any testing/trials in their plans. I also think the drug they're fighting is probably Modafinil--not Adderall. And I don't care how the jar feels in my hand... fancy packaging is money wasted.

</comment><comment><AUTHOR> Anyone tried L-theanine + caffeine? How's the price of SPRINT compare to existing ways to take the same combo of nootropics?</comment>


Amazon cloud drive desktop app for windows and mac - <AUTHOR> http://www.amazon.com/gp/feature.html/?ie=UTF8&tag=thepartim-20&linkCode=ur2&docId=1000796781&camp=1789&creative=390957

<comment><USER2> The desktop app allows you to manage a few settings or buy more space; it doesn't allow you to store files without going to the cloud drive website. You also can't simply put things in a local folder, like dropbox.

It feels like amazon is both late to the party and underdressed at this point. The only place it competes is price -- its about half the price of dropbox unless you count the referral space @ dropbox then its not quite half.

In my opinion, the lack of features isn't worth dealing with to save a few dollars. I'm interested to see how fast they make it more dropbox-like, and if they go that direction at all.</comment>


CSV 1.1 – CSV Evolved (for Humans) - <AUTHOR> https://csv11.github.io/ <comment><USER2> There are a ton of problems here:

* From the documentation "No quotes needed for values ... Use dual quotes ... Use triple quotes"

* You haven't solved the problem of describing what the columns are. In fact, its worse because you are encouraging people to put units in the field

* Spaces matter! What if the data is literally " Word " vs "Word". This format makes them both the same.

* For some reason, the header row is removed. The only thing in a CSV that provides structure is gone. In the first "improved" example, I have no way of knowing what the columns meaning without them.

The only good improvement in this is the "#" convention as a comment. If we could get every CSV parser in future to agree (we already can't get them to agree now) to ignore lines that start with "#" then CSV's would be vastly improved.

<comment><USER3> Hello, I'm the original author of CSV 1.1.

There are no problems :-). You're making up things / problems.

Nowhere says it that the header row is removed. You CANNOT auto-detect a header, you have to tell your parser (see the csvreader library as an example - [https://github.com/csv11/csvreader](https://github.com/csv11/csvreader) \- from my humble self.) if you have a header or not (it's optional).

If Spaces matter! but them in quotes. By default you don't need quotes (and, thus, discouraged).

> You haven't solved the problem of describing > what the columns are

That is solved / done by a schema with a (tabular) datapackage, see [https://github.com/csv11/csvpack](https://github.com/csv11/csvpack) as a real-world example how that works in practice or use csvrecord, see [https://github.com/csv11/csvrecord](https://github.com/csv11/csvrecord)

<comment><USER2> > If Spaces matter! but them in quotes. By default you don't need quotes (and, > thus, discouraged).

So, just like a regular CSV?

</comment></comment></comment><comment><USER4> Well, if you want to improve tabular data formats:

1\. Add a version identifier / content-type on the first line!

2\. Create a formal grammar for this CSV format

3\. Specify preferred character-encoding

4\. Provide some tooling (validation, CSV 1.1 => HTML, CSV => Excel)

5\. Add the option to specify column type (string, int, date)

6\. Specify ISO-8601 as the preferred date format

7\. Allow 'reheading' the columns in the file itself. This is useful in streaming data.

8\. Specify the format of the newlines.

<comment><USER5> CSVW: CSV on the Web [https://w3c.github.io/csvw/](https://w3c.github.io/csvw/)

"CSV on the Web: A Primer" [http://www.w3.org/TR/tabular-data- primer/](http://www.w3.org/TR/tabular-data-primer/)

"Model for Tabular Data and Metadata on the Web" [http://www.w3.org/TR/tabular-data-model/](http://www.w3.org/TR/tabular-data- model/)

"Generating JSON from Tabular Data on the Web" (csv2json) [http://www.w3.org/TR/csv2json/](http://www.w3.org/TR/csv2json/)

"Generating RDF from Tabular Data on the Web" (csv2rdf) [http://www.w3.org/TR/csv2rdf/](http://www.w3.org/TR/csv2rdf/)

...

N. Allow authors to (1) specify how many header rows are metadata and (2) what each row is. For example: 7 metadata header rows: {column label, property URI [path], datatype URI, unit URI, accuracy, precision, significant figures}

With URIs, we can merge, join, and concatenate data (when e.g. study control URIs for e.g. single/double/triple blinding/masking indicate that the [https://schema.org/Dataset](https://schema.org/Dataset) meets meta-analysis inclusion criteria).

"#LinkedReproducibility"; "#LinkedMetaAnalyses"

</comment></comment><comment><USER6> What problem is this trying to solve?

CSV's problems are the nature of a very flexible convention. It's so simple that everyone writes their own generators and parsers that are slightly different. That's what happens when you use a convention like csv.

Revving the spec won't help anything... Because csv is the kind of convention where no one reads the spec anyway!

</comment><comment><USER7> I even don't see how it is better.

        ID,Name,Capital,Area,Tags
        bc,British Columbia,Victoria,922509,en|western canada

vs

        ###########################
        # Oh, Canada! 10 provinces and 3 territories
        #
        # see en.wikipedia.org/wiki/Provinces_and_territories_of_Canada
        #
        # note: key is two-letter canadian postal code
        #
        # for regions tags see
        #   en.wikipedia.org/wiki/List_of_regions_of_Canada

        bc, British Columbia,       Victoria,     922 509 km², en|western canada

In 1.1 I don't see what is what - what is `bc`? What is `en|western canada`? In vanilla CSV you can clearly see what is what because it shows you on top.

Why not just write a CSV formatter?

<comment><USER8> Right, where are the tag names specified? What if a new column is added, and my reader needs to handle both formats?

Also, seems like specification-by-example, unless I'm missing something.

<comment><USER9> To be honest, I'd be quite happy if CSV just had a 'frontmatter' block. Sometimes a bit of background info or description of what the data represents or its source is really useful.

Removing column headings would be a killer. Although efforts in the past to add type information are misguided, by that point you may as well just use XML.

</comment></comment></comment><comment><USER10> I can't see this gaining any traction. There's nothing wrong with CSV as it is for basic data transfer. CSV had been unchanged now for so long, that any attempt to update it will fail - there are simply too many CSV parsers already implemented.

If you are worried about people in the year 3000 understanding your data, add a .txt file alongside the CSV explaining the fields.

<comment><USER11> I agree. The biggest problem with CSV is Excel's terrible support.

And while CSV is primarily used for data transfer, Excel could hypothetically be a great debug-viewer if it didn't do brain dead stuff like unrecoverably corrupt UPCs and other long numbers by default.

There's an Excel Uservoice about this very issue from 2015:

[https://excel.uservoice.com/forums/304921-excel-for- windows-...](https://excel.uservoice.com/forums/304921-excel-for-windows- desktop-application/suggestions/10374741-stop-excel-from-changing-large- numbers-actually?page=8&per_page=20)

They're "looking into it." Just as they've been looking into it for the last twenty years. Any. Day. Now.

<comment><USER12> Excel's braindead csv corruption is why I never use it for csv now. Add guids to the list of things it can corrupt. (if the first group is numeric it parses as an expression which then causes an error because the next group has hex digits, what!?) Also any long string that happens to only be digits. And two numbers separated by a "/".

It's so bad I check any csv file I get if it's been opened in Excel, and if I find any evidence that it has I reject it and make whoever I got it from give me the original source, or transform it with literally _any other tool_.

</comment></comment></comment><comment><USER13> While it is nice to add some spacing and make things a bit easier to read, I don't really see the point in this.

If you want it to be easily parsable by a human then there are hundreds of applications designed for this. The most common being spreadsheets like Excel.

And what happens if you have a list of domains, but one contains a really long url (300 characters wide?). It will mess up the columns for every single row above and below it as well.

Also adding comments will completely break any existing software (I think most programs can handle some more whitespace, but you couldn't import the final example into Excel, I don't think (untested))

</comment><comment><USER14> For those looking for a modern take on improving on the CSV format, I'd recommend Frictionless Data's Datapackage specification[1] which basically consists of a json file of metadata that accompanies a CSV file that describes column types, versions, sources, and how to validate correctness of the CSV's data. This allows for quite a lot of tooling and workflow improvements to CSV files without mucking with the CSV itself

Another hack to improve CSV workflows is OCHA's HXL[2] that is used by humanitarian organizations. Basically adding a row of hashtags in addition to column names, which is surprisingly useful considering the ease of adding these to a file.

[1] [https://frictionlessdata.io/docs/tabular-data- package/](https://frictionlessdata.io/docs/tabular-data-package/) [2] [http://hxlstandard.org/](http://hxlstandard.org/)

<comment><USER15> One of the authors of the Frictionless Data specifications here. The spec directly relevant to CSV is Table Schema [1], and we've also got some nice tools that leverage Table Schema and the family of specifications, such as goodtables [2].

[1] [https://frictionlessdata.io/specs/table- schema/](https://frictionlessdata.io/specs/table-schema/) [2] [https://github.com/frictionlessdata/goodtables- py](https://github.com/frictionlessdata/goodtables-py)

<comment><USER16> This looks interesting and useful. It might be a feature that wouldn't get used a lot but I've always felt that it would be great to have a 'unit' field to specify the physical units of the values. Have you ever discussed adding this as an optional field to the specification?

<comment><USER15> We've looked into it:

[https://github.com/frictionlessdata/specs/issues/537](https://github.com/frictionlessdata/specs/issues/537)

Feel free to add your use cases to that issue.

</comment></comment></comment></comment><comment><USER17> I was extremely confused why the website nor the repository does not contain the specification. There is an informal specification littered in a separate repo [1], and the whole organization [2] is dedicated for the documentations and associated Ruby implementations. So well, good luck, but next time try to be more specific.

[1] [https://github.com/csv11/csv-next](https://github.com/csv11/csv-next)

[2] [https://github.com/csv11/](https://github.com/csv11/)

<comment><USER3> FYI: The readme (the website) is the specification :-), see [https://github.com/csv11/csv11.github.io](https://github.com/csv11/csv11.github.io)

PS: csv-next is NOT an informal specification - these are notes (collection of ideas).

PPS: How do I know? I'm the author of the website - I should know ;-).

<comment><USER17> Oh, hello! My gut feeling was that csv-next is _pointing_ to the informal specification (and they read like it, but specific to a single problem point).

I guess you might not have understood my complaint (my bad), so let me give some examples how the specification should look like.

An informal specification looks like this [1]. You should give examples and rules enough to use your format and sufficient to implement most of the things. You should give definitions for keywords that are specific to your specification (but you can omit common definitions). You should give some (but probably not all) ideas where the specification can go wrong: Unicode whitespaces, Byte Order Mark, platform-specific newlines, escape sequences, duplicate keys in the front matter (well, actually there is no provision how to put the front matter at all), numeric overflows, and so on.

A formal specification looks like this [2]. In addition to what an informal specification provides, you should give lots of examples and formalized rules (most frequently ABNF [3]) to implement all the things. There should be clear and reasonable error handling policies. The wording of the specification should be clear, unambiguous and preferably standardized (there are specific meanings to "MUST", "SHOULD" etc. [4]). The specification should be honest about its pros and cons. You should be explicit about the flexibility of the format: you should give a list of what can be extended or modified later and what can't.

I strongly suggest you to put an informal specification at the least, and to prepare for the eventual development of a formal specification by pondering about missing pieces (it is a daunting task, I know).

[1] [https://github.com/toml- lang/toml/blob/bb47759841ac368d86eb7...](https://github.com/toml- lang/toml/blob/bb47759841ac368d86eb7a459bd7eea7162b9a80/README.md#user- content-spec)

[2] [https://tools.ietf.org/html/rfc7049](https://tools.ietf.org/html/rfc7049)

[3] [https://en.wikipedia.org/wiki/Augmented_Backus%E2%80%93Naur_...](https://en.wikipedia.org/wiki/Augmented_Backus%E2%80%93Naur_form)

[4] [https://tools.ietf.org/html/bcp14](https://tools.ietf.org/html/bcp14)

<comment><USER3> I think you don't understand :-). A specification doesn't always have to be complicated, that's the point. I'm all for tech notes with all the details. Here's a challenge for you - write a csv parser - I know - it's a daunting task. For inspiration, here's an example from your humble self - [https://github.com/csv11/csvreader](https://github.com/csv11/csvreader)

<comment><USER17> > A specification doesn't always have to be complicated, that's the point.

That's right, but your "specification" was not even enough for writing a CSV 1.1 file. For example I have mentioned a front matter issue---there is no example using it, and I'm deeply confused how the front matter looks like (or even where it is). I'm seeing numeric units there, but I don't know how they are handled (or, say, what happens if different units like m² are there) from your page.

I'm not saying I'm a good specification writer (and I'm not even a native speaker of English), but I at least try. I have once written an informal specification [1] which should be almost enough to write valid files and start implementing parsers, without being too complicated. You can avoid a complicated specification without being too vague.

> For inspiration, here's an example from your humble self - > [https://github.com/csv11/csvreader](https://github.com/csv11/csvreader)

That's frankly much better (in terms of explicitness) than the front page, why not linking to it? :-)

But I believe that it is still not sufficient. Rather, I'm now unsure about the goal of your project: is it a set of Ruby libraries with fancy, modern but incompatible (by itself) CSV extensions? Or is it hopefully going to be a universally used format? If your intention was the former, then your front page should have been clear about it---it is not about a format but about a library. And if you are going to have a format, then my suggestions hold.

[1] [https://github.com/lifthrasiir/cson](https://github.com/lifthrasiir/cson) (with a bit of formal materials, just for the clarity)

<comment><USER3> The goal of the project is to improve CSV :-) - it's in the title e.g. CSV Evolved and in the version 1.1 (that is, it's not version 2.0). It's about not breaking things - it's about small changes (mostly) for humble humans (not machines).

It's NOT about one universal csv format and the ultimate specification to settle the matter until the end of history etc.

Thanks for the detailed suggestions. I see your points. I appreciate your helpfulness.

</comment></comment></comment></comment></comment></comment><comment><USER18> The best thing we have today for "CSV evolved" is JSON array of arrays, IMO. It's much better specified, everything agrees on how things should be quoted, it's perfectly human-readable if you do one array per line, its semantics follows straightforwardly from that of JSON, and we already have parsers for it that produce a reasonable output.

Is it slightly more verbose? Sure, but why does it matter? Having to quote all strings is really not a big deal.

<comment><USER3> Good point. See the CSVJSON format - [http://csvjson.org](http://csvjson.org) \- Love it. I will add a new variant called CSV <3 JSON shortly to the CSV 1.1 repo and csvreader etc. too.

</comment></comment><comment><USER19> This is not VCS friendly. Resize any column and you have to rewrite 99% of the file, tracking changes becomes hell on earth.

<comment><USER20> What do you mean by "resize any column"?

<comment><USER21> I presume the parent means changing the data s.t. the max length of strings in a column changes; this would cause the padding after that column to change, so as to keep subsequent columns aligned & looking nice. This doesn't play well w/ VCS diffs, as a bunch of data that isn't changing semantically is now changing syntactically, and a simplistic textual diff reflects now that syntactic change, whereas a human reader would want to see the semantic change.

Now… most diff tools also have options to ignore whitespace changes, for cases like this.

</comment></comment></comment><comment><USER22> CSV is hell, glad to see improvements. Some idiot somewhere decided that Comma Separated Values in certain locales should be based on semicolons (who would have thought files would be shared across country borders!?), so when we open CSV files that are actually comma separated all the information is in the first cell (until a semicolon appears).

To get comma separated CSVs to show properly in Excel we have to mess around with OS language settings. CSV as a format should have died years ago, it's a shame so many apps/services only export CSV files. Many developers (mainly US/UK based) are probably not aware of how much of a headache they inflict on people in other countries by using CSV files.

<comment><USER18> Semicolons come from many countries using the decimal comma in lieu of the decimal period. Of course, locale-specific number formatting should be a big no-no in any kind of a portable format anyway. But...

</comment></comment><comment><USER20> Some time ago I've worked on a similar, CSV-like thing that improves on readability and canonical representation, and which can hold multiple tables in a single file or stream like a database: [http://jstimpfle.de/projects/wsl/main.html](http://jstimpfle.de/projects/wsl/main.html) . (I hope it's not rude to reference it here)

<comment><USER3> No worries. Thanks for highlighting WSL - whitespace separated literals. The world needs more of this. Keep it up.

</comment></comment><comment><USER23> I like the intent "Easy-to-Write, Easy-to-Read", but the contents from the README leave much to be desired. There are far too many corner cases which the README doesn't address (many already mentioned below), and I can think of more, like, how does a parser know "city: Halifax" as "key: Value" and not just value="city: Halifax"?

Even if a `spec` is created which clearly defines these cases and standardizes how they should be handled, does it qualify to be called CSV 1.1 (or 2.0? really, as it probably won't play nice with existing CSV 1.0 implementations). It is almost as-good-as creating a new format altogether. And there are many to complete with.

I also wonder if it REALLY solves the problems it aims to solve (even if the tech specs were in place). CSV in any form is not human-friendly. This is especially true when you have wide columns (say >30). Even if the records were spaced-out in a human-friendly way in STATE-1, editing records where values are of highly varying width (within a single column) will soon mess the justification when you get to STATE-2. If you skip the requirement for `fixed- number-of-fields` and `key:value` style named values; that messes readability (by humans) even more!

I've had good success with `reading` CSVs using the `csvtk` ([https://github.com/shenwei356/csvtk](https://github.com/shenwei356/csvtk)). It provides excellent support for pretty-printing CSVs, filtering select fields etc. `writing` is still a pain, but I still feel spreadsheets are the way to go. They have been around for decades. Its sad if the formatting by specific tools is screwing CSVs, but, solving the composition/modification requirement purely by way of formatted-plain-text is a really tall ask.

</comment><comment><USER24> ASCII defines control codes for record and field separators, thus avoiding CSV's problems with quoting and escaping characters. The only hassle is entering and displaying the control characters.

[https://en.wikipedia.org/wiki/C0_and_C1_control_codes#Field_...](https://en.wikipedia.org/wiki/C0_and_C1_control_codes#Field_separators)

</comment><comment><USER25> Good effort. And I like the minimal documentation. However, and I would love to be proved wrong, I think you are about 20 years late for this project to get any interest aside from some effimerous moments in hackernews...

<comment><USER3> No worries. It's just a write-up for what gets used in football.db, beer.db, world.db, etc. day in, day out. Also remember - as the say it's never too late :-).

[1] [https://github.com/openfootball](https://github.com/openfootball) [2] [https://github.com/openmundi](https://github.com/openmundi) [3] [https://github.com/openbeer](https://github.com/openbeer)

</comment></comment><comment><USER26> If it ain't broken don't fix it.

<comment><USER27> it was already broken, but i dont think this will fix it either

</comment></comment><comment><USER28> I'm sorry, but when do you ever need to read a CSV file in an environment where you don't have a spreadsheet editor anyway?

I work in the public sector, we use a lot of CVS, even to non tech savvy people.

I've never heard the use case for this project.

Now that might not matter, but you're breaking CVS to fulfill a nonexistent use case.

</comment><comment><USER29> How does this improve on RFC 4180? (I only _wish_ excel and many other parsers behaved predictably like that RFC, but it is a decent proposal. )

( [https://www.ietf.org/rfc/rfc4180.txt](https://www.ietf.org/rfc/rfc4180.txt) )

<comment><USER3> The [strict] RFC 4180 CSV Format memo is way too simplistic and "specifies" a format for machine-reading and writing. It's not a human csv format. What's human? It's a format that you hand edit and so on - thus, you want comments, blank lines, short cuts etc.

Just for the record to quote from the memo:

This memo provides information for the internet community. IT DOES NOT SPECIFY AN INTERNET STANDARD OF ANY KIND. IT DOES NOT SPECIFY AN INTERNET STANDARD OF ANY KIND. IT DOES NOT SPECIFY AN INTERNET STANDARD OF ANY KIND.

Just repeating it three times in case you missed it, see [https://www.ietf.org/rfc/rfc4180.txt](https://www.ietf.org/rfc/rfc4180.txt)

</comment></comment><comment><USER30> Uh... so the bottom one of the or is CSV 1.1?

How do you define headers? There's no header for the bottom csv version so I have to manually type headers for my data frame?

I got a dataset with 87 obs and ~8700 columns here. I'm not going to manually name those columns. What's the solution?

<comment><USER3> Yes, you can add a header, of course. No worries. CSV 1.1 is still CSV. It's not CSV 2.0 :-).

</comment></comment><comment><USER31> [ ["If", "we're", "going", "to", "improve", "CSV"], ["Let's", "do", "it", "with", "some", "JSON"] ]

<comment><USER3> Good point. You're not the first one (even in this thread). To repost:

See the CSVJSON format - [http://csvjson.org](http://csvjson.org) \- Love it. I will add a new variant called CSV <3 JSON shortly to the CSV 1.1 repo and csvreader etc. too.

</comment></comment><comment><USER32> [https://xkcd.com/927/](https://xkcd.com/927/)

</comment><comment><USER33> You can already do this in R with a fixed-width text file, comments, and the names of variables in a header. The commas serve no actual purpose AFAICT.

</comment><comment><USER34> The number parsing is going to be a nightmare with this!

</comment><comment><USER35> Reminds me of old IBM punch card fixed-column-aligned tables... Still in use in many systems (such as the US ACH wire system...)

Everything old is new again.

</comment><comment><USER36> What if you want to start a row with a field that has a `#`? Not to mention how to handle backwards compatibility with the above concern...

<comment><USER3> If a field starts with # you have to escape it, that is, put it into a quote e.g. "# no comment".

</comment></comment><comment><USER37> I don't get the point. If your are on to replace your CSV im-/exporter you can switch to a non-ridiculous format anyway.

</comment><comment><USER38> Why not use tab-separated values, you get improved readability and less quoting needed, or am I missing something?

<comment><USER3> See the Awesome CSV page, for the discussion about tabs vs commas - [https://github.com/csv11/awesome-csv#tab](https://github.com/csv11/awesome- csv#tab)

Yes, ideally the tab is perfect (no escape rules needed, etc). In practice you cannot tell if you're human if a tab is a space or a space is a tab and than you will get into trouble reading / writing your data etc. Anyways, both are great and needed and use tab2csv or csv2tab to convert or pipe (when using command line tools) :-)

</comment></comment><comment><USER39> Or just open the CSV in a spreadsheet?

</comment><comment><USER40> Yeah... No. This doesn't solve anything.</comment>


Fire Eagle goes to open invite status - <AUTHOR> http://fireeagle.yahoo.net/ This is basically me giving away some invites I have to people who might want to check it out:
The eagle has landed! Fire Eagle's Beta is now open by invitation only. We've chosen to invite you because we think you're a super-cool hacker, super-cool early adopter, or just super-cool ... and also because you gave us your email address and told us to let you know when we launched (thanks for doing that!).
Fire Eagle is the secure and stylish way to share your location with sites and online services, while maintaining unprecedented control over your data and privacy. It's built on open APIs so you can build all kinds of applications that respond to your location.
We're also giving you four more Fire Eagle invitations to pass along to a few friends. These are also one-time invitation codes and can't be reused, so share them wisely! http://fireeagle.yahoo.net/ticket/PiybqXZSW2CJen7C http://fireeagle.yahoo.net/ticket/WQi54p9MdWiQCNFC http://fireeagle.yahoo.net/ticket/nrKGwlmzFDtyzIT2 http://fireeagle.yahoo.net/ticket/PMMhrqllLvcwRVWz
Finally, if you want to get started building an application that's powered by Fire Eagle, then you should join our developer group: http://new.groups.yahoo.com/fireeagle <comment><USER2> I've got a few spare invites. First 4 replies with an email address to this comment can have them.

</comment><comment><USER3> this is already been done with multiple types of content feeds... and an iphone app

<http://www.groovr.com/></comment>


Ask HN: Anyone sells YouTube video lists? - <AUTHOR> I am planning to make a youtube video search engine(hopefully better than what youtube offers currently). What is a good way to collect  a million video URLs other than scraping or using youtube APIs? Does anyone sell a premade list?<p>Followup question. How do podcast search engines get the data? Do they scrape from podcast websites? <comment><USER2> Personally, I would add some data to your search engine every time a person makes a new search query by searching that same query on youtube's api ([https://developers.google.com/youtube/v3/code_samples/](https://developers.google.com/youtube/v3/code_samples/)) or by scraping the data. Now that you've got some relevant results, you can sort through those (or all of the URLs you have collected) to give a more refined list of search results. If you want to pay me to get it done, I would be fine with that too.

The problem with buying a premade list (even if there is such a thing) is that it would be a whole lot of trouble to keep it updated.

<comment><AUTHOR> I agree with the keeping updated part. I am just looking for a starter list of sorts. Thanks for idea regarding calling youtube in the backend, atleast until I have enough data. I will do with it.</comment></comment>


Nowden Live – With Startpage.com Private Search - <AUTHOR> https://www.youtube.com/watch?v=FwOIgOH_S18 <comment> ar-jan Title should start "Snowden".</comment>


The bonnethead, the world's first omnivorous shark - <AUTHOR> https://www.cnet.com/news/meet-the-bonnethead-the-worlds-first-omnivorous-shark/ <comment><USER2> The title, in the context of hacker news, makes it sound like a new product announcement.

</comment><comment><USER3> There was a bit about a ferocious sea weed-eating shark on one Simpsons episode. Funny how these things turn out... :)

<comment><USER4> "…but none attacks its prey with more fury than the seaweed shark"

[https://www.youtube.com/watch?v=bgybr7BWY-w](https://www.youtube.com/watch?v=bgybr7BWY-w)

<comment><USER5> The really interesting joke here is from SpongeBob: Sea-weed composition = 50% Sea, 50% weed

Sharks seeming to be able to digest some celulose from seagrasses is not enough to classify them as omnivores, (cats eat grass sometimes but are still hypercarnivores) but is very strange in a fish, because fishes are notoriously bad using sugars. They use fat as main source of energy. Cats have similar problems and will be normally uninterested in sugar.

<comment><USER6> Humans can't digest cellulose. We call it instead dietary fiber, and it's important for our health. Not sure if this shark is actually directly digesting the cellulose or relying on gut bacteria like ungulates. This shark possibly notwithstanding, I don't believe there are any vertebrates than can directly digesting cellulose

Regularly consuming plant material for nutrition seems like enough of a bar to render the shark an omnivore.

<comment><USER5> > Regularly consuming plant material for nutrition seems like enough of a bar > to render the shark an omnivore.

Not if forced artificially to do so. Keeping its health after consuming plant material would be a necessary step.

There is a reason for giving a shark a diet composed in a 90% of plants and is the same for having vegan dog food. Is a lot cheaper. As animal protein is expensive, we are in a rush for feeding captive carnivores as many plant protein as possible without killing them. You need to tag the animal as "but is an omnivore so this is fine" first.

I had keep a lot of fishes and know when a fish is starving or not having enough food. They turn like tadpoles: big head in a small skinny body. The fish shown in the photo has this symptoms. Either is a very young fish or is sacrifying its own muscle just to survive.</comment></comment></comment></comment></comment>


How the CIA Trains Spies to Hide in Plain Sight - <AUTHOR> https://www.wired.com/story/mastermind-cia-disguise/ <comment><USER3> Previous discussion -- [https://news.ycombinator.com/item?id=18291780](https://news.ycombinator.com/item?id=18291780)

</comment><comment><USER4> In a different life of mine, a long time ago, my surveillance team and I were doing a multi-day follow of a non-us person of interest in a major city.

I recall one day in particular, this person, who didn't know we had significant camera coverage of their route, changed clothing twice, right in front of our cameras. Like the short article described, it was pretty easy to don stuff, like a hat with built-in wig, reversible backpack/carrying case and obvious accessories.

So you'd be surprised how little it takes to fool a team of trained people following one person in a crowded area. We were lucky that our coverage caught everything, because had we not caught the person in the act of changing, we would have probably lost them.

Anyone who has a kid will recognize this phenomena from what happens at a crowded playground.

<comment><USER5> I had some connection to the flip side of the urban spy game. Cops trying to tail a man on a motorcycle in British Columbia (drugs). On a mountain highway. He meets another biker at a lookout where they trade bikes/helmets jackets ... or maybe they didn't. Then he gets onto a drive-on-drive-off ferry (bikes are filtered to the front) and they've lost him.

When two spies want to get together without tails, it isn't at a downtown coffee shop. It is deep in the woods on a quiet road. It is somewhere surrounded by people in helmets and driving generic vehicles. They drive the vehicles that cops don't. Motorcycles. Bicycles. A vespa scooter can out- maneuver any car attempting to follow it.

Want to get really hollywood. I've heard a legend about two russian spooks having a meeting while _surfing_. Alone out in the waves, nobody can see the exchange.

</comment></comment><comment><USER6> Reminder: Any CIA or MI6 agent who lasted more than a week in the Soviet Union had diplomatic cover. The NKVD and KGB caught all of our deep cover agents within days; in some cases within hours.

The United States excels at sigint and aerial/orbital recon, but the Russians have been top-notch at humint for a long time.

<comment><USER7> > The United States excels at sigint and aerial/orbital recon, but the > Russians have been top-notch at humint for a long time.

Maybe, but there's also an inherent counterintelligence asymetry between a liberal democracy and a totalitarian police state (which is why in a liberal democracy, the counterintelligence services are often trying to simulate a totalitarian police state.)

<comment><USER8> > _Maybe, but there 's also an inherent counterintelligence asymetry between a > liberal democracy and a totalitarian police state_

You could just as easily argue it in the opposite direction, it is much easier to co-opt the corrupt Russian government than it is to co-opt the less corrupt American government. An individual corrupt _unaccountable_ person of interest in a totalitarian police state is far more valuable than an individual corrupt person of interest _that is being watched like a hawk by the public_ in a liberal democracy.

I think a much more reasonable explanation for why government organizations are always trying to become more totalitarian is simply that the officials would rather have more power than less power. The only difference between here and there are the checks placed on them to limit what they can do.

<comment><USER9> >it is much easier to co-opt the corrupt Russian government than it is to co- opt the less corrupt American government.

Can you give a cold war example of America effecting who ran russia that is as dramatic as the Trump story in the other direction?

<comment><USER10> There's no evidence that Russia had any significant effect on who runs the US, even if you accept all of the claimed attempts as true, whereas it's universally accepted and documented that the US got Yeltsin elected: [https://www.theguardian.com/world/2016/aug/03/spinning- hilla...](https://www.theguardian.com/world/2016/aug/03/spinning-hillary-a- history-of-america-and-russias-mutual-meddling)

<comment><USER11> I'm fairly certain the thousands of dollars Russia spent on Google and Facebook ads, overwhelmed the $1.2 billion that was spent in total on the Clinton election campaign.

<comment><USER12> The ads are a red herring (and uncomfortably visible for both parties).

The real activity was on Facebook groups, viralizing anti-immigration and far- left news, controling the narritive on 4chan pol and /r/the_donald, and botted up - and downvote brigades.

It was so blatant it was visible to a non-expert on information warfare like me.

The people doing this probably got paid about as much as a paid advertisement.

<comment><USER9> [https://www.nytimes.com/2018/12/13/us/politics/butina- guilty...](https://www.nytimes.com/2018/12/13/us/politics/butina-guilty.html)

[https://www.theatlantic.com/politics/archive/2018/11/wikilea...](https://www.theatlantic.com/politics/archive/2018/11/wikileaks- trump-mueller-roger-stone-jerome-corsi/576940/)

I suspect we will see more of this sort of thing as the investigation continues, but of course, the investigation isn't over yet. My point is just that it was a lot more than 4chan and facebook.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER13> There are two very useful books on the subject - Surveillance Tradecraft by Peter Jenkins, and Close Protection by Richard J Aitch. Read those two and you should have no problem surveilling or following someone whilst remaining undetected, and similarly no problem in not being surveilled/followed.

<comment><USER14> Amazon has 'Close Protection' for $1229 and 'Surveillance Tradecraft' for $54. My library has neither book, which is not terribly surprising. Sadly, that means I'm unlikely to get a chance to read either one.

<comment><USER15> Buy it from the source for 65 GBP [http://www.cpbook.co.uk/](http://www.cpbook.co.uk/)

</comment></comment></comment><comment><USER16> Amusingly, when I lived in Germany in my twenties, people never seemed to assume I was an American military wife and homemaker. They would greet me in German, assuming I was a local national, or assume I was French or assume I was a teacher. I apparently didn't read as American.

<comment><USER17> While in the US, people would often tell me that my style of dress and my accent was hard to place, which I always found a bit odd since I was born/raised in the US (to a long line of Americans). And now after 10 yrs abroad, following that period, I can only guess I've got better at hiding it, merely through osmosis.

In my time abroad, I've become good enough at Brazilian Portuguese (my 2nd language, learned as an adult) that people in Portugal, where I've been living, often think I'm from Brazil. I always mention quite early on that I'm from the US but, nonetheless, I'm still intrigued with the idea of disguising oneself.

Linguistically, I'm highly doubtful that one can reach C2 status (that is, consistently sounding like a native) in a second language learned as an adult. Of the Brazilians I met in the States, I paid attention to how old they were when they arrived stateside, and judged their cultural/linguistic knowledge with people who were born in Brazil and never left. Anecdotally, I found the age of assimilation (of their home culture/language) was around 16. That is to say, if they arrived in the US at age 14, for example, they'd turn out to be more American than Brazilian by their early 20s. But I wonder how that can be entirely true if I myself don't seem to fit in that box, regarding my American qualities.

<comment><USER16> My mother is a German immigrant to the US. I speak a little German and I likely dressed a lot more like a German than an American because my mother had a strong influence on my fashion sense. So it's not real shocking that I would sometimes be assumed to be German.

What was bizarre to me was that it seemed like no one ever pegged me as an American. IIRC, I was on an American military base when someone asked me if I was French.

So this didn't just happen out on the streets of Germany. This happened in spaces where you should by default be assuming that most people there were American.

Granted, a lot of soldiers married foreign nationals, so it wasn't quite that simple. I mean, my dad was a soldier who married a German woman while he was stationed in Germany. But it seemed to me like the guess as to my nationality should skew towards _American_ when I was actually on an American base and not, say, fluently droning on in some foreign language.

</comment></comment></comment><comment><USER18> the cynical side of me assumes that they'd only show this information off if they had a new system for disguise which was far beyond what they showed here.

i mean, look at the image of the lady in the mask meeting HW bush -- that image is ~30 years old. they've had 30 years to improve on that...

</comment><comment><USER19> Interesting. "We can turn a women into a man... But it's almost impossible to turn a man into a women."

<comment><USER20> We all start out female in the womb, and then some of us turn make given the right chromosomes. If you look at those who have transitioned their genders, female to male almost always look more "correct" than male to female... it's hard to remove the effects of testosterone, but easy to add it.

<comment><USER21> True although paradoxically there is one aspect that is reversed - <USER2> surgery is harder and less functional to construct a functioning penis from a vagina than to create a more or less functional vagina from a penis. Put crudely but succinctly as 'it is easier to make a hole than build a pole'.

Reminds me of the one bit of weirdness with birds - they have the opposite convention of sex assignment being based upon ZW where males are ZZ and females are ZW. Females also have two ovaries but only one of them is active and the inactive one produces testosterone - meaning that if the primary one lost functionality they would start spontaneously masculinizing.

<comment><USER22> This is one of the reasons you see a higher standard deviation for traits in female birds than male birds. The opposite of humans.

<comment><USER23> Am I correct in interpreting your comment as saying that human women generally have less variance in (gendered?) traits than human men. That's an interesting observation if it is what you meant.

<comment><USER21> Well part of it is also that any X-linked traits are more likely to tend dominant with the back-up copy. Look at colorblindness for one. Women would need a 'bad' gene on both Xs. A man would lack a backup copy on the Y and thus only needs one for it to express.

</comment></comment></comment></comment></comment></comment><comment><USER24> Former CIA Chief Explains How Spies Use Disguises - [https://www.youtube.com/watch?v=JASUsVY5YJ8](https://www.youtube.com/watch?v=JASUsVY5YJ8)

</comment><comment><USER25> Marking this so I can find it again. The video is excellent.

<comment><USER26> Quick tip: you can mark stories as favourite with a link at the top of the page, and comments on their individual pages (accessed by clicking on the "x hours ago" link).

You can see your own upvoted stories and comments, as well as favourited stories and comments at the bottom of your profile page.

<comment><USER25> Ah thanks.

</comment></comment></comment><comment><USER27> The same assholes that posed as Doctors Without Borders so they could gather dna evidence to confirm where Osama Bin Laden was hiding, thus hampering any efforts to eradicate polio because nobody trusts DWB anymore? _That_ CIA?

[https://www.scientificamerican.com/article/how-cia-fake- vacc...](https://www.scientificamerican.com/article/how-cia-fake-vaccination- campaign-endangers-us-all/)

I can only assume they have spooks running around pretending to be the Red Cross.

</comment><comment><USER28> this article is blogspam for an AMA [https://www.reddit.com/r/IAmA/comments/a32j7e/im_jonna_mende...](https://www.reddit.com/r/IAmA/comments/a32j7e/im_jonna_mendez_the_former_chief_of_disguise_for/)

<comment><USER29> An interesting quote from the AMA:

"I have difficulty separating my life from the work at CIA. This is true for a lot of my colleagues. Early on I found that the men in my office, upon retirement, had an average life-span of 18 months. Eighteen! Their work was their life, and most of them didn't have outside interests. They literally lived their careers inside the CIA. We changed our retirement procedures to deal with this problem and it got better. Tony Mendez used to say that working at CIA was like drinking from a firehose and that retirement was like jumping from a moving train."

I wonder what it is about the work. This must be true in other professions as well. But at least in computer software after retirement you can still build things and practice your craft.

<comment><USER30> My father grew up in Oshawa, Ontario where there was a big GM factory making cars. Hard work with decent pay but good pensions- if you lived long enough to enjoy them.

As the stories went, a lot of guys, they put in their 25 years at the plant, retire, and drop dead within 2 years. The only thing worse for you than two and a half decades of hard labor is suddenly stopping.</comment></comment></comment>


The Irrationality of Giving Up This Much Liberty to Fight Terror - <AUTHOR> http://www.theatlantic.com/politics/archive/2013/06/the-numbers-dont-lie-its-irrational-to-give-up-this-much-liberty-to-fight-terror/276695/

<comment><USER2> Discussion: [https://news.ycombinator.com/item?id=5854818](https://news.ycombinator.com/item?id=5854818)

<comment><AUTHOR> thanks, didn't realize it's a repost

</comment></comment><comment><AUTHOR> For over a decade the US Gov't has used the idea of terrorism to justify many wars and legislation produced over that time. It's starting to get annoying. Hopefully this article will help people understand why it is an irrational fear.</comment>


China's voluntary organ donation data perfectly fit quadratic equation - <AUTHOR> https://mobile.twitter.com/mprobertson/status/1195151387585216512 <comment><USER2> If the organ donation data is cooked, imagine the incentive to massage market moving data like GDP, industrial production, inflation....</comment>


Inside Trump's Cruel Campaign Against the U.S.D.A.'s Scientists - <AUTHOR> https://www.vanityfair.com/news/2017/11/usda-food-stamps-school-lunch-trump-administration <comment><USER2> The history would probably benefit of a little less verbosity IMHO</comment>


Why Do People Say "Yes?" The "6 Weapons of Influence" - <AUTHOR> http://www.fripp.com/art.of_influence.html <comment><USER2> She is basically repackaging Aristotle's Rhetoric for modern business folk who skipped that class in college. "Of the modes of persuasion furnished by the spoken word there are three kinds. The first kind depends on the personal character of the speaker; the second on putting the audience into a certain frame of mind; the third on the proof, or apparent proof, provided by the words of the speech itself." To that, she has added various forms of bribery, which aren't really forms of persuasion.

Read the original here: <http://classics.mit.edu/Aristotle/rhetoric.html>

</comment><comment><USER3> This article does not do the book/author justice. Buy, 'Influence, the Psychology of Persuasion'. Quick read, and it is one of the best books you will ever read on marketing and human behavior.

<comment><USER4> I'll second that, and it goes beyond mere marketing. For example, you're outside at some group event, say, a concert, and you find yourself having a heart attack. What do you do? Do you call for help? "Help, I'm having a heart attack!" That might work, but what works better, according to research, is to call out to a single individual. "You, sir, in the red shirt, yes you! Help me, I'm having a heart attack." This is due to the phenomenon known as "pluralistic ignorance," where a group of people will collectively do nothing in response to some threat/problem, because they're collectively looking around at each other to see what to do (and they see each other doing nothing but looking around...) By singling out someone, you break through the group psychology.

The book emphasizes the subconscious and automatic nature of influence techniques. You may think "oh, I'm rational and can't be manipulated," and you would be wrong. Only becoming aware of them can you have some hope of avoiding them.

</comment></comment><comment><USER5> Lousy article. Where's the "because it's in their best interest" option? Sometimes people say yes because they think they're getting a good deal, not because you've used psuedo-psychology to twist their arm.

<comment><USER6> People do act rationally. But it's a small minority of the time!

Okay, that's a bit of an exaggeration. We tend to act rationally a lot of the time. But those are often "no brainer" situations where there is little question about the best option. The 6 factors in the article come in to play when decisions are less clear.

</comment></comment><comment><USER7> I liked it better boiled down and including a story about a Persian rug.

<http://news.ycombinator.com/item?id=270145>

</comment><comment><USER8> check this one out too: theories about persuasion - [http://changingminds.org/explanations/theories/a_persuading....](http://changingminds.org/explanations/theories/a_persuading.htm)

</comment><comment><USER9> This is a bad rehash of "The Power of Persuasion" by Robert Caldini. One of the best sales books I've ever read.

</comment><comment><USER10> great article none the less</comment>


Open Arms: Open-source Prosthetics - <AUTHOR> http://www.spectrum.ieee.org/mar09/7908

<comment><AUTHOR> Via LWN, where a commenter also mentioned a Scientific American article on the topic: <http://www.sciam.com/article.cfm?id=open-source-thinking>

<http://lwn.net/Articles/323763/></comment>


Middle-class China aims for the Ivy League - <AUTHOR> http://www.iht.com/bin/printfriendly.php?id=19063547 <comment><USER2> I see you revised your title while I was attempting to comment at first. The article you've linked to is an interesting discussion of incentives on parents to prompt their children to apply to certain colleges.</comment>


Merry Etc... - <AUTHOR>

Happy Xmas, Merry New Year, hanukah is over already but happy that, etc, ad infintum. <comment><USER2> Merry Gifting day!</comment>


How the Berlin Wall Really Fell - <AUTHOR> http://www.nytimes.com/2014/11/07/opinion/how-the-berlin-wall-really-fell.html

<comment><USER2> > many Americans have come to believe that the wall fell thanks to President > Ronald Reagan's direct, personal intervention

Is this truly a belief held in the USA? As an east-german this seems equal parts baffling and insulting to me.

If so, i can only thank the author for correcting this belief in a quite public manner.

<comment><USER3> I think this story is wrong on many accounts.

For starters, I don't think many Americans understand any details other than the anecdote that there was a wall in Germany, and it came down. Please, rest assured east Germans, we have done worse than steal your achievement- we don't give it any real thought anymore.

Americans glibly joke about the fear culture around communism from that time, but we have no awareness of the irony of our current fears of isis or Ebola.

Reagan's speech at the wall was iconic, but you really need to be over 35 to be aware of it.

<comment><USER4> I am an former Est Germany. I am older than 35. I can assure you, the Reagan speech did noting.

There where only one or two important speeches, may be 3 if you count Kennedy in.

One of the most important was, when Willy Brand had its visit in East Germany. The only important visit by a western politician in East Germany. The other one, which was not really a speech, but very important, was the one by Genscher for the release of the people into West Germany after occupying the West German embassy in Praha, CZ.

<comment><USER5> It was clearly more directed at the USSR and specifically Gorbachev. Khrushchev and Ulbricht had unwisely created the concrete, so to speak, symbol of the evil empire, a very visible wall dividing a historic, major city. Where families trying to cross it were shot at, often successfully.

It had to have been terribly embarrassing to Gorbachev to be called out like that, and it strongly cut against many of his and the USSR's themes, e.g. if it's so great, why do you need that wall keeping people inside of your socialist worker's paradise?

Getting back to the 1989 fall, _something_ stayed the hands of the USSR, unlike Hungary in 1956 and Czechoslovakia in 1968. I believe it's impossible to make an honest case that Reagan had nothing to do with that.

</comment></comment></comment></comment><comment><USER6> As an East-German I find this slightly offensive (edit: Reagan bringing the wall down). It was the peaceful people on the streets, a bit of luck (no hot- heads in the police and military who wanted to open a civil war on their own people), a completely paralysed government and the Russians who didn't want to intervene. Besides, we all know the only American who had a hand in this was Hasselhof, not Reagan ;)

<comment><USER7> You may want to read the actual article. It says pretty much exactly that.

> In the decades since, many Americans have come to believe that the wall fell > thanks to President Ronald Reagan's direct, personal intervention. In a 1987 > speech in front of the Brandenburg Gate in a divided Berlin, he told Soviet > leaders to "tear down this wall" — and so, we've been told, they did.

> This misreading of the actual fall of the wall is, at best, incomplete; at > worst, it's dangerous, contributing to the belief that American leaders can > go "from Berlin to Baghdad," shaping world events while ignoring the complex > realities of the locals.

<comment><USER6> Yes, sorry for being unclear. The article is a good thing and hopefully helps to clean up the misconception that Reagan single-handedly liberated us. My impression of him was more that he was a diehard cold-warrior who'd rather start dropping the nukes. Basically the western equivalent of Brezhnev and pretty much the opposite of Gorbatchev. But that was a long time ago, and I was a teen in the 80's so my memory might be a bit off.

<comment><USER8> Can you provide any examples related to Reagan being a risk to start dropping nukes? Or where he ever said or gave the impression that his preference was to start dropping nukes?

I think you're extremely wrong in your character assessment. There are numerous books available on Reagan in relation to what he actually said and wrote about his time as President, none of it indicates he was eager to start a nuclear war.

<comment><USER5> I can believe that flohofwoe honestly got that impression growing up behind the Iron Curtain (per his website, 12 in 1984). Heck, at the time most of the US Main Stream Media (MSM) portrayed Reagan as being like that. I'll bet that if he tuned into the BBC back then he'd had heard the same thing.

</comment></comment></comment></comment></comment><comment><USER9> I'm surprised the author makes no mention of the exodus of East Germans to the West via Hungary, which by the time the Berlin Wall came down had been ongoing for 6 months.

[http://www.politics.hu/20140627/hungary-austria-slovakia- mar...](http://www.politics.hu/20140627/hungary-austria-slovakia-mark-25th- anniversary-of-iron-curtain-opening/)

</comment><comment><USER10> How on earth anybody can write a story about the fall of the Berlin Wall (indeed, the entire collapse of the USSR) without discussing the economic collapse in the face of dropping oil prices is beyond me. The details, such as minister speeches and ignored guards are interesting in the color they lend to the collapse, but the principle story is one of overwhelming macroeconomic forces at play.

[http://marginalrevolution.com/marginalrevolution/2007/06/why...](http://marginalrevolution.com/marginalrevolution/2007/06/why_did_the_sov.html)

<comment><USER5> From the quoted article:

" _The timeline of the collapse of the Soviet Union can be traced to September 13, 1985. On this date, Sheikh Ahmed Zaki Yamani, the minister of oil of Saudi Arabia, declared that the monarchy had decided to alter its oil policy radically._

That doesn't quite fit the _connotations_ I associate with "overwhelming macroeconomic forces at play". That was a specific policy decision ... which I don't believe you can isolate from things like the USSR's genocidal occupation of Afghanistan, and the Muslim world's many reactions to it.

As I understand it, we strongly encouraged the Saudis in this policy. Visibly, by things like selling them F-15 and E-3A AWACS planes in the early '80s before that policy change ( _very_ controversial arms deals back then).

As I mentioned elsewhere in this thread, as part of Reagan's unique determination to _end_ the Soviet Union, we the USA did everything we could to deny it hard currency, like obstruct and sabotage the export of natural gas to Western Europe.

And like good capitalists, after Reagan took office, resumed making a buck in the process by selling them grain ^_^.

ADDED: Why the _bleep_ didn't Gorbachev et. al. allow a massive increase in household plots: [https://en.wikipedia.org/wiki/Household_plot](https://en.wikipedia.org/wiki/Household_plot) We always expected (see for example _Red Storm Rising_ if I remember correctly) that when things got tight, the restrictions on them would be relaxed and the resulting productivity increase would tide them over. Or was that attempted but failed? Or is this "oil and grains" economic picture a bit too simple (as I've noted natural gas also played a role).

<comment><USER10> The argument would be that the Policy Decision was simply the reaction to larger forces at play - in much the same way high oil prices resulting in fracking in the US and the Oil Sands in Canada being developed, and taking away market share from Saudi Arabia (weakening their geopolitical position), one could say that it was a foregone conclusion that Oil prices were going to drop in the 80s after the 1973 oil crisis.

That's the thing - every time you see a huge increase in the cost of anything, economic forces kick in, and there is a response.

That's a long winded way of saying there were much, much bigger forces at play than Sheikh Ahmed Zaki Yamani in 1985, and, his announcement can be seen as little more than the result of those forces.

</comment></comment></comment><comment><USER11> They quote:

\- The political reform started 4 years earlier by Mikail Gorbatchev, which was designed to reinforce the party's strength in exchange for softening very few things.

\- The disorganization of the Party and the lack of trust within the Stasi,

\- The press conference of Nov 9., where the administrator said something about travels, which wasn't reported acurately but which included words like "effective immediately".

\- The massive amount of people in front of the gates.

I would love to have more details about the Perestroika and the reform background. One does not simply walk up with a crowd to a checkpoint without thinking about Tiananmen. I've always wondered whether Gorbatchev had desired freedom for his people and subtely acted to make such events happen.</comment>


Goodbye cars, hello colour: the great reinvention of city intersections - <AUTHOR> https://www.theguardian.com/cities/gallery/2018/sep/20/goodbye-cars-hello-colour-the-great-reinvention-of-city-intersections <comment><USER2> Where I live they have been using really bright green paint to designate certain spots for bicycles. For example, where a bike lane crosses a car lane.

Lately I've noticed that with the accumulation of road grime and/or with the paint fading in the sunlight, the green no longer stands out. The painted areas are getting less and less obvious.

Point being, the idea has a lot of potential, but it is going to require a lot of maintenance. All that paint needs to be redone (cleaned and/or repainted) periodically.

Whereas if you put in a curb or physical barrier to demarcate where pedestrians can stand or where bikes should be, that lasts longer / is more permanent.

<comment><USER3> Not all "paints" are equal.

In Indianapolis, for a while they were primarily using "Ride-A-Way", but recently switched to "CycleGripMMAX", which is more expensive per square foot, but lasts longer. Additionally, there were complaints that the Ride-A-Way would wear down to where it was difficult for bikes to get good enough traction to safely take curves; CycleGripMMAX is way grippier.

For smaller sections (like crosswalk-type "dotted" strips, like these[3]), they might actually be strips of thermal-plastic melted in to the asphalt. That should last as long as the asphalt does; yeah, it will eventually need re-done, but at about the same time that the asphalt needs re-surfaced.

Then you'll talk to a city council person from another town who says that they tried doing green bike-lanes, but they wore out very quickly... and reveals that they just used outdoor paint from the hardware store.

[Ride-A-Way]: [https://www.ennisflintamericas.com/catalog/product/view/id/1...](https://www.ennisflintamericas.com/catalog/product/view/id/1032/category/81)

[CycleGripMMAX]: [https://www.ennisflintamericas.com/catalog/product/view/id/9...](https://www.ennisflintamericas.com/catalog/product/view/id/945/category/81)

[3]: [https://www.google.com/maps/place/New+Albany,+email@example.com...](https://www.google.com/maps/place/New+Albany,+email@example.com,-85.8129673,38m/data=!3m1!1e3!4m5!3m4!1s0x88696c3c70677891:0x69e372d31b3e1b7a!8m2!3d38.2856247!4d-85.8241312)

</comment></comment><comment><USER4> A old term used by traffic engineers... Paint is the cheapest form of traffic management.

</comment><comment><USER5> Should people be encouraged to hang around near an intersection? Surely the pollution is off the charts. How about just banning cars 1 day a week? Then 2, then 3...

<comment><USER6> Banning cars is a political disaster without serious alternatives. The paint is mostly to improve public safety when crossing the road, and if you have a lot of leftover space all of a sudden, it's better to convince people to use it rather than turn it into a desolate space orphaned and abused by the community that becomes dirty and crime-infested.

</comment></comment><comment><USER7> The base idea is pretty good, but I guarantee some of those designs are going to seem tacky as hell in a couple decades</comment>