GNU Taler – Electronic payments for a liberal society - <AUTHOR> http://taler.net/ <comment><USER2> This is a digital payment scheme that respects the customers privacy, when buying from merchants. The alternative besides a cryptocurrency, would be a prepaid cc you buy with cash in a store, but then VISA knows everything you buy with it, whereas this system the payment provider (the "mint") and merchant are separated. If you directly buy things with SEPA then your bank knows everything you buy. This allows for lawful payments, allows any company to become a mint (no payment monopolies) and is free software so you can make your own wallet client and not have to rely on proprietary software wallet apps.

People can still become a black market escrow, keep creating new merchants and facilitate user to user transactions unless there is some kind of cost or identification required to accept Taler payments, or exchange Taler for bitcoin. US law makers will likely try to prohibit Taler though if any online gambling accepts it as payment, for example Paysafecard had to design a special UIGEA compliant card in order for it not to be able to be plugged into a gambling site to get into the US payments market.

Speaking of bitcoin pretty much every online exchange will start accepting Taler, then they aren't so exposed to banks shutting them down and can trade in any currency.

</comment><comment><USER3> This sounds a lot like David Chaum's eCash of the 90s. You transfer ordinary money into a Taler account in a Taler bank and transfer them to other Taler accounts, and from there to ordinary money. That's unlike Bitcoin in pretty much every respect.

It takes some courage to make a crypto-payment protocol explicitly taxation- friendly, but they claim the tax man only gets to see the amounts, not the identities. What a pity the design doc isn't yet on their website.

And I hope the eCash patents have expired...

<comment><USER4> For those who don't know about DigiCash:

[http://www.chaum.com](http://www.chaum.com)

[http://cryptome.org/jya/digicrash.htm](http://cryptome.org/jya/digicrash.htm)

[https://en.m.wikipedia.org/wiki/David_Chaum](https://en.m.wikipedia.org/wiki/David_Chaum)

[http://www.hit.bme.hu/~buttyan/courses/BMEVIHIM219/2009/Chau...](http://www.hit.bme.hu/~buttyan/courses/BMEVIHIM219/2009/Chaum.BlindSigForPayment.1982.PDF)

</comment></comment><comment><USER5> Consider how valuable this could be to privacy.

Governments likely will insist on a mechanism to track electronic currencies, for tax, regulatory and other reasons. Governments seem to have very little concern for privacy. So the mechanism they choose could very well ignore privacy.

By building a mechanism that meets government needs and by being an early mover, Taler could be the de facto solution by the time governments get around to dealing with this issue, and could make privacy the policy.

<comment><USER2> There's a global racket in market data mining too, to track demographics and see what they are buying. This system defeats any large payment entity from payment tracking.

Taler can also compete with Ukash, since it's free/open any merchant can implement their own Taler for cash business without signing large exclusive contracts with Ukash or waiting a month to be settled by them, as you can freely choose which Mint to do business with.

<comment><USER6> > This system defeats any large payment entity from payment tracking.

Merchants of physical goods must know the identity and delivery address of their customers in order to deliver them. Given how easy many companies find it to embed 3rd party trackers or adverts on their site, it seems reasonable to believe that they will also find it easy to sell this data ("to help detect and prevent fraud") to the payment processing companies.

<comment><USER2> True, but at least Taler allows for the consumer to chose merchants who specify they don't sell data. Right now I can't buy anything online and not be tracked by the payment provider unless I use bitcoin. The prepaid credit cards here all require online registration and in store identification, Visa/Mc/Amex then track my purchases. If I show ID to buy $200 worth of Taler in a 7-11, there's no way for my Taler provider to see what I used it for.

<comment><USER5> > Right now I can't buy anything online and not be tracked by the payment > provider unless I use bitcoin

Here are a couple of solutions I've seen but haven't tried:

* Masked Cards by Abine (a privacy service): Abine gives you a card to use and you pay Abine. Obviously, you'll need to trust Abine.

[https://dnt.abine.com/#feature/payments](https://dnt.abine.com/#feature/payments)

* PaySafeCard: A prepaid card; "You don't have to enter any personal information or bank account or credit card details. Your privacy remains completely protected at all times with paysafecard!"

[https://www.paysafecard.com/en-us/](https://www.paysafecard.com/en-us/)

</comment></comment></comment></comment></comment><comment><USER7> No thank you. With regards to my government, I would like to see more privacy rather than less. This seems to give me less.

I do like the idea of complete transactional privacy between end users and/or merchants. But we already have this today with CryptoNote based cryptocurrencies like Monero - with optional transparency through "view keys".

<comment><USER8> As far as I understand it, Taler seems to be great for privacy __and __society (ensuring income is taxable). Governments can 't see how you spend your money, but they can see who receives payments.

Bitcoin and others will never succeed because they are too friendly towards illicit behaviour. Taler seems to strike a genuinely interesting and ethical balance here.

<comment><USER9> The underlying assumption is that governments don't care about how you spend money. This is patently untrue, because "liberal" governments explicitly use taxation as global social modification schemes, for all sorts of things beyond income redistribution - from minimizing percieved vices (alcohol, tobacco) to minimizing environmental footprint, etc.

The idea that this sort of activity is able to be decoupled from surveillance is one of the more unfortunate delusions in politics.

<comment><USER10> Why did you specify liberal governments (is this not something all governments do?), and put it in quotes?

<comment><USER11> Other less liberal governments will simply ban something they consider sinful instead of try to make money off it while letting their populace (who they are supposed to care about) slide into hell (if the thing is truly sinful). Historically this doesn't work so well in the US, but in other places like Singapore it works better.

<comment><USER12> Not true at all. Conservative governments often desire things like trade protection tariffs. They may ban 'immoral' things rather than merely tax them differently, but they are still very interested in controlling the flow of money.

</comment></comment></comment></comment></comment></comment><comment><USER13> Uses GNUnet's crypto library to generate keys. Biggest problem I've found is fraud potential, the mint operator can run a fractional reserve since it relies on "government authorities doing regular audits" and also that it touts turning Mastercard/Visa into Taler. Carders will become their own merchants, card a mint to get Taler and then cash it all out before anybody knows what happened. You also can't transfer Taler p2p, all transactions go through the mint to prevent "black markets".

Edit: Uses this API [http://api.taler.net/](http://api.taler.net/) plus seems to have future GNUnet compatibility built into it's system for payments over it.

<comment><USER3> Does it use GNUnet? I only see them mention REST here, HTTPS there.

As to the carder-friendlyness: credit card companies are known to treat payment aggregators with lots of suspicion, so I suppose there just won't be many credit-card-to-Taler "banks".

</comment></comment><comment><USER14> > Taler ensures that governments can easily track their citizen's income

So exactly why would I want to use this?

<comment><USER15> Because unless the government can track citizen's income it cannot tax it, and without taxes modern society cannot function at all.

Technologies like this would be a boon to many developing countries like mine where tax evasion and corruption is rampant. It would enable the government to collect funds for infrastructure, education, environmental protection and healthcare (and yes, I am a progressive liberal who would rather have the government provide these at subsidized cost rather than private entities; even at the cost of "efficiency" if need be).

<comment><USER16> The question remains, why would anyone choose to use a system like this? Wealthy individuals and companies (aka those with most of the money) have a vested interest in having their transactions _not_ easily tracked by the government.

<comment><USER17> Simple: this aims for regulatory support. If Taler (or a system like Taler) is embraced by governments, and is therefore given the full support of the financial system, Bitcoin could be permanently relegated to black-market-only use.

<comment><USER18> Why would we want bitcoin to be relegated to a black market only use?

Bitcoin is more valid money then paper currency. we can directly verify the creation of the transaction.

</comment></comment></comment></comment></comment><comment><USER19> From the description it seems like "GNU PayPal" or "XMPP for Stripe".

To me it seems quite desirable: a decentralized protocol for sending government-backed money in a way that is legal (strong crypto isn't protection from being arrested for money laundering), and yet offers better privacy and is more open to competition than existing networks of banks/credit cards/PayPal.

<comment><USER20> > From the description it seems like "GNU PayPal" or "XMPP for Stripe.

The name itself screams "money": a taler is a thaler is a dollar and they all come from from the valley of Saint Joachim (Joachimsthal) where they had a silver mine. You may thank the Holy Roman Empire for making good coinage.

[https://en.wikipedia.org/wiki/Thaler](https://en.wikipedia.org/wiki/Thaler)

</comment></comment><comment><USER21> This is the most political open source project I've seen in a while. I love the idea that it's designed to sit inside a progressive society where we all agree we have a social contract to the state. That is _very_ different to a lot of similar efforts, and at the very least it's an important part of the discussion.

Right now it's vaporware, but I'm very interested in its release. I'd love to help give it a friendly face and integrate it with existing open source social platforms, to make it easier to send and receive money regardless of your technical ability.

<comment><USER22> Where is this social contract? What are its terms and where is my signature?

<comment><USER23> In the case of modern countries: The contract is the law + the Constitution (or your local equivalent if not in the US) and your signature is living in and benefiting from the nation/society you live in.

</comment></comment></comment><comment><USER22> From [http://taler.net/governments](http://taler.net/governments)

> _Thus, Taler will enable competition and avoid the monopolization of payment > systems that threatens global political and financial stability today._

The irony.

</comment><comment><USER24> I'm not sure how desirable the new features are compared to existing currencies. It also doesn't seem any freer. Don't know what the association with GNU entails. Furthermore, its name is easily confused with the ancient denomination of "Thaler", I guess it's on purpose, but, once again, maybe not the greatest idea.

If they want to centralize a currency, perhaps that would be better done by a government or other experienced institution.

<comment><USER25> As near as I can tell, they aren't trying to be a digital currency, just a digital payment system.

This is really an iteration on PayPal (and similar businesses): the new feature is that it is a system for letting multiple PayPals compete. Instead of being forced to make an account with a specific company, you can pick from a hopefully-large number of mints, who will be competing on things like fees and the customer experience. That will hopefully make things suck less.

BitCoin solves a lot of problems most people don't care about. Most people don't care about Byzentine Generals or trustless transactions or very limited forms of anonymity. What most people care about are low-cost transactions which don't suck. We'll have to wait and see if Taler can actually deliver.

<comment><USER24> >a digital payment system

Existing cryptocurrencies already perform this function well. If that's the sole claimed advantage of Taler, it's a step backwards.

<comment><USER25> Cryptocurrencies (by which I mean, Bitcoin, since it is the one with significant adoption) perform this function, but not particularly _well_.

For a payment system, being a currency is a negative -- fluctuations between the value of the digital currency and the value of the currency goods are priced in is a headache few people enjoy.

A payment system that is set up as a payment system -- instead of a decentralized trustless network -- can also better optimize things like transaction time (transactions can happen ~instantly if you're willing to trust central agencies) or having a known transaction cost. Right now Bitcoin's transaction cost is ¯\\_(ツ)_/¯ -- some combination of inflation from printing money to pay the miners and direct payments to the miners per transaction and payments to the exchanges to put cash in or take cash out and the risk of currency fluctuations mid-transaction. Regardless of whether it costs a little more or a little less, just _knowing_ the transaction cost is a huge benefit to consumers and merchants.

</comment></comment></comment></comment><comment><USER26> Why, as a rational actor, would I voluntarily choose this system over one that gives me the freedom to decide what the government sees about my spending?

<comment><USER27> _> Why, as a rational actor, would I voluntarily choose this system over one that gives me the freedom to decide what the government sees about my spending?_

Because this system ensures fair play of all participants, therefore making cooperative behaviour the rational choice, therefore enabling more efficient solutions on a societal level.

Proper game theory (which I assume you were referring to) is bit more complex than the prisoner's dilemma.

<comment><USER26> It doesn't ensure fair play of the most powerful participant (the government), and it doesn't ensure fair play of people who have the knowledge or material means to escape surveillance.

</comment></comment></comment><comment><USER28> Much love and respect to Richard Stallman / the GNU Project for all the pioneering work and advocacy they did in OSS, but I personally don't want them near my $s. The number of religious / flame wars persistently raging in OSS circles just can not, would not be tolerated in this domain.

<comment><USER29> Your comment is completely baseless FUD, I wonder if you actually believe it.

P.S. Stallman is a FS advocate, not an OSS advocate.

<comment><USER28> Ah, trolling a comment that is anti-flamewars. so meta-cool!

Sorry, FS, that was indeed my mistake.

<comment><USER29> You're still not making any sense.

</comment></comment></comment></comment><comment><USER30> Lets make it quite simple.

If you can't mail order LSD with it, in privacy, then it's too submissive to the state.

</comment><comment><USER31> _Unlike BitCoin or cash payments, Taler ensures that governments can easily track their citizen 's income and thus collect sales, value-added or income taxes. Taler is thus a currency for the mainstream economy, and not the black market._

Sounds pretty useless to me then. I would think, as technologists, we would adhere to a mindset of "we can do better". That is, we can do better than basing our society on a State apparatus that is rooted in the use of violence (or threat, implied or otherwise) of violence, to coerce people into following its dictates. Taxation is damage that we should be creating ways to route around, not something we should be building tools to support.

<comment><USER32> I will take nonsense like this seriously when its proponents no longer avail themselves of the benefits of assets created through taxation. Which would be never.

<comment><USER33> This is just the reverse of the (also bad) argument that people who advocate for higher taxes shouldn't be taken seriously unless they voluntarily donate some of their income to the government. For example: [http://bleedingheartlibertarians.com/2012/04/want-higher- tax...](http://bleedingheartlibertarians.com/2012/04/want-higher-taxes-pay- them-yourself/).

<comment><USER34> Thus taxes should never change as people would rationally never want to pay more nor receive less benefits. Hence we must have exactly the right amount of taxation at all times.

</comment></comment></comment></comment><comment><USER35> _Unlike BitCoin or cash payments, Taler ensures that governments can easily track their citizen 's income and thus collect sales, value-added or income taxes. Taler is thus a currency for the mainstream economy, and not the black market._

HAH, I love it, they lump both BitCoin _and cash!_ as currencies fit for the black market! I wonder how cash-only convenience stores will feel about that, those outlaw troublemakers! ;p

</comment><comment><USER36> > Unlike BitCoin or cash payments, Taler ensures that governments can easily > track their citizen's income and thus collect sales, value-added or income > taxes. Taler is thus a currency for the mainstream economy, and not the > black market.

This simultaneously implies that both Bitcoin _and_ cash are not suitable for the mainstream economy. I think the opinion that cash should be given up would be fairly controversial, even today. Furthermore, a law abiding citizen will still pay taxes on their Bitcoin income - the same way they would for cash. Just because Bitcoin makes it harder to prove tax evasion doesn't mean we should eschew Bitcoin and cash. The technology exists regardless, governments will have to adapt.

> When you pay with Taler, your identity does not have to be revealed to the > merchant. The bank, government and mint will also never learn how you spent > your electronic money. However, you can prove that you paid in court if > necessary.

This feature is offering nothing not already available, although it seems to imply that one could not prove payment with Bitcoin, which is false, as it is trivial.

<comment><USER37> > This simultaneously implies that both Bitcoin and cash are not suitable for > the mainstream economy.

No, it implies that Bitcoin and cash are both alike in not simultaneously being:

(a) suitable for the mainstream economy, and (b) not suitable for the black market.

It neither states nor strongly implies which of those two things bitcoin or cash fail on (nor does it state or strongly apply that they each fail on the _same_ one of those two things), though there's a fairly weak implication that Bitcoin and cash both fail on the second (that, whether or not they are suitable for the mainstream economy, they are quite suitable for the black market.)

</comment></comment><comment><USER38> I have this strange idea that once the government-agents really understand what e-payments, even block-chain-powered e-currencies bring to the table, in terms of money control, they'll embrace it. It's a no-brainer.

That said, taxes are not inherently bad. Governments on the other hand... Have an awfully bad track-record of power abuse.

<comment><USER39> Several central banks are interested in cryptocurrencies because it allows them to devalue the currency from a central point. Cash is hard because it can be hoarded, therefore indirect measures are taken to devalue said currency.

[http://www.finextra.com/news/fullstory.aspx?newsitemid=27870](http://www.finextra.com/news/fullstory.aspx?newsitemid=27870)

[http://in.rbth.com/economics/finance/2015/09/21/rusias- bitru...](http://in.rbth.com/economics/finance/2015/09/21/rusias-bitruble-to- be-worlds-first-state-controlled-cryptocurrency_426465)

[http://cointelegraph.com/news/115312/azerbaijan-mulls-its- ow...](http://cointelegraph.com/news/115312/azerbaijan-mulls-its-own- cryptocurrency-cryptomanat)

<comment><USER40> "If the controls we've imposed aren't working to revive the economy, it must mean we need to use new and ever more intrusive ones!"

<comment><USER39> "It is difficult to get a man to understand something, when his salary depends on his not understanding it."

Seriously. You have an aging population causing consumption to decline, a global recession along the horizon, ever-increasing productivity replacing the need for labor, and you think your interest rates are going to a damn bit of good? We'll need to keep interest rates at 0 just to keep the whole thing going, forget about negative interest rates (which is going to cause a run on the bank and asset bubbles in commodities, real estate, anything else that can be a value store).

</comment></comment></comment></comment><comment><USER41> Not sure how one can both have traceability with merchants and: "However, we also want to stay out of the immediate personal domain, so sharing funds within a family or copying coins between devices should not be subject to monitoring by the state."

I'm guessing this means that you have either "free" trust (help from the thaler system to ensure fair transactions), or "no" trust -- essentially you hand over cash, and hope the other party gives you goods in return?

I have a hard time seeing how you could prevent "anonymous transfer of coins" from one device from the other -- across the Internet (provided the parties are prepared to risk fraud)?

</comment><comment><USER42> So inflows (their amounts) are public (i.e taxable), but outflows are not?

Interesting claim.

</comment><comment><USER43> Taxable & anonymous at the same time? The marketing of this sounds shady. I don't know if the "inventors" or whatever really know what they are doing...

<comment><USER35> It's in development at a university that's known for its computer science research and work: [http://www.inria.fr/](http://www.inria.fr/)

For example, Facebook has partnered with the university and it's also the birthplace of the OCAML language. They have some sharp people doing research work there, if this was a VC-backed company or a kickstarter, I would be much more skeptical and think it's shady. However, the bar at a university is much higher so we'll see what they do deliver and how it works whenever they deliver it in the next year.

<comment><USER44> TU Munich too!

</comment></comment></comment><comment><USER45> Is it just me - I cannot understand how it works? There are diagrams and explanations about banks and wallets but no actual "how"?

</comment><comment><USER46> I don't understand why people seem to constantly associate bitcoin with tax free. There are no inherent reasons why cash transactions are taxable or non- taxable, it just turns out that people don't want to be destroyed by the IRS so they make an effort to pay their taxes.

<comment><USER41> There's also the aspect of currency vs goods. If I've understood correctly, bitcoin is taxed as (virtual) goods in Norway, meaning that if you make money from price change, the tax is much higher than it would be for making money off of stock/currency etc (which would be 28% off the profit, or a 28% deduction from the loss). Just because some people would _like_ bitcoin to be a currency doesn't make it so (in the eyes of the taxman).

I'm not sure how/if thaler would change that -- but it's worth keeping in mind that "new" assets will be taxed according to how the government thinks it should be taxed, not according to how you might feel it should be taxed.

</comment></comment><comment><USER47> What's crucially important, and conspicuously unmentioned, is not taxes but monetary policy as a whole: can the central bank (or "mint") control the interest rate? Is inflation possible?

<comment><USER48> It's right on the home page:

 _Taler uses an electronic mint holding financial reserves in existing currencies. This means that Taler is not a new currency with the inherent currency fluctuation risks, but instead the cryptographic coins correspond to existing currencies, such as US Dollars, Euros or even BitCoins._

So, yes, inflation is possible, since the digital coins are tied to existing currencies.

</comment></comment><comment><USER49> sounds like a shitty version of OpenTransactions

<comment><USER50> even shittier?

</comment></comment><comment><USER51> This seems to bake in some curious assumptions.

</comment><comment><USER52> This is good but how does it compare to internetpeso.com?

<comment><USER53> That site seems to give no details about how internetpeso works; it confuses "how does it work?" with "how do I use it?".

Also, Taler seems to be very much a Free Software project. internetpeso seems to be proprietary, and also self-contradictory:

> Other cyber-currencies require that you trust complete strangers to verify > transactions. This is a patently unsafe way of handling your life savings.

> We leverage best-of-breed closed-source software

<comment> ryan-c I suspect that is a parody and you have been trolled.

</comment></comment></comment><comment><USER54> Is the word "liberal" being used ironically?

<comment><USER55> Liberty is to be under no restraint apart from standing rules to live by that are common to everyone in the society. Persons have a right to follow their own will in all things that the law has not prohibited and not be subject to the inconstant, uncertain, unknown, and arbitrary wills of others.

Is that a acceptable definition for you?

</comment></comment><comment><USER56> This is a sheep that goes walking to the wolf and hopes to be devoured.

</comment><comment><USER57> This just seems like the Hurd of digital currencies.

</comment><comment><USER36> So GNU just released FedCoin? Interesting...

</comment><comment><USER58> As I understand the scheme, I inform the mint I will have, say $200, wired to it. It gives me a code which I give to my bank when they do the transfer, so that the mint knows the money I told it would be arriving did actually arrive from me. I then get a cryptographic assertion that I have $200 in my wallet. Anonymous purchases work, as when I go to buy something all the information I need to give the seller is the assertion I have the available amount of money, and essentially the seller bills the mint for the cost of the game, which the mint transfers to the seller's bank. The first question is how is bookkeeping done so that money can't be spent twice? One option would be that when I go to buy, say a $60 video game the system cryptographically bounds the purchase to my wallet, so if I go to another retailer it says that I have spent $60 of the original $200 in the wallet, and therefore have $140 left. The other option would be that any transfer into my wallet can only be used once and in full. The discussion does say the wallet holds a transaction history, but that doesn't delinate which of the two previous modes of operation is the case.

There are a few issues with this if using the "wallet-bookeeping option" where proofs-of-purchase are bound to the wallet. One, it doesn't protect against dishonest sellers. Say a customer comes in with a wallet that originally held $200 but they've spent $80. The seller can still bill the mint for $150, and there is no way for the mint to confirm that the customer didn't have that much still left over from their transfer to the mint. Secondly, presumably the proof-of-purchase is cryptographically bound to the wallet, else the buyer will simply discard it. With it bound, discarding the proof-of-purchase discards the remaining money on it. However, for this to work, there must be no recourse for the buyer to recoup the money in a lost wallet. But the user can do this by backing up (which the discussion of Taler notes can be done), which would allow the user to simply discard the version of the wallet with a payment and simply return to the one without it.

Because of this, it seems likely that Taler would make any transfers one-use only, to prevent against this issues. In such a case, the user would then transfer the exact amount of funds needed (or something close, hoping to get change in cash, which wouldn't be sustainable for the sellers if all their buyers are using Taler) as needed right before making their purchase. In that case, it is fully reasonable for the mint to be able to link purchases to bank transfers, since there will only be so many $273.67 transfers (remember, the mint knows who the transfers are from) followed within the next five minutes by $273.67 purchases (and the mint will know the name of the seller).

Fundementally, the issue with all these schemes is that for the mint to not be able to track the purchase, it needs to be able to decouple the processes of taking deposits and paying out purchases among the folks who have money in the mint. It can do this either by making the money holder responsible for the bookkeeping, opening the door to fraud, or to make (as Chaum's Digicash did) each certification a one-time use only, which incentivizes buyers to transfer funds as needed, which allows for at least partial linkability, probably not something which would standup in court in and of itself, but would probably be sufficent circumstantial evidence in a number of cases.

I'm wondering if I'm misunderstanding something, since these issues have all been pointed out with previous anonymized e-cash attempts, and it seems unlikely GNU would go to build a scheme that has these well-known weaknesses. Hopefully I'm wrong, and there is some additional information here that I'm misunderstanding or has yet to be presented. Also, for a useful read on this type of thing see Brands' criticism of Chaum's pseudonym approach in "Building in Privacy: Rethinking Public Key Infrastructrures and Digital Certificates" (pgs. 25-32, although the whole intro is very useful).

EDIT: Making my points more clear, got pretty jumbled in my original post...

</comment><comment><USER59> Free software for an unfree currency.

<comment><USER60> Very well said. As long as our monetary system is not freed from the grasp of the government/central banks/banks in general, reasonable currencies will never be widespread... Sane money is money that cannot be printed in arbitrary amounts. The money monopoly is particularly harmful to economy & citizens.

<comment><USER61> Thankfully there are not vast numbers of people that buy into that string of assertions.

</comment></comment></comment><comment><USER62> I don't mean to be a dick, but the default bootstrap look is not making it easier to take this as a serious project.

But I like how this is not a giant project to waste electricity breaking useless hashes.</comment>


Shakti, the new data platform from Arthur Whitney - <AUTHOR> https://news.efinancialcareers.com/uk-en/3002752/shakti-arthur-whitney <comment><USER2> Morgan Stanley in recent years has used this where they'd previously used kdb:

[https://github.com/Morgan-Stanley/hobbes](https://github.com/Morgan- Stanley/hobbes)

From the article, it sounds like the Kx folks are working on the issues that led to the development of hobbes in the first place.

I'd be surprised if they've changed their opinions about type systems though (or bullying people with lawyers — those guys get nasty when their ideas are challenged).

<comment><USER3> Does hobbes have any kind of db or persistence functionality? As a programming language, k/kdb is nice (and I personally prefer it to APL and J), but without the db part I suspect that it wouldn't have been as popular (in the niches that it is popular).

Seems like a mostly eager haskell; am I correct? How does it compare performance-wise to ghc? (It obviously does much better on embedding and interfacing C++)

<comment><USER2> Yes there are persisted data structures and databases built on top of them. The whole type system applies to persisted data structures so you can make your own data structures and fill out a few type class instances to query them with plain list comprehensions (e.g. btree indexes are defined and queried this way in some internal tools).

Eager Haskell with a structural type system, deterministic garbage collection, and no overloading penalty, that would be a fair summary.

It'd be interesting to see how GHC would do in the same area, there would be some significant challenges. I haven't tried to use GHC there so I can't say.

</comment></comment></comment><comment><USER4> An evaluation copy is available for installation via Anaconda:

[https://anaconda.org/shaktidb/shakti](https://anaconda.org/shaktidb/shakti)

The tutorial mentions _streaming_ and _distributed_ capabilities, but then doesn't cover them:

[https://shakti.com/tutorial/](https://shakti.com/tutorial/)

Having used q/kdb+ a bunch in the beginning of my quantitative finance career, I hold a special place for Arthur's languages. I look forward to what Shakti develops into.

<comment><USER5> How widely used is kdb+ today?

</comment></comment><comment><USER6> When this was first announced, they used the word "blockchain" a lot. It's funny that this is now deleted, as they've learned the word is credibility poison.

</comment><comment><USER7> Nitpick from the article: it suggests kdb underpins most algorithmic trading systems. This is simply not true. I've worked in a number of banks and hedge funds, and most have their own home-brew storage and analysis engines (I've helped build a few myself).

</comment><comment><USER8> A group of k enthusiasts working under the 'kparc' brand has been writing docs for this new in-development k:

[https://kcc.kparc.io/](https://kcc.kparc.io/) [https://ref.kparc.io/](https://ref.kparc.io/)

</comment><comment><USER9> The submitted URL was [https://shakti.com/database-software- history/](https://shakti.com/database-software-history/).

Their press page links to [https://news.efinancialcareers.com/uk- en/3002752/shakti-arth...](https://news.efinancialcareers.com/uk- en/3002752/shakti-arthur-whitney), which reads like PR but arguably contains a bit more information (e.g. that it was released last month), so I guess we'll link to that instead.

</comment><comment><USER10> It seems like Shakti is just the k language version 7 as per README at [https://github.com/kevinlawler/kona](https://github.com/kevinlawler/kona)

I remember reading an interview with Arthur and he said that he writes each version of k from scratch. Most likely this version has also been written from scratch.

</comment><comment><USER11> It sounds like they are using the AVX primitives to accelerate certain operations, when they talk about 256 and 512 bit wide instructions.

<comment><USER12> Maybe this is why shakti fails to start when I have tried it with 1.6 GHz Intel Atom and Celeron CPUs. When I tried on Ubuntu with Celeron I got "Illegal instruction (core dumped)"

<comment><USER13> I doubt there would be a hard requirement on any AVX instructions, just that it can take advantage of them if available.

<comment><USER14> The version I downloaded from the site does (did?) depend on AVX instructions and would die on startup. There are other ('nightly builds') that fix this. One of the goals of the current team seems to be portability.

<comment><USER15> FWIIW J is moving in this direction as well; the last release version of it (807) needed a special build for older intel hardware (they subsequently made separate binaries for older hardware). AVX has been around since 2011/2012, so it doesn't seem like that big a deal.

Admittedly I myself have older hardware kicking around I see no real reason to upgrade, but AVX is such a big win on array processing you might as well make it a dependency for the kinds of use cases Shakti or J get used for in business.

</comment></comment></comment></comment></comment><comment><USER16> As a k email list reader i noted in

[https://news.ycombinator.com/item?id=21660005](https://news.ycombinator.com/item?id=21660005)

that shakti(k7) MAY become FOSS...

<comment><USER17> highly unlikely it'll go that route IMO, the messages in google group are just wishful thinking

</comment></comment><comment><USER18> I'm super skeptical of these articles about programmers with superhuman powers.

<comment><USER19> I suggest reading the J interpreter Whitney wrote in a weekend and reconsidering!

[https://code.jsoftware.com/wiki/Essays/Incunabulum](https://code.jsoftware.com/wiki/Essays/Incunabulum)

<comment><USER20> Writing in that style is not that hard, I do something similar if I need to make a prototype of something. The short variable and function names really save time as long as you remember what is what. Reading or debugging it even a week later is torture tho.

Writing the interpreter in an afternoon is impressive but most likely he had done something similar before and had most of the algorithm in his head before he even began.

You don't need to be superhuman to do that.

</comment></comment></comment><comment><USER21> I'd like to see some real metrics. All the words are subjective and the technical ones are common to other databases as well. How many results can it return in how many seconds from a table with how many rows? Can it do full text indexing? Can it aggregate?

It feels like something from wallstreet, the language is almost pump and dump "I have this really great stock tip from this guy who made millions once, he's a legend. He says buy, so you should."

We need more tangible and quantifiable information about this database. Until then it's not even hype it's vaporware.

<comment><USER22> Calling anything from Arthur "vaporware" is really laughable.

This article isn't great, but it's from a finance site, not a technical site: of course it's going to be low on technical details. You would think that people who want technical details would know how to use a search engine.

Further, kdb/kdb+ (which Shakti is the successor of) is a columnar-based database. "How many results can it return [...] from a table with how many rows?" is completely irrelevant.

There are hundreds of benchmarks for kdb+, and the speed of k is unrivaled.

<comment><USER23> >There are hundreds of benchmarks for kdb+, and the speed of k is unrivaled.

Eh, I worked on array languages in grad school and the speed of k was significantly over-hyped. It's trivial to beet with Python and no contest if you want to work in a compiled language. Wall Street develops weird religions around tech.

<comment><USER3> In my experience it is mostly trivial to beat with C if you adopt columnar storage (which isn't idiomatic), but even numexpr doesn't beat k (though it seems like it should), and numpy is still way behind; pandas and plain python not even in the same ballpark.

What approach did you use?

The one thing nothing beats k on is functionality per time spent (if you are proficient). It is often 100 times shorter but only takes 10 times as long to write.

<comment><USER23> I'm going to sidestep arguments about its terseness since some people are into that and it's really an aesthetic choice. But for machine learning or numerical algorithms, I found it quite hard to make k nearly as fast as vanilla NumPy code. Overall there's really no good reason for it to be faster. It didn't have (as of ~2011) a JIT or any mechanism for operator fusion, so you end up allocating many intermediates, sometimes using quadratic space which gets immediately reduced along some axis.

Yeah, I know "Arthur Whitney is really smart"...but that's not actually a technical reason why k would be faster. I don't remember the specifics but I think that many of the array operators weren't even multi-threaded and I remember only a few of the array operators getting FLOPs that would correspond to SIMD acceleration. So, for the most part, it seemed like a lot of vanilla single threaded C code behind the operators.

It's possible that the implementation got smarter over the past decade, but when I was working in the space it seemed like the hagiography and actual performance were worlds apart.

<comment><USER3> I didn't try modern machine learning stuff, so I cannot comment on that.

It doesn't do operator fusion or JIT, but neither does numpy ... numexpr, numba and pythran do, but you were referring to "plain python" in your original post.

Indeed, K is mostly vanilla single threaded code. When I started using K, i was also thinking "neat, simplifiied APL, but it won't be fast" (I had previous positive experience with APL). But it was way faster than plain threaded code should be. The reasons, AFAICT at the time were:

1\. Access patterns are extremely predictable, which means much of the data is in L1 much of the time; That, on its own, delivers a factor of ~10 speed compared to having the same data scattered in L2 if you are memory bound.

2\. The code is incredibly small and in 2003 when I looked at it, mostly fit inside the I-cache, which meant even the branch prediction available at the time was doing way better than one would expect.

3\. The primitives are very well chosen and play well together.

The hidden elephant in the china shop is that your code has to be idiomatic for these to matter. See e.g. [0] from Stevan Apter, for aggregating functions in a toy database e.g. group[avg] - the version he uses, vs. 4x slower one in the comment.

I have no doubt it is possibly to beat k with numpy; it might even be easy if you are more proficient with the latter than with the former. However, for the kind of exploratory data work I did in the day, it is very hard to beat overall when you take thinking, debugging and experimenting into account. And if/when you have converged onto a well-enough defined computational pipeline, the best tool for that specific job (C++, CUDA, TF, MKL) is often the only reasonable choice.

> when I was working in the space it seemed like the hagiography and actual > performance were worlds apart.

I don't know who you were talking to, but for me it was always an optimization on the "effort/result" metric, not on either on its own. Python often requires less effort, but iteration runtime speed (even with numpy) is horrible. C+CUDA delivers faster results but the iteration development speed is horrible. For me K struck a sweet spot. YMMV

[0] [http://www.nsl.com/k/t.k](http://www.nsl.com/k/t.k)

</comment></comment></comment></comment></comment></comment><comment><USER24> q is an interesting language and am using it for the advent of code. While the language superficially seems simple, it features a lot of hidden complexity and dubious design decisions. Every operator is massively overloaded on rank and type. There are functions that accept functions or lists or atoms as arguments: supposedly there's supposed to be mathematical commonality, but coming from the statistically typed FP world, this kind of polymorphism makes me extremely uncomfortable.

The dynamic typing bugs me as well, it would really help to have an editor be able to tell ahead of time whether an expression is valid.

Speaking of valid code, the error messages are absolutely terrible and stink of arrogance. I'm talking about ed level error messages: no words, no pointing out the error, just some one letter message that is supposed to indicate a type or function argument error.

A lot of the culture of the language is built in Arthur Whitney's own image: error messages are bad, comments are for stupid people, white space is for "Qbies"

Speaking of cultural, the jargon of the language is didactic and annoying. All over the documentation references to monadic or niladic or diadic functions. Also calling things verbs or adverbs or nouns is annoying and unhelpful. Even in the book "Q for mortals" the author prizes using different terminology than literally anyone else and strikes me as trying to be deliberately obtuse. He even spends an entire chapter talking about the mathematical nature of natural numbers, which strikes me as both unhelpful and narcissistic.

Going back to the language, not only are functions overloaded to a ridiculous level, but the syntax of the language itself is inconsistent. Multiple ways to call a function, all in the name of reducing unnecessary typing. Also the syntax and name of some functions strikes me as very add: '/' is not used for division, but instead for both comments and folds. ',' is used for appending lists, while ';' is used to separate function parameters. Instead of parents, Q uses '[]' for function parameters. Here's an example of finding the square of the hypotenuse in idiomatic q:

pyth_sq:{x^2+y^2}

One can use xyz without explicitly declaring them as parameters, which is both nice and arbitrary. I would much prefer to have something like Scala's underscore or maybe something like _1,_2,... Here's the function written with explicit parameters:

pyth_sq:{[x;y]x^2+y^2}

Personally I think a syntax (and spacing) like:

pyth_sq = { (x, y) => x^2 + y^2 }

would be nicer.

While the naming of the functions and syntax is arbitrary, I beginning to understand the rational and design of the language. For Whitney, code is the root of all evil: less code, less bugs. Also less code, the more you can fit in your head. This is the reason why everything is overloaded I think: because Whitney ran out of ASCII symbols and didn't want to use names (more information to remember).

Regardless, the real reason people use q isn't because of the language, it's often despite of it. The kdb+ database and q sharing the same memory space is the real advantage. The performance of the system is so exceptional that a single server can outperform even a large Hadoop cluster: the code is out of this world optimized.

I've began to think that q is a natural and easier to use extension of Chuck Moore and Forth's philosophy and language. The languages both share the same tenets of performance, terseness, and minimal syntax. While forth works on a fixed number of stacks, q works on a arbitrary number of n-dimensional vectors, a natural extension to the one dimensional stack.

Anyways, sorry for the rant. I'm still learning q so take what I say with a grain of salt. I can imagine an expert q developer becoming extremely productive and comfortable: I'm personally just not there yet. When I finish advent of code, I'll have a better perspective.

<comment><USER25> > Speaking of cultural, the jargon of the language is didactic and annoying. > All over the documentation references to monadic or niladic or diadic > functions. Also calling things verbs or adverbs or nouns is annoying and > unhelpful.

That is a valid opinion, but this is not a k thing. The same jargon is used by APL, J and practically any other array programming language. In fact, the term monad was first used in this context [1].

[1] [https://en.wikipedia.org/wiki/Monad_(functional_programming)...](https://en.wikipedia.org/wiki/Monad_\(functional_programming\)#History)

</comment></comment><comment><USER26> Ten trillion rows, zero information, nice</comment>


Ipv6 traffic just shy of 10% of total - <AUTHOR> https://www.google.com/intl/en/ipv6/statistics.html#tab=ipv6-adoption&tab=ipv6-adoption <comment><USER2> The per-country stats are interesting. US passed the 20% mark some time ago. Seems like some Canadian ISPs have started deploying IPv6 (other than beta), since it's now at 6% (it was around 1% for a long time). UK, Spain and Sweden are surprisingly low.

On the corporate side, it's annoying that Github, Twitter and OSUOSL (which hosts a lot of Free Software projects) still don't have IPv6 enabled (although I was told 2016-Q1 for OSUOSL).

<comment><USER3> I heard a Dane say the Nordic ISPs had enough 'foresight' to hoard enough IPv4 addresses for the foreseeable future.

<comment><USER4> Yes the major Swedish ISPs also say the same thing, except for the "foresight" part. One of our ISPs is even giving out "unlimited" public IPv4 addresses on their consumer subscriptions (previously they had a limit on 5 addresses, but now they have apparently removed the limit).

The ISPs even refuse to offer IPv6 when it comes to "big" customers, like apartment cooperatives with hundreds of apartments... You have to turn to the small/local ISPs for (consumer) IPv6 here.

<comment><USER5> Any way for someone outside of Scandinavia to purchase/rent some of these public IPv4 addresses?

VPS? Datacenter?

<comment><USER4> No, it's for consumer subscriptions only, and commercial use is prohibited. On the same ISP, commercial users have to pay depending on IP address block size and justify how they will use the addresses etc.

</comment></comment></comment></comment></comment><comment><USER6> This is almost entirely attributable to v6-only mobile networks, right (at least in the US)? Anybody know what the stats look like if you exclude those, and only look at fixed connections?

<comment><USER7> I don't think so, Comcast residential is dual-stack nowadays so that would be significant traffic.

<comment><USER8> ...assuming that the router on the premises supports IPv6-PD, that RAs to the LAN has been turned on, and that the machines on that LAN haven't had their IPv6 support switched off... of course. :)

<comment><USER7> That's all true, but I think that's a more normal case than it might seem.

I've been running IPv6 on Comcast for two years using one of their recommended Motorola (Arris) Surfboard modems and an Airport Extreme, and didn't have to configure anything at all.

Given a typical user that doesn't know what IPv6 even is, if they just plug everything in and have a Mac or recent Windows box it should all just work.

<comment><USER8> Yup. Apple routers know how to get and advertise IPv6 allocations. IIRC, they'll even establish a (6to4?) tunnel and advertise space in that if the upstream network doesn't have IPv6 service.

New gear is substantially more likely to have good (and on-by-default) v6 support than old gear, but there's a lot of old gear out there as well as many cargo-cult sysadmins. :)

</comment></comment></comment></comment></comment><comment><USER9> This isn't 10% of traffic. Merely that 10% of connections to google were from IPv6 capable clients.

<comment><AUTHOR> D'Oh! thanks

<comment><USER10> That isn't right. "The graph shows the percentage of users that access Google over IPv6." So the graph shows the percentages of users who accessed Google over IPv6.

</comment></comment></comment><comment><USER11> Are there any benefits to a website supporting IPv6?

I'm looking at the "per country" graph, which highlights latency issues, but it looks like they wouldn't really be noticeable to the user. However, is there much point when IPv4 works fine?

(Perhaps 10% of our users are from the countries marked in pink/red on Google's map.)

<comment><USER12> It doesn't really work fine - you're just used to (or don't notice) all the ways that it doesn't work. We ran out of IPv4 addresses a while back, so we're doing all sorts of stupid stuff like selling addresses, dynamic IPs, and address translation to distribute and squeeze more life out of the ones we have. This makes a lot of networks much more complicated than they really should be, and will only become a bigger problem as more devices come online.

Universal IPv4 would enable some cool stuff, like giving every device on the planet a unique IP (or even several to increase anonymity), as well as vastly simplifying many common networking problems. Here's one example: Have you ever tried to use a home internet connection to host a server, say for home automation, file sharing or gaming? It's a huge pain for anybody who isn't network-inclined. By giving everything a unique IP, routing setup for this sort of thing becomes much easier since every device has a unique, static IP address that it can always be found at.

Also, IPv6 has some security and encryption features that are not found in IPv4, which should basically end packet spoofing as an attack. It's also a bit faster too, because packet integrity isn't as rigorously checked and routing is simpler.

<comment><USER11> We have enough IPv4 addresses (1024?) that every device and VM has a public IP.

Even if we were short, would we necessarily want to enable IPv6 on the webserver? I'm interested to know if it makes any difference to users:

\- Does it make things noticeably faster (or slower) for any?

\- Does it totally break (or fix) access to the website for anyone?

Faster for many but slower for a few would be OK, but faster for some but broken for a few wouldn't be acceptable.

<comment><USER2> If your infrastructure has less than 5 years projected life-span, maybe you don't need to make it a #1 priority. Otherwise, if you don't start planning IPv6 now, in 5 years your infra is likely to be seen as an aging infra with too much technical debt. (similar to people just starting to realize they should probably adopt https)

One major advantage of IPv6 for the web I run into regularly: debugging attacks/issues/traffic from folks behind carrier-grade NAT is extremely annoying.

</comment></comment></comment></comment><comment><AUTHOR> I've been following this for months, and by my projection it should have cleared 10% this past weekend, but it didn't... Recent weekly increases have been very impressive.

</comment><comment><USER13> Maybe these trends will actually drive AWS to support externally addressable IPv6 for EC2 instances.

<comment><USER14> Same goes for Google's own Compute Engine, which doesn't support IPv6 at all.

Curiously Google's managed MySQL instances receive IPv6 addresses by default, but I believe that's the only component of their service that does.

</comment></comment><comment><USER15> I don't use IPv6 even though my ISP supports it, because every time I try to enable it, it causes all my network connections to randomly choke.

This has happened with every Linux desktop I've ever used. If IPv6 is enabled, then half the time I try to visit a website, the connection doesn't go through. No error either; it just sits there forever and ever. It's particularly aggravating when AJAX is involved--you get to experience Google Maps stop working for about five minutes while you're using it.

I've observed it across multiple Linux distributions for several years now. I've learned to _always_ add "ipv6.disable=1" to my sysctl.conf if I want a working network.

<comment><USER8> That's very, very, _very_ strange. I've been running IPv6 (first through a tunnelbroker.net tunnel, and -much- later using native IPv6 through Comcast) for roughly eight years now. Haven't had any issue with Linux, Mac, or Windows systems.

To diagnose a thing, what happens when you disable Router Advertisements on your router, then enable IPv6 on a machine, and run Teredo [0] on that machine? (After Teredo assigns an IPv6 address to the teredo virtual interface, start a 'ping6 google.com'. It will take a few minutes for the Teredo machinery to figure out how to make a connection, but it should eventually do so.)

Are you -perhaps- rejecting or dropping ICMP on your router? If you are, don't. ICMP is _required_ to make IPv6 work.

[0] This is the Teredo implementation that people use on Linux systems [http://www.remlab.net/miredo/](http://www.remlab.net/miredo/)

<comment><USER15> I'll have to look into this sometime.

Thanks for the advice!

</comment></comment></comment><comment><USER16> 39,69 % for Belgium looks like to be the highest? One of the Biggest PSPs flipped the switch

<comment><USER17> Belgium has been top for a while. All ISPs support IPv6* here on home internet. Mobile not yet though.

* Some need a newer modem -- which might take a while before everyone switches.

</comment></comment><comment><USER18> Meanwhile my ISP still doesn't support it. And I'm paying $60 a month for 10/2\. 'Murica. (and thats not even bad, 3 years ago I was getting 3/1 for $40)

<comment><USER19> It was recently that I could even talk my ISP into giving me a static ipv4, so I'm not going to try and push my luck on ipv6 just yet.

</comment></comment><comment><USER20> Why is the chart go up and down?

<comment><USER13> If you zoom in you'll see higher IPv6 usage on weekends. This points to higher penetration on IPv6 on mobile and consumer (cable modem and DSL) networks, and lower penetration on work or corporate networks.</comment></comment>


Survey of research into the effects of pornography - <AUTHOR> https://aeon.co/essays/does-too-much-pornography-numb-us-to-sexual-pleasure <comment><USER2> Solid article backed up with research put in context.

TL;DR: Almost everyone watches porn, and people are starting to watch younger and younger. However, almost all research shows it's not harmful. Porn is also a reflection of the viewer and society.

</comment><comment><AUTHOR> There is also a related and thoughtful discussion on Aeon's own website: [https://aeon.co/conversations/is-it-wrong-to-watch- pornograp...](https://aeon.co/conversations/is-it-wrong-to-watch-pornography- and-i-don-t-mean-does-pornography-cause-harm)

</comment><comment><USER3> The referenced Newton episode is here:

[https://www.youtube.com/watch?v=2PeQG61W4i0](https://www.youtube.com/watch?v=2PeQG61W4i0)

The Scandinavian approach to sexual education goes a long way to ensuring children's confidence about their body and their feelings.

</comment><comment><USER4> What are the chances someone interested in researching about porn is going to find that porn is harmful?

</comment><comment><USER5> "More than 36% of the internet content is porn". Is this in terms of bandwidth, size of files, or some other metric? If it's size of files then of course porn in going to be a large percentage of files because video files are large.

</comment><comment><USER6> Slight NSFW warning (blurred tits).

</comment><comment><USER7> tl;dr: There is no evidence that pornography has ill effects, and a modicum of evidence that it can be positive. The (sometimes) conventional wisdom about porn -- objectification of women, its addictive nature, destructiveness in relationships -- is flatly wrong.

<comment><USER8> I read this as a single researcher (Prause) in a university with an agenda to push.

Sometimes conventional wisdom is conventional for a good reason.

<comment><USER9> You should look up Masters and Johnson. The first time any science at all was performed to test conventional wisdom showed conventional wisdom was mostly nonsense. Often nonsense passed down from one person to another with negative effects on people. That was mostly the physical stuff. Now, we need to know the psychological stuff.

As before, people's pre-existing beliefs and social structures make it hard to even test the conventional wisdom. Contradicting it is declared evil or foolish before-hand. Given history of "wisdom" in this area, it's right to test it in ways that might show contradictions. People have always been exceedingly foolish on these things when they cast judgment before collecting data. No reason to believe it will be different this time.</comment></comment></comment>


The SCRAM Authentication Protocol - <AUTHOR> https://www.cleeus.de/w/blog/2018/02/13/The_SCRAM_Authentication_Protocol.html <comment><USER2> Nice write-up of it, though I disagree that you can (or should) "recover" from a database breach in that way. If you detect a database breach, it's likely considerably after the event, and you should enforce password changes (and TOTP resyncs).

Also, there's no mention of Channel Binding, which adds considerable protection to MITM attacks aimed at obtaining the ClientProof off the wire.</comment>


Fake Obama speech is the beginning of the end of video evidence - <AUTHOR> https://boingboing.net/2017/07/17/fake-obama-speech-is-the-begin.html <comment><USER2> Fortunately this will counteract a total lack of privacy caused by ubiquitous microscopic drones and cameras. You come forward with a celebrity/political/ceo sex tape? There is no difference between the "real" tape and thousands of already synthetic ones; no one could know the difference. In other words, this technology will return privacy to everyone by allowing us to all "hide" in plain sight.

</comment><comment><USER3> I dare say deaf people who lip read could pick it, with forewarning I could see discrepancies.

<comment><USER4> Yeah same. Seems to struggle with sounds that require more vertical shapes, or closed lips for short periods

</comment></comment><comment><USER2> Politicians/business/others will have to cryptographically sign their official audio/video

</comment><comment><USER5> I don't know, this doesn't really impress me. It's not that hard to do using After Effects. Also, they're still using his real voice, if you're going to fake something like this, you have to have the person say something they never said.

<comment><USER6> look into project voco, a feature that was introduced to Adobe Creative product suite last year or so...you can make anyonr say virtually anything, by typing in text. This combined with video manipulation can be very effective.

</comment></comment><comment><AUTHOR> The researchers' own Web page: [https://grail.cs.washington.edu/projects/AudioToObama/](https://grail.cs.washington.edu/projects/AudioToObama/)</comment>


Don't Go Back to School: How to Fuel the Internal Engine of Learning - <AUTHOR> http://www.brainpickings.org/2013/05/13/dont-go-back-to-school-kio-stark/

<comment><USER2> Great find. Speaking from my own experience as a 5th year undergrad student set to graduate in May 2015...when I am in class I am eagerly watching the clock waiting to be dismissed so I can return to the web to learn something new. Whether it be a new Photoshop technique or learning simply through hacking something together out of thin air. School has lost its taste to me and many others around me...why is this? Why is creativity and expression not encouraged in the classroom? Graduation has now become a roadblock to my imagination and passion for entrepreneurial ventures rather than a goal and token of achievement. But we press on!!</comment>


Aaron Swartz "JSTOR" case indictment revised/expanded - <AUTHOR> http://sethf.com/infothought/blog/archives/001476.html <comment><USER4> Here, the government is pursuing criminal charges for someone who has taken research papers (funded, by and large, through public research grants) to presumably make them available to the public.

In doing so, he wronged various publishers, who have not financially supported any research, who have not financially supported the scientific review of said research, who have financially gained by (likely) not only charging the original author for the submission but also the universities which provided the infrastructure critical to the majority of research.

Spearheading this noble effort to right the wrong and restore law and order is US attorney Carmen Ortiz, who is cited for the wise words: _"Stealing is stealing"_.

Lets be reasonable here: a decently sized server farm could probably keep the entirety of documents hosted on JSTOR in RAM. Today alone, imgur burned through 50TB of traffic; it delivers a petabyte a week. I'm not going to believe a sob story about how distributing 100KiB PDFs to someone running 'wget -r' is DoSing their systems.

<comment><USER5> > Lets be reasonable here: a decently sized server farm could probably keep > the entirety of documents hosted on JSTOR in RAM.

One largeish science publisher I worked with had 9-10TB of (pdf) document data. In addition to that there's a search engine, and image versions (of everything) to allow for look inside. Then there's dynamically generated HTML for online view. Electronic publishers do with large amounts of data and it isn't wholly static content. I don't know much about JSTOR but I think it's safe to assume that the implementation is non-trivial.

<comment><USER6> I believe Swartz's point is that 10TB of data is not something you can build a self-perpetuating bureaucracy around, anymore, and that this implementation is, in fact, simple enough that it can be made available and maintained for a tiny fraction of the cost of JSTOR.

<comment><USER5> I'm not really talking about what Swartz's argument is, but you seem to have missed the point that there's more to being a publisher than holding data.

<comment><USER7> I don't understand your point either. Yes, publishers do more than merely hold PDF files. Ergo what?

<comment><USER5> Ergo keeping all the documents in RAM isn't a technical solution. I think you've mentally lost track of what I originally said.

</comment></comment></comment></comment></comment></comment><comment><USER8> Despite what Aaron's intentions were, he clearly knew he was doing something illegal. He broke into a building (closet?), setup a computer illegally and transferred information illegally. In addition to all of that, he tried to evade security when he knew he had been discovered.

It seems like there were many other (more reasonable) avenues that he could have taken to accomplish his goal. He's really the only one to blame for getting himself into this mess. I sincerely hope that he gets the punishment he deserves which should be a firm slap on the wrist. I'm uncomfortable with the idea of him getting any jail time, but if it comes to that then he'll have to deal with those consequences.

<comment><USER9> > I sincerely hope that he gets the punishment he deserves which should be a > firm slap on the wrist.

A felony, even without any jail time, is an irreversible and life changing punishment. He will no longer be able to vote or (possibly?) leave the country, or work at various organizations and corporations that automatically do not hire felons.

Edit: didn't know that felons could still get passports. They can have trouble getting visa's for sure though.

<comment><USER10> > He will no longer be able to vote or leave the country,

Why leaving the country would be a problem? I haven't heard of that one before. Say what if wants to fly to Europe for vacation, he can't because he was convicted of a felony?

<comment><USER11> this link says that's only in specific cases: [http://www.travelinsurancereview.net/2010/04/20/internationa...](http://www.travelinsurancereview.net/2010/04/20/international- laws-can-a-convicted-felon-travel-outside-the-us/)

interestingly, if you have a DUI conviction, you cannot enter Canada.

</comment></comment></comment></comment><comment><USER12> The government --- and a grand jury.

                       A TRUE BILL

                       b r e n d a   l    h a n n o n
                       _________________________________
                       Foreperson of the Grand Jury

        scott l garland             9 - 12 - <USER2>
        _______________             ___________
        Assistand United States Attorney

        DISTRICT OF MASSACHUSETTS
        September 12, 2012
        Returned into the District Court by the Grand Jurors and filed.

Granted: grand juries rarely refuse to indict. But it's not just the prosecutors behind this decision.

<comment><USER13> This made me wonder how often. <http://www.freedomlaw.com/archives/oldsite/GRANDJRY.html> says 0.6% in 1984.

<comment><USER14> Minor correction: 0.4% (the article reports a 99.6% indictment rate).

<comment><USER13> Thanks! I wrote that on an iPad at 3am and tried to check if I'd goofed, but it had no within-page search.

</comment></comment></comment></comment><comment><USER15> As Aaron is a founder of <http://demandprogress.org> I am not surprised by this. Their campaign emails are highly critical of the current political establishment fighting SOPA, PIPA, NDAA and related issues quite effectively through social media. This is deliberate over-reaction to petty protest crime. Like WikiLeaks, I imagine he has developed enemies due to these efforts. This entire situation reminds me of the treatment of political protest punk band Pussy Riot in Russia recently.

<comment><USER16> You need only reflect that one of the best ways to get yourself a reputation as a dangerous citizen these days is to go about repeating the very phrases which our founding fathers used in the struggle for independence.

\- - Charles Beard

</comment></comment><comment><USER17> Copied from "Show Dead"

To me the Largest Sadness is that he did not succeed in releasing the corpus to the public, which I presume was his real goal. Making this information accessible would be one of the largest contributions an individual could make to the world, possibly worth the trade of spending the rest of one's life in prison. The tragedy is doing so without the reward.

The only greater contribution I can immediately come up with that is definitely accessible to some of the readers of this board would be leaking the Google Books archive. Depriving the world of life saving knowledge for purpose of profit is as great a sin as genocide. Sometimes conscience must trump the law.

Sincerely,

A long time member resorting for the first time to an anonymous account.

</comment><comment><USER18> As a result of Swartz's conspiracy to release JSTOR articles to the public, it led to the company making many articles free for public for viewing. A lot of good has come about from what Swartz did, even if the morality of his illegal actions are under dispute. There has been a lot of recent criticism against academic journals and their publishing companies whose business model is keeping all science and research behind gated walls and away from the public. Even if the research was funded with public money!

<comment><USER19> _it led to the company making many articles free for public for viewing_

That is a nice result indeed, do you have a reference?

<comment><USER14> They've decided to make pre-1923 articles from American journals, and pre-1870 articles elsewhere, available publicly (w/o any subscription or even a free account needed): <http://about.jstor.org/service/early-journal-content-0>

Their original FAQ on it used to have a question about whether this was related to the "Swartz situation", where the answer could be paraphrased as "no but sort of yes", basically that they had been planning it all along but may have moved up some initiatives in response to the publicity. Doesn't seem that their current FAQ has that question anymore.

</comment></comment></comment><comment><USER20> _One may well ask: "How can you advocate breaking some laws and obeying others?" The answer lies in the fact that there are two types of laws: just and unjust. I would be the first to advocate obeying just laws. One has not only a legal but a moral responsibility to obey just laws. Conversely, one has a moral responsibility to disobey unjust laws. I would agree with St. Augustine that "an unjust law is no law at all."_

Martin Luther King, 16 April 1963, "Letter from Birmingham Jail"[1]

[1] [http://www.africa.upenn.edu/Articles_Gen/Letter_Birmingham.h...](http://www.africa.upenn.edu/Articles_Gen/Letter_Birmingham.html)

<comment><USER21> The Nuremberg laws are a prime example of laws that were so intrinsically wrong that disobeying them was a moral imperative.

</comment></comment><comment><USER22> Could someone who knows about such matters comment on whether this is the usual consequence for what he did, or if it's likely that he's been given "preferencial" treatment for his political activism?

<comment><USER23> I don't think it's for the political activism as much as it's for someone's friend at JSTOR.

<comment><USER24> JSTOR have said that once they had a guarantee the articles had been secured-- which I believe means that Aaron claims he no longer has them and has not, and will not distribute them--they had no more interest in pursuing the case. This is the feds acting on their own.

<comment><USER25> JSTOR said that publicly, and they could well have said that privately. But I'd be interested to know what the various journal companies have said, either to the prosecutor or to the various politicians they lobby and contribute to.

<comment><USER24> Yeah, it would definitely be interesting to know that.

But even so, is it really that hard to believe that the feds have their own agenda and are acting on their own for this one? I realize I have no proof to back up my assertion that this is all them, but it's not outside the realm of possibility, that's for sure.

<comment><USER25> They could have their own agenda, sure. But given how much money there is to influence American government these days, I think it's also worth asking "who could benefit financially from this action?"

</comment></comment></comment></comment></comment></comment><comment><USER26> Good men must not obey laws too well - <USER3>

(not that I equate JSTOR to slavery)

<comment><USER27> > (not that I equate JSTOR to slavery)

Not to belabor the analogy too much, but in academia at large, there's a decent argument to be made....

<comment><USER28> No.

Having to pay for access to papers and being forced to pick cotton in the blinding sun every day of your life under pain of the whip _are not the same thing_.

<comment><USER27> I'm going to go out on a limb and assume that you completely missed the point of my analogy, which was a reference to the exploitative, all-but-mandatory unpaid 'research assistant' jobs or 'internships' and the like. We're fortunate that this is unheard-of in our industry, but in some fields, it's the norm.

Though I would thank you for deleting the completely out-of-proportion reference to Hitler that was in your original comment ("Hitler liked cakes, therefore anyone who likes cakes is a Nazi"). It seems even Hacker News is not immune to Godwin's law.

</comment></comment></comment></comment><comment><USER29> Wow, I didn't know that the government compelled Quinn Norton and Alec Resnick on this one too. What exactly is aaronsw's defense going to be?

</comment><comment><USER30> It was mentioned in the blog post, but probably not everyone will read it. There's a legal defense fund for Aaron Swartz: <https://free.aaronsw.com/></comment>


Does India need its own vernacular internet? - <AUTHOR> https://www.techinasia.com/talk/india-vernacular-internet <comment><USER3> I am not sure if I subscribe to the findings of this article completely. Popular English language words have made their way into almost all Indian languages. When you are accessing the internet, unless you are reading long articles or online books, all you need is passing reading ability of some English words to get through. To say that Instagram doesn't have a Hindi equivalent of "filter" is simply missing the point - "filter" itself is called "फिल्टर" in Hindi which has the same English pronunciation and many people will get that. Also, many of my friends communicate with me in Hindi by literally using English alphabets but writing Hindi sounding words - <USER2> thing which the author fails to mention in his article.

<comment><AUTHOR> I agree to the first point you mentioned. In certain utility apps like payments etc this would hold true. In an internet without words i.e. images and video that would be true as well. But currently a large part of internet we use is text. Case in point, this article itself and this discussion. With the current technologies we have, for most part of Hindi speaking users, it is easy to speak in Hindi but very difficult to type in it.

As per using hindi words in english alpbhabets, my argument is - majority of India can't even read or speak english, how will they type in English? A lot of your friends do that because Hindi input is very hard and we need better tools.

<comment><USER4> > majority of India can't even read or speak english, how will they type in > English?

They won't. They type in Hindi, using the roman alphabet. Big difference. If anything, the roman alphabet is simpler than the syllable based Devanagari.

[https://en.wikipedia.org/wiki/Devanagari](https://en.wikipedia.org/wiki/Devanagari)

Apparently SMS is big in India:

> SMS is hugely popular in India, where youngsters often exchange lots of text > messages, and companies provide alerts, infotainment, news, cricket scores > updates, railway/airline booking, mobile billing, and banking services on > SMS.

[https://en.wikipedia.org/wiki/Text_messaging](https://en.wikipedia.org/wiki/Text_messaging)

I assume it has largely been done with cheap dumbphones even after 2007, which would mean no access to Devanagari characters, so there has been a long time, and a huge incentive to learn transliteration.

<comment><USER5> > If anything, the roman alphabet is simpler than the syllable based > Devanagari.

Sure, until you need to figure out which of the four phonemes a letter corresponds to which is very difficult for foreign language learners of Hindi, as it can be for English. I am a native English speaker but I picked up Devanagari in a day or so because it's actually relatively consistent due to the phonetic syllabic nature.

<comment><USER6> But that problem is entirely due to English, not the roman alphabet. Same in French. No such problem in German or Italian.

<comment><USER5> You would need quite a few diacritics to map to all of the Hindi phonemes, especially including loanword phonemes (/z/, etc.) I'm not saying this is bad, but it's not the easy solution that it was made out to be. Meanwhile, Devanagari handles consonant and vowel phonemes quite consistently as others have pointed out.

</comment></comment></comment></comment></comment></comment><comment><USER7> Fantastic article. I'm not Indian, so I apologise for my ignorance - but I would have liked to have read a bit more under the 'education' heading.

I would (this is where I fear I may be ignorant of the truth) assume that there is perhaps a rather strong correlation between money/education and English-fluency?

With that premise, it would seem that those with the power to effect change (educated engineers and entrepreneurs) are less incentivised to do anything about it - since they're presumably able and content to use "the English- language internet".

I suppose money can be the motivator though, if there's ~0.9*1.2B people clamouring for more internet in their native tongue.

I am trying to learn Hindi, so having read this I think it'd be interesting to toy around with pan-alphabet internationalisation when I've got a bit of a grip on it.

<comment><AUTHOR> Thanks OJFord. Under education, I tried to highlight a few short term opportunities for indian startups to focus on. From K 12 education to college education, all technology solutions have been built only for English speaking users.

There is no mobile based learning app for students to learn Math in Hindi language. No test prep platform in Hindi/Bengali or any other language (there are tons of test prep apps in English) No employee training or certification providers provide their online platform in any regional language. Not even an LMS exits (to be fair not many regional language medium institutions would be willing to pay for one) There is no Khan Academy in Hindi language.

As per the second point, yes I believe that should be true. Its not necessarily true that these non-english speakers don't have propensity to pay.

</comment></comment><comment><USER8> Am I the only Indian who thinks this is not such a bad thing, prefer one language to unite us all (isn't that the common base for most far-future novels?), don't care which, I can converse in three Indian languages, and know bit of french. In Sweden, all that knowledge is useless, all the main sites are in Swedish, now I am learning å,ö,ä, guess how much fun I am having?

<comment><USER9> > prefer one language to unite us all

Its not a bad thing if that happens properly. But instead currently we have small number of 'elite' people who always prefer English over native language and we have large number of people we don't know how to read & write English. Since the output(business/knowledge/tech) of this 'elite' group almost always in English and by definition they are in control its not easy for person who don't know English to break loop. Its very easy for the person who know English to get the job compared to who don't. One might say, just learn English and become the 'elite'! Well, learning English is not at all easy for a kid from rural background. This initial barrier dividing among ourselves and now we now have invisible class system based on language. this is apparent if you visit any restaurants/supermarket in cities or just visit city outskirts. My point is process of implementation of single language for all is very difficult.

<comment><USER10> Why would they want to break the loop?

If one language to unite us all is a good thing, then either kids from rural backgrounds have to be taught English, or everyone has to learn the native language of that kid. This will be impossible for a kid from a different rural background. Worse than English, even, since most other languages don't have the same amount of cheap available learning materials.

</comment></comment></comment><comment><USER11> English is one of the official languages in India for historic reasons. The number of English speakers in India surpasses the population of the United Kingdom.

Now, the field of automated translation is making enormous progress lately. I think eventually whatever the gaps in understanding are, will eventually be solved.

<comment><USER12> Google search across languages would be amazing, e.g., accessing old scholarly research written in a native tongue. Wonder what the underlying structure of the neural network is for translations between all languages:

[http://www.kurzweilai.net/googles-new-multilingual-neural- ma...](http://www.kurzweilai.net/googles-new-multilingual-neural-machine- translation-system-can-translate-between-language-pairs-even-though-it-has- never-been-taught-to-do-so)

</comment></comment><comment><USER13> In India, the state is simply lazy.

All most all English words should have an equivalent in all native languages. But, unfortunately no one forms any new words.

If apps can be fully translated to the local languages, it will help everyone.

<comment><AUTHOR> I don't think its the responsibility of the state. It is of the startups to move out of their english speaking bubble and realise the potential of non- english speaking india.

<comment><USER14> Buying power is largely concentrated with the English speaking though, even if it isn't their first language. Most people who do not speak English are also lower on the income scale. As in less than $500 per month in income

<comment><USER10> The people who have the most purchasing power and speak English, are they bilingual or do they only speak English?

If not, could language be used as a tool for market segmentation? Give price- sensitive people lower prices to increase the size of the market without hurting the profits extracted from the wealthier people.

</comment></comment></comment></comment><comment><USER10> The article says "88 percent of Indians can't speak English (let alone read it)"

I'm not a native English speaker, but that sentence implies that reading English is more difficult than speaking it. But in my experience, most people who learn English as a second language learn to read first, especially if they specifically learn it to use the Internet.

For native English speakers: Is my interpretation of that phrase correct?

For non-native English speakers: Did you learn to read English first or did you start by learning to speak it? Does it vary on first language, literacy level, motivation?

<comment><USER9> > reading English is more difficult than speaking it

I'm a non-native English speaker. I agree, reading English is easier than speaking. In rural places basic grammar is not taught properly. Even if they do learn to read somehow with the help of dictionary or internet, Speaking English is far out of their reach. For using App reading ability is enough. But for overall, English as medium of communication this is very problematic. This is more frustrating when you are among elite (for academics or career purpose) who speak English fluently.

Most important thing is they can't express their idea. For using App this isn't big problem, but as overall expressing idea this is a barrier.

</comment></comment><comment><USER15> (Vernacular - "Language of the slave").

It's far more complicated that "need its own vernacular internet". The internet is an emergent thing, and what one sees are just the symptoms. This is also the reason India's literacy is so low (even compared to Africa/Middle East).

If India wasn't a linguistic apartheid regime, we'd already have seen a native ecosystem which is lacking quite badly. This needs fixing at the state and political levels, which I have exactly zero hope of ever happening. This despite having some nauseously xenophobic organizations like ShivSena (in Maharashtra), the DMKs (in TN), the KaRaVes (in KA). This is in addition to the "nationalistic" organizations like RSS and BJP at the central level.

These orgs are essentially vehicles which instrumentalize the widespread disaffection from the apartheid state, in order to put themselves in power (Advani's use of Ayodhya is a nice study). If you study their policies carefully however, you realize they plan to do precisely nothing that is the cause for the inequity.

This is not different from the independence movement, where a bunch of Brown folk wanted to run the colony.

Digital India, be in no doubt, is meant for the 200 million Brown sahibs, who do know English. The rest are peasants who have been at the losing side of the inflationary system today, and of the cruel taxation system of the British, kept at bay via endless subsidies (a dog hardly bites a master ?) and mutual bickering.

There is zero empathy from the former class, and these pretentious people are the source of endless pain for the red-pilled; it is very disappointing to live in communities where the kids start speaking English before anything native, and worse when the state hold such clones in such high regard.

May kek not have mercy on the clones (apologies for the 4ch lingo).

Technical:

I think Sailfish has better localization than Android. Keyboard is a disaster everywhere, since no one in India uses native language keyboard/input. They are very rare, if at all available, and next to no one knows they exist - it's in fact easier to find such resources in the US than within in India. Swarachakra is too crowded and very very information inefficient (unlike the 5-vowel Japanese system its based on), which leads me to believe even their creators don't use it actively.

The lack of feedback means that the ITRANS layout is very very bad, and unusable (xkeyboard moth-balled most layouts due to disuse).

Again, the roots of the problem lie with the education & state policies that are reinforced via state violence. China gets this right because India is a colony & it isn't.

(Downvote all you want clone people; सत्यमेवजयेत् !)

[1] [https://news.ycombinator.com/item?id=12237411](https://news.ycombinator.com/item?id=12237411)

<comment><USER16> What do you mean by "linguistic apartheid regime"?

<comment><USER15> See,

[http://www.forbes.com/sites/realspin/2014/11/06/the- problem-...](http://www.forbes.com/sites/realspin/2014/11/06/the-problem-with- the-english-language-in-india/)

The "socialist" state runs a systematic unwritten discrimination regime where every thing from higher education to state services are restricted only to English speaking class; yet, it is the remainder that get the boot due to the inflationary forces of the currency. It is by every meaning of the word, an apartheid; this state mandated scheme of slavery is kept in check by various schemes of cultural propaganda and distractions by the Anglical state. Of course, without education, feudalism becomes only too normalized.

This system is widespread all over former colonies in Africa and Asia.

</comment></comment></comment><comment><USER17> Cash transactions provide Privacy/Security; Govt must give Gun-Licenses to Common man if it really wants a Cashless society; [https://en.wikipedia.org/wiki/Estimated_number_of_guns_per_c...](https://en.wikipedia.org/wiki/Estimated_number_of_guns_per_capita_by_country)</comment>


Scala use is less good than Java use for at least half of all Java projects - <AUTHOR> http://www.javacodegeeks.com/2011/09/scala-use-is-less-good-than-java-use.html <comment><USER2> Some kind of dupe, I think? I guess Java Code Geeks is just an aggregator: <http://news.ycombinator.com/item?id=2967578>

<comment><AUTHOR> Sorry for that. Didn't notice.</comment></comment>


Covid-19 Global Chart - <AUTHOR> https://www.covid-19-live.com/ <comment><USER2> Does WHO provide an API or do you scrape web site yourself? In any case the data doesn't seem to be updated in real time. I also developed a dashboard myself using Wolrdometer as data sources which is updated every 10 min: [https://coronavirus.gg/](https://coronavirus.gg/)

</comment><comment><USER3> Also: [https://gisanddata.maps.arcgis.com/apps/opsdashboard/index.h...](https://gisanddata.maps.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6)

</comment><comment><USER4> Country selector resets to top-5 every time you change graphs :(</comment>


Why Canada is prospering and the U.S. isn't - <AUTHOR> http://finance.yahoo.com/blogs/daily-ticker/why-canada-is-prospering-and-america-isn-t-150359533.html

<comment><USER2> tl;dr:

\+ No bailouts/Better banking = Zero crisis!

\+ Encourages inmigration/Easier entry

\+ Less corruption</comment>


Where is a company where a developer will ACTUALLY be challenged? - <AUTHOR> Where is a company where a developer will ACTUALLY have to really use their mind to solve NEW and INTERESTING problems on a daily basis?  I feel like every company says they're a good environment to grow, expand your mind, be challenged etc. when in actuality it is 99% banal tedious stuff and the biggest challenge is slogging through it all (not really anything that makes you think or expands your mind much). <comment><USER2> AI is a bit of a buzzword, but working for an AI startup, I see a lot of talented developers doing some very clever things with code. All of them are seriously engaged with the projects they're working on and work long hours because they want to solve the problems.

Beware though, a lot of jobs advertised as 'AI' may not be and may wind up being banal/tedious as you describe because AI as a field is very, very young and not fully defined.

</comment><comment><USER3> In the worse case, try to automate that boring stuff, build metrics, and see the improvement.

A chaotic CEO will challenge you at any time too. And will generate new (and curious) problems. There are many of them.

I use to ask two questions at interviews:

How is a common day for a team member?

What will I be working on from first day?

</comment><comment><USER4> games are a domain where creativity and programming move side-by-side. i don't know if this is true for companies</comment>