One Way to Deal with Internet Thieves - <AUTHOR> http://pzxc.com/internet-is-still-wild-west

<comment><USER2> This guys website is just others content, HE is taking flash games created by others and publishing them on his website FOR PROFIT. I spotted some friends games on there that they are unlikely to gave permission to put on there.

Also, he has taken a popular t-shirt design, copied it and then is selling it for profit here <http://www.cafepress.co.uk/playitontheweb> as advertised on his website.

If the guy who posted this is reading, where do you get the games from you're posting? Are you downloading them from other flash game sites and uploading them? Are you scraping those sites? I bet it's the latter. You're only slightly less shameful than this guy you're attacking.

irony eh.

<comment><USER3> In addition, if you visit www.playitontheweb.com, you'll see in the right column he prominently features Mario and Sonic flash games, games that make unauthorized use of copyrighted and trademarked characters, as well as game art and sound effects.

This guy really has some nerve complaining about someone copying his crappy site design.

<comment><USER4> Indeed. Perhaps this is more of a case of "what comes around, goes around."

</comment></comment></comment><comment><USER5> Okay, I'll take the alternative view: this is embarrassing for the OP.

What worries me is that the OP found the name Bart Burns on the WHOIS for the domain, and assumed (a) that's who registered the domain, and (b) that the information he found via a quick Google is actually Bart Burns. I can change the information on my domain to be whoever I want, particularly if I'm a scammer. Whatever I put in NameCheap's panel shows up minutes later. If someone doesn't believe me, I'll alter the registration on one of my domains. Just ask.

More interesting than that, however, is that the OP is completely aware that there are avenues of recourse for this and that he is technically able to determine who hosts the scraped domain. Rather than pursue those avenues of recourse, however -- said facilities have been keeping the Internet from devolving into a "wild West," at least in ARIN and RIPE regions -- this guy had to take a play from 4chan's book and attack possibly the wrong guy. If everybody did what the OP does, you're absolutely right: the Internet _would_ be a wild West.

As an administrator at a very large ISP, I am completely aware that some people lose faith in abuse desk contacts. At my employer, we receive countless abuse complaints and _we handle every single one_. This is a pattern you will observe in ARIN and RIPE regions, but less so for APNIC and other parts of the world (with exceptions). Particularly for someone hosted with JustHost, a DMCA complaint gets the job done in hours. If it doesn't you can go after the ISP. I was never a fan of the DMCA, but this is a prime example of a use case for it.

Above everything else, though, even if the OP is right about the identity of the scammer, calling out the scammer's employment means the OP could be attacking his family. If I were the aforementioned company I'd cut my losses and terminate the employee for bad PR. Now the OP has potentially hurt the suspected perpetrator's family based on actions his family is _probably not even familiar with_. Because he copied the HTML for a site I've never even heard of. That's icing on the cake, to me, because I protect my family. With teeth.

In short, I am totally unimpressed by the OP's vigilantism and I am equally disappointed that it is so popular in this forum. Demonstrating this sort of arrogance to Hacker News is probably a good way to burn your bridges at the innumerable employers and business contacts that frequent the site.

<comment><USER6> "WHOIS may be wrong" is a very good point.

However, if he were actually certain to get the right guy, what's _so_ bad about "attacking his family" as you define it? Putting a thief in jail does reduce the income of his/her family, but I wouldn't blame the victim for doing so!

You _could_ argue that the punishment doesn't fit the crime, but that's a much more subtle argument than you appear to be making.

<comment><USER5> > However, if he were actually certain to get the right guy, what's so bad > about "attacking his family" as you define it?

Spoken like a bachelor. Two things I firmly believe in, _especially_ when it comes to Internet squabbles, disagreements and such:

\- Don't mess with someone's livelihood.

\- Don't mess with someone's family.

They're related. You're going to sit here and tell me that it's justifiable that the alleged guy lose his job over this, possibly taking food out of his kids' mouths? If I copied your Web site and someone pulled a stunt like that with my employer, I'd be out for blood. You hurt my family, I hurt you. You'll understand when you have one.

You can't parallel this to putting someone in jail, because there wasn't really a crime committed (I'd like to see a prosecution), and OP is not the law.

<comment><USER6> I upvoted your comment because you are contributing to the discussion, and because I hope to understand you. You're right that I have no kids (I do have a girlfriend, though). I've also explicitly assumed throughout that the guy who was fingered really was the guilty party.

Under this assumption, though, I don't see what's so bad. First off, the guy did do something that he shouldn't; secondly, the actual punishment is almost certainly zero. Not only are we, indeed, not talking about jail time, we're not talking "meticulously SEO johndoeisathief.com to the top of Google" either; we're not talking "start a Twitter shitstorm until his boss fires him"; we're not even talking "publicly call him out" - the data in the post probably probably does identify exactly one person, but it's hard to actually make that link. I do admit that blacking out some more data would be prudent, but - what exactly would you be so upset about?

<comment><USER5> That's a different argument than you started with. You lost the plot somewhere, which is that I think attacking his employment over this goes beyond a line. There is only one reason to proudly put in your attack:

        Someone that works at X should know better!

What reason do you think that is? Now they need to take a hard look at their decision to hire this guy; what is he up to in his private time? Suddenly, his private actions have become publicly tied to our company.

There's a reason I don't have my employer in my profile -- I have personally seen someone take a comment the wrong way, and spend days calling the employer to report what a piece of shit the employee is, and how much he was into drugs, and so on. Merely naming the guy's employer here casts them in a bad light.

Think back to Noirin Shirley accusing the guy from Twitter of sexual assault via her blog. When the media inevitably ran with it, it wasn't "an Apache member accuses another Apache member," it was "a Google employee accuses a Twitter employee," because that's a much juicier story. Now Google and Twitter have been sideswiped in something that isn't even their business, and I wouldn't have been surprised if Twitter had cut him loose over it. I don't think they did, so that is a lucky thing for him.

In this economy, I'm just as stretched as everyone else. Anybody who goes after my paycheck is playing with fire; that's all I'm saying.

<comment><USER6> I see where you are coming from. I agree with you that e.g. phoning his employer would be an overreaction; and I agree with you that my earlier message was _very_ unclear in that respect, although I wouldn't have posted it if he did, in fact, try to get the alleged thief fired.

Right now, though, "company X" does not seem to be heavily involved - the text in the original is in an image and not indexed. Don't you think that the actual damage is likely to be small?

</comment></comment></comment></comment></comment></comment><comment><USER7> I would've just sent the DMCA notice. It's not a "whole process", it's a couple lines of text you copy/paste, put in the URL of your site and the copy, change the date, and email to the host. It'll take you a few seconds to modify and send out. Most web hosts have a specific mailbox for DMCA notices on their contact page or in their AUP.

      Subject: DMCA Notice of Copyright Infringement

      The copyrighted work at issue is the text that appears on: [URL(s)]

      The URLs where our copyrighted material is located include: [URL(s)]

      You can reach me at [email] for further information or clarification. My phone number is [phone] and my mailing address is [physical address].

      I have a good faith belief that use of the copyrighted materials described above as allegedly infringing is not authorized by the copyright owner, its agent, or the law.

      I swear, under penalty of perjury, that the information in the notification is accurate and that I am the copyright owner or am authorized to act on behalf of the owner of an exclusive right that is allegedly infringed.

      [your name]
      [today's date]

Never once have I had a DMCA notice not result in the material taken down within 24 hours, even when the host was outside the US. _None_ of them want to potentially be a party to a copyright infringement suit, however unlikely it may be that you'd take it to court and drag the host into it.

<comment><USER8> _Never once have I had a DMCA notice not result in the material taken down within 24 hours_

 _I_ have!

The culprit -- whose copyright agent [edit]appears not to[/edit] respond to DMCA takedown notices unless they come on law firm letterhead -- is none other than Amazon.com. (I've caught scumbags selling books of mine online as ebooks via the Kindle store. Amazon don't listen unless you carry a big stick. My solution was to grab the nearest big stick, in this case SFWA's Griefcom process, but something like that's not really an option for most self- published authors.)

(In contrast, Apple -- same situation, only in the iBook store -- were the very model of efficiency and helpfulness.)

<comment><USER5> > The culprit -- whose copyright agent doesn't respond to DMCA takedown > notices unless they come on law firm letterhead -- is none other than > Amazon.com.

That's illegal. There are well-defined bullet points that make a valid DMCA complaint that must be acted upon, and I doubt Amazon would open themselves up to liability based on "big sticks". They'd lose safe harbor for not complying, which is _extremely_ bad news for EC2 and any Internet hosting provider.

<comment><USER8> Legal or not, it's what they're doing. And it's not just me:

<http://nielsenhayden.com/makinglight/archives/012933.html>

<comment><USER5> If someone were to pursue it further and remind Amazon that not following OCILLA means being liable for every single copyright infringement that they host, whether on EC2, S3 (looking forward to that one!), or the Kindle store, they might change their tune.

</comment></comment></comment></comment></comment><comment><USER9> If the OP's planted information identified the wrong guy, then that guy, who was wrongly accused of being a willful copyright infringer, might be able to sue successfully for libel.

Otherwise, though, one legal irony about this posting sort of tickles me:

* Suppose that the (human) scraper were to sue the OP for libel. And leave aside for now the fact that (at least in the U.S.) truth is usually an absolute defense to a defamation action (although that's not the case in false-light or invasion-of-privacy cases).

* The OP should win, it seems to me, if for no other reason than that _it was the scraper, not the OP, who published the putatively-libelous information about himself._ Presumably, if the scraper himself had not copied and published (what he thought was) the OP's site, then the supposedly-libelous information would have stayed hidden in the OP's server, never seeing the light of day.

Of course, I could imagine some judges thinking that the case was like the spring-gun cases you read about in first-year torts class: If you booby-trap your vacant house with a shotgun to protect against burglars, you can be liable to a burglar who gets shot, even though he's breaking the law by breaking in.

</comment><comment><USER10> I don't quite see the argument myself. As a former dancer in the breakdancing scene the premise of "biting" (stealing) moves comes up a lot. After a lot of consideration I came to the conclusion that if you perform something publicly its pretty much fair game for copying. I don't think there is a benefit in spending your time trying to hunt down these copycats. The time is much better spent creating new works. The Open Source world teaches us that product is not just the object the item being copied but the service, the knowledge, the understanding of the product and the ability to create more. For creatives and dancers it's the brand, and/or your name.

By all means if you think its worthwhile you can hunt down people ripping off your stuff but if you're becoming tempted to just sit on your prior work and prevent anyone else from touching it you're becoming stale. Case in point would be Cliff Richard who the RIAA like to roll out whenever they want to extend the terms of their copyright because apparently Cliff Richard relies on this royalties as his pension. I often wonder why he didn't/doesn't have to save for his retirement like everyone else.

<comment><USER11> Yo, where did you used to break?

</comment></comment><comment><USER12> "I don't make idle threats. I have all kinds of information about this person, and I'm going to be keeping an eye on him, this particular domain, and any other domains run by him or his host that access my servers."

^ Still the wild west indeed.

<comment><USER13> ??

So what's your proposal to deal with such a problematic behaviour?

The guy is quite nice - he only made the thieve risk his reputation by the consequences of his own action. He didn't involve anything like law enforcement, the hosting company or the banners network - which would have cost them time thus money.

All he did is take some of his own time so that the scraper would get the "accusing" version of the website next time. He fixed that at his own expenses.

I believe many of us wouldn't have been that nice. And yes I believe he should keep an eye on this thief, because from what he said (the games didn't play) the guy didn't spend a lot of time on that borked job.

For all we know, this might be his full time job - copying websites, replacing copyright notices and authors, making money out of it with banners. All he needs is a process that scales well financially to be a "scamtreupreneur" (Couldn't find a word so I made up one. Do I get karma claims on that word?? :-))

With bots, selecting target websites etc. it might be possible to make a living out of it while true entrepreuneurs are starving in bootstrapped companies :-/

<comment><USER14> I don't know if the poster was criticizing the owner's response; I read it as just an observation about the state of things. Also, you're right - the thief got off easy.

</comment></comment></comment><comment><USER15> Disclaimer: Most of my knowledge of the Wild West comes from the movies.

In the WW they used to quite like public hangings, both legal and illegal. Why do you think they did this rather than just shoot people beside a grave and roll them in? As a deterrent to others is my guess.

I appreciate jedsmith and others views, and do somewhat support them, but lets say the OP had just issued a DMCA (and it was actioned). What has that achieved. The copier loses his site, so from the OP's perspective for this instance the problem is solved. Until next week the guy registers another domain, and another, and another. And his friends do it too.

I bet the guy doing the copying is going to think about it twice next time, and I bet one or two people will have read the blog post and rethought future plans.

Don't underestimate the amount of stress, time and legal costs that some people would spend trying to resolve a problem like this.

I know this Wild West killing is not the best approach but at this point in the history of the internet I am seeing few alternatives to prevent (ie not resolve post event) these things happening.

</comment><comment><USER16> This is interesting, because it works on the same principal as software piracy: The original isn't "stolen" it's merely copied. The original is still intact.

I'm wondering if the same people that are so zealous and pro-piracy would be fine with someone taking their site and duplicating it.

<comment><USER17> IS IT the same principle? Usually pirated software distributors don't claim original authorship.

I'm not saying that the "merely copied" argument is correct, I'm just saying that I think the situations are different.

<comment><USER16> The main argument for 'piracy is not stealing' is that there is still an original copy.

Just like with the OP's site.

<comment><USER18> "The main argument for 'piracy is not stealing' is that there is still an original copy."

No, the main argument for 'piracy is not stealing' is that piracy is not stealing. Stealing is the substraction of property, which piracy isn't.

This isn't stealing either, this is plagiarism.

If we don't name things with their correct names, we create false associations, and that makes arguing about those things needlesly complicated. It's hard enough getting a consensus on anything related to IP as it is, so we should avoid ambiguous syntax wherever possible.

<comment><USER19> Violent agreement. You and GP seem to be saying exactly the same thing -- because the original still exists, there is no subtraction of property, and it is not stealing.

<comment><USER20> They are saying the same thing as far as that goes, but rick888 then uses that as a jumping-off point to misrepresent a (fairly OT) position he doesn't agree with. He's conflating the arguments of "piracy is not stealing" and "piracy is not necessarily bad," which are only thinly connected (being theft would make piracy almost certainly wrong, but _not_ being theft doesn't confer any moral value either way).

</comment></comment></comment></comment></comment></comment><comment><USER21> You'd also think a cisco engineer up to such shenanigans would know enough to use an anonymous/proxy whois registration.

<comment><USER22> Or use someone else's name, which may be what the owner of the site actually did. How can we know that the whois information is valid?

<comment><USER23> I wouldn't be surprised if the advertising provider requires and checks for real whois info. If not, there's probably a business model in there somewhere.

<comment><USER24> Small advertising networks are notoriously lax. The ad network in the screenshot, pogads.com, has this testimonial on their front page:

"I've been banned from adsense for advertising online with companies offering popups, I registered with pogads as an adsense alternative and I see the real money potential in my service Roger, IL"

It's positioned as the first testimonial of several.

</comment></comment></comment></comment><comment><USER25> one sneaky thing you could have done is to modify your site very slightly so that his site would load your ads instead of his. He wouldn't have noticed ;)

<comment><USER26> Just adding some javascript that just made sure the right ads are showing would have been a much more subtle response.

I wonder if there is a possibility of a JS library that could detect this type of scraping and reserving automatically. Would probably leads to an interesting arms race with the scrapers.

</comment></comment><comment><USER27> Was leaving the domain name that easily identifies the person whose name and other details were censored done intentionally?

</comment><comment><USER28> It'd be interesting to hear the technical details of how the switcheroo was accomplished

<comment><USER29> Check web logs for regular visits from a specific IP address. Setup alternate site (eg: on a different port) and use firewall rules to direct http traffic from that IP to that port. Make a small change to the site and verify that the copied site is altered. Then have fun.

<comment><USER24> It would be simpler to handle it in the application.

if ($ip == 'XXX.XXX.XXX.XXX') { require 'poison.html'; exit; }

<comment><USER29> Setting up a new site is trivial to do and allows you to mess around with it without modifying your main site... which may not be so simple as your example if it's based on a CMS or other framework.

</comment></comment></comment></comment><comment><USER14> Speaking of Internet thieves - check this out:

GearSpoke.com Derekfrye.com Badrobots.org

Why would anyone copy a website consisting of a single picture? Automated bot scraper?

<comment><USER24> GearSpoke.com - 188.8.131.52

derekfrye.com - 18.104.22.168

badrobots.org - 22.214.171.124

And 22.214.171.124 is a Slicehost IP. Looks like someone just neglected to update their DNS settings.

<comment><USER14> Ah hah! Nice catch, didn't think of that.

</comment></comment></comment><comment><USER30> well played, sir!

</comment><comment><USER31> Other fun ideas:

1) Redirect all of his traffic to your site

2) Put an iframe with his site in it. His hosting company would probably shut him down very quickly.

3) Put goatse / tubgirl on all pages (for his scraper only).

</comment><comment><USER32> Great!

</comment><comment><USER33>
      updating, come back later...

The site in question has been updated quite quickly I suppose.

</comment><comment><USER34> I keep trying to scroll up farther than the page allows</comment>


How Much Do Music Artists Earn Online? (2010) - <AUTHOR> http://www.informationisbeautiful.net/2010/how-much-do-music-artists-earn-online/ <comment><USER2> Yeah, um, this assumes each stat to be the sole source of income, and to compare to minimum wage we must assume 40 hour work weeks.

Anyway, if you can't make it as a musician, I guess you have to do something else productive. Corey Doctorow gets it when he describes himself and other people who make a living in creative arts as being in the 0.001%, the extremely lucky few who get to have that as their career.

All professional musicians could disappear tomorrow and we'd still have enough great and varied music to listen to for all of anyone's lifetime and also have massive amounts of high quality participative music making by non-pros.

Incidentally, I'm a semi-pro musician myself but I long ago stopped fighting the I-deserve-a-living-as-a-musician battle and switched to figuring out how to be sure my contribution to the economy actually mattered.

<comment><USER3> Yes, your statement is accurate given that you don't think art or music need to evolve with culture. That they don't play a vital part in human evolution: spiritually, emotionally, culturally, philosophically.

If so, yes, the music that has been made is more than enough. And we don't need any more music.

<comment><USER4> You completely miss the point: All _professional_ musicians only account for a tiny fraction of actual music output. If they all disappeared it would not prevent art or music from evolving.

<comment><USER3> See my reply above. I doubt you, or anyone you know, voluntarily and primarily listens to amateur music.

<comment><USER5> He's saying professional as in 'all i do is make music'. There are plenty of local bands where the band members have 2nd jobs.

<comment><USER3> yeah, and that's who you listen to all day? Really?

<comment><USER5> Sure, I knew people in college who were part of the local music scene and listened to it primarily.

I don't see why it is so surprising. People choose genres and primarily listen to it all the time. The kind of music popular to make in your locality, is definitely not the same genre that's played nationwide no matter how close the similarities may be.

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> This is from 2011. It's best to add a (2011) to the title in situations like these. It's a good topic and good info, but considering the speed at which music industry tech is evolving, the date of publication should have more attention.

</comment><comment><USER7> I don't think the world is ready yet.. but here is my idea on how to let artists make money in a fair manner without consumers paying too much.

Imagine a clone of Spotify. You can do offline streaming and such for all the music in it. You pay $10/month for the privilege (which I think is fair). $5 or so goes to the company to maintain their servers and such. The other $5 though you can do interesting things with. Basically, each month you have $5 worth of "tip money". You can send however much money you want to whatever artist you want.. or you could alternatively setup other neat dynamic setups (the author gets 50 cents when you favorite his song, etc). .. The point being that it's under the user's control. And, if the user chose to not tip any artists this month(or they have left over tip money), then it's evenly split between all the artists they listened to for the month (depending on song count or whatever).

This model I believe would work because the biggest thing standing in the way from giving your favorite band a tip is that it's through services that you are not already using. I don't want to go to their website and then sign up for paypal to give them a $1. The thing with this method payment is already accounted for. You already paid. Now you just get to pick which band deserves your money. This could even work with a free version by doing a model like for every 10 ads you must listen to in a month, you get 10 cents added to your tip money..

If I knew anything about the music industry, this would be the startup I'd be behind. This wouldn't work with the label model, and giving artists tips isn't really something I think most people would understand at this point... but some day... some day.

<comment><USER8> That "totally fair" $10/month figure is the problem.

Spotify's benefit to the consumer is _they pay less for listening to the same music_. Compare that to iTunes which doesn't save you any money, it just makes it a whole lot more convenient. (As an aside, I was far more appalled by the discrepancy between retail albums and iTunes albums: Both cost the consumer $10, but the label gets 5x more from iTunes while the artist gets slightly _less_ than the $1 they get from retail.)

You can save the consumer a certain amount of money by cutting out middle-men, but at the end of the day if consumers are paying less money the producers are earning less money. It only makes sense to the producers if it doesn't cannibalize their existing revenue streams.

<comment><USER9> I pay £10/month to Spotify.

I was not spending anything like £120/year on music for several years before I got my Spotify subscription.

<comment><USER8> ... and you use Spotify because you pay less than £120/year for more music than you would otherwise get for that value?

<comment><USER9> I use Spotify because I have the convenience of not having to think about music - it's just _there_ whenever I want it, without me having to transfer it anywhere, dig out CDs to rip, etc.

I've listened to vastly more new music since I got Spotify, because I could try things without having to buy a single or an album.

That's worth £10/month to me.

</comment></comment></comment></comment></comment><comment><USER10> A musician who relies on Spotify to make a living isn't a musician. In the music industry money comes from concerts, there's no way around it. Recorded songs are best thought of as marketing.

<comment><USER11> So, by that logic, the Beatles at about the time Revolver was released... were not musicians. Because, as you may or may not know, with the Revolver album they became a studio-only band. They neither _wanted_ to play concerts (they were sick of playing concerts, started fearing getting shot, etc.) nor _could_ they play because of limitations of live performance technologies at that time.

But ignoring the Beatles for a moment, what about totally electronic music that's of a kind that can't be played live in any meaningful way? And what about people like me -- who have absolutely no interest in buying band t-shirts (I prefer wearing plain clothes) or attending live concerts?

<comment><USER12> > Because, as you may or may not know, with the Revolver album they became a > studio-only band.

That's famously _almost_ true ;)

[http://www.youtube.com/watch?v=fFJFsyAAoZ4](http://www.youtube.com/watch?v=fFJFsyAAoZ4)

<comment><USER11> The album 'Let It Be' (which was performed at the famous rooftop concert) came to be precisely because of McCartney's insistence that they for once create an album without any double-tracking, fancy technology, etc. and perform it live in a televised setting (as opposed to live in a concert setting -- which they were still unwilling to do).

</comment></comment></comment></comment><comment><USER13> Not the best submission title.

Spotify isn't mutually exclusive of the myriad other ways musicians can earn from their music online, many of which are described on this very same infographic.

<comment><AUTHOR> I thought that was clear from "...to earn minimum wage _on Spotify_ ". I'm happy to change the title if you can suggest better wording?

</comment></comment><comment><USER14> This doesn't actually feel like an unreasonable number. Even a radio station with a small broadcast radius playing your song once could get that many people to hear it per day - I think people may be assuming that the value of a single person listening to a song once is worth more than it actually ought to be.

</comment><comment><USER15> I find the graphic very misleading in that it mixes units. It seems very reasonable that price-per-album would be 10x the price-per-song since an album generally has about 10 songs on it. It also seems reasonable that price-per- song would be 50x price-per-play (using low-end royalty deal and Rhapsody payments) since, on average, I probably play songs about 50 times before I move on. So for synthesizing "replacing album revenue" the Rhapsody payments seem fair.

"Replacing radio revenue" [ie. you don't control song order/flow] seems to be about 30% of "replacing album revenue" so Pandora is $3/month instead of $10/month.

If the above is reasonable then Spotify seems well off the mark on pricing. Also, I don't understand why the label share is 80% of revenue.

</comment><comment><USER16> This was posted a few months ago. I asked a question then, that I will ask now. How much money does Spotify have to pay the record label? Usually the artist gets most of their money taken by the label.

See here: [http://www.salon.com/2000/06/14/love_7/](http://www.salon.com/2000/06/14/love_7/) and here: [http://www.techdirt.com/articles/20100712/23482610186.shtml](http://www.techdirt.com/articles/20100712/23482610186.shtml)

Record company math is very similar to movie studio math. Both systems are designed to make people very rich, just not the ones actually doing any of the work.

</comment><comment><USER17> Although this number clearly states the unfeasibility of subsisting solely on Spotify plays of a few songs composed, it fails to translate this number to the present value of the expected plays throughout the years before the song becomes public domain.

Such value would constitute a fraction of the payment the artist would get for composing those songs, since, as mentioned in another comment, "selling" your songs on Spotify does not require you to refrain from selling them elsewhere.

Only given the sum of the present value of all the different passive sources of revenue the artist will get from a song and the time he spent writing it can we begin to compare his remuneration with the minimum wage.

<comment><USER17> Other point I saw recently here at HN is that the recording companies get the lions share of the revenue that comes from recording because investing in a up and coming artist has a high risk, therefore must bring high profits to be a viable business model.

Those business who don't milk the successful artists don't have money to take a chance with the up and coming artists while rewarding the investors at a competitive rate, which means they don't get investors for future growth. Other sources of money for a business include it's own money (which we established such a business wouldn't have) and loan (which require the business to have a higher return rate than the loan's interest).

</comment></comment><comment><USER18> Musicians make money from touring and selling merchandise primarily. This statistic is still quite interesting, it really puts it into perspective how little money there is for most artists to make on platforms like Spotify.

</comment><comment><USER19> Which is equivalent to what, one play per day on traditional radio? Maybe less?

</comment><comment><USER20> Or just 6,929,156 for one week of the year, around Christmas time...</comment>


AMD Threadripper 2990WX Linux Benchmarks: The 32-Core / 64-Thread Beast - <AUTHOR> https://www.phoronix.com/scan.php?page=article&item=amd-linux-2990wx&num=1 <comment><AUTHOR> Long years of supporting NUMA in kernel seems to pay off now.</comment>


An Android 8.0-9.0 Bluetooth Zero-Click RCE - <AUTHOR> https://insinuator.net/2020/04/cve-2020-0022-an-android-8-0-9-0-bluetooth-zero-click-rce-bluefrag/ <comment><USER3> I really hate this software world where my phone stack is generally hidden away from my ability to fix it or change it. It's true for both apple and android generally, even if I can see some pieces of android in the public sources it's basically impossible to change out alot of the inner stack. I know there are endless attempts to let us have control over our phones. But we programmers are never the customers. And the vendors never open source their drivers. The various open software/hardware schemes never seem to reach maturity. Is there any hope here?

<comment><USER4> The two big projects working on this are the Purism Librem 5 and the PinePhone, both run stock Linux with no binary blobs aside from an isolated cellular modem.

The Librem 5 has been delayed for years and the behaviour of the company is kind of sketchy, however going by Purism's videos the software is pretty good and getting better rapidly (and they upstream their changes back to Gnome).

The PinePhone has shipped to some developers and the company has a history of actually making functional products, but the software is still a WIP, and Pine64's products are cheap (which is great for many people, but I would rather have a $400 phone than a <$150 phone, especially given that these ones won't suffer from software obsolescence).

I can't wait for these things to become at least somewhat functional - <USER2> personally will be buying one as soon as they get phone calling, SMS, and a web browser (the Librem has them, and other Gnome applications, but it's still in preorder).

Pinephone: [https://www.pine64.org/pinephone/](https://www.pine64.org/pinephone/)

\- [https://news.ycombinator.com/item?id=21824962](https://news.ycombinator.com/item?id=21824962)

Librem 5: [https://puri.sm/products/librem-5/](https://puri.sm/products/librem-5/)

\- [https://news.ycombinator.com/item?id=21369733](https://news.ycombinator.com/item?id=21369733)

\- [https://news.ycombinator.com/item?id=21303770](https://news.ycombinator.com/item?id=21303770)

\- [https://puri.sm/posts/librem-5-vs-android-which-boots- faster...](https://puri.sm/posts/librem-5-vs-android-which-boots-faster/) (stupid comparison vs a 6 year old Android phone - how out of touch is their marketing team and CEO to allow this to happen???)

<comment><USER5> These things look fun but if they don't get the app support they'll just go the same way that Firefox OS, Ubuntu Mobile and Windows Phone did.

Unfortunately other than the hacker niche that's on this forum you won't get the average consumer using these devices/OSes and therefore you won't get the apps that attract people to use the devices in the first place.

<comment><USER6> This is why I want Firefox to prioritize Progressive Web App support, and for PWAs to become more popular. Then an open device that can run Firefox can have access to a variety of apps that will work just the same as they would on an Android device. I understand there are drawbacks compared to native, but I really think they're worthwhile tradeoffs to enable new platforms to be immediately compatible with an existing set of cross-platform apps!

<comment><USER7> Except it is Google and Microsoft that are driving the show in regards to PWAs and their OS integration.

</comment></comment></comment></comment></comment><comment><USER8> Fixed in [https://android.googlesource.com/platform/system/bt/+/3cb714...](https://android.googlesource.com/platform/system/bt/+/3cb7149d8fed2d7d77ceaa95bf845224c4db3baf) of [https://source.android.com/security/bulletin/2020-02-01](https://source.android.com/security/bulletin/2020-02-01):

      -        packet->len = partial_packet->len - partial_packet->offset;
      +        packet->len =
      +            (partial_packet->len - partial_packet->offset) + packet->offset;

I wonder how many devices are running that patch level.

<comment><USER9> My Google Pixel 1 isn't running this patch level, because Google dropped support for it entirely. Which just sucks, because there's no reason not to support it anymore, it's just planned obsolescence.

<comment><USER10> The Pixel 1's kernel and drivers are no longer maintained upstream.

<comment><USER11> Do you mean there's a Linux tree upstream of Google but downstream from Linus who have stopped work outside of Google control?

<comment><USER12> Yes, sort of. Google depends on Qualcomm to make updated drivers, as Qualcomm owns the intellectual property. All Pixel phones are based on a Qualcomm platform.

Qualcomm stops supporting their processors after a few years, usually three. They can make more money by selling new chips. Although I say this cynically, Qualcomm is actually the best third party vendor in this regard.

<comment><USER11> They must have the source code for any chipset vendor kernel drivers due to the GPL so they could keep producing security patches, or require them contractually. Some other Linux vendors keep supporting old ketnel versions for 10+ years after all.

<comment><USER12> Linux kernel drivers don't have to be GPL though. There exist plenty of closed source drivers for GNU/Linux. The "kernel" on Android devices are a small open source kernel and a huge amount of closed source blobs for wireless connectivity, sensors and the GPU.

<comment><USER11> That legal interpretation has been contested a lot. In any case that wouldn't be a blocker for providing security patches to the kernel, just like other LTS Linux distributions do.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER13> A do remember an "SMS storm" for Sony ericsson A200 from 15 years ago.

You get a garbled binary SMS, and then the virus resends itself to every number in your phonebook.

<comment><USER14> What was the end goal of the virus?

<comment><USER15> Before monetization through bitcoin was easy, viruses were usually made and released just because you could. To experiment and see how far it could spread. And maybe just to wreak havoc and troll people.

[https://techgaming.pk/2015/09/07/the-most-dangerous-mac- viru...](https://techgaming.pk/2015/09/07/the-most-dangerous-mac-viruses/)

> This virus first appeared in 1988 on earlier versions of Mac OS. Initially, > the program displayed a message about Michael Dukakis (Democratic > presidential candidate):

> I was created by mischievous 14 year old, and am completely harmless. > Dukakis for president in '88.

<comment><USER14> That's awesome, thanks for sharing.

I assume via BTC or any crypto, it's just a matter of running a "miniminer" of some sort across all effected machines?

</comment></comment></comment></comment><comment><USER16> So uh, as someone stuck on Android 8 forever, what am I supposed to do? Just get a new phone?

<comment><USER17> Disable Bluetooth. Use a cabled connection if you need headphones/mic.

If you need Bluetooth then check whether your phone has the same Broadcom driver - you might be fine depending on hardware. Or check if you can install open source firmware that includes a fixed driver.

There are other things you can do depending on device.

<comment><USER18> How would one disable Bluetooth?

<comment><USER19> apple makes it kind of hard; three taps vs. android one swipe and one tap or are you implying it cannot be disabled b/c it is needed?

<comment><USER20> I am not sure how that is relevant, since this is about a current _Android_ vulnerability.

But it's less of a problem in iOS anyway, because Apple provides updates for iPhones much longer. They are still releasing security updates for the iPhone 5s and 6, which are from 2013 and 2014 respectively.

[https://en.wikipedia.org/wiki/IOS_12#12.4.6](https://en.wikipedia.org/wiki/IOS_12#12.4.6)

</comment></comment></comment></comment></comment><comment><USER7> Yet another typical C exploit.

No wonder that Android 11 will require hardware memory tagging and is now introducing GWP-ASan for the devices without it.

</comment><comment><USER21> Seems like every time there's a new mobile RCE they compel us to buy new phones. And for average person who don't, they'll compound the problem with system level contact tracing.

I give up on technology.

<comment><USER22> Just give up on smart phones . iPhones though on the other hand will get updates for about 5 years which is decent

<comment><USER7> A security crazy person also cannot make use of feature phones as they are already smart enough with networking capabilities.

So it is back to rotary phones and public booth.</comment></comment></comment>


Watch NASA inflate the space station's expandable habitat - <AUTHOR> http://www.engadget.com/2016/05/21/nasa-beam-inflation-livestream/ <comment><USER2> It is about time they got around to it. If you have ever seen a mockup of the space station you know that 10' diameter cylinder that is 13' deep is _huge_.

And with no equipment on the walls it could be a good place to try out activities where you might bump uncontrollably into the wall by accident :-) Either way, its going to be a really interesting space to hang out in.</comment>


HTML to Excel data extraction - <AUTHOR> http://www.listly.io/ <comment><AUTHOR> Thank you, all.

Listly.io is my private work built days ago. I hope to hear opinions if it is useful for you... or not.

Listly.io turn HTML to Excel in seconds without coding. It finds the pattern of repeated structure and extracts all of image links and texts. It does find not tags (table, ul ...), but the structure.

Ideally for developers, I think API would be the best way to adapt this extractor to other scraper or your own scraper.

<comment><USER2> This site is actually really awesome, and has worked for every website I've tried! My only slight issue with it however is it took me a few minutes to actually work out what "HTML codes" were and even then it was only from watching the video. Have you considered renaming it to something like "HTML Source Code"? It also seems to struggle on web pages it can't find tables, such as the following website I made which contains no information:

[https://hastebin.com/eguluvoquq.html](https://hastebin.com/eguluvoquq.html)

<comment><AUTHOR> I appreciate your hard testing and feedback. Your suggestion is very good to me.

Actually, any (partial or full) HTML source code is available;

,

, ,, and etc. Following to your advice, I changed the placeholder description to "any HTML Source Code".

Secondly, my server returns 500 error only if there is nothing to extract such as your code. I will fix it soon. Thank you.

</comment></comment></comment><comment><USER3> I think Google Spreadsheet had something similar

[https://support.google.com/docs/answer/3093339?hl=en](https://support.google.com/docs/answer/3093339?hl=en)

I use it all the time for better sorting, filtering, etc.

<comment><AUTHOR> I have used Google Spreadsheet to extract
 or
 content. It works very well with them.

Compared to it, listly.io works well with all types of tag if there are repeated structures.

In my experiment, it works well with hunderds kinds of web sites.

e.g. Google/Bing search result, Amazon/Walmart/Ebay product list, Twitter/Facebook/Tumblr posts, Twitch list, Bloomberg finance info, Threads of a forum, Instagram comments, and etc.

</comment></comment><comment><USER4> On macOS, you can literally copy and paste tables from Safari into Numbers. Numbers can export to Excel, if you need to.

</comment><comment><USER5> Tried on Indeed jobs listing, Amazon product search, Craigslist and cannot get it to work. I suggest you test the tool with top 10-20 most popular websites that contain listing type data. Our company also did a little side project similar to yours and packaged it as Chrome extension. We learned that it is quite hard to make a unversal tool to guess where data is. Especially that so many websites use
 and
 with CSS to form table like structures instead of plain
. If you want, take a look at our tool: [https://chrome.google.com/webstore/detail/instant-data- scrap...](https://chrome.google.com/webstore/detail/instant-data- scraper/ofaokhiedipichpaobibbnahnkdoiiah)

<comment><AUTHOR> I tested now on Amazon product search, [https://www.amazon.com/s/ref=nb_sb_noss_2/130-9531298-529675...](https://www.amazon.com/s/ref=nb_sb_noss_2/130-9531298-5296754?url=search- alias%3Daps&field-keywords=tv) . It works well though the result comes out slow (about 45 seconds). In result page, you can find the product list with the number of 28. For better speed, I agree to publish API or chrome extension.

In addition, it also works well in seconds with Craiglist apts / housing page. [http://seoul.craigslist.co.kr/search/apa](http://seoul.craigslist.co.kr/search/apa)

Sorry for being slow. This is my private work. I could not predict a lot of new visitors, I need to scale up and out the server.

</comment></comment><comment><USER6> Looks like import.io.

[https://www.import.io/](https://www.import.io/)

I tried writing a script to do the same thing before - turns out finding the element on the page with the most children and assuming each child is an entry works surprisingly often.

<comment><AUTHOR> The difference:

Import.io needs user's click to determine what to extract, thus, the user has to repeat it whenever the web page changes.

Listly.io needs URL or HTML codes. It always works even if the web page chages.

</comment></comment><comment><USER7> I use this extension for tables. Gets the job done: [https://chrome.google.com/webstore/detail/table- capture/iebp...](https://chrome.google.com/webstore/detail/table- capture/iebpjdmgckacbodjpijphcplhebcmeop)

<comment><AUTHOR> I think Chrome extension is the best way for end-users, too.

</comment></comment><comment><USER8> I'm guessing this is doing some kind of tree-diff on the DOM?

Now if you would have this generate a graphQL spec file, you could run a graphQL server acting as a proxy to lots of websites. That would be interesting. Not sure how that fares with the websites' owners' ToS though.

<comment><AUTHOR> Thanks for the idea. It makes me think how to build API. I need to take a look at graphQL.

</comment></comment><comment><USER9> Parseur is doing the same for emails. It's a bit more manual at first but it works better IMHO.

[https://app.parseur.com](https://app.parseur.com)

</comment><comment><USER10> I use a chrome web scraping extension:

[http://webscraper.io/](http://webscraper.io/)

</comment><comment><USER11> Falls flat on this very website.

<comment><USER12> To be fair to the guy, hn is among the worse in markup out there. Great lack of js and css, but boy oh boy is that html ugly.

</comment></comment><comment><USER13> outwit hub is my go to for a large, complex extraction: [https://www.outwit.com/](https://www.outwit.com/)

their marketing is poor but the product is very powrful

</comment><comment><USER14> This looks good, can't wait for the scraper feature.</comment>


Bitcoin value based mostly on belief - <AUTHOR> http://sociable.co/technology/cryptocurrency-value-bitcoin/ <comment><USER3> These ideas are very, very old - Adam Smith and John Locke were writing about the difference between an item's "use value" (how useful it is) and its "exchange value" (what you can trade for it) in the 1700's, and Karl Marx spent a lot of time on this in his book Capital, first published in 1867.

I guess it's worth restating this stuff (because obviously everyone _didn 't_ get the memo) but it's misleading to present these insights as if they were new. It may be that the ideas have never occurred to some people before - <USER2> while it's nice to think up smart things all by yourself, it's even nicer to begin with an education about what's already been figured out, so one can work on new problems instead of centuries' old problems.

</comment><comment><USER4> This is such a hilarious non-article. Just a bunch of people being quoted repeating Econ-101 truisms, and even those presented in a shaky manner.

> "Kardashian is only famous for being famous, as she enjoys no other > intrinsic, artistic value as a performer or entertainer."

Even if she did have artistic talent, the 'value' of that talent would still be based on whether people wanted to listen to her or watch her perform, so it's still fundamentally linked to her desirability and people's belief in it or valuation of it.

</comment><comment><USER5> As opposed to which fiat currency?

<comment><USER6> I think that Bitcoin has done one thing really well and that is to make people understand more about money and how it works. This argument that it has no intrinsic value can be applied to every fiat currency used the world over. Whenever someone asks this I ask them what intrinsic value EUR, GBP or USD has, some people genuinely think/thought that the bank of England will exchange you £10 worth of gold for a £10 note. Explaining to them that not only is this not true but that there is nothing that gives a £10 note any value other than other people believe it to be worth £10 is a real eye opener for them. Explaining then that bitcoin is worth $x because people think it is worth $x makes so much more sense after that.</comment></comment>


Ask HN: Help me with the architecture of my SAAS web application. - <AUTHOR>

I'm about to begin the development of my first ever SAAS web application. I already have 20+ customers of a niche business segment willing to pay for it. Our application will let customers create & manage their business websites through our service either through subdomain or their own domain. e.g., customersite.oursite.com
I am comfortable with both python and php. I have developed web apps before but have never dealt with SAAS. What are the things I need to watch out for? Any known pitfalls? I also have some questions regarding architecture of SAAS apps -
1) Should I put all customer's data in single database or should I use one separate database per customer?
2) I want customers(and 3rd party developers) to be able to develop their own themes much like Wordpress. How should I architect my app to do that?
3) Should I go with Python/Flask or Python/Django or PHP/<Some framework>? I'm comfortable with all three but I don't know which one is better for SAAS app.
Any advice, books, articles, resourced will be nice. <comment><USER2> > Should I put all customer's data in single database or should I use one > separate database per customer?

Do a financial forecasting: given your pricing, how many people do you need to subscribe to break even? Can you enlist enough people for the same vision to be able to maintain your business from SAAS alone?

If yes, keep the whole thing in a single database (reduces maintenance); if the number seems daunting, you might want to consider also selling customization via consulting.

For providing customization, it's generally a good idea to have each client's database seperated from the other, so that individual changes impact only a single client.

> 2) I want customers(and 3rd party developers) to be able to develop their > own themes much like Wordpress. How should I architect my app to do that?

One way how you might achieve that is keeping the code - design HTML strictly separated. At the simplest level, you can do string substitution (eg: <%content%> in the design HTML gets filled up with the content from the database); a templating engine might be useful for complex cases.

> 3) Should I go with Python/Flask or Python/Django or PHP/<Some framework>? > I'm comfortable with all three but I don't know which one is better for SAAS > app.

Go with the combo you have the most understanding of. You will invariably come across edge cases, that aren't covered by these, so you will need to know how you can drop into the framework, and fit it to your needs.

</comment><comment><USER3> I built my SaaS application using Rails & decided to have a separate PostgreSQL schema for each customer's data. It seemed a neater option to me rather than putting account_id kind of field in all of the tables. Also, having a full fledged database for each customer was an overkill for my application.

You can refer these resources:

Architecture Strategies for Catching the Long Tail - <http://msdn.microsoft.com/en-us/library/aa479069.aspx>

Multi-Tenant Data Architecture - <http://msdn.microsoft.com/en- us/library/aa479086.aspx>

p.s. I am also based near Delhi. Feel free to connect with me if you would like to discuss anything in detail.

</comment><comment><USER4> OK. I am developing something similar and here is what I do:

1\. For db, it really depends on the level of security and you app architecture. I am planning on different dbs for each paying customer.

2\. Use a template engine like Jinja(Python) or Mustache(support for a slew of languages).

3\. Python/Flask is good. I looked at both Flask and Bottle but I decided to go with something else that fits my needs. If you want PHP frameworks, there is FuelPHP, Kohana, CodeIgniter, etc.

<comment><AUTHOR> Which framework you have decided to go with and why?

<comment><USER4> luajit within nginx. I use many third party modules and decided to go along with it.

</comment></comment></comment><comment><USER5> Biggest suggestion anyone can give you: Don't prematurely optimize. Period. Python/Django is a good starting point given the good community support. Start with a standard project structure, so that asking technical questions can be answered by anyone out there.

</comment><comment><USER6> Your questions make me think you should start with a much smaller project.</comment>


1Blocker – the most advanced iOS 9 content blocker - <AUTHOR> http://1blocker.com/ <comment><USER2> There is something odd about the "business" of "selling" Adblockers.

<comment><USER3> On the contrary: there is something odd on 'everything is for free'. Because there must be money involved at some point.

<comment><USER2> There are a lot of things in life which are free. Just look around. Not everyone's into making money, when they make something. I will give you an example because I was just contemplating on this the other day (I have no involvement in this). Take a look at this communication platform (1). Read up its FAQs, its TOS, its history of existence, etc. Now tell me, how and why is this free? Then reflect on your own comment about money being involved in "everything".

(1) [https://www.threadthat.com](https://www.threadthat.com)

The Ads exist because they are selling on top of something else altogether. When the Adblockers come on board with the same goal, they defeat the purpose. I am just indicating this apparent cycle, which is fundamentally odd. I am not making a case for or against Ads or Adblockers.</comment></comment></comment>


EVscope – 100 times more powerful than a classical telescope - <AUTHOR> https://www.kickstarter.com/projects/unistellar/evscope-100-times-more-powerful-than-a-classical-t <comment><USER2> So...it's a single purpose digital camera? Cool? I...guess? Can someone into astrophotography weigh in on why this is a meaningful development, as opposed to, say, mounting a DSLR on a motorized base and then also having a nice DSLR to take on vacation?

<comment><AUTHOR> Best to read the comments at [1] as given in another comment

In short, no, it's probably not better than the setup you described.

But I think astronomy clubs would like it. At such events, you have to share scopes and probably have to keep switching between seeing a few stars or planets among a group of people. Having a scope that automatically keeps track of the target and can augment it by displaying some information on what you're seeing in the scope would probably be fascinating and exciting experience.

[1] [https://news.ycombinator.com/item?id=14831715](https://news.ycombinator.com/item?id=14831715)

</comment></comment><comment><USER3> See also:

[https://news.ycombinator.com/item?id=14831715](https://news.ycombinator.com/item?id=14831715)</comment>


Early British Computers (1980) - <AUTHOR> http://ed-thelen.org/comp-hist/EarlyBritish.html <comment><USER2> One of my favourite things to hack around with a few decades back was the EDSAC emulator written in BCPL:

[http://www.cl.cam.ac.uk/~mr10/BCPL/bcplprogs/edsac/](http://www.cl.cam.ac.uk/~mr10/BCPL/bcplprogs/edsac/)

(EDSAC: [http://en.wikipedia.org/wiki/Electronic_Delay_Storage_Automa...](http://en.wikipedia.org/wiki/Electronic_Delay_Storage_Automatic_Calculator))

It was great to be playing with BCPL to emulate these old machines because its so weird - but everything you need to know to write a machine emulator is in this funky language from the past .. and was quite approachable.

So one time for fun I ported the BCPL compiler to the GP2X portable gaming console, just so I could boot up an EDSAC on it and say 'I have one of the oldest computers in the world in my pocket' .. this before the days of iPhones and supercomputers in ones pocket, of course.

Well as a result I was lucky enough to have attracted the attention of Mr. Martin Richards as a result of that port - which I turned over to him after introducing him to the GPX, and consequently spent a few weeks back and forth with him talking about the 'golden age of computing' .. what a blast to have had direct contact with one of the early scientists who brought so much to this industry.

And nowadays I'm still blown away, even though its 'normal' now, at the sheer power of emulation that is occurring.. the folks who bring old architectures to new platforms really get my respect. Nothing like knowing there's a dynarec going on underneath the layers of the toys that my kids are booting up first thing in the morning ..

</comment><comment><USER3> Less old, and a little less british: the linn rekursiv was an odd case. [http://www.brouhaha.com/~eric/retrocomputing/rekursiv/](http://www.brouhaha.com/~eric/retrocomputing/rekursiv/)

</comment><comment><USER4> Both my parents worked on this computers: [http://ict1301.co.uk/1301ccsx.htm](http://ict1301.co.uk/1301ccsx.htm)

One day, when they stop dicking about at bletchley my daughter will be able to program it.</comment>


When One Protected Species Kills Another - <AUTHOR> https://www.scientificamerican.com/article/when-one-protected-species-kills-another-what-are-conservationists-to-do1/ <comment><USER2> So here's a radical idea that's fun to think about. Usually we think of nature as "conditions before humans came along." And we figure that if humans disrupted it, then humans should try to undo the disruption.

But that's just fixing one intervention by means of another intervention, and the fix likely has just as many, if a different set, of poorly-understood and unintended consequences. So lately I've started to define nature a little differently - maybe "nature" isn't what earth looked like before humans, it's what earth would look like if humans disappeared. Or to put it less fantastically, a scenario of minimal human intervention from this point forward.

In that context, the blackberry, kudzu and ivy vines that we introduced, are the new landscape, the new nature, the thing that ends up taking over the sites we disturbed. Similar dynamics would presumably play out in the oceans and everywhere else.

The thing this definition of nature has in common with the more usual one, is that phrase "minimal human intervention." The difference of course is that it doesn't try to undo the recent past, which in general throughout the universe but also specifically in this context, I think is pretty much impossible. So this definition, and the course of action that follows logically from it, has the bonus attribute of being a bit more consistent with current reality.

Of course it'll never happen that way unless _homo sapiens_ really does "disappear," i.e. suffers a dramatic decrease in either demand for intervention (read: population), or ability to intervene, in ecosystems. But it's interesting to think about.

<comment><USER3> "Nature" could also include humans. I haven't seen anything to suggest that the natural outcome of billions of years of evolution isn't some highly intelligent, dominant organism that ends up shaping its environment to an extreme degree.

<comment><USER4> > "Nature" could also include humans. I haven't seen anything to suggest that > the natural outcome of billions of years of evolution isn't some highly > intelligent, dominant organism that ends up shaping its environment to an > extreme degree.

I wonder if the narrative that humans are not of nature makes it easier for us to harm the environment, since we do not feel part of it (as evidenced by our use of the word nature as some _other_).

<comment><USER5> I think this point has merit and it's important for us to think of ourselves as part of the environment, but also, there are lots of other animals (like most of them?) that don't care at all about the environment and the only reason they don't destroy it in pursuit of their vital resources is that they can't. There are endless stories of animals eating and breeding until their ecosystems are destroyed and they suffer massive die offs. Of course that tends to happen most often to invasive species, but nature certainly does not keep itself in balance through conscientiousness.

<comment><USER6> > here are endless stories of animals eating and breeding until their > ecosystems are destroyed and they suffer massive die offs.

I believe the hypothesis that we're _not_ one of these animal species may well prove false.

> Of course that tends to happen most often to invasive species, but nature > certainly does not keep itself in balance through conscientiousness.

Neither do humans. We tend to fuck things up until they become real problems, then mitigate the damage, sometimes with success. That's not really conscientious, that's "winging it". This is the true nature of the human being.

</comment></comment></comment></comment></comment><comment><USER7> It probably wouldn't be practical and woukd require a lot of preresearch and funding to get right but I could see a "buffer" common species to make a more appealing prey. Say getting a huge hatchery full of fish and mass releasing them to boost non-endangered prey numbers for example. Of course doing that without making another disaster would be easier said than done.

<comment><USER8> Yeah, looks like the old engeneering strategy "solve problem A by creating problem B, solve problem B by creating problem C, solve problem C ... etc etc."

Not that I have a better idea how to get out of this mess...

</comment></comment><comment><USER6> 99% of species went extinct before humans arrived. Adapt or die is how life works. Can't survive alongside humans? Tough call.

That's not to say conservationism is futile, I fully agree we should save those Pandas, or various sea mammals, or monkeys and apes. Why? Because they're _fascinating_ and _adorable_. Not out of principle.

But let's be real, even if it wasn't for humans some of these would've been destined to go extinct with just a minor tilt of the ecosystem.

</comment><comment><USER9> we should leave nature alone.

<comment><USER10> Most endangered species would not be so if it wasn't for human intervention and over-consumption. It's arguable that saving a species from extinction _is_ leaving nature alone.

</comment></comment><comment><USER11> I was listening to an interview of Shane Dorian, a well-known big wave surfer from Hawaii. He was remarking on how, when he was growing, when there was a shark attack the locals would go on a shark hunt, killing numerous sharks. Around the nineties sentiments around these hunts changed, and that combined with the depleting of fisheries, shark attacks in Hawaii have risen dramatically. I think it's time we brought back these hunts in the name of protecting otters. I am obviously not just saying this as a biased surfer...

<comment> crooked-v Hawaii's population has also increased from 1.1 million people in 1990 to 1.4 million people today. There's no great peculiarity around the fact that more people means more encounters with sharks.

Also, "dramatically" is quite the claim, when there's a total of only 137 recorded shark attacks in Hawaii ever (including incidents without injuries) dating back to the 1700s.

<comment><USER11> I was just parrotting the interview I listened to but after looking into it it's not false to say that shark incidents are definitely increasing in frequency from using information from [https://dlnr.hawaii.gov/sharks/shark- incidents/incident-grap...](https://dlnr.hawaii.gov/sharks/shark- incidents/incident-graphs/)

Just eyeballing here there was 6 shark atacks from 1980-1985 to about 50 in the last 5 years. Seams like that backs up his claim that there has been a dramatic increase.</comment></comment></comment>


Operators and Meta-operators in Perl 6 - <AUTHOR> https://perl6advent.wordpress.com/2013/12/22/day-22-a-catalogue-of-operator-types/ <comment><AUTHOR> You can check the whole series: [https://perl6advent.wordpress.com/category/table-of- contents...](https://perl6advent.wordpress.com/category/table-of-contents/)

</comment><comment><USER2> Perl 6 is really an interesting language. There are a lot of tools like this that give you the power to say a lot with a very small number of characters.

(The flip side, of course, is that you can easily make incomprehensible code—I wouldn't want to read code using any but the most natural user-defined operators—but if you set that aside, Perl 6 has a ton of cool ideas.)

</comment><comment><USER3> The meta-operators remind me of adverbs in J

</comment><comment><USER4> So what's the situation of Perl 6? Is it production ready? Any demonstration code?

<comment><USER5> (Assuming you're asking from a really long distance out) Well, there's two current implementations of it (with Rakudo being actively developed): [http://www.perl6.org/compilers/features](http://www.perl6.org/compilers/features)

As for example code, there's some on [http://rosettacode.org/wiki/Perl_6](http://rosettacode.org/wiki/Perl_6) in the bottom section. With all that said, is it production ready? I don't know. I've heard people joke that Rakudo crashes every couple hundred lines of code it executes. Though the parrot developers have said that parrot itself is extremely stable, and they've had decently sized programs running for some time on it. That, and Rakudo is currently working on two new backends besides parrot: JVM and a new vm called MoarVM.

<comment><USER6> While Rakudo certainly crashes more than anyone wants, it doesn't crash anywhere near as often as you suggest. It's a good thing, as it is implemented in itself; you couldn't get to line one of your script without crashing if it crashed every couple of hundred lines.

Actually, I don't know when the last time I got a random crash in Rakudo was; it may have been years ago. Getting a crash because you ran out of memory, on the other hand, definitely can happen in bigger programs.

</comment></comment></comment><comment><USER7> I'm genuinely curious - why are programmers still using Perl instead of Python?

<comment><USER8> Really? Python has magic everywhere and requires methods to take self as an argument. Indentation instead of brackets. It's one big hack. Perl is a much nicer language. The real question is, why would I ever use Python over Perl?

<comment><USER9> What in Python is "magical"? The fact that methods take a "self" argument is a result of it _not_ being magical.

<comment><USER8> Methods taking "self" as an argument is a dirty hack, ain't no magic there. The language is just pretty ugly. It's got the whole whitespace thing going on, which is forced, and some might think it gives a good aesthetic but that is dust in the wind in terms of help for Python's ease on the eyes. The whole standard namespacing and mix of oop/procedural global functions is just..schizophrenic, the capitalization scheme is dastardly. I haven't seen many languages that look more _basic_ than python. It's a frankenstein of Perl/Ruby and Basic. Forced verbosity is egh.

Here's your magic methods and properties: [http://www.rafekettler.com/magicmethods.html](http://www.rafekettler.com/magicmethods.html)

Those magic methods kind of remind me of clever hacks while coding in PHP.

<comment><USER10> Magic methods have always been my favourite part of python, they're the developers way of ensuring we can do anything that they can do, if there isn't a magic method for it, it's not possible. We can create our own version of anything in the standard library.

Self is one of those things, if you wanted to you could easily add another instance variable, just implement getattr and have it return a partial function. Although I will admit it is one area which does cause more than its fair share of errors, but I prefer it to Ruby's methodology.

<comment><USER8> Are the magic methods API frozen?</comment></comment></comment></comment></comment></comment>


Microsoft's strange new ads for Internet Explorer - <AUTHOR> http://www.idsgn.org/posts/microsofts-strange-new-ads-for-internet-explorer/

<comment><AUTHOR> I've been hyper-critical of Microsoft's recent ad campaigns, but there is _nothing_ not to like about these. Fantastic spots! If the browser they're hawking had benefitted from the same attention to detail, Safari/Chrome/Firefox would have a real fight on their hands.

Oh, and _go Dean Cain_!</comment>


NYC Proposes API Requirement Bill - <AUTHOR> http://fortune.com/2017/05/23/new-york-city-government-apps/ <comment><USER2> I read the title and I immediately went to a dark place. "Oh no, do they want to somehow regulate APIs?"And then I read it and realized what this is and it is a brilliant initiative.

Government needs to be more accessible.

</comment><comment><USER3> this would be a great step towards open government code</comment>


Hello, Rust – Go vs. Rust: Concurrency and Race Conditions - <AUTHOR> https://www.youtube.com/watch?v=B5xYBrxVSiE <comment><AUTHOR> One could also write the Go version using channels [1] and the Rust version in a more functional way [2], but overall the code is quite straightforward in both languages.

Only thing I don't like about the Rust version is, that the error message is still a bit cryptic:

``` Cannot borrow data mutably in a captured outer variable in an `Fn` closure. ```

But the important part is that Rust prevents a data race at compile time, which is nice.

[1]: [https://github.com/hello-rust/show/pull/46](https://github.com/hello- rust/show/pull/46) [2]: [https://github.com/hello- rust/show/pull/45](https://github.com/hello-rust/show/pull/45)</comment>


My Favorite Algorithm: Metropolis-Hastings - <AUTHOR> http://flynnmichael.com/2015/06/01/my-favorite-algorithm-metropolis-hastings/ <comment><USER2> I like the build-up explanation in this chapter (Monte Carlo Methods) from MacKay's book: [http://www.inference.phy.cam.ac.uk/mackay/itprnn/ps/356.384....](http://www.inference.phy.cam.ac.uk/mackay/itprnn/ps/356.384.pdf)

</comment><comment><USER3> In practice, there are some nicer algorithms for doing Markov Chain Monte Carlo, such as Goodman & Weare's Affine Invariant sampler, as implemented in [http://dan.iel.fm/emcee/current/](http://dan.iel.fm/emcee/current/) . This algorithm has the advantage of not needing a proposal distribution.

</comment><comment><USER4> I always go back and forth on which of the two things are more amazing: the fact that we can accurately estimate the rendering equation in tractable time, or the fact that the universe manages to do it in real time.

<comment><USER5> The universe can take all the time it wants. You wouldn't notice.

</comment></comment><comment><USER6> Listen to Press on the naming of the algorithm, seems like Metropolis just put his name on the paper:

[https://youtu.be/4gNpgSPal_8?t=938](https://youtu.be/4gNpgSPal_8?t=938)

<comment><AUTHOR> This is true! It should be called the Rosenbluth-Hastings algorithm.

</comment></comment><comment><USER7> Interesting choice...

Seems like genetic algorithms, particle swarms etc. would be more attractive choices since they solve the same problem and are inherently parallelizable while Metropolis Hastings is almost 100 years old and designed for a 4 function calculator.

Although I guess some people have meta-parallelized it... but still seems like a patch job compared to modern likelihood navigation algos.

<comment><USER8> One of the parallelized versions is Gibbs sampling that is used for sampling from Bayesian networks. In this case you don't even need a proposal distribution; neither would you need the test from MH.

I think Graphlab ([https://dato.com/](https://dato.com/)) comes implemented with something of this kind.

The trouble with Particle swarms/Genetic algorithms is that they aren't guaranteed to sample from the underlying p.d. It is not yet apparent whether you can find the mode of a distribution faster by choosing a Markov chain whose stationary distribution is different from the underlying one.

<comment><USER7> Are you sure that Gibbs sampling isn't just the multivariate version of MH?

What I'm saying is that convergence speed for MH is limited by the fact that guesses cannot communicate with each other... which doesn't matter when you have a pencil and a 4 function calculator like when it was designed.

A genetic algorithm or a particle swarm algorithm is capable of much swifter convergence because the guesses _can_ communicate and influence the direction of the drunken walk.

<comment><USER8> It _is_ a multivariate (co-ordinate wise) version of MH. You can paralleize it because the Bayes net allows a decomposition of the p.d.

I feel like while your comment on Global-optimization algorithms may indeed be true, I don't quite yet believe that the hacks they involve are quite that general yet.

MH wasn't designed for Global optimization, and there is only one "particle". I guess this what you meant by "parallel" ?

<comment><USER9> Well, you can run multiple simultaneous particle chains and sum their results. There's some wasted work since each will need to burn in, but modern algorithms can make that go quite quickly.

<comment><USER8> _Sure_ \- but that's not what the OP is referring to. There is no "sync" step in multiple parallel chains, unlike particle swarm & genetic algorithms.

MCMC is also used for finding MAP solutions, an operation which is strictly less difficult than computing the partition function.</comment></comment></comment></comment></comment></comment>


Exploration of NYT Crossword answers from 1994-2017 - <AUTHOR> https://github.com/jtanwk/nytcrossword <comment><USER2> It sounds like they wanted to do the scraping themselves, but for anyone else who wants to play with this data, it's available for download at [http://xd.saul.pw/data](http://xd.saul.pw/data).

<comment><USER3> Also here: [https://github.com/doshea/nyt_crosswords](https://github.com/doshea/nyt_crosswords) and a couple of other repos on GH.

<comment><USER2> Thanks for posting this, but not all data is created equal. These crosswords only go back to 1976, are in .json format, and require nontrivial processing to get into a useful format for data analysis. By contrast, I very meticulously designed the formats available at xd.saul.pw to be easy to use. You can even extract some interesting results with just e.g. `grep` and `wc`.

<comment><USER3> Thank you, I did not realize at first that the link you posted is so complete! I PM'd you on reddit a little while ago asking for the dataset, but didn't have the time to play with it yet. Thanks again for your time and consideration.

</comment></comment></comment></comment><comment><USER4> This was actually really fascinating to me. I loved to see all the words that are _important_ :

\- nsfw - This one is hilarious. Of all the internet terms (e.g. AFAIK, AMA, LOL), this is the most used.

\- laiditontheline - Why is this important in 2016? Does "Laid it on the line" have a particular meaning?

\- idriselba - This one is great too. Stringer Bell!

</comment><comment><USER5> It's interesting that word length emerges as such a clear proxy for difficulty. I often find it easier to think of a longer word or phrase (assuming no wordplay) than something short and obscure -- the parameter space is much smaller.

Also interesting to see the step increase between Thursday and Friday. This finding is consistent with my anecdotal experiences, but I doubt it's deliberate on the part of the editors. I think there is probably a technical explanation for this. For example, maybe there are fewer words available of medium to long length between the set of shorter words and multi-word answers, so no smooth transition between Thursday and Friday is possible.

<comment><USER6> Sunday's word length slots in nicely between Thursday & Friday, which matches what I've read the NYT editors desire (for Sunday to be "roughly as difficult as thu/fri") ... and gainsays your hypothesis =)

Rather than the set of possible answers being U-shaped, maybe it's a size thing: it is hard to fit in N medium-sized words unless you have the extra space of the Sunday puzzle.

The top & bottom of Fri/Sat puzzles are often split into just two words (one long, one short), this might inform our developing hypothesis.

</comment></comment><comment><USER7> That third chart is a thing of beauty!

<comment><USER8> what was the clue for CCCCCCC?

14 across - sound made by someone choking</comment></comment>


The new and improved Programming Languages Zoo - <AUTHOR> http://math.andrej.com/2016/09/07/the-new-and-improved-programming-languages-zoo/ <comment><USER2> Great post, this was an old goldmine that is easy to forget and think it's just stacking dust.

Many thanks to everyone.

</comment><comment><USER3> Link to the Programming Languages Zoo, [http://plzoo.andrej.com/index.html](http://plzoo.andrej.com/index.html)</comment>


Music Notation with HTML5 Canvas - <AUTHOR> http://0xfe.blogspot.com/2010/05/music-notation-with-html5-canvas.html

<comment><USER2> You may want to look into the Lilypond syntax for textual score notation. - <http://lilypond.org/>

<comment><USER3> Lilypond's web site could benefit from some examples of its output directly on the front page. I tried three different approaches in my casual attempt to see just how good it is, and didn't reach any PDFs or printouts.

<comment><USER2> Yes, examples and docs are a bit buried, but there is a huge amount of info to be found.

Example outputs - <http://lilypond.org/switch/tour>

Example syntax - <http://lilypond.org/switch/howto>

</comment></comment></comment><comment><USER4> Oh man, music notation on a computer is a path you don't want to start down. Take the oldest, crustiest, hacked together duct-taped programming language you can think of -- imagine what that language would be like if it would be still in use hundreds of years from now. That's music notation, today.

Even the really good programs like Sibelius (and I guess Finale...) still have a ways to go in terms of accurate notation. Attempting to reduce notation to a JSON-style dataset seems, well, extraordinarily challenging.

<comment><USER5> Isn't the majority of the challenge in nice notation similar to typesetting with text? The difficulty is in making it look nice, not knowing what data you're attempting to display.

I should think you could encode pitches, start times and durations in a JSON- style format without many problems.

<comment><USER6> You could start out that way and get pretty far, but eventually the language will get complicated enough that you'll want something more like LaTex than like JSON if you want it to be readable.

Musical notation is often deliberately ambiguous to allow for interpretation. Tempo is often approximate. Durations are often approximate or implied (such as swing time). Volume is very approximate. And then there are various kinds of hinting. The hints need to be in the source code because nobody wants to specify them with more precision.

But that said, I'd be perfectly happy with something basic that runs in a browser.

<comment><USER7> For some arenas of music, such as jazz combo-oriented groups, a very basic melody lead sheet with chords and song structure elements (section a, section b, repeat, etc.) is plenty.

Maybe rather than focusing on building a full music notation system for the web, focusing on making something for lead sheets / rhythm charts / etc would be a better path. I really don't like breaking out Finale to do jazz rhythm charts, but right now it's just about the best option. Make something better, and I for one would pay for it.

</comment></comment></comment></comment><comment><USER8> If the goal is to build a wysiwyg editor, finale and others have some implementation that regular musicians have learned to work with.

For a lot of people, the ability to do midi input is really important because when they write the music down a lot of the time they have to transcribe from their motions anyway. This is possibly more important than the actual explicit way that you write things down.

</comment><comment><USER9> With MathML in the new Firefox engine and now music notation in canvas, browsers are going to be a bit more fun and beautiful.

</comment><comment><USER10> Is there a link to the unminimized source for this? Looks awesome!</comment>