4 million songs on Spotify have never been played. Let's change that - <AUTHOR> http://forgotify.com/ <comment><USER2> There was a funny story[1] (transcript[2]) on the NPR program _On The Media_ a couple of weeks ago about someone[3] who writes and records dozens of songs per day to put on Spotify (playlist[4]). Many songs deserve to not be played.

[1] [http://www.onthemedia.org/story/100-songs- day/](http://www.onthemedia.org/story/100-songs-day/)

[2] [http://www.onthemedia.org/story/100-songs- day/transcript/](http://www.onthemedia.org/story/100-songs-day/transcript/)

[3] [http://www.moternmedia.com/bio.html](http://www.moternmedia.com/bio.html)

[4] [http://spoti.fi/1gnQ7nd](http://spoti.fi/1gnQ7nd)

<comment><USER3> There are also hundreds of "cover" songs on Spotify that, from what I can tell, only try to earn plays through mistaken clicks.

For a good example, search for Thrift Shop -- you might be inclined to play "Thrift Shop" by "Thrift Shop" on his hit album "Thrift Shop." It has 675k plays already.

<comment><USER4> AC/DC has an interesting twist related to cover bands -- the band only has one song (through an old soundtrack) on Spotify. About a dozen cover bands have filled the gap; the top tunes for each of them collectively have over a million listens.

</comment></comment></comment><comment><USER5> Most songs don't deserve to be listened to.

In the old days one had no choice - whatever was on the radio was what you heard, and it was very difficult to hear the whole CD/tape/ancient-media. I used to have a 3-4 song rule - if I didn't like 3-4 songs on the album I wouldn't buy it. It was very hard to adhere to, and breaking the rule almost always resulted in disappointment.

MP3 changed that, but not because of pirating - because listeners could start sampling the whole product to see if they wanted to buy it. Tech improvements keep making that easier. Bands will have to work harder to produce quality work for their fans (including having a good handle on what 'quality' and 'fans' means for them), instead of producing one good song and ten fillers and calling it an album.

</comment><comment><USER6> It's not very (at the track level) - but at the same far too (at the 'release' level) - accurate.

"Wayward Wind" by Gogi Grant (off "Sweet Memories") is listed -but- the track itself is listed as one of the most popular if you search for her, just not off that particular album. There's a metric ass-ton of releases which probably explains why one particular release has zero even when the track is clearly very popular.

F-. MUST TRY HARDER.

<comment><USER7> This is very annoying on Spotify itself as well. If you use the 'radio' feature (basically random tracks based on some genre/artist), it often repeats the same track over and over, from different releases.

<comment><USER8> Oh, so THAT'S why it repeats tracks! I always assumed that the radio feature just wasn't very smart.

</comment></comment></comment><comment><USER9> Does OP or anyone know how it's built?

I see there's a track popularity figure in the API [1] so maybe these songs have zero popularity. And then I can only assume they've crawled every track as the search API [2] doesn't support filtering by popularity.

1\. [https://developer.spotify.com/technologies/web- api/lookup/](https://developer.spotify.com/technologies/web-api/lookup/)

2\. [https://developer.spotify.com/technologies/web- api/search/](https://developer.spotify.com/technologies/web-api/search/)

<comment><AUTHOR> Sorry, I don't. Your explanation sounds pretty plausible.

</comment></comment><comment><USER10> This is the antithesis to sites promoting quality content (hn, reddit, etc). It only encourages people to publish shit. Like this guy: [http://www.gigwise.com/news/88051/spotify-joker- earns-%C2%A3...](http://www.gigwise.com/news/88051/spotify-joker- earns-%C2%A313000-uploading-slightly-rubbish-tracks-online)

May as well try and rack up views for all the unseen ads on the internet.

<comment><USER11> The "Knights of 'New'" on reddit have to wade through a lot of junk, but everyone else who uses the website benefits from their efforts.

This is a similar deal for Spotify. Some percentage of these songs are good, it might be less than one percent, but if you can get 500 people with decent ears to listen to 1 song each, you can move the needle.

</comment></comment><comment><USER12> Would be nice to have a counter of the number of forgotten songs, updating in realtime.

<comment><USER13> I absolutely agree!

</comment></comment><comment><USER14> Their cache engine will love you for this!

</comment><comment><USER15> I'm gonna be "that guy" and complain again that developers (web developers in particular) really need to learn the basics of scaling. There is _no_ point having your app publicly available if you're not prepared for small spikes like this.

Edit: At the time of posting, the service was 503ing.

<comment><USER16> Is there a service that tests the scale of web apps? I'm _not_ thinking of software, rather a service that web developers could use to test their app with simulated "real" traffic..

<comment><USER17> There is Loader.io from SendGrid ([http://loader.io/](http://loader.io/)) that lets you load test your web apps with thousands of concurrent connections.

<comment><USER16> Thanks..

I wonder if there's a marked for (helping to) optimizing sites and web-apps..

</comment></comment></comment></comment><comment><USER16> This is great for hipsters who want to start listening to a band before they're popular :)

Since the chances are kind of slim, you'd have to listen to a lot..

<comment><USER18> Thou shalt not stop liking a band just because they have become popular.[1]

[1] [http://www.youtube.com/watch?v=CWrMGXwhFLk](http://www.youtube.com/watch?v=CWrMGXwhFLk)

<comment><USER19> Yeah, you just gotta tough it out; eventually, _every_ band becomes unpopular again ;)

</comment></comment></comment><comment><USER20> There probably a very good reason why people don't listen to those songs ;)

<comment><USER6> Well, I can vouch for Dimitris Arapakis being a bit hard going.

Imagine a half speed version of Nusrat Fateh Ali Kahn mating with a Klezmer band and you'd be about right.

<comment><USER21> > Imagine a half speed version of Nusrat Fateh Ali Kahn mating with a Klezmer > band and you'd be about right.

Thanks, I will definitely be checking this out later.

</comment></comment></comment><comment><USER22> When DistroKid launched, I put a song on iTunes, Amazon and Spotiy. So far it has had no sales, but 3 streams, earning me 2 cents.

I highly suggest everybody streams 'Goatsemon - Penis Bird' or buys it on Amazon. [http://www.amazon.com/Penis-Bird- Explicit/dp/B00FQS30W6](http://www.amazon.com/Penis-Bird- Explicit/dp/B00FQS30W6)

<comment><USER23> I'd rather download it off the Pirate Bay.

<comment><USER22> I just checked, it's not there. But it is on YouTube and my old Geocities page has been mirrored. You could find it if you wanted too, but it's not worth the effort.

</comment></comment></comment><comment><USER24> Spotify has around 20 million songs [1], so around 20% of songs are never played.

Two random questions:

\- How much is Spotify paying to host those unplayed songs?

\- How many of songs have a small number of plays? (< 5 plays, for example)?

[1] [http://press.spotify.com/us/information/](http://press.spotify.com/us/information/)

<comment><USER25> Storage is cheap with digital music. Even assuming 10 MB/song, you're looking at 40TB, less than $4k/mo on S3. It's not trivial, but it's probably a drop in the bucket compared to their other costs. Realistically the file sizes are much less than 10MB.

<comment><USER26> They have multiple encodes in various qualities for each title too, though.

<comment><USER27> Do they produce these from the source audio _before_ they're listened to though?

</comment></comment></comment></comment><comment><USER28> 4M? Interesting. One of the recurring themes of "long tail" discussions is that _everything_ is requested at least once every month or so. Would like more consideration of why that's not the case here.

<comment><USER29> I was in a band in my college years that played this "indie / experimental" instrumental rock. We only played maybe a dozen shows that were sparsely attended. There's one show we played that had at least 50-100 people, but that was because we opened for a semi-popular local band. In 2003-2004, just about when I left the band, we recorded an album in on of the band member's bedroom. I doubt more than a handful of people have ever listened to it.

This album is on Spotify. If it has any listens it's probably because I listened to it, or maybe another one of the former band members did.

<comment><USER30> Alright what is it. Now I want to hear it.

<comment><USER29> Album is _Hurt Seconds_ by The Shy Trafficker. I'll warn you that it's a long poorly recorded slog. There's a couple songs with some vocals but most of it is instrumental. Instrumental rock tends to be pretty unpopular I've found so I felt like the "warning" was needed.

<comment><USER30> Well Explosions in the Sky do pretty well.

</comment></comment></comment></comment></comment><comment><USER31> Fun times listening to:

Lesson 7: Education and Government part 5 by Morris Schreiber

</comment><comment><USER32> It would be nice to have this as a real Spotify app, they're written in HTML5 so it's pretty easy to do.

</comment><comment><USER33> "Sorry, we're not available where you are. Leave your email to stay tuned for good news."

Thanks Forgotify!

<comment><USER34> I really wanted to listen to the first random song, as well! What is the "I have Spotify" button for? It doesn't work for me... I was hoping that would let me log into my (US) account without being in the country, but nothing happens when I click it.

</comment></comment><comment><USER35> i only get Indian music :p

<comment><USER36> Getting mainly german schlagers here

<comment><USER37> Mexican music here.

<comment><USER38> Service Unavailable music here

<comment><USER39> I love that stuff.

The 404's playing their 808's, buzzing happily into oblivion.

</comment></comment></comment></comment></comment><comment><USER40> I got a cool looking jazz trumpet piece, but it seems to want me to sign up for a Spotify account. I guess I should have known, I'm just too used to sound cloud.

</comment><comment><USER41> HTTP Error 503. The service is unavailable.

</comment><comment><USER42> 503 Error.

</comment><comment><USER43> I was going to listen then I saw how the extension was Coldfusion...

</comment><comment><USER44> Surprisingly, just heard a song that I loved on it, [http://open.spotify.com/track/6cZ1tXMm5BaYQzddgcY7W9](http://open.spotify.com/track/6cZ1tXMm5BaYQzddgcY7W9)</comment>


Why scientists and engineers don't make billions of dollar as businessmen do? - <AUTHOR> <comment><USER2> They (or anyone for that matter) can make billions if they learn the art of selling.

<comment><AUTHOR> If so, most salesman would be billionaires ?

<comment><USER2> But salesmen are not the owner of the stuff they are selling. You should own the stuff (patent etc) and should be able to sell it to people.

</comment></comment></comment><comment><USER3> Businessmen hire scientists and engineers, not the other way around.

</comment><comment><USER4> They do when they become businessmen like Gates and Ellison.

<comment><AUTHOR> I guess the question is really why is it that only businessmen make the bulk of money. Is sweet talking, managing, being social and creating hype really needed to make money even in this age of Internet.</comment></comment>


Third Party Sellers Need To Rethink The Amazon FBA Program - <AUTHOR> http://www.startupnation.com/start-your-business/plan-your-business/third-party-sellers-need-to-rethink-theamazon-fba-program/ <comment><USER3> Here's an interesting hypothetical.

Step 1: User A purchases a shrinkwrapped DVD with excellent packaging from a counterfeiter in China.

Step 2: User A sells the DVD to an FBA merchant, who forwards it on to Amazon.

Step 3: User B buys the DVD from Amazon; due to commingling they get the counterfeit DVD. Before they can open it they get hit by a bus.

Step 4: User B's executors sell the DVD back to an FBA merchant, who forwards it on to Amazon without a second thought (since it's a shrinkwrapped DVD straight out of Amazon's warehouses.

Step 5: User C purchases the DVD from the FBA merchant, opens it, and finds its counterfeit.

In short, the FBA/commingling stuff means that merchandise you buy directly from Amazon can be untrusted. In turn that means that, as an FBA merchant, even merchandise that came directly from Amazon can be untrusted. Any shrinkwrapped DVD that has passed through an Amazon fulfillment warehouse is actually of entirely unknown provenance; it could have been purchased at a stall in Shanghai for all you know. And if your business relies on never selling a counterfeit DVD, then that means nothing you buy _or_ sell can ever touch an Amazon warehouse.

I doubt this is a real concern (this is the first I've heard of any issues with counterfeiting and FBA), but it's still an interesting structural flaw. Given Amazon's volume, they're probably shipping out multiple counterfeit DVDs right now, under their own shipping label as well as that of various FBA merchants.

Fascinating.

<comment><USER4> > Any shrinkwrapped DVD that has passed through an Amazon fulfillment > warehouse is actually of entirely unknown provenance; it could have been > purchased at a stall in Shanghai for all you know.

Not to disagree, but as someone who lives in Shanghai and occasionally buys pirated DVDs, the quality (for a western show) is generally _at least_ as high as you'd expect of official merchandise. I got all of Futurama on many fewer discs than the official stuff uses, at no visible hit to quality.

I find it unlikely that a user, as opposed to a media company running stings, would ever complain.

<comment><USER5> To disagree, no.

 _the quality (for a western show) is generally at least as high as you 'd expect of official merchandise_

 _I got all of Futurama on many fewer discs than the official stuff uses, at no visible hit to quality_

If you got Futurama on many fewer discs than the official edition, it is extremely likely that the quality was not "at least as high as you'd expect of official merchandise." You may not have noticed it, in your viewing scenario, but it is almost guaranteed that your copies were compressed at a much higher level of compression loss. That, and the physical media itself is of unknown quality.

As a user, I would complain. Loudly.

<comment><USER4> Higher level of compression, of course. Why higher level of loss? There are various compression technologies. Some are better and worse, according to your purpose.

My viewing scenario was a 16:10 17" screen (I believe 1440x9XX) and Bose headphones. That beats the resolution expectations of broadcast TV, and definitively crushes the audio expectations.

Full disclosure: I've watched Futurama in the form of torrented episodes (quality varies), chinese bootleg DVDs (high quality), and official DVDs (also high quality).

May I ask about your experience with Chinese DVDs?

<comment><USER5> I hesitate to respond, only because this is becoming somewhat off-topic to the article itself.

That said, I was assuming the scenario of a counterfeit DVD that would play on any standard DVD player. Sure, there are more efficient compression algorithms now, but most aren't supported as standard on every DVD player out there. The base DVD specification is rather limited in codec support.

Those Futurama counterfeits - which compression algorithm are they using? Will they play on all standard dvd players?

<comment><USER4> Sadly, I don't know the answer to this; even the impeccably-licensed official translated Disney movies I've bought don't tend to work in US DVD players (region coded, I guess), so I never try. I play DVDs via computer (for one thing, that means I don't have to buy a DVD player in China; I already have computers). I don't have the discs with me, so it's impossible to check. :(

I can think of a couple scenarios -- (a) my error, it's actually the same number of discs, but packed into one box instead of four boxes; (b) they're nonstandard, but still recognized as DVDs by VLC; (c) (highly speculative) something about the limited palette of cartoons makes them more susceptible to compression, and that was exploited.

For what it's worth, the reason I included the qualifier "(for a western show)" is that I once bought a set of bootleg DVDs for a chinese TV show, and they were compressed to the point where I found them unwatchable. They still played through VLC; my best guess is that DVD-making software was used on ripped, heavily-compressed video files. It is entirely possible that something similar was done for my Futurama discs, although no compression was visible in that case.

</comment></comment></comment></comment></comment></comment><comment><USER6> Mysterious.

"80 companies were sued in the past 12 months".

Where did the new counterfeit items come from? Is someone making them and feeding them into these guy's pipeline? How does that happen?

How were there enough counterfeits to sue 10 companies a month? What did the purchase history look like for his last buyer of a new "the mentalist"?

Did the author mean to imply that Amazon might have shipped a forged copy that Amazon procured?

Why is the tone of the article so glib? Why was it posted on a site like that, and where is the follow up, now that it's a year later? And whats with his next article about selling xbox games in June

I'm a cynic and conspiracy theorist, but I can't help but wonder if this tribe of amazon merchants found a source for "new" DVDs that was too good to be true, and looked the other way while passing them to amazon. A more outlandish theory would be that the rights holders set these guys up to destroy huge volumes of used merchandise.

<comment><USER7> "A more outlandish theory would be that the rights holders set these guys up to destroy huge volumes of used merchandise."

My first thought when reading the OP was that this is what probably actually just happened

</comment></comment><comment><USER8> Certainly avoid buying books sold through this process, Amazon doesn't pack them carefully like it does their own books.

And I must say, I always wondered why Amazon seemed to be so "fair" to apparent competitors. Looks like "seemed" was right.

Also emphasizes the wisdom I read a while ago about YC not touching music related startups with a 10 foot pole. Media with insane rights holders does not sound like a safe domain to play in.

<comment><USER9> I think Amazon's thought process here is that if there's going to be an after market, they might as well get paid for it. This also could grow the total market size if people can easily resell their books, rather than pitch them or give them to goodwill.

It's kind of like companies selling their premium soap under one label, and then selling the excess inventory to private label brands.

<comment><USER8> Oh, I agree 100% on the after market part, and Amazon makes a not inconsiderable amount of money from me that I do not begrudge in the least because of the quality of their marketplace, including useful quality scores for their merchants (e.g. I've learned to not touch anyone below 96% outside of special cases).

However here we're talking about direct competitors in selling new goods. Look closely at the right items, and you'll see that Amazon sometimes does two amazing things: if someone else is offering something new for a lower price, they, not Amazon will get the sale if you just click on the Buy button. (Of course, I'm often willing to pay a bit more to get it directly from Amazon.) If 2 or more including Amazon are offering it at the same price, Amazon sometimes rotates who gets the sale if you just hit the Buy button.

That's what I meant by "fairness", although I couldn't find any examples of this just now (ADDED: that might be because according to another commentator in this topic they only offer this for highly rated merchants). Then again if Amazon is analyzing my purchasing habits, they know a) I prefer to buy direct from them and b) for most of what I buy from them unit wise (but not price), I always check out lower priced new and used offers from their merchants.

And, yeah, especially with how much media I buy that's out of print, it's win- win-win for them. But to the extent this story is true ... well, it was a "new" item that was claimed to be counterfeit.

</comment></comment></comment><comment><USER10> Co-mingling is a nightmare, even if you aren't an FBA seller.

Amazon was a reseller for a product we launched on Kickstarter. Our product was a precision optic, made in the USA. A few months in, a counterfeit knock- off of our product was manufactured in China and distributed in great quantity. The counterfeit items had the same bar code and nearly identical packaging (except it said "Made in China"). The knock-off wasn't optical quality and took lousy pictures.

Third party FBA sellers were buying the counterfeit unit and stocking them at Amazon. These would appear as "Other Sellers" on our product page, and would usually be selected by default because of crazy low pricing. Amazon co-mingled the counterfeits with their own inventory of genuine goods that we sent them. They began to draw from the counterfeit stock for their own sales instead of the real deal. Customer reviews tanked.

We figured out what was happening since we knew these other sellers weren't legitimate, and we received counterfeit returns. We were able to ask Amazon to shut down the sellers, but the tainted goods remained in inventory. Amazon was either unwilling or unable to screen their inventory to eradicate the counterfeit goods. Since they were useless in fixing the problem we just stopped selling to Amazon and labeled the product page as counterfeit.

</comment><comment><USER11> The Seller (Amazon client) operated his business from paycheck to paycheck (i.e. invoice to invoice) with no reserve fund, no insurance and no set-aside for legal. He lost the lawsuit because he never filed an appearance. There are lots of lawyers who would handle such a case for much less than a $25,000 retainer. At the end of the day, though, if your business depends on Ebay, Google, Amazon or their ilk, you have no security.

<comment><USER12> >" _He lost the lawsuit because he never filed an appearance._ " //

Is that how law works in USA, forget finding if the claimant has a case just destroy the little guy?

As this was a tort case, assuming it's like English law (big assumption), then the measure of guilt is the balance of probabilities. On balance with the evidence available did the defendant commit copyright infringement? Doesn't appear so - there's one DVD that no one can actually link to the supplier.

Also "He lost the lawsuit". Wasn't he just the CEO of an LLC?

<comment><USER11> Depends on whether he was named personally or not. Based on his post, a default judgment was entered. You'd have to take a look at the complaint. What the facts are in reality really doesn't matter--unfortunately--because he did not appear in a tribunal to prove or assert those facts. Don't forget that in U.S. courts (as in the UK) the uncorroborated testimony of a single witness is enough to meet the burden of proof, even in a criminal case. Whether you prevail or not is something else, but in terms of a minimum threshold, all you need is the testimony of the person who placed the order and who received counterfeit goods. The burden then shifts to you to prove that a) the witness is mistaken or lying or b) the goods are not counterfeit.

But he never showed up. Moral of the story: buy insurance. Budget for legal in the same way you budget for accountants.

</comment></comment></comment><comment><USER13> As someone with 20,000 units in FBA right now, that is horrifyingly scary.

<comment><USER14> So, what are you going to do about it?

(I am asking honestly, the last time someone I know said something along these lines they just said "Oh my god, that is terrible" then promptly did nothing and just hoped for the best.)

<comment><USER15> What can they do? Amazon charges a fortune to return FBA items or dispose of them...

<comment><USER14> Wind down stock and use a different program / partner and abandon the FBA program would be one option.

</comment></comment></comment></comment><comment><USER16> It was Warner Brothers.

[http://www.ecommercebytes.com/cab/abn/y13/m02/i25/s01](http://www.ecommercebytes.com/cab/abn/y13/m02/i25/s01)

</comment><comment><USER17> I work for a book store that sells roughly 40,000 items on Amazon (some but not all of those via FBA), and since I've been in charge of the Amazon sales, I've had our account "permanently" shut down several times. Each time, I've written an appeal that gets our account reinstated in just a week, and I can tell you that, based on this article, the seller did not write the correct kind of appeal.

When you're kicked off, Amazon very clearly says (I'm paraphrasing) "If you appeal, explain why this happened and how you're going to prevent it from happening again". At this point, your only option (if you want to continue selling on Amazon) is to suck up your pride and do exactly that. You can not argue against their decision. You have to tell them you messed up and tell them you're going to fix it, whether or not you messed up and whether or not you're actually going to change anything. It's essentially that simple. Just try to make your "changes" sound as bullet proof as possible, and if possible, mention that you've already implemented them.

I am by no means excusing Amazon, mind you. I completely sympathize with the OP, and dealing with Amazon in any way is the worst kind of bureaucratic BS I've ever dealt with. (It once took me over 6 months to get the title of a book corrected, and during that entire time I had the book in my possession and sent Amazon copious pictures of it with the corrected title.) The only reason I knew how to write successful appeals was because another seller once told me. Otherwise, I would have done exactly the same thing, and suffered exactly the same problems.

edit: I should add that it seems as though all correspondence with Amazon is handled by very very low level employees, and there always seems to be a checklist or script they're always working off of. Unless you hit certain points on their presumed checklists in your emails (including your appeals), nothing gets done at all.

<comment><USER18> He talks quite a bit about how he did the appeal and talked to his "Amazon contacts" about the appeal. His resulted in a lawsuit and I'm assuming none of yours has had that outcome. I can't imagine Amazon would've re-stated the above account if Warner Bros, for example, had sued your company as a result.

<comment><USER17> I apologize for the confusion, but I meant the official Amazon appeal that you can do when you're kicked off. It's a one-shot thing, and his description of the appeal said that he tried to ask for more details and state that he has a good Amazon record. It makes no mention of suggesting fixes or anything like that. I admit, he may have said that, but from what Amazon seems to want in the appeal, the tactics he says he details trying are completely wrong.

What went on after his official appeal (including talking to his "amazon contacts") is the bureaucratic BS I briefly mentioned. Almost nothing gets done that way, especially if one's previous official attempts (ie. the official appeal) were already concluded (and failed).

</comment></comment></comment><comment><USER19> Email jeff at amazon.com he sometimes personally responds or forwards problems to his minions to fix.

<comment><USER20> Oh God, I still have nightmares about emails with the subject line "Fwd: Fwd: Fwd: Fwd: Fwd: Fwd: Problem with my account". It'd be from my manager, with a note that said "Look into this, this is your top priority. I'd scroll down, he was forwarding a note from his manager, which would read "<boss>, look into this immediately please, and keep me in the loop." In horror, I'd keep scrolling. Oh, there's my director. And my VP. And my SVP. And at nearly the bottom, there'd be an initial email from jeff to my SVP, and its entire contents would be "?".

There is nothing that terrifies Amazon management like a "?" email.

<comment><USER21> > There is nothing that terrifies Amazon management like a "?" email.

Not true, the ":(" email is much much worse.

<comment><USER22> No, you forgot about the horrifying "):" email.

</comment></comment></comment></comment><comment><USER23> Why do I have the feeling this is just a blanket lawsuit and product may not even be counterfeit.

Big companies do it all the time and smaller companies that don't have the financial means to defend themselves loose because justice is blind to everything except money.

<comment><USER24> Amazon also would've had a pretty good idea about the financials of his business, since most/all of the sales were happening through Amazon.

</comment></comment><comment><USER25> I haven't been involved in Amazon in a few years, but when I was, buyer feedbacks could really turn sales off and on like a switch.

A seller needs to have 'good' feedback to show up in the More Buying Choices box on a listing - getting a couple of silly negatives ("I ordered the wrong item and they sent me what I ordered! 1/5") on a Friday night can push you down so you only show up on the Used and New page for the weekend resulting a lot fewer sales.

I also recall that buyers leaving 4/5 instead of 5/5 carried more weight than it should've done, because then you didn't have 'perfect' feedback

<comment><USER8> " _I also recall that buyers leaving 4 /5 instead of 5/5 carried more weight than it should've done, because then you didn't have 'perfect' feedback_"

URK! Could you be a bit more specific? I rarely give out 5 stars, since in my mind "Excellent" requires something above and beyond "normal", "Good" service, which is the expected level.

Does this just matter for the rare bird who has a 100% rating or thereabouts?

<comment><USER25> I'm guessing it happened because the stars are really a percentage, so a 4/5 would count as 80%; although whenever we asked amazon 'why has xyz strange thing happened?' they seemed to just respond with a stock 'it's how the algorithm works'.

It probably matters less for sellers who shift a lot of stuff, but there was always the problem of annoyed customers being more likely to leave feedback. It can get pretty serious because your whole account can get suspended out of the blue if your feedback goes too far south.

Perhaps a better system would be to assume that customer is fully satisfied if they haven't left a negative rating after x amount of days.

</comment></comment></comment><comment><USER26> This story is frightening, but at the same time inspiring too, considering the volume they managed to achieve. Seems like everything would have turned out OK if the product had not been media. Slightly confused about how the second business ran though, as it says they bought private collections of DVDs, but they must have been new, otherwise couldn't put them co-mingled on FBA.

<comment><USER3> I got the feeling they bought lots of DVDs for a variety of sources; estate sales, going-out-of-business sales, disposals from other merchants or whatever. There's no reason to think some new/shrinkwrapped items wouldn't be in there.

For example: My mother has an enormous collection of DVDs, some of which she rented from the local DVD rental shop, loved, and then purchased a (new) copy from Amazon "just so she owns a copy" but has never rewatched it or even unwrapped it. As I don't watch DVDs, in the (hopefully long away) event of her passing, I'd be highly inclined to see if I could find someone to just take the entire collection off my hands for a reasonable sum. In theory, some of those DVDs, including "new" shrinkwrapped DVDs could end up as FBA merchandise.

In fact, it's entirely possible that she has a shrinkwrapped, counterfeit DVD right now without even knowing it (due to commingling, who knows where the DVDs she orders, overwhelmingly from Amazon, actually came from?); a timebomb waiting to take out some FBA merchant if it ever finds its way back into Amazon's supply chain (which isn't unlikely).

The more I think about it, the more FBA and commingling seem like utter disasters. It's mixing untrusted merchandise into a "trusted" pool; I can't know if the DVD I buy from Amazon came from Amazon or not.

</comment></comment><comment><USER27> So if you do use FBA.. Tag everything?... even the new shrink wrapped stuff... Then you can verify if it was yours or not.

</comment><comment><USER28> Interesting story that might affect my company. We have a large amount of sports apparel with Amazon FBA. Obviously there is a high amount of knockoff sports apparel in the industry. I wonder if I should investigate how this might affect us.

Anybody else have experience with Amazon FBA for textiles/apparel?

</comment><comment><USER9> Is there another side to this story? Not about the individual, but Amazon overall?

For instance - could it be that Amazon was historically guilty of passing on a large amount of counterfeit goods via 3rd party resellers, and they are now just being overly strict as a counter-reaction?

Or is this an issue of Amazon being forced to respond this way, similar to Youtube taking down 3rd party videos?

I'm curious, as this is a very damning report against them, and behavior like this could kill them as a preferred supplier for used goods resellers.

</comment><comment><USER29> having some of the largest rights-holders in the world identify your organization's destruction as a significant revenue opportunity puts you in a rather haphazard spot. if you don't have a war chest, you're not going to have a very fun war. we've all seen how these corporations have acted. they have been stockpiling legal weaponry and applying it belligerently for a decade. something about a lot of hammers and you look like a nail.

<comment><USER8> Well, this works both ways. If you don't have a war chest, the rights holders aren't going to get much money from you (blood from a stone and all that).

The theory that this results in a side effect of Amazon destroying massive quantities of used merchandise, since that's the cheapest option for the target, is just about as plausible. A number of rights holders have been whining about used media, now that it's a LOT more durable than in the bad old, pre-polycarbonate, pre-digital era.

</comment></comment><comment><USER30> I hope something good eventually comes of this. I wonder if they do in fact destroy your media or do something else with it when you pay for destruction?

</comment><comment><USER12> I don't understand one point - he was protected by an LLC, how then did he have a personal liability for the goods held by Amazon when the LLC failed.

Similarly with the tort case for alleged copyright infringement - <USER2> Warner/Sony/etc. CEO's fined for their companies infringement of the music for the "you wouldn't steal a car" ads a while back?

<comment><USER31> Your opponent can argue that your LLC's business and your personal business were not separate, and therefore the LLC's liability protections don't apply ("piercing the corporate veil"). So, the LLC liability protections only matter to the extent you can defend them in court.

</comment></comment><comment><USER32> How did Warner Brothers find out about this? Did Amazon contact them when a customer sent in a allegedly counterfeit DVD? Was Warner Brothers the customer who purchased the DVD?</comment>


Ask HN: How to deliberately practice life-long learning? - <AUTHOR>

<comment><USER2> For me, the biggest thing is just making time for it. If I don't schedule continuing-education into my week then it's not going to happen.

If you're fortunate enough to have an employer that encourages this practice, (ask them if you're not sure) they may help you out with the costs of attending conferences, buying textbooks, taking classes, getting certified, etc. In theory, their return on investment should be quite high for this.

But even if you're a freelancer (or work a job that doesn't promote life-long learning) there's still quite a bit you can do. My big 4 are: Meetups, conferences, whitepapers/articles/books, and the occasional class (either online or in person). Some of these cost money, others don't. But once you make them part of your normal routine, it'll feel like second nature. You'll look back with utter contempt on those days when your skillset stayed stagnant for months.

<comment><USER3> "Meetups, conferences, whitepapers/articles/books, and the occasional class (either online or in person)." -> those are all good, but none of those are what's technically known as deliberate practice.

<comment><USER2> I see your point, but I think it depends on _what_ you are deliberately practicing. I would argue that if engaging in those activities regularly _is_ what you want to be good at (if those activities are an end not just a means to an end), then simply making time for them and _just doing it_ , would be deliberate practice.

</comment></comment></comment><comment><USER4> Dedication and focused practice. I'm going to leave this here, several of the same principles can transfer. Purpose, living with intention and not living life running on auto-pilot.

[http://www.youtube.com/watch?v=ggYF7JeH1Tw](http://www.youtube.com/watch?v=ggYF7JeH1Tw)

</comment><comment><USER5> Try Coursera.org - the classes are of decent quality, even the free ones are really, really good. Make yourself do it. I only completed half of the classes I signed up, but that's better than nothing....</comment>


How we hacked the hotel industry to save $200+ per night - <AUTHOR> http://blog.localfu.com/post/105466825757/how-we-hacked-the-hotel-industry

<comment><USER2> This is not a hack. It's a thing non-tech people have been doing for quite some time. Indeed, you can simply _call_ the hotel and ask them if they have any discounts (including event-related discounts) during the period of your stay, and they'll gladly tell you if it means they get your business.

It's always jarring to read something that techies call a "hack" which is something that non-techies consider common sense. What is localfu going to "discover" next? That you can ask the concierge desk to book reservations at popular restaurants?

</comment><comment><USER3> Correct me if I'm wrong, but aren't those Google Forms? I don't think I'd trust my credit card information to one if it's reminding me not to enter my password in one!

This "hack" appears to just be abusing a conference organiser's pre-arranged rate for hotels near the area. I'm assuming the Google Form submits your information to the conference organiser, not to the hotel. They should (but apparently aren't) cross-reference the attendance list before passing the approved reservation to the hotel.

<comment><USER4> Yep, that looks like Google Forms. GAH!</comment></comment>


Ask HN: Software gigs involving small, scripting-type tasks? - <AUTHOR> I love writing small, elegant programs--under 500 lines, say--that do one thing well. Scrape a website and dump the data to a CSV; take a directory full of video files and rename them based on the metadata; poll a JSON API for a retail site to check the price of an item and email someone if it decreases.<p>I like coding, but it isn't my passion, so it's hard to get my teeth into developing a big web app, or making small enhancements and bugfixes on someone else's. But I always love those little projects when I get the chance to do one. There are lots of non-programming fields that involve this kind of thing (data science, devops, IT, etc.). But are there fields or sub-industries of the software world wherein it's possible to do this kind of old-school scripting for a living? <comment><USER2> You could find applications in almost any field, given that there is work done on a computer or have access to one regularly. As for it being a big portion of daily work, I'd guess that most would be technically oriented, similar to those you listed (data science, dev ops, IT)

</comment><comment><USER3> Not sure why you want to so quickly discount data science, dev ops, etc. But you could also consider many research areas in computer science that use light scripting to prove out new algorithms and techniques. Another area is computer security/pen testing.

</comment><comment><USER4> Most of my daily work involves the type of scripting you mention, granted we do have some bigger projects as well but we spend most of our time making small scripts. I work for a small "analysis house" (finance sector)

<comment><USER5> do you do a lot of web scraping? what sort of scripts are these designed to do?

</comment></comment><comment><USER6> [https://www.fiverr.com/categories/programming- tech](https://www.fiverr.com/categories/programming-tech) sounds like it might be a possibility.</comment>


File System Latency: part 4 - <AUTHOR> http://dtrace.org/blogs/brendan/2011/05/24/file-system-latency-part-4/

<comment><AUTHOR> Part 1: <http://news.ycombinator.com/item?id=2539268>

Part 2: <http://news.ycombinator.com/item?id=2560682>

Part 3: <http://news.ycombinator.com/item?id=2560684></comment>


Project PERRY - An enhanced stored-value card - <AUTHOR> http://urbanscale.org/2011/02/15/project-perry/ <comment> _-af-_ Couple of major issues with this (as an EE who has designed products involving both RFID and E-Ink):

1\. E-Ink requires a controller -- you can't get those at wafer scale

2\. E-ink has a hard plastic substrate, you can't have it on a flexible substrate therefore it will break very easily under pressure.

3\. E-ink requires 1mW to switch + power to the controller chip... RFID can't provide that much power.

4\. You can use an NFC equipped phone to do the same thing, much easier (but that has its own issues)

5\. E-ink is expensive. It will cost at least $20 in volume for a small display the size of your CC -- even a 120x72 strip will be about $10.

5\. You can't easily interface the RFID chips with an e-ink display. Those RFID chips (MiFare DESFIRE, etc.) are all effectively enhanced memories. There is no processing power that you can control to interface w/ an e-ink display.

<comment><USER2> 1\. Yep, e-paper _does_ require a controller. We're developing one at the appropriate thickness. 2\. It's reasonably conformable, but it doesn't need to be any more so than the plastic substrate of e.g. an Oyster card. 3\. That number strikes me as a little off, but in any event we're comfortable with the differential between the power we've got coming off the coil and what the display + controller need. 4\. Agreed - but that's a different use case entirely. 5\. You'd be amazed what volume can do. 6\. Again, we're developing a custom controller.

What kind of an engineer looks at a situation like the one you describe and sees it as anything but a tasty challenge, anyway? : . )

Thanks for taking the time to think this through, though.

<comment> _-af-_ I'm not trying to be a naysayer. I'm trying to give you info because we've done a number of RFID and e-ink designs and I'm very familiar with both technologies. Neither of our designs involve a card.

BTW, some credit cards Visa is doing a field trial with have a display but it is LCD COG + Polymer battery for exactly the e-ink restrictions I've laid out below.

1\. To fit inside a card you'll need wafer level thickness. Any package will be at least 2mm thick.

2\. The screen will break if you put any force on it. I have dozens of samples of this to show you. Get a kindle, take the screen out, use a a dremel to cut it down into a card size, now put it in your wallet and sit on it.

3\. That number is actually a low ball. We're seeing close to 2mW for switching. 1mW is from the datasheet. We turn on the controller, update the scree and go back to sleep -- close to 2.8mW gone. Also switching requires +25, -25 and a VCOM, I'd love to see how you're going to put a dual supply charge pump in such a small package and power it from a coil.

4\. --

5\. Well the pricing we have is for 100K volume but maybe you guys can get it down to couple of cents.

6\. E-Ink requires custom waveforms which you can only get under license from PVI. They don't even give it to Epson, Motorola, or TI. You put the controller then e-ink gives you a file w/ the waveforms for that batch of screens.

Also, the e-ink display + substrate is much thicker than you think. Even their segmented display stuff which has much simpler requirements can't be driven off of what you want to drive it off of.

No one minds a challenge, provided the circumstances are realistic. Anyhow, good luck with the project.</comment></comment></comment>


It's Real: YouTube Debuts Live Streaming Platform With Two-Day Test - <AUTHOR> http://techcrunch.com/2010/09/12/youtube-live-streaming/ <comment><USER2> The problem they'll have the most trouble with in live streaming is simultaneous viewers. YouTube can fudge it with On Demand videos now by using progressive download and a lighttpd plugin, but streaming live to that large of an audience will choke them.

Wow, 105 people watching now at probably 300kbps and it's incredibly blocky and jittery. Now it's having technical issues, off the air.

</comment><comment><USER3> The legal ramifications will be interesting once it is released to the masses and not just a few partners.

Org's such as the UFC or NFL will hate this far more than they hate justin.tv/ustream.tv because it will make live streaming more mainstream for the masses.

<comment><USER4> Except of course that YouTube won't allow their site to turn into [http://www.justin.tv/directory?order=hot&lang=en](http://www.justin.tv/directory?order=hot&lang=en)

The current sites could prevent it, but they have VCs to impress and no assets to protect.

<comment><USER5> i see a lot of "canned" content getting streamed over there. what's the point of that?

i can just as easily download that stuff, in higher quality, watch it whenever and wherever i want, and i can even pause and seek.

<comment><USER6> It's communal viewing. Watch a program and chat about what is happening.

</comment></comment></comment></comment><comment><USER7> > Live streams will only be available on YouTube proper (you can't embed it in > widgets yet), and YouTube won't be archiving the footage, either.

Someone make an archiver of youtube live steams.

</comment><comment><USER8> I think this is a natural step that youtube should be taking and I would say good bye to most of the online streaming sites.

I don't think youtube will open up the live stream feature to the masses, where anyone can live stream, like ustream or JTV because their infrastructure can't handle it and honestly most of the stuff regular people produce are horrible and a waste of resources.

They will in the short run offer live streaming to content partners and special people (celebrities) that produce content that people actually want to watch and has production value.

Ustream is known for having celebrities on their platforms, but once Youtube live stream platform comes out they will be gone. No celebrity will be able to resist the amount of eyeballs that youtube can bring and the switching cost is very low. It would be safe to say that people who have Ustream accounts also have Youtube accounts and Ustream hasn't signed any exclusive deals with celebrities.

Youtube will be the distribution platform of the premium live streaming content and the rest will be for the low brow stuff and illegal content.

<comment><USER8> I don't get why this is being down voted, but i don't care. It will happen just wait.</comment></comment>


Learn HTML & CSS in one easy to use guide. - <AUTHOR> http://learn.shayhowe.com/html-css

<comment><USER2> Nice and beautiful site, but really for beginners. I wonder how difficult it would be to make this guide more interactive by allowing users to modify/play around with code. May be I am a little spoiled by CodeSchool.</comment>


A Dog Who Took the Witness Stand - <AUTHOR> https://narratively.com/the-dog-who-took-the-witness-stand/ <comment><USER2> Interestingly enough, we now have better ways of proving ownership that weren't possible back then: Get your dog microchipped and tattooed if you haven't already. They're far better ways of proving ownership than feelings.

Also, if I were there, I'd have snuck a treat into the court room. It's far more foolproof if I'd been able to condition the dog to expect treats by coming to me in a few 20 minute vists.

Also, does anyone have suggestions for international pet databases? My country doesn't have a national database.

<comment><USER3> I don't see how a microchip (that a thief would have removed/replaced) helps prove ownership, and tattooing has been available for a long time.

A friend of a friend lost their cat once: it suddenly disappeared. Several years later they found it dead in a public place, but apparently well fed. It appears that someone else had adopted it, but they never even found out who, let alone how or why. I don't know how they identified the animal. Some people can identify animals from their appearance as easily as most people identify humans; some people cannot even identify humans very reliably.

Put lots of pictures of you and your pet on social media, perhaps, and summon an animal super-recogniser as an expert witness?

<comment><USER4> > I don't see how a microchip (that a thief would have removed/replaced) helps > prove ownership...

A microchip (aka RFID tag) contains a unique serial number. An entry corresponding to this serial number is added to national databases that are easily searched by vets, shelters, and animal control.

It's nontrivial to remove a microchip (requires a vet to perform an X-ray and surgery). You're talking about removing a grain of rice that's buried somewhere within the dog's shoulders.

On the other hand, it's easy to transfer registration data (modifying the database entry) if you have the approval of the original owner.

At any vet worth their board certification, they would scan the chip and contact the named owner before performing this surgery.

Microchips are well known to be effective ownership proving tools, and their prevalence even without a matching tattoo is usually an effective theft deterrent. If you think you can demonstrate otherwise (notwithstanding anecdotes concerning lost/stolen unchipped pets), I'd love to hear it!

<comment><USER3> An RFID tag is an excellent way of labelling a furry animal, no doubt about it, and a national database administered by regulated and trustworthy professionals is an excellent way of making it possible to prove ownership, no doubt about that, either. However, as I see it, it's the database plus regulated professionals that's the essential ingredient here, not the particular way that labelling is implemented.

Though I introduced the thought myself, perhaps removing microchips is not all that relevant. If there was no reliable database and vets were all corrupt then you could have an animal with six RFID tags attached to it, and six people each claiming to be the owner of each of those tags, and how would that help anyone prove ownership?

Nevertheless, if one did want to remove an RFID tag, I'd hope there would be a better way of locating the thing than X-rays. It does, after all, contain a transponder and I seem to recall that when RFID tags were first proposed one of their advantages was supposed to be that you could use them to precisely locate things. If someone had attached an RFID tag to my property and I was worried that they wanted to do something nefarious, how would you recommend that I locate and remove/destroy that tag?

<comment><USER4> > _Nevertheless, if one did want to remove an RFID tag, I 'd hope there would > be a better way of locating the thing than X-rays.

Ultrasound might be useful, too, especially for determining the depth of the tag, but most vets would reach for the radiograph. The ultrasound head will be right over top of the tag, which would make it difficult to mark the location and see what you're doing during surgery.

> _It does, after all, contain a transponder and I seem to recall that when > RFID tags were first proposed one of their advantages was supposed to be > that you could use them to precisely locate things.*

RFID provides location tracking by being an easily scanned unique identifier, not by transmitting location data. You'd need to have a reader within a few inches of the tag to locate it, at which point your nefarious actor is fully in posession of the pet (just as they were when they implanted the tag). It could locate your animal as having last gone through a pet door in one direction or the other with a reader on each side of the gate, it can't put a dot on a map for a wardriving Cruella DeVille. Think permanent, easy-to-read barcode sticker, not GPS transmitter.

> _If someone had attached an RFID tag to my property and I was worried that > they wanted to do something nefarious, how would you recommend that I locate > and remove /destroy that tag?_

Bring your animal to your vet. They'll recognize you and your pet's history (you are up to date on your vaccinations and animal registration, correct?) and will be able to scan and see your microchip, as well as the new data. Then you can file a police report.

<comment><USER3> Thanks for that information!

What if I'm worried that someone has attached an RFID tag to my car, my bicycle, or an item of clothing?

(The tags used in animals might only be readable at a range of a few inches, but: "Gen2 tags can have a read range of over 16 meters or 52 feet when using the full 4 Watt EIRP legally allowed on the readers by FCC and other global regulators"; "a model using a beam-steerable phased-array antenna can interrogate passive tags at a distance of 600 feet or more")

</comment></comment></comment></comment></comment></comment><comment><USER5> I've seen a similar story(video) with Judge Judy pop up on reddit a few times.

[https://www.youtube.com/watch?v=IqZKFzFZEqc](https://www.youtube.com/watch?v=IqZKFzFZEqc)

</comment><comment><USER6> Ain't no rules says a dog can't play witness.

</comment><comment><USER7> I thought this might be about the opposite: in some cultures animals could be tried and convicted of crimes: [https://en.wikipedia.org/wiki/Animal_trial](https://en.wikipedia.org/wiki/Animal_trial)

<comment><USER8> Then the dog would be the defendant, not a witness.

<comment><USER7> Defendants take the stand all the time.

<comment><USER9> Would they be considered witnesses though? I'm honestly not sure whether the legal classification overlaps or not. I suppose that one doesn't necessarily have to be a witness to "take the witness stand" even if not, though.

</comment></comment></comment></comment><comment><USER10> Dogs can look guilty. And lie. So we probably can fMRI a dog to get a sense of if it can pass the McNaughton test ("would you have done it, if a policeman was watching")

</comment><comment><USER11> Sounds like a cheesy Brady Bunch plot, but is a great story regardless. As a former dog owner, it's not hard to relate to the owner here.</comment>


Show HN: From RSS to Social networks - <AUTHOR> http://shre.io/

<comment><USER2> I like the "one-click decision" idea. I find it time-saver as I don't have time to copy/paste/edit posts for each articles I'd like to share. Bookmarked it.

<comment><USER3> +1

<comment><AUTHOR> Thanks you. We really wanted the core feature to be this ready-to-be-published concept. Glad you like it.</comment></comment></comment>


When AES(☢) = ☠ – a crypto-binary trick  - <AUTHOR> https://speakerdeck.com/ange/when-aes-equals-episode-v <comment><USER2> That was a great read. I saw the title and figured it would quickly go over my head but it's all pretty understandable.

Does anyone know where I can download the src to have a look through?

Edit: found it [https://code.google.com/p/corkami/source/browse/#svn%2Ftrunk...](https://code.google.com/p/corkami/source/browse/#svn%2Ftrunk%2Fsrc%2Fangecryption)

<comment><USER3> The play of unicodes reminds me of some awesome slides made by Larry Wall and other Perl hackers. A good slide show is about to convey messages in a visually efficient and succinct way, not in an overloaded and bloated one.

</comment></comment><comment><USER4> There's also a word play in the title. "AES" transliterates to "АЭС" (acronym for " _А_ томная _Э_ лектро _с_ танция") in Russian (and some other Slavic languages), which means "nuclear power plant". Thus, the "☢" sign.

<comment><AUTHOR> that's totally unintended: I just looked for some interesting unicode characters and chose these 2 for the video game references ;)

</comment></comment><comment><USER5> Actual link: [https://speakerdeck.com/ange/when-aes-equals- episode-v](https://speakerdeck.com/ange/when-aes-equals-episode-v)

</comment><comment><USER6> Recording of the talk: [http://podcast.raumzeitlabor.de/#wbHkVZfCNuE](http://podcast.raumzeitlabor.de/#wbHkVZfCNuE)

</comment><comment><USER7> That's amazing. Didn't think it's even possible, however it turns out to be surprisingly simple. Also, laughed out loud because of that guy's twitter nickname on the 3rd slide.

</comment><comment><USER8> Cool trick, I have encountered something like this in a steganography wargame before, the only difference is they used Base64 encoding on the original picture instead of AES :)

<comment><USER9> Base64 is not true encryption, it doesn't require a key it is merely obfuscation.

<comment><USER10> In the general usage case base64 is not even obfuscation, really (though it does make it non-human-readable... for most humans), just an encoding for dealing with situations where you need to store or transmit 8-bit or multibyte character data into a system that may otherwise be incapable of dealing with it reliably.

</comment></comment></comment><comment><USER11> Does any one know the name of the hex editor used in these slides, the one showing the PNG chunks and JPEG information?

<comment><AUTHOR> that's [https://bitbucket.org/haypo/hachoir/wiki/Home](https://bitbucket.org/haypo/hachoir/wiki/Home)

</comment></comment><comment><USER12> A good example of why a MAC after encryption is also needed. And blocking length extension attacks.

<comment><USER13> Where do length extension attacks come into play with angecrypt?

</comment></comment><comment><USER14> I love the "HexII" hex-dump format he links to, it's so much less cluttered than the traditional one. I'm definitely going to have to try that out the next time I'm picking apart some binary file.

<comment><AUTHOR> thanks - HexII is in very early development for now.

</comment></comment><comment><USER15> this is awesome. now I hide secret information in a seemingly innocent image. no one would want to use AES to decrypt it if the image looks fine.

</comment><comment><USER16> Where do you store the IV? Do you just append it at the end of the file?

<comment><USER17> You have to supply that, see the python script, at the end it prints what to run to decrypt, the determined IV is passed as a param:

[http://corkami.googlecode.com/svn/trunk/src/angecryption/ang...](http://corkami.googlecode.com/svn/trunk/src/angecryption/angecrypt.py)

</comment></comment><comment><USER18> What's the benefit of AES using such small blocks?

<comment><USER19> The block size of a symmetric cipher will generally be in the neighborhood of the key length. You can imagine the problem if you had a 128-bit key but an 8-bit block size -- for each of the 256 possible inputs you would have only 256 possible outputs. There would be multiple keys that produced the same mapping and you could produce every possible mapping with only 65536 keys. So the block size needs to be near the key size. Making it larger than that wouldn't make it any more secure but would make it slower.

<comment><USER20> This doesn't sound right to me. There are 256! permutations of the set {0, 1, 2, 3, ..., 255}. 256! is much bigger than 2^128, so I see no reason that each key cannot produce a unique mapping.

> So the block size needs to be near the key size

Note that AES-256 has a 256 bit key, but the block size is 128 bits, which is not near the key size.

I believe that the main constraint on block size is that a small block limits the length of messages you can safely encrypt with a given key. If the bad guys see a lot of cipher text encrypted with the same key, they have a better chance of a successful attack. What "a lot of cipher text" means depends on the block size. The bigger the block size, the more cipher text is needed to constitute "a lot of cipher text".

<comment><USER13> A smaller block also gives you less room to maneuver when designing modes of operation; for instance, it can be tricky to implement CTR with a 64 bit block --- the convention is to split the block into "counter" and "nonce", and you need enough space for the counter that it can't conceivably wrap.

<comment><USER21> This may be a stupid question, but why is there a convention to split?

I saw that in set three of the crypto challenge, as well, and wondered.

Is it so that you can always start counting at zero when re-starting the application, as long as you're randomly picking a new nonce?

And if you just picked a random counter value at each restart you might get very unlucky and at some point reuse a counter value, so by this convention you're separating counter values belonging to different restarts?

<comment><USER22> Conventions and usage vary by setting, but yes, it's to prevent nonce reuse.

A typical scenario might use a single encryption key for many different messages. A simple strategy is to allocate 64 bits to a message counter and 64 bits to a block counter. For each encryption you can increment the message counter by one. The block counter starts at zero and increments for each block of the message.

Note that this imposes hard limits on both the number of encryptions you can perform with this key (2^64) and the number of blocks a message can contain (also 2^64). As long as we respect these limits, we're guaranteed never to reuse a nonce.

If we use a random 64-bit nonce in place of the message counter, the picture changes a little. Due to the birthday paradox, we can now expect a collision in around 2^32 encryptions, which is a little close for comfort.

Fortunately, we can tune the bit allocations based on the needs of the application. So a reasonable strategy might be to bump the random message nonce to (say) 96 bits and leave 32 for the block counter. This still allows individual messages of around 64GB.

Of course, parameters like these should be considered implementation details for 99% of application developers. The best thing to do is to use a high-level library like NaCl that makes reasonable choices and then abstracts them away.

</comment></comment></comment></comment></comment></comment><comment><USER23> impressive! a very cool hack!

</comment><comment><USER24> Impressive.

That's also the reason why one should limit the max-length of a password field (something reasonable), if one is using the _salted-password in db_ approach. Otherwise someone could enter a very long password to do the trick (MD5/SHA1), see [http://en.wikipedia.org/wiki/MD5#Security](http://en.wikipedia.org/wiki/MD5#Security) .

<comment><USER25> I guess I'm not following your logic. If there is a salted, hashed password in a db, allowing arbitrary length passwords shouldn't matter? HMACs and KDFs work very differently from symetric-crypto primatives.

<comment><USER26> The point is that it makes it easier for an attacker to find a hash collision. It's much easier to construct data which hashes to a given value if it can be of arbitrary length. I don't immediately see the connection with this article however.

<comment><USER27> I don't follow. Breaking a password hash your not trying to find a hash collision you need to break the preimage attack. Sure are some stage in the future there might be a preimage attack that requires a large amount of data to use. But really if your worried about theoretical preimage attack you aren't using md5.</comment></comment></comment></comment>


Free idea: Craigslist alerts - <AUTHOR>

I'm sure this has been written... but a (very) quick search didn't turn up anything.<p>I've been on the lookout for some stuff and wished I could get alerted when a posting was made for these items... of course I could use RSS, but that's not KISS for normal people. <comment><USER2> We've been thinking about this idea for a long time. For example, I'm in the market for a platform bed (because right now my mattress is on the floor), but I'm not going to check every day. This application would probably be similar to EBay's "Want It Now" application, but would be more geographically based. It's a great idea, and I think we put it on our YC application (as one of the ideas we thought about).

<comment><AUTHOR> Ya... exactly what I was thinking... for what I need right now, I wouldn't travel to Austin or Houston (I'm in the Dallas area), but for other things (cars), I would want to know when a 68 corvette comes into Austin, Houston or Dallas.

</comment></comment><comment><USER3> This is a pretty good implementation of your idea. <http://www.craig2mail.com/> I've used it to find items that are rarely posted.

Edit: my bad, they got shut down by Craigslist <http://www.craig2mail.com/craigs.html>

Aside: What is up with CL? Their policies seem very short sighted.

</comment><comment><USER4> Maybe a person could suggest they keep there system "pure" the way they want it and let you setup some "enhancements" that people would pay for or would be ad supported (by google adwords (ads on the ads, I know that sounds kind of strange)) then share the revenue.

I believe they just don't want to lose control but at the same time they want to keep there system "pure".

</comment><comment><USER5> The problem with that kind of thing is polling the servers of Craigslist, I suppose. Would Craigslist allow it? Suppose you have thousands of items to poll for every day.

<comment><AUTHOR> Ya... I think they busted that one guy that was doing the CL pictures... Maybe you could use the google cache of CL? Seems the site is indexed quite quickly...

Not sure how to monetize it either...</comment></comment>


Ask HN: Building a job board. Worth the time? - <AUTHOR>

As idea's are a dime a dozen and what really matters is the execution, I wanted to share mine and get opinion. There are a lot of job boards out there but most charge big dime to post a job. Following are 30 day job posting costs: Dice - $479 SO - $350 GitHub - $300 37Signals - $400
So when a small company wants to post a job, and for less money...they either start by reaching out to their network first and then end up posting jobs on craigslist ($25 for 30 days).
I want to get HN communities opinion on whether it would make sense to invest time in making a job board that concentrates only on small businesses, provides them features comparable to dice but still at the price of craigslist.  Also, the motto of "do no evil" aka "don't sell candidates to recruiters" would be utilized.
What's your opinion? <comment><USER2> Two problems: the classic chicken-and-egg problem, coupled with trying to differentiate yourself on price. Even for small businesses, $500 is a trivial cost when compared to the benefit of making a good hire, and anyone not willing to pay that is probably not going to be a very attractive employer.

<comment><AUTHOR> I understand the chicken-and-egg problem but I have some contacts that would be willing to post jobs up there to begin with. You are correct that if an employer is not willing to spend $500 to post a job then they are not attractive. However, still everyday I see an increasing number of jobs being posted (for IT field) on cragislist. There must be something attracting those employers/employees?

<comment><USER2> Job listings on Craigslist are part of a general-purpose, locally targeted system. That's different than a special-purpose, globally targeted system (like Dice).

The question I'd be asking myself is: what "special sauce" do I have that is going to help me attract more programmers looking for work than, say, the StackOverflow job board.

If you can crack that nut, you don't really need to worry about pricing.

</comment></comment></comment><comment><USER3> The success of a job board is based entirely on it's ability to attract candidates. So you've designed a decent site, priced yourself competitively now how are you going to attract a huge amount of candidates?

The other drawback of cheaper job boards is that more senior candidates tend to avoid them due to the assumptive association that they are essentially cattle marts for employers to find low quality talent.

 _don't sell candidates to recruiters_

Most job boards, particularly smaller job boards, rely on recruitment agencies as their primary source of income. You would be missing out on a huge market here.

<comment><AUTHOR> point well taken on sr. candidates not applying. not selling candidates to recruiters was a way to attract candidates to post their CV's. This way they would be safe from spam about jobs 600 miles away from them.

</comment></comment><comment><USER4> It's worth the time if you are sure you can give it something special. But you can't know for sure till you build it. So ship and see.

Even if it fails, it will add to your experience.

<comment><AUTHOR> I like that. Intend on doing this. thanks</comment></comment>


Think GPS is cool? IPS will blow your mind - <AUTHOR> http://www.extremetech.com/extreme/126843-think-gps-is-cool-ips-will-blow-your-mind

<comment><USER2> Only in the hyperventilating startup bubble world can tracking people around a shopping mall with wifi be cooler than tracking your position anywhere on earth with satellites that were put there by huge rockets.

<comment><USER3> I also get the feeling the author has never heard of differential GPS: <http://en.wikipedia.org/wiki/Differential_GPS>

Which actually leads to something I've wondered about before: Why couldn't you (with appropriate FCC clearance) set up GPS transceivers around a shopping mall calibrated to rebroadcast the signal at the appropriate time offset? It would work with everyone's existing hardware and be completely transparent.

<comment><USER4> The location of each GPS transmitter must be known exactly for the triangulation to work, and existing GPS receivers know nothing about the location of shopping-mall transmitters. They only know about satellites.

So that's why you couldn't do this with existing receivers, but I don't know of any reason why you couldn't make new receivers that cooperate with a new type of ground-based transmitter built into cell towers or similar. Multipath might be a problem but it seems surmountable. Do any radio experts here know why nobody has done anything like this for cell phones? (The Wi-Fi/Bluetooth positioning talked about in the article is based solely on signal strength which is far inferior to the GPS method).

<comment><USER5> I don't think the GPS approach can work with ground stations - what GPS has going for it is line-of-sight to the satellites. Multipath would be a hell of an issue in an urban environment.

That said, here's a paper describing various positioning methods - [http://emits.esa.int/emits-doc/1-5200-RD25-Signal- processing...](http://emits.esa.int/emits-doc/1-5200-RD25-Signal-processing- techniques-network-aided-positioning.pdf)

EOTD looks rather interesting - <https://en.wikipedia.org/wiki/E-OTD>

<comment><USER6> WAAS is a combination of GPS and ground stations. It is accurate to about 6 feet. Widely used in aviation.

<comment><USER4> The ground stations don't transmit to your device. WAAS still needs the satellite signals to work and so won't help indoor reception problems.

<comment><USER7> Correct. The ground reference stations multiplex their data to the WAAS satellite, which then pushes the data on WAAS-specific channels (hence, why your GPS receiver specifically needs to be built to support WAAS)

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> As a robotics hobbyist I've done a lot of research into the 'localization' problem (which for robot folks is localizing where you are relative to the space around you). And there are lots of ways to do it, some more interesting than others. The simplest being landmark navigation. Sebastian Thrun of the self driving car fame did some interesting graduate work in Germany where a robot uses the doorways to figure out what room it was in and then could navigate to any other room.

However what some people fail to realize is that _stores_ really don't want this technology to exist. You see they, like Casinos and other places with 'impulse actions' want you to wander around looking for your target and being exposed to various other temptations. How much less per shopper would you imagine there would be if a shopper had a pre-planned route available for the three things they needed at a Target or a Walmart?

A really simple solution would be to program the ceiling of a place into an app. When you're looking at your phone, its front facing camera can look past you to the ceiling and figure out where you are. Roboticists have known for a while that ceilings are remarkably good navigation references, try as you might they are often quite different to the discerning camera.

<comment><USER9> "Roboticists have known for a while that ceilings are remarkably good navigation references, try as you might they are often quite different to the discerning camera."

Would this work in an absolute sense, providing absolute location, or in a relative sense, like an optical mouse, sampling the ceiling and comparing each sample to the previous?

<comment><USER10> It ought to be good enough for absolute position. The general arrangement of lights, air vent, smoke alarm, sprinkler, motion detector, and wall proximity are more than likely unique.

<comment><USER11> Assuming this technology because popular enough to affect the stores' bottom line, how much do you think an enterprising person could charge them to install a system specifically engineered to confuse this?

I'm thinking, Portal-like attachments in the ceiling that randomly move around - probably at night, to avoid scaring shoppers.

Or maybe you could get away with a couple of computers hooked up to half a dozen projectors, projecting random imagery on the ceiling. You could even do something interesting like a rainforest canopy, clouds, etc.

<comment><USER8> I really love this visual. Black inverted beetle hump devices, masquerading as security cameras, moving from place to place on the roof at night when the lights are out to confuse nav algorithms that defeat 'enchhanced discovery' layout.

</comment></comment></comment></comment></comment><comment><USER12> GPS categorically does not "only really operate in two dimensions." You get altitude above mean sea level as part of the standard GPS reading.

<comment><USER13> The article claims that " GPS and GLONASS can measure altitude, but generally the data is inaccurate and too low-resolution (on the order of 10-25 meters" - which I'm fairly sure is totally un-true.

<comment><USER14> No, it's true. The possible geometry of visible GPS satellites at any given moment (namely that they're all above you and none are below) means that the vertical error is pretty bad in comparison to the horizontal error.

<comment><USER15> GPS altitude readings seemed pretty fine on the numerous mountains I've climbed. I almost always cross-checked with a calibrated barometric altimeter.

<comment><USER13> Interesting. I wonder if the altitude readings were WAAS augmented, or if this is a case where theory doesn't translate in to reality.

<comment><USER15> Ah. You may be right, there. WAAS is enabled on my GPS (a Garmin 60CSx).

Link for bystanders: <http://en.wikipedia.org/wiki/Waas>

_WAAS uses a network of ground-based reference stations, in North America and Hawaii, to measure small variations in the GPS satellites' signals in the western hemisphere. Measurements from the reference stations are routed to master stations, which queue the received Deviation Correction (DC) and send the correction messages to geostationary WAAS satellites in a timely manner (every 5 seconds or better). Those satellites broadcast the correction messages back to Earth, where WAAS-enabled GPS receivers use the corrections while computing their positions to improve [3D] accuracy._

<comment><USER7> And if you need even higher resolution (centimeter resolution), use use NOAA CORS sites (either with post-processing, or real time with their experimental datafeed)

<http://geodesy.noaa.gov/CORS/data.shtml>

<http://beta.ngs.noaa.gov/CORS/NGSRealtimeGNSS/>

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> One more piece of 'revolutionary technology' whose main goal seems to be to sell me things or make me spend my money in other ways.

<comment><USER17> When did commerce become a bad thing?

<comment><USER16> Commerce isn't a bad thing, but exciting new technology to try to separate me from my money faster may be a cool thing for Google, but not for me--the consumer. Maybe I'm weird, but I already know what I want to buy, I don't need technology to try (and usually fail) to generate some new consumption desire.

<comment><USER18> Be honest though - it's not like companies like to develop technology that has no commercial value. Whether or not it's used in that way is a different question, but if it's a big company developing something, you bet it went through twenty levels of market research to make sure there is a buck to make at the other end.

It's still your choice what you do with it once the technology gets to you. I find it far more fruitful to look at what people do with the technology and how _they_ end up screwing themselves with it. And by fruitful I mean frustrating.

</comment></comment></comment></comment><comment><USER19> I cannot wait for this to get some adoption! I love augmented reality, but the current state-of-the-art is not exactly suited for AR applications. If this chip can deliver on its promises, I think we'll see an explosion in AR as it finally starts to look real. The main problem with AR on mobile devices as I see it is a visual one - very intense jitter due to inaccurate GPS and gyroscope data. If IPS can pinpoint AR objects not based on visual markers or inaccurate GPS, then I think we are quickly headed into Dennou Coil territory. (<http://en.wikipedia.org/wiki/Denn%C5%8D_Coil>)

<comment><USER20> I too cannot wait for true AR. I have been dreaming about making AR apps for the iPhone since it was released, but have never found satisfactory solutions to the positioning issues. Can't wait to play a real world RPG, with GPS used for the world map, and IPS for dungeons with AR objects accurately placed.

(Just started Dennou Coil. It rules.)

</comment></comment><comment><USER21> "Roving groups of attenuating meatbags" as a synonym for "people" is my new favourite phrase.

<comment><USER22> Sounds like something out of a Neal Stephenson novel.

</comment></comment><comment><USER23> "Unlike GPS and GLONASS, there isn't a standard way of building an indoor positioning system. "

I don't really get it then. So Broadcom built some sensors into its latest GPS chip, but what standard is it using to get a location from them?

<comment><USER24> There isn't a standard yet; wifi locationing currently requires a site survey to gather location data. It's potentially possible to crowdsource this data, but currently indoor location tracking has nothing like a GPS standard – each site is different.

</comment></comment><comment><USER25> Think wi-fi is cool? Sky-fi will blow your mind! It's just like wi-fi, only it uses satellites instead of terrestrial nodes for communication.

That is to say, my main problem with IPS is that it sounds like GPS, but isn't at all the same. One is a massive global satellite network, and the other is glorified RFID with a hefty glob of marketing on top. Call it something else, at least, please?

</comment><comment><USER26> This can also be done by using LEDs and smartphone cameras, which actually gives a higher resolution. <http://www.technologyreview.com/computing/39685/>

</comment><comment><USER27> I was desperately hoping this would be "Interstellar Positioning System", and not something that tracks people around in shopping malls.

<comment><USER28> There were recent stories about using pulsars in an interstellar positioning system.

<comment><USER7> <http://www.bbc.co.uk/news/science-environment-17557581>

</comment></comment></comment><comment><USER29> Nothing pertaining to shopping malls and consumerism has ever or will ever blow my mind.

Sorry I'm not sorry.

Can we get back to using our powers for good now?

<comment><USER30> Just because the article shows examples of how that tech is applied to consumerism doesn't mean thats all IPS is limited to. I'm sure you can think of many applications that are not solely to make more money.

<comment><USER29> You're right... it could also be used to oppress people.

</comment></comment></comment><comment> excuse-me We had this nearly 25years ago. <http://www.cl.cam.ac.uk/research/dtg/attarchive/ab.html>

It was vaguely useful for finding your supervisor - assuming he a) actually turned up at the lab and b) remembered his badge

</comment><comment><USER31> A cool system would be a tool to find the location of a point relative to the sun. So I can whip out my smartphone while in the asteroid belt and have an intuitive metric system to identify any point in our solar system.

<comment><USER11> You'd need at least two more points of reference, I think, in order for this to be effective. By itself, a picture of the sun (and I'm assuming it's an incredibly clear picture taken with a magical instrument) would only tell you how far away from the sun you were.

With an additional known point (and an accurate date and time), you'd know where you were within a circle around the line between those two points. That might actually be good enough for a rough estimate, if you can assume that you're in the plane of the solar system.</comment></comment>


How do you organize what you've learned so far? - <AUTHOR> I surmise many of HN readers can relate to this problem, but how do you manage to organize what you've learned over the years? I feel like I am absorbing a copious amount of new information but feel like I rarely organize them in a fashion where I can revisit them and brush up on them at a glance. <comment><USER2> I try to put as much on paper or a digital equivalent of it as possible. My co-founder and I have habit of writing everything we hear, read about, or learn down so we can come back to it later. If it's for personal use - keep a word doc and just write down thoughts and discoveries, perhaps with links or references for you to re-read and re-visit later. For our business at [http://Airdyme.io](http://Airdyme.io) we keep a collection of spreadsheets pertaining to customer research, product development, fundraising and other activities. The more you can document the better. I believe even Airbnb's co- founder have mentioned a habit of writing down all kinds of crazy ideas on a simple notepad. You can watch that video here: [https://vimeo.com/23275754](https://vimeo.com/23275754)

</comment><comment><USER3> I have found this very helpfull: [https://medium.com/@RyanHoliday/this-unique- notecard-system-...](https://medium.com/@RyanHoliday/this-unique-notecard- system-will-help-you-remember-organize-and-utilize-everything-you- read-9f3ddabaa3c)

I use this same concept but use Evernote instead of physical cards

</comment><comment><USER4> I don't. I let my brain do its own thing.

If I forget something it's because I'm not using it much. If I forget something and need it again I search for it on the internet.

It's beautifully simple and requires zero configuration, cost or maintenance.

</comment><comment><USER5> I sleep so that my brains sorting algorithms can do their thing.

<comment><USER6> Yes, let them fight and may the strongest ideas survive.

</comment></comment><comment><USER7> Evernote. Bear.</comment>


Cerberus – Semantic models for C - <AUTHOR> https://www.cl.cam.ac.uk/~pes20/cerberus/ <comment><USER2> This is cool work. Peter Sewell has been giving talks about this, and he is a great speaker. Here is one from this year's EuroLLVM developers' meeting, including some demos of the Cerberus tool: "The Cerberus Memory Object Semantics for ISO and De Facto C" [https://www.youtube.com/watch?v=SI3uycKDH8Y](https://www.youtube.com/watch?v=SI3uycKDH8Y)

</comment><comment><USER3> This is scary. I love and hate C, been using it almost 40 years. Its use seems more and more like juggling razor blades, wishing not to get cut but still always wondering when it will happen next.

Rust, Pony, Crystal, Go, Elixir, Haskell, OCaml, Lisp, Clojure have their merits to address most of C's shortcomings, offer much higher levels of abstraction (productivity) and provide these features that didn't yet exist when C grew up. Languages were mostly built bottom-up.

But really, C is often used as a portable-ish lingua franca, along with similar but different: JVM bytecode and LLVM IR, for writing nearer to bare metal (native assembly or runtime).

<comment><USER4> Those features did exist on ESPOL, NEWP, PL/I, PL.8, PL/S, BLISS, Concurrent Pascal,....

They just lacked an OS being offered almost for free, with source code to come along.

<comment><USER5> That's a wrong reading.

C hit it big in the 1980s as a language that was portable between smaller (PDP-11) and larger (VAX) minicomputers as well as microcomputers that had a 16-bit address space at best with memory management limited to switching a bank of ROM in and out.

I spent about $80 for a C compiler that run on the OS/9 operating system on my TRS-80 Color Computer. When I upgraded to a PC with MS-DOS there was a Microsoft C compiler that cost about $500, a Turbo C Compiler from Borland that costs around $100 and Tom Mix had a compiler that cost about $20. This was long before GCC.

C was astonishingly portable across machines of variable capabilities; I remember many Unix games like Nethack that would run on MS-DOS as well as they did on Sun workstations. Commercial C compilers supported a multiplicity of "modes" to work with the strange segmented architecture: for instance you could have a 64k address space for data and 64k code.

My favorite language of the late 1980s was Turbo Pascal which had extensions to support systems programming but was a better language than C in most respects and had OO extensions that were much easier to use than C++. I switched to C when I went to college though because I had access to Sun Workstations where C was the way to go.

GNU tools were already establishing themselves as the Cadillac of userspace (e.g. there wasn't commercial competition to make a better awk for HP/UX) but it was another three years before Linux came around and 32-bit x86 chips were starting to whup SPARC and PA-RISC on performance, never mind price- performance.

<comment><USER4> Of course it hit big in the 80's.

It was only about 1986 that AT&T was lifted the ban that prevented selling UNIX, which triggered the law suit against BSD a couple if years later.

The damage to other commercial OSes was already done by then.

I got to use Turbo C in 1992, after a couple of years with Turbo Basic and Turbo Pascal. Thankfully the teacher that brought us Turbo C, also had a copy of Turbo C++ 1.0 around.

So Xenix was the only UNIX where I was forced to use C, and a couple of university assignments a few years later.

Everywhere else, just following Bjarne's footsteps, I enjoyed C++ on UNIX, even with the quirks of catching up to C++ ARM.

</comment></comment></comment></comment><comment><USER6> Is there a set of preprocessor directives for use in Cerberus that will make programs compatible? That is,ignoring the undefined issues.</comment>


Ask HN: How do you offer bug bounties for open source projects? - <AUTHOR> I am the maintainer of an open source project and would like to offer bug bounties and rewards for testing the application. The thing is that platforms like Issuehunt or Bountysource seem to be limited to Github only. Other platforms like HackerOne look more like an enterprise SaaS and seem to not fit.<p>Does anyone have an recommendation or idea? <comment><USER2> You could try [https://www.upwork.com](https://www.upwork.com)

I've hired a lot of great people from there. So you can either post a job with a fixed price or pay by the hour.</comment>


Apple quietly bought an unusual startup - <AUTHOR> http://fortune.com/2015/05/28/apple-augmented-reality/

<comment><USER2> This was a long time coming. It definitely shakes up the AR space. They were one of the only real independent players in the industry.

It's a good play by apple too. It almost brings them up to speed with the other major players. MS has the Hololens team, google's got Tango, Oculus has been snapping up computer vision companies left and right. Up until now Apple didn't have anything in the AR space.

Metaio had a strong computer vision patent portfolio too [https://pbs.twimg.com/media/CGGGaveUgAAKFje.jpg:large](https://pbs.twimg.com/media/CGGGaveUgAAKFje.jpg:large)</comment>


Flexible systems are the next frontier of machine learning [video] - <AUTHOR> https://www.youtube.com/watch?v=Jnunp-EymJQ#t=245 <comment><USER2> Talk starts at 4:05, you can skip the intro without missing anything: [https://youtu.be/Jnunp-EymJQ?t=245](https://youtu.be/Jnunp-EymJQ?t=245)

<comment><USER3> Added above. Thanks!</comment></comment>


Make mobile Application like a powerpoint - Victor-he-sun https://appstylo.com/ <comment> Victor-he-sun We're in beta testing stage. We offer to our early adopters 3 months of free usage. No credit card is required.

I hope you like our product and we appreciate your feedback very much.</comment>


My Favorite Deep Learning Papers of 2017 - <AUTHOR> http://cachestocaches.com/2017/12/favorite-deep-learning-2017/ <comment><USER2> Super stuff. Can anyone here who reads these research papers tell me their process to understanding them? Does anyone try to recreate the models using the math from the papers? I'm interested in this stuff at a hobby level, but find my mind glazes over when trying to process what is being talked about. It would help if these papers were more interactive, or perhaps I need to sit down and try and build these models in tensor flow perhaps.

<comment><USER3> Agh I typed up a response but it got deleted somehow. But basically, I view there as being 3 layers of understanding. For whatever level you're stuck at, there's different ways to get "unstuck".

1\. Understand the task they're solving, how they did it, and their results. Anybody with a basic understanding of the domain should be able to do this by reading the abstract/intro and conclusion. If you find yourself having trouble here, you probably just need more background in the field.

2\. Gain some intuition of why their method works. This is probably one of the hardest parts to figure out how to do, and is probably the part that most people stumble on. Really, this is basically the entirety of what you're trying to do when you learn math. There's also varying levels of intuition. There's "I get why this might work", "I get why this works", and "I get why it's impossible that this doesn't work", in order of difficulty. The more background you have, the easier this intuition is to grasp. Alternatively, you can bootstrap your intuition by reading other people's blog posts, talking to somebody who understands the paper, asking the authors, playing around with your own implementations, etc. I'm not sure anybody has any good answers for this stage. Personally, I really like good blog posts from bloggers I know are good, but unluckily, many papers do not have blog posts attached :(

3\. Finally, there's the strict mathematical rigor part. These levels aren't really strict; oftentimes, I'll treat math I'm not familiar with as a black box theorem. If you don't have the math background for these proofs, there's usually not much better recourse than learning the subject properly.

Luckily, many ML papers barely have any mathematical proofs :)

Alex Irpan has a great explanation of the Wasserstein GAN paper here: [https://www.alexirpan.com/2017/02/22/wasserstein- gan.html](https://www.alexirpan.com/2017/02/22/wasserstein-gan.html)

If you're looking for interactive blog posts, distill.pub probably has the best: [https://distill.pub/2017/research- debt/](https://distill.pub/2017/research-debt/)

And one final note: Many papers (especially papers that aren't math papers) are often surprisingly simple to get to step 2; it's just hidden behind a lot of cruft. I will say that it's wise to be careful about so called "intuitive explanations" of a concept. If somebody gives an "intuitive" explanation for why X is true, but that intuitive explanation doesn't explain why !X is false, it's not very useful.

<comment><USER2> Thanks for the links, hadn't even heard of distilling but the papers on that site are a lot more approachable for sure. Thanks for the other insights too, I will keep them in mind for the next paper I read.

</comment></comment></comment><comment><USER4> Are GAN's pretty hot these days or is that just a coincidence of the authors preferences? I just received Ian Goodfellow's Deep Learning textbook [1] and I know he pretty much invented the technique, so I'm wonder how influential / important GANs are in the field.

[1] [http://www.deeplearningbook.org/](http://www.deeplearningbook.org/)

<comment><USER5> GANs are for graphics, not machine learning. They have no test scores as they cannot run on the test set. Therefore, its hard to tell how much they overfit.

But they have uses, like for lossy compression of images or texture generation. Stuff which focuses on the graphical side of things where it outperforms machine learning methods with its crisper samples.

For an overview of this argument: [https://arxiv.org/abs/1511.01844](https://arxiv.org/abs/1511.01844)

The idea of adversarial training is important and relevant in ML though! It allows for setting up losses which are hard to formulate otherwise.

<comment><USER3> Huh? GANs are for graphics, not machine learning? I mean, first of all, how are GANs not machine learning? They are definitely "machine learning", and I don't think anyone would disagree.

Second of all, GANs are most definitely not just for graphics. They've been applied to text generation, to generating adversarial examples, to data preprocessing, etc.

Third, I have no idea what you even mean by "test set" in the context of GANs. It is true that it's hard to tell their performance, but that's irrespective of whatever you're talking about. It's hard to evaluate performance because we're usually judging the quality of the generated images, and we don't have any good ways of evaluating "perceptual loss", or how real an image looks.

As for the OP, GANs have been a very hot topic. Not as hot as this blog post makes them look perhaps (with nearly every paper about them...), but I wouldn't really disagree with any of the papers posted. Only one I'm not familiar with is the "most useful" one, but the rest were all pretty great papers imo. As for Ian Goodfellow, he's a very smart guy who seems to do a pretty good job explaining things. I saw a couple YouTube videos from him at a meetup covering his DL book, and he did a great job teaching.

<comment><USER6> Although I would agree that GANs are part of machine learning, some people definitely do disagree, and their concerns are valid. It's definitely an area of open research.

Your third point is actually the point of those who disagree. It's the same reason why we have the principle of unfalsifiability in science.

<comment><USER3> I'm a bit confused about the point that you're stating. I've never seen anybody not group GANs under machine learning.

Machine learning is typically split into supervised learning, unsupervised learning, and reinforcement learning, and GANs are usually considered part of unsupervised learning. I guess the part I don't understand is what you mean by "their concerns are valid"? What are their concerns about? Whether GANs are a promising path of research? And if GANs aren't part of machine learning what are they?</comment></comment></comment></comment></comment>


Ask HN: How to decide on price for first freelance project? - <AUTHOR> I have recently gotten involved with a project with a trader who wants to build an algo trading platform to perform his strategies a bit less manually. It sounds like the scope is quite big and it will obviously have a front end and back end.
We're deciding on price right now and I'm really struggling as I haven't done any freelance projects before (only personal). We have agreed on about 15 hours a week and I'm based in London. The only thing that gives me some indication is that I tutor quite regularly and charge £60/hour. I have more than enough students and I'm sure if I tried I could get the same 15 hours a week. I have almost 3 years professional experience in finance companies.
Would I be charging too much or too little? If so, what would be a better way to find my price? <comment><USER2> If the scope of the project is huge then don't go for fixed-bid quote, go for a T&M quote. Also, when freelancing you need to consider other costs (e.g. insurance, medical, pension, electricity, computers, internet connection, furniture, taxes etc) that the employer will not be paying but you will have to pay. So factor that in to your quote. The London contract rates is somewhere around £65 to £200 per hour depending on the project and your skill- sets.

<comment><AUTHOR> It's not so much that it's huge, more that it will almost definitely be an ongoing project. I agree not to have it as a fixed quote. I have a full time job too so I won't need any of those things (bar taxes/accountant of course) , this is purely part time and remote.

Thanks for the advice!</comment></comment>