Show HN: Spamnesty: Waste spammers' time - <AUTHOR> https://www.stavros.io/posts/spamnesty-waste-spammers-time/ <comment><USER2> I'm just waiting for someone to hook this sort of stuff to some kind of machine learning system, to learn and adapt and come up with new replies. And some 100 years hence we'll be explaining to kids that the singularity arose out of people's annoyance with spammers ;)

<comment><USER3> I think that this or something very similar was a plot point in Neal Stephenson's Anathem. Cannot find a good reference, but it was pretty entertaining.

<comment><USER2> The Artificial Inanity and bogons, yes.

</comment></comment></comment><comment><USER4> See also, The AI that wastes telemarketers time [https://news.ycombinator.com/item?id=13013327](https://news.ycombinator.com/item?id=13013327)

And this script that plays what sounds like an old man losing his memory: [https://www.reddit.com/r/itslenny/](https://www.reddit.com/r/itslenny/)

<comment><USER5> How can I get Lenny? :) That is awesome.

These is how to use it: Transfer, conference, or forward your telemarketing calls to 1-347-514-7296

Can't wait to try this out.

<comment><USER6> Doesn't work on cell phones.

<comment><USER7> I've always wondered why smartphone apps don't have better access to the telephone portion of the device. Why is it so difficult to do something as simple as recording calls? It would be neat if I could get apps to change my voice or encrypt the audio.

<comment><USER8> I don't know if this is why, but there are a bunch of laws and stuff about what you can and can't do on a telephone. For example, many states have limitations on who can legally record a phone conversation, with or without various forms of consent. Maybe phone manufacturers don't want to facilitate easily breaking those confusing laws.

<comment><USER7> They don't seem to mind people using the phone when they drive and that's also illegal in many jurisdictions.

<comment><USER9> They do mind, there just isn't a clear cut of a solution they can force someone to put into code. You can't reliably detect if the phone is being used by the driver or by a passenger.

Without being able to make that distinction, it's nearly impossible to force phone software makers to write rules preventing "illegal" phone use in situations where it is illegal.

Instead, they punish using verifiable methods such as direct observation (cop sees you doing it).

The situation may be similar for recording calls (I'm not familiar with exact laws in this area), but I'm assuming there are also privacy restrictions in calling which lead to the software restrictions.

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> This is great and seems to work, check out this thread:

[https://spa.mnesty.com/conversations/hszegufh/](https://spa.mnesty.com/conversations/hszegufh/)

<comment><USER11> I can't believe I just wasted time reading voluntarily through a thread of spam mails

<comment><AUTHOR> You have been foiled by my evil plan, my goal was to sell you penis enlargement pills all along.

</comment></comment></comment><comment><USER12> The problem with Spam has always been the minuscule cost of doing it. If the cost can rise above the profit, spam could become unattractive. Imagine if spam attracted a sudden swarm of bots, just as a new email addy attracts spammers now.

<comment><USER13> And bots could do pretty well there. I recently read an article (cannot find it; but it may have been even linked from here) which explained the benefit of poor grammar in the spam (i.e., they want to put off more knowledgeable users as the cost of the follow on emails is significantly higher).

El cheapo bots have pretty basic English skills which can help them fit naturally into the spam target patterns.

<comment><USER14> The paper you're referring to is probably this one: [https://www.microsoft.com/en-us/research/publication/why- do-...](https://www.microsoft.com/en-us/research/publication/why-do-nigerian- scammers-say-they-are-from-nigeria/)

The same logic that favours using an opening gambit that scares off all but the most unworldly of people works against using bots for the followup conversation. You don't want to finally find someone that genuinely believes that a random central African stranger might want to split $138,400,000 with them and has funds ready to send only for your bot to fail the Turing test by misunderstanding their question about how to pay the deposit.

Also, anyone smart and knowledgeable enough to be able to write bots capable of persuading exceptionally gullible people to part with their money is probably capable of moving to the next level and going after the slightly-less gullible, who are more numerous and richer.

It's also probably a bit flattering to the average scammer to assume the flaws in their patter are a deliberate filter pattern though. They're just as likely to open with a highly plausible (probably copied) offer of high value second hand goods on a listings website that probably attracts dozens or replies and then respond to each of those genuine expressions of interest from people willing and able to pay with badly-written response(s) that conflict with the detail of the original ad, not to mention totally forgetting that little old Scottish ladies don't have email addresses registered in the name of Nigerian men.

<comment><USER15> "It's also probably a bit flattering to the average scammer to assume the flaws in their patter are a deliberate filter pattern though."

It doesn't have to be deliberate. It probably evolved. If worse spelling and grammar work better than correct spelling and grammar, then even if no spammer ever consciously says "Ah, I need to make this crappier", the spams will evolve to be crappy.

Moreover, while our brains in this discussion are taking cognitive shortcuts and putting spams on a "correct <-> crappy" single dimensional continuum, the reality is probably even more complicated, and it's actually _specific_ crappiness that works better than others. For instance, I could hypothesize that it's not just typos, but typos that the recipient plausibly believe are the result of a non-native speaker from the relevant country. (I say "plausibly believe" rather than whether the non-native English is truly representative of the relevant country, because that's what matters.) That's an obvious possibility; more subtle things are possible and probably even likely. And again, the spams will evolve into those more subtle possibilities and exploit them even if no individual participant is sitting there and trying to deliberately figure out the best spams to send... which is, obviously, almost certainly untrue as well. And whoever is sitting down to write The Perfect Spam is probably doing so with a lot more data and a lot less scruples than you or I would apply to the problem.

I've often thought one of our best anti-spam defenses is the sheer mailbox stuffing quantity of them. Even the best crafted spam about how I won the lottery I never entered looks a lot less plausible the thousandth time I receive it.

<comment><USER14> Occam's razor implies that if the people originating the scams mostly are based in central Africa (true) and display the specific crappy spelling patterns and linguistic quirks of many other moderately-educated central Africans using the internet for non-nefarious purposes (also true) one needn't assume that there's any particular process that leads people to write like their compatriots. Especially not when similar imperfections betray their attempts to masquerade as a Scottish widow selling their husband's canal boat or Australian trader that wants to buy your product if you'll send him the import tax.

I'm also little inclined towards scepticism there's much evolutionary optimization going on when the standard email scam format hasn't even adapted to get around now unbiquitous standard webmail spam filters, which are essentially orthogonal to the gullibility of the account owner.

Besides, wouldn't an evolutionary process that wasn't a conscious attempt to avoid generating false positives (or slavish copying by people that don't really analyse in much depth) tend to optimise for reusing the emails that generated more responses rather than less?

</comment></comment></comment></comment></comment><comment><USER16> That is reasonable, but do you think you can go into more detail about your main product? What is it about, exactly?

<comment><AUTHOR> Hello my beloved friend,

I hope you and your family are healthy and happy. As I have detailed in my previous correspondence, my offering to you is a way to waste spammers' time by making them reply to emails. I am sure your esteemed self would have great use for such a service. Please wire $1500 to my account for attorney's fees.

Thank you and God Bless,

Stavros

<comment><USER17> Hello Stavros, I'm not sure I understand. Could you go into more detail?

*In all seriousness though (lol): You could have different types of personalities, depending on the email address of the person who doesn't want the spam. Like someone pretending mnesty.com is their work address but is still treating it like a personal message. That way the spammer will disregard the fact that the reply email isn't the same as the original. EDIT: Nevermind, I see you said you did that already... I think. ... Help?

<comment><AUTHOR> It's too hard to write generalities, I'm afraid. You can only write about 10 different sentences before you start to repeat yourself (or at least I can't do it). Feel free to send me sentences for me to add!

<comment><USER18> This is amazing. Looping in my supervisor. [add a second email agent to thread. The two email agents will fork the email chain for the spammer and ask similar questions, hopefully wasting more of their time.

</comment></comment></comment></comment></comment><comment><USER19> Someone posted about Lenny [1] the other day. I've had fun listening to him on youtube waste telemarketers' time.

[1] [https://www.youtube.com/playlist?list=PLduL71_GKzHHk4hLga0nO...](https://www.youtube.com/playlist?list=PLduL71_GKzHHk4hLga0nOGWrXlhl- i_3g)

<comment><USER20> Lenny is great. My 11-year-old and I play the Lenny game sometimes -- one of us is telemarketer, the other plays Lenny. Lenny is a brilliant piece of software. I just worry that eventually most call centers will be onto Lenny. Already some telemarketers recognize him. Luckily, as AI and voice recognition improve, more such systems will come online for the amusement of youtube audiences and to thwart telemarketers and scammers.

</comment></comment><comment><USER21> I'm guessing the emails this generates will become flagged as spam themselves. Do spammers use spam filters too or would that prevent them from getting replies?

<comment><USER22> Spam filters could do very little, because it's almost impossible to tell whether a conversation was run by a bot - and it's outright _impossible_ to do so for a single response.

However, you raise an interesting point: blacklists could reasonably be put into place. I'll make a merge request to use Spintax.

<comment><AUTHOR> If you do, I would definitely be interested in that, Spintax seems like a very easy way to add some variation.

</comment></comment></comment><comment><USER23> Very nice, and a great test bed for AI code. Lots of low hanging fruit, like follow-ups to specific branches of conversation, extracting certain information from the spammers replies to make the answers look more real, etc.

Someone should organize a competition to see who can create the longest running conversation (or the funniest one).

<comment><AUTHOR> > Very nice, and a great test bed for AI code.

Yeah, that's an interesting aspect of it for me. I'll have to clean up the code a bit, because it's very "weekend project" quality, but it's an interesting way forward.

</comment></comment><comment><USER24> I had a friend who tried messing with spammers and then was hit with a massive DDOS.

<comment><USER25> Was your friend working for Blue Frog [1]?

I always thought it interesting that no one seriously tried to replicate a tool so good at its job that spammers were actually scared.

[1] [https://en.m.wikipedia.org/wiki/Blue_Frog](https://en.m.wikipedia.org/wiki/Blue_Frog)

<comment><USER24> No. He set up some one-off script to annoy people who spammed him. One of the spammers didn't take to it.

</comment></comment></comment><comment><USER26> I love this. I've actually presented on the concept before - more so about ensuring that people targeting your user base have to waste a lot of time without realizing it by doing things like shadow bans weeks before actually "detecting and cancelling their account" but this is a perfect example of the concept.

Everything, even spam, is based on ROI.

</comment><comment><USER27> Have you thought of adding in a check in if they haven't responded after a week?

Perhaps, "Hi there, I thought we were really making progress, but I haven't heard from you for a week. Is there still a chance I could get in on this great opportunity?"

<comment><AUTHOR> Yes, but right now it's very event-driven (each email triggers a new one), so it would be more complicated for little benefit (given that spammers wouldn't reply anyway, they only stop replying when they think you're wasting their time, and that wouldn't change with an additional email).

</comment></comment><comment><USER28> Really cool. Two remarks:

1) If lots of people start using this, spammers might learn to avoid wasting time with Mnesty's CEO. Having a few more fake companies would help.

2) The bot could probably be made a lot smarter with some natural language processing. But it seems to fill its purpose pretty well already. I think this just shows how desperate for leads spammers are.

<comment><AUTHOR> 1) Yeah, there's support for multiple domains, but I have no domains right now :) If anyone wants to donate one, please open an issue!

2) Exactly, they're so desperate for leads that they will spend lots of time chasing someone who doesn't appear to make lots of sense.

<comment><USER29> I have a few domains you could borrow? I can at the least offer you 5 domains for half a year, shoot me an email if you're interested.

<comment><USER30> I've now read so much spam, I can't help but see it everywhere! ;-)

<comment><USER29> You too can be the proud borrower of 5 domains! Just shoot me an email at email@example.com.

jk

</comment></comment></comment></comment></comment><comment><USER31> How is the bot tailoring the replies, for example

> Hello, I have wired the funds you asked for. Could we please proceed to the > next stage? Daniel Wong CEO, MNesty, LLC

Is that a canned response? This happens later on in the thread (does it recognise that the topic of the conversation is "funds" rather than say "product")

<comment><AUTHOR> It has various categories of replies, which right now you can specify when you forward a message (it sends you a management URL back). Later on, when I have enough data, I'll do some simple ML to have it pick a most likely category as a default.

Each category comes with its own set of responses, but you can see where the category changed because the bot will start talking about a product and then switch to talking about funds.

The code is here if you want to see:

[https://gitlab.com/stavros/Spamnesty/](https://gitlab.com/stavros/Spamnesty/)

<comment><USER14> It'd be more effective if it didn't re-use responses later in the chain (unless it'd totally run out of alternatives). Although, arguably, it's funnier when the spammers/scammers do continue to find new ways to reply to the same couple of questions.

<comment><USER32> To improve negotiations the bot should extract money amounts from the mail and start negotiating (and never stop).

</comment></comment></comment></comment><comment><USER33> Ahah, this is fun :) But please someone, let google know this exists. I just had a forward refused by gmail because "it contains viruses", and I realize I've forwarded 5 spams before that. I would not want google to think I'm "actually" sending spams :D

<comment><AUTHOR> Google refused because the forward contained viruses? Hmm, maybe if you strip the link?

<comment><USER33> Yep. That's ok, I have forwarded plenty enough :)

What surprises me is that a mail with viruses is in my mailbox (my spambox, granted). I guess it was not know it was a virus when it was received.

</comment></comment></comment><comment><USER34> > Spammers will usually realize they're talking to a bot (or at least to > someone who isn't interested or not going to give them any money) after > around 3-5 messages, but some have sent up to 15 messages before giving up > in frustration.

So it passed the Turing test?

<comment><AUTHOR> No, the Turing test requires that the interviewer not be trying to extract money from the interviewees :P

</comment></comment><comment><USER35> Finally an actually useful application for AI! No perfect result is required and this is also not something you want actual humans to do.

</comment><comment><USER36> [https://spa.mnesty.com/](https://spa.mnesty.com/) = server not found.

Anyone else seeing this?

<comment><AUTHOR> It has something to do with (the move to) Cloudflare. Refresh once or twice, it will load.

[https://www.timetaco.com/jaspsa/](https://www.timetaco.com/jaspsa/)

<comment><AUTHOR> It turns out that some DNS servers cached the old nameservers and are now failing to resolve those. It's unfortunate, but all we can do is wait for the cache to expire :(

<comment><USER37> Did you forget step 4 from [https://cr.yp.to/djbdns/run-server.html#move- zone](https://cr.yp.to/djbdns/run-server.html#move-zone) ?

<comment><AUTHOR> No, it looks like Namecheap stops serving the old records when you change nameservers.

</comment></comment></comment></comment></comment><comment><USER38> In my days we called this an autobaiter :)

</comment><comment><USER39> Nicely done. I worked a bit on the same concept, mine is not as good as yours, and it's an android app. If anybody wants to have a look, I'm really opened to suggestions. [https://play.google.com/store/apps/details?id=net.scampong.s...](https://play.google.com/store/apps/details?id=net.scampong.scampong&hl=en)

Funny thing is that most of the people who downloaded it came from Nigeria and seemed to think that it was a tool that allowed them to scam people. It was quite funny when they put their own contributions and then one of them was talking to a real scammer trying to scam him. Had to delete the thread cause it didn't make much sense though.

</comment><comment><USER40> This could works in short run as long as this type of service stay as niche and not too popular. Spammers always, always have a creative work around it. I imagined a counter measure from them by build a bot detector that will spam back the spambot and one hell of continuos email loop doesn't do any good to our internet traffic.

</comment><comment><USER41> We need a Chrome extension for this!! I would create it myself .. but there's not any public API.

<comment><AUTHOR> What sort of API would you need? What do you want the extension to do? Just show conversations?

</comment></comment><comment><USER42> I've tried forwarding a spam email from Apple Mail, but I got a response that it didn't work out. This is the raw email message I sent - [http://pastebin.com/qNwT04my](http://pastebin.com/qNwT04my)

<comment><AUTHOR> Thank you, it looks like your mail client adds forward headers in a way Spamnesty doesn't understand yet. I'll try to fix it soon.

EDIT: Can you add an issue to the tracker with the forward header verbatim so I don't forget?

</comment></comment><comment><USER43> There is one major problem : the answers are sent immediatly after receiving the email.

This is obviously suspicious, considering most of the mails require at least a few minutes of reading + time to write the mail.

I guess adding a delay would improve the bot's credibility by a lot!

<comment><AUTHOR> It actually has a random delay of 1-8 hours, which shows in the timestamp. It only sends the first email right away, because it presumably took you some time to forward it to Spamnesty.

<comment><USER44> What are you using for the event management? I wonder if using celery and rabbitmq or redis as the queue would help simplify things.

With it, you just schedule asynchronous tasks in the future ... you don't have to manage the event queue.

<comment><AUTHOR> The box these are running on is already pretty full (it's where I keep all my side-projects, and it's got around eight running on it now), so I'm trying to keep it lean. Features definitely get cut on a cost/effectiveness ratio.

</comment></comment></comment></comment><comment><USER45> Using machine learning like this [https://arxiv.org/pdf/1506.05869.pdf](https://arxiv.org/pdf/1506.05869.pdf) would be fun for this and avoid(?) canned responses...

</comment><comment><USER46> I forwarded a single spam email to Spamnesty at 8:46 AM today. Since then, it has been spamming me saying it received the email...

Timestamps:

8:46 AM - initial receipt email

8:56 AM

9:11 AM

10:11 AM

12:11 AM

The conversations linked in the subsequent receipt emails lack the spammer's email. They only contain the response from the Spamnesty bot.

<comment><AUTHOR> Yes, this is a known issue:

[https://gitlab.com/stavros/Spamnesty/issues/6](https://gitlab.com/stavros/Spamnesty/issues/6)

Can you post the IDs of the emails you received so I can look into the issue?

<comment><USER46> OK, I replied to the issue with the links. :)

</comment></comment></comment><comment><USER47> I usually just pretend that i just killed someone to get the money they asked for.

<comment><USER48> Lol. I used to bait spammers as a hobby, and after a few exchanges with one, I'd tell them how lucky I was that I had been identified as the real owner of $324 million in a lost government account. That I had to have been the luckiest person in the world, because not only that, just last week, I won the national Nigerian lottery, and the week before that, a distant great-aunt from Nigeria had named me sole heir in her will!

</comment></comment><comment><USER49> This is not a bad idea. Perhaps if the major email providers had bots that did this sort of thing, spammers would be overwhelmed with responses, with the tiny proportion of real fish lost in the mix.

</comment><comment><USER50> I've said it in the other thread where you mentioned it the other day, and I'll say it again: this is an awesome service!

And thanks so much for sharing the source on GitLab to reproduce it.

Really great work!

<comment><AUTHOR> Thank you!

</comment></comment><comment><USER51> I forwarded an email and got a response, but [https://spa.mnesty.com/](https://spa.mnesty.com/) doesn't load for me

<comment><AUTHOR> There appears to be some problem with DNS after I moved to Cloudflare. Keep refreshing, it will work, but sometimes people get NXDOMAIN/SERVFAIL when resolving.

</comment></comment><comment><USER52> I have never understood what were the goals of these messages. People saying they have a fund of millions. Is everything arranged for me to send money to them? Simpĺe as that?

<comment><USER53> One of the goals is to find someone who'll fall for such a scheme. If it was more plausible, they'd waste time on someone who'd end up not pulling the trigger when they realized it was too likely to be a scam.

<comment><USER54> Once spammers start using bots to carry conversation, this might change. If bots can handle part of the communication, it probably makes sense to start planning more subtle scams.

<comment><USER55> I'd assume they are already.

</comment></comment></comment></comment><comment><USER52> What happened to hashcash proposals to email protocols?

<comment><USER56> The problem with hashcash for email is that there's no way for the sender and receiver to negotiate the amount of work to be proved, so they must correctly guess how much to perform/require. The most sensible requirement to guess is zero, since _all_ existing email is sent without any proof of work, and the value of interoperability is higher than blocking spam. After all, if you're willing to lose interoperability, you can block 100% of spam by just not using email at all!

Note that it may be possible to have _servers_ negotiate an amount of work to perform, but having someone else (the server) perform the work defeats the point of the system. The _client_ must be told to do the work, but it's a core feature of email that it's not realtime: messages can be composed offline, and queued up by servers, so by the time a "needs more work" response is received, there's no way for the server to send a message back to the client.

<comment><USER57> Could the work requirement be advertised as part of the destination's MX record or something? I know that in theory you're supposed to be able to send the message to your upstream server and let it worry about how to deliver, but in practice it seems like these days you can count on being able to look up the target's DNS info.

Another problem is finding a quantity of work which is high enough to stop spam, but low enough not to stop legitimate traffic. You want people to be able to send email from their RPiZero or original iPhone, but stop spammers who may have massive botnets, or at least decent PCs.

<comment><USER56> I can't think of a way to make this work with email, but I do think a proof of work like hashcash would be useful for Web APIs instead of using API keys:

\- Give a 402 reponse if hashcash isn't included or isn't enough, with a link to a price URL

\- GETting a price URL may return different values depending on identifying information like IP address, user agent, etc.

\- Prices can be adjusted based on server load, whether we recognise this agent as malicious or benevolent, etc.

\- Different types of request can have different prices, e.g. GETting a specific resource could be cheaper than searching or performing some expensive computation.

There would also need to be a mechanism to avoid replays, which would make things slightly less RESTful. I haven't thought of anything more useful here than an increasing request ID.

<comment><USER57> I've done a crappy implementation for my blog to fight spammy comments. It works OK, probably more because it's a totally custom thing that isn't worth time for spammers to fight, rather than because it's actually effective.

The way I did it is:

\- When the user focuses in a comment field, the page makes a request to the server asking for initial parameters.

\- The server returns the number of leading zeroes required, the number of distinct hashes it needs, and a salt to use. (This is just a constant in my code right now, but could be varied based on client specifics.)

\- The page then crunches on the work as the user types their comment. The submit button is disabled until it's complete.

\- The proof of work is submitted to the server along with the comment. The server then checks to see if it's good and accepts or rejects. A properly working client should never be rejected (since it fetches the required parameters in advance) so the rejection doesn't have to be too fancy.

\- Replays are prevented by storing the salts in a database, and deleting them once they're used for a comment.

I changed the standard hashcash technique a bit by requiring the client to submit multiple distinct hashes. Requiring only one hash works fine, but results in a lot of variance in how long it takes to compute the proof of work. You might tune it for an average of 30 seconds, but a decent percentage of clients will get it in 1 second, or will take 60 seconds. By, for example, requiring 7 fewer leading zeroes but requiring 128 distinct hashes, you get the same average but with a lot less variance. You can also display a semi- accurate progress indicator this way. The downside, of course, is that you have to send more data and the server has to do some extra work to verify.

<comment><USER52> Is that code public and in Python/Go/Javascript?

<comment><USER57> My web site is a hacked-together thing that's been gradually accumulating since the late 90s, so it's kind of ugly. The hashcash code is not very modular, either.

If you'd like to see it anyway, I pulled out the relevant parts here:

[https://gist.github.com/mikeash/daf02b6cc3017560f930515c3c17...](https://gist.github.com/mikeash/daf02b6cc3017560f930515c3c17af43)

The comment-inline.js file is directly embedded in the HTML for the comments area, and is the glue code between the actual UI and the hashcash computation code. The hashcash.js file is where all the client-side work happens, and it handles the actual hashing, making multiple attempts, checking to see if an attempt produced a good result, and such. Then commentsubmit.py handles the server side by returning hashcash parameters when requested, and checking the provided hashcash for validity when submitting.

I have a brief blog post about it here, which you can also use to see the system in action:

[https://www.mikeash.com/pyblog/testing-hashcash-based- anti-s...](https://www.mikeash.com/pyblog/testing-hashcash-based-anti-spam- measures.html)

If you have any questions, comments, or criticisms, please feel free to get in touch.

</comment></comment></comment></comment></comment></comment></comment><comment><USER58> Don't you think spammers will react like antibiotics? Getting better at it while we're building better bots?

<comment><AUTHOR> All we need to do is make bots that are more convincing than their victims. Can't be too hard, since they prey on the least savvy people.

</comment></comment><comment><USER59> Wow. How can I set a filter so that all email marked as spam by gmail is automatically forwarded to your service?

<comment><AUTHOR> Probably don't do that, I'm not sure I can handle that much spam. Mailgun would begin charging for that much email, for one (I'm on their free tier now).

</comment></comment><comment><USER43> Is there any way to have multi-language support?

</comment><comment><USER60> Can't you connect it to a chatbot api?

</comment><comment><USER61> This is pretty awesome!

</comment><comment><USER41> Love it, good work!

</comment><comment><USER62> This is hilarious

</comment><comment><USER63> This needs to send fake name address and bank account details when rerequested. That's where the threads would start to get more interesting.

Another idea is to send two spammers responses to each other.

</comment><comment><USER64> I am surprised (and a little embarassed for my race) to see so many Indian names in the About page of [http://www.mlooper.com](http://www.mlooper.com)

Most appear to be "Web Developers". Ex: See this Conversation: [http://www.mlooper.com/conversations/2912/](http://www.mlooper.com/conversations/2912/)

Is this common? Do a lot of you get spammed with Indian companies / individuals claiming to do web development? I have received a few, and the funny thing is, NONE of the ones I received even had a Web Site?! How does that work?!!

</comment><comment><USER65> This seems like a noble endeavor which will be useful (albeit under different names over time, I assume) right up until the spammers start using bots to hold their ends of conversations, to make their own operations more efficient.

<comment><USER28> If they do so they will likely lose the very few leads that actually earn them money. Remember, for them it's a real business.

<comment><USER65> Eh, machine learning is advancing enough that the sort of people that would fall for Nigerian prince emails could be conceivably convinced they're talking to a real person.</comment></comment></comment>


eBay launches visual search tools that let you shop using photos - <AUTHOR> https://techcrunch.com/2017/10/26/ebay-launches-visual-search-tools-that-let-you-shop-using-photos-from-your-phone-or-web/ <comment><USER2> Some brilliant mind who lurks on HN should create a competitor to eBay. Not some online yard sale where you have to meet people in person for the exchange, but a nice online marketplace for people to sell their stuff.

If this already exists, do tell.

In my opinion, eBay for sellers (outside of maybe the power sellers. I've sold <100 items) is a nightmare.

I sold a Tiffany necklace a few months back. The buyer reported to eBay it was fake and I was ordered to refund the seller and they could keep the fake item. Long story short, over a span of a few weeks, I was lucky enough to get an original receipt from Tiffany and supply it as evidence, which almost still did not work.

More recently, I listed an old iPhone with a "buy it now". Sold in like 30 minutes to someone with an "@god[dot].com" email requesting I send the phone first and then he will pay me. It took a week or longer for me to challenge this. I even have settings to disallow certain types of eBay users based on ratings etc. eBay still charged me a listing fee.

I had success doing a non buy it now sale after that. Maybe that is my only option now.

I don't even feel like getting into how slow their seller admin tools are ...

<comment><USER3> >If this already exists, do tell.

Yes, it does exist, and it's completely open source. It's called OpenBazaar, and it's a fully decentralized marketplace. There's no middlemen at all.

[https://www.openbazaar.org/](https://www.openbazaar.org/)

It's backed by the OB1 company, which has raised $4.25 million from a16z, USV, and BlueYard.

It uses IPFS so that stores don't go offline, and all payments are settled in Bitcoin.

We love code reviews and pull requests. The back end is done in Go:

[https://github.com/OpenBazaar/openbazaar- go](https://github.com/OpenBazaar/openbazaar-go)

The front end is an Electron app:

[https://github.com/OpenBazaar/openbazaar- desktop](https://github.com/OpenBazaar/openbazaar-desktop)

<comment><USER4> Being a desktop application is a pretty serious barrier to usability for the average person. So is using Bitcoin, to be honest.

<comment><USER5> Bitcoin is an accessibility barrier, but OpenBazaar is also an example of the kind of models bitcoin reduces the hurdles for.

<comment><USER6> it is certainly a barrier, but if sellers recognize that sales over bitcoin has lower costs (cost of fraud) - certain items should show up on these marketplaces and people knowing about that tech will be really happy to get bargains... or so it should work :)

</comment></comment></comment></comment></comment><comment><USER7> I thought the title said "let you STOP using photos (when selling)", which is what has basically already happened due to the prevalence of stock images and CGI items on a white background.

I'm still looking for the browser extension that lets you filter out any listing with a pure white or gradient background from your eBay, etsy, pinterest, etc. searches.

Would be a HUGE indicator of an actual real used item sold by a human, vs. a new dropshipped item from China. You know, what I want to use eBay to buy.

<comment><USER6> Amazon for example requires you to put picture on white background. They do not actively enforce it, but if someone complains - they will remove picture which does not fit their requirements.

<comment><USER7> Makes sense for Amazon, which is mainly a store for new items / open box used items. Don't care about pictures of the _actual_ item.

If I'm buying vintage/antique items on etsy or eBay I want to see the real item, not a stock photo.

</comment></comment></comment><comment><USER8> I'm pretty sure Taobao in China has had this for ages. Search suggests[0] since as early as 2011. To test it, go to Taobao[1] and click the photo icon at the right of the search box (previously only available in some countries and product categories, now apparently global). If this was the other way around it would be "OMG China is copying the west!" No such discussion the other way around. Just sayin'...

[0] [https://www.chinainternetwatch.com/1189/taobao-imagine-an- im...](https://www.chinainternetwatch.com/1189/taobao-imagine-an-image-search- engine-for-shoppers/)

[1] [http://www.taobao.com/](http://www.taobao.com/)

<comment><USER9> Sort of; they also have links to visually-similar items if you hover over a listing, if there are the 'right' amount of matching products.

I mean, how many relevant listings are you going to find for a stock photo of a 7x7mm QFP32 chip? Still, that's a risky thing to buy on taobao anyways, and it is useful for things like specifically-shaped buttons, connectors, etc.

</comment></comment><comment><USER10> I have been trying to use Amazon's visual search for the past week or two and out of many trials, it has only worked once.

I wonder whether eBay will beat Amazon in this category as the types of photos they have access to (amateur/home vs. professional/stock) are very different. Either way, I am not confident that taking a picture is a better solution than just typing in the name for the vast majority of items.

<comment><USER11> >Either way, I am not confident that taking a picture is a better solution than just typing in the name for the vast majority of items.

It's a great solution for "I want another one like this one but searching for it returns 10000 different items"

<comment><USER10> That seems like a rare case to me. Typically there's a company name or model number somewhere on the item, and often that identifier returns a very specific item.

Looking around my house, I only see a few items that don't have unique identifiers like furniture or plates/utensils. However, I suspect the majority of things sold through Amazon/eBay are more likely to be video games, books, toys, etc. with very visible and highly unique names.

<comment><USER12> Consider fashion or home goods. Frequently you see something you like based on pattern, texture etc that cannot be easily described in words. Image search is the perfect fit for these kinds of purchases.

</comment></comment></comment></comment><comment><USER13> My pet startup idea, going back I suppose 15+ years is a service that does this:

You have something. You like it. You want another one. The service gets it for you. This doesn't have to work from a picture -- it could use a UPC barcode or a text description.

Idea originally inspired by a school friends of mine who wore the exact same type of boots for years and years. When the old pair wore out, he'd buy another pair the exact same.

Over the years I've noticed many instances where I really want another one of something I have but it turns out to be either painful or impossible to find. You know however that somewhere someone has a warehouse full of the things.

Also like the electronics industry brokers who can track down an old component for you to repair gear or re-start manufacturing on some old product. Those have existed since at least the 1970's.

A few years ago I tested Amazon's visual product search feature to see if it would do what I wanted. It didn't. I showed an image of some brown boots I was wearing, and certainly the suggested products were also brown boots, but not the same kind.

<comment><USER14> I would totally use this. 8 years ago I bought a pair of Nike shoes in Paris. I loved those shoes. I tried millions of things to buy a similar pair but I was unsuccessful at finding them.

Maybe the service should be like FlightFox, but for people who are really good at finding stuff. You pay a commission to the individual who helps you find the item.

<comment><USER15> I've been building an automated service very similar to this for womens fashion (mens fashion soon) in my free time over the past 5 months using deep learning. The goal is to index existing items, and then expand to items that are no longer available. So you could take a picture of your shoes/shirt/hat/etc and it would ideally find the same one. If it doesn't find the same exact item, it would find similar items at different price points that you could buy. Getting similar items isn't that hard -- getting the exact item is much trickier though. With the pace of improvement in these types of models, there will probably be a lot of these types of apps popping up over the next few years.

</comment></comment></comment><comment><USER16> It seems like ebay would have a ton of user-provided photos which would be a possibly _huge_ corpus for machine learning to help identify objects like this. It will be interesting to see how well this works!

<comment><USER17> The yellow sofa example in the article looked very convincing.

</comment></comment><comment><USER18> I know a bunch of sites has this feature already but I'm pretty excited to try it out since eBay is probably the website with the best dataset. I do not think any other site has as many home taken pictures as eBay. This would definitely make the image recognition much more robust. I'm guessing the problem with Amazon or TaoBao is that most of the pictures are studio pictures and does not scale well when a person takes a picture in a random environment.

That being said though, the eBay buying and selling experience is a hit or miss. It sucks how there is group of people preying on the site to rip people off. I had an selling experience so bad that I vouched never to sell anything via eBay anymore. Craiglist tends to be a much better transaction.

</comment><comment><USER19> To test a machine learning algorithm, one can use "Fashion-MNIST": [https://github.com/zalandoresearch/fashion- mnist](https://github.com/zalandoresearch/fashion-mnist)

"Fashion-MNIST is a dataset of Zalando's article images—consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. We intend Fashion-MNIST to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms. It shares the same image size and structure of training and testing splits."

</comment><comment><USER11> Seems like a great way for eBay to undercut brick and mortar stores (and for consumers to save money) on cheap junk from China that people would rather pay $0.99 and wait 3wk for than pay $4.99 and have right now.

<comment><USER20> Also dropshippers who resell such Chinese products under different names for a margin.

</comment></comment><comment><USER21> I have made a browser extension[1][2] for those looking to perform reverse image searches with multiple engines at once. It's more reliable than Chrome's built-in image search, which appears in the context menu only for <img> elements, while this extension parses the clicked area to extract images. Support for shopping sites may come in the future.

[1] [https://chrome.google.com/webstore/detail/search-by- image/cn...](https://chrome.google.com/webstore/detail/search-by- image/cnojnbdhbhnkbcieeekonklommdnndci)

[2] [https://addons.mozilla.org/en- US/firefox/addon/search_by_ima...](https://addons.mozilla.org/en- US/firefox/addon/search_by_image/)

</comment><comment><USER22> Interesting how we built a raw prototype about eight months ago ([https://imgur.com/OIaQu7x](https://imgur.com/OIaQu7x)), but it never took off (at least for the customer whom we've pitched). To add to other's comments, the visual search can be quite useful comparing to terms search specifically for non-native speakers and people who are not skilled in fashion terms (i.e., the name of the material, pattern). As for how accurate it is, this is indeed an existing problem. If you product set is significant (>10k products), then the visual search results can be entirely off the target, not surprised Ebay is dealing with the same problem.

<comment><USER23> I didn't find that surprising since your blue bag results in the demo returned a bunch of bags that didn't look at all like the original.

<comment><USER22> The fuzziness was added on purpose in a way to recommend you products you may be interested in (not necessarily the exact match).

<comment><USER23> You won't succeed in business if your response to feedback is defensive and you don't even try to get to know your target user.

Your reply _should_ have been 'oh really, why is that?'.

</comment></comment></comment></comment><comment><USER24> Can someone explain to me why eBay is trading only at 5.5x P/E? I fully understand that it isn't growing tremendously but with all its classified assets, I can't help but feel that it should be trading at a much higher market cap.

<comment><USER25> Interesting point. But p/e so long, and they are never paying dividends, what is happening with that money?

I don't think of ebay highly, desides what they did to Craigslist.

They had Skype. They could have been Facebook, but no. They had Paypal. With Skype and paypal they could have easily made micropayments on cell phones, but no. I hardly ever see a company with so many wasted chances. They are in a "winner takes it all" niche but sooner or later someone (Open Bazaar?) will challenge them and I don't see any innovation from this company.

</comment></comment><comment><USER26> Last weekend, my dad needed new printer cartridges and so we went sifting through the internet for different offerings. By chance, we got onto a page on the manufacturer's website, which had images of all their cartridge packs. Among those, we also found an XL version of what we initially thought to buy.

So, there we were, needing exactly this feature. If I didn't know that it's impossible for them to have developed this in such a short time, I'd be a little freaked.

Thankfully, though, TinEye served the exact same purpose...

</comment><comment><USER27> Can anyone recommend a good tutorial or reading on how to setup content-based (visual) image searching using a CNN to process images. I'm looking to build a POC of a reverse image search trained with in-house product data. In the past I've used the imgSeek but it is dated and not using neural nets.

<comment><USER28> For a POC you could use Tagbox, just a REST API packed as a Docker container from [https://machinebox.io](https://machinebox.io), here the blog post [https://blog.machinebox.io/visual-search-by-machine-box- eb30...](https://blog.machinebox.io/visual-search-by-machine-box-eb30062d8abe)

Disclaimer: I did both of them so, I'm a little bias :)

</comment></comment><comment><USER29> We provide similar technology as a service if anyone is interested to hire us. Check our demo at: [https://app.turingiq.com/demo/image/client](https://app.turingiq.com/demo/image/client)

</comment><comment><USER30> Surprisingly, it works well. I tried searching for few items and it showed similar items on ebay..

</comment><comment><USER31> Shameless plug for my friend's startup, which has prototyped the concept and got a few customers for the last year.

[https://www.mirrorthatlook.com/](https://www.mirrorthatlook.com/)

</comment><comment><USER32> It would be interesting if they'd use this tech to deduplicate the gazillion listings for many cheap (chinese) goods which seem to use common set of images with maybe varying watermarks etc.

</comment><comment><USER33> Just adding - AliExpress has had similar feature for more than a year when I started using it. It doesn't work really well though.

</comment><comment><USER34> As I understood, there are 2 different apps: "Find it" and "Image search". If yes, .. why?

</comment><comment><USER35> Happen to know that is the work of the eBay team in New York. Really great engineers in that lab.

</comment><comment><USER36> I wonder if I snap a fake rolex it will find an actual fake rolex.

</comment><comment><USER37> Someone's been watching Big Bang Theory reruns.

</comment><comment><USER38> Pintrest should've hit this first.

<comment><USER15> Pinterest does have something similar and has a few blog posts about how they did it

</comment></comment><comment><USER39> How can I use this from desktop?</comment>


Three Simple Rules Poor Teens Should Follow to Join the Middle Class (2013) - <AUTHOR> https://www.brookings.edu/opinions/three-simple-rules-poor-teens-should-follow-to-join-the-middle-class/ <comment><AUTHOR> The rules:

\- finish high school \- get a full-time job \- wait until age 21 to get married and have children.

"Our research shows that of American adults who followed these three simple rules, only about 2 percent are in poverty and nearly 75 percent have joined the middle class (defined as earning around $55,000 or more per year). There are surely influences other than these principles at play, but following them guides a young adult away from poverty and toward the middle class."

<comment><USER2> Ironic, then, that poverty greatly hampers the ability to follow nos. 1 and 2.

Full-time employment increasingly requires a college education, which not only requires high school, but is increasingly expensive as well. I've known several promising people who, rather than undertake loans, simply dropped out and took a few part-time jobs to help their families with immediate needs.

Not to say that the rules can't be followed or don't help, but poverty is a trap and it's not correct to invert these rules - that is, to say "you're struggling so clearly you didn't do the right things" is wrong.

</comment></comment><comment><USER3> YouTube series _Some More News_ briefly addressed this study in their episode "Perhaps Ben Shapiro Shouldn't Be Taken Seriously By Anyone About Anything".[0]

Brookings also followed this up with a report entitled "Following the success sequence? Success is more likely if you're white."[1]

Note that I don't mean to make any claims as to OP's political outlook or motives in sharing this article.

[0] [https://youtu.be/aDMjgOYOcDw?t=594](https://youtu.be/aDMjgOYOcDw?t=594)

[1] [https://www.brookings.edu/research/following-the-success- seq...](https://www.brookings.edu/research/following-the-success-sequence- success-is-more-likely-if-youre-white/)

<comment><AUTHOR> OP here. I found the article via a link elsewhere but checked on Brookings' reputation before submitting. From my (granted non-American) perspective and based on various links and citations on their Wikipedia page, they seemed about as centrist as you might be for a think tank.

[https://en.wikipedia.org/wiki/Brookings_Institution](https://en.wikipedia.org/wiki/Brookings_Institution)

My own political outlook is largely libertarian but with a predilection for voting for whatever local politician brings the Tour de France within spitting distance of my apartment. ;)

</comment></comment><comment><USER4> So... what are the three rules? The author mentions three 'stark realities'\--poor children don't get support, enter kindergarten behind their peers and make bad decisions. But those aren't rules. Am I missing something?

</comment><comment><USER5> As a conservative, I have to say, this is extremely reasonable advice coming from the left-of-center Brookings Institution.

As someone in his sixties, however, I would like to add that as much as we would like to believe in nurture over nature, it does seem that the biggest difference between those who thrive and those who don't is that some seem to lack the drive, the initiative to make things happen in their lives. If it is in your personality to be ambitious and optimistic, probably nothing can hold you back. For people with naturally passive personalities, you had better have been born with lots of external support.

Life isn't fair, but as a conservative I am not in favor of enforcing equal outcomes for everyone. But just about everybody who "made it" in their chosen life's work can think of several people who reached out to give them a chance. Be that person if you can.

<comment><USER6> Maybe poverty and the stress of living in poverty has an effect on with drive and initiative.

<comment><USER5> No doubt.</comment></comment></comment>


ASK HN: Review our new product: WordPress Module: Security, Auditing and Backup - <AUTHOR> http://sucuri.net/wordpress-security-monitoring

<comment><AUTHOR> I would love some feedback on a new product we just released to the public.

It is a security plugin that will add some things that are missing on WordPress. Including:

-Full auditing trail (so you can track everything that happens, including logins, logouts, new users, new posts, posts being modified, etc, etc)

-Active response - Based on the audit trail, it will block certain behaviours (multiple failed logins, spam bots, attacks against known WP vulns, etc)

-Backup - Daily backup of your site (DB + files)

-1 click hardening - A quick option to run some best-security-practices hardening into WP.

Link: <http://sucuri.net/wordpress-security-monitoring>

Anyone can download it from here (only parts of the plugin that are paid - auditing + backup), the rest is free: <https://wordpress.sucuri.net/wp-sucuri- plugin.zip>

Any feedback on the landing page, plugin itself, screenshots, etc are more than welcome.

thanks,

</comment><comment><USER2> Much needed for WP :) But I am biased, since I helped with this work...</comment>


Q&A with Professor Hal Abelson of MIT - <AUTHOR> http://research.google.com/university/relations/visiting-faculty/professor-hal-abelson.html

<comment><USER2> "I had students two years ago who did a project which is way more obvious now than it was two years ago; it turns out you can tell people are gay by analyzing their social networks."

This is one of the reasons I use social networking so sparingly. It seems like it would be fairly easy to do analysis based on a large stream of social networking information and use that to determine traits of a person that they did not intend to share. I'm sure that a lot of it is used for targeted purposes, like advertising, but it can also be dangerous. Maybe I'm just paranoid.

</comment><comment><USER3> Earlier he was using Kawa Scheme on Android at Google Research. I wonder if the educational program will use it.

<comment><USER4> I hope I can say this...but the answer is yes! internally at least.

<comment><USER5> You can!

</comment></comment></comment><comment><USER6> I think if Google got Android into the schools it might even make CS more attractive for women. If it meant you could build a cool app for your friends.

" What we're working on, having passed the gauntlet of Google's trademark lawyers, is called App Inventor for Android, and the idea is to make it easy for beginning computer science students, non-computer science students, maybe eventually high school students, to build mobile apps."

</comment><comment><USER7> <http://research.google.com> is new for me; thanks for introducing me to it

</comment><comment><USER8> When I read interviews like this, it makes me realize just how cool it must be to work at Google.

<comment><USER9> Depends on your status, methinks. The guys in Adwords customer support aren't having so much fun.

</comment></comment><comment><USER10> when i read this, i involuntarily started every one of his answers with a haughty HA!</comment>


A gentle introduction to pointers using C (2019) - <AUTHOR> https://www.brainstobytes.com/a-gentle-introduction-to-pointers-in-c/ <comment><USER2> A meta comment on tutorials to teach pointers:

If pointers are so "simple", why are they so confusing?!? Because pointers are one of those concepts that's easier to _use_ than to _explain_.

Some concepts like _pointers_ (and also _monads_ and also English words like _" a"_ and _" the"_) are easier to teach by _examples_ instead of by _definitions_.

E.g. Consider the English words "a" and "the": a 5-year old can fluidly _use_ those connecting words when talking but a 50-year old adult that's been speaking English for _decades_ will have a tough time _defining what the words "a" and "the" _mean__ without giving a confusing and circular explanation. Attempted explanation: the "a" is the indefinite or general signifier while "the" is the definite or specific signifier -- what's the problem?[1]

How did small children _learn_ how "a" and "the" work without being given a formal definition? Because of numerous _examples_.

By working through enough examples, the lightbulb moment happens ("oh gee, is that all pointers is?!?") and the learner is able to derive his own definition. But the irony is that the learner now has _The Curse of Knowledge_ [2] and he/she now thinks he can write a "better" tutorial to explain pointers but new learners reading that tutorial will still say _" I don't understand your explanation at all."_

[1] riff on : [https://stackoverflow.com/questions/3870088/a-monad-is- just-...](https://stackoverflow.com/questions/3870088/a-monad-is-just-a- monoid-in-the-category-of-endofunctors-whats-the-problem)

[2] [https://en.wikipedia.org/wiki/Curse_of_knowledge](https://en.wikipedia.org/wiki/Curse_of_knowledge)

<comment><USER3> > pointers are one of those concepts that's easier to use than to explain.

My experience is the exact opposite.

A pointer is just an address for something. There is a "reference" operator to get a variable's address (pointer) and a "dereference" operator to get the data at a given address. That's it.

The hard part for me has always been using them correctly - keeping track of what points to where in my head, especially for void pointers. I haven't done any significant development in C since my early programming years, but that's the part I remember struggling with.

They're sort of like GOTO in that they're easy for me to understand, but they can quickly make my code hard to follow.

<comment><USER4> But then you need to explain to a newcomer that you can have `int __ptr;` (pointers to pointers)... It 's pretty odd for someone coming from say, Python, Java, or JavaScript. (which, many go through years before realizing that pass by reference happens in their language too)

<comment><USER5> > you need to explain to a newcomer that you can have `int ptr;` (pointers to > pointers)...

I think HN swallowed the asterisks there. I presume you meant:

        int **ptr;

> It's pretty odd for someone coming from say, Python, Java, or JavaScript. > (which, many go through years before realizing that pass by reference > happens in their language too)

That doesn't sound right. Java always passes by value. [https://stackoverflow.com/a/40499/](https://stackoverflow.com/a/40499/)

<comment> jagged-chisel > For objects, the pass by value is the value of the reference to the object.

That's exactly what "pass by reference" means. According the SO definition, C is always pass by value: the value of the 'primitive' (int, long, char) or the value of the pointer (an address.) It pretty much applies to any language: ultimately you're passing a _value_ from the caller to the callee. Whether that value is 'raw' (say, an int) or a pointer or reference is all in how you (or your compiler, or your runtime) interpret that _value_ you received.

<comment><USER5> > That's exactly what "pass by reference" means

No, that's wrong. Pass-by-value with references as values, is not the same as pass-by-reference.

Plenty of people make the mistake of equating the two. They can get away with the misconception as long as they don't encounter a language like C# that features both reference types and optional pass-by-reference.

> According the SO definition, C is always pass by value

That's correct. C is always pass-by-value. You can get a similar effect to pass-by-reference, by passing a pointer.

> It pretty much applies to any language: ultimately you're passing a value > from the caller to the callee.

That's incorrect. If the language has pass-by-reference semantics, then it does not copy the value, it passes the callee a reference to the variable. It's not true of pass-by-name, either, but don't know much about that strategy.

A good article on this topic is the documentation on C#'s _ref_ keyword, which enables pass-by-reference. The article does a good job explaining the related concepts. [0]

The acid test for whether pass-by-reference is happening, rather than pass-by- value, is whether the asserts can fail in code along the lines of this Java code:

        int myInt = 42;
        doStuff(myInt);
        assert(42 == myInt); // Ok

        String myString = null;
        doMoreStuff(myString);
        assert(null == myString); // Ok

In Java, the asserts will never fail. The two methods called do not have access to either the _myInt_ local or the _myString_ local, they only have access to copies of the values held in the two locals. This is the reason Java would permit us to use the _final_ modifier when declaring the two locals, without needing to know anything about either of the called methods.

On the other hand, C# permits pass-by-reference. Here we'll suppose that both _doStuff_ and _doMoreStuff_ declared their parameters using the _ref_ keyword:

        int myInt = 42;
        doStuff(ref myInt); // If callee uses 'ref' keyword, caller must use it too
        assert(42 == myInt); // May fail

        string myString = null;
        doMoreStuff(ref myString);
        assert(null == myString); // May fail

In this case, either assert may fail, depending on the behaviour of _doStuff_ and _doMoreStuff_. The callees are given references to our locals, and so they may mutate our locals (as well as reading their values of course).

Related to this, this is illegal in C#:

        doStuff(ref 42); // Compile error. 42 is not a variable!

[0] [https://docs.microsoft.com/en-us/dotnet/csharp/language- refe...](https://docs.microsoft.com/en-us/dotnet/csharp/language- reference/keywords/ref)

<comment> jagged-chisel > ...it passes the callee a reference to the variable.

And that reference itself is a value-- it's a thing passed to the function and copied from somewhere in memory to a register. It's the interpretation of this value that makes it a reference to some other tangible item. Further, it's the implementation details behind the syntactic sugar, compiler optimizations, and runtime support that allow the developer to reason abstractly with references.

Your Java example with String overlooks the fact that String is immutable: the object itself can't be modified. Use any other object type, allow doMotreStuff() to operate on the argument's properties, and you get the same effect as if passing by reference.

My point is that the SO explanation isn't a good one precisely because you get this pass-by-reference behavior in Java. Yes, indeed, technically there's this _value_ that gets passed to a function, and it's the value of the reference. And, yes, everything that "passes by reference" has to pass _something_ and that something is the "value of the reference" so everything is technically, at the bottom of the abstract stack, "pass by value." But that's not helpful when attempting to help the newb understand why his object mutated in a method when "pass by value" is what he expected.

<comment><USER6> "call-by-reference" already has a technical meaning, and it is widely- misunderstood because "reference" is an overloaded term.

Variables are _cells_ that contain values. A better name for "call-by- reference" semantics is "call-by-variable", because the variable -- the value- containing _cell_ \-- is what is semantically received by the called function. Moreover, it doesn't matter how this is actually implemented, so long as a caller may effectively _rebind_ a caller's variable.

In Java, variables may contain either "reference values" or "primitive values". It is never possible, in Java, to rebind a caller's variable. (Consider that there is never a call site that could be rendered illegal by making a non-final argument final.) We should not call it "pass-by-reference", not only because it's incorrect but because, to your point, it's entirely misleading when "reference" already has a distinct meaning in Java.

Pointers are strictly more powerful in a language than pass-by-reference. If you have pointers and pass-by-value, you can emulate pass-by-reference by providing a pointer _value_ that points to a variable _cell_. So if you have pointers already, you don't strictly need call-by-reference.

C# doesn't have pointers in the managed subset, because raw pointers can't be managed. But call-by-reference can be added relatively easily, so you can recover a small amount of what could be done if you had pointers. That's the purpose of the `ref` keyword.

Java doesn't have pointers, either, but it also doesn't have call-by- reference. That makes it impossible to write the canonical example of call-by- reference, `int a = 1; int b = 2; swap(a, b)`. Instead, you must reify the concept of a cell in the first place, e.g. by wrapping each value in a one- element array, and pass those cells to `swap` instead.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> This article and most of the comments here are just over complicating a straightforward idea. I simply don't get why people always think "pointers" are difficult to understand; it is manipulating them to do advanced stuff that gets tricky. But nevertheless it gets easier with time and experience since you start picking up on the common idioms.

For anybody beginning C, my advice is to not make an "imaginary dragon" out of "pointers" but just study examples and write your own programs using them and soon they will seem "obvious".

<comment> globular-toast I agree. I really can't understand what is complicated about them. I find the same with programmers who find the DAG in git difficult to grasp. It's just linking stuff together. It's how the the web works. It's how linked lists work. What's difficult about it.

I think exposure to assembly demystifies all of this stuff about C. Most of C is just making stuff really convenient that you would do anyway in assembly. A pointer is just a memory address, but in C it also has a type. The only thing the type is for is so the compiler knows how many bytes to add/subtract when you increment/decrement the pointer. That's it. In assembly you'd just have to remember how many bytes to increment/decrement yourself.

Reading Knuth also helps with this. It's one of the reasons he uses a synthetic assembly language rather than a high-level language. There are no pointers. There is just memory, and you need some way to find stuff in memory, otherwise it is just noise.

<comment><USER8> IMO I don't think the problem in C is pointers, it's that pointers _are_ arrays, in the sense that they are syntactic sugar. This is incredibly confusing, and probably part of the reason why most modern systems languages wisely make arrays _more_ e.g. {pointer, length}, sometimes introduce slices, where length is runtime, and make you do some syntactic work to pull out the pointer from the array.

<comment><USER9> No, pointers are not arrays. Pointers can point to arrays and, because of the language rules, you can do pretty much anything you can do to an array through pointers, but they are definitively not arrays.

<comment><USER8> I think we see the problem here. I've been using C on and off (granted, never formally educated in it) for 20 years now and I don't know the difference.

<comment><USER9> An array is a variable that is declared to contain sequential data (using square brackets and the desired number of elements). It is immutable. A pointer is a mutable variable that can be made to point to anything, and it can be used to reference an array or any of its elements. These are different concepts, although the language allows a lot of operations in common between them.

<comment><USER8> I see. I mean, yes I obviously know the difference between pointers and arrays as general CS concepts (and articulated how other languages get it correct), but we're not talking about CS, we're talking about C. Perhaps I should have said in my original context, "array _syntax_ " is the same as "pointer _syntax_ ". But while the CS concepts of "array" and "pointer" are indeed different, the C syntax _elides_ the two. C does not track the immutable length of the array for you.

<comment> globular-toast I think what you're getting at is that arrays are not really first-class objects in C. Rather it just gives you a bit of syntactic sugar to do the same thing that you could do with pointers anyway. But it's up to the programmer to complete the abstraction.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> > Pointers are variables that hold a reference to something in memory.

This sentence is extremely confusing or meaningless. I would rather say: "A pointer is an integer that indicates a position in the RAM."

Thus, if a pointer "p" indicates a position in memory, then "p+1" is a pointer that indicates the next position in the memory. The value on that cell of memory is often written as p[1].

<comment><USER11> > A pointer is an integer that indicates a position in

RAM, ROM, NVRAM, flash, etc.

A pointer stores a number that represents an address, without making reference where is located. Can be 0 (NULL).

"p+1" can be the next byte in RAM, or the next 4KB if p is of type "4KB struct".

<comment><USER12> > A pointer stores a number

I disagree with this, and I think it's important to understanding pointers.

A pointer _is_ a number. A variable stores a value.[1] A variable has an address, a value does not; a value might be stored at an address, but it doesn't intrinsically _have_ an address. A person might live in a building, but the building has the address, not the person, who can move.

[1] (In C-like languages this is true. In Python, a variable stores something you're largely not allowed to inspect, which might be a pointer to a data structure with data and metadata. Haskell has no variables, just names for constants.)

<comment><USER11> If I put you in front of a wall full of numbers, and I tell you "point a number with your finger and remember it", what is your finger? A number? Or the thing you use to point a number?

</comment></comment></comment></comment><comment><USER13> Seeing _why_ pointers are used is especially important. Understanding that functions in C pass by value and what that means gives context to pointers as a useful tool instead of a longer and more complicated way to do things. Students become a lot more frustrated if they don't see a reason behind learning something, especially if there's any sense of elitism going on.

Additionally, a lot of these terms like that pointers now introduce (address, reference, etc) are being introduced without any sort of context, usually they're brand new to students. If you've never seen (and may never see) assembly then you miss out on a lot of what's going on at the beginning and eventually when you get it, as well. It's an abstracted view of what's going on.

</comment><comment><USER14> I learned pointers by drawing boxes and arrows which I think is the best high- level explanation. Basically a variable is a container which holds a value from a predetermined set of values (also known as a type). For instance, an integer variable x holding the value 37 can be visualized as a box with the name x:

      x
     +--+
     |37|
     +--+

A pointer variable p can be visualized as a box which "contains" an arrow pointing to another variable of a certain type, for instance

      p       x
     +--+    +--+
     | -|--->|37|
     +--+    +--+

which correspond to the C declarations

        int x = 37;
        int *p = &x;

<comment><USER15> And that's the way they're shown in K&R2

</comment></comment><comment><USER16> Coming back to C after several years, I now think one of the hard things about pointers in C is the C syntax for pointers, not just the idea of pointers in general. Eg, sometimes the type declaration wraps around both sides of the variable name:

        int a[2];
        int (*ap)[2] = &a; // pointer to array of int

It makes it harder to see what's what.

<comment><USER17> I agree that it is confusing. Here [1] is something that can help new C programmers to understand these kind of declarations. It gets easier after a while.

[1]. [http://c-faq.com/decl/spiral.anderson.html](http://c-faq.com/decl/spiral.anderson.html)

</comment></comment><comment><USER18> I feel like spending some time programming in assembler, even just Z80 or 6502, would give most people an understanding of pointers in very short order.

<comment><USER19> True, but you do not need to know assembler to have a general idea of what computer memory looks like; but more importantly, C's notion of a pointer is more complex as it is tied to the data type and its 'sizeof' and is further complicated by association with arrays.

</comment></comment><comment> prosaic-hacker Not a simple general intro to pointers A useful gentle intro to the C type feature of pointers to arrays.

The C language has more than one way of accessing arrays. Other languages don't.

In the classic 80s teaching language Pascal if you started with pointers it was because you where working with linked lists.

Most people I have taught programming to conflated the pointer access in C and the linked list concepts to their disadvantage.

I taught linked lists first and left the point abstraction intact. Later I crossed the the layer of abstraction to explain memory access details. I have had more success (less learning difficulties) this way.

<comment><USER20> > The C language has more than one way of accessing arrays. Other languages > don't.

Like this?

        procedure ArrayDemo;
        var
            data : array [1..20] of Integer;
            ptr : ^Integer;
            i : Integer;

        begin
            ptr := @data[low(data)];
            for i:=low(data) to high(data) do
            begin
                WriteLn(ptr^);
                ptr := ptr + Sizeof(Integer)
            end
        end;

</comment></comment><comment><USER21> I remember pointers being very hard to grok when I was first learning about them. And then once I was finally starting to understand, I saw a char __and my head exploded. It took some time & practice for it to become second nature, but eventually it did.

It's interesting how most languages have this concept, even if they don't call it "pointers". In some ways that reduces complexity and learning curve, but in other ways it leaves invisible footguns lying around.

JavaScript comes to mind as one where references and referential equality are very important to understand, especially in modern frontend dev with its focus on immutability, but there's no syntax like C's `*` and `&` to lend clues about what's going on. I ended up writing a visual guide to "pointers for JS devs" [0] to help explain the concept, because I think it's really easy to get bitten by reference bugs when your mental model doesn't match what's happening.

[0]: [https://daveceddia.com/javascript- references/](https://daveceddia.com/javascript-references/)

</comment><comment><USER22> I wish someone would do a gentle introduction of header files. I sort of understand pointers, at least on a basic level. Headers files are apparently much easier, so they get the briefest of introduction in most "Learn C" tutorials and books, but I don't understand how they work or how to use them.

<comment><USER23> Imagine you have a function f in f.c:

        void f() {
            puts("hello, world!");
        }

and you want to call it in main.c:

        int main() {
            f();
        }

this is an error because the compiler treats each .c file separately, so when it's compiling main.c, it isn't looking at f.c, so it has no way to know the function f exists. So you need to "declare" it at the top of main.c. A declaration is similar to a definition, except it has no code, and just tells the compiler that the function exists:

        void f();

But now imagine f.c had many functions. It would be extremely cumbersome to repeat all those declarations in every file that wants to use them. So you can collect all the declarations in some file f.h, and include it from main.c or anywhere else you want to use functions from f.c. When the compiler sees #include "f.h", it operates as if the contents of f.h were copied and pasted at that point.

You might ask "why not just include f.c from main.c instead?" Because then you would have two _definitions_ of f in your program (one in f.c and one in main.c), which is illegal, as opposed to two declarations, which is fine.

<comment><USER24> > this is an error because the compiler treats each .c file separately, so > when it's compiling main.c, it isn't looking at f.c, so it has no way to > know the function f exists.

But it's not an error and the compiler does not need to know the function exists. If there's no declaration, then the function's type is implicit and simply assumed to exist.

If the function doesn't actually exist, then it's your linker that will complain.

<comment><USER23> You're right (even with `gcc --std=c99 --pedantic`, which surprised me!) I should have said that the compiler won't be able to type-check the arguments and return value.

</comment></comment></comment></comment><comment><USER25> What I find most annoying about tutorials on pointers (as someone relatively new to C picking it up for work and fun) and memory management is that every single one of them seems to discuss the trivial case of a hard coded char or int array.

Which like, is great for the absolute basics, but not for just about every serious piece of software where in practice what you need is to build stuff like e.g. dynamic accumulators of strings -- and virtually no tutorials or books I've read talk through best practices for doing this.

So I've sort of kludged together my own probably wrong set of patterns for this. Reading the source code of others on github helps a bit but I have yet to find a good example of how to e.g. read strings from say a CSV file and parse it into an array of arrays of strings

Does anyone have any pointers here? (sorry)

<comment><USER26> Perhaps that's because C pointers are usually (always?) exactly that - a hard coded char array, which starts at memory offset 0.

Depending on the circumstances, it may be easier to think about it as int array, or some other type, but the easiest on-ramp for pointers is still char[] = 0x0.

</comment></comment><comment><USER27> I think people get scared of pointers because the numbers are big and are in hexadecimal. But they don't have to be. I didn't understand pointers until I thought of them as house numbers in postal addresses. In a world where everyone lives on the same street, you can send anyone a letter by mailing a letter with their house number written on the envelope. You don't even need to know their name -- thus SPAM mail is born. The same analogy holds for phone numbers. The pointer is like a phone number or house number. The person on the other end is the value you get when you dereference a pointer.

</comment><comment><USER28> The point where I actually understood everything about pointers was when i realized that they just hold memory address. After that everything like pointer arithmetic and dereferencing pointers become clear as day.

<comment><USER19> It is easy to believe that you understand something while in reality things are more complicated. What is an address? Is it like an integer value? (Is it really? Or can it be converted to one? Even if so, how many bits wide is it?) Or, if not, what is its structure? Back in the day, a question would be is it a 'far' or a 'near' pointer or we are living in a flat address space? Also, how is pointer arithmetic affected by alignment?

<comment><USER29> Yeah, you may use a pointer in C, but it will compile to an instruction that fetches some data from a location in memory relative to the currently executed instruction.

</comment></comment></comment><comment><USER30> I recommend anyone saying that pointers are simple, just integer memory locations, to read this article: [https://www.ralfj.de/blog/2018/07/24/pointers-and- bytes.html](https://www.ralfj.de/blog/2018/07/24/pointers-and-bytes.html)

The fact that in C pointers point to specifically objects and not just simply to a random place in memory makes all the difference and is source of oh so many bugs

</comment><comment><USER31> Here's my explanation.

A pointer stores a memory address.

But I hear you say: can't an int or long also do that? All you need is the number of the memory address.

Well, the magical thing about a pointer is that it can use a special operator!

What? (I hear you say)

Yes!

Here is what it does: it allows you to obtain the value that is stored at that memory address.

What does a pointer store again? (a memory address)

But it can also obtain a value at that memory address.

You can't do that with an int, I've tried [0].

We call this operator the dereference operator [1].

It looks like a star

*

That's the one. That's the dereference operator.

One of the confusing things is that when you declare a pointer, you also use a *

Example

      int *a;

      //or

      int * a;

      //or

      int* a;

In those 3 examples, the * is not a dereference operator. It simply indicates that it is a pointer. In this case we have an int pointer.

I always like to view it as: we have an int! And this is how many times we're allowed to use the dereference operator: one time

How many times are you allowed to use it on this pointer?

      int *****super_pointer

That's right, five times!

So when you have

      int val = 10;

      int *a = val;

Then what is the value of a? It's 10. What do pointers store? Addresses. So what address is this? Address 10!

But when it comes to memory addresses in computers, there is a tricky part to it. In normal operating systems, low addresses are reserved for special things, I think it's the kernel in this case. So if you dereference that particular address, the OS/runtime environment won't allow you to. They will give you an error called a segmentation fault.

So you need to get a valid address of the variable val, but how do you do that? We have another operator for that, which I like to call the address-of operator.

So when you have

int val = 10;

int *a = &val; // & is the address-of operator

Then the variable a has the valid memory address we want!

There's much more to learn, but I'll leave it at this for now.

\---

I recommend people who start learning pointers/programming to use GDB with the TUI graphical interface. One could also print the values, but it's easier to see what really happens. For that there are two steps you need to do:

\- Watch a GDB with TUI tutorial: [https://www.youtube.com/watch?v=AxsSXMPD3N4](https://www.youtube.com/watch?v=AxsSXMPD3N4)

\- And then also display all the cpu registers. The reason for that: your variable values will be visible in there (in hexadecimal).

All the GDB commands: [https://sourceware.org/gdb/onlinedocs/gdb/TUI- Commands.html](https://sourceware.org/gdb/onlinedocs/gdb/TUI-Commands.html) (look for the regs option)

[0] [https://stackoverflow.com/questions/35379191/cannot-cast- var...](https://stackoverflow.com/questions/35379191/cannot-cast-variable- type-to-its-pointer-type-e-g-int-to-int-i-guess-its-a)

[1] Side note: I used to have trouble with the word dereferencing. If you have to, here is why it's called that way: [https://stackoverflow.com/questions/35372786/what-does- the-d...](https://stackoverflow.com/questions/35372786/what-does-the-de- prefix-in-dereference-mean-is-there-a-linguistic-explanation)

<comment> andrew-nguyen I'd rethink the this part of the explanation:

 _And this is how many times we 're allowed to use the dereference operator: one time_

I can see what you meant, but I initially thought this was trying to say that a pointer declared with a single asterisk could only be dereferenced once and never dereferenced again afterwards (as if there was a "one-time-use only" property to pointers). In the same vein, the "super_pointer" example could be misinterpreted like "Oh, so if I want a pointer that will let me dereference it on 5 different occasions, then I'll need to declare it with 5 asterisks?" I think a better way would be to replace the "how many times we're allowed to use the dereference operator" explanation and start with explaining the idea of a double pointer

      int **doublePtr

in depth first. With a solid understanding of a double pointer (an address that points to a memory location that contains an address that points to another memory location), making the mental jump to understand

      int *****super_pointer

should be easier.

Also, I think low addresses aren't used for various reasons, but I believe the kernel is stored in the top, high-address region of the address space. Otherwise, I think this is a really solid explanation of pointers!

<comment><USER31> > I can see what you meant, but I initially thought this was trying to say > that a pointer declared with a single asterisk could only be dereferenced > once and never dereferenced again afterwards

Ah, haha, I see! You're right. This could've been me back in the day.

> Also, I think low addresses aren't used for various reasons, but I believe > the kernel is stored in the top, high-address region of the address space.

Ah, yea, I was foggy on this.

For what reasons aren't they used then?

</comment></comment></comment><comment><USER32> Ted Jensen's tutorial helped me a lot back in the day: [https://pdos.csail.mit.edu/6.828/2014/readings/pointers.pdf](https://pdos.csail.mit.edu/6.828/2014/readings/pointers.pdf)

</comment><comment><USER33> as a C noob, this example is humorously confusing...

declare an array, fair enough, makes sense.

Declare a char *pointer,(a "reference to something in memory") ok if you say so.

Assign the char pointer the ARRAY name.... erm, ok.. so ... that would make the char pointer the 'first char in the array' then?

lol no you silly old duffer, it sets the char pointer to the memory address of the first char in the array.. not the contents of that location, obviously ;)

<comment><USER11> > that would make the char pointer the 'first char in the array' then?

That would make the char pointer _to point to_ the 'first char in the array'.

<comment><USER34> And why is that? I am a certified C noob and this really confuses me. Why `alphabet_pointer = alphabet;` points exactly to the first element in the array? Why not the whole array? Why not the last? Can we have a pointer that accepts a whole array?

Thanks

<comment><USER35> a c array is literally just a sequence of objects in memory (possibly with padding, but you can ignore that for a while). you can think of pointing "exactly" to the first element as being equivalent to pointing to the whole array. you need to know where the array begins and its size to do anything with it. but once you know where it starts, you can access the next element by adding sizeof(char) (or whatever the element type happens to be) to the pointer.

<comment><USER36> > possibly with padding

What padding?

<comment><USER11> If you declare a structure of unaligned size, the compiler might (probably must) introduce some padding bytes at the end of the structure to fit the alignment. In some architectures you cannot do unaligned accesses (ARM for example).

Unless you _pack_ the structure with the alignment you want.

<comment><USER36> Yes, so the structure itself has padding (added by the compiler). But that is regardless of whether it is in an array or not. OP seemed to suggest that C arrays introduce padding of some sort, which they do not.

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> A pointer monad, anyone?

</comment><comment><USER37> Is adding "(2019)" really necessary? Are the contents no longer valid or only meaningful when set in its historical context?

<comment><USER38> It's long been the convention here, and it's useful for people to know how recently it's been published.

With an article title like this, it could easily be anything from 1 to 30 or more years old, and showing the date makes it easier for readers to know if they've already seen it before clicking through.</comment></comment>


Ask HN: Startup jobs site - <AUTHOR>

I'm graduating from college this year and the job search has begun. One of my main criteria is a relatively small company, preferably a mature startup. Is there somewhere outside of HN that I can find these postings without having to look up every single company that posts jobs to see how big their company is and how old they are? <comment><USER2> This really depends on where you live!

For startup hubs such as San Francisco, New York etc, I think the best job boards are [https://jobs.github.com/](https://jobs.github.com/), [http://careers.stackoverflow.com/jobs/](http://careers.stackoverflow.com/jobs/), [http://www.authenticjobs.com/](http://www.authenticjobs.com/), and [https://angel.co/jobs](https://angel.co/jobs)

Angellist as mentioned on another comment is also great for just finding startups.

I also used to find really good city specific sites with a google search like "startups [CITY NAME]" or even "startup jobs [CITY NAME]"

for berlin for example one of the first hits is: [http://berlinstartupjobs.com/](http://berlinstartupjobs.com/)

Also, there is a huuuge list with job boards, currated by the guys from underdog.io -> [https://docs.google.com/spreadsheet/pub?key=0AlOCi7qqICWzdEh...](https://docs.google.com/spreadsheet/pub?key=0AlOCi7qqICWzdEhzX1V5RVJMb1BKVjB0bWFER3dCdFE&output=html)

if you are interested in remote jobs, and dont want to wade through all those job boards yourself, subscribe to [http://remoteworknewsletter.com](http://remoteworknewsletter.com) and set a couple of filters to only receive jobs that are relevant to you

</comment><comment><USER3> > One of my main criteria is a relatively small company, preferably a mature > startup.

Search the news archives for startups in the local business journal corresponding to the market where you're interested in living/working.

Go back to 2009-13, put together a hit-list of promising targets, see where they are today. In Dallas> [http://www.bizjournals.com/dallas/print- edition/2013/09/20/n...](http://www.bizjournals.com/dallas/print- edition/2013/09/20/north-texas-cities-are-trying-to.html?page=all)

*Pro-Tip: Reach out to the CEO directly, not all jobs are posted.

</comment><comment><USER4> One option might be to go to sites that only display startups, eg. CrunchBase (crunchbase.com), AngelList (angel.co/jobs), Startupers (startupers.com), StartupHire (startuphire.com), and just apply to or cold-email companies listed there.

Beyond that, feel free to e-mail me (my address is in my profile, slightly obfuscated)...I'm founding a startup that's aimed at people like you, and I'd love to get your feedback on a new product concept.

<comment><AUTHOR> Oh that's great information. I just got started on AngelList. I went to StartupHire briefly but haven't searched there yet as I'm still intrigued by AL. I'll surely be giving all those places a try, though.

</comment></comment><comment><USER5> often times startup ceos are accessible, so you might really dig into a company and its ceo, and write a few well thought out letters to the ones that resonate with you the most. be careful here - there's a ton of smoke and mirrors in this space and you could end up with a real egomaniac that's done better blowing smoke than delivering value. startup ceos want talent who will scale our their visions - the more that you can demonstrate that you just want to work hard learning how to solve the problems theyre working on, the more compelling you will be. good luck - theres a lot of opportunity out there.

</comment><comment><USER6> Hi! Hopefully this is cool/helpful:

[http://www.breakoutlist.com/](http://www.breakoutlist.com/)

would love your thoughts. email in profile.

</comment><comment><USER7> If your based in the UK you can try [http://workinstartups.com/](http://workinstartups.com/)</comment>