Edge Computing at Chick-fil-A - <AUTHOR> https://medium.com/@cfatechblog/edge-computing-at-chick-fil-a-7d67242675e2 <comment><USER5> I work in midtown NYC near a Chick-fil-a location. Used to be a line down the block, went with a friend yesterday thinking we caught the line at a short time but turns out they were using smiley employees with ipads coordinating orders and that they were serving arguably more people than before. Very impressed with how seamlessly they've integrated tech in their brick and mortars in contrast with the countless "cashless" restaurants that just add cruft, inefficiency, and customer resentment.

<comment><USER6> I have a friend who used to be a shift manager at Chick-fil-A. I can't remember exactly what he called that process... It was like meet and greet or walk and talk, but it's surprisingly efficient for the drive thru. He said it cuts down the average wait time for customers around 15% at his location.

I don't know what it would be like at NYC since it's not a drive thru.

<comment><USER7> It's crazy efficient. When you walk in someone with a tablet takes your order, and by the time you get to the cash register your food is ready. Works so well because the employees are able to catch customers at the end of the line.

<comment><USER8> Here's "Line busting" the drive thru 68+ cars deep. It's astonishing the WiFi goes out that far.

[https://www.youtube.com/watch?v=TIQWyYQ0EYg](https://www.youtube.com/watch?v=TIQWyYQ0EYg)

FWIW, also an edge approach, with Prometheus etc.

<comment><USER9> Slightly off-topic, but what's the reason that Americans all tend to line-up at the drive thru? I was in LA, there was a huge line at the drive thru, so we parked, went inside and got our orders really quickly.

<comment><USER10> The comfort of still being in your own space. For example, listening to your own music, no excursion for your kids to belabor, precisely don't have to park and get out.

<comment><USER11> Yeah, I think I would rather wait 10 minutes in my car than 5 minutes in line. I'm sure other people may feel differently. And something about the drive-thru still "feels" faster, even if it's not. I think it's just the idea that you get to pull out and be on your way after you've gotten your food.

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> If you want to see the actual hardware used at the restaurant, check out the previous blog post: [https://medium.com/@cfatechblog/bare- metal-k8s-clustering-at...](https://medium.com/@cfatechblog/bare- metal-k8s-clustering-at-chick-fil-a-scale-7b0607bd3541)

Spoiler: It's a stack of Intel NUCs.

<comment><USER13> NUCs are surprisingly well-built little machines. I have one in my car, and I give them my absolute seal of approval. I've heard of people running little VMware clusters on them too.

<comment><USER14> It's actually suprising that people think they're not "normal" computers. I provisioned a few of them and they're little beasts. Most in use are I7, 16-32gb ram, Dual NVME. The Skulltrail nucs are extremely powerfull and on the lower end the atom based units are solid too. I also ran a nuc in my car for a while, mostly stumbling radio / wifi spectrum as I drove around, but it was perfect with its relatively low power requirements.

<comment><USER15> This is Brian -- I wrote the article. Super cool use for a NUC! I'm going to follow the links and check out what you did some more.

</comment></comment></comment></comment><comment><USER16> On a related note, Chick-fil-A using Deep Learning models for object detection with MXNet to track how long fries have been waiting: [https://www.youtube.com/watch?v=3Uuq_cX8b1M](https://www.youtube.com/watch?v=3Uuq_cX8b1M)

<comment><USER17> This seems like such an over-engineered solution.

It would've been much simpler to have a plate with multiple pressure sensors and a multi-color LED per spot. When you set the fries down on the sensor it activates and the LED turns green. After X amount of time the LED can turn yellow, meaning that it's becoming stale. Finally, after X+N time has passed and it's no longer fresh it can turn red. Removing the fries turns it off. Aside from being way cheaper, I'd conjecture that this would be much more reliable. I'm pretty sure I could get a prototype of this up and running over the course of a weekend or two.

If you wanted to get really fancy I guess you could also track the room temperature and moisture levels and use that to get a better guess of how long a group of fries will remain fresh. Although I don't know if environmental factors like these have enough impact on fry freshness to be worth taking into consideration.

Anyway, it looks like they were just doing this for fun and learning, so I guess it doesn't really matter.

<comment><USER15> This was a good learning experience for our engineers at our Innovation Center at Georgia Tech in Atlanta. It may bear fruit in the form of a useful solution in the future, though. What is unique to Chick-fil-A is 'volume'. We do a lot of sales in our restaurants, so anything we can do to try and make our team members lives easier is important to us. We want them to enjoy their jobs and we want to do the best we can to consistently create high quality food experiences for our customers. Our teams in restaurants are the heroes, but we are trying to use technology to help them do what they do. <thumbs up>

</comment></comment></comment><comment><USER18> Would just like to say Chick-fil-A should publish more like this. They are absolutely knocking it out of the park with tech. My wife and I are friends with a few operators (they don't have "owners"). I recall them, several years back, trying to get me to quit consulting to come work with them and open more stores. "Food service, ugghhh, no offense" I said. A friend whipped out her phone and showed me their really great in-house app that showed detailed analytics for her store (everything you could imagine) and tools for for doing purchasing and much more we didn't have time to get into. This is circa 2010 if I recall. For reference the iPhone SDK dropped in late 2007 and these folks were already way onboard with mobile. I was impressed.

</comment><comment><USER19> Unrelated but I've done a good amount of nonprofit work and everytime I've had to ask for food donations chickfila has given food free of charge.

<comment><USER20> I see people like this the way I see my Mormon family. They usually try to be very kind and I value that.

At the same time, they harbor beliefs which are incredibly damaging to both me and those I love, so I keep a healthy distance while understanding that they mean well.

Don't mistake their kind acts for open-mindedness or acceptance, though.

<comment><USER21> Out of curiosity, in what way is Chick-fil-a not open-minded?

<comment><USER22> [https://www.forbes.com/sites/clareoconnor/2014/03/19/chick-f...](https://www.forbes.com/sites/clareoconnor/2014/03/19/chick- fil-a-ceo-cathy-gay-marriage-still-wrong-but-ill-shut-up-about-it-and-sell- chicken/#4532740d2fcb)

Anti gay marriage

<comment><USER21> I understand that you disagree with the CEO, but how do you get from there to the conclusion that he is not open-minded?

<comment><USER20> Because he claims that homosexual relationships do not deserve the same state recognition as heterosexual relationships.

<comment><USER21> Wait, so in order for a person to be open-minded, they have to not only honestly consider other ideas and perspectives (which is what being open- minded actually means) they also have to agree with your perspective?

A person's open-mindedness is independent of whether or not somebody else agrees with them. Otherwise, by definition nobody - including you - is open- minded, since there are people that disagree with you. It's completely nonsensical.

<comment><USER20> It's the fact that they are unable to honestly consider it because of their dogma rather than the fact that they don't agree.

<comment><USER21> I'm fascinated by your stance - genuinely fascinated, I'm not trolling you, I'm serious.

So to recap, the CEO of Chick-fil-a is part of a group of people ("they") who are actually _unable_ \- as in, incapable - of honestly considering certain viewpoints? How did you reach this conclusion, and how did you come to be so certain that this CEO is in that group?

Is your understanding that he gave no serious thought to his opinion (i.e. it was just imposed on him), or that he thought about it lightly but lacks critical thinking skills, or something else?

Is there any possibility that he simply has a different opinion than you, and that's all there is to it? Like, he educated himself like you did, thought about it like you did, etc. but just reached a different conclusion than you? Or is the only explanation that he is flawed/limited in some way?

Apologies in advance if anything I've said comes across as demeaning, I'm genuinely trying to understand. Thanks!

<comment><USER20> I was raised in a cult. I know what it's like to be incapable of considering others' views because of dogma because of my experience.

I am assuming that his Christian perspective is similar to the one I had. My experiences with Christians since have reinforced this expectation.

Does that help?

<comment><USER21> Ok, I appreciate your willingness to share.

FWIW it's virtually impossible to determine the open-mindedness of someone from afar, and to make assumptions about their _ability_ to even consider other ideas seems like a way to insulate oneself from difficult ideas - <USER2> declaring that someone is incapable of an honest evaluation of your perspective, it prevents your perspective from being scrutinized (which, ironically, could lead to being close-minded if you're not careful).

A parting thought: most people (a) can be reasoned with if not attacked - <USER3> can find common ground and have a good discussion with nearly anybody (b) on every "interesting" social/political issue there are intelligent and thoughtful and just really great people with nuanced and reasonable perspectives all across the spectrum - the extremist nutjobs on the fringes are wildly outnumbered by pretty normal people

So as a random dude on the internet, just a friendly warning that if the above strikes you as untrue (e.g. if you believe large chunks of people are intolerant or incapable of being reasoned with, or if you see a difficult social issue as pretty much black and white and can't understand why so many people don't get it), then there's a good chance that you've been duped and have been sucked into a form of modern tribalism.

Have a great day!

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER23> I don't know why, but for some reason the juxtaposition of traditional, brick- and-mortar brand images with tech blogs about building modern application technologies is amusing to me. It's seeing two completely different sides of the same organization. There's 0 condescending sentiment behind it, it's just like "wow, that was unexpected."

<comment><USER24> I worked for Nordstrom for several years and whenever I describe their technology department to another developer they're always pretty shocked. It was pretty shocking to me when I started there too.

<comment><USER25> I'm curious about Nordstrom, I know nothing of their infrastructure.

<comment><USER24> Their tech stack is similar to what you'd expect from any large tech company. I can't speak much to the server side of things (I worked on the iOS app) but I know they had everything ranging from older monolithic .NET apps to Node and Go micro services.

Their GitHub page ([https://github.com/Nordstrom](https://github.com/Nordstrom)) doesn't have much in the way of internal projects, but it has lots of forks that will give you an idea of some of the tools they used.

<comment><USER26> Nordstrom also does polymorphic React with server-side rendering via .NET [0]. I've always been impressed with this part of their stack.

[0] [https://libraries.io/github/Nordstrom/React.NET](https://libraries.io/github/Nordstrom/React.NET)

<comment><USER27> Latest commit e56c9be on 24 Apr 2015 - wonder if it's still in use internally?

</comment></comment></comment></comment></comment></comment><comment><USER28> I enjoyed reading this post, but came away with more questions.

They talk about providing a service for developers to deploy stuff easily, but I'm wondering how that works in practise.

I've very little experience with Kubernetes, the author mentions they run 'mini' kubernetes clusters at each restaurant. Does that mean they have to deploy software/container updates to each restaurant one by one? Or is it abstracted above that level, where they can see ALL of the restaurants as one "big" cluster?

<comment><USER29> Hey! Caleb here... I'm the lead SRE building the clusters on the restaurants.

Each individual restaurant gets it's own cluster (we aren't fully deployed yet). There are too many network latency challenges and too much immaturity around federated clusters to take the route (unfortunately).

We currently use gitops, which houses all the configs per cluster in a single repo per restaurant (CRAZY amount of corgis!)... we call that Atlas (the repos) and we made a little pod called Vessel that polls and applies configs in those repos.

We're almost done building something called Fleet that will generated and manage those repos (Atlas) at scale. Ie; a UI where we can say "send this version to 10% of restaurants" and it will regenerate and deploy those configs to all the appropriate Atlas repos, which will then get pulled down by Vessel.

We tried doing all this with Helm but failed miserably. Maybe it was us? But templating vs gitops... the choice seemed obvious.

<comment><USER30> What were the main issues you faced with Helm?

<comment><USER29> The primary challenge was just reasoning with the template and using Helm at scale... ie; what exactly did we deploy on those hundreds of varying clusters?

Other issues included; tiller would sometimes become unstable... version mismatch issues between helm local and roller... lack of a clear, outage free canary deployment... we even found cases where helm would not cleanup after itself during a deployment and retain previous config settings within k8s.

</comment></comment></comment></comment><comment><USER31> Sure, they have the biggest cloud in the US, but it's down on Sundays.

Kidding aside, do you guys do maintenance on Sundays or is that forbidden by company policy?

<comment><USER15> This is Brian -- I wrote the article. We are closed on Sunday but its not dogmatic in nature. We want to make sure that everyone at Chick-fil-A has a great work/life balance. Occasionally we need to do some work on a Sunday but we try to architect and engineer in a manner that supports business-hours- friendly deployments, or scheduled deployments over night to protect that. This is truly a first-class design consideration. In my 14+ years at Chick- fil-A, I have only worked a handful of Sundays for any reason. Its a tribute to leadership at all levels and a great company culture that values people (and their need for times to rest).

</comment></comment><comment><USER32> They call their "cloud": Cloud-Fil-A.

Par excellence.

<comment><USER29> We will have an order of magnitude more cloud datacenters than AWS! :)

<comment><USER32> Do you also have a CDN (Chicken Delivery Network)? (Stolen from a guy in a slack channel.)

<comment><USER33> Indeed they do... that's their core business.

</comment></comment></comment></comment><comment><USER34>
        Since we have a highly available Kubernetes
        cluster with data replicated across all
        nodes ..

This sounds like one of the more cutting-edge bare-metal kubernetes setups I've heard of!

Curious what the kubernetes implementation / configuration looks like for the setup they've described. Particularly in terms of HA-cluster k8s configuration, networking solutions integrated, and replicated storage.

Considering there is no AWS / GCE / Azure block-storage primitive to build on, it'd be cool to understand the choices and tradeoffs made in this scenario.

Is all data being stored on something like HDFS or CockroachDB? Or perhaps the k8s minion slave nodes launch containers with Ceph volumes.

<comment><USER29> Well, it's a 6 man team so most of our trade offs were for expedience, not the worlds greatest architecture. We cheated and sync'd data using a HA MongoDB setup amongst the cluster. RKE for K8s clustering was a life saver (nothing easier at this point on bare metal IMO), although RKE can be brittle at times.

</comment></comment><comment><USER35> Having worked with some of CFA's competition and comparing their IoT architectures, I'd say these guys have their heads on straight. Kudos to them.

<comment><USER36> I've never been to a retail establishment with anything close to the credit card speed of CFA. The cashier can swipe and grab receipt in one smooth and quick motion. It's so fast you feel like it's broken or slight of hand.

I wonder how they do it or why other companies can't or don't do it. CFA stores are so much busier than even an average McDonald's, maybe the efficiency gains are just worth more.

No idea how they handled the chip card shift.

<comment><USER35> From what I understand, merchants can do a quicker swipe (basically record the card # and transaction amount without waiting for authorization), but the merchant fees are slightly higher because of the possible higher fraud/chargeback rate. It's up to the chain to decide on what to do.

Perhaps CFA can do it even better since they could possibly keep a database of bad cards at the edge. Remember those books that cashiers used to thumb through to verify if a card was stolen?

<comment><USER37> Or keep a Bloom filter of hashes of repeat customers' cards on the edge to accelerate their experiences.

<comment><USER35> The goal is to detect stolen/deactivated cards. Not sure how that helps.

<comment><USER37> If it's a repeat customer, at the same location, then you can avoid contacting the payment processor gateway, and just record the transaction on the local cluster for later payment processing. The expected value of the loss if someone stole their card, and used it _at the same restaurant that they frequent as a regular_ , after the card was canceled, and the restaurant gave the thief a chicken sandwich, and now corporate needs to eat that cost because the asynchronous CC transaction doesn't go through when they finally run it... is absolutely miniscule.

</comment></comment></comment></comment></comment></comment><comment><USER38> _We think of our Edge Computing environment as a "micro private cloud"_

We call local clouds "fogs."

<comment><USER39> Aren't most edge computing reffered to as fog-computing? Or is it just in academia

<comment><USER15> Hey this is Brian -- I wrote the article. I have heard the term a lot over the years. I think you could call it that -- the idea being its the "foggy" area between the user and the cloud. The reason we like "micro private cloud" is that 1) its a tiny environment with limited cloud properties (no elastic scale) but 2) we do try and provide a platform with cloud-like services for our internal developers, and we like the cloud paradigms being applied to how we think about compute at the edge, in restaurants.

</comment></comment></comment><comment><USER40> This is a pretty cool example of cybernetics ala Stafford Beer - <USER4> information and reaction to streamline production. I'm excited to see this be brought in to more production (though maybe it has, and most places aren't as upfront as this). It's truly a shame the Chilean experiment in the early 70s didn't work out.

On a side note, it's interesting that Kubernetes is the word from which cybernetics is derived, meaning "governance".

</comment><comment><USER41> with articles like this and Mac'd from this YC cohort it seems like an increasingly interesting time in the restaurant/technology space.

<comment><USER42> the domain is interesting, but I worry about the long-term effects of maximizing efficiencies (read: automation) forpeople who rely on jobs in that industry.

<comment><USER43> A myth about manufacturing jobs is that they all went overseas. Reality is that most of them were replaced by robots and automation.

Low skill, low knowledge, repetitive work is key target for automation.

Fast food workers wouldn't have large downstream effects. Automated truck driving would though. There are likely entire towns right off of highways that depend on humans passing through daily - restaurants, repair shops, etc. If automated trucks don't stop anymore, a town could literally be a ghost town.

<comment><USER15> Hey this is Brian -- I wrote the article. I do want to add that Chick-fil-A isnt looking to replace jobs with "robots". We really want to assign our restaurant team members to tasks that add maximum value to customers, which are generally service oriented (iPad in Drive Thru, personal service in the dining room, etc). Our automation will help our team members be more effecient and enjoy their jobs more rather than replace them.

<comment><USER43> Of all places I doubt CFA would replace workers, having nice, helpful people is the second awesome-est thing there (chicken and waffle fries tops that).

I can see the automation and analytics helping staff appropriately and offer more options for efficiency and waste reduction.

</comment></comment></comment></comment></comment><comment><USER44> From other retail IT I've been involved with, these guys could make an incredibly successful side business doing this to pretty much any retailer.

However when I see that retail stores are using dialup to do batch uploads of inventory and transactions I wonder if their central IT systems would be remotely ready for something like this.

<comment><USER15> You hit on a great point -- Edge infrastructure _buffers_ outbound internet traffic but does not replace the need for a good circuit.

</comment></comment><comment><USER45> Pretty interesting, but good lord I've never seen such buzzword-density in an article that's making real sense

<comment><USER15> This is Brian and I wrote the article, and I am taking this as a massive compliment ;)

</comment></comment><comment><USER46> Interesting that they use Kubernetes exclusively for edge computing resources (as opposed to their cloud architecture). Never really considered this approach before.

<comment><USER29> If you decide to play around with it, we use RKE to build and manage the k8s environments locally.

</comment></comment><comment><USER47> When it comes to Chick-fil-A, there's also the regressive side in addition to the modern and traditional sides.

[https://hornet.com/stories/chick-fil-a-anti-lgbtq- donations-...](https://hornet.com/stories/chick-fil-a-anti-lgbtq-donations- two/)

<comment><USER48> Your comment is essentially "the owners hold certain values I disagree with regarding sexuality, therefore I'm surprised their technical team is any good."

I don't see how that follows.

<comment><USER49> Political values are important to point out. Companies create these tech blogs as a recruitment vehicle. You should know if the company you're considering applying to is funding bigotry (or "wholesome family values" if that's your angle).

<comment><USER50> > 'Bigotry'. _noun_ : intolerance towards those who hold different opinions > from oneself.

Surely castigating the owner of Chick-fil-A for advocating his personal beliefs is the truest example of bigotry here.

<comment><USER51> You should probably not rely on Google's fairly crappy dictionary definitions.

<comment><USER52> The quoted definition of _bigotry_ as "intolerance toward those who hold different opinions from oneself" is from the _New Oxford American Dictionary,_ from Oxford University Press (the OED people). If that's what Google is using, I don't think I can rely on your definition of "crappy."

<comment><USER51> Yep, it's a crappy definition that does not reflect current usage. It doesn't even reflect the definition in the OED, if you want to check there. 'Bigotry' is not limited to difference of opinion. Nobody uses it like that, you can take a look at the quoted usage examples here, which are from current publications.

[https://www.merriam-webster.com/dictionary/bigotry](https://www.merriam- webster.com/dictionary/bigotry)

<comment><USER53> Ahh, yes. Always useful to continuously cherry-pick or alter definitions to tailor words that have an objectively negative moral implication to apply to exactly the standards you want.

Just because some publications use the word in that way does not mean the definition of the word should be shifted.

<comment><USER51> I'm sorry but this is a pointedly stupid comment. I'm doing the exact opposite of cherry-picking - I'm saying the definition is much broader than the inaccurately narrow one posted.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER54> One thing I question is what kind of WAN link a single chicken fast food restaurant can get, or is economical. A lot are in places where the only last mile ISP options will be adsl2+ over copper or docsis cablemodem. I don't see it being a good use of money to spend $600/mo to pay some local ISP for a dedicated 100Mbps line on a 5 year contract.

But then again it's highly likely that what they do is not bandwidth intensive and will work fine on a 5Mbps x 1.25 Mbps DSL connection.

<comment><USER38> _One thing I question is what kind of WAN link a single chicken fast food restaurant can get, or is economical_

At this point, I don't think it's a problem for CFA.

CFA doesn't try to go into every little town in the nation like McDonald's. It's expansion is very deliberate and only seems to go into very high value locations, where internet presumably isn't a problem.

Moreover, landlords fall all over themselves to land a Chick-fil-a. Almost as much as they do to get an Apple Store. A landlord will make business-grade internet happen, if it means getting a CFA.

<comment><USER55> Meh. Unless you personally own a franchise, or work for corporate, this is pure conjecture.

The article did mention if a failed node is detected, a next-day restore 'disk' is delivered to the store, so obviously that is faster/more reliable than random ISP.

<comment><USER19> Having a chickfila greatly increases realestate value for the landlord though, more so than other restaurant chains like McDonalds.

<comment><USER55> That doesn't mean the local cable company will put in $100k of poles and coax to your restaurant.

Also, McDonald's corporate owns all their real estate, franchisees, too.

<comment><USER19> Im not 100% sure its all of them (malls and airports come to mind) but yes its a vast majority.

True I didn't consider the cable investment either. Normally the landlord pays for this but I imagine chickfila probably pays for part of the bill. That I dont know though

<comment><USER54> For an ISP, if it requires a costly outside plant build to reach a premises for a business circuit, it will usually be rolled into some combination of an upfront connection fee (NRC) and monthly rate (MRC) to recoup the costs and have some positive ROI over a 3 to 5 year term. If it's something like a multi tenant office building they may gamble and take a calculated risk to lose money on the first customer over 5 years (example: $800 NRC and $500 MRC that comes nowhere near xoveinft the cost of the build), with a new expensive fiber build, and hope the sales team can sign up more tenants ASAP.

</comment></comment></comment></comment></comment></comment></comment><comment><USER56> I'm going to guess the surprising (to the layperson) technical savvy of CFA has some relationship to Georgia Tech

</comment><comment><USER57> This was posted in a devops-related slack not too long ago and caused quite a stir over CFA's lack of, uh, diversity. I know it's technically off-topic and frowned upon here, but I believe the fact that they make their beliefs such a big part of the company that it deserves to be pointed out.

<comment><USER58> In what manner is CFA non-diverse? Are you talking racially or belief wise?

If it's the latter, would you class the big tech companies as equally non- diverse, since Christians are frequently treated with derision there?

<comment><USER57> To put it nicely, CFA is extremely unfriendly to the LGBTQ community.

</comment></comment></comment><comment><USER59> That level of infrastructure investment is really playing the long game, especially for a brick and mortar business. I often feel like "traditional businesses" see technology's disruption effects but not the positives they can bring. Other businesses could benefit from this example.

</comment><comment><USER60> Not that surprised, a fully automated restaurant is every restaurant chain's dream, reduces labour costs and logistical accuracies go up at real-time speed.

</comment><comment><USER61> Can anyone explain what the actual problem is they're solving using 'edge computing'?

I get the feeling this is a completely unnecessary use of technology.

<comment><USER29> Here's the abbreviated reason;

Internet connectivity for restaurants stink.

So we put our compute at the edge.

Trust me... as an SRE I would much prefer we shove it in the cloud and call it a day, but alas... no such luck.

<comment><USER61> Yeah, but what compute? what are they computing?

The driving need for edge computing is low-latency. There just isn't the need for it in a restaurant.

I do hope they're not doing real-time control of kitchen equipment via something in a container, containers in Linux are nowhere near deterministic enough.

This strikes me as someone who is familiar with only web technologies but unfamiliar with control systems attempting to create a control system.... completely inappropriate tools for the job.

<comment><USER29> Could you define what is not deterministic about containerized computing?

Examples of things we are controlling are timers for food, cameras that can recognize and track food items, drying machines that will automatically trigger and fry fries.

<comment><USER61> Linux is general is not a real-time system, in that you cannot get deterministic response time from it.

For example I'm guessing it doesn't matter if one of your timers are a second late due to Linux deciding to swap a process out or prioritise some upgrade check but imagine if your time constraints were a lot tighter.

Similarly if you're attempting to track food on a conveyor belt and Linux decides to prioritise the file indexer on your filesystem... oops, a lettuce has been thrown on the floor.

Real-time operating systems exist for a reason.

<comment><USER29> Our tolerance requirements for synchronicity are broad enough that we can tolerate blips like this... at the end of the day we are automating away some simple human interactions (ie; fry the fries or track the food), we aren't performing surgery with these systems.

So far we've been satisfied with Linux (Ubuntu 18.04 to be exact) and it's overall capabilities.

Also, don't forget that cost at scale is a big factor... doing things perfectly but expensively is not profitable at 2k locations.

<comment><USER61> The point about real-time systems is not the your tolerance of 'blips' its about analysis of your worst-case timing requirements. This is what real-time control systems are all about.

Sure, you may be able to tolerate a 1 second 'blip'... how about ten seconds? a minute? do you know for sure how long Linux may put your process on hold for? Linux provides _no_ guarantees at all.

It seems to me that what you are really creating is a traditional control system with web technologies and using the hype-of-the-day (i.e. edge computing) to justify it... with the corresponding increase in overhead and cycles.

This does not seem like a good solution to me, the only novelty here is the inappropriate use of technologies. i.e. you've managed to dig a tunnel with a spoon.

<comment><USER62> Seems like a very appropriate use of technology to me?

If they can get "good enough" results using Linux and running with technologies that the average developer is more likely to know, is it worth the investment in running a RTOS? Especially given running a RTOS will make any higher level app integration more difficult?

The benefit of running with Linux outweighs the risk of "lettuce on the floor", imo. Yes, there's probably overhead in terms of clock cycles, but there's also probably dramatically less overhead in terms of developer resources.. and when you're running off commodity hardware you can guess which overhead will cost more!

<comment><USER61> It would also be 'cheaper' to build a bridge using lego....

Correctness of the solution is important, otherwise we're just playing at being engineers.

<comment><USER63> To quote you:

> At some point, you will realise that your job is not to create an idol to > the software-engineering gods but rather to have a working product.

> Does this product work? if so, its doing its job.

[https://news.ycombinator.com/item?id=17824782](https://news.ycombinator.com/item?id=17824782)

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER64> Chickfila is very much a top-to-bottom ownership company. They own every step of the process in the cooking, from the machines, to servicing, to the logistics behind maintenance.

It's really exciting to see a company like this just want to own everything they do, and not reach out for 3rd party tools and SaaS and such.

<comment><USER19> They don't own everything, their logistics and servicing is through Trimark, granted many of those employees there only work on Chick-fil-a. But yeah Chickfila has always been one of the most high tech restaurant chains out there, its not common knowledge either. They have one of the most unique kitchen setups in the industry as well.

</comment></comment><comment><USER65> Is this use of "Edge Computing" a term that anybody else uses when they are not talking about an always online distributed network in datacenters?

"I have some Raspberry Pi's in my basement and at my vacation home"

VS

"I'm keeping bare metal close to the action with our uniquely scalable Edge Computing implementation"

</comment><comment><USER66> What data does CFA actually /need/ to collect in their restaurants? Orders and supply chain metadata? Store traffic statistics? Cleanliness and user ratings?

I hope it isn't Bluetooth or WiFi (or worse yet, GSM/LTE) metadata from their customers.

<comment> mh- _Our hypothesis: By making smarter kitchen equipment we can collect more data. By applying data to our restaurant, we can build more intelligent systems. By building more intelligent systems, we can better scale our business._

<comment><USER67> Yeah, I wish the article had more detail on the data and how they want to use it instead of the same overview of containerization you could find a thousand other places.

<comment><USER68> Best overview of containerization is here (the video parody is actually funny, perhaps NSFW):

[https://zwischenzugs.com/2016/04/12/hitler-uses-docker- annot...](https://zwischenzugs.com/2016/04/12/hitler-uses-docker-annotated/)

</comment></comment></comment></comment><comment><USER69> Pretty neat.

</comment><comment><USER70> Should I read this, after Chick-fil-A's "THING" about LGBTQQIAA people? _cough_

<comment><USER71> I've always found CFA employees to be kind, courteous, and respectful.

</comment></comment><comment><USER72> Sorry to interrupt the eager participation in this engineer recruiting effort (wow, iPads, cool!)

Don't forget that the people who run Chic-Fil-A support a toxic version of Christianity that, among other things, holds that God himself is going to punish the United States because I'm allowed to get married.

(Also, in case you're a few years ahead of the current western ethical consensus, they torture and kill a massive, genocidal number of chickens every year in order to better monetize related public health crises.)

Maybe don't work for them if you value modern society.

<comment><USER71> I've always found CFA employees to be kind, courteous, and respectful.

<comment><USER73> You've repeated this often enough in the thread that it's actually a kind of trolling. Please don't do that.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

<comment><USER71> Ah, good to know, thanks for the heads up.</comment></comment></comment></comment>


Amazon reportedly in talks to launch a Netflix for books - <AUTHOR> http://thenextweb.com/insider/2011/09/11/amazon-reportedly-in-talks-to-launch-a-netflix-for-books/

<comment><AUTHOR> With the tablet including prime and a discount (or included) version of this service, it could bring in another population to the tablet form factor. I'm thinking those that only have a Kindle now. The Kindle brand could continue to grow.</comment>


Category theory for beginners - <AUTHOR> http://www.slideshare.net/kenbot/category-theory-for-beginners <comment><USER2> The best exposition I've found for demystifying category theory is the first few chapters of Paolo Aluffi's Algebra: Chapter 0 [1]. The central emphasis is on universal properties, which I see most treatments for beginners de- emphasize or ignore (as these slides do). From my experience, universal properties are the main tool for unifying and abstracting concepts into the language of categories. Aluffi uses category theory to unify the treatment of groups, rings, modules, linear algebra, and then goes on to more abstract category theory. I have also written a bit on this [2], regrettably choosing ML as the language of implementation.

[1]: [http://www.amazon.com/Algebra-Chapter-Graduate-Studies- Mathe...](http://www.amazon.com/Algebra-Chapter-Graduate-Studies- Mathematics/dp/0821847813)

[2]: [http://jeremykun.com/2013/04/24/introducing- categories/](http://jeremykun.com/2013/04/24/introducing-categories/)

<comment><USER3> I'm a big fan of that book, but there were a few too many important proofs "left as exercise for reader" for my taste.

<comment><USER4> From my experience, if you don't work through those proofs that are "left as exercise for reader" then you won't ever really come to understand the material. It's working on proofs that gives you the mastery.

It's like learning music theory without ever composing and performing. It's like being a eunuchs in a harem; knowing how it's done, seeing it done, but unable to actually do it.[0] You need to do it to gain any real understanding and/or ability.

It's like reading about programming, but never actually writing programs.

[0] [http://www.goodreads.com/quotes/33229-critics-are-like- eunuc...](http://www.goodreads.com/quotes/33229-critics-are-like-eunuchs-in-a- harem-they-know-how)

</comment></comment></comment><comment><USER5> I scanned through the first 50 slides or so. I've read and drawn out the first few chapters to myself of "Categories for the Working Mathematician", so for me - this was a very nice presentation that graphically illustrated the key forms and key semantics.

I look forward to reading the code parts.

But I have to agree with other posters - it took a lot of studying to get to the point where I can understand your slides. I think by the time you can read mathematics like this, it is because you already understand it verbosely.

As for people who don't agree that this is for beginners, I don't think you will be able to feel like you get it until you sit down with the work and stare at it a lot until it makes sense to you. It sounds stupid and silly, but that's the only way I can really communicate how to learn this stuff with a feeling of adequate understanding. The simplicity and elegance of the pictures do not adequately convey how difficult this stuff is to comprehend, especially if your mathematical foundation involves reasoning about real things. It is descriptive of abstract layers of terms, and the relationships they construct. Category theory is essentially the math of maths.

</comment><comment><USER6> A serious question, and one not meant to cause offense: why is category theory being pushed so aggressively? I've seen numerous "Category Theory for Dummies"-style posts here over the years that all imply CT is somehow extremely relevant to working programmers in ways that other branches of mathematics like analysis or abstract algebra apparently aren't. Considering that most programmers have never rigorously studied even formal logic or set theory, both of which are directly applicable to formal methods, electrical engineering, and relational database theory, wouldn't those subjects be better studied first?

Instead of clicking through a 209-slide PowerPoint presentation you likely won't be able understand, I suggest reading these free books on those subjects:

[http://www.fecundity.com/codex/forallx.pdf](http://www.fecundity.com/codex/forallx.pdf)

[http://i.stanford.edu/~ullman/focs.html](http://i.stanford.edu/~ullman/focs.html)

[http://www.people.vcu.edu/~rhammack/BookOfProof/](http://www.people.vcu.edu/~rhammack/BookOfProof/)

<comment><USER7> I believe it is part of the gradual realization in the commercial software development community that object oriented programming is a dead end on one hand and the fact that category theory captures abstractly patterns of composition, which with enough reflection yield elegant and efficient software designs.

Trivial examples are things like Map-Reduce, the first step describes is done by applying a monoidal functor

T B -> T A,

the fact that the functor is monoidal allows for the work to be distributed across lots of nodes with no communication, the second step uses some associative algebra

T A -> A

to compute the final result, associativity allows for easily parallizable computation of subresults.

<comment><USER8> I'd argue more people know that we can use parallelism when reducing with an associative binary operator, than understand what a monoidal functor is.

</comment></comment></comment><comment><USER9> As a beginner, I can assure you that without someone explaining the point of the arrows, this is most certainly not for beginners.

<comment><USER10> Right, so can someone explain what the hell these slides mean? I studied a lot of abstract mathematics at uni but I wasn't exposed to category theory, and these slides aren't helpful.

They start off with objects, what are they? Are they sets? I'm going to assume they're sets. Then the arrows represent mappings, something like a function that takes an input and maps to stuff in one of those other objects (set of objects?). Then the codomain is the set of objects you can get to by calling that function on the domain of objects.

Am I in the right ball-park here?

<comment><USER11> Ah, the joys of learning math from slides without the talk the slides were helpers to.

The objects here are not necessarily sets, and the arrows are not necessarily functions -- this is actually one of the points of category theory, to abstract the notion of morphism away from set theory. This becomes necessary, when in your regular mathematical life you encounter morphisms which are not strictly functions, but nevertheless in many ways behave as such. Best example would be the homotopy classes of maps between topological spaces. Homotopy classes do not map points from domain space to codomain space, but nevertheless you can compose them, there's a homotopy class of identity, which acts as an identity on composition, sometimes there are homotopy classes that are inverses and so on.

If this sounds complicated and unnatural, that's because it is: it is pointless to learn category theory before learning where and how it is useful. This is completely opposite order than in which it was conceived, and totally misses the purpose the category theory, which is to make everything clearer, to reduce the number of notions, to make it easier to notice analogous behavior between separate concepts, to reduce number of repeating arguments. Learning what a morphism or functor or natural transformation is, without having tens of examples of these notions, only increases the number of concepts without a corresponding increase in understanding.

My advice is, if you want to learn category theory, learn algebraic topology first, or at least abstract algebra (groups, rings, modules, fields). Otherwise, you'll see very little purpose, and very little use.

<comment><USER12> I disagree with your last point, to some extent.

While not strictly necessary, learning a bit of category theory makes FP abstractions such as functors and monoids a bit more accessible.

I can't disagree with the fact that learning category for that sole purpose sounds like a lot of work for too little benefit, though.

</comment></comment></comment></comment><comment><USER13> This is a top-down approach starting with CT fundamentals and connecting them to CS concepts/patterns after 150 slides.

The air is thinner near the top of the abstraction ladder and more or less unbreathable for people who don't spend considerable time in the Ivory Tower.

<comment><USER14> I think if I were attempting to make it more accessible for beginners, I would start with a concrete example of a category (like Set) and introduce the concepts in what is, for most people (programmers in particular) familiar territory, and then attempt to show how abstracting those concepts is useful and applicable in other situations.

</comment></comment><comment><USER15> These slides do the same misleading thing all the other slides I've looked at do: show examples where there are only an arrow or two between objects. I still don't really understand category theory, but one breakthrough I had was when I realized that two objects can (and often do) have LOTS (or even infinite) arrow between them.

For example, in a category of "types", each object could be a type (e.g.: int, float, list(int), etc.), and each arrow could be a function between those types.

With that example, just think of all the arrows between, say, string -> int! There's "length()", "md5()", "to_i()", etc. And don't even get me started on the "identity" arrows of string back to itself (upcase, downcase, strip, etc). We're talking gazillions of arrows.

<comment><USER16> So, on slide 33, the "+1" and "bestFriend" arrows are identities of Int and Person? On slide 35 it says that identity is associated with the identity function, but to me that means a function that returns what you give it, rather than "+1", which gives you the next number. So the identity of Person just means any function with the type signature Person->Person?

<comment><USER7> No, the identity id:A->A is characterized by the property that for all arrows

f:B->A and g:A->C

(P1) id . f = f and (P2) g . id = g, in particular it is unique, since if id':A->A had the same property, then

id' = id . id' = id,

where the first equality follows by (P1) and the second by (P2').

<comment><USER16> Ok, I think I got the wrong idea that the identity was any arrow pointing back on to an object. Thanks.

</comment></comment></comment></comment><comment><USER17> If I didn't have a couple years of shipping Scala code into production I'm not certain I'd understand the slides by themselves. (Having audio and/or video to go along with the slides would be a great help!)

I don't have a math background, but write a lot of Scala anyway. These slides filled in the blanks in a lot of places for me though largely because I could relate to how I've written code.

This is an excellent resource and its piqued my interest in learning more about category theory.

</comment><comment><USER18> See also Bartosz Milewski's very nice WIP blog series[1] on category theory for programmers.

[1] [http://bartoszmilewski.com/2014/10/28/category-theory-for- pr...](http://bartoszmilewski.com/2014/10/28/category-theory-for-programmers- the-preface/)

</comment><comment><USER19> is there any video/audio that complements this slides ?

<comment><USER20> That is my question. I'm not sure how I'm supposed to learn anything from a powerpoint presentation (the worst way to convey any type of information).

</comment></comment><comment><USER21> If two categories are related by a functor, does that imply that the first category is isomorphic to the second category?

(I'm very new to this topic, but I have an applied math background.)

<comment><USER2> That's half of it. If you have another functor going the other way, with some properties on how the two functors relate, then you get what's called an equivalence of categories. [1] Basically, you need both ways to compose the two functors to be "equivalent" to the identity functor in each category (equivalent as shown by a reversible natural transformation).

[1]: [http://en.wikipedia.org/wiki/Equivalence_of_categories](http://en.wikipedia.org/wiki/Equivalence_of_categories)

</comment></comment><comment><USER22> As thankful as I am that this was made and published, I find only slides with no talk very difficult to try to learn from. I kinda wish slideshare wasn't a thing :/

</comment><comment><USER23> I am a bit confused, I had thought there could not be multiple isomorphisms from object A to object B. I thought an Iso was an epi and mono, doesn't that mean the arrows must all be the same? Thanks in advance if someone wants to go through all the ways I'm misunderstanding. :)

<comment><USER12> if f is an isomorphism between A and B, it simply means that there exists f-1 such that f(f-1) = idB and f-1(f) = idA.

I don't think anything in this definition allows us to conclude that there can only be one such f between A and B. Indeed, if A and B are sets of identical cardinality, I'm pretty sure you can create n! isomorphims between the two (where n=|A|). Drawing a picture of of two sets with two elements each should let you see it.

<comment><USER23> Oh. I think I was confused, arrows can be equal (extensionally so), but not the 'same'.

</comment></comment></comment><comment><USER24> This is an ok introduction, but for a much much better introduction check out the book "Conceptual Mathematics". It does a little better than using abstract words like 'objects' (which has a particular connotation for programmers).

<comment><USER12> It also takes bloody ages getting to the "good" parts like monoids and functors :)

</comment></comment><comment><USER25> Hi everyone, author here.

One thing to remember is that each slide had a lot of verbal explanation around it, so I'm not surprised that it's confusing by itself!

We took a video, so hopefully it turns out OK so we can put it online.

</comment><comment><USER8> Could someone explain slide 90-92 to me?

I know what map does and I know what a functor is, but I don't know scala, so I can't understand what the slides are saying.

<comment><USER12> Well, they're a bit weird. I'm not sure why the signature of map magically changes between slide 91 and 92 and find it very confusing.

Slide 90 is straightforward: Functor is a trait (sort of like an abstract class) that takes a type parameter F, where F is a type constructor that takes one type parameter.

Examples of such type constructors are List (used to create List[Int], List[String], ...), Set (Set[Int]...), Option (Option[String])...

The Functor trait declares a map method that, for any two types A and B, take an instance of F[A], a function from A to B and returns an instance of F[B].

A concrete example: F is List, A is String and B is Int. map takes a list of strings, applies a function that turns strings to ints (say, strlen), and returns a list of ints - the Functor instance for List replaces each of its elements by applying the specified function to it.

A functor in FP really is an endofunctor in the category of types, in which:

\- objects are concrete types

\- arrows are unary functions

That is, a functor in FP must map:

\- concrete types to concrete types

\- unary functions to unary functions.

In slide 91, you see that map maps instances of concrete type F[A] to concrete type F[B] - List[String] to List[Int], for example.

In slide 92, you "see" that map maps functions from A to B to functions from F[A] to F[B] - it lifts strlen into a function that takes a List[String] and returns a List[Int].

I'm assuming that slide 92 is meant to show partial application of map, but it really could be made clearer. The author probably explains things more while giving the presentation, but the shift from slide 91 to 92 really is a bit handwavy and magical.

Hope this clarified things a bit, but feel free to ask for more if not, I'd be happy to expand on / correct the parts that you still find obscure.

<comment><USER8> Yeah, that helped clarify it alot :)

Just one question to check my understanding. That definition of a Functor is not completely general right? Since it always takes an object _ to an object F[_].

Like one functor could be to compose an arrow with addition of 0.5 . And that would map X->int arrows to X->double arrows, right? But that would not be compatibe with the Functor trait?

<comment><USER12> I'll be using pseudo code and skip over some technical details that are entirely irrelevant to this conversation in my answer. I hope I understood your question correctly and not answering something else entirely.

First, a functor doesn't go from A to F[B] but from F[A] to F[B]. A to F[B] looks suspiciously like something a monad does, a topic we don't want to get into.

Your example is a functor that maps unary functions to unary functions by composing them. It's a good if slightly complicated example: unary functions have two type parameters (A and B in A => B), while the slide's Functor only takes one. That is, you want to write a Functor[A => B], but you cannot, or at least not directly.

What you do is fix one of the parameters. You don't write a Functor for all unary functions, but a Functor for all unary functions whose parameter type is C (for concrete): Functor[C => A], where C is a concrete type and A the functor's type parameter.

This functor's map method takes an instance of (C => A), a function f: A => B, and yields a function C => B. The only non-pathological implementation of this is composition: C => A => B

What I just wrote makes it sound like you need to write a Functor implementation for all possible parameter types, but that luckily isn't true: writing a function that yields a Functor[C => A] for any given type parameter C is trivial. I'll put the code here just to show it is, but you don't really need to understand it:

// I know the C => ? isn't standard Scala, I'm using kind-projector syntax to make this readable. implicit def functionFunctor[C] = new Functor[C => ?] { override def map[A, B](fa: C => A)(f: A => B): C => B = f compose fa }

To go back to your concrete example: you just need to get functionFunctor[X] and call map on (x => x * 0.5) to get your function from X to double. Note that some technical details are skipped (partial application, calling the binary function map with a single parameter), but I arbitrarily deem them irrelevant to my point.

<comment><USER8> You misunderstand my question. I realize in hindsight how it was ambiguous.

I used the word "object" like it is used in slide 55. Functors map objects to objects and arrows to arrows. I.e. arrow is the (co)domain of the functor, and object is the (co)domain of the (co)domain of the functor.</comment></comment></comment></comment></comment>


Amazon adding lockers for customers at 7-Elevens and drug stores (video) - <AUTHOR> http://www.wcsh6.com/news/watercooler/article/216376/108/Amazon-adding-lockers-for-customers-at-7-Elevens-and-drug-stores

<comment><USER2> If you live in SV/BA, there's a couple of these around. I tried one in Mountain View, and I totally love the idea of not having a box with something expensive in it sitting at my doorstep for 6 hours.</comment>


Create your itinerary on-the-go, meet people and bring back home a story to tell - <AUTHOR> https://www.travsolo.com <comment><AUTHOR> Hey All!

We recently launched our iOS planning app for solo travelers and are excited to get people to solo travel easier than ever!

Plan your trip, meet people, and stay safe – all while sharing memories of your journey on the go.

Step 1: Choose your interests Step 2: Choose either or not you want to meet people Step 3: Get your Itinerary for the day & add your own content - photos, short videos and notes. Step 4: Turn your itinerary into a story video to share with your loved ones, publicly or privately.

TravSolo champions independent travel, promotes diversity abroad and instills the freedom to travel at the beat of your own drum. Traveling alone doesn't mean spending your trip alone or that you don't have friends or family, it means you stopped waiting for others to live your life your way.

Disconnect while staying connected. No more editing or getting distracted with endless feeds or notifications while traveling. Allow your loved ones to follow your journey wherever you go, in real time. Travel blog easier than ever without the hassle. Connect with travelers, meet locals and backpackers when you're traveling.

Solo travelers are active people who want to see the world but want to do it safely. Currently, there is no travel app focusing on women safety concerns.

Available to download now on the App Store in North America and Europe .

\--> [https://apps.apple.com/us/app/travsolo-solo-travel- easier/id...](https://apps.apple.com/us/app/travsolo-solo-travel- easier/id1459854419)</comment>


Design is Open Source - <AUTHOR> http://www.markboulton.co.uk/journal/comments/design-in-open-source Mark's a great designer.  Interesting to see his thoughts on design / development in the open source world.  A bit long, but a pretty good read. <comment><USER2> Please fix the title. It's "Design in Open Source".

For some background, this particular situation was kicked off by a comment at <http://twitter.com/markboulton/status/3576335793> which led to a response at [http://angrydonuts.com/an-observation-about-designers- versus...](http://angrydonuts.com/an-observation-about-designers-versus- developers)

<comment><AUTHOR> Sorry about the title. Thanks for the extra context.</comment></comment>


Ray tracing has failed to deliver on its promise - <AUTHOR> https://www.pcgamer.com/au/ray-tracing-has-failed-to-deliver-on-its-promise/ <comment><USER2> That's like going back to when Terminal Velocity had just been released for the S3 Virge (which just barely managed to do bilinear filtering at 640x480 at 15fps) and saying welp, this sucks, show's over boys, hardware accelerated rendering has failed.

<comment><USER3> I had forgotten about that trash board. Games of the time (Descent, etc.) looked better AND ran faster using only CPU. My first 3dfx Voodoo was a revelation.

</comment></comment><comment><USER4> Or GPUs haven't been powerful enough to support it properly for ages

</comment><comment><USER5> TL;DR

> Even here though, I didn't feel like ray tracing was actually doing much to > improve, or even change, gameplay.

Why would eyecandy be expected to change gameplay?</comment>


How Do I Manage An Unmanaged VPS Hosting? - <AUTHOR> http://www.bloggingguts.com/manage-unmanaged-vps-hosting/

<comment><AUTHOR> Hello Folks, Lets discuss some of your ways to manage an Unmanaged Hosting platform. Here's My Story behind Managing an Unanaged Hosting.

I am Using DigitalOcean's $10 Droplet which includes, Ubuntu 14.04 x64 and LEMP Stack. Actually I am a Beginner and I have tried several times to host my WordPress on it, But Unfortunately, Each time My WordPress Url return blank. Finally One of my friend recommend me to use EasyEngine and Now I am doing my best with EasyEngine it's rock and Open Source Linux Shell Script which helps me to setup my WordPress blog with just 3 commands.

What do you think about EasyEngine, Have you used it before? Share your wisdom about ee or Is there any other Open Source program you use to manage your Unmanaged Hosting?</comment>


Stalking a person with only their email & IP address - <AUTHOR> http://www.attackvector.org/invasion-of-privacy/ <comment><USER2> Whoever wrote this is a complete idiot and doesn't know the first thing about computer forensics.

Check out this gem:

"I use spammers and pedophiles as test subjects when I'm working on something. This is mostly because it's unlikely that they would go to the authorities and point the finger at me, knowing that I could easily turn around and say something to the effect of, "Well, yes I did pwn his box.. but you should have seen all the child porn I found on it." owned x 2."

Well, owned x 3 because the first thing that this would do is backfire, after all if you admitted to hacking someone's box they could make a fair claim to you having put that data there.

The last thing you do when you go after a pedophile is to hack their computer, any evidence found there is tainted, and you are now a suspect with a confession on record.

There are several other laws this guy broke in his vigilante action against this spammer, he's set himself up quite nicely for a lawsuit, after all he's handed the prosecution all the documentation they could possibly wish for.

<comment><USER3> It is moronic that he thinks that informing cops about child porn on a computer he hacked would get him out of trouble, or get the pedophile in trouble - but the odds that a pedophile is going to call the police and complain about being hacked seem to be to be about where he puts them. He's not talking about "going after" the pedophile, he's talking about hacking the pedophile to have somebody to hack.

Out of curiosity, what possible laws did he break by going through a few public databases, facebook, whois data and google, and what does it have to do with computer forensics? I did this same type of thing to go after a 4chan- type troll who was attacking my blog a few years ago. and it was a joy to really effectively stop somebody who thought they were anonymous on the internet.

<comment><USER2> He should have left the guys wife out of it for one, he also should not admit in public to hacking computers. Regardless of who owns them that's breaking the law.

<comment><USER3> He definitely shouldn't have mentioned the guy's wife and kids, but it's certainly not against the law. It's definitely not bright to admit in public to hacking computers in the same way it's not bright to admit in public to doing drugs, but it isn't actually against any law but the law of prudence and good sense.

<comment><USER2> Before you ever decide to hack someone's computer I suggest you consult a lawyer.

<comment><USER4> I think I missed something. How did he hack the other guy's computer? He based his analysis on email headers received in his own computer, and used Google, whois, fb, etc. after that. Right?

<comment><USER2> He didn't hack this guys computer but he pretty much documents for the world to see that he's hacked other computers.

There's stupid and then there is this article.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> This guy is clearly a complete moron, but the thing that annoyed me the most was not the fact that he seems to think he's a genius for being able to use Google, it's his total misuse of the phrase "work cut out for".

 _Also, consider what you're sending in this email. What if this guy had sent me an email trying to extort me, threaten me, whatever? I could turn this over to the authorities and they'd have their work cut out for them._

<comment><USER6> I noticed that as well. I had to re-read it a few times before I believed it.

</comment></comment><comment><USER7> This is Dumb catching Dumber. Only the bottom of the barrel spammers in the year 2010 would use their real non-proxied IP address and a non-private WHOIS record. I would also point out that GeoIP does not point out the address of the person using an IP address, just their ISP (or whoever owns the ARIN block).

<comment><USER8> Rule #3: Spammers are stupid.

<http://bruce.pennypacker.org/2005/02/28/the-rules-of-spam/>

The barrel is immense, and even the bottom is quite large.

</comment></comment><comment><USER9> I stopped reading when I read:

 _Just put the IP address in the box and hit "search". Here's what we find.

        Region: Washington
        City: Spokane
        Postal code: 99205

So, we're narrowing it down.. we now know that it's Spokane, Washington._

Erm, no you don't know that he lives there, it just means that the IP address he happens to hit the internet with belongs at that address. If I geoIP myself it comes up about 300+ miles from where i live , I geoIPd some of the public wifi connections I used and it's not even the same country that I am in.

I blame google analytics for this, I often hear people say stuff like "hmm , all our visitors seem to come from London, let's optimise our site for people in London".

</comment><comment><USER10> It's also possible that the spam email came from a computer infected with malware, making the target of this "attack" an innocent bystander.

Whether this was the case, in this instance, is insignificant, it's just one of those things that the author didn't think of, it shows the author did not think very deeply about the situation, and simply wanted to flex his technical jock.

<comment><USER11> _The email was coming from his email address, using his business's name, and advertising his business_

</comment></comment><comment><USER12> The realities of life are that at some point you're going to have to give out your personal information. From a fragment of that it's pretty easy to reconstruct the rest.

The only thing the internet really changes is using a browser vs. going down to the courthouse.

Finding a name, address and phone number are not really a big deal. Life is far better knowing lots of people than it is knowing a few people. My time is worth far more than anything I could hope to gain from tracking down someone who sends me email I don't want. I'd much rather just click the Spam button and get on interacting with the wonderful people in my life.

"OMG, someone on _the internet_ found my name, address, and phone number. I don't know how I could go on living my life dealing with _phone calls_ and _mail_ , my life has been completely destroyed!"

If this person could put my name, phone number and address infront of every person on the planet I'd gladly pay for that service, it would be extremely valuable having 7 billion people know how to reach me.

</comment><comment><USER13> Whois info can and does easily get falsified. So the basis of his entire research which was built on checking whois is incorrect. In other words there is nothing preventing anyone from registering a domain name using someone elses contact info. Which is actually done quite frequently by web designers as only one example on behalf of their customers for legitimate reasons.

Separately, this is also incorrect:

"and it's the administrative contact, which means he owns the domain"

The "registrant" actually owns the domain. Not the admin contact. Of course the admin contact can own the domain. And all the info can be false anyway as mentioned.

For example in the case of the domain "ycombinator.com" the admin contact is "Kirsty Nathoo" who, according to linkedin, is the "VP Finance and Operations at Y Combinator". The registrant is Ycombinator LLC and of course that makes sense that they are the owner.

Geo locating ip addresses also produces far from gold standard results.

</comment><comment><USER14> Hm. He keeps [redacting] Nancy's name, but leaves it in all the screenshots. Sloppy work, there.

<comment><USER5> _UPDATE (1/12/2011): I received an email from Steve regarding this post. He sincerely apologized for his actions and realized now that what he did was wrong and simply asked that I modify the post to protect the identities of his family. I felt that this was a fair request, considering that his family had nothing to do with what Steve did and it doesn't jeopardize the impact of the article. So, if you're wondering why you're seeing all the "[withheld]"'s, that's why!

PS – Yes, I realize the names are still shown in the images, but they're not indexed by Google_

<comment><USER15> _"but they're not indexed by Google"_

...yet.

</comment></comment></comment><comment><USER16> I do this sort of stuff on a near-daily basis. All the tools he refers to are quite common.

The only things I'd add is that if you're a state of Washington resident, your personal details are particularly open to the Internet. They seem to have every little government database open for use. I always breathe a sigh of relief when whoever I have to track down lives or has lived in WA.

In numerous other cases, I've had to track down people merely with a street address, and getting a full dossier on those people is equally easy.

If I were to use illegal techniques, it would be orders of magnitude scarier.

I track down criminals or long lost loved ones normally, but how do you combat being found? Dis-information. I could go on about this, if people were interested.

Summary: keep your personal life off the Internet.

</comment><comment><USER11> _whois ricardo.cc_

ring me up :)

I'm not worried about my address, phone, name or email. You can get those from the doormen, co-workers, friends, google and hundreds of documents scattered around the world (bills, contracts, registers, etc).

The best you can and should do is protect your personal life. I recently switched my FB account to "friends only" - that should be the default for everybody.

<comment><USER8> You realize of course that your friends include any attorney with a subpoena, the worlds' intelligence agencies (who's to say there isn't Intelligence life in space), and all the criminal hackers on the planet.

Sleep tight.

<comment><USER11> I do. I don't carry any secrets though (and for sure wouldn't share them on Facebook), and am more afraid of being ran over by a bus. Otherwise I would just get off the internet.

</comment></comment></comment><comment><USER9> On a somewhat related note , this talk from defcon:

<http://www.youtube.com/watch?v=fEmO7wQKCMw>

Is actually pretty cool, albeit a little far fetched in parts.

</comment><comment><USER17> It seems inevitable that all public information about people will become organized, and you just won't be able to keep secrets like this any more. Better than it be available to everyone than only to sneaky governmental agencies.

<comment><USER18> Just 2 cents- Bankruptcy details are from PACER (pcl.uscourts.gov). The rest is obvious, of course.</comment></comment>