Ask HN: How do you continuously monitor web logs for hack attempts? - <AUTHOR> what is the generally accepted best practice to be monitoring web logs for anomalous accesses ?
do you guys just throw cloudflare in front and forget about it ? Or do you have engineers who work like data scientists - eyeball the logs ?
I have heard suggestions of using a firewall - but I'm genuinely curious on how do security focus companies discover things like "oh, we got attacked by Bitcoin miners from North Korea". Are there sophisticated tools that do this for you.. or is there a generally accepted practice that has evolved for even regular engineers to do this ?
P.S. I'm personally more interested in an API-focused answer, but I guess the same thing applies for websites. <comment><USER2> The answer isn't quite "don't bother" but it's definitely not "install these tools and get started".

Most of the things you can easily set up to watch for security events are looking for problems you simply shouldn't have.

For instance: you can set up fail2ban, sure. But what's it doing for you? If you have password SSH authentication enabled anywhere, you're already playing to lose, and logging and reactive blocking isn't really going to help you. Don't scan your logs for this problem; scan your _configurations_ and make sure the brute-force attack simply can't work.

The same goes for most of the stuff shrink-wrap tools look for in web logs. OSSEC isn't bad, but the things you're going to light up on with OSSEC out of the box all mean something went so wrong that you got owned up.

Same with URL regexes. You can set up log detection for people hitting your admin interfaces. But then you have to ask: why is your admin interface available on routable IPs to begin with?

What you want is an effectively endless sink for developer logs (Splunk is fine but expensive, ELK is fine but complicated to manage), and you want to add structured instrumenting logs to your applications in all the security- sensitive places, like authorization checks (which should rarely fail), and especially a very solid, very long-term audit trail of admin-like actions, so you have some prayer of tracking insider misuse. You could instrument your SQL queries and catch any query failures --- which should also be rare. That kind of stuff. I don't think there's a good shrink-wrap package that does that.

But really: focus on removing attack surface, rather than continuously monitoring it.

<comment><USER3> I came here to add an important detail about WAFs that's missing in this thread. For context, I am the original author of ModSecurity (but haven't worked on it since at least 2009). I wrote ModSecurity for three use cases that still make a lot of sense sense; they are: 1) full HTTP transaction logging (meaning request and response bodies), 2) rule-based analysis of HTTP traffic (largely to trigger logging and otherwise highlight anomalies) and 3) virtual patching.

The first two use cases are all about traffic visibility, which has been, and remains a big problem. If we're talking about improving security, the third use case—virtual patching—is the exciting part. The story goes something like this: you discover a vulnerability in your code but you can't fix it immediately, for whatever reason (take your pick). In practice, it may be weeks until the problem is fixed. However, if you have ModSecurity installed, you can write a virtual patch for your problem, often to fix it 100% reliably.

However, it's very important to understand that virtual patching doesn't mean just slapping together some rule that appears to be doing the job. Blacklisting doesn't work. To properly virtual-patch, you have to fully understand the vulnerability and the entry point to the application. You have to carefully whitelist all parameters in that particular location. In other words, you have to almost replicate the work that should have been done by the developer.

Even in this context, it's not all rosy. Virtual patching requires a great deal of expertise to do properly. It's almost always better to spend your time fixing the actual problem. Sometimes, however, that's not possible to do in a reasonable time frame, and that's when a WAF (according to my definition), can be a life saver.

<comment><USER2> Thanks for this. I admire ModSecurity! But the subtext of my comment is the needs of typical HN people, most of whom manage applications they wrote, for small companies. Overwhelmingly, those people should spend any time they would spend on things like ModSecurity instead shoring up the security of their code.

</comment></comment></comment><comment><USER4> I assume that I suck at security, everything I have ever built is badly compromised and I'm too dumb to know it. I'm so bad at security that evil criminals actually celebrate my birthday.

Since I'm this bad, I assume that any monitoring tool I could install on the boxes themselves, up to and including all my logs are compromised. I'll usually still install monitoring tools for compliance, basic diligence and because they make wicked graphs that impress my superiors.

Since I'm that bad, I'd rather not invest 40 hours a week combing through fake logs unless I'm validating that I've taken care of the basic security fundamentals for my threat level.

Rather, I like to make sure that my security fundamentals are taken care of so that I can stop the simple attacks. If my threat/compliance model dictates, from there I'll possibly step things up into a web application firewall (WAF).

But, through all of this, I know that there are people out there who are so much smarter than me that the best I can do is little more than an annoyance for them. So, again, I like to assume that I'm badly compromised. This implies that I'm very careful about what data I keep, and I take efforts to protect my data at rest.

</comment><comment><USER5> I break into stuff professionally.

There is nothing I love more than the "just throw {Cloudflare,Akamai,Fastly,etc} in front of it" crowd. If you are going to deploy a WAF [1] it _needs_ to be at the edge of your datacenter or cloud environment. No matter what fancy feature your provider wants to sell you to fix this, there is a way to bypass it.

* Make sure you are logging centrally, and logging the right things.

* Make sure someone is looking at the logs, it doesn't have to be a full time person, but it needs to be someones job.

* Depending on how big you are, hire people to break in on some regular basis. It could be anything from hiring a cheap contractor to run a bunch of automated tools, to full on 10 person adversarial teams. They will find some stuff for you to fix, but more importantly you will know whether the first two points are working based on what you see.

1\. [https://www.owasp.org/index.php/Web_Application_Firewall](https://www.owasp.org/index.php/Web_Application_Firewall)

<comment><USER6> > to full on 10 person adversarial teams

Okay this sounds incredibly interesting. Are there any open source groups that do this "for free" sorta? Not asking to get free working, asking so I could possibly jump in and learn.

<comment><USER7> There are websites that let you practice. hackthebox.eu is really popular. And there is a bit of a test to even get access to that.

<comment><USER8> Wow! Thanks for this! I "signed up" and it looks very interesting so far

</comment></comment></comment></comment><comment><USER9> In short - security is such a big and multifaceted topic that it's hard to know where to start, but I think that learning Kali Linux tools would cover a lot.

I started working at a cybersecurity company - our product is ridiculously stupid and I try and work as little as possible. In the time I'm not working I try to at least learn stuff. Since I'm at a cybersecurity place I sometimes try to learn about cybersecurity. It seems that even though for profit cs companies are making crazy dough, a lot of open source solutions out there do the same thing. There are so many attacks - they can be anywhere from the application level to the level of physical devices like routers having some kind of exploit. For any type of attack there is a good or multiple good open source tools that can be used to execute or defend (depending on what color hat you have), and a ton of them come standard installed on Kali Linux, but before you learn these you have to understand networks. As a python/Django dev I didn't really have a deep knowledge and still don't have a deep knowledge of this. Nor have I done more than dabble with any of these tools. But I know that there are a lot of videos on how to use these - including this 15 hour (!) one [https://m.youtube.com/watch?t=1177s&v=vg9cNFPQFqM](https://m.youtube.com/watch?t=1177s&v=vg9cNFPQFqM) \- if you were to go through that you'd probably be a security wizard

<comment><USER10> Thanks for the youtube link. Seems interesting. I am going to watch it in full.

</comment></comment><comment><USER11> We also use Fail2Ban like a poor man's Web Application Firewall - for a Rails app you know you'll never have legit traffic to anything 'cgi' or 'php' or 'exe', so crafting a couple of regexps to match those and then ban them works well.

We use CloudFlare so we feed the IP bans into CloudFlare's firewall through their API as well as local IPTables. (We also use CloudFlare's WAF - we like redundancy in our paranoia.) If you had more demanding requirements you could look at mod_security for Apache or Nginx.

We also implement rate-limiting using Fail2Ban, which helps to trip the dumber automated scans.

Every time Fail2Ban blocks an IP address, it emails our admin team with the matching log entries and a whois on the IP address, which we then eyeball to make sure there's nothing weird going on.

TBH, easily 90% of the IP addresses we block got caught looking for 'wp- admin/admin.php', which is probably a good reason to never use WordPress.

<comment><USER12> Are these bots causing problems like consuming excess resources? If not it doesn't seem worth the effort to ban them.

These days I only block bots when they're e.g. filling out forms or posting junk comment that a human has to moderate.

<comment><USER13> It seems you never had to deal with the amount of crap bots throw at every minor website out there.

If you don't want your logs flooded with vuln. scanning and referrer spam, use fail2ban or a waf.

<comment><USER12> I have, in fact, been doing this for years. Why should I care about having pristine logs?

<comment><USER13> Yeah, who cares about the extra disk space, the extra overhead, the waste of resources, filtering through noise to understand some issues with the app, right?

Isn't it lovely when every other referrer is trying to promote some online pharmacy or testing to see if you have a wp vulnerability?

Sure you can filter it, but I'd rather deal with garbage at the source.

<comment><USER12> A lesson I learned the hard way: if random bots are causing significant resourcing issues, then you need to address that directly. And if their impact isn't significant, why are you spending time on it? The disk space my log files consume is trivial.

</comment></comment></comment></comment></comment></comment><comment><USER14> > I'm personally more interested in an API-focused answer

OSSEC [1] is my go-tool for this type of tasks, it offers a language-agnostic interface to monitor any type of events. I used it along with my team to power the early version of the Sucuri Firewall, which later became the GoDaddy's security system. The project has matured a lot and is heavily tested using millions of HTTP requests every day. We also implemented a behavior-based algorithm to detect malicious requests, this part is not open-source but I can tell you that it was relatively easy to integrate it with the core interface.

Splunk [2] is also a good option, slightly more difficult to configure though. I have had the opportunity to meet some of the developers working in this company during a short visit to their Vancouver office and was genuinely surprised with the level of engineering that goes around this product. Definitely, worth checking it.

NewRelic [3] while the purpose has nothing to do with security, it also offers an easy-to-use interface to monitor events in the system. With some extra configuration you can make it work as a rudimentary firewall and benefit from all the great features that the product provides.

Disclaimer: I am an active contributor to the core features, but I am neutral on my recommendation.

[1] [https://en.wikipedia.org/wiki/OSSEC](https://en.wikipedia.org/wiki/OSSEC)

[2] [https://en.wikipedia.org/wiki/Splunk](https://en.wikipedia.org/wiki/Splunk)

[3] [https://en.wikipedia.org/wiki/New_Relic](https://en.wikipedia.org/wiki/New_Relic)

</comment><comment><USER15> What you need is information, what's easy to get is data. Generally, if you have data on what "normal" looks like, and data on what the current state is, you can view the difference as information that indicate an abnormal state; be that a full/failing disk - or someone downloading all tour data via blind sql injection.

That's the theory. I've yet to achieve this in a timely fashion, within budget limitations - but on my list is to have another look at graylog2 for logging (if you have the money, splunk is probably easier - but they do charge for the value they provide).

But I'm hoping to find the time to mash graylog2 with something like collectd, and maybe a general event collector for exceptions/stack traces. The goal would be to have it all on a similar timeliness in a unified interface (this spike of exceptions happened after this series of requests from there, and after the disk went full). And ideally with a sensible way to automate "normal" vs "exceptional" (alert on high cpu, low disk, high network bandwidth etc).

Sure, people should read logs/alerts - but it's a challenge to filter out irrelevant (ssh pw bruteforce against a ssh server with only key auth, Web app crashing due to #wontfix (yet) bug) - so the relevant becomes obvious (successful key based login from different continent - where none of your team are).

[https://github.com/Graylog2/graylog2-server](https://github.com/Graylog2/graylog2-server)

</comment><comment><USER16> You don't monitor web logs, you ship the logs to cold storage every [crontab spec units for time :)] and monitor your hosts and network for unusual activity.

If anything happens, THEN you look at the logs. The signal/noise ratio of the average web host logs makes combing through them just not worth it.

There are tons of products out there - have a look at OSSEC ([https://www.ossec.net/](https://www.ossec.net/)), it's open source and free.

</comment><comment><USER17> Don't.

You shouldn't care about hack attempts. There will be thousands each day. They don't matter.

You should care about your systems not being easy to hack. Unless you're a very high value target then "practice basic security hygiene, install your updates, don't reuse passwords" is enough to achieve that.

</comment><comment><USER18> Using Splunk to monitor traffic either via web logs or via more detailed packet capture solutions.

Just good web logs is enough to detect all kind of malicious anomalies.

Splunk machine learning functions can cluster data and detect anomalies in anything without rules or signatures:

[https://imgur.com/a/IFhUZuH](https://imgur.com/a/IFhUZuH)

</comment><comment><USER19> I'd recommend only really addressing the most obvious attacks, rather than trying to follow up on everything that even looks like it.

I once got an "emergency" call at 7am (I had been up doing something rather late), about a machine I run attacking this guy. After some digging with him, the machine was our public-service NTP server, and the packets were on udp/123, in response to packets his system was sending out. Ended up with him yelling at me and hanging up, and me shutting down our public NTP server for good.

</comment><comment><USER20> 1) fail2ban 2) log everything to an ELK stack (or your preferred alternative) and set up dashboards and reports to feed you malicious crafted URLs 3) Reject traffic in your web proxy based on a maintained list of hostile URLs. Ban the source of this traffic. Don't send back 500s, just slow down and eventually stop answering the requests. HAproxy lets you do this easily with a few simple rules and a stick table.

</comment><comment><USER21> most successful organizations ive seen don't think about their API/app, but rather their stuff of value: usually data. If someone lifts your entire database, can you spot that on the way out the door? How quickly would you notice 100% of your fattest internet pipes spewing data at some poor souls website? You mention bitcoin; how long until you spot all your machines at 100% CPU?

</comment><comment><USER22> Separating this out:

1\. Monitoring logs for security -- people don't, but machines do. Initial security comes more from segmentation architectures (bastions, whitelisting, ..) and network/host intrusion detection systems. Logs do come into play later for app-specific rules (unpatched apps, fraud tipoffs, ...) and eventually automated UEBAs (exabeam, ..) and analyst threat hunting (plug: [http://www.graphistry.com](http://www.graphistry.com) !).

2\. Forensics for post-breach attribution & containment/mitigation. Let's say something does happen (... it will.) Logs are important here.

\-- Threat intel: Known indicators of compromise (IOCs) -- IPs, URLs, downloaded file hashes, ... -- get published. Team setup always-on correlation rules against their internal logs via say splunk or hadoop, and if hit, they know the threat intel of it. Or they'll get alerts from OpenDNS that an internal device is talking to a known bad actor. That's generally pretty shallow threat intel on the attack (ex: VirusTotal.com entries), but a start. There are multi-step ways to iteratively correlate across these sources, but takes effort. Attacker attribution happens here, but typically more important is identifying further incident IOCs in case of a bigger breach.

\-- IR / DFIR: Teams care more about defense and explaining what happened internally (incident killchain, scope, ...). Say there was a malware alert from AV, if the dwell time was hours or the device important, teams should investigate what else happened for the device/user/etc, or if anyone else got hit. THAT'S where logs really shine. Small/immature teams will check individual tools (firewall device, ...), while more mature teams check centralized logs.

We find manual log analysis shallow in practice for a decently sized org... so another plug for [http://www.graphistry.com](http://www.graphistry.com) for setting up more comprehensive multi-step analysis flows for the human side of looking through stuff like malware, phishing, and account hijacking. Tools like ours work on top of log stores like Splunk/ELK/Hadoop/etc. (more scalable) or bring their own DB (less scalable.) This can get pretty intense: some companies log all the way through process activity.

At the extreme end (FBI, Mandiant, ...), companies will quarantine a device for activities like analyzing RAM & disk images.

</comment><comment><USER23> I sleep a lot better since I use IP white lists on any port/is that is not public facing. Central reliable location with a signed whitelist that the servers checks regularly and adds to its firewall. No security measure is bulletproof, particularly against zero days but that reduces the risk by orders of magnitude.

<comment><USER24> Nice idea, but how does it improve over "have important stuff on an administrative VPN"? To me the whitelist seems more effort (but we often have non-static IPs at home; with dual stack lite aka native(?) ipv6 plus a private ipv4 behind a huge, ISP-wide NAT making things even worse), and it doesn't give you additional encryption.

<comment><USER23> Agree. I have scripts regularly updating the whitelist with dynamic IPs every 5 min. But I don't use VPNs for various reasons.

OpenVPN on windows is very slow (unlike on Linux), it's hard to get above 5-7MB/s, which is a problem for large file transfers (and that was pre meltdown and spectre patches). It's also hard to configure, and the server still has WAN connectivity so it's easy to misconfigure it and get no protection. I also found that having an OpenVPN LAN with no gateway in addition to your default LAN on win7 tends to interfere with Chrome on the client, which will occasionally complain that it has no connectivity.

Being a belt and suspenders guy, and only dealing with windows machines, I found that a good solution is to combine it with IPSec, which is straightforward to configure on windows, fast, and adds encryption when the underlying protocol doesn't have any (SMB2). You can configure the firewall to only use IPSec for certain ports. But it's a nightmare to interop with non windows machines.

</comment></comment></comment><comment><USER25> Marcus J. Ranum's "Artificial Ignorance" concept is pretty good. I did an implementation and have been running it for more than 10 years:

[http://www.ranum.com/security/computer_security/papers/ai/](http://www.ranum.com/security/computer_security/papers/ai/)

Someone's also done a Python implementation: [https://github.com/lchamon/ai](https://github.com/lchamon/ai)

...and it looks like there's one in syslog-ng Premium Edition:

[https://syslog-ng.com/documents/html/syslog-ng-pe-latest- gui...](https://syslog-ng.com/documents/html/syslog-ng-pe-latest- guides/en/syslog-ng-pe-guide-admin/html/concepts-artificial-ignorance.html)

</comment><comment><USER26> I highly recommend you try out Sqreen. [https://www.sqreen.io/](https://www.sqreen.io/) It's basically newrelic for security. Takes 2m to add to your web app and it works super well. It blocked an attack automatically on our service recently!

</comment><comment><USER27> A tad off-topic, but one thing I've enjoyed doing in the past is giving my attackers something to get distracted on, allowing me to observe their behavior.

[https://www.youtube.com/watch?v=smvwW3vW5rM](https://www.youtube.com/watch?v=smvwW3vW5rM)

<comment><USER24> Wow, it's interesting to see how bad this hacker is with a shell. Even the "better" attacker from the video you linked in the comments doesn't seem to really know what he is doing at times.

Sadly, I feel not all hackers are dilettantes.

</comment></comment><comment><USER28> I can not recommend watching this Defcon talk enough: [https://www.youtube.com/watch?v=_SsUeWYoO1Y](https://www.youtube.com/watch?v=_SsUeWYoO1Y) \- "You Spent All That Money"

<comment><USER29> Dude I'm a security consultant for a major vendor and I can't tell you the number of times I've

A) Found an incident while installing new equipment

B) Installed new equipment last year and came back this year because they still got hacked

and/or

C) Ripped out their old gear and put in my company's gear after a hack because the sales guy told them they wouldn't get hacked with our (passive, monitor- only) gear.

You need the right tools, no doubt. But you need to understand your needs first. Centralized logging is the first step, but without knowing what you want to get out of it, those logs just sit there unnoticed. Even intelligence platforms like Splunk or ArcSight, unless you know what you're looking for and write it into rules, it's just sitting there.

Knowledge is what matters, the tools just make it _easier_.

</comment></comment><comment><USER30> During my time at Wanderu we set up Filebeat on all app machines, forwarding json logs to Kafka, and then on to ELK and Pipelinedb for analysis/alerting/session killing/ etc.

It worked very well, as there are almost always application edge cases that should not be blocked immediately with Cloudflare or at a FW level.

ELK was used for exploration and/or setting alerts on thresholds that we had found/defined beforehand.

Pipelinedb was used to run continuous aggregates on a stream, augment (pop json log lines off a stream, enrich them with Maxmind, etc) logs in "realtime", and immediately surface malicious behavior. We'd then programmatically kill sessions or add Cloudflare rules upstream, based on the behavior that surfaced in a Pipelinedb continuous aggregate.

Wanderu: [https://www.wanderu.com/](https://www.wanderu.com/) Filebeat: [https://www.elastic.co/products/beats/filebeat](https://www.elastic.co/products/beats/filebeat) PipelinedB: [https://www.pipelinedb.com/](https://www.pipelinedb.com/)

<comment><USER31> I'm curious how you got Filebeat to push to Kafka. Did you write a custom modification to Filebeat/libbeat?

</comment></comment><comment><USER32> I use a home-grown solution called txrban:

[http://www.kylheku.com/cgit/txrban/tree/](http://www.kylheku.com/cgit/txrban/tree/)

It consists of several processes written in the TXR language. One for monitoring Apache, one for sshd and one for Exim (mail server).

TXR's standard library has "tail streams": streams which automatically follow a rotating log file.

These txrban scripts simply open tail streams on their respective log files, use TXR pattern matching to scan for what they are a looking for, and then use functions in the tiny txrban utility library to do the banning. Banning is instigated by issuing iptables commands.

The banning isn't persisted, but when you restart the scripts, they retroactively replay everything by scanning the available logs from the beginning.

Banning events are reported via syslog. A "dry run" configuration variable allows testing for the rules without actually modifying your iptables.

I never packaged this program into a redistributable form. It contains some hard-coded log-file paths that work on my server, and no documentation.

</comment><comment><USER33> Could something like poor man's primitive check work?

\- Create a custom http header and find a unique way to create/identify it. So theoretically another layer of authorization, just like APP ID for instance.

\- Create a middleware that would scan the request and check whether it has the required custom header

\- If not log everything and create a notification/slack/allow/deny and if it's good let through the request.

</comment><comment><USER34> I used to run a web service, an API, that created 3D models of people from photographs. It was a character creation service that made likeness models of real people from a photo. I knew we'd get a lot of hack attempts, and a consultant recommended we get a high end Sonic Wall appliance, a $20K piece of hardware that cleanses all network traffic to and from my data center.

Best decision EVER! We'd get hack attempts, we'd get DoS attacks to the tune of 130K hits per hour, and that Sonic Wall shrugged it all oof like nothing. For a while, it looked like every script kiddie was trying to hack into our servers. No one got through, and we could care less how or what they were doing in their attempts. We had real work to do, that did not include fucking around with those idiots.

We also had a block list for customers from China. Not a single transaction from China was legitimate, all forged credit card data, so we just blocked their billions of potential customers because not a single purchase from China was real.

</comment><comment><USER35> For me monitoring the logs is already too late to do anything about the hacking attempt. Usually web servers write logs after the request was served. One approach (I have seen commercial products do it) is to hook into the web server code and inspect every request real time before it hits the application servers and make a decision if it is a legitimate request or not.

<comment><USER36> It does take a number of malicious requests before someone can find an oppening. If you ban them after the 1st or 2nd request, you will be safe. It is going to be a miracle if some vulnerability scanner finds and oppening after 1 request, not impossible but HIGHLY unlikely

<comment><USER4> > If you ban them after the 1st or 2nd request, you will be safe.

Sadly, it is too easy to get a new IP address now. I think as part of our security mindset, we should consider it trivial to scan for several hundred vulnerabilities using relatively unique IP addresses.

I'm not saying that you shouldn't ban traffic as it's required under several compliance programs and it is another form of demonstrated diligence. But, unfortunately, banning individual IP addresses only works against the absolute lowest hanging fruit of the criminal world.

</comment></comment></comment><comment><USER37> previously worked at a major shared webhost a few years back (which used Apache)

we used mod_security with common rulesets + a custom list which was reviewed/updated based on what hacks were coming in (with hacks both passively monitored by customer base/compromised spam reports and via active scans for malware deposited by attackers which were then cross-correlated/traced back to application logs to find the entry point/compromise.

support staff was ~600 (including front end call center and billing people), with probably 250 of those doing L2+ support and ~50 doing the security work which fed into the ruleset, for a server farm of ~2-4000ish (and exponentially more sites/accounts since it was shared webhosting). In some ways our network could have been viewed as a giant honeypot because of all of the outdated webapps that were being run by the customers, plus their compromised PC's saving passwords, etc.

<comment><USER36> I see a lot of mod_security misconfiguration across hostings. While I think it is a great tool, these misconfigs make it difficult for me to recommend it.

</comment></comment><comment><USER38> What is "an attack?" You need to define this better to get more focused answers.

Log monitoring is good for finding bad access patterns, but can't be trusted to find a problem as it happens.

Look for repeated errors. Hitting the same service over an over? Login attempt failures from the same few IPs? It's either a script, or the Bing crawler. Write application level code and network level configuration to mitigate the problem.

How about legitimate users doing social engineering attacks on each other? Sybil attacks in ratings or voting? Attempt to reuse mobile purchase tokens on different accounts? Some new SSHD credential leak zero day? Using your cat picture upload service for storing pictures of dogs? Each of these need different approaches, and a good logging system that can change with your needs is super helpful.

But you can't read the logs in real time, looking for bad patterns. That needs customized analytics.

</comment><comment><USER39> We use a combination of [https://www.templarbit.com](https://www.templarbit.com) and Splunk. Templarbit allows us to get real-time notifications when someone is running a vuln scan, violates one of our security headers or is trying payloads that look funny.

</comment><comment><USER40> Fail2ban looking for web pages that don't exist on your web site (like ../wpadmin) works pretty well at getting rid of script kiddies.

I suppose you could add a honeypot page that took form submissions and looked for any sequence attempting a CSRF, XSS, or SQL attack and ban the source there as well.

</comment><comment><USER41> This is a list of what you can do for application security with Nginx (mostly with open source tools): [https://github.com/wallarm/awesome-nginx- security](https://github.com/wallarm/awesome-nginx-security)

My talk from Nginx conference: [https://www.nginx.com/blog/build-application- security-shield...](https://www.nginx.com/blog/build-application-security- shield-with-nginx-wallarm/)

Important note. Care about vulnerabilities. Not about attacks. Buy Burp license. Run appsec training for all of your developers; it's easy while you're small.

Disclaimer: I am a co-founder of Wallarm mentioned in preso.

</comment><comment><USER12> I used to spend a lot of time combing my logs with custom scripts, but honestly I think the effort is better spent elsewhere. There're countless automated attacks a day on a big site but they really aren't anything you need to worry about.

</comment><comment><USER42> I've learned that at the very least to ensure that everything is being properly backed up and that the restore works. Testing this regularly puts my mind at ease that even if my site, application, whatever gets destroyed, stolen, etc I can bring it back to life in a very short time. It's very difficult to ensure that something is 100% secure (not even sure if possible since the major players like Twitter make mistakes). Of course, this doesn't solve the issue of data being taken but that's why it's important to understand the data you actually store and if you truly need to store it or if some could be handled another way.

</comment><comment><USER43> There's really no way to eyeball logs in any sort of complex environment. What you're looking for is a system that will automatically analyze logs and/or traffic for malicious events and alert on them. What you're looking for are Introsion Detection Systems (IDS) and Security Information and Event Management (SIEM). For IDS there are a whole bunch of open source solutions. Essentially these systems ingest logs into a centralized location and run them through a rules engine, which gets updated with new rules on a weekly or daily basis.

<comment><USER44> > _What you 're looking for is a system that will automatically analyze logs > and/or traffic for malicious events and alert on them._

(I don't pick on you, so I hope this comment doesn't come off that way.)

What are "malicious events" and how will you know them when you see them?

I would argue that, instead, your system should automatically analyse logs for benign (a.k.a., completely normal, expected, innocent) events, toss them out, and leave humans to review anything that's left over.

With any new "system", it will take a bit to build up a complete "whitelist" so that the amount of data to review is down to a manageable level but you'll be rewarded with much more meaningful and actionable data when you get there.

<comment><USER45> > What are "malicious events" and how will you know them when you see them?

I suspect they are simply defined as "rare" events.

</comment></comment></comment><comment><USER46> fail2ban, but I'm not sure that's what you are looking for. You'd have to customize it for an API.

</comment><comment><USER47> Ha I remember awhile ago doing something similar with our AWS server logs, turned out we were always under attack 24/7, rather amusing but not very useful apart from scaring juniors a little bit.

</comment><comment><USER48> 2nd'ing fail2ban, it's simple and awesome.

Beyond that, tripwire or it's modern equivalent. Traffic monitoring, "deep packet" inspection on the network side. Run all outbound through a proxy.

<comment><USER49> Nah, I've waned to use fail2ban multiple times and failed.

Unless something has recently changed it is some of the most poorly documented software on the planet.

I've never been able to configure the thing or figure out what it's doing or... really anything besides find the homepage.

It's also just not worth the trouble. Globally open SSH port? That's like 1992 level Linux box in your parent's basement stuff.

<comment><USER11> While it monitors SSH by default out of the box, people here are talking about creating custom filters which will monitor their Apache or Nginx logs. (Although, that said, we also have it monitor our non-standard SSH port with v. aggressive blocking rules, even though it's also behind a firewall. Defence in depth, and all that.)

The documentation isn't the greatest, but this page explains the concepts well enough and was easy to find using Google.

[https://www.fail2ban.org/wiki/index.php/MANUAL_0_8](https://www.fail2ban.org/wiki/index.php/MANUAL_0_8)

<comment><USER49> I understand the concepts but... how do you actually use it?

<comment><USER11> A filter matches lines in the nginx log using regular expressions. If the line matches, it uses another regexp to extract the IP address, and then calls out to scripts to block that IP address.

I'm not going to post the exact configuration files I use, but the GitHub repo for fail2ban contains examples.

[https://github.com/fail2ban/fail2ban/blob/0.11/config/filter...](https://github.com/fail2ban/fail2ban/blob/0.11/config/filter.d/apache- shellshock.conf)

That's a filter that protects Apache against the 'shellshock' ([https://en.wikipedia.org/wiki/Shellshock_(software_bug)](https://en.wikipedia.org/wiki/Shellshock_\(software_bug\))) vulnerability, for example.

</comment></comment></comment></comment></comment><comment><USER50> Dont do this in production without backup tools but we use unsupervised learning for anomaly detection. We're machine learnists so this is natural for us and we originally built the system for a mega Corp and made it open source (though not all parts are yet public). Catch is it's not real time but he system outputs outliers every few hours or daily depending of the size of your traffic (daily for around 50M proxy daily logs). Outliers are then reviewed by scripts and few operators / analysts.

</comment><comment><USER51> Computers are better at the eyeballs-on-logs tasks. You teach them what is "normal" to be ignored and they may warn you about what is left, even the old logwatch is pretty capable for that.

Of course, there some basic security practices. Firewall, fail2ban or similar to stop/ban misbehaving IPs, monitoring systems metrics to detect abnormal activity/usage (not just for intrusion detection), and maybe feeding logs to i.e. ELK to notice new/unknown patterns.

</comment><comment><USER52> I use logwatch. It gives a large informative report every day. Mostly it shows me that fail2ban and my secure configuration are doing what they are supposed to do.

</comment><comment><USER53> There is a product called Unomaly [1] which does exactly what you wrote.

1\. [https://unomaly.com/](https://unomaly.com/)

</comment><comment><USER54> WAFs like CloudFlare help, but stuff still gets through. Last night an obvious brute force attack went undetected through CloudFlare. I just finished writing a post about it: [https://hauptj.com/2018/05/08/brute-force-login- attacks/](https://hauptj.com/2018/05/08/brute-force-login-attacks/)

</comment><comment><USER55> I use Fail2ban [http://www.fail2ban.org/wiki/index.php/Main_Page](http://www.fail2ban.org/wiki/index.php/Main_Page) for auto blocking.

I take an 80/20 type approach and see what types of access is not covered by current rules, then I add a custom rule to cover that type of access attempt.

</comment><comment><USER56> The most important things for this are: \- A good idea of what is normal traffic and behaviour on the web service. \- Good threat intel.

These 2 combined will make it fairly obvious what needs to be done, but obtaining these 2 items is not a trivial task, infact, they are both continuous processes.

</comment><comment><USER57> I maybe have a counter-question. Once you identify an attack, what do you do? Firewalling the attacker is an option but not really effective, the attacker might just change IP address and keep going. We've only inconvenienced them a bit. What else can be done?

</comment><comment><USER58> I work for: [https://www.signalsciences.com/](https://www.signalsciences.com/) and our tool is specifically designed for use cases like this. Let me know if you would like more info cody-at-signalsciences-dot-com

</comment><comment><USER59> > I'm genuinely curious on how do security focus companies discover things > like "oh, we got attacked by Bitcoin miners from North Korea". Are there > sophisticated tools that do this for you..

I work in a Fortune 100 Security Operations Center (SOC). We have a dev team that builds tools for our Incident Response team, Threat Intel team, etc, etc and an engineering team to manage those tools. And lots of other teams that specialize on lots of other things (malware reverse engineering, spam/phishing, red team, etc). We have to build a fair amount of our own stuff because we operate at levels above what a lot of commercial tools can do. We have hundreds and hundreds of custom alerts that fire based on various network traffic and sysmon activity that our threat team has written rules around.

This, of course, is out of the question for most.

So, what should you do? First, get your logs all in one place. The ELK stack is probably the easiest way to do that. ES makes things searchable and Kibana is a pretty good UI for a lot of this stuff.

Second, if you aren't monitoring your internal network, you really have no idea what is going on, so look at what you can accomplish quickly with distributions like Security Onion (which includes the ELK stack): [https://securityonion.net/](https://securityonion.net/)

Third, there's a fair amount of Open Source Intel (OSINT) on threats. You'll need to find a way to integrate that information and scan your logs for it. Malware, like Coinminers, will have certain Indicators that will tell you if they are on your network. If you're on the ELK platform, look at something like ElastAlert. You write rules in Yaml and then set up Alerts to fire into your Slack channel or Email you or whatever. There is no real shortcut here.

Commercial tools that can do a lot of this stuff includes FireEye's TAP, but they definitely aren't free: [https://www.fireeye.com/solutions/threat- analytics-platform....](https://www.fireeye.com/solutions/threat-analytics- platform.html)

And, as mentioned elsewhere, absolutely remove any attack surface that you can. Scan your IP ranges and domains with tools like Shodan.io, HackerTarget.com's DNS tools (DNSDumpster.com has a nice, free DNS search), etc and make sure you have a handle on what, exactly, you're exposing.

And, finally, if you're big enough then every single day a computer will be compromised in some way. There is no company in the world who doesn't experience this so you need to have a clear set of tools and playbooks to handle that. If you can respond to most incidents in days (not weeks), you'll be better than 90%+ of the companies out there.

Edit: clarified acronym

<comment><USER7> Im currently building a SoC for a fairly large company in Germany and before that I worked for what probably by now through M&A is the largest MSSP in the world. I dont know if I agree with you that having a MSSP is out of the question. we had plenty of clients who only paid us maybe 2 to 3 thousand a year if even that. Most MSSPs will charge on a contract to contract basis revolving around log volume and estimated daily event creation. If you are only creating an estimated .1 events per day you could very well have a rather cheap contract.

<comment><USER60> If it costs 2 or 3k a year your MSSP is probably not spending more than a few minutes a year understanding your logs. Realistically at that price the most likely number is zero minutes.

There is room in the industry for players that aggregate logs in interesting ways and can share threat intel and track attack patterns between customers, but a lot of the time with MSSPs you will just find yourself paying for a really expensive splunk instance.

Logs are fairly useless without occasional review (to at least verify you are collecting the things you think you are) and a decent understanding of the infrastructure they support.

I don't know your business model or capabilities though so this might not apply in your case. Hopefully not.

But there are plenty of MSSPs out there that will ingest logs, charge out the %]%{^{ for "monitoring" and add very little value. After the nth time you get alerts for PHP vulns on stuff which is clearly not running it, or something stopped sending logs months ago and no one noticed, or you find that firewalls logs from anything not in an unpublished support list don't really get parsed (yeah they will ingest it but just not tell you that nothing meaningful will happen with it..)...

To be fair, monitoring by third parties is one of those things that you can't "do and forget about it", you need shared responsibility and a clear understanding of who is doing what. None of this is cheap though.

</comment></comment></comment><comment><USER61> For web logs specifically, even if you had a system that would 100% tell you what is a hack attempt and what isn't, you would probably not want to look at the output.

The web server _will_ be hit by thousands of "hack attempts" a day. Most of them will be directed at software you don't even use.

You care about _successful_ hacks, not random attempts (targeted attempts may be different, but good luck finding them in the noise of web logs). Web logs are probably the noisiest, least useful place to look for those (although they can be useful to determine how an attacker got in/where they came from after the fact).

The only thing that could make sense for web logs is looking for brute-force attempts, but that is better done with application-level logging (and countermeasures like CAPTCHAs).

Things you could look at instead are e.g. firewall/netflow logs: If your server is making unexpected _outbound_ connections, there's a good chance it got owned and someone is now downloading their toolkit onto it.

</comment><comment><USER62> SpectX is a tool that can run fast sql-like queries on top of distributed text files. It was initially built to parse logs by an infosec team. Hope it helps!

</comment><comment><USER63> It's called SIEM, Security Information Event Management, there is a lot of companies doing this and providing cheap and expensive solutions.

</comment><comment><USER64> OSSEC is a nice FOSS tool for this

</comment><comment><USER65> Another question- how do you comply with GDPR and do that? What if a black hat sends you a SAR request?

</comment><comment><USER66> ossec + syslog , how you display the data (zabbix, elk, etc) is up to you.

</comment><comment><USER67> error/exception monitoring</comment>


Show HN: Crowdsourced Salary Predictions  (update) - <AUTHOR> https://salaryfairy.com

<comment><AUTHOR> When we were first starting out, the feedback from our initial show HN post tremendously helped us shape Salary Fairy. We want to show you how much it has changed over the last few months and hopefully get more of your feedback.

</comment><comment><USER2> Co-founder of PayScale here. Fun concept!

<comment><AUTHOR> Thank you!! If you have any feedback for us, it'd be very valuable coming from you.</comment></comment>


12 Facts About Entrepreneurs That Will Likely Surprise You - <AUTHOR> http://onstartups.com/tabid/3339/bid/10561/12-Facts-About-Entrepreneurs-That-Will-Likely-Surprise-You.aspx

<comment><USER2> It's interesting to see how different this picture is from the typical YC startup or many other similar lean startups we hear so much about.

'Ramen profitable' is very, very different, for example, when comparing two single guys hacking in a bedroom of a rented apartment with a dozen roommates to that of a married family guy with kids and a mortgage in Northern California.

Being in the second category, I can say that there are some advantages (focus, time management, experience), but large blocks of time and the ability to go without financial resources for months on end are not among them.

I'm really curious how the startups that apparently make up this survey make it work in the beginning.

</comment><comment><USER3> I would love to know the actual breakdown of the demographics of the sample.. and how that changed across industries.

I particularly wonder if you limit it to internet founders if the data would skew younger, and thereby affect other answers as well.</comment>


Ask HN: Consulting in tech. How to start? - <AUTHOR> So, I've been thinking about starting consulting (not freelancing).
I am working on my own company that sells a news API. We've launched a month ago, it's fully bootstrapped. I have money for a few more months but at some time might search for a "job".
Does anyone here works as a "self-employed" consultant?
How do you find your clients? <comment><USER2> _How do you find your clients?_

Always one at a time and usually painfully if you have to ask. Clients are generally a lemon market. Good clients (well paying with realistic expectations — born of professional experience —and regular work flow) already have a list of consultants who meet their needs and with whom they have working relationships. By default almost any client you encounter as a new consultant will be lacking in some or all of wherewithal to pay, realistic expectations, and experience. Sure sometimes they have lots of work but it will tend to be bad projects.

To put it another way there is not a lot of unmet demand for consulting and building a client list takes many years of relationship building, sufficient capitalization and luck.

Luck because clients come one by one. And the world is full of buses. Most people didn't appreciate Charlie. Because we had this in common we understood each other in a particular way and could cut out a metric ton of bullshit and just work together. Charlie ran a growing retail company and had several decades experience. There were lots of viable future projects of the type I was cut out for. A really good fit.

A few months in Charlie abruptly retired to his farm with an aggressive brain cancer. Dead six months later. That was my luck. It wasn't good just better than Charlie's.

</comment><comment><USER3> I'd recommend to read whatever patio11 wrote about consulting like [https://www.kalzumeus.com/2012/09/17/ramit-sethi-and- patrick...](https://www.kalzumeus.com/2012/09/17/ramit-sethi-and-patrick- mckenzie-on-getting-your-first-consulting-client/) or [https://www.kalzumeus.com/2012/09/21/ramit-sethi-and- patrick...](https://www.kalzumeus.com/2012/09/21/ramit-sethi-and-patrick- mckenzie-on-why-your-customers-would-be-happier-if-you-charged-more/) . Also, there is a monthly freelance thread here [https://news.ycombinator.com/item?id=23379195](https://news.ycombinator.com/item?id=23379195)

</comment><comment><USER4> What distinction do you mean to make between freelancing and consulting? Those terms usually describe the same thing.

[http://typicalprogrammer.com/how-to-start-freelancing-and- ge...](http://typicalprogrammer.com/how-to-start-freelancing-and-get-clients)

<comment><USER5> Consulting to me carries connotations of much higher skill and much higher pay by orders of magnitude.

<comment><USER6> _Multiple_ orders of magnitude? Work for a month, relax for a decade?

It's funny how people understand terms differently. To me, consultant says mostly "working for a large-ish consultant company on large projects for large companies", while freelance is "self-employed", working on your own.

<comment><USER7> I aim for 3:1 or so when working alone, work real hard for three months then have the rest of the year to pursue my own projects.

<comment><USER6> Don't you get tempted to just work the whole year (and the next two or three) and just retire?

<comment><USER7> Too many side projects... I get tempted not to work at all and cut my expenses even further. Also, I'd rather just cherry pick the good jobs instead of lowering the price to be busy all the time.

</comment></comment></comment></comment></comment></comment><comment><USER8> You can contact one of the big recruiting agencies like Robert Half and they will negotiate contracts for you. They will also take some unknown percentage above whatever hourly rate you negotiate with them.

However, if a client likes you they can hire out outside of the recruiter (usually by paying off the recruiter for the option.) I've gotten hired for a few jobs like this and then I had a network of clients in my niche who got me more work by word of mouth, without any recruiter involvment.

Other jobs I've gotten by simply being prepared for pure luck to happen, like the time when my landscaper said that his full-time employer needed an app. I already had my LLC setup, I had contract document templates ready to be filled in and signed. I had business cards. A single page static website to describe my capabilities. After taking some notes and spending a weekend building it - I walked into their office with a rough prototype of what I heard that they needed and after a second meeting I locked them in for a year contract at $90/hour. (The recruiters were giving me $50/$60 per hour and probably collecting $90-$150! (they don't really tell you).)

<comment><USER9> Margin is typically 35% (less if the contract is longer or if you're part of some bundle) so yeah, you bill at 90, that's client billed at 90 / .65 = 140 or so.

</comment></comment><comment><USER7> I've written a whole slew of blog posts on the subject, quite a few of which got plagiarized to greater or lesser extent. Enjoy:

[https://jacquesmattheij.com/categories/consulting/](https://jacquesmattheij.com/categories/consulting/)

<comment><AUTHOR> seems like in depth. thx

</comment></comment><comment><USER10> Some good reads for this topic:

[https://training.kalzumeus.com/newsletters/archive/consultin...](https://training.kalzumeus.com/newsletters/archive/consulting_1)

[https://chrisachard.com/how-to-find-consulting- clients](https://chrisachard.com/how-to-find-consulting-clients)

[https://tomcritchlow.com/2019/02/27/i-consultant/](https://tomcritchlow.com/2019/02/27/i-consultant/)

[https://news.ycombinator.com/item?id=18227768](https://news.ycombinator.com/item?id=18227768)</comment>


Bill Gross: The single biggest reason why startups succeed – TED - <AUTHOR> https://en.tiny.ted.com/talks/bill_gross_the_single_biggest_reason_why_startups_succeed <comment><USER2> Note that all the companies he mentions are B2C.

I see a strong connection between the importance of "timing" mentioned in this talk with the importance of riding a consumer "wave" mentioned by Justin Kan in his article "Why I Love B2B over B2C". In short, it's difficult to succeed without riding a huge wave. It's difficult to pinpoint waves in the first place because waves are only obvious in retrospect (i.e. when another company has had substantial success already).

</comment><comment><USER3> anti clickbaiting:

> The number one thing was timing. Timing accounted for 42 percent of the > difference between success and failure.

</comment><comment><USER4> unfortunate headline :) i'm sure anywhere between 99.999% and 100% of startups succeed without any help from bill gross.

</comment><comment><USER5> Ray Kurzweil has written extensively on the timing aspect. He attempts to identify the necessary prerequisites. I think it's an important question to ask when thinking of a potential startup: "Is the market ready for this?". Then equally important is not squandering that first mover advantage.

[https://www.technologyreview.com/s/402705/kurzweils-rules- of...](https://www.technologyreview.com/s/402705/kurzweils-rules-of- invention/)</comment>


A Large-Scale Analysis of Early Domain Deletions - <AUTHOR> https://www.usenix.org/conference/raid2019/presentation/barron <comment><USER2> > Bitsquatting it very similar [to typosquatting], but it involves registering > do- mains with an ASCII encoding one bit different from the target to > capitalize on random bitflips during the resolution process.

Wow, kind of mind boggling that one can profitably (?) capitalize on random bit flips. More details here: [http://dinaburg.org/bitsquatting.html](http://dinaburg.org/bitsquatting.html)

</comment><comment><USER3> [http://www.iana.org/assignments/registrar-ids/registrar- ids....](http://www.iana.org/assignments/registrar-ids/registrar-ids.xhtml)

How many registrar IDs does DropCatch.com need

Interesting how many registrars have been terminated

</comment><comment><USER4> Why cant ICANN itself start auction and back order service? In that way it can make good amount of money too.</comment>


Ideas are Worthless? Wrong! Ideas can solve problems - <AUTHOR> http://juwo-works.blogspot.com/2007/05/ideas-are-worthless-wrong-ideas-can.html <comment><AUTHOR> Agree/Disagree?</comment>


Why are there so many Russians in crypto? - <AUTHOR> https://qz.com/1267356/why-are-there-so-many-russians-involved-in-crypto-and-icos/ <comment><USER2> In Soviet Union, computers encrypt you.</comment>


Ask HN: Setting up training systems, solutions for new platform? - <AUTHOR> We have a new platform we are building for a customer, that will also be open to third party partners. It will need training, I'm looking for resources on setting up training. Hiring trainers, how much they should be paid, setting up curriculum etc. Breakdown on offering technical help (problems, more in depth solutions required would mean going to programmer, probably me so we want to keep this down, but also have some ideas as to what to charge in the eventuality) <comment><USER2> To me, this sounds like it's outside your organization's area of expertise. That suggests an outsource and markup approach to the service: find an org specializing curriculum development and delivery, get a quote, add overhead and profit and put it in a proposal to the client.

It will be easier to find a good organization than to build a new team because the best people already have jobs that pay good rates and specialized orgs know how to recognize and attract productive talent in the specialty.

Owning problems outside your core mission is a bad idea. Marking up high priced consultants is usually more profitable. Good luck.</comment>


The Yoda of Silicon Valley (2018) - <AUTHOR> https://www.nytimes.com/2018/12/17/science/donald-knuth-computers-algorithms-programming.html <comment><USER2> One the aspect I like the most about this man is how approachable and down to earth he still is after all these years. I'll give one example. Late on night, my friends were in the basement of the Gates building at Stanford working on their CS107 Heap Allocator project. Lo and behold, Donald Knuth walks by and sees them drawing all sorts of things on the white board in the hallway. "What's that?" he asks, to which my friend responds about the heap allocator project. "Oh, I know a thing or two about heap allocators; let me guide you" :)

edit: lo and behold!

<comment><USER3> I think you meant "lo and behold", but lord sounds good too in this context

<comment><USER2> ah! I wish I was this clever--sadly, I am just bad at writing...

<comment><USER4> It's a great story!

btw, "Lo and Behold (Reveries of the Connected World)" is the name of a documentary by Werner Herzog about the history of the Internet. "LO" was the first text sent on UCLA's Internet test. They probably intended to enter "LOGIN" but the network crashed. :)

[https://www.rottentomatoes.com/m/lo_and_behold_reveries_of_t...](https://www.rottentomatoes.com/m/lo_and_behold_reveries_of_the_connected_world)

<comment><USER5> Of course the phrase itself existed long before the documentary.

<comment><USER5> For example, the phrase goes back at least as far as Shakespeare.

</comment></comment></comment></comment></comment></comment><comment><USER6> The following anecdote about Steve Jobs is from [1]:

 _I was sitting in Steve 's office when Lynn Takahashi,. Steve's assistant, announced Knuth's arrival. Steve bounced out of his chair, bounded over to the door and extended a welcoming hand._

 _" It's a pleasure to meet you, Professor Knuth," Steve said. "I've read all of your books."_

 _" You're full of shit," Knuth responded._

[1] - [http://www.folklore.org/StoryView.py?project=Macintosh&story...](http://www.folklore.org/StoryView.py?project=Macintosh&story=Close_Encounters_of_the_Steve_Kind.txt)

<comment><AUTHOR> I think this is suspect on the face of it--was Steve Jobs known for fabrications? Giant dreams, yes, but known to say outrageous falsehoods?

Further, Donald Knuth, as is pointed out in another link on this thread, is quite humble and polite and unlikely to have called anyone full of shit.

This is uninteresting and very likely wrong. Let's not post junk like this.

<comment><USER7> >> was Steve Jobs known for fabrications? ... known to say outrageous falsehoods?

Just off the top of my head... He lied to Woz about being paid to develop break out and stole money from him to go to India; He presented essentially a block of wood as the finished, functioning iPhone in 2008-ish; oh, and he denied paternity of his daughter Lisa as in "I am not the father" even after a paternity test. The last one seems pretty outrageous.

I'm not convinced he viewed himself as outright lying; He probably had seen Knuth's books and in his brain that meant he had read and understood them in their entirety.

So maybe don't be so quick to dismiss this as junk. It's a pretty innocuous example but completely inline with his behavior.

<comment><AUTHOR> These make sense, but somehow they are more of the manipulative variety, whereas the incident with Knuth was outright bragging, which seems less like him.

<comment><USER8> Seems more like pandering than bragging. Or maybe just lying out of self consciousness.

</comment></comment></comment></comment></comment><comment><USER9> I first met Prof. Knuth at a conference in 1995 where James Gosling and I gave a talk about Java, and at the same conference I told folks that I was leaving Sun and joining a startup.

Three years later, while talking to Don at a picnic, he said, _" When I first met you I couldn't tell if you were really smart or really stupid."_ :-) He thought that being part of the original Java team would be the most exiting place to be. Then a couple of years later (2001, post dot com crash) he told me he had decided I had made a pretty smart choice, all things considered. That was a good day.

<comment><USER10> I want to mention you are one of the handful of people I recognize on this site so it's interesting to learn more about your background.

<comment><USER9> There is always this [http://mcmanis.com/chuck/](http://mcmanis.com/chuck/) :-)

</comment></comment></comment><comment><USER11> There are many more important things about Knuth and his work, but one of my favorite stories about him is that he showed up at Randall Munroe's Google tech talk [0], and, during Q&A, asked Munroe "Have you thought about animated cartoons?" and "What is your n log n algorithm for searching?"

I have a special respect for people who are both brilliant and humble, and Knuth is my poster person for that.

[0] [https://www.youtube.com/watch?v=zJOS0sV2a24](https://www.youtube.com/watch?v=zJOS0sV2a24)

<comment><USER12> 21 mts in: [https://youtu.be/zJOS0sV2a24?t=1288](https://youtu.be/zJOS0sV2a24?t=1288)

<comment><USER13> Headphone users beware

</comment></comment></comment><comment><USER14> Is he really? The title is so mis-leading. Coach of SV - Bill Campbell. Yoda of SV - Knuth? Not really.

Knuth is more like the Yoda of CS. What is SV really? He is so far away from all the SV excesses - greed is good, VC shenanigans, founder hubris, drinking cool-aid, becoming new defacto destination for MBAs instead of Finance, FANG mania and worshipping, Leetcode grinding to get into FANGs and their brethren to chase insane comps, starry eyed and naive entrepreneurs who want to change the world

Where does any of this overlap with Knuth. Lot of pirates in SV. Yoda like figure - I can't even think of one. If tweets are any indication, I would nominate Naval and PG as Yoda's of angel-funded/pre-series-A start-up landscape.

<comment><USER15> _What is SV really? He is so far away from all the SV excesses_

That's precisely the point: He's the icon of the ideals that SV _claims to_ espouse - but has obviously long since deserted.

That's why he deserves to be called Yoda.

</comment></comment><comment><USER16> I thought this was going to be about some dumbfuck VC but OK, for Knuth this silly headline is acceptable.

<comment><USER2> "who bears a slight resemblance to Yoda" \-- though I found this comment somewhat disrespectful, HE would be the kind of person finding this amusing.

</comment></comment><comment> ur-whale [http://archive.is/cIJxe](http://archive.is/cIJxe)

</comment><comment><USER17> Prior comments: [https://news.ycombinator.com/item?id=18698651](https://news.ycombinator.com/item?id=18698651)

</comment><comment><USER18> Obligatory mention: the Knuth Reward Check. [0]

I've learned about Donald Knuth and TAOCP about 22-23 years ago. Since then, I have dreamed of collecting one check and framing it on a wall in perpetuity.

"Intelligence: Finding an error in a Knuth text. Stupidity: Cashing that $2.56 check you got."

[0]: [https://en.wikipedia.org/wiki/Knuth_reward_check](https://en.wikipedia.org/wiki/Knuth_reward_check)

</comment><comment><AUTHOR> I have two (weak) connections to folks connected to Knuth. The first is in the bibliography for a paper he wrote in High School--check out Fibonacci Nim. I got to work with the author of the paper for a number of years. Exceedingly bright.

The other was a key part of the MWC story. Bob called up Knuth asking if he knew of any bright programmers and Steve had been a masters student of Knuth's.

</comment><comment><USER19> I have read parts of TAOCP vol 2 and the TeXBook. His attention to detail inspires me. I have found that whenever I pay close attention and immerse myself in details of a paper, shutting off my computer, it is very relaxing.

</comment><comment><USER20> I don't fully understand the warnings at the end and the refusal to cover ML.

</comment><comment><USER21> I had no idea Donald Knuth towers over most at a height of 6'4. Wow

<comment><USER22> He just towers over most, elegantly.

I remember working in an Internet cafe and in the evenings out of boredom would read Knuth. I used to visit uni libraries and hunker down with TAOCP for a few hours until cramp, physical and mental, became too painful. Oh my golly what a privilege.

These are the tablets of our age. You'll note these tomes are granted the only definite article in English.

But mainly the thoroughness. Every path is an avenue of thought in this vast mapping of computation. It's always graceful, terse, and full of pleasure. The guy can carve candyfloss with a jackhammer and weave a mountain out spider's web.

Gates famously said anyone who'd read the books would get an interview. Yeah! "So what about ..., pretty lovely eh?"

And he's still going strong. A true hero.

I wonder what he thinks about HOTT... any clues? A quick ddg didn't show anything :(

(edit:spelling)</comment></comment>


Chrome and Firefox sometimes display text unpleasantly - <AUTHOR> http://avodonosov.blogspot.com/2014/01/blog-post.html <comment><USER2> I tried his code.

It renders with proper anti-aliasing in Firefox 26 and Safari 7 on Mac OS X 10.9.

Perhaps his problem has its roots in his operating system, or rather browser developers' use of that operating system? I admit to ignorance about whether browsers are doing their own font rendering or if they are handing that off to the OS...

<comment><USER3> Looks fine on Chrome 32 / Win 7

Here's a fiddle for the lazy: [http://jsfiddle.net/garnwraly/CqW6k/show/](http://jsfiddle.net/garnwraly/CqW6k/show/)

</comment></comment><comment><USER4> Convenient link (unfortunately not linkified):

data:text/html,<h1%20style="font-size:300%25;color:%234f81bd;font- style:italic;font-family:Times%20New%20Roman,Times,serif">WINTER%20ACTION</h1>

</comment><comment><USER5> Have you disabled ClearType in the operating system and IE ignores that setting and uses ClearType anyway, while the other two browsers don't?

<comment><AUTHOR> Yes, you are right, others suggested to check that too, and enabling ClearType fixed that

</comment></comment><comment><USER6> Doesn't look like that on my machine. Anything more you want to tell us about your setup?

</comment><comment><USER7> Not only does it look OK in firefox for me (ff26, windows 7), but if I CTRL-+ to boost the text size, it always looks antialiased. So the supposition that it renders then scales is false.

</comment><comment><USER8> This bug is pending for almost a couple of years in Chrome - [https://code.google.com/p/chromium/issues/detail?id=137692](https://code.google.com/p/chromium/issues/detail?id=137692)

You can try opening this icon ([http://fontawesome.io/icon/windows/](http://fontawesome.io/icon/windows/)) in all three browsers to test - check for the smallest size. Works fine in Firefox and IE, not in Chrome

<comment><USER9> The bug suggests that it's Windows-only, and the bug also looks like it saw some activity last month. Has someone tried the --enable-direct-write flag that's mentioned in it?

<comment><USER8> Yes I tried it and it works fine once you enable that flag. But fonts look much thinner than what they should be

</comment></comment></comment><comment><USER10> Back in the day this was some kind of problem in Linux. I always had to do stuff[1] to get MSWindows fonts to the Linux OS. But I don't think anyone needs to do this anymore. Gotta know more about this person's environment.

1\. [http://community.linuxmint.com/tutorial/view/29](http://community.linuxmint.com/tutorial/view/29)

</comment><comment><USER11> I am marking this bug as closed: unable to reproduce.

Really, though, you should do a little more testing before you try to go public like this. This code renders almost the same for me on all three browsers: [http://i.imgur.com/fmEhGOB.png](http://i.imgur.com/fmEhGOB.png)

<comment><AUTHOR> OK. I am glad to hear this works for others.

I would now change the title to something like "Chrome and FireFox sometimes display text unpleasantly" but the edit link is not available.

<comment><USER12> When you do that, also add the words "for me" at the end. Thanks.

</comment></comment></comment><comment><USER13> Looks perfectly fine for me on firefox/windows.

</comment><comment><USER14> Windows 7 x64 : [http://i.imgur.com/fsXFgAw.png](http://i.imgur.com/fsXFgAw.png)

Chrome is a little choppy, but Firefox and IE 11 look exactly the same.

All browsers use hardware acceleration nowadays, so I'm suspecting that something is wrong with OP's graphics settings. Are you running those browsers in a VM by any chance?

</comment><comment><USER15> I dealt with this after a user complaint last week - The "-webkit-text-stroke: 1px" hack mentioned in the Chromium thread does indeed work, but only for headings. If you use a webfont for body copy, and absolutely need cross- browser support, probably just have to fall back.

</comment><comment><USER16> Looks perfect on chrome32/Ubuntu.

</comment><comment><USER17> Looks more like three is no anti aliasing. And as James say does not look like that for me.

</comment><comment><USER18> Maybe because proper font rendering trapped is behind patents? Chrome and Firefox probably rely on public system APIs. IE probably uses private APIs to render fonts. That's just my guess.

</comment><comment><AUTHOR> FireFox 26.0, 32.0.1700.76 beta-m, Windows 7

<comment><USER19> For Firefox at least, this is probably because you are not using DirectWrite - that in turn is probably because you are not getting D3D10 (you are probably using D3D9 or no hardware acceleration), the usual cause of this is due to out of date drivers or black-listed hardware. You can check what hardware acceleration you get in the graphics section of about:support.

If you are not getting HWA, you can try forcing it on by setting 'layers.acceleration.force-enabled' to true in about:config (I would do that in a throw away profile though because it might crash Firefox; that overrides the black list).

<comment><AUTHOR>
      Graphics

      Adapter Description	NVIDIA GeForce GT 230M
      Adapter Drivers	nvd3dumx,nvwgf2umx,nvwgf2umx nvd3dum,nvwgf2um,nvwgf2um
      Adapter RAM	1024
      ClearType Parameters	DISPLAY1 [ Gamma: 2200 Pixel Structure: RGB ClearType Level: 100 Enhanced Contrast: 200 ] DISPLAY2 [ Gamma: 2200 Pixel Structure: RGB ClearType Level:   100 Enhanced Contrast: 200 ]
      Device ID	0x0a28
      Direct2D Enabled	Blocked for your graphics driver version. Try updating your graphics driver to version 257.21 or newer.
      DirectWrite Enabled	false (6.2.9200.16571)
      Driver Date	7-23-2009
      Driver Version	18.104.22.16844
      GPU #2 Active	false
      GPU Accelerated Windows	0/1 Basic Blocked for your graphics driver version. Try updating your graphics driver to version 257.21 or newer.
      Vendor ID	0x10de
      WebGL Renderer	Blocked for your graphics driver version. Try updating your graphics driver to version 257.21 or newer.
      windowLayerManagerRemote	false
      AzureCanvasBackend	skia
      AzureContentBackend	none
      AzureFallbackCanvasBackend	cairo
      AzureSkiaAccelerated	0

<comment><USER20>
        Driver Date	7-23-2009

When will users learn to upgrade their drivers properly?

EDIT: For everybody who thinks this should be easy, what's the latest version of the NVIDIA drivers on Windows Update? Is it not "version 257.21 or newer"?

<comment><USER21> To be fair, I think the better question is when will driver vendors learn to have their drivers update automatically? Checking to see if your drivers are up-to-date and installing new ones is a tedious process, and I shouldn't have to do it just to have a usable machine.

<comment><USER20> [http://www.nvidia.com/object/nvidia- update.html](http://www.nvidia.com/object/nvidia-update.html)

> I shouldn't have to do it just to have a usable machine.

The complaint isn't that Chrome and Firefox don't work.</comment></comment></comment></comment></comment></comment>


Steam Greenlight - <AUTHOR> http://steamcommunity.com/greenlight <comment><USER2> Valve continues to innovate in the distribution market. Here's to the death of monolithic publishers and retail!

<comment><USER3> Except in this case, aren't they the monolithic "publisher"? Not technically publishers but they are now holding the gates to all the users who are not willing to register / pay on a separate site or run down to a store. Sort of similar to the apple app store.

<comment><USER2> Compared to self-published works, sure. But compare them instead to the old standard, the likes of EA, Ubisoft, and Activision, with their buyouts, exclusive contracts, editorial control, and very small volume of profits returned to developers. This is a big step forward, and will help break ground for others to follow in the future and take the ideals even further.

(Of course, the old publishers also made large initial investments in games/game studios, but there's good evidence that such investments are no longer necessary for quality games to be produced, and when they come with so many strings attached it's easy to see how the quality can suffer)

</comment></comment></comment><comment><USER4> Comes with the danger of a popularity contest... part of Steam's existing brand is its fairly strict curation.

<comment><USER5> As I read the page, this is in addition to whatever methods they have currently. A good thing.

Go Steam!

<comment><USER2> If your brand is based on strict curation, then accepting the union of the selections of your curators and popular demand removes that strictness. It didn't sound to me like they're accepting the intersection of the two methods.

<comment><USER5> The intersection of these two would be pointless. Even less stuff would get made. The goal of this is more money. Steam has proven sales way before the game is released.

<comment><USER2> Well, the original commenter might've thought that they were becoming _more_ selective, or even just letting the community select titles for final consideration.

</comment></comment></comment></comment></comment><comment><USER6> Are they going to put the next Half Life 2 episode on there? That would surely get the go-ahead from fans.

</comment><comment><USER7> Sounds like Kickstarter: Steam Edition.

<comment><USER8> Sort of yes. For many games its not the funding, but the exposure a kickstart campaign gives.

</comment></comment><comment><USER9> Will Valve be promoting Steam Greenlight using Greenlight itself?

If not, why not?

<comment><USER10> Steam Greenlight isn't a promotion platform, it's a tool for community-driven curation. It would be kind of hard to use Greenlight to vote when it's not live yet, and there would be no point in voting on Greenlight after it's live. :-)</comment></comment>


Trulia to go public at $369 to $422 million valuation - <AUTHOR> http://www.sec.gov/Archives/edgar/data/1349454/000119312512383756/d352334ds1a.htm

<comment><USER2> Finally an IPO for a startup that has real money making potential. Trulia is a great app. If any of you guys here are are Tulia employees, congrats on building a great product.</comment>


New Drag and Drop Voice IVR, SMS and Business Critical Email Platform - <AUTHOR> http://www.upwire.com/?hn-001 <comment><AUTHOR> The importance of reliable and dynamic communication channels is absolute.

Having to wait for a developer to make code changes to existing message flows takes hours, even days.

By using the Upwire platform you can have complex Voice IVR's, Two Way SMS and Business Critical Email channels setup in minutes. Better still you can change the content of your campaigns in real time with zero impact on connecting to your customers.

Take the load off your developers and get them working on the important stuff.

#CommsWithoutCode</comment>


Female genetic superiority: two X chromosomes beat an X and a Y - <AUTHOR> http://cbc.ca/radio/quirks/jun-20-a-cosmic-iceberg-visitation-female-genetic-superiority-and-more-1.5606487/female-genetic-superiority-when-it-comes-to-survival-two-x-chromosomes-beat-an-x-and-a-y-1.5617471 <comment><USER2> Would three be even better?

<comment><USER3> Good question. Klinefelter syndrome often leads infertility so in terms of passing off your genes it is quite maladaptive. In term of general health outside of reproductive health, the conclusion is a maybe. I doubt there is much research into covid-19, although it would be interesting to read.</comment></comment>


The Value in Wowing Your Customers - <AUTHOR> http://blogs.hbr.org/cs/2012/03/the_value_in_wowing_your_customers.html

<comment><USER2> I've read books on this subject, most notably, 'Free Prize Inside' by Seth Godin- a perfect sequel addressing 'the how', to the book about 'the why' to be remarkable: 'Purple Cow'.

Whenever I read on the subject it gets me to thinking about how my particular company might go about implementing practices & policies which encourage such exceptional customer experiences.

What I've come up with so far, is to invest in developing services that could be sold as 'add-ons', but which exist only to be offered as bonuses to customers when the time is ripe to affect & influence their opinion of us & the value we offer.

Of course, I aim to hire employees who will seek opportunities to go the extra mile, as do those in this article, but I also aim to empower them to& create a company culture that encourages such actions, and thus can be constantly innovating on the subject of customer satisfaction. In fact, our values statement says 'Our clients must be enthusiastically satisfied all the time', and our culture will hopefully facilitate that.

Seth Godin's book 'Free Prize Inside' covers the topic of employee involvement in this regard quite well, and I encourage anyone who finds themself inspired by this HN post to buy, read & give that book to an employee.</comment>


Ask HN: Migrating away from React after license decision? - <AUTHOR> About a month ago the Apache Software Foundation, made the decision to disallow the BSD + Patent license. This is precisely the license that React is using. There was discussion on this announcement on HN at the time. [1] Naturally, this also generated debate and discussion on GitHub as well. [2]
Facebook recently blogged that despite the change in status from ASF for the BSD + Patent license, they would not be making any changes. [3]
So, I'm curious to find out how many teams out there are actually going to take action based on the aforementioned events and current licensing status. Also, if you are migrating away from React, what are your plans going forward (Move to Preact, Vue, something else)?
1. https://news.ycombinator.com/item?id=14779881
2. https://github.com/facebook/react/issues/10191
3. https://code.facebook.com/posts/112130496157735/explaining-react-s-license <comment><USER2> as long as you are trying to build the next facebook or something similar to facebooks business i dont think you will not have any problem with react, legal at Netflix doesnt have a problem with so I dont see how it will be a problem to regular community. If the hell really broke loose though I would prefer Vue any day, it strips off all the complexity given by Angular/React and makes single page apps simpler. And Easy.</comment>