'Crack baby' study ends with unexpected but clear result - <AUTHOR> http://www.philly.com/philly/health/20130721__Crack_baby__study_ends_with_unexpected_but_clear_result.html <comment><USER2> Science vs. hysterical racism, yet again. There's a reason it's "crack baby", and not "cocaine baby". Helps imprison the right sort of person. ([http://en.wikipedia.org/wiki/Fair_Sentencing_Act](http://en.wikipedia.org/wiki/Fair_Sentencing_Act))

There's a much more dangerous drug than cocaine, for babies. ([http://en.wikipedia.org/wiki/Prenatal_cocaine_exposure](http://en.wikipedia.org/wiki/Prenatal_cocaine_exposure)) It's alcohol, a legal drug. ([https://en.wikipedia.org/wiki/Fetal_alcohol_syndrome](https://en.wikipedia.org/wiki/Fetal_alcohol_syndrome))

<comment><USER3> Alcohol isn't really that dangerous for babies either. The prevalence of fetal alcohol syndrome is less than 5% for babies of women who drink more than 14 drinks per week during pregnancy: [http://www.motherisk.org/prof/updatesDetail.jsp?content_id=2...](http://www.motherisk.org/prof/updatesDetail.jsp?content_id=299). There's very little evidence that low levels of alcohol consumption during pregnancy are bad for babies: [http://www.time.com/time/nation/article/0,8599,2023984,00.ht...](http://www.time.com/time/nation/article/0,8599,2023984,00.html). Indeed, the available data shows the opposite: babies whose mothers drink lightly during pregnancy have better balance and higher IQs: [http://www.cbsnews.com/8301-204_162-57589867/moderate- drinki...](http://www.cbsnews.com/8301-204_162-57589867/moderate-drinking- during-pregnancy-may-not-harm-babys-neurodevelopment). This is largely due to maternal predisposition; higher-IQ women are more likely to drink during pregnancy.

It turns out that the placental barrier is pretty awesome.

<comment><USER4> You're presenting facts, the "total abstinence" people don't like that.

I find it funny how the "total abstinence" people come up with such amount of unproven facts, like "any consumption of alcohol" will cause harm. You'll be hard pressed to find a mother that hasn't consumed any alcohol whatsoever during pregnancy in Europe.

This looks like it works as well as the "total abstinence" for sex works.

<comment><USER5> Can you really speak with such authority about all of Europe?

In Sweden, where I live, alcohol and pregnancy are not suppose to mix. For example, translating from [http://www.vardguiden.se/Tema/Gravid/Livsstilsfragor/Alkohol...](http://www.vardguiden.se/Tema/Gravid/Livsstilsfragor/Alkohol/) : "Since no one knows where the border lies, the recommendations in Sweden and many other countries are to not drink alcohol at all during pregnancy."

Or translating from [http://www.lakartidningen.se/07engine.php?articleId=12235](http://www.lakartidningen.se/07engine.php?articleId=12235): "It's been known for a long time that alcohol consumption during pregnancy can damage the fetus, and most women also stop drinking alcohol when they discover that they are pregnant."

I'm lead to believe that the other Nordic countries are similar.

<comment><USER4> You're right, that's the recommendation

But about this I'm not so sure: "most women also stop drinking alcohol when they discover that they are pregnant."

<comment><USER5> Just like I and others are "not so sure" about your original statement.

Again I ask, how well can you claim to speak for the entirety of Europe? My observation is that Nordic women abstain from drinking during pregnancy at least as often as women who are native born to the US.

What would you accept as evidence that your original statement is incorrect? Perhaps the journal article "Alcohol consumption among pregnant women in a Swedish sample and its effects on the newborn outcomes", which finds: "Before pregnancy, 89% of the women regularly consumed alcohol and 49% reported occasional or frequent binge drinking. Nicotine was used by 15% before and by 5% during pregnancy. During pregnancy, 12% continued using alcohol and 5% also admitted binge drinking." ?

Compare this to the statement in "FOCUS ON: Biomarkers Of Fetal Alcohol Exposure And Fetal Alcohol Effects", which says that "roughly one of every eight women in the United States continues to drink during her pregnancy". 16% is higher than 12%, though I don't know the error bars to tell if that's significant.

If you don't like self-reported tests, then perhaps "Measurement of direct ethanol metabolites suggests higher rate of alcohol use among pregnant women than found with the AUDIT—a pilot study in a population-based sample of Swedish women" is useful. It concludes "Twenty-six women (25.2%) were identified as possible alcohol consumers by the combined use of AUDIT and direct ethanol metabolites."? (Note though that hair cosmetics and other things can cause false positives in a direct metabolites test.)

I interpret this to mean that 75% of Swedish women certainly do not drink alcohol while pregnant.

In other words, it's very easy in Sweden to find women who don't drink while pregnant.

Then for Germany, in "Smoking, alcohol and caffeine consumption of mothers before, during and after pregnancy--results of the study 'breast-feeding habits in Bavaria" the researchers report "25.3% of the mothers reported any alcohol consumption during pregnancy, 69.0% of pregnant women were drinking caffeine-containing beverages. The consumption rates were reduced clearly during pregnancy."

And for France, in "Is pregnancy the time to change alcohol consumption habits in France?" reports that "A total of 52.2% of women indicated that they had consumed alcohol at least once during their pregnancy".

Why did you make such a large, encompassing statement about all of Europe, when (1) there is such a variable rate, depending on the country, and (2) it looks like better than even odds that a 'European' woman does not drink once she knows she's pregnant.

<comment><USER6> I agree with your broad argument, but it isn't clear to me if you are referencing actual studies. If you are, please at least state how you are finding them and/or the journals in which they are published so that others can do so too.

<comment><USER7> They're article titles, Google finds them easily.

<comment><USER5> Yeah, I thought the part where I said "journal article" sufficed, and that a trivial Google search would verify all of them. For what it's worth, I found the papers through PubMed searches.

I'm surprised sometimes the lengths people will go through in order to not admit they can be wrong.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> Bad story design... trying to talk about the typical result, then late in the article drop the bombshell "The team considers Jaimee and her mother, Karen, among their best success stories.". If they "have to" go anecdotal, they could have at least picked the median subject. Like doing a report on the health effects of smoking, and intentionally selecting the healthiest 99 year old smoker in the world rather than the most likely outcome.

The most interesting part of the whole situation can be summed up by one of the lines describing the babies, "nearly all were African Americans." Even since the first days, It never was about medical issues or science, just a sorta-stealthy way to bash black folks. You'll note there was carefully no outage at the time, or long term medical study, at white coke snorting suburban women, although the blood levels of coke the babies experienced probably were about the same in the end. By analogy it would be like creating a social meme and scientific study of the negative pregnancy impact of malt liquor consumption (by urban black women), carefully ignoring the consumption of fruity margaritas (by suburban white women). Because you can't bash black people unless you can "other" them first.

I guess the two startup lessons are if you're trying to make median situation analysis, policy, and decisions, and you must use an anecdote, don't chose an extreme outlier, use a median... unless you've got an axe to grind and you're trying to mislead people, in which case unusual sample selection can be a powerful tool to mislead people. Startup lesson two is one popular way to scam people is to play word definition games as a strategy for divide and conqueror, so look out for that gameplay technique, and/or use word redefinition as a weapon of your own.

<comment><USER9> _Bad story design..._

I disagree. If you're trying to tell a compelling story, you want to reinforce the story that you're trying to tell in every way that you can. That makes it more compelling, and sells more page views.

The danger being, of course, that making reasoned decisions based on compelling story lines is dangerous. But this article isn't in that business - it wants to sell page views.

<comment><USER10> Journalists have a responsibility to the public that goes beyond page views. That is what makes them journalists.

<comment><USER9> If you wish to be consistent and say that the authors of the articles in virtually every blog of note are not journalists, then I will agree.

Unfortunately lots of people who have nothing resembling "journalistic integrity" make a living that way. If you're bored, you can turn on Fox News and see how quickly you can identify one.

</comment></comment></comment></comment><comment><USER11> _Hurt 's study enrolled only full-term babies so the possible effects of prematurity did not skew the results._

Doesn't this mean that their selection sample effectively excluded any babies that had a noticeable physical reaction (other than having cocaine in their system) to the effect of their mothers' cocaine use?

<comment><USER12> Yeah, I don't really get that part either. Prematurity is something that is itself heavily studied; prematurity with crack seems like the logical thing to study.

<comment><USER13> > prematurity with crack seems like the logical thing to study

It is a logical thing to study, and most likely is being studied, by someone else. After all, this study wasn't looking at premature babies. That doesn't prematurity with crack isn't a problem, only a different problem.

<comment><USER12> My issue is less with what the experiment studied and more with how it was summarized (particularly by the headline).

</comment></comment></comment></comment><comment><USER14> tl;dr - "Poverty is a more powerful influence on the outcome of inner-city children than gestational exposure to cocaine."

<comment><USER15> Damn lies and statistics, though. Being poor does not cause lower IQ, nor does lower IQ cause you to be poor. However, you'll find a lot of lower IQ people at poverty level. You can't guarantee that any college candidate that is poor is not going to excel, so poverty, sex, race, religion, sexual orientation, should have absolutely nothing to do with acceptance into any organization, assuming that everything else is equal, which it isn't. Unfortunately clothing, shared experience and knowledge, language, etc. significantly influence testing and decisions.

<comment><USER3> So the kids in the study were selected because their parents were poor, and the average IQ of the kids in the study was around 80. That certainly signals a correlation, and since these are kids and don't have agency over their economic situation, it suggests a direction of causation too.

<comment><USER15> Logically money should have no direct affect on intellectual capacity (e.g. see my other argument about taking the money away and giving it to someone else).

In this case, I think the causation is in the other direction. Over time, people with less intellectual capacity will be less successful, and therefore poorer. That is rational. The other direction is irrational.

<comment><USER3> It depends on how you ascribe causation to hereditary factors. Does it make sense to say that a 4 year old is poor because of his low IQ? I don't think so, because a 4 year old has no agency over his economic circumstances.

<comment><USER9> _Does it make sense to say that a 4 year old is poor because of his low IQ?_

No.

But it does make sense to say that a 4 year old is both poor and has a low IQ because his parents do. And this applies whether you believe that his low IQ is due to heredity or a poor environment caused by his parents.

</comment></comment></comment></comment></comment></comment><comment><USER16> My old room mate did cocaine studies with pregnant mice. He'd inject pregnant mice with cocaine and then raise the mouse pups and place electrodes in on of the "pleasure centers" of their brains. He would then measure the curve of how hard the mice were willing to spin a wheel for a given amount of current into their brains.

The model was that mice willing to beat their bodies more for the same amount of stimulation were more susceptible to a wide range of addictions. He found that pups exposed to cocaine in utero did in fact as adults spin the wheel harder for a given amount of stimulation, indicating higher susceptibility to a wide range of addictions.

My old room mate would have really liked to perform the same study with nicotine, since many many more human mothers dose their fetuses with nicotine as compared to cocaine. For what the wild speculations of an experienced researcher are worth, he suspected that mouse pups exposed to nicotine would also be more susceptible to addiction (supposing he was actually measuring susceptibility to addiction).

However, politicians and lobbyists have made it much easier to get federal grant money for cocaine studies vs. nicotine studies, despite nicotine having a much larger impact on society.

On a side note: in 1999, 4.7% of US 8th graders were willing to admit to having used cocaine, so the test group appears to be below US averages for cocaine use, despite their mothers using cocaine. I imagine that being predicated upon having mothers caring enough to place their children in these studies, and mothers responsible enough to stay in contact with researchers, and the subjects knowing they were being studied, skewed the drug usage portion of the study. Nationally, (for those outside of medical trials) I can't imagine the cocaine use rate for those whose mothers used cocaine to be below the rate for those whose mothers did not use cocaine.

</comment><comment><USER17> This is getting into questions you can't ask, and IQ is a severely flawed metric, but I wonder what the IQs are of the parents, and if the children's scores are higher or lower, and if we can rule out heredity.

<comment><USER18> The results of IQ tests are strongly driven by poverty as well.

<comment><USER19> Related: Six Brain-Damage Scourges That Cripple IQ in Sub-Saharan Africa

[http://www.amren.com/news/2012/04/six-brain-damage- scourges-...](http://www.amren.com/news/2012/04/six-brain-damage-scourges- that-cripple-iq-in-sub-saharan-africa/)

<comment><USER20> That's a fun comment thread. Poe's Law illustrated beautifully.

</comment></comment></comment></comment><comment><USER21> "They found that 81 percent of the children had seen someone arrested; 74 percent had heard gunshots; 35 percent had seen someone get shot; and 19 percent had seen a dead body outside - and the kids were only 7 years old at the time."

That's not really much different than the national average for kids whose parents aren't crackheads. E.g. 1 in 20 kids see someone get shot every year, so you would expect that by age seven that 7 in 20 would have, which is in fact exactly 35%. C.f. [https://www.ncjrs.gov/pdffiles1/ojjdp/227744.pdf](https://www.ncjrs.gov/pdffiles1/ojjdp/227744.pdf)

N.b. that some of these statistics are pretty wonky, e.g. they count getting beat up for your siblings as assault, or getting flashed as being a victim of sexual assault.

<comment><USER22> _1 in 20 kids see someone get shot every year, so you would expect that by age seven that 7 in 20 would have_

No I wouldn't, because:

a) I doubt the age distribution for "kids who see people get shot in a year" is constant. The linked article doesn't include age distributions, but they surveyed children up to age 17, and I would expect many more 15 year-olds to see people get shot than 7 year-olds.

b) I doubt the system is stochastic. It's probably more likely for the 1 in 20 from year 1 to see another few of the 7 shots in those 7 years (due to geographic and socio-economic factors) than it is for the other 19.

In other words, even for a country where gun violence is rampant, a study population where 35% of children below 7 have seen someone get shot is far from average.

<comment><USER21> Good points. In terms of lifetime prevalence, that study says that, "Similarly, 3.5 percent of 2- to 5-year-olds had witnessed a shooting during their lifetimes, whereas more than one in five 14- to 17-year-olds (22.2 percent) had witnessed a shooting."

So the population in question is definitely no where near average, my was just that strictly in terms of the amount of violence witnessed, the differences probably aren't as great as one would otherwise assume.

<comment><USER23> What the hell!? One in five teenagers has witnessed someone being shot? I must live a sheltered life…

</comment></comment></comment></comment><comment><USER24> What's more likely, that seeing dead people and being poor lowers your IQ, or that having a low IQ means you'll end up poor?

They preselected a bunch of poor people, which means the low IQ could be explained by the fact that their existing IQ put them in this place in society along with the predisposition to cheap drug addiction.

<comment><USER25> It seems like what's most likely is that you didn't read the article correctly. They were measuring the IQ of the children, not the parents. These children didn't 'end up' poor; they started life that way. Their IQ was then 10 to 20 points below average by the time they were four, regardless of the parents' cocaine usage. Or are you suggesting that four year olds with low IQ are responsible for their place in society and the poverty of their parents?

<comment><USER24> IQ most likely has a genetic factor to it. Otherwise, evolution would be independent of IQ and our society would not have shown an increase in IQ over time (which studies have shown to be true). So no, I did not misread the article.

</comment></comment></comment><comment><USER26> I got the part about not seeing a relationship between crack usage and mental functioning. It's a very interesting result. What I don't understand is "Hurt and her team began to think the "something else" was poverty."

Is there any data referenced in the article to actually support that claim? It's the central thesis here, and I don't see any supporting argument. I see some text around seeing people arrested, dead bodies, and so on, but there are lots of poor rural kids who never see that. This is much more a function of urban poverty.

Maybe I missed it.

<comment><USER27> It wasn't in the article -- the past few years have been disruptive in the social sciences because of multiple replicated studies that show stress, including poverty stress (which is a well-quantified phenomenon) have a profound impact on working memory ("how much RAM your brain has").

Google it. I first read about it in the Economist, but really, it's where the conversation starts these days.

Brain damage is one of the main mechanisms by which poverty passes itself down to subsequent generations.

</comment></comment><comment><USER28> The lede is buried a bit:

"Poverty is a more powerful influence on the outcome of inner-city children than gestational exposure to cocaine," Hurt said at her May lecture.

</comment><comment><USER27> TL;DR "White folks attack wrong devil, again. Lots of black folks dead or suffering, again."

It isn't science that's racist. What's racist is the thousands of brilliant minds that took _THIS LONG_ to look in the right direction. Each one of them minutely racist on its own -- it was just one tiny blind spot. One tiny speck on the lens. On every lens.

That's all it takes to destroy a community. That and some SWAT boots.

</comment><comment><USER29> This isn't in fact news, it has long been known that there is no scentific/medical 'crack baby' phenomenon, as documented in the wikipedia article. [http://en.wikipedia.org/wiki/Prenatal_cocaine_exposure](http://en.wikipedia.org/wiki/Prenatal_cocaine_exposure)

What's perhaps more important to talk about is the demonstrated harm of poverty and violence on children.

</comment><comment><USER30> "Crack baby" ALWAYS was simply a synonym for "poor and black".

It turns out that racism harms its victims. Who knew?

<comment><USER23> Do you think racism harms crack users more than their crack use?

</comment></comment><comment><USER31> So, vitamins are actually _bad_ for you and there's really no such thing as a 'crack baby'.

It's a bad week for things I learned growing up in the 80s/90s ...

</comment><comment><USER32> The study outcome is _poverty_ has the bigger impact. Yet there is a lot of discussion about anything else like Alcohol etc. But think about the uncomfortable issue: What to do against poverty? Isn't it a hint that our concern should be "a human right to access minimum wealth" and how to enable it?

</comment><comment><USER33> FYI, the information starts at paragraph 18.

</comment><comment><USER34> _Of the 110, two are dead - one shot in a bar and another in a drive-by shooting - three are in prison, six graduated from college, and six more are on track to graduate._

It seems rather depressing to me that the college graduation rate by 23 is under 10%. Talk about different worlds

</comment><comment><USER35> I've read that it's notoriously difficult to measure the impact of crack on prenatal development because so many crack-smoking pregnant women also smoke cigarettes, and the latter is so harmful already that it's hard to isolate the effects of the former.

<comment><USER10> That doesn't make a lot of sense since there are plenty of pregnant tobacco smokers that don't smoke crack.

<comment><USER35> I'm not following; could you elaborate?

<comment><USER36> The control group should be smokers not non-smokers and if all the cocaine using group are also smokers the difference should be the effect of the cocaine.

</comment></comment></comment></comment><comment><USER37> [http://www.nytimes.com/2013/05/20/booming/revisiting-the- cra...](http://www.nytimes.com/2013/05/20/booming/revisiting-the-crack-babies- epidemic-that-was-not.html)

Great video from the nytimes on this story from may

</comment><comment><USER38> "Jaimee Drakewood hurried in from the rain" What? This is supposed to be a scientific report? A lay science report? A newspaper? Why does the very first sentence read like a badly written paperback?

</comment><comment> guard-of-terra I guess it's time for a War on Poverty.

Make poverty illegal @ Jail anybody caught poor.

<comment><USER39> I thought we started that program in the 80s?

</comment></comment><comment><USER40> _" Poverty is a more powerful influence on the outcome of inner-city children than gestational exposure to cocaine," Hurt said at her May lecture._

And yet somehow immigrants (Nigerian, Vietnamese, Romanian, Armenian) living in poverty somehow find a way to work hard enough to become successful. They make their own outcome. Why is it only the impoverished Americans that cannot succeed.

I think we have to look here at what it takes to succeed.

1\. Dedicated parent(s). Two of them (a mom and a dad) can do far more than one so probability of success rises if both stick around. These inner city kids almost NEVER have a dad in the picture. This leaves mom to work or she is a complete welfare loser and probably on drugs too.

2\. Standards/Morals. One bad choice can ruin a person's life. We all have turning points in life where we choose to take a hit off a pope, or pass on it. Have unprotected sex, or abstain. Assault someone, commit a robbery, ETC. These choices have a ripple effect on the rest of our lives.

3\. Hard work. Why work if you can live off of the dole? Why work for white people and help their society since they are so racist? (this is a common attitude).

Immigrants come to America with almost nothing. They see opportunity though. They work hard and succeed. They stick together. Dad does not leave, mom does not smoke crack or sleep around. They have values, morals and a strong work ethic. This is what separates immigrants living in poverty and your more typical inner city situation in Philadelphia.

</comment><comment><USER41> Whats the TLDR of the article?</comment>


Tcpdump is amazing - <AUTHOR> http://jvns.ca/blog/2016/03/16/tcpdump-is-amazing/ <comment><USER2> Tcpdump is amazing. The quickness that you can have with it over Wireshark is awesome. I love that it is a command line tool instead of a GUI tool, since I needed to analyze TCP packets for quick debugging purposes.

But for the people who are new(ish) to tcpdump have you heard of libnet and libpcap? You can basically build your own tcpdump! :D

I was amazed at the speed packets fire when you program it yourself in C.

See: [https://github.com/the-tcpdump-group/libpcap](https://github.com/the- tcpdump-group/libpcap) <\-- CAPturing packets [https://github.com/sam- github/libnet](https://github.com/sam-github/libnet) <\-- sending packets

Libnet tutorial that I used religiously: [https://repolinux.wordpress.com/2011/09/18/libnet-1-1-tutori...](https://repolinux.wordpress.com/2011/09/18/libnet-1-1-tutorial/#example-5)

<comment><USER3> Wireshark also comes with "tshark" which is a CLI tool that gives you access to display filters and some of the GUI features. In particular, it's great for scripting if you're trying to pull some stats while troubleshooting.

<comment><USER4> Tcpdump is cool, tshark is cool, but nothing beats raw libpcap for grabbing raw data off the wire. It is simply so stable at high throughput it's often the only option.

Wireshark starts to break down at a certain point. When that happens, I've found scapy (Python pcap parser) very useful. Once that breaks down, a good option reading the hex directly in C.

Once that breaks down... Well, there be dragons.

<comment><USER5> Got to add dpkt to this list for parsing pcap streams - allows you to write some awesome python analysis tools - e.g. For this 5GB packet capture find me all the streams where tcp packets were retransmitted more than twice.

<comment><USER4> Dpkt is also a very good, although I've had some bugs with it on Mac. Probably just odd network issues.

</comment></comment></comment></comment></comment><comment><USER6> >> _Also if you understand how to reason about the overhead of using tcpdump ( "below 2 MB/s is always ok"?), I would REALLY REALLY LOVE TO KNOW. Please tell me._

I think when Dick Sites from Google says he has a rule of "stay below 1% of overhead" when analyzing traffic on datacenter nodes he wants you to select the right tool for the job. In the context of tcpdump you can run it with so many options that makes it very powerful. But that power is dangerous in the hand of a novice user. A simple error in how you run it (maybe missing filters or too wide an address space) can cause you to shoot well above the theoretical 1% limit. But that's not the tools fault IMO.

Anyway this seems more theoretical, because in practice I'd prefer a hardware based network tap and analyze that without creating any risk to the live traffic

<comment><USER6> thinking about it further I wouldn't want even a 1% fluctuation caused by analysis since it distorts the data captured in statistics. And as a datacenter guy who hasn't written the software I might not know the sensitivity level that some of my software is depending upon (timeout, max- latency, etc) which could spin it well above 1% not because tcpdump but because of a domino effect.

So if it would be _my_ datacenter I'd have a rule of no analysis of non-tap data.

<comment><USER7> >I'd have a rule of no analysis of non-tap data.

This is a ridiculous rule, you are cutting off your nose to spite your face. The saying "perfect is the enemy of good" comes to mind.

To your above point

>maybe missing filters or too wide an address space

There is merit in just capturing a shitload more than you need (provided of course, you're not trying to cap a full 10Gbit) because it is often the filters that are the cause of any "performance" issues, however you define that.

<comment><USER6> the rule depends on context which I failed to mention. I agree with you in context of non critical infrastructure. Though I'm not talking about web apps in AWS but places where you still want to have your own datacenter like in a banking or telecoms, or insurance or medical environments.

EDIT: whether you allow any access to a node for whatever purpose other than the software that was meant to run on that node would probably depend on what damage is done if that node goes down. If the damage is a blip in statistic and you can live with that fine but that's not always the case

<comment><USER7> You're still overthinking things and dealing in hypotheticals it sounds like to me.

Almost the _only_ time you would want a physical tap is if you need a permanent tap capturing everything over a long term - often for the purpose of running through an IDS/IPS, an even then SPAN/RSPAN/ERSPAN works pretty well.

Even in those industries you mention, most of the other time you are doing a capture is to troubleshoot something, so you don't need to run things for a long time, nor does any "perf issue", which is overstated, probably matter, since things are possibly already half way to fucked. And the compliance argument doesn't hold either vis-a-vis installing tcpdump - your processes and policies would be written as such to allow for debugging (or should be)

</comment></comment></comment></comment></comment><comment><USER8> As a network engineer, tcpdump and friends are on my list of most frequently used applications.

If you're not a network engineer, you'd be amazed at how many times we get issues escalated to us exclaiming that "it's the firewall" and demanding that we fix it.

It's usually not the firewall, though, and unfortunately it falls on us to prove that that's the case. It's not always easy but, luckily, tcpdump and friends allow me to show that and I can punt the issues back to where they came from.

(Several years ago, I was able to prove it was a _customer 's_ on-premise firewall -- managed by them -- and not _our_ firewall based upon the packet timestamps and this little thing known as "the speed of light".)

<comment><USER9> I'm not a network engineer and oftentimes have to use tcpdump and mtr on behalf of network engineers combined with Chrome devtools to show application developers that there's nothing wrong with AWS or even the internal enterprise network and that it's closer to them. The hard part is trying to make these diagnostic workflows scale for tens of thousands of application developers when you have maybe a handful of people that can help them in this manner.

</comment></comment><comment><USER10> It's a pity the article doesn't touch on BPF. BPF is the magical bit in tcpdump.

[https://blog.cloudflare.com/bpf-the-forgotten- bytecode/](https://blog.cloudflare.com/bpf-the-forgotten-bytecode/)

<comment><USER11> She mentioned it, just not explicitly. BPF is the super efficient optimized compiler that tptacek tweeted about.

<comment><USER12> No, BPF is the extremely simple and limited virtual machine. The optimized compiler is in libpcap and does some neat stuff to eliminate redundant code. There are implementations of the pcap filtering language that do not compile to BPF, and there are other languages compiling to BPF which do not benefit from the compiler in libpcap.

It's worth noting that optimizer is also absolutely necessary since the design of the pcap language is such that very simple filters are easy, anything even remotely complicated becomes very verbose and repetitive.

<comment><USER13> I'm pretty sure the three things (BPF, tcpdump, and the pcap compiler) were developed in tandem, or, at least, the latter two were.

I had a job, about 15 years ago, hacking on a customized version of libpcap (mostly to do filter merges). There is a surprising amount of stuff going on there.

<comment><USER12> Indeed. There's a great talk about those early days at [https://www.youtube.com/watch?v=XHlqIqPvKw8](https://www.youtube.com/watch?v=XHlqIqPvKw8) ; I'd recommend anyone who is interested in the design and implementation of little languages to watch that. Few things will make a language designer throw out everything and restart from scratch, but I guess having Van Jacobson dismiss the initial design as unusable would do it!

But after that common starting point they quickly developed lives of their own, and should not be treated as a single unit. You'll get things like pflua as a completely distinct implementation of the filtering language compiling to lua instead of BPF, pfmatch with language extensions that are not really compilable to BPF, seccomp and other uses of BPF in the Linux kernel for things that have nothing at all to do with packet processing, a (e)BPF code generator backend in LLVM, and so on.

<comment><USER13> BPF and libpcap are certainly separable, but tcpdump is practically just the test driver program for libpcap. :)

</comment></comment></comment></comment></comment></comment><comment><USER14> Actually, you can decode a stream as HTTP traffic in Wireshark. It assembles related TCP packets and puts together the conversation from the viewpoint of the application layer.

<comment><USER15> even with SSL you can MITM yourself with it

[https://jimshaver.net/2015/02/11/decrypting-tls-browser- traf...](https://jimshaver.net/2015/02/11/decrypting-tls-browser-traffic-with- wireshark-the-easy-way/)

<comment><USER16> Nitpicky, but: this isn't MITM, this is the client side of the connection writing out the keys necessary to decrypt the wire traffic. This is important because the process doesn't change alter the network data in any way, whereas a MITM proxy would.

<comment><USER15> You're right. Not MITM but decoding tap.

If you want a true MITM proxy then I recommend Burp

[https://portswigger.net/burp/](https://portswigger.net/burp/)

<comment><USER17> unfortunately this is only for http

</comment></comment></comment></comment></comment><comment><USER18> You can use netsh on Windows without need to install anything external (beside capture file viewer - Microsoft's Message Analyzer - but that can be done on your workstation rather than servers)

      netsh trace start capture=yes IPv4.Address=10.2.0.1

      netsh trace stop

The last command will output path to .etl file containing captured packets.

[https://technet.microsoft.com/en- us/library/dd878517(v=ws.10...](https://technet.microsoft.com/en- us/library/dd878517\(v=ws.10\).aspx)

[https://isc.sans.edu/forums/diary/No+Wireshark+No+TCPDump+No...](https://isc.sans.edu/forums/diary/No+Wireshark+No+TCPDump+No+Problem/19409/)

<comment><USER5> What's really irritating about the windows stack is how difficult it is to dump local host traffic compared to the same problem on Linux. I believe it's because the loopback interface isn't as completely implemented.

</comment></comment><comment><USER19> I read the title and think "Ok, lets see what tcpdump has that wireshark doesn't" and what do I find inside ?

Article about wireshark :)

<comment><USER20> I guess the most obvious thing is what it not has, which is a massive GUI. It is much faster to work with when you know what you are looking for in my opinion. In most cases running wireshark without tcpdump first is really inadequate.

<comment><USER21> For wireshark, there is tshark for a nice command-line based version with none of the GUI overhead. But usually, I'd rather just use tcpdump.

</comment></comment></comment><comment> pi-rat I really love ngrep[1] for debugging network traffic (for smaller problems).

Give it a filter (BPF) and the pattern you're looking for, and off you go: $ ngrep -W byline -d en0 "INVITE" port sip and host sip.phone.tld

[1]: [http://ngrep.sourceforge.net/usage.html](http://ngrep.sourceforge.net/usage.html)

<comment><USER22> What's the advantage of using ngrep over tcpdump?

<comment><USER23> It is based around filtering content, not metadata. Breaks down with TLS though.

</comment></comment></comment><comment><USER24> Wireshark is great, but it's fundamentally an interactive tool; you fire it up when you have a problem to look into and close it down afterwards. Running wireshark/tcpdump all the time seems like it would generate too much data. I'm not sure how well Wireshark handles dumps larger than available RAM, either.

The author mentions the pcap filter language, but one of the misfeatures of Wireshark is that it has a _different_ filter language for the GUI filter box.

<comment><USER25> The capture filters and display filters are interfaces to two sort of unrelated pieces. Maybe the misfeature is that wireshark doesn't do good job of communicating that.

BTW, If you want constant monitoring for your applications, you can buy a shark appliance that you can always go back to to investigate issues.

</comment></comment><comment><USER26> Some Linux distros (Debian family) separate tshark from wrireshark which is great because that means you can install tshark without X-Windowsetc... I wish the other predominant distro family (Fedora) did the same.

I have not used tcpdump a lot, but I believe there is nothing it can do that tshark cannot.

Tshark can capture only the portion of traffic you want via filters, which can reduce the size of your pcap files considerably, and possibly haves less of an impact on performance.

When using tshark, make sure you capture the traffics to a file too, so you can go back to look at something that happened x seconds or minutes ago.

I understand the security concerns, but forms being able to debug with tshark (or an equivalent tool) is a very good reason for not using HTTPS internally.

<comment><USER11> > I have not used tcpdump a lot, but I believe there is nothing it can do that > tshark cannot.

How about go two months without a security vulnerability in packet processing logic? Drop slightly fewer packets? (Yes, dumpcap too)

> good reason for not using HTTPS internally.

Nevermind.

</comment></comment><comment><USER27> Tcpdump use the BPF syntax to filter packages. In the current linux kernel,BPF was implemented and extended as a kernel virtual machine,when cooperated with the perf module,they can be used for collect trace info of the system. see [https://lwn.net/Articles/599755/](https://lwn.net/Articles/599755/)

<comment><USER13> No, tcpdump uses libpcap to filter, and libpcap compiles down to BPF.

<comment><USER27> Thanks for your explanation,I'll dig it deeper.

</comment></comment></comment><comment><USER28> One of my favs, a tcpdump filter for HTTP including request headers:

sudo tcpdump -s 0 -A 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420'

This is useful for troubleshooting outbound requests that your backends are making. I've had the interesting logic explained to me but can't remember the details.

<comment><USER29> It's good to have things like that in your toolbox, but it's probably also a good idea to figure out how they work and what their limitations are going to be, otherwise you'll may hit situations where they don't work and you don't understand why.

So, start with the magik number. If you look up those 8-bit ASCII codes you'll see that it spells out GET followed by a space, which should give a clue as to how it's working. So it will capture a lot of HTTP requests, but it may not be getting them all.

</comment></comment><comment><USER30> Aren't tcpdump and wireshark some pretty interfaces to libpcap? Also linking the home page of tcpdump/libpcap [1], and check the documentation about the packet capturing.

[1] [http://www.tcpdump.org/](http://www.tcpdump.org/)

<comment><USER31> They're intelligent interfaces to libpcap. If you have a pretty interface, you get nice opaque data you'll have to understand yourself. With intelligent interface you can do high-level processing like "I only want to see GET requests".

But they're still fairly stateless... without some extra scripting you cannot do a query for "TCP connections with more than 3 restarts".

<comment><USER30> What I meant with pretty, is pretty for human eyes, unless you are able to read TCP/IP headers ( for example ) directly in hexa.

Regarding ther other topic, nothing stops someone to capture the packets to a generic DB ( sqlite or berkleydb ) to operates such queries. Looks like a weekend project!

</comment></comment></comment><comment><USER32> I started really using tcpdump when I discovered "-X" switch. It displays all the traffic as it happens in hexadecimal, so you can then use other standard unix tools (grep, less, redirection...) to check the traffic. For inspection recording is still better, but nothing beats "tcpdump -X" when you just want to know if the packages are arriving at some port.

<comment><USER33> You also may want to use the "-A" switch if you are looking for ASCII content, the command ngrep..

</comment></comment><comment><USER34> tcpflow is another useful tool. It's similar to tcpdump, but reassembles connections by sequence number. I've found it easier to use for application- layer analysis, where you care more about the data being sent than what literally appears on the wire.

</comment><comment><USER35> One thing that you have to watch out for with tcpdump in the field (and especially in production) is that it doesn't rotate its capture files by default, and so you'll eventually end up with very large capture files which aren't loadable in Wireshark (which struggles to load capture files of size ~= avaliable RAM).

See [http://superuser.com/questions/904786/tcpdump-rotate- capture...](http://superuser.com/questions/904786/tcpdump-rotate-capture- files-using-g-w-and-c) for an example of the required syntax.

Note that tcpdump's default behaviour here is better than Wireshark's; last I checked Wireshark just crashes when your capture file exceeds the available RAM. Again, you can enable file rotation, but many don't realize this until they have been bitten by this attempting to do an overnight capture of a production issue...

</comment><comment><USER36> I have used tcpdump in the past to capture traffic when I had physical access and ability to implement a hardware tap and analyze packets after feeding in packets with tcpreplay from another network's pcap file (IIRC). The point was to configure an IDS like Snorby using rules I derived from the packets I analyzed in Wireshark (from the resulting pcap file).

However, I haven't seen a need to use tcpdump in awhile since my problem domains have been quite different in that my focus back then was primarily network monitoring. Usually performance problems where I have worked have been easy enough to identify at a higher layer (e.g. n+1 select issues with SQL).

</comment><comment><USER37> Julia Evans' blog is pretty great overall.

<comment><USER38> Her talks are great too, I particularly enjoyed her strace talk [0].

[0] [https://youtu.be/4pEHfGKB-OE](https://youtu.be/4pEHfGKB-OE)

</comment></comment><comment><USER39> Here's my primer for anyone interested:

[https://danielmiessler.com/study/tcpdump/](https://danielmiessler.com/study/tcpdump/)

</comment><comment><USER40> Tcpdump and wireshark are built on top of libpcap. If you are interested in learning how to write programs in C or Go using libpcap check out these posts.

[http://www.devdungeon.com/content/using- libpcap-c](http://www.devdungeon.com/content/using-libpcap-c)

[http://www.devdungeon.com/content/packet-capture- injection-a...](http://www.devdungeon.com/content/packet-capture-injection- and-analysis-gopacket)

</comment><comment><USER41> Just yesterday tcpdump saved me an indeterminate amount of time working with a Horribly Problematic vendor's support team by allowing me to sniff the loopback interface to pinpoint which of two (closed-source) components was introducing a sizeable processing delay in our real-time network event management system.

I am grateful for the tool.

</comment><comment><USER42> You can capture with tcpdump while analyzing it with wireshark at the same time ([https://wiki.wireshark.org/CaptureSetup/Pipes#Remote_Capture](https://wiki.wireshark.org/CaptureSetup/Pipes#Remote_Capture)).

</comment><comment><USER43> If you are interested in tcpdump and use it for debugging, you might potentially also be interested in the Bro network monitoring system ([http://bro.org](http://bro.org)).

It gives you very deep visibility in the supported protocols, dumps easy to parse log-files by default (see e.g. [https://www.bro.org/sphinx- git/httpmonitor/index.html](https://www.bro.org/sphinx- git/httpmonitor/index.html) for HTTP information) - and it is fully scriptable.

(Disclaimer: I am involved with the project.)

</comment><comment><USER44> We use tcpdump all the time to capture traffic that we later analyse with Wireshark, so I'm a bit surprised by some comments trying to confront the two.

</comment><comment><USER45> I apologize if someone already mentioned it, but you can capture packets from Wireshark directly as well, without the need for tcpdump or tshark. However, this is only useful if you want to capture packets on the machine you are running Wireshark, obviously.

<comment><USER46> true, for machines without a monitor you normally use tcpdump and tshark then do local analysis with wireshark, plus wireshark is a little heavy for real time capturing sometimes.

</comment></comment><comment><USER47> With tshark used to be possible to capture packets using display filters, which was pretty handy in lots of situations. When privilege separation was implemented, the feature was lost(bug 2234).

I understand the importance of privilege separation, but I miss the feature.

</comment><comment><USER48> The problem is that encryption (https everywhere) spoils its usefulness for the larger part.

<comment><USER49> If you have the private key, you can decrypt it in Wireshark.

<comment><USER50> Inaccurate. Most browsers will now be using ephemeral key exchange. You pretty much have to configure one of the end points to dump session keys to a log file, then load that in to wireshark alongside the packet dump.

<comment><USER48> It would be nice if there was some automated way to do this.

<comment><USER51> You can use mitmproxy [1] to dump the TLS Master Secrets for all connnections it intercepts [2]. The dumpfile goes straight into WireShark.

Obligatory disclaimer: I'm one of the mitmproxy authors - happy to answer any questions.

[1] [https://mitmproxy.org/](https://mitmproxy.org/)

[2] [http://docs.mitmproxy.org/en/stable/dev/sslkeylogfile.html](http://docs.mitmproxy.org/en/stable/dev/sslkeylogfile.html)

</comment></comment></comment></comment></comment><comment><USER52> This person's blog in general seems pretty great. This was posted a while back from there: [https://news.ycombinator.com/item?id=8167546](https://news.ycombinator.com/item?id=8167546)

</comment><comment><USER53> tcpdump is fantastic; I often use it in lieu of Wireshark if I can. It's also a bit faster, which kind-of doesn't matter for me since I usually have it output the trace to a file and then use less to go through it.

windump is the windows version of tcpdump: [https://www.winpcap.org/windump/](https://www.winpcap.org/windump/)

I haven't used it yet but it's libpcap based so I can't imagine it being too different. It has to be at least 2000x better than the piece of shit Microsoft Network Monitor (it's like Wireshark, except so much worse...oh, and it doesn't do promiscuous mode)

</comment><comment><USER54> I love that the PCAP format has become the defacto standard for analyzer tools to use. I just saw an HN article about a new analyzer based on PCAP.

</comment><comment><USER55> If you are on Windows, I highly recommend starting with Netmon (Network Monitor) instead. It is 100 times easier to use than tcpdump and Wireshark.

</comment><comment><USER56> It's also great combined with something like CURL you can upload the captures to CloudShark and skip using Wireshark entirely.

<comment><USER57> What do you see as the advantage of using CloudShark over Wireshark? I see CloudShark costs between $200-$9000 a year and Wireshark is, of course, free so I'm wondering why CloudShark is preferable. Is it the collaborative aspect?

</comment></comment><comment><USER46> love this, I always thought tshark is similar to tcpdump for CLI, did not realize it knows more protocol than tcpdump, learned something new today. Thanks!

</comment><comment><USER58> Didn't use -n in any of the examples...please...

</comment><comment><USER59> particular advantages over wireshark?

<comment><USER60> tcpdump is just a way of capturing the traffic, it's not a great analysis tool. It's simple, command-line, and part of the base install for many distributions. Generally very useful when you're remote debugging where you don't have a GUI: the workflow is tcpdump -> download .pcap to local machine -> analyze offline with wireshark. Alternately, tcpdump -> analyze in situ with tshark. (You can also use tshark directly, but then you don't get a do- over if you want to change your display filters and such). Also, easily scriptable.

</comment></comment><comment><USER61> it's really amazing

</comment><comment><USER62> Tcpdump for president!

</comment><comment><USER63> For a moment I read Tcpdump as Trump</comment>


10 people who don't matter (2006) - <AUTHOR> http://money.cnn.com/2006/06/21/technology/10dontmatter.biz2/index.htm <comment><USER2> OOF:

 _In entrepreneurship, timing is everything. So we'll give Zuckerberg credit for launching his online social directory for college students just as the social-networking craze was getting underway. He also built it right, quickly making Facebook one of the most popular social-networking sites on the Net. But there's also something to be said for knowing when to take the money and run. Last spring, Facebook reportedly turned down a $750 million buyout offer, holding out instead for as much as $2 billion. Bad move. After selling itself to Rupert Murdoch's Fox for $580 million last year, MySpace is now the Web's second most popular website. Facebook is growing too - but given that MySpace has quickly grown into the industry's 80-million-user gorilla, it's hard to imagine who would pay billions for an also-ran._

As I remember it, Facebook didn't have wireless either, and had less space than a nomad.

<comment><USER3> I don't get it— do these people hit more often than they miss? Why do we still listen to them?

<comment><USER4> Well, if you look at the rest of the list, Zuck's the only one who really _stands out_ as a "they sure couldn't have got that more wrong".

<comment><USER5> Netflix CEO, Linus. Both are as relevant or more relevant than ever. Netflix is getting to be as big a force, in revenues and viewership, as any big cable company. Linux is very much still relevant in the server world and in the mobile and tablet computing world (android). And Linus' Git is growing in popularity (github?) and influence.

It's questionable whether the Vodaphone CEO is "relevant", but Vodaphone is definitely still a major competitor, they make more money than google and amazon combined.

<comment><USER6> Yes, plus Linus is relevant and important for _both_ linux and git.

Git has been hugely, positively disruptive in a way no Dvcs was prior.

And anyone (cnn evidently) that thinks Linux developers are fungible misses the point of having a fierce, opinionated, blunt semi-dictator running the show. Great things can happen. He's the anti-steve jobs.

<comment><USER7> "fierce, opinionated, blunt semi-dictator running the show." That doesn't sound like the anti-Steve Jobs to me.

His complete lack of (aesthetic) taste might make him the anti-steve jobs though.

<comment><USER8> He has aesthetic taste, just for other things. He's the Steve Jobs of system internals.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> Sparing the pain of paging through the list one javascript link at a time:

         1  Steve Ballmer, Microsoft
         2  Jeffrey Citron, Vonage
         3  Reed Hastings, Netflix
         4  Ken Kutaragi, Sony
         5  Warren Lieberfarb, HD-DVD promotion group
         6  Rob Malda, Slashdot
         7  Aurun Sarin, Vodafone
         8  Jon Schwartz, Sun Systems
         9  Linus Torvalds, Linux
        10  Mark Zuckerberg, Facebook

<comment><USER10> i was just about to do the same! :)

<comment><USER9> I might've let it alone if the Firefox AutoPager extension had worked on it, but no.

I fucking hate that linkbait shit.

<comment> eric-hu Do they do this so they can show ads on multiple pages? I couldn't tell since I'm always browsing with an adblocker on

<comment><USER9> That and simply inflating pageviews / counts / visits and user time-on-site, all of which are typical performance metrics used by websites.

</comment></comment></comment></comment></comment><comment><USER11> I'm impressed with the list actually. It's hard to create something like this- in order to be interesting you have to pick people others would consider relevant.

There are only two picks that are obviously wrong: Reed Hastings and Mark Zuckerberg. And, it's a testament to both of them that they succeeded in challenging circumstances.

And, they did a great job picking Ballmer and Torvald. Their explanation of why Torvald is included (open-source as a movement has out-grown just his brilliance) is very prescient.

<comment><USER12> I consider Torvalds one of the big mistakes on the list. This predates the rise of git, which is more innovative (though possibly less revolutionary) than Linux.

I wouldn't even write him off today, though it's hard to imagine what else he could touch that could be as world altering as Linux and git.

Also, this was before Android, which is now the most important mobile OS. So, Torvalds' impact continues to resonate and grow. I agree with them that Linux has outgrown a single person...and so, they may have been somewhat right, had git not sprung almost fully formed from that massive brain of his.

<comment><USER4> My gut instinct to the inclusion of Torvalds was "No, that's totally wrong", but on further reflection I think it makes sense. Linux is much bigger than him now. The whole point of the list is "In the areas where these people are considered 'important', who could I totally piss off and not see too many negative ramifications?'" I just don't see Torvalds wielding the kind of power or influence to really make or break a person/business.

Don't get me wrong, I'm not trying to besmirch Linus, I think he's pretty awesome. I'm just saying, after thinking about it some, I understand why he might be on this list.

And I don't know that I would call git innovative, _per se_. git wasn't the first dvcs, and it's not substantially better than some other systems (mercurial, for example). git's prevalence today is purely because it won the popularity contest, plain and simple (being Linus project certainly helped!). On top of that, while git may be a big deal for developers, git's importance to the world at large is _tiny_ \-- frankly I'd argue that github is more important than git itself.

<comment><USER12> _git's prevalence today is purely because it won the popularity contest, plain and simple_

I don't believe that is at all plain and simple (or correct). I worked with many DVCS systems before git. git was obviously better, from the moment I first tinkered with it, and the more I understood of its internals the more obviously better it seemed. There's a reason that after _years_ of DVCS fighting it out for turf and never really gaining any traction, git exploded onto the scene and took over a huge portion of projects. git got everything right...that's why it won the "popularity contest".

 _On top of that, while git may be a big deal for developers, git's importance to the world at large is tiny_

Developers make the world as we know it today. And git is what developers use. github is awesome, but git is the engine to github's fancy automobile exterior.

<comment><USER13> And people buy automobiles, not engines.

<comment><USER12> The internal combustion engine changed the world. It did it by way of automobiles (and other powered machines), but the internal combustion engine, in and of itself, was required for any of that stuff to come into existence. Likewise, git is the engine for numerous things.

I'm not sure I understand how _hackers_ can continually underestimate the impact of our own tools and technology. There are _millions_ of developers using git today, and the ones who use git are the most important and influential developers in the world. Sometimes I wonder if nerds even realize how much impact they have on the "real world"....

<comment><USER13> It is the _idea_ of the combustion engine, and the fact that it became technical possible to build them that changed the world. It's not any particular combustion engine, but the combustion engine industry as a whole.

Yes, the combustion engine is very important for a car, but it became a commodity.

Yes, DVCS changed the world. Git maybe more than others because of its popularity, but slowly I think they are becoming commodities. Github could function with any other DVCS and practically all major DVCS (git, mercurial, darcs, bazaar) are interoperable one with another, making the choice more personal than technological.

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> I'd like to play devil's advocate for a moment. Many of us would have agreed on Zuckerberg and Netflix on this list back in 2006 and probably on Linus as well. Some of us would have even been pretty vocal in defending Blu-ray and/or HD-DVD. Maybe at this very moment someone is not convinced that Steve Ballmer should be on the list.

The fact that 5 years later some events altered the course for some of these guys' projects doesn't negate on the arguments put forth to include them.

We already know what Hastings and Zuckerberg did to stay relevant. Torvald's inclusion is explained as being _by design_. That is, Linus himself is designing the growth of his baby toward as much self-sufficiency as possible. Lucky for us there was the Bitkeeper debacle and he had to pull his ass out to give us Git. Right now the kernel is at version 3.0, most distros out there (including Ubuntu) still uses something like 2.6.xx and people are nonetheless very happy. I'd say, seen from a purely Linux angle, it would make Linus a lot less relevant.

Now, something that would've made CNN look much better 5 years after posting this, would have been to moderate their zeal by excluding stuff like _"But as respected as they might be for their past achievements, their best days are behind them"_ , especially when covering the ever changing tech business.

<comment><USER15> It has been said many times, but Linux 3.0 is just really 2.6.xx+1 with a change in version numbering.

<comment><USER16> Exactly. Not to mention 3.0 has been out for less than a month. Are you really acting as if the fact that major distributions haven't switched yet is a point against it?

</comment></comment></comment><comment><USER17> I can't believe in 80+ comments no one has defended Ballmer. They basically predicted his imminent retirement but in reality Windows 7 was released and became the best-selling of version of Windows ever. Also, it looks as if he has locked in his CEO position of Microsoft for years to come. You don't have to be a Microsoft fan-boy to point out this article was wrong in their future prediction of Steve Ballmer.

<comment><USER9> Given that the userbase of personal computing devices (not necessarily PCs or laptops) is still growing and will be for a while, the point at which Microsoft offers an operating system that _isn't_ the "best selling version ever" will be a disaster for the company.

Truth is: it's very difficult to buy large classes of equipment without buying Windows due to OEM relationships, same as it's ever been. I've got Win7 that came with my current Thinkpad, but I've booted it maybe two or three times, getting as far as logging in. Otherwise it's strictly been Debian. Haven't even bothered setting up Win7 under VirtualBox yet.

I don't know that Ballmer's been a disaster for Microsoft, or if it's just running down the Christensen "innovator's dilemma" curve, but the company's best days are behind it.

In ten years I see it as a legacy provider of business servers to late- adopting companies, ossified industrial giants, some government sectors, and, principally, gaming and entertainment. Possibly merged with Sony.

<comment><USER17> I am not trying to argue he is awesome, he isn't, just that the list was wrong about him. There is no chance he will be considered a revolutionary but he has done an adequate job of maintaining against really strong competition.

</comment></comment></comment><comment><USER18> These lists are always bullshit, whether they're positive or negative. They're nothing more than a few random people's opinions assembled for the benefit of easy page views.

</comment><comment><USER19> For additional giggles, under the related articles section: 50 people who matter.

" _Rank: 14

Brian McAndrews

CEO, aQuantive

Why He Matters: McAndrews is the most important adman on the Internet, and his innovations have played a big part in making the Web an effective marketing platform._"

Today?

According to Google, " _Quantive is now Microsoft Advertising._ "

Talk about also-rans.

<comment><USER20> Microsoft bought aQuantive for $6 billion.

<comment><USER21> That'll do nicely, sir.

</comment></comment></comment><comment><USER22> The real point of this is: don't turn to CNN for your technology industry news.

<comment><USER21> Don't turn to CNN for your technology industry predictions. They're fine reporting facts, but predictions are hard for anyone!

</comment></comment><comment><USER19> The fact that Netflix is on the list amuses me as well.

Although I love how they decided to call it both ways by putting Blu-Ray _and_ HDDVD on there.

It is a little weird to see companies that I literally don't remember on it. Vodaphone and Vonage both ring very, very, _very_ faint bells in my mind.

<comment><USER23> _Vodaphone and Vonage both ring very, very, very faint bells in my mind_

I'm afraid that's a problem with your mind, not with Vodafone.

Wikipedia sez:

 _Vodafone Group plc is a global telecommunications company headquartered in London, United Kingdom. It is the world's largest mobile telecommunications company measured by revenues and the world's second-largest measured by subscribers (behind China Mobile), with around 341 million proportionate subscribers as of November 2010. It operates networks in over 30 countries and has partner networks in over 40 additional countries. It owns 45% of Verizon Wireless, the largest mobile telecommunications company in the United States measured by subscribers._

</comment></comment><comment><USER24> Save some time and go all the way to the last person on the list, Mark Zuckerburg.

[http://money.cnn.com/popups/2006/biz2/peoplewhodontmatter/fr...](http://money.cnn.com/popups/2006/biz2/peoplewhodontmatter/frameset.exclude.html)

<comment><USER25> "In entrepreneurship, timing is everything. So we'll give Zuckerberg credit for launching his online social directory for college students just as the social-networking craze was getting underway. He also built it right, quickly making Facebook one of the most popular social-networking sites on the Net. But there's also something to be said for knowing when to take the money and run. Last spring, Facebook reportedly turned down a $750 million buyout offer, holding out instead for as much as $2 billion. Bad move. After selling itself to Rupert Murdoch's Fox for $580 million last year, MySpace is now the Web's second most popular website. Facebook is growing too - but given that MySpace has quickly grown into the industry's 80-million-user gorilla, it's hard to imagine who would pay billions for an also-ran."

<comment><USER26> Don't discount also-rans. Google was an also-ran. In 2000, Apple was an also- ran. Microsoft was (and arguably still is) the _master_ of the successful also-ran.

</comment></comment></comment><comment><USER27> Not that it matters in all the noise here, but here's my comments on their picks:

Steve Ballmer: He got where he is entirely by riding on Bill Gate's coattails. He did and still does belong on this list.

Jeffrey Citron: Obviously a correct call now. I don't know enough to say whether I could have called this one correctly at the time.

Reed Hastings: Maybe I'm remembering wrong, but it seems like from the very minute on-demand media became really feasible, Netflix was dragging Hollywood along kicking and screaming. I'd say he should never have been on this list.

Ken Kutaragi: You can argue now about whether or not he has been influential in the last five years, but I know that at the time I definitely would have agreed with CNN on this one.

Warren Lieberfarb: CNN and I would have agreed and we would have both been correct.

Rob Malda: CNN called this one correctly. I probably would have disagreed with them at the time, and I would have been wrong.

Arun Sarin: Same as with Citron--obviously correct in hindsight, but I couldn't have called it one way or the other at the time.

Jonathon Shwartz: Obviously, in hindsight CNN called it correctly. At the time, though, I had really high hopes for Jonathan Shwartz. Regardless of how talented he may be, he was put in an almost impossible situation as CEO of Sun --too much damage had already been done.

Linus Torvalds: First, a personal disclosure--I am a huge Torvalds fanboy, and would certainly have disagreed with CNN on this one. That said, you could now make a good case that CNN got this one right. In the last five years, Linus really hasn't done anything big from a business perspective. Absolutely he has been busy creating software that people use and care about (Git and continuing Linux development). However, none of has recent activity has had a major economic impact. (Can you name an industry that's been disrupted by Git?) Even the major things that Linux has done in the last five years (Android, for example) happened without being driven by Linus. I'm going to say I was wrong and CNN was right on this one.

Mark Zuckerberg: Obviously wrong in hindsight. I'm not sure whether I could have done any better, though. In 2004 I definitely would have accepted the notion that Facebook was unlikely to be worth more than Myspace. I'm not sure whether I would have still thought that in 2006.

<comment><USER28> > Can you name an industry that's been disrupted by Git?

Well, it's certainly decreased the significance of commercial and closed- source development tools & support. It's also made the market for programer talent more transparent and meritocratic, thus potentially increasing compensation for talented developers. One would expect this to lead to shifts in the employment from other sectors into programming.

Doesn't that count as disruptive? Sure he didn't make a bundle in the process, but he did make the world a better place.

<comment><USER29> You really need to get out of the HN bubble. I could pick 10 random developers off the street and I'm guessing you'll get 1 or 2 that know what git is.

</comment></comment></comment><comment><USER30> This is a trite and insulting article. Saying an individual "doesn't matter"? There are very respectable people in that list, and they've all reached a very successful point in their careers. They definitely all have a lot of power to do good in the tech world, and try really hard to do so.

Maybe it's just a semantics thing...but I find saying someone "doesn't matter" is way over the top.

</comment><comment><USER31> Thanks for posting this. By far my favorite is the Facebook versus MySpace. It's amazing how quickly things change.

</comment><comment><USER32> One name (Zuckerberg) tells how foolish anybody who predicts future can seem. And therefore tech journos almost always do. 'Idiots just shutup' I say to them!

Another one (Torvalds) screams of _insensitivity_ to the real impact created by some one. The journalist(s) deserves a whack in the back for this.

There are many people who do something noteworthy, and then fade away, for various reasons. Calling them out, in a derogatory manner is bad. 'Idiots just go away. Look at your loser self, in the mirror, when creating a list like this' I say again.

<comment><USER33> Most of Zuckerberg's $13 billion net worth is based on an imaginary valuation of Facebook, apparently engineered by Goldmans. There's still plenty of opportunity for Facebook to disappear into obscurity, overshadowed by the next big thing, just like Myspace, Mixi, Friendster and Livejournal.

<comment><USER32> What you say can happen. Agree with you. My main peeve is with any such list in the first place - "actors who don't matter...", 'techie greats who don't matter' etc. etc. Makes me very angry, purely from basic human feelings POV.

And its obviously a link bait - '10 people who don't matter...' easily attracts people on to read (or just page visit) '50 people who do matter...'.

</comment></comment></comment><comment><USER34> It's likely anyone who makes a "10 people who don't matter list" will look like an idiot in 5 years, because there's little insight to be gained from trying to understand the tech scene as a series of people who "don't matter" -- a poorly-defined category of people whose "best days are behind them." And the inclusion of Torvalds is bizarre, regardless of whether he "matters." Everyone else on this list is a founder or an executive, working in worlds with entirely different definitions of success.

</comment><comment><USER35> "Ironically, the whole [Blu-Ray/HD-DVD] debate may well be pointless. There's little evidence that consumers are eager to upgrade their existing DVD collections, and by the time the latest format war is settled, most of us will simply download movies in our living rooms instead of hoarding them on little plastic discs."

The above was a pretty accurate call, minus the surrender of the HD-DVD camp — and I love how Netflix, far from becoming irrelevant, became one of the top players in the streaming movie camp!

<comment><USER36> I think the jury is still out on netflix. They are king of the hill today, but there's lots of companies with deep pockets that want to change that.

</comment></comment><comment><USER37> Reed Hastings and to some extent Linus Torvalds are a bit ironic as well.

<comment><USER2> The logic behind putting hastings on the list (that Netflix would be eclipsed by VOD) is funny. I think the hook for Torvalds being on the list was uncharacteristically insightful for Business 2.0, though.

<comment><USER37> Yeah, that's why I said "to some extent" for Torvalds. Their reasoning was sound, but he's still far more influential than that reasoning predicted.

</comment></comment></comment><comment><USER38> Haha: " After selling itself to Rupert Murdoch's Fox for $580 million last year, MySpace is now the Web's second most popular website. Facebook is growing too - but given that MySpace has quickly grown into the industry's 80-million-user gorilla, it's hard to imagine who would pay billions for an also-ran."

</comment><comment><USER39> I think Ballmer at number one makes the list. Anybody trying to make a statement at any point in time about which tech products will be relevant in the future will be hard-pressed to hit better than 50%.

Microsoft repeatedly throws good money after bad, chasing the latest fad but never getting any of them completely right or, more importantly, creating any game-changing innovations to speak of. They saw Sony making the PS3 and decided they had to build a competing product and threw away billions in the process. They saw Google, and assumed they needed to build the same search engine to remain relevant. I can't believe that Ballmer has managed to stick around as long as he has, but if I were a Microsoft employee or shareholder, I would want his head to roll.

</comment><comment><USER40> Steve Jobs might've been in this list if the article was written 10 or more so years earlier.... And look what happened instead!

</comment><comment><USER41> To be fair to the writers of this article, M. Zuckerberg took the criticism of himself seriously and worked hard to become a great executive -- the news affected reality. One could argue that if all the press about Facebook (and Zuckerberg)at the time was positive, Facebook might not be as successful today.

</comment><comment><USER21> Tough job making any prediction like this.

They were right about Aurin Sarin and Sony - but for the wrong reasons.

</comment><comment><USER42> The Business 2.0 Magazine staff are missing from the list.

Also: stop trying so hard to come up with justifications for idiotic choices like Mark, Linus, etc.

</comment><comment><USER43> > its[Netflix] raison d'etre itself - the DVD

Ha! Anyone who believes DVD is what Netflix is still about is either so far out of touch they should be banned from spreading their ignorance publicly or are on the payroll of the the dead but don't believe it yet media industry who still hopes "the Internet" will just go away.

<comment><USER44> you realize this was posted in 2006, right (which is the point)?

<comment><USER43> Nope, author(s) weren't worth paying attention to then either.

</comment></comment></comment><comment><USER45> Well, folks, when you're right 52% of the time, you're wrong 48% of the time.

</comment><comment><USER46> this just proves how fast things can change in our industry. don't be afraid of the criticism, because you never know what will happen in 5 years.

</comment><comment><USER47> Ahh..so this is what it looks like when you have egg on your face.

Why do people keep doing this? Over and over again.

<comment><USER12> Seven out of ten isn't all that bad.

Sure, they didn't realize what a force git would become, and Zuckerberg showed them a thing or two, and Hastings/Netflix turned out to be smarter than anyone gave them credit for. But, the rest of them...yeah, pretty good calls.

Some of their 50 people who matter were stupid choices, however.

<comment><USER48> > Seven out of ten isn't all that bad.

Actually, it's pretty terrible - bear in mind that they could have picked ten (reasonably well-known) people at random, and chances are 8, 9, or all of them would be "has-beens" in four years.

<comment><USER12> These aren't merely "reasonably well-known" people. They're people who changed the world in one way or another. Though, the people who had most changed the world back then, turned out to continue to change the world...their boldest choices turned out to be the most wrong.

So, you might be right. Where they went bold, they got it wrong. Where they chose people obviously in decline (Ballmer, Schwartz, etc.), they turned out to be right...but it wasn't surprising to anyone (me included, as I remember reading this article when it was new).

</comment></comment></comment></comment><comment><USER49> ahhh !! just what it would feel like to have your foot in your mouth !!

</comment><comment><USER50> who reads CNN Money anyway?

</comment><comment><USER51> My favorite in the "not-getting-it" department from this list is Reed Hastings of NETflix. Way back in 2003, I remember being struck by how brilliant the name NETflix was. If I'd had the idea for DVDsByMail, I would have opened DVDsByMail.com Hastings was smart enough to call the company NETflix, which works fine for DVDs by mail, but really shines as the name of a streaming media company.

So, I find it hilarious to see people writing him off presuming he didn't have a streaming media plan. It is in the name!

Also, Linus shouldn't matter. I've met him. He's a decent guy, and that he "doesn't matter" in the way Steve Jobs _does_ matter, is a good thing. Linux should be bigger than him... at some point if your platform is dependent on you that's a weakness in the platform.

</comment><comment><USER52> Anyone want to deconstruct this article. These are 10 people who "you can safely snub at conferences". But by doing so you are also submitting to CNN's authority.

</comment><comment><USER53> I can only imagine the dickbag who came up with this idea at the editorial meeting.

</comment><comment><USER54> this article is total BS (accept for Balmer of course) it's easy to say "timing is everything" after the fact, but Zuckeberg did an awesome job. as in, series of great and innovative moves .. not just good timing. and " Last spring, Facebook reportedly turned down a $750 million buyout offer...Bad move" Facebook made 2 bil in 2010.. what am i missing here?

<comment><USER55> You missed the (2006)</comment></comment>


Check out the outrageous perks top companies offer - <AUTHOR> http://www.recroup.com/perks

<comment><AUTHOR> Soo many awesome perks ...</comment>


Execute - <AUTHOR> http://executebook.com/

<comment><USER2> Looks interesting. I like the video, I felt inspired just watching actually. Except that the poor girl forgot her laptop on the beach.

$15 for eBook is a hard sell but I'll probably pick it up eventually if only because I'm one of those chumps that sits on ideas instead of executing. Reading this book will be a good excuse to not do anything productive.</comment>


Tempting Software Startup Categories - <AUTHOR> http://onstartups.com/tabid/3339/bid/11978/The-10-Most-Tempting-Software-Startup-Categories.aspx?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+onstartups+%28OnStartups%29&utm_content=Google+Reader

<comment><USER2> Guilty. Invoicing and time tracking fall under category #1 along with project management and bug tracking as one of those things that naturally come to mind when thinking about a startup to work on.

In my case, I'm glad I started Ronin, despite the level of competition out there. I really felt (and still do feel) that I had something unique to bring to it. In my case, it was a certain workflow that I had experience with coupled with strong UX/UI experience. And, I felt the current players in the market were insufficent (for my tastes).

The real key here is to realize that in such a field, there is a large noise to signal ratio. Unless you can rise above that, it will just be 1 major player getting the lion's share of the users and the rest tripping over each other. (In my field, for example, Freshbooks is still the major player.)

On the other hand, if you succeed to the point where you are a level above the noise, you have a built in insolation layer that discourages overly simplistic and naive attempts at entering the market.

If you think you can commit to bringing a noticeable, appreciable and sufficient difference in quality, there's no fear in wading into a well- established field. Almost everyday I get emails from users that have noticed the difference in quality that we provide. However, if you're just there to throw up a half-assed attempt or to be a player with no discernible quality differences, I'd seriously reconsider.

</comment><comment><USER3> I've come back to this list several times since I first saw it last week. Doesn't it freak you guys out that you're mining for ideas within such a small pie-slice of the human experience?

There are lots of things that can bring a startup down. In my experience most founders are so close and in love with their precious idea that they only pursue evidence and opinions that support their position. So even if there is an opportunity to ask "is there a critical flaw that realistically should keep me from even starting it" most geeks will find a way not to take it. That's why Steve Blank's "customer development" reads as radical to many geeks but it's beyond obvious to people who develop products in the real world.

Fact is, most startup concepts are doomed before the first line of code is written; not because of poor execution, but because the ideas just aren't that great — and they only ask people who love them "too much" to be objective for their opinion. When I tell would-be clients that their idea is not going to be a home run, that they should stop while they're ahead and take the time and money the were about to throw away and take their family on a nice vacation... some get angry, others thank me, and one fella bought a third of my company.

TL;DR version: if your idea is on this list, you're not trying hard enough. Nobody needs your Basecamp clone, but everyone sits on chairs. Design a better chair.

<comment><USER4> Cloning an existing service is a sure way to fail.

Looking at how existing products service a market, finding their collective blind spot, and then kicking its ass; maybe you'll be successful.

</comment></comment><comment><USER5> I have fallen into the trap of trying to come up with an idea that is completely unique. Its tempting to look for new categories to attack, while dismissing categories with entrenched players as the lack of competition looks so appealing. In reality, unless you have some highly unique attribute to bring to market, trying to create the new market can be highly problematic. If it hasn't already been done, are your potential customers even looking for a solution to a pain? Also, are you certain that if you found a new market, that the lack of competitors isn't a sign that there really isn't a market?

Even though existing markets are full of competitors, the opportunities in existing areas are much more plentiful. Instead of looking for the next big thing, why not focus for a niche in an existing market, and grow it from there? The competitive landscape is constantly changing, and new opportunities are always being created as the market shifts like the tides. What it often takes is just a new twist on an existing concept. If you attack an existing market, all you really need is an opening through a target niche to gain a foothold, and from there, iterate as fast as you can.

<comment><USER6> I think that there are many, many markets that have not been noticed by most progressive developers and people who are interested in creating startups. (By progressive developers I mean those who develop really polished, usable software in the vein of web 2.0 - there is software for every conceivable market already, but there are plenty of markets that are dominated by hard-to- use, crappy software.)

I suspect these areas tend to be those that are not at all among the core competencies or hobbies of software developers, for obvious reasons. For example, I recall someone who started up a site devoted to exchanging knitting patterns (or something similar) that enjoyed a lot of success.

Just think about how many markets for software you could find if you just talked to __women __about what they want.

I bet there are tons of huge moneymakers lurking in fields that most software developers think are silly - like people who love to ride horses, or love to do crochet, or enjoy (or make a living from) any of a number of fields that are far removed from the average software developer's mind. Right now, those markets are dominated either by amateurs making crappy sites, or big established players with no creative spark.

</comment></comment><comment><USER7> This is actually a list of obvious and highly competitive categories. Perhaps the most highly competitive categories.

</comment><comment><USER8> Most of these categories are niches that the average person has some experience with and don't require specialized knowledge or connections to create a successful product. Most programmers don't know enough about non- programming fields to build a successful product, so they all tend to pile on in the few niches they understand. There are probably a lot of business opportunities out there for programmers who also have experience in non- software fields.

</comment><comment><USER9> I often get the sense that any idea I come up with, somebody with funding and development resources has already done it. Is nothing new in the world?

<comment><USER10> Yeah, great point. With so many bright minds out there, and the technology knowledge threshold becoming so much lower, it seems that all bases have been covered a few times.

But, there are many examples of companies that were not first, but who have become dominant.

Reminds me of a story I heard of two economists walking down the street. They spot a $100 bill lying on the ground, and one says to the other "Hey, a $100 bill. Should we grab it?" And the other says "Nah, it must be a fake. If it was real, someone would have picked it up already."

*edit for clarity

</comment></comment><comment><USER11> We should turn this into a startup/programmer decathalon. So far I've done 4 of these...to a kid with a hammer the whole world looks like a nail. Maybe I should write the others for fun.

</comment><comment><USER11> I've heard of machine shop operators that built their own machines. "All you need is a Bridgeport and a Lathe and can build the rest".

Also, I recently shopped for a year for a good time tracking/billing system. I tried many products but non were a perfect fit. Then in desparation I built one in a few days. It improved my skills and improved my code libraries.

(edit: I suppose it's different because I did a year of market research before committing to build something.)

</comment><comment><USER12> The cup is either half empty or half full. You can look at this list as a list of oversaturated markets or as a list of markets that have produced some great innovation and probably will continue to do so in the future. Hell, take Mint for example. They came after wesabe, fit squarely in the personal info space and are considered successful by most (not based on their sale, just the service). Looking at issue management: are Jira, lighthouse, trac, bugZilla, etc the best we can do? I doubt it.

Fundamentally I think any idea you are passionate about is worth pursuing. Nevermind the judgements of naysayers and lists of supposedly typical idea space.

</comment><comment><USER13> "I just find it curious that these specific themes tend to occur again and again."

The markets for each of these categories is huge with plenty of room for innovation in specialized niches. It makes sense to me to see plenty of competition in each category.

That said, they do tend to line up pretty well with the demographic involved in your typical internet startup:

Project Management/Bug Tracking = Developer

Discussion Forums = Gamer

Music/Events = Hipster

Dating and Match-Making = Single

And there you have your computer programmer with ironic t-shirt, no girlfriend and a level 70 mage in WoW.

<comment><USER14> Please, he should be lvl 80 if he's serious at all.

</comment></comment><comment><USER15> Personalized News Aggregation/Filtering: Having personally witnessed a few flame outs in this category I can honestly say though this area has lots of ideas and interest you will find very people willing to actually pay for personalizing it at any price point. Move along. No revenue to see here.

<comment><USER16> Unless you are going for mass market and can find a payer that is not paying for the service directly but rather for some kind of access to your customer. Ads come to mind, but there are other ways as well.

</comment></comment><comment><USER17> since when does "over-saturated" == "tempting"? But I think I get the point ... none of these are tempting to me (except maybe a twist on a dating site) but I suppose they are for some.

</comment><comment><USER18> It would be interesting to see a brainstormed list of the most _underserved_ categories. I remember that one of PG's essays has a list of things he thinks could use some startup love, but I'd be really interested in what other people think, too.

<comment><USER19> How about financial consolidation and reporting? Having had a fair bit of experience with "market leading" applications there is definitely a better solution out there waiting to be built.</comment></comment>


Using chip memory more efficiently - <AUTHOR> http://news.mit.edu/2017/using-chip-memory-more-efficiently-cache-hierarchies-0707 <comment><USER2> For those interested MIT 6.851 Advanced Data Structures covers Cache-Oblivious algorithms if your interested in the general CS/Coding techniques to maximize cache usage

OpenCourseWare [https://www.youtube.com/watch?v=bY8f4DSkQ6M](https://www.youtube.com/watch?v=bY8f4DSkQ6M)

</comment><comment><USER3> Take what is fixed and make it flexible.

Take what is one and make it many.

Take what is repeated and make it one, and fast.

<comment><USER4> Where is this from?

<comment><USER3> These are common patterns I have extracted from things that are innovative. Most of the time, the innovation is a simple perspective transformation.

Read 1000 patents and extract the abstract refactorings that made the idea possible.

</comment></comment></comment><comment><USER5> FPGA BlockRAMs have been configurable for the past two decades! The CPU architecture world is borrowing some good ideas from the FPGA architects.

<comment><USER6> To be fair, BRAMs aren't reconfigurable on the fly. That is, there's no assumption of data preservation when you change them.

Processors would need to temporarily hang to reconfigure hierarchy. Execution pipelines would stop issuing and resolve all in-process instructions and push registers to a stack, the memory system would need to quiesce the memory system and flush all structures, and finally the processor could rewrite low- level addressing crossbars. Once that's done, pop the registers back into the stack and hope there's no weird interactions with any number of legacy features in any older processor architecture. :)

<comment><USER5> That is a great point. Yes, most of the configuration parameters for FPGA RAMs are statically configured and would need to be "reconfigured" safely. Certain parameters are dynamically controlled (can be changed during execution without reconfiguration) and do not affect data safety, like cascade control if you want to stitch together multiple RAMs on-the-fly.

</comment></comment></comment><comment><USER7> I wonder if projects like openblas can make use of this. Maybe it could be baked into the cpu too somehow.

On the other hand, when reading the title I was preparing for a gory Amiga article. In this case it should be on-chip memory or is my non native english fooling me here? But it was at least in the same spirit ;)

</comment><comment><USER8> I wonder how the OS is supposed to deal with this complexity. It probably needs an API for processes to reconfigure the cache. But then the cache can be private to a process or to a group of processes. Complicated stuff.

<comment><USER9> Ideally, it will be selfcontained at the silicon level, perhaps with support for the OS to report statistics to the hardware.</comment></comment>


How PCI express devices talk - <AUTHOR> http://xillybus.com/tutorials/pci-express-tlp-pcie-primer-tutorial-guide-1 <comment><USER2> This is pretty cool, thanks for the intro.

The funny thing is this is _almost_ what I wanted the system interconnect to evolve towards back in the early 100Mbit ethernet days. After AVB and I wandered around the valley convincing the card people that you could signal over copper at 100Mbit, and they bit and started building cards, I started thinking about the future.

It was fairly clear that the hardware guys got it at 100Mbit, they could keep the packet format and keep making the interconnect faster and faster. The hump they had to get over was that the packet was the thing that had to stay the same, you can use whatever tricks you want to signal at the next speed level. So I was thinking wouldn't it be cool if they really pushed it so that it was viable for system interconnect. I half heartedly pushed for it inside of Sun but never got anywhere. At the time I was pushing for ethernet out to the disk drives (with power over ethernet so you just plug in more storage, easy peasy) but I definitely had the switched interconnect in mind as a replacement for the system bus.

So this seems like that vision, a switched network, but at a slightly different level than ethernet, it's got more end to end stuff in it.

I sort of wonder if we'll get there at some point, where the system interconnect is sort of like Formula 1 ethernet and the networking cards are like commercial car ethernet where some of the Formula 1 stuff has been trickled down to high volume stuff.

Eventually we'll run out of ethernet addresses though I bet anything we could just have a rollover, start reusing any address that is more than 20 years old (or for that matter, just have a block of addresses that are domain specific and are reused in each computer's internal interconnect).

</comment><comment><USER3> If the author sees this message, thank you! Just the kind of tutorial I was looking for!

I'd love to see one that built on this a little and showed a packet from kernel space in one device being sent over a PCIe NIC and then bring received by another device, and DMA'd from the receiving NIC into kernel space etc. I already know that whole process more or less but it would be nice to see it and have my understanding confirmed :)

</comment><comment><USER4> I love the comic on the right ([http://xillybus.com/media/comic-300px.jpg](http://xillybus.com/media/comic-300px.jpg)). Never heard of this software yet. Amazing that it is part of the Linux kernel (in terms of part of the Ubuntu distribution, for instance).</comment>