DNA in the dock: flawed techniques send innocent people to prison - <AUTHOR> https://www.theguardian.com/science/2017/oct/02/dna-in-the-dock-how-flawed-techniques-send-innocent-people-to-prison <comment><USER2> This seems like an ongoing problem with science and the criminal justice system, especially in the US. Science, and the trustworthiness of science, become an object of adversarial argument by non experts that have directly opposing interest, and which often face large disparities in resources with which to make that argument.

I had a discussion years ago at a conference with an EFF member that they really ought to (obviously easy for me to suggest) hold training sessions for judges on the basics of how technology and the internet works. Otherwise, the courts seem enormously susceptible to bad analogies ('an I.P address is just like a physical mailing address') that are used in place of actual evidence.

</comment><comment><USER3> The DA has an incentive to convict people. All the people who supply the evidence have incentives to help out. Who gets convicted isn't all that important as long as there's a pretty good correlation between a crime being convicted and someone being punished. Doing "my PHD is dependent on this and I'm not going to f-it up" work is time consuming and expensive. Whether the lab and investigative work done is high quality or sloppy it doesn't matter as long as the strong correlation between a crime being committed and someone being punished remains.

Something something metrics and targets.

</comment><comment><USER4> I took a genomics class. We had a guest lecture by: Frederick R. Bieber about medical Forenics. Fascinating stuff.

Its interesting the way they try to identify people is they pick 13 locations (more as of this year, the article says 16 locations for england) then count what are known as "Short tandem repeats"(STR). These STR markers exist on many chromosomes and are short sequences that repeat, everyone has a different count. (there is a standard for these locations in the US called CODIS: [http://strbase.nist.gov/fbicore.htm](http://strbase.nist.gov/fbicore.htm))

If you have good reads the probability of a random match is in 1 in 594 trillion. Its complicated as the DNA can degrade and the reads can be not so good.

Some places allow Forensic DNA analysis by data mining, in which case you may find relatives of a criminal. Some states require felons DNA to be added to the database. This can be searched though its controversial and familial searches are blocked by some states.

One case helped by this was Lonnie Franklin Jr. aka the "grim sleeper" who took a long time to apprehend.

[https://en.wikipedia.org/wiki/Grim_Sleeper](https://en.wikipedia.org/wiki/Grim_Sleeper)

"Police had found no exact match between DNA found at the crime scenes and any of the profiles in California's DNA profile database, so they searched the database for stored profiles that demonstrated sufficient similarity to allow police to infer a familial relationship. They found similar DNA belonging to Franklin's son, Christopher, who had been convicted of a felony weapons charge. "

Exonerations are possible using dna too: [http://www.brandeis.edu/investigate/innocence- project/dennis...](http://www.brandeis.edu/investigate/innocence- project/dennis-maher.html)

DNA analysis is just a tool, it can also lead to post conviction exonerations. I think its best used a evidence for building a case, not having the whole weight of the case on just DNA evidence.

</comment><comment><USER5> I'm wondering if it could ever get to the point where you should never give away your old clothes in case the next owner dies and you're incarcerated for murder.

Incineration-As-A-Service anyone?</comment>


Ask HN: Opportunities for combining programming with math/physics - <AUTHOR> Hi! I'm a software engineer, in my late 20's, located in Europe, working on Java enterprise applications.
I love physics, mathematics. I simply love the clarity of mathematics, the abstract yet elegant way of reasoning, and how everything falls together. I am also curious about the world around; and I simply find reasoning about the natural world, using mathematics is abstract way of thinking, and witnessing how this abstract reasoning leads to deep insights, later confirmed, an irresistible satisfaction. I am amazed about how many things we know about the Universe around us, and about how many things we discovered in the last century.
In my free time, I allocate time for learning mathematics, physics. But I'm worried about my future (will I be employable in my 40's, 50's?), and about the fact that I'm losing time instead of preparing for my career. I say to myself, that due to current problems (global warming, pollution) there will be a big need for physicists in the future, that by combining knowledge of math/physics with machine learning I will be able to track energy, new materials design, environmental problems, that quantum computers are the next big thing, and so on.
From your knowledge or experience, do you know if there are career path combining programming with mathematics/physics skills? Am I a dreamer in the sky? <comment><USER2> I was a 20 something developer, now, I'm in my early 30's and still on something similar.

If you find yourself enjoying math, cryptography and security may be for you. It's not a hype like 'data science' or 'AI/ML' but one look at the security community will tell you how important the area is. Give it a try, it may change your life.

</comment><comment><USER3> I don't have any direct experience here, but sensor design comes to mind. Satellite sensor design especially tends to be a secure job since they have long design/build times, and long mission lengths.</comment>


Data-Oriented Programming in Python - <AUTHOR> https://www.moderndescartes.com/essays/data_oriented_python/ <comment><USER2> I went in thinking this would be data-oriented in the ECS sense, ie., the way that modern games are designed to optimize class structure for cache lines.

However it's about the limitations of python's computational model (ie., how it imposes large constant factors on most operations, due to pointer indirection). And how numpy solves that in one case (homogenous arrays).

I was hoping it would include a discussion on the julia solution: a sort of partial static-typing, where type information is use to generate optimised code.

It should be possible for python to follow a similar route: an (optional?) compilation pass which uses type information to eliminate pointer indirection (, etc.).

However I'm not sure how much resistance the actual object model of the VM poses: would you, in effect, just be compiling to a different VM?

The present cpython approach to optimization is hacky and specific, eg.,

       sum( a * a for a in range(10) )

is faster than any equivalent loop/etc. approach, because `sum()` has a manually coded fast-path within its implementation.

<comment><USER3> Cython offers the type of targeted compilation options you suggest. Also numba offers them in a more opinionated way that requires less overhead than Cython.

These are both long standard tools in scientific Python.

<comment><USER2> My hope is that there's some easy, non-trivial, improvements in just basic type awareness, eg.,

        xs: List[Int] = list(range(10))
        total : Int = 0

should permit an optimised,

        for x in xs:
            total += 1

(almost?) as fast as the equivalent sum.

Python hasn't had typing for long enough for this thought to have percolated deeply, ie., I believe at the moment it is just unconsidered rather than infeasible.

I suspect that there's a few cases that can be hard coded into the VM that will make a big difference.

eg., namedtuple/dataclass, List[String], Dict[String, String], etc.,

<comment><USER3> Unfortunately what you suggest just isn't conceptually possible (by design) in Python, and would require removing many of the language features that make Python successful for a wide set of use cases.

Type hinting (eg List[Int]) has no control over the actual byte layout, it's just purely metadata. Nothing stops someone from changing xs[4] to be some complex object in your example, regardless of the type hint, so at a pure CPython level it's fundamentally impossible to benefit from static typing & optimization purely on the basis of type hinting.

This is why Cython and numba play that role. They change the location of type metadata - it's not mere runtime metadata, rather it is compile-time static type annotation in the real sense, for either ahead of time compilation (Cython) or just in time compilation (numba).

<comment><USER2> I'm not convinced. The compiler can emit different byte-code. The proposal includes adding a compiler-aware phase to processing type "hints" (here now, actual static types).

If you type code "xs: List[Int]" and try to change xs[4], the _compiler_ (phase) would have to reject your code.

This isn't "removing features" as typing is optional. And as long as a lists in "successfully compiled python program" express the same API, it doesnt matter that their implementations differ.

<comment><USER3> > " If you type code "xs: List[Int]" and try to change xs[4], the compiler > (phase) would have to reject your code."

This just can't work - because lists are mutable, so code that mutates them must be valid and permitted by any such compiler. Similarly, values are dynamically typed, so the value of the assignment might be an int or might not, and it cannot be decided until runtime (and this inability to prove at compile time can't cause the program to fail to compile, it's Python).

Additionally, Python only has infinite precision ints, so it would be quite hard to get anything useful out of this, unless you wholesale switch to using ctypes or something, but then just use Cython and it will be way easier.

I'm not convinced this could ever be anything more than a super limited toy.

</comment></comment></comment></comment></comment></comment><comment><USER4> This is a niche where [https://julialang.org/](https://julialang.org/) is really poised to shine: (nearly) As fast as C; (almost) as approachable as Python.

</comment><comment><USER5> I thought that data oriented programming was a paradigm that sits next to Object Oriented and Functional programming and I had a lot of trouble understanding the DOP-related articles that have been trending on HN recently.

But apparently it is just what we in R and Python call vectorised functions.

To boil it down:

non-vectorised:

        square(2) -> 4
        square([2,4]) -> error

vectorised:

        square(2) -> 4
        square([2,4]) -> [4, 16]

Which doesn't care whether you are passing messages between objects or avoiding side effects through pure functions and monads or imperatively muting your data left and right.

Or am I completely off?

</comment><comment><USER6> The two-language problem is well-known. People wanted performance, which is reserved to languages like C, C++ or Java, but they didn't want to use these languages, since they are objectively ugly and a pain to write. Thus, languages like Python were born, but we were warned that they were going to be slow because something something dynamic typing something something the compiler can't optimize blah blah blah. And so we were told to avoid doing too many loops, or load too many objects in memory, or indeed even attempt push the language to match one's actual use cases, because Python wasn't well-built for it.

But in the meantime, languages like R or Matlab had figured a solution: write all the heavy-lifting ultra-optimized algorithms in C or Fortran or some equally ugly language that no one but really smart nerds wants to touch, and wrap it in a semantic that makes loops and loading many objects unnecessary, called 'vectorized operations'. In R, for instance, you think you're manipulating mere strings or logicals, but you're in fact manipulating _vectors of length 1_ and of type 'string', 'logical', etc. But doing operations on vectors or arrays became as seamless as doing them with mere scalars, with hardly any loss in performance. And so the R world thrived, although we were still cautioned to use weird lapply/sapply/rapply magic instead of doing proper loops because something something compiler something something slow blah blah blah.

And so the Python world saw that the R and Matlab world thrived, and wondered if they could do the same. A bunch of really smart nerds sat down with their laptops and wrote a bunch of ultra-optimized algorithms in one of those ugly languages no one else wants to touch, and lo, in the mid-2010s Python had finally achieved feature parity with R and Matlab twenty years ago. Yet the trend showed no sign of slowing, as Python was not only useful for scientific computing, but many other use cases as well (you ever tried to write an interface or webserver in R?), and sometimes researchers have the audacity to want to do several things at once with the computer. And so Python achieved its present ubitquity in data science.

There's trouble in paradise, however. As with R, we were cautioned to avoid doing too many loops because something something you know what I mean, and instead use vectorized operations. And little by little, we had to learn every day a little more of numpy's arcane API, the right magical formulas to invoke in order to avoid losing performance. We had to learn which operations are in- place and which ones create a new array (knowing this could change over multiple versions), which appropriate slicing and indexing to use, which specific functions to call. And the more our use cases deviated from the documentation, the more magic we had to learn. At some point we had to learn obscure methods beginning with an underscore, or even (the horror!) mind whether arrays were ordered C-style and Fortran-style, or even told to use Cython (!), nevermind your desire to absolutely avoid touching these languages in any way. May Allah be with you should you ever want to manipulate sparse data.

Aware that the community had to learn magic whose complexity on par with the ugly languages they'd sworn off, really smart nerds took it upon themselves to... write more magic in order to avoid writing the older magic. And so we got dask, which is as powerful as it is painful to use. We got numba, which seems to work automagically in the official demo snippets and zilch in your own. 'That's because you're using them wrong', the smart people tell you on stackoverflow. 'Teach me how to use them right', you beg. And so your mental spellbook thickens with no end in sight...

Enter Julia. Julia doesn't have that any of the above dillemas, because Julia is _fast_. Julia doesn't care whether you vectorize or write loops, but you can do either. Julia doesn't force you to declare types, but you can if you really want to. Julia doesn't require you to write advanced magic to do JIT compilation. Julia doesn't see itself as an R or Python competitor: why, Julia _loves_ Python and R, and in fact you can just call one from the other if you feel like it! Go on, just RCall ggplot on an array created with PyCall("numpy"), it just works! Julia was _built_ with parallel computing and HPCs in mind, so no need to fiddle with dask boilerplate when it just works with @macros. Julia knows programmers are afraid of change, so it syntax is really, really close to Python's. Julia has a builtin package manager. Julia lets you use the GPU without having to sacrifice a rooster to Baal every time you want to install CUDA bindings.

Of course Python isn't going anywhere, just like R is still going strong even after Python 'displaced' it. And of course, Julia's ecosystem is smaller (but growing), its documentation is lacking, it doesn't have millions of already answered questions on Stackoverflow...but if you know where the wind blows, you know where the future is headed, and its name rhymes with Java.

<comment><USER7> > if you know where the wind blows, you know where the future is headed, and > its name rhymes with Java.

[https://www.rhymes.net/rhyme/java](https://www.rhymes.net/rhyme/java)

<comment><USER8> I have no clue where the rhyming future is headed. Anyone solved this riddle?

</comment></comment></comment><comment><USER9> It's interesting that this execution speed optimization technique has adopted the name traditionally used in altogether other meanings (like the quote about a program's data structures being more important than the code, or how FP and programming with immutable values is actually data-first programing).

</comment><comment><USER10> What I'd like (I don't know how possible) is being able to define ufunc(-likes) without resorting to C extensions. Obviously won't be as fast, but some benefit should be possible?

<comment><USER11> I'm not sure if Numba is in the spirit of your question, but it does make it easy to write ufuncs without explicitly dropping into C.

[https://numba.pydata.org/numba- doc/latest/user/vectorize.htm...](https://numba.pydata.org/numba- doc/latest/user/vectorize.html)

<comment><USER12> numba is a true horror to use, because supported subset of python language is an ever-moving target and overall small. cython is good but needs time to meddle with. For loop-intensive, simple function I'd advise FFI.

<comment><USER13> I disagree that it's a "true horror to use." The set of supported built-in classes grows significantly by the day. It's not as good when used for wholly unstructured streams of data (e.g. tuples of mixed type, dicts with complex objects inside of them), but if you can spend the design time to arange things in a structured manner, it's super easy to use, and can seriously boost performance on simple algos.

I've had a ton of success using it in statistical and computational geometry applications.

<comment><USER12> Let me explain my line of reasoning here (been there at least three times, situation gave 3 different outcomes) :

\- case 1 : need to make calculations with a specialized library in C (precision arithmetics). Build a bare FFI, later replaced with CFFI.

\- case 2 : loop-intensive on very simple calculations, time-constrained development. Identified as horrible performance in python : tried numba didn't work, ended up using cython, worked really well.

\- case 3 : optimize numpy-intensive routine for performance. Tried numba, didn't work. Too expensive to recode in cython. Look at numpy C/C++ interfaces, and numpy-friendly C++ alternatives. Also try to trick numpy to function better (do not ever try this, it'll give worse results). Ended up doing nothing as time to develop was the main constraint here.

If you have a lot of time and work on a small program, maybe you can spend the time to optimize. Team producing lots of complicated algorithms and no way to re-develop everything, stick to python, identify perf losses and choose wisely what you'll optimize.

Numba has progressed but is still not a "drop-in decorator" as advertized. Can even give worse performance in some cases. Nevertheless the idea is good and I praise the effort, when it's done it'll be massive !

<comment><USER3> Numba failing to work in case 2 is extremely bizarre. Can you post that code or a similar example?

Not only should it work for case 2 - it should only require a one liner decorator add and no user specified static typing.

I've used numba in production for around 6 years now, and never encountered the problems you describe.</comment></comment></comment></comment></comment></comment>


Greenpeace raids the cloud - <AUTHOR> http://www.roughtype.com/archives/2010/03/greenpeace_in_t.php

<comment><USER2> What about all of the energy savings that computers bring us, Greenpeace?

It's a lot more efficient to make a couple Google searches than to drive to the library.

It's a lot more efficient to buy something on Amazon and have it delivered by UPS, which can combine trips like crazy, than to drive to the store--or possibly several stores--to buy it.

It's a lot more efficient that I used Campfire as an integral part of working remotely last summer rather than if I had had to fly across the country for the job.

Sure, it is still important to reduce energy consumption, and lots of computing doesn't directly replace more energy-intensive tasks (sending a tweet rather than ... driving to all of my friends' houses to tell them a very short message and then leave?), but computing has lead to lots of gains.

<comment><USER3> You're misreading the article. Green eace isn't saying that cloud computing should go away. It's saying that the energy that runs major data centers should be green, or if not green, at least not coal powered.

</comment></comment><comment><USER4> Centralizing computing is good, though. When you buy a PC, there is no real incentive for the manufacturer to care about efficiency. 200W vs. 300W is a few bucks a month for you and no bucks a month for them. So who cares.

But when one entity is running all the powerful computers, then shaving off a few watts makes someone a lot of money. So they'll do it.

As for the green issue... I think it's good that Greenpeace is reminding folks that just reducing the number of watts you use isn't enough. You need to get those watts from something renewable or clean; not coal. Otherwise you're still destroying the Earth... just not quite as quickly.

</comment><comment><USER5> They (G'peace) would no doubt decry energy and resource consumption at factories were America ever to regain its much-missed and sorely-needed position as a thriving manufacturing economy.

The posts below bring out good points. Energy use is not in itself evil. One must consider what good was done with the energy. Watt-for-watt, those server farms are doing a huge amount of work on behalf of people.

</comment><comment><USER6> If the apple data center in NC is being built anywhere near the research triangle (and why wouldn't it?) - the electricityy is likely not coming from the coal plants but from Sharon Harris nuclear power plant.

<comment><USER7> Greenpeace are anti-nuclear power.

<comment><USER8> Greenpeace are against pretty much anything which would allow technology and civilisation to proceed at it as it currently does.

</comment></comment></comment><comment><USER9> Do you know what would impress me a lot more than protesting? Lead by example.

If Greenpeace and the rest of their ilk care enough, they should lead by example - announce that they'll no longer use cars, telephones, computers, heaters, air conditioners, or anything that was made by a process that also created pollution. No email, no internet, nothing more than riding on horseback to drop a letter made on hand made paper off. I think that would be good for them as an organization and good for the rest of the world.

<comment><USER10> Ah yes, the old discussion poisoning fallacy of "drawing a straw man and implying the other party are hypocrites for not following/implementing/adhering to the straw man".

And somewhat less abstract: if someone argues in favor of a smaller government, they are not automatically anarchists that think there shouldn't be a government at all. Painting a world without a military and challenging them to go live in that world is not a proper counterargument.</comment></comment>


Why a Transaction Fee Matters to You - <AUTHOR> http://davidbrin.wordpress.com/2013/02/10/why-a-transaction-fee-matters-to-you/ <comment><USER2> This is a terrible idea that for some reason keeps cropping up again and again. The major problem facing the Global Financial System(tm) is not high- frequency trading, it is large, illiquid, unhedged assets held by systemically important financial institutions. The nature of an illiquid asset is that it is traded infrequently, which makes it especially un-affected by a transaction tax.

OTOH, a tax like this may discourage high-frequency trading [despite it not being at all a systemic risk], but it also discourages certain legitimate and extremely useful forms of hedging, such as large notional currency swaps [<http://en.wikipedia.org/wiki/Currency_swap>]. And in the likely event that some esoteric forms of derivatives are exempt from the tax [they may be executed overseas, or just be exempted from the statute due to lobbying pressure] then it will encourage banks to shift trading activity from actual assets to much-more-fragile derivatives.

Bottom line: it's very likely that the net result of any Tobin Tax type implementation will be to actually make the Global Financial System(tm) _more_ dangerous. And that's assuming you can actually discourage HFT, which is not at all clear. It will probably just migrate to some island that sets up a tax- shelter exchange.

I have no great love for HFT, but this proposal is the epitome of dangerous, populist feel-goodery. It makes no regulatory sense, and the fact that the Europeans keep threatening to hold hands and jump over this cliff together should not make you think that it is in any way good public policy.

<comment><USER3> What exactly are these large, illiquid, unhedged assets? Bad mortgages?

Who would shift trading from assets to derivatives? It's my impression that HFT traders make money from HFT; that is there is no "core" business that would continue without HFT. I'm also not clear how they're supposed to HFT the derivatives from another country when apparently every foot of cable counts.

<comment><USER2> Bad mortgages are one type of large, illiquid unhedged asset, yes. But my general point is that the main danger to large institutions is a huge asset that loses value very quickly and can't be easily sold. If you accumulate too many of those, other banks can lose faith in your ability to meet short-term obligations, and then you're essentially up the creek. That is exactly what happened during the 08 panic.

As for #2, the main way trading shifts from assets to derivatives is this template: the government says I have to pay a tax if you sell me one share of AAPL. Instead, you sell me a contract that says, "if the price of AAPL changes by $X, I will give you $X. This is financially equivalent to a sale of AAPL stock for both parties, but does not require you to transfer ownership of a share of AAPL stock, and so you can avoid the transaction tax. Even if the US tries to impose the transaction tax on the derivative, it can be moved offshore, and thus avoid the tax.

What I just described is a simplified version of a Contract For Difference (CFD), as mentioned by another commenter. But there are innumerable other sorts of derivatives that could accomplish the same basic purpose.

</comment></comment></comment><comment><USER4> The problem is that any transaction fee is going to be strongly lobbied against and if it passes will likely have loopholes. I'd prefer a simpler structural change that isn't tied to a specific wording or implementation (like modifying the tick size/sub penny rule[0]).

[0] [http://marginalrevolution.com/marginalrevolution/2012/06/hft...](http://marginalrevolution.com/marginalrevolution/2012/06/hft- versus-the-sub-optimal-tick.html)

<comment><USER5> Thank you, came here to mention the penny-spread regulation. It's a little known fact about HFT that the reason speed is the primary advantage, is because the exchanges don't allow orders at prices which are fractions of a cent, and its first-come first-serve at each price point. The minimum spread permitted on the NYSE was 1/16 of a dollar (6.25 cents) until 2001, when the SEC forced all exchanges to the "decimal system". So now the minimum permitted spread is one penny.

If the exchange allowed orders at prices which are fractions of a penny, then algorithmic trading systems would have to compete on price as well speed. But it would also narrow the spreads, which would mean less profit for the "market makers".

</comment></comment><comment><USER6> Where is the evidence that HFT is affecting anything? History had plenty of bad market crashes long before computers were a twinkle in anyone's eyes. (South Sea bubble, anyone?) And HFT didn't corrupt Enron and Worldcom's leadership, or robo-sign a bunch of bad mortgages.

Also worth noting is that equity markets are a tip of the iceberg: there are also foreign exchange and bond markets. If anything, charging transaction fees will suck up liquidity in the secondary markets, causing bond issuers to have to raise rates to compensate for the illiquid secondary market. (And in the case of treasuries, guess who pays the extra interest? Not Wall Street: you.)

<comment><USER3> HFT is not just a risk of synthetic crashes, it's essentially a tax collected by private companies. I say I want to buy XYZ, you run in front of me and buy the last XYZ at that price and kindly offer to sell it to me for just a trifle more. Why thank you for providing me such a fantastic service.

I've never seen anyone else in the markets proclaiming how great it is that we have HFT providing liquidity, not like the bad old days 5 years ago or so, when it was _so_ very difficult to push a market order through. It's always the HFT companies themselves insisting how vital they are.

<comment><USER7> Do you have any evidence that the business model of any particular HFT firm involves front-running? If so, you should probably do something about it beyond leaving comments on the internet. Front-running is, of course, against the law.

5 or so years ago a large number of your counterparties were probably algorithmic market-making strategies.

</comment></comment></comment><comment><USER8> HFT serves the same purpose that human Market Makers and Specialists do, only better. Kill it, and you will end up paying more.

Did you complain when human travel agents were replaced by expedia and like? Would you complain if car salesman as a profession is gone? Do you see your profit when amazon is competing with all brick and mortar shops? What makes HFT so special in that list?

So strange to see that sentiment from a science fiction author. Afraid to lose to reality evolving faster than you can imagine?

<comment><USER9> There have been studies proving that the majority of HFT traders are net liquidity 'takers'. The notion that HFT provide liquidity is false but the industry still propagates the myth to main street.

Have a look at a recent paper: [http://www.bankofcanada.ca/wp- content/uploads/2012/11/Brogaa...](http://www.bankofcanada.ca/wp- content/uploads/2012/11/Brogaard-Jonathan.pdf)

<comment><USER8> It is an ecosystem, of course traders with fast computers would try to fill all the niches they can. Take a look at a broader picture: <http://www- rcf.usc.edu/~lharris/ACROBAT/Zerosum.pdf>

Is suspect those in "Panel C: Losers who expect to profit from trading but will not" are complaining the most.

<comment><USER9> You're citing a very old paper that makes no mention of HFT? Not sure I follow the reasoning for the citation.

No one is saying that speculative traders don't serve a function in the market place. However you said "HFT serves the same purpose that human Market Makers and Specialists" when it has been proven they don't. HFT want you to believe that they serve some altruistic purpose to the marketplace to legitimize their existence. The truth is that HFT serves as a quasi-tax on each and every share traded because to execute a trade in today's marketplace non-HFT volume almost invariably passes through the hands of HFT volume thereby shaving pennies of profit off each trade.

</comment></comment></comment></comment><comment><USER10> It's interesting to note that the effect of this transaction tax would be to 'reduce volatility' in the market. I say this because recently I watched the Authors@Google video of Nassim Nicholas Taleb (<https://www.youtube.com/watch?v=S3REdLZ8Xis>) where he stated that volatility in the market place is a good thing, creative destruction and all that.

More precisely the point he made was it was good for the market to appear and be volatile because otherwise it would only appear stable but with high and disasterous volatility brewing underneath the surface, ready to explode (the great moderation and subsequent GFC in the video).

<comment><USER11> I think it's important to distinguish between volatility within a market (e.g. hog futures) and volatility in The Market, meaning the whole economy. The first is perfectly normal and its absence across the economy would be very strange if not dangerous.

The second one is the problem -- when it happens it's because you're on the losing side of a systemic risk. It isn't that hog futures go up or down but everything is still fine for normal people because the DJIA only moves by a tenth of a percent, it's that some jackass causes a panic and a trillion dollars disappears out of the economy for no good reason. The latter is something everyone should want to prevent, which is why all the talk at the individual level about diversification and hedging. But the same goes at the macro level: We need to "diversify" the banks and major industries so that no individual company is too big to fail anymore, so that we _can_ have volatility within specific industries without it breaking the whole world economy.

And the sort of volatility HFT creates is the second kind. When something bad happens, it happens across the board, regardless of the "fundamentals" of the underlying industries. Even when it's "working" the result is only to transfer wealth from ordinary stockholders to high frequency traders, which is a net loss for anyone who isn't a high frequency trader.

<comment><USER10> My understanding of the point in the video is that the volatility resulting from the systematic risk is a symptom of the market not having enough normal/baseline volatility to clear out the dead wood, so to speak. Quite analogous to forest management, where more smaller fires clear out the fuel on a regular basis so that a large scale fire cannot occur.

Now it is probably the case that the HFT traders make The Market far more intertwined than is sensible, and hence far too combustible as a whole. In the video Taleb points out that he favours many smaller organisations over fewer larger ones for the simple fact that within an ecosystem a few small systems can fail with little negative effects on the whole. Where as if a large system fails it is far more catastrophic.

So perhaps the solution here is to make some sort of upper limit in financial (HFT, Banks, etc) firms so that there would be more smaller ones, where failure would be more common and the destruction of smaller companies should not disasterously affect the system as a whole.

That said, the recent wiping of trillions of dollars in value, seemed to be an effect of the many years of apparent calm. With a lot of companies creating new derivatives with the appearance of safety but were anything but safe.

To bring it back to the parent article, I do agree that everyone on the exchange should be paying some sort of transaction fee, I have to pay one when trading shares, and so should other traders.

</comment></comment></comment><comment><USER12> I don't know why techies dislike HFT. If anything HFT is helping to slim down Wall Street, replacing human traders with computers.

<comment><USER13> Why accept gaming markets for profit, instead of honest work for profit, at all?

<comment><USER12> Traders don't "game markets" for profit. They're part of the infrastructure that makes markets work. It's like saying E-Bay "games markets for profit" because they take a piece of every transaction. Being an intermediary != gaming the market.

<comment><USER13> Oh? How can something like this happen then? Did some people just loose track of the mission of helping everybody out and whatnot?

[http://www.eurolabour.org.uk/Call_for_EU_action_to_stop_fina...](http://www.eurolabour.org.uk/Call_for_EU_action_to_stop_financial_speculation_linked_to_African_famine)

 _2011-09-15

The European Parliament has called for changes to EU law to stop food price speculation that has been linked to the famine that has claimed tens of thousands of lives in East Africa.

A hard-hitting resolution adopted by Euro-MPs on Thursday 15 September calls for changes to EU directives on market abuse and financial trading to stop "abusive speculation" which has been identified as a contributing factor to the current famine in the Horn of Africa.

While it is widely accepted that the humanitarian emergency in the region was triggered by drought, a recent World Bank report identified high food prices as a key contributing factor.

Academics and international development charities working in the sector believe that food price volatility caused by speculation in agricultural derivatives on the financial markets are exacerbating the situation._

<comment><USER12> How does what happen? A drought led to reduced supply and higher food prices. That's exactly what you'd expect in an efficient market. People blame the traders because they're the mechanism through which the market incorporates the fact of the drought.

You see the same crap-throwing with traders in oil. People blame the traders for "bidding up" oil prices. The traders are just helping the markets reach a price--it's the market that's bidding up oil prices. And that's exactly what you'd expect when India and China are guzzling up the stuff but production has been flat since 2005 (and we hit peak discovery in 1965...)

<comment><USER14> Also, ethanol.

</comment></comment></comment></comment></comment></comment><comment><USER15> As someone currently in HFT, I think a financial transaction tax would be very good for me/my firm.

<comment><USER16> Can you explain why? Or is that part of your secret sauce.

<comment><USER15> The same reason McDonald's would benefit with a tax of 10 million dollars for all restaurants. It would eliminate a large swath of weaker competition and they would make more profit with reduced competition.

</comment></comment></comment><comment><USER17> That's a terrible idea. Destroying liquidity and masking market volatility. You don't destroy your clock when upon looking on it you figure you don't have enough time. The clock is the only thing telling you time is running out.

What you have to do instead is legislate and enforce transparency in all global organisations and put in jail high level executives who (may have) turned a blind eye to deliberate borrower misrepresentation (may be fraud) in investment banks and which on-sold those loans on to investors, knowing the risk of those loans were understated.

</comment><comment><USER18> If anyone can find a link to the Joint Committee on Taxation's report for the Financial Transaction Tax (AKA: Securities Transaction Tax / Tobin Tax) it would be much appreciated. The ProPublica article[1] and Defazio's press release[2] mention the JCT report but I have not been able to locate the actual report. It is even referred to in a CRS report[3] but it seems that everyone is taking Defazio's word and nobody is bothering to look for the report. I scoured the jct.gov website but could not find the publication listed anywhere.

[1] <https://www.propublica.org/thetrade/item/the-03-solution>

[2] [http://www.defazio.house.gov/index.php?option=com_content...](http://www.defazio.house.gov/index.php?option=com_content&task=view&id=736&Itemid=70)

[3] [http://www.hsdl.org/?view&did=715463](http://www.hsdl.org/?view&did=715463)

</comment><comment><USER19> HFT is an abomination. Anything that will curtail it is good in my book.</comment>


The dark side of .io: How UK makes DNS profits from a shady Cold War land deal - <AUTHOR> https://gigaom.com/2014/06/30/the-dark-side-of-io-how-the-u-k-is-making-web-domain-profits-from-a-shady-cold-war-land-deal/ <comment><USER2> Might is right. This is senseless crying. You live in a land that enjoys security and prosperity because of centuries upon centuries of violence and conquest.</comment>


Software vendor argues that it has copyright in output of its CAD software - <AUTHOR> http://www.maw-law.com/copyright/output-copyright-protected-software-program-protected-copyright/ <comment><USER2> This is actually surprisingly interesting case. While I think most would agree that when software does purely "algorithmic processing" then the output would not be considered derivative work. But increasingly software suites include various templates and content libraries. When the output is then a composition of those templates and library content then I'd agree that there is an argument to be made that such output could be considered derivative work.

<comment><USER3> By that line of reasoning, your compile's output is derivative work of the creative effort of the compiler writers.

<comment> alok-g Isn't that true? A compiler's output does include many portions which include creative effort of the compiler writers. Licenses for compilers thereby explicitly include a clause on giving up that ownership over the output of the compiler. In other words, the line of reasoning has always been true; perhaps you were unaware like most people who never actually read the license text.

<comment> vertex-four GCC does not have such a clause. What it does have is a clause allowing a small library - the GCC Runtime Library - to be linked to and included in its output, provided certain conditions are met. The library is separate from the compiler input and not a result of it - it contains some core functions involving things like exception handling, arithmetic on some architectures, etc. It's held to be self-evident that compiler output that is a direct result of compiler input does not come under the license of the compiler.

<comment><USER4> GCC does not, but Autoconf and Bison for example do. GCC's templates are a handful of instructions long (at most) and are subject to plenty of transformations before they end up in assembly code; Autoconf and Bison literally copy hundreds of lines of code verbatim. You need a license that is separate from the GPL to use that output in a non-GPL-compatible derivative work.

<comment> vertex-four That makes sense - if autoconf source is closer to a set of instructions "please include library functions A, E and G in this order" than C source, it's obvious that each function is under copyright and so the output would come under the terms of the license covering those functions.

</comment></comment></comment></comment></comment></comment><comment><AUTHOR> This case is with the Ninth Circuit on appeal. If you are interested in how courts conduct oral arguments in cases like this, there is a video recording available at [http://www.ca9.uscourts.gov/media/view_video.php?pk_vid=0000...](http://www.ca9.uscourts.gov/media/view_video.php?pk_vid=0000010345)

<comment><USER5> I like how they really grill the lawyer of the plaintiff. Frankly, her explanations - especially when trying to come up with a test to distinguish their software from Photoshop - are pitiful.

<comment><AUTHOR> It gets slightly better later when she explains how some of the elements of the output is controlled by the CAD software, but it is still a very weak argument.

<comment><USER5> That still doesn't distinguish it from PS, though; just look at Content-Aware Fill.

<comment><AUTHOR> As I mentioned, weak argument.

</comment></comment></comment></comment></comment><comment><USER6> What I take from the description is that the defendant, Unigate Enterprise (UE), used the software (SDS/2) to produce drawings for its Chinese contractors. The drawings are in a proprietary file format that can only be viewed by SDS/2 (written by Design Data Corporation (DDC)).

Chinese contractors use illegal copies of SDS/2 to view the drawings. So yes, the Chinese have broken the law. It's Unigate's problem that they chose software that produces files that can't be openly shared.

In claiming that the output is copyrighted, DDC seems to be resorting to a last-ditch (and groundless) effort to sue their American client, UE, because they have little prospect of getting anything out of the Chinese.

Bottom line: [1] Somebody broke the law. [2] It has little to do with copyrighting a program's output.

<comment><AUTHOR> Those are not the facts. UE didn't use SDS/2\. They subcontracted work that requires the use of SDS/2 to a Chinese subcontractor who used an allegedly pirated copy of SDS/2 to do the work. It has to do with copyright in program output because that is what DDC is claiming. I would agree that the claim in this case is weak but it is before the 9th Circuit and their opinion is going to be very interesting either way.

<comment><USER7> Doesn't this bit also matter?

> _Judge Callahan and Judge Hurwitz were both troubled by the fact that UE had > advertised on its site that it used the SDS /2 software. UE's counsel > answered that UE was counting on contractors to use it, but admitted that UE > had never asked DDC if it was indeed true that the contractors were legally > using the software._

<comment><USER8> Is it normal to contact vendors of software you use to find out if your contractors are also legally using it?

</comment></comment></comment></comment><comment><USER9> > UE admitted that SDS/2 had been used to create files and drawings in five of > its projects, but argued that they were made by contractors in China.

China piracy story time. I was talking to makers of a niche CAD type software package. It was niche enough that they simply knew all their customers pretty much by name. They went to a trade show to exhibit their software, and a group of Chinese engineers approached them, thanked them and told how much the love their software, it is the best really for what it does, etc, etc. So that was good, everyone was happy and smiling, except one thing - the company shown on their badges was not a client.

<comment><USER10> ...continue

<comment><USER9> Well they had a problem. Their software was being pirated. There was talk of a selling a hardware dongle with it. But I am not sure what came of it, since I lost contact with them.

Here is what I wonder. Even if they lowered the price just for China, would there be a point where they would choose to pay for it or it would never matter a pirated copy would always be preferred.

<comment><USER11> I have done a few HW and a few SW startups. For the HW I would eventually help us move into all of Asia. For the SW I would help move into Asia, minus China.

<comment><USER9> At our company we did eventually end up with a hardware dongle. But it helped that it was a HW/SW solution anyway. We just added a few operations that needed to go through a hardware module. It was just enough hoops to jump through to hopefully dissuade the casual "pirate".

It was an interesting problem anyway. A lot of obvious solutions didn't seem to work, like say just checking a flag in an "if ... else" branch because that can be disassembled and bypassed with a patch. Some functional operations that are critical to the product had to actually go through the hardware... Now looking back we probably over-engineered the heck out of it.

</comment></comment></comment></comment></comment><comment><USER12> In other news... Bic pen, makers of ubiquitous writing implements, claims copyright over all writings made with said implements.

Where does it end?

<comment><USER13> If we're asking "should a user be able to continue using files created with a program beyond the point that they are paying from that program?" we're not that far from the practical effects of such a decision already.

I'd wager that a good 90% of the world's graphical/artistic material is locked up in Adobe-proprietary formats for which the only practical editing software has moved to a cloud licensing model. Dispute with Adobe? Say goodbye to all your company's assets...

<comment><USER14> Though, adobe themselves provide the documentation to unlock the format

[https://www.adobe.com/devnet- apps/photoshop/fileformatashtml...](https://www.adobe.com/devnet- apps/photoshop/fileformatashtml/)

<comment><USER15> To which I humbly submit one of my favorite pieces of writing about the PSD file format.

From [https://bitbucket.org/WAHa_06x36/old- xee/src/fe1cec107972603...](https://bitbucket.org/WAHa_06x36/old- xee/src/fe1cec107972603523e67e7be29a83e493f5cac2/XeePhotoshopLoader.m?at=default&fileviewer=file- view-default#XeePhotoshopLoader.m-108)

        // At this point, I'd like to take a moment to speak to you about the Adobe PSD format.
        // PSD is not a good format. PSD is not even a bad format. Calling it such would be an
        // insult to other bad formats, such as PCX or JPEG. No, PSD is an abysmal format. Having
        // worked on this code for several weeks now, my hate for PSD has grown to a raging fire
        // that burns with the fierce passion of a million suns.
        // If there are two different ways of doing something, PSD will do both, in different
        // places. It will then make up three more ways no sane human would think of, and do those
        // too. PSD makes inconsistency an art form. Why, for instance, did it suddenly decide
        // that *these* particular chunks should be aligned to four bytes, and that this alignement
        // should *not* be included in the size? Other chunks in other places are either unaligned,
        // or aligned with the alignment included in the size. Here, though, it is not included.
        // Either one of these three behaviours would be fine. A sane format would pick one. PSD,
        // of course, uses all three, and more.
        // Trying to get data out of a PSD file is like trying to find something in the attic of
        // your eccentric old uncle who died in a freak freshwater shark attack on his 58th
        // birthday. That last detail may not be important for the purposes of the simile, but
        // at this point I am spending a lot of time imagining amusing fates for the people
        // responsible for this Rube Goldberg of a file format.
        // Earlier, I tried to get a hold of the latest specs for the PSD file format. To do this,
        // I had to apply to them for permission to apply to them to have them consider sending
        // me this sacred tome. This would have involved faxing them a copy of some document or
        // other, probably signed in blood. I can only imagine that they make this process so
        // difficult because they are intensely ashamed of having created this abomination. I
        // was naturally not gullible enough to go through with this procedure, but if I had done
        // so, I would have printed out every single page of the spec, and set them all on fire.
        // Were it within my power, I would gather every single copy of those specs, and launch
        // them on a spaceship directly into the sun.
        //
        // PSD is not my favourite file format.

<comment><USER16> Here's a mobile-friendly copy of that:

/*

At this point, I'd like to take a moment to speak to you about the Adobe PSD format. PSD is not a good format. PSD is not even a bad format. Calling it such would be an insult to other bad formats, such as PCX or JPEG. No, PSD is an abysmal format. Having worked on this code for several weeks now, my hate for PSD has grown to a raging fire that burns with the fierce passion of a million suns.

If there are two different ways of doing something, PSD will do both, in different places. It will then make up three more ways no sane human would think of, and do those too. PSD makes inconsistency an art form. Why, for instance, did it suddenly decide that _these_ particular chunks should be aligned to four bytes, and that this alignement should _not_ be included in the size? Other chunks in other places are either unaligned, or aligned with the alignment included in the size. Here, though, it is not included. Either one of these three behaviours would be fine. A sane format would pick one. PSD, of course, uses all three, and more.

Trying to get data out of a PSD file is like trying to find something in the attic of your eccentric old uncle who died in a freak freshwater shark attack on his 58th birthday. That last detail may not be important for the purposes of the simile, but at this point I am spending a lot of time imagining amusing fates for the people responsible for this Rube Goldberg of a file format.

Earlier, I tried to get a hold of the latest specs for the PSD file format. To do this, I had to apply to them for permission to apply to them to have them consider sending me this sacred tome. This would have involved faxing them a copy of some document or other, probably signed in blood. I can only imagine that they make this process so difficult because they are intensely ashamed of having created this abomination. I was naturally not gullible enough to go through with this procedure, but if I had done so, I would have printed out every single page of the spec, and set them all on fire. Were it within my power, I would gather every single copy of those specs, and launch them on a spaceship directly into the sun.

PSD is not my favourite file format.

*/

</comment></comment></comment></comment></comment><comment><USER17> This seems like a bad idea for the vendor. Now all of their customers will have to evaluate the legal risks of merely using the software.

<comment><AUTHOR> That's not a bad idea for any software that is critical to one's business.

<comment><USER18> What, it's not bad to scare off their own customers? After such claims, they'll ditch this vendor, and will go looking for more sane competitor who isn't a copyright freak.

<comment><USER19> I believe the parent meant that it's not a bad idea to reevaluate the usage of any software that is business-critical and may have similar repercussions should your license to use it expire.

<comment><AUTHOR> That is exactly what parent meant.

</comment></comment></comment></comment></comment><comment><USER20> Once again the dangers of proprietary software in the businesplace raise their heads. I wish people with the ability would work on contributing more to the open source cad projects, because at one point suffering under autodesks draconionan and expensive licensing I tried to get a business to start switching but the open source stuff simply isn't up to snuff in most cases.

<comment><USER21> How do you mean danger?? The body of the article is considerably less hysterical than the title.

<comment><USER20> I mean you won't find GNU projects doing stuff like this, but only in proprietary land do you find companies who think it's ok to copyright output. (which is a symptom of the bigger issue of culture around non-free software in general)

[http://www.gnu.org/licenses/gpl- faq.en.html#GPLOutput](http://www.gnu.org/licenses/gpl-faq.en.html#GPLOutput)

<comment><USER22> Actually, the output of Bison is GPL'd on the same grounds as presented in this case, that a significant portion of it includes code from Bison itself. You can't link and distribute a Bison-generated parser with a non-GPL parser generator (for other kinds of programs, the license contains an exception that allows linking).

<comment><USER23> That was the case, but now there's an exception: [http://www.gnu.org/software/bison/manual/html_node/Condition...](http://www.gnu.org/software/bison/manual/html_node/Conditions.html)

<comment><USER22> Yeah... I mentioned that exception in the parenthetical statement.

</comment></comment></comment></comment></comment></comment><comment> alok-g Many EULAs explicitly include a clause stating that the vendor does not hold any rights over the output of the program. (A common example is when you use a compiler to create a binary. Read what the license says.)

IANAL.

<comment><AUTHOR> And many do not. I have negotiated software licenses for 15+ years and had to fix this issues many times. Just as many software licenses explicitly only license object code, but the licensed product turns out to contain quite a bit of source code, which if not addressed the licensee is not allowed to use (copy).

</comment></comment><comment><USER24> Defendant in this case probably knows and purposely uses Chinese contractors to get around licensing expensive software. So not exactly someone to cheer for. But plaintiff also wants to set a pretty bad precedent.

<comment><USER5> What's pathetic is that they actually did download the software, which will probably come back to bite them. Sounds like we can hope this is a case in which everybody loses :)

Well, except the lawyers, of course.

<comment><AUTHOR> They appear to argue that the download was some sort of trial copy or at least that they thought it was. It appears undisputed that they never used the software.

</comment></comment></comment><comment><USER25> That's nonesense. This is like somebody claiming they own copyright on your book or code because you used their text editor.

<comment><USER26> It's not quite that. If the output of the program was a standard text file, then no, because nothing of the program was copied. If for example the output of the program included certain fonts or graphics (e.g. PDF), those are still copyrighted. To use those fonts, you'd need a licence, in this case the same licence you need for the program.

</comment></comment><comment><USER27> Software vendor loses all of its customers.

<comment><USER4> Parent didn't read the article?

<comment><USER28> From the Hacker News Guidelines[0]:

> Please don't insinuate that someone hasn't read an article.

[0] [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

</comment></comment></comment><comment><USER29> Why not argue that because the external contractors in china are using an illegal copy, therefore there is no client that paid a license to use the software, and that DDC (software maker) should be able to assume the copyright of the output because they are the only viable and legal copyright holder of the software in use. AND therefore the only one with a valid license of the software and its output.

<comment><USER30> If I steal a pen and write a poem with it I still hold copyright over the poem, how the poem was transcribed has no bearing on copyright.

<comment><USER29> great analogy, thanks!

</comment></comment></comment><comment><USER31> Nintendo has a similar take on its software. It claims rights on the game footage uploaded to youtube by users.

<comment><USER32> That makes more sense as they own the art assets that are being displayed in video.

</comment></comment><comment><USER33> So, does that make them responsible for the things done with their software?

</comment><comment><USER28> From the Hacker News Guidelines[0]:

> please use the original title, unless it is misleading or linkbait

[0] [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

</comment><comment><USER34> The company is accused of pirating the software. If they had it lawfully, they wouldn't be in this fix. The question here is whether the program outputs from a pirated program are free of copyright restrictions.

<comment><USER5> No, the fact that they "pirated" the software is a separate claim, since the output files were not produced by their copy of the software. They would be in this fix even if they had never "pirated" it.

</comment></comment><comment><USER35> So if I wrote a novel using Microsoft Word, Microsoft has the copyright to the novel?

</comment><comment><USER36> Software vendor is wrong.</comment>


The History of a Security Hole - <AUTHOR> http://www.os2museum.com/wp/the-history-of-a-security-hole/ <comment><USER2> Worth mentioning: FreeBSD had an equivalent bug on _amd64_ , because the I/O permissions array got confused with a pointer to said array. This was fixed in April 2005 -- and was discovered by someone saying "that's weird, I can startx as a regular user even though the X server binary isn't setuid...".

</comment><comment><USER3> How this works in emulators/VMs is also of interest --- because it could be another way in which code running on real hardware differs from that in VM, and can be used to distinguish.

</comment><comment><USER4> Forbidden

You don't have permission to access /wp/the-history-of-a-security-hole/ on this server.

<comment><USER5> [https://web.archive.org/web/20180903022426/http://www.os2mus...](https://web.archive.org/web/20180903022426/http://www.os2museum.com/wp/the- history-of-a-security-hole/)

</comment></comment><comment><USER6> Seriously? OpenBSD developers didn't consider the C compiler adds padding to the structure?

<comment><USER7> No, we've never heard about this padding of which you speak.</comment></comment>


Search warrant overrides 1M users' choice not to share DNA with cops - <AUTHOR> https://arstechnica.com/tech-policy/2019/11/search-warrant-overrides-1m-users-choice-not-to-share-dna-with-cops/ <comment><USER2> Major problem with a heavily underfunded understaffed company, they simply don't have the resources to put up any kind of legal resistance to this type of judicial pressure.

Edit, Also incredibly relevant: [https://news.ycombinator.com/item?id=21461957](https://news.ycombinator.com/item?id=21461957)

<comment><USER3> >they simply don't have the resources to put up any kind of legal resistance to this type of judicial pressure

a very simple solution to this, don't keep the data once you've provided a service to the customer. Or anonymize it and scrub any association.

It's never a question of "we don't have the resources to protect your data". The conclusion should be, if you don't have the ability to protect your customers you don't get to harvest their information.

<comment><USER4> This is the only way things will change. Companies need to be our in the position where holding personal data is a major liability. And not gpdr type that relies on admins or politicians. It needs to be something that excites the trial lawyers approaching the level of mesothelioma.

Idk how we get there, whether it be through the courts, Congress, or some other way. I think getting rid of third party doctrine will do more towards this goal than most people realize, but it has to be in a way that treats any data acquired by government as if it were collected by the government, to 4th amendment levels of scrutiny.

But if things are going to change, major liability on government and private sector will have to be the result, unless we want eventual relapse.

</comment></comment></comment><comment><USER5> Things like this are why tools like Ring doorbells or Alexa make me scared. Sure, nobody is normally paying attention to that data. But it's all just a subpoena/warrant away from being searched. If it's physically possible to look through a dataset, then there are legal pathways for the government to use that data.

<comment><USER6> >legal pathways

And barring that, they'll just suck it all up anyway in secret and claim immunity from oversight due to undefined national security reasons.

</comment></comment><comment><USER7> I wonder how a bigger corp, like 23andme, would handle this.

Interestingly, their transparency report[1] claims they've received 7 requests from law enforcement, and denied all of them.. shows they're willing to put up somewhat of a fight at least.

[1] [https://www.23andme.com/transparency- report/](https://www.23andme.com/transparency-report/)

<comment><USER4> It seems to me all these companies would benefit by pooling resources into an industry wide organization for defending against this sort of thing.

If case law is set by the little guy losing, it doesn't matter how much money 23 and me is throwing at their cases.

</comment></comment><comment><USER8> >GEDmatch hit the spotlight in 2018, when DNA data from its site led to the eventual arrest of a man suspected to be the "Golden State Killer," responsible for dozens of rapes and murders in California between 1976 and 1986.

This Golden State Killer case was the biggest public relations boon to police wanting access to these DNA. In general the public wants rapists and killers to be caught. I think the public thinks that if these people don't care enough about their privacy to send DNA samples to a private company (that is not a health care company), then they should not object if police use that information to catch killers and rapists.

<comment><USER9> Wasn't it actually a relative of his that submitted their DNA, not him?

<comment><USER10> Yes it was a relative's DNA. So you have to not only submit your DNA, you have to talk your all your kin out of submitting theirs. That includes sisters or brothers separated by adoption in early life.

</comment></comment></comment><comment><USER11> At this point, particularly as a matter of principle, I think it's better to pull your DNA profile out of the system, and request that it be destroyed.

<comment><USER2> That may not be enough.

>Asked if there was evidence the database had already faced concerted attacks, scraping, or scanning, Rogers said, "I don't want to get into it."

>"Not that I am aware of," he added. "I don't know."

>Rogers declined to comment on whether he'd been approached by national security officials about the site.

Good luck trying to destroy a digital record at this point.

Edit, Referenced: [https://www.technologyreview.com/s/614642/dna-database- gedma...](https://www.technologyreview.com/s/614642/dna-database-gedmatch- golden-state-killer-security-risk-hack/)

<comment><USER4> He's already been busted lying about providing access when it was against policy. After being in the bad position of not having a policy for police requests, he set a policy then almost immediately broke it.

I think policy was only for a murder, and he personally approved access for a non-lethal stabbing, or something similar.

</comment></comment></comment><comment><USER12> If you don't want your DNA searched in the GEDmatch database, then don't upload your DNA to GEDmatch.

<comment><USER13> You also have to ensure that none of your close relatives upload their DNA (very similar to yours) to the database either.

<comment><USER14> I've tried. So very hard. And I failed.

People, like my siblings and mother, see this as a game. They see no problem with giving a private company access to their most uniquely identifying information. They'd likely hand over their fingerprints and retina scans for a free photo of themselves doing it. My mother shreds every piece of mail but jumped on 23andme without hesitation.

The world has become dumber and less concerned with personal information than some weird shared social experience. I'm not worried about the feds getting my data (ish) as much as I am about Bad Actors in the future. (I still very much want the government out of my business and personal life though.)</comment></comment></comment>


Embrace Minimalism - <AUTHOR> http://www.techsoomer.com/embrace-minimalism/

<comment><USER2> _Why write a 5 paragraph email that can be expressed in 3 lines? Why waste your coworker's time with a 2 hour presentation? It's probably because your email is garbage and your presentation sucks._

Or, you know, you might have a complex point to put across. Otherwise we might as well throw email out and stick to twitter. Throw out books, and stick to blogs,

Minimalism is not its own reward. Seek neither to minimise nor to maximise; aim to optimise. There is a sweet spot for everything. Why should I only have one bank account? The problem would not be having three bank accounts, but having more than I needed.

<comment><USER3> Perhaps the article was a more minimalist way to say we need to head in the direction towards minimalism in order to find the optimum. But that would've taken too long to say...</comment></comment>


Women Accusing WikiLeaks Founder Maintains Charges of Sexual Misconduct - <AUTHOR> http://www.newsweek.com/blogs/declassified/2010/08/25/lawyer-for-women-accusing-wikileaks-founder-maintains-charges-of-sexual-misconduct.html

<comment><USER2> It's somewhat implied in Sweden that if you hire Lief Silbersky the defense lawyer, it means you did it and that you stand a good chance of getting away with it.

 _A detailed, chronological account of the women's alleged encounters with Assange—which in both cases began with consensual sexual contact but later included what the women claimed was nonconsensual sex, in which Assange didn't use a condom_

I'm confused I thought non-consensual sex was rape, or is there a law in Sweden that says women who regret their sexual encounters can now withdraw consent retroactively?

<comment><USER3> I think that the main issue was that he didn't use a condom in those additional encounters, which resulted in the withdrawal of consent. One assumes that consent may be offered conditionally, such that consent is withdrawn automatically if the conditions are violated - therefore, one might say "I consent if you use a condom," and once he did not wear a condom the consent was withdrawn.

More pessimistically, in the US, at least, there is no requirement to communicate the withdrawal of consent - during an act, a partner may decide to withdraw consent, and it is withdrawn regardless of whether it is communicated or indicated in any way; combined with the issue of proving whether consent was given or not, this means that retroactively withdrawing consent is trivial. I am not familiar with Swedish law, but I doubt that it fixes these issues - preventing false negatives seems to generally be considered more important than preventing false positives, to most people.

</comment></comment><comment><USER4> That article, for being so wordy, is remarkably slim on details. If the issue is that he refused to use a condom, then that doesn't constitute rape if they decided to have sex with him anyway. If he actually did force himself on someone without their consent, that's a whole different (and much more serious) issue and he should face the repercussions.</comment>


Bigger Tweets than Twitter - <AUTHOR> http://woofertime.com/ Woofer = Twitter * 10; <comment><USER2> 10 times BIGGER Tweets than Twitter. But a 1400 character limit? Maybe they should change this to a maximum. It sounds good. But actually it doesn't help us any. First we couldn't type more than 140 characters. Now we cant type less than 1400. So basically you are still screwed if you want to type 141-1399 characters. Big gap=even bigger opportunity. We have the microblog, now the macro blog. We need the "mid-croblog".

</comment><comment><USER3> I think twitter is fine. I'm reminded of Spolsky's comment about how the Spanish Steps is an example where the environment drives behavior. Twitter's short posts create an environment that drives behavior, short and to the point. If someone is going to blog about their cats, I want them to be restricted to 140 characters.</comment>


Virginia's $40 Toll Road Better Be the Future of Driving - <AUTHOR> https://www.wired.com/story/virginia-i66-toll-road/ <comment><USER2> The Northern Virginia traffic situation is more the result of a failure to follow the projections put out in the 60s and build the required roads for the expected growth. I66 was recommended to be a minimum of 4 lanes each way at the time, but only half of that was put in. The number of roads and lanes on those roads is totally inadequate.

The main driver was that Southern Virginia had the political power at the time and refused to allocate money for roads in the north, preferring instead to build multi-lane highways to nowhere in the south. It's all different now with the explosion of the size of the federal government increasing the population of the north so that they're now in the driver's seat - 30 years too late now that the growth has occurred.

They did build the amazing HOV lanes on I95/I395 that manage to fit 2 lanes of traffic into the same space where 6 lanes could have been built. The 2 lanes change direction mid day further inhibiting traffic.

</comment><comment><USER3> >The scourge we know as traffic costs the American economy about $125 billion to traffic per year, according to traffic analytics company Inrix.

I never understood how they got these numbers.

<comment><AUTHOR> I don't know how they got those numbers either, although I could speculate on a simple way to do a back-of-envelope calculation by Multiplying the following:

* commuter average hourly wage * time spent in traffic (basically time the commute took in traffic minus the amount of time the commute would hypothetically take if drove at the speed limit)

That's easy although imperfect...might also want to account for other things like delayed meetings due to traffic.</comment></comment>


Do we really need another Apollo mission? - <AUTHOR> http://trueslant.com/gregfish/2009/07/31/do-we-really-need-another-apollo/ <comment><USER2> NASA is underfunded ? The whole Space Shuttle program is a huge waste of money. Robots are far better in space than humans.

<comment><USER3> I would say NASA is woefully underfunded, especially compared to the seemingly infinite budget for national defense. The defense budget for the US for FY2009 was $515.4 BILLION (from Wikipedia) compared to NASA's FY2010 REQUEST for $18.686 BILLION (Official Press Release). To add a little more perspective, you can take just a couple of projects that the DoD works on and make it == NASA's ENTIRE budget!

 _Missile Defense - $8.8 Bil._

 _F-22 Raptor - $4.6 Bil._

 _V-22 Osprey - $2.6 Bil._

 _DDG 1000 Destroyer - $3.5 Bil._

This comes out to $19.5 BILLION! All of the projects listed above are in my opinion relics from the cold war arms race, combined with the military complex's deep hooks in government. Also, these are just the projects that we KNOW OF, how much money do you think all of the black projects take up?

If we took away these projects and gave most of the money to NASA (effectively doubling their budget), I think that we would see lots of great work and scientific achievements, along with a increase in patriotism.

On the topic of whether we should go back to the Moon, I believe that we should. I feel this way not because I think it would be cool, or even that me might learn some new things (we would), but mainly because it would be a stepping stone to Mars. We could use the Moon to build and launch our manned ships to Mars and other planets and their moons. We could also more safely store nuclear reactors and materials on the moon, allowing us to build advanced nuclear engines just like the 'mad scientists' dreamed up back in the olden days. If anything, the ISS has been a waste of time, I think we should have built an extensive base on the moon, instead of putting some tinker-toys together in orbit.</comment></comment>


Netstat without netstat - <AUTHOR> https://staaldraad.github.io/2017/12/20/netstat-without-netstat/ <comment><USER2> > _Since I wasn't root and couldn't install any tools (guess I could have > copied a netstat binary across)_

If you have the ability to write anywhere to disk (e.g., even /tmp, /run), dropping a copy of Busybox (a statically linked, minimal clone of many of the base tools like ifconfig, netstat) has helped me more than once.

(I didn't have a working scp on a machine, and it was the ssh "gateway" to another machine that I needed access to. To jump to the next machine, I needed to run either ssh or nc, both of which were defunct (the HDD was dying). Copied busybox over for nc, and it worked like a charm. I had to write a small python script to transform the binary into a printf command to write it out to disk, as essentially nothing but bash builtins worked due to the condition of the disk. Had a minor panic attack when I realized I needed to chmod +x it, but chmod still worked. In hindsight, I think I could have set the umask prior to the printf to avoid that step.)

<comment><USER3> I know you've already solved this problem, but a few tips I've learned along the way in case you - or anyone else - runs into this problem again:

If only Bash worked then you can use Bash's builtin TCP/IP stack[1][2] to transfer data (inc files)[1][2]

[1] [http://www.linuxjournal.com/content/tech-tip-tcpip-access- us...](http://www.linuxjournal.com/content/tech-tip-tcpip-access-using-bash)

[2] [http://www.linuxjournal.com/content/more-using-bashs- built-d...](http://www.linuxjournal.com/content/more-using-bashs-built-devtcp- file-tcpip)

This is how I got around a similar problem as yourself regarding needing to get a file on a system which was severely locked down.

Also you can get around the `chmod +x` problem by calling ld directly[3]

[3] [https://superuser.com/a/341471](https://superuser.com/a/341471)

<comment><USER4> I like Bash's TCP thing. First time I saw it thought a coworker was pulling my leg. Since then I've used it in a few places like talking to a metrics daemon on localhost, or just debugging various issues. The most fun thing is to show it to others as almost everyone is surprised about it.

</comment></comment></comment><comment><USER5> AWK-less version. Assumes only sed and printf builtin.

Bonus: aligned columns[1]

(tested on 80x25 and only with text from blog, not actual /proc/net/tcp)

[1] Hack. Probably there is a better way to do alignment using only printf; alas I only know a subset of printf features.

       #! /bin/sh
       printf '%s\t\t\t%s\n' Local Remote
       sed '
         /: /!d;
         s/.*: //;
         s/ /-/;
         s/ .*//;
         s/[0-9A-F][0-9A-F]/0x& /g;
         s/ 0x//4;
         s/ 0x//7;
         s/://g;
         s/-//;
        ' /proc/net/tcp \
        |while read a b c d e g h i j k;do
         s=$(printf '%d.%d.%d.%d:%d %d.%d.%d.%d:%-22d\n' \
                     $d $c $b $a $e $j $i $h $g $k);
         l=${s%% *};r=${s#* };
         if test ${#s} -lt 45;then
         printf '%s\t\t%s\n' $l $r;else
         printf '%s\t%s\n' $l $r;fi;
         done

</comment><comment><USER6> The 'modern' iputils alternative to netstat is 'ss'. May or may not exist on some systems where netstat does not.

Good info, though. Lots of info to bad had in /proc but needs parsing.

<comment><USER7> ss is also one of the biggest travesties: every other UNIX-like and UNIX OS has netstat, just GNU/Linux won't because someone thinks they know better.

<comment><USER8> Unlike other Unix, Linux doesn't control its userland. net-tools (ifconfig, netstat, arp, route, etc.) was maintained by a separate person, was mostly a dead project (it has been revived very recently) and many scripts heavily relied on their interface (both input and output). To be able to push new features, kernel devs had to develop a set of tools to expose those features to users in a coherent way.

So, yes, maybe they know better.

<comment><USER7> Only in GNU/Linux can it happen that core networking utilities like netstat, arp and ifconfig fall into neglect. Because no, they don't know any better.

</comment></comment></comment></comment><comment><USER9> Pro-tip: if you pop a shell, only use it to run your own dropper. Use locally available tools as little as possible. Automate your entire post-exploitation payload and cleanup and get out as soon as possible.

Reading this article is like reading about elite navy seals who storm an enemy outpost, get inside, and then get distracted trying to refashion a stapler they find laying around into a lock-picking device to open the safe.

<comment><USER10> "only use it to run your own dropper"

Can you clarify this? I'm not really sure what it means.

<comment><USER9> [https://en.wikipedia.org/wiki/Dropper_(malware)](https://en.wikipedia.org/wiki/Dropper_\(malware\))

</comment></comment></comment><comment><USER11> netstat and ifconfig tend to be deprecated on Linux in favor of ss and ip. I do have to wonder if ss was tried, but still, the learning experience is quite valuable. :)

<comment><USER12> Exactly what I thought.

I've also had to deal with a lot of this 'security'. It really inhibits normal work so much that security is probably _reduced_.

Any machine should have tcpdump, netcat, lsof, strace as well. If you want to make it more secure - well set permissions appropriately and/or use selinux/apparmor.

Hell, may as well put osquery on it as well. Why make life difficult for people to do their jobs?

At an interview I was asked how to diagnose a running program. I mentioned strace of course, but they were also looking for gdb use. I've not had gdb on any prod machine for 'security reasons'. :-(

</comment></comment><comment><USER13> I wonder if ss was available? I'd recommend looking over "Deprecated Linux Networking Commands and their Replacements"[1]. In the case of netstat, `ss -a` is a good place to start.

1\. [https://dougvitale.wordpress.com/2011/12/21/deprecated- linux...](https://dougvitale.wordpress.com/2011/12/21/deprecated-linux- networking-commands-and-their-replacements/)

</comment><comment><USER14> I can't speak to ss and ip, but apparently netstat actually operates by opening these exact files [1].

[1] [https://github.com/ecki/net- tools/blob/master/lib/pathnames....](https://github.com/ecki/net- tools/blob/master/lib/pathnames.h)

<comment><USER15> ss too, uses the same files than thew article, see:

        strace -e open ss -s

<comment><USER8> This is more an exception than a rule. ss mostly uses the netlink interface, except when using "-s".

</comment></comment></comment><comment><USER16> Reminds me of an article shared a while ago on HN where someone deleted / and ended up with all binaries gone but still had a working bash session and the /proc fs. Can't find the link, sadly.

</comment><comment><USER17> Super helpful article for a budding penetration tester such as myself! Thanks for sharing.

</comment><comment><USER18> pretty interesting.</comment>