Ask HN: Are submissions allowed more than once on Hacker News? - <AUTHOR>

Yesterday I came across an interesting wikipedia page and thought of sharing the site on HN. It was "Timeline of the far future".
Someone commented and noted that the same link had been posted multiple times before (link: https://news.ycombinator.com/item?id=8029426)
In the past, I had been unable to submit links that were already submitted at one point. Does anyone know if multiple submissions are allowed? Is this a bug? <comment><USER2> The first problem with your submission is that you submitted an incorrect URL that doesn't point to that page.

If a story has had significant attention on HN in the last year or so, we kill reposts as dupes. When posting, you should first check HN Search: [https://hn.algolia.com/?q=timeline+of+the+far+future#!/story...](https://hn.algolia.com/?q=timeline+of+the+far+future#!/story/sort_by_date/0/timeline%20of%20the%20far%20future). As you can see, it appeared in the last year. I think 23 points counts as significant, especially since there was a huge thread the year before.

Coincidentally, we updated the HN FAQ last night to answer this question.

Finally, please re-read the HN guidelines. Questions like this aren't supposed to be posted to HN itself. You should email email@example.com.

<comment><AUTHOR> I apologize. Thanks for the explanation.

Just gave the HN guidelines a thorough (and much needed) read.

<comment><USER2> > Just gave the HN guidelines a thorough (and much needed) read

Music to my ears!

</comment></comment></comment><comment><USER3> It bothers me as an older user(six years) because I see a lot of the same links. (Good lord the number of times certain Feynman articles get published is just too much—and I'm a fan that used to study physics!)

Lately though, this has seemed like less of a problem, so I wonder if something was adjusted in the HN ranking sort.

</comment><comment><USER4> This is a bug, a difficult to solve bug. The server don't canonize the url, so you have: "http" vs "https", "example.com" vs "www.example.com", "example.com/whatever" vs "example.com/whatever/", "blogspot.com" vs blogspot.co.uk" vs ... , ...

The problem is that canonization is very difficult, it's very easy to get false positives, and each site needs a slightly different canonization rules. So the current method is to use manual moderation.

And when the server restarts, it forgets the old url's for a while, so there are some possible repetitions.

And in a very few times, resubmissions are good.

Most of the resubmissions are accidental, good faith resubmissions. Someone just copy the url and the browser / link / site internationalization was different.

I think there are not official rules, but the unofficial algorithm is:

* When a submission was recently discussed, and it's resubmited the moderators may kill it.

* If the discussion was a long time ago, the resubmission may be left alive.

* If none of the previous submissions get many comments / points, the resubmission may be left alive.

* If there is an intentional systematic resubmission, the moderators may hellban the user and put the domain in the autokill list and other nasty things.

</comment><comment><USER5> Submissions to HN are allowed based on unique URLs.

Each of the submissions to that wiki article have slightly different link structures. For example, one has a trailing "/" and one does not.

</comment><comment><USER6> ... and if your attempts at self-promotion using Show HN fail miserably, is it acceptable to the community if you try again using a different headline? If so, how long in between, weeks? Months?

<comment><USER4> A better idea is to write a technical blog post about a unique feature of your site. (Bonus poits for nice graphics or photographs.)

For example read: [http://blog.wolfram.com/](http://blog.wolfram.com/) All the blog can be resumed as "You can do a lot of calculations and nice graphics with Mathematica" But many of the individual post have interesting information (and nice graphics) and got a lot of points [https://hn.algolia.com/?q=wolfram.com#!/story/forever/0/blog...](https://hn.algolia.com/?q=wolfram.com#!/story/forever/0/blog.wolfram.com)

</comment></comment><comment><USER7> My understanding is that submissions that are not accessed for some time drop out of the cache and then can be re-submitted. I can't find the comment from PG or dang though.

</comment><comment><USER8> You can do it by adding "/#" anchor link at the end of the link over and over.

Link.com/mysubmission

Link.com/mysubmission/#

Link.com/mysubmission/##

Link.com/mysubmission/###

Link.com/mysubmission/####

I'm not sure if it pisses people off but that's the technique I've used before. Resubmit at different times of day. It's tough to get your submission exposure because different crowds are on at different times of day. So really good articles can be ignored simply because they were submitted at the wrong time or wrong day and no one was there to upvote it. That's what I've always hated about Digg/Reddit/HN. You have to submit stories at some magical time.</comment>


All three major Russian carriers dump iPhone - <AUTHOR> http://www.rethink-wireless.com/2013/07/16/all-russian-leaders-dump-iphone.htm <comment><USER2>
      This is a country which Apple has never found easy - notably,
      Russia bans handset subsidies, making the iPhone an expensive
      option for most customers.

The effect is exactly the opposite; those countries where phones are mainly subsidized have also the highest total cost of ownership and ARPU (average revenue per user). The subsidized phones aren't cheap, quite the opposite.

Carrier subsidized model should be ended to bring down consumer telecommunication costs. It only benefits the carriers.

<comment><USER3> > Carrier subsidized model should be ended to bring down consumer > telecommunication costs. It only benefits the carriers.

In the US at least, everyone has to get on LTE before that becomes a reality. Sprint and Verizon 3G and older handsets can't speak the same language, I don't think. AT&T and T-Mobile both use GSM based systems so phone calls are possible using a handset form the other carrier, but their 3G frequencies are different - so you're hosed on mobile data. So, there's a de-facto lock into a carrier with your device even if there isn't one in the contract. (And $600 for a new device just for a different carrier is a hard sell, in a world where subsidies are gone.)

It's happening though. The problem is that companies like HTC charge the same price Apple does for the iPhone for what is a lower quality of product, in my experience. Once again, personal experience, but the number of hardware problems I've had within the walled garden of Apple are few compared to those in my Android days. And those that I've had have been fixed quickly by an Apple store representative, whereas with HTC or Verizon it was a multiple week ordeal.

That's something that makes the product itself even more valuable to me.

<comment><USER4> I've always wondered: can a single phone have radios for all the different carriers? Could they share a single antenna? Could one be activated by the user selecting which network to get on?

<comment><USER5> > can a single phone have radios for all the different carriers?

All carriers in a single country? Yes, many do. All carriers worldwide? You'd be putting a lot of radios in the thing.

> Could they share a single antenna?

Not a radio expert, but I think each antenna has to be tailored to each wavelength. Maybe you can get away with one per band (one for 800-900, one for 1800-1900, one for 2100-2600)

> Could one be activated by the user selecting which network to get on?

You'd need a SIM card for each carrier you want and swap them out, and for American CDMA carriers you'd have to have your ESN registered with their networks beforehand.

<comment><USER6> All carriers worldwide would be tough, but a sufficiently multi-band GSM phone will typically give you pretty much worldwide roaming from a variety of carriers.

I'm in Europe. The last time I had a phone I couldn't effortlessly take to the US or Asia and use roaming was probably about '99.

<comment><USER5> Yeah, most modern phones will work most everywhere, but there'll always be one carrier's 3G or 4G network you'll miss out out. Some phones have really impressive radios in them now. Here's the Sony Xperia Z Ultra:

2G Network: GSM 850 / 900 / 1800 / 1900

3G Network: HSDPA 850 / 900 / 1700 / 1900 / 2100

4G Network (C6833 model): LTE 800 / 850 / 900 / 1700 / 1800 / 1900 / 2100 / 2600

4G Network (C6806 model): LTE 700 / 850 / 900 / 1700 / 1900 / 2100 / 2600

With one of these babies, pretty much all you're missing out on is extended range areas on Docomo Japan (800 MHz)

</comment></comment></comment></comment></comment></comment><comment><USER7> Not a single Russian iPhone user I know bought it in carrier store. Mostly they've got it from Europe trips, others in non-official stores where prices 15-20% lower.

So, despite official sales numbers are low, there is a remarkable iPhone user base here.

</comment><comment> guard-of-terra People around me link this to official Apple Store opening.

You can now order iPhone there over the internet, carriers can't realistically compete with this because they'll lose money selling for less that Apple Store and sell zero phones selling for more.

(I've never owned nor bought anything from Apple)

<comment><USER8> The carriers sell the phones for less and make money back through the service plan. That's the basic model for carriers offering phone subsidies.

<comment><USER9> Not sure about Russia, but in other countries there are tight regulations for phone subsidies (treating them for what they are: dumping of retail price with the purpose of locking in users - a blatantly anticompetitive practice that would be illegal in most industries).

Strong competition has a similar effect. If the SIM free contracts are good enough, people are not going to get into lengthy, expensive contracts. This doesn't happen in places like the US because there's a very strong cartel/oligopoly there that affords them the right to simply deny you a terminal if it's not under the carrier's conditions.

If Apple is selling for cheaper than the carriers can afford to sell, then they will simply do what they are doing: stop selling it.

<comment><USER10> How is selling you something on credit 'anticompetitive'?

<comment><USER11> Danish (and I believe EU) regulation has a comprise. Subsidized phones are legal, but 1) the longest time you can be bound by a "plan" is 6 month, and 2) all advertising must include the total cost of ownership in the binding period.

This makes it blatantly obvious to the consumer that the unsubsidized phones are cheaper.

</comment></comment></comment></comment></comment><comment><USER12> Ex-russian here.

This might be counter-intuitive to many people who never lived in Russia, but: mobile phone service contracts in Russia almost do not exist, entire market model for carriers is quite different.

Carriers don't sell SIM-locked phones for over decade now in Russia. Most common contract features are implemented as pay-as-you-go with extra packs (quite similar to what giffgaff does in the UK). This is probably due to the fact that CRA and other similar financial institutions in Russia are currently in their earliest stages of development.

Mobile phones are usually purchased absolutely independently from the mobile service contract, and almost never purchased from carriers: electronics stores usually offer better prices, and all Russian carriers are happy to sell SIM- only prepaid contracts (also see PAYG model notice above).

So overall you should read this news as: "major russian mobile companies decided that long-term contract model required to sell 'cheap' iPhones in russia is still too risky and not feasible", no conspiracies.

<comment><USER13> This is not Russia specific... it's like that in most countries of the world.

</comment></comment><comment><USER14> "Apple dumps all three major Russian carriers" is how I would read this heading. Everyone I know in Russia has or wants an iPhone, Apple isn't losing any market share there, because you can get an iPhone from Apple directly.

<comment><USER15> Yep, we have our share of Apple funboys here in Russia)

</comment></comment><comment><USER16> >> Russia has over 180m mobile subscribers

The whole population is 143M (2011) including infants and very old people. How is it possible? Multiple phones per person?

<comment><USER17> Brazil has around 190 million people[1] with about 249 million mobile phone lines enabled [2].

People that have more than one line usually buy one line from each provider so they can talk "for free" between the same providers. It is not uncommon for people to ask you what is your provider so they will mark it and call from a phone that uses that provider when trying to reach you.

We also have a high number of people with phones that allow multiple providers with one device.

Business and personal cellphones are not uncommon either.

[1] [http://www.ibge.gov.br/home/estatistica/populacao/censo2010/...](http://www.ibge.gov.br/home/estatistica/populacao/censo2010/default.shtm) [2] [http://extra.globo.com/noticias/celular-e- tecnologia/quinto-...](http://extra.globo.com/noticias/celular-e- tecnologia/quinto-pais-no-numero-de-celulares-brasil-pode-subir-ainda-mais-no- ranking-com-desoneracao-de-smartphones-8123620.html)

<comment><USER18> To make a example as brazillian:

My close family is 4 people (my dad, mom, me, sister).

We have 6 lines from one carrier (we got it in a special package, we use 1 line per person + 2 backup lines for whatever we need, since those lines can call very cheap each other, one use for backup line for example is borrow it to extended family)

Also my dad has one workplace line with another carrier, and a Nextel Android.

And I had for a while a extra line to communicate cheaply with people of the same carrier, thus in total we have 9 mobile phone lines for 4 people.

Sometimes you see particularly important business people (like a CEO) here walking with 4 or 5 phones on his belt (one of each carrier + 1 for strictly personal stuff)

<comment><USER19> A CEO is carrying 5 phones on his belt so he can save a few pennies using same-carrier free calling? What, does he have those 5 numbers also printed on his business card?

<comment><USER20> In most countries the caller pays, not the callee. So he is saving his callers money, not himself.

</comment></comment></comment></comment></comment><comment><USER21> FTA "which has helped enable the Microsoft OS to overtake iOS, gaining 8.3% of smartphone sales compared to 8.2% for Apple"

So maybe windows phone will actually become a competitor to android in russia?

<comment><USER22> Unless they have a very loose definition of smartphone or I'm missing something, it doesn't sound like anything is competitive with Android in Russia?

</comment></comment><comment><USER23> Any one care to tell us how good/bad is a market that outlaws phone subsidies?

Is there better competition on service quality and cost or does it end up being just the same?

<comment><USER24> In three words: it is great! Not sure what lack of phone subsidies has to do with it though, or even if there is such a thing.

I lived in Russia and now live in Canada, and I would take Russian mobile market over Canadian any day. Dirt cheap, fierce competition between operators, over-abundant coverage, hidden fees are much less prevalent, there are contracts but pretty much everyone is on prepaid, buying new SIM card is a breeze.

I do not know if subsidized phones are forbidden, but much more likely the tradition of buying own phones is historical. Back in 90s owning a phone was a status symbol, there were no credit bureaus, no credit histories, inflation was rampant and ruble FX rate was very volatile. Everyone trusted only cold hard cash (preferably USD).

Later the country was swamped with lots and lots of cheap phones. So-called "grey" phones (euphemism for "smuggled") were brought in circumvention already lax or non-existent import regulations.

So I guess it is just not in the Russian tradition to sell subsidized phones.

<comment><USER25> I completely disagree. Passport information is collected upon purchasing a SIM-card diminishing all hopes of privacy. Furthermore, there is roaming between states/provinces in Russia, so a SIM bought in one state is extremely expensive to use in another. I know for a fact that this isn't the case in the US and Canada. But, yes, generally prices are lower in Russia.

<comment><USER26> > Passport information is collected upon purchasing a SIM-card

Not always. You can purchase anonymous SIM too. I did it two years ago on a train station. Price for calls is a little higher this way though.

</comment></comment></comment></comment><comment><USER27> I wonder how much of it's related to fears that every iPhone is a secret NSA listening device?

I mean if you knew the KGB could possibly listen in on your phones, how likely would US carriers keep it?

I've been thinking an overseas consumer and business/government backlash against US technology products is inevitable now that the NSA being in bed with every major US technology company is public knowledge.

<comment><USER28> What phone isn't? Is even cyanogenmod safe from the NSA?

<comment><USER29> I think so, since it's fully community made, and beyond the "official" CyanogenMod team, there are hundreds more developers making "unofficial" CM ROMs for other phones, and they get to see the code, too.

Plus, the CyanogenMod team is now working on encrypted end-to-end messaging that is built-in:

[https://plus.google.com/+CyanogenMod/posts/23vfN2qdZTu](https://plus.google.com/+CyanogenMod/posts/23vfN2qdZTu)

<comment><USER30> No, it's not, because everything useful that happens with your phone happens over a carrier network or the internet, both of which are available to the NSA.

End-to-end encrypted messaging is a good sign, but it doesn't matter how "community made" your endpoints are - the NSA has the network.

<comment><USER29> Yes, but if the communication is encrypted it doesn't really matter. So these text messages should be by and large _safe_.

I agree about the voice calling, though, which is why I'd like them to use RedPhone as the default dial app, too, so at least you can have encrypted voice conversations when connected to the Internet.

</comment></comment></comment></comment></comment><comment><USER31> So, none of the operators were angry about possible (likely) NSA leaks?

<comment><USER7> Why would they bother? There is a local version of PRISM named SORM](http://en.wikipedia.org/wiki/SORM">SORM) deployed at every provider.

</comment></comment><comment><USER32> [http://shop.beeline.ru/msk/iphones](http://shop.beeline.ru/msk/iphones) this is weird. Maybe they haven't updated their site though.

</comment><comment><USER33> "draconian contracts" for me this means unlimeted Data Plans, some carriers only offer that for iPhone - at least this is valid for Romania (last time I checked).

</comment><comment><USER34> If the iPhone is considered "expensive" due "high import taxes" at $920, how is it selling on the black market for $3500?

<comment><USER35>
      On the black market, iPhone 5s have even been reported reaching over $3.500

Maybe a typo but this is an unreleased product.

<comment><USER36> That's "iPhone 5s", the plural of "iPhone 5". "iPhone 5S" is the unreleased product.

<comment><USER37> That's a written language problem Apple didn't see coming. It not as bad as the situation you get when you try to discuss the new New iPad though. The Le Ferrari has managed a similar achievement too.

</comment></comment></comment></comment><comment><USER38> Russia leading the way once again.</comment>


Color Thief, script for grabbing the color palette from an image - <AUTHOR> http://lokeshdhakar.com/projects/color-thief/ <comment><USER2> I built a similar tool for this purpose recently. It uses Canvas, but falls back to Flash if Canvas is not supported. Here's an example - hover and click the image - <http://www.imagecolorpalette.com/image/40>

<comment><USER3> Why does it turn magenta when you double-click (on two distinct pixels specifically)?

<comment><USER4> That's just the highlight color. Assuming you're using Chrome, it's pretty eager about highlighting random stuff; other browsers may be similar. Try selecting some text.

<comment><USER5> There is a css rule to disable user selection which might be handy here: <http://stackoverflow.com/questions/826782>

</comment></comment></comment></comment><comment><USER6> I'm not a programmer, but I see an opportunity here:

Go to istockphoto.com - this is a site where you can buy images for use in any type of publication, advertising etc.

The opportunity I see would be to offer to write software which indexes their entire collection by color.

Then when someone wants to buy an image for a publication they could put their colors in and have istockphoto only show them images which fit the color palette.

This would save designers a ton of time.

<comment><USER7> iStockPhoto already has a "search by color" feature which does pretty much what you're suggesting. <http://screensnapr.com/v/gTjWCw.png>

<comment><USER6> sorry, that must be new then. It has been a little while since I logged in there.

<comment><USER8> Sorry in advance as I'm not trying to make you feel stupid but.. I used to use that feature a lot before they were acquired by Getty (and their prices got so ridiculous I stopped using them.) So at least six years ago.

</comment></comment></comment></comment><comment><USER9> Wow, I can use this. This is great. I run an online football game <http://deeproute.com> and players upload helmets to go with their team.

I can use this to check that helmet image they are using, and grab the color palette and then use those colors to drive their color scheme.

Brilliant! Thanks!

</comment><comment><USER10> This could produce an interesting and possibly strange image viewer if you use it to control the color scheme of the page. For example, you upload an image and the css of the page itself updates to be complimentary to the photo.

</comment><comment><USER11> Very nice. I would expect a provision to upload (or add URL of) an image and test it on the demo page before downloading it and using it on my website.

</comment><comment><USER12> You can build a similar thing server side using Imagemagick. It has a great suite of tools for doing image quantization and other manipulations.

<comment><USER13> I've tried. I've failed. I would have loved seeing this been done in Imagick on PHP.

<comment><USER14> You can do it like this with ImageMagick on the command line. This will quantize the image to 8 colors and list them in descending order of occurance:

      convert image.jpg -resize 400x400 -format %c -dither None -quantize LAB -colors 8 -depth 8 histogram:info:- | sort -r

<comment><USER15> Sorting should be done with _sort -r -n -t :_

</comment></comment></comment></comment><comment><USER16> Could you make it show the RGB of the color of the square color block I'm hovering over in the palette area?

</comment><comment><USER17> Clusterfck is a nice JS library for hierarchical clustering: <http://harthur.github.com/clusterfck/>

I put together a tiny tool like the posted link for playing with clusterfck:

<http://dylanfm.github.com/Ladderfck/>

Kind of handy for getting a feel for what you can do - e.g. trying different distance metrics and linkages. It too uses canvas.

</comment><comment><USER18> Can someone please write this in ruby? :)

<comment><USER19> no, now go jump into a lake

and take your hipster language with you

<comment><USER20> Hacker News has standards of discussion. Learn them.

</comment></comment></comment><comment><USER21> Clever use of <canvas>. I made an Android app that does something similar:

[https://market.android.com/details?id=uk.co.opeso.android.co...](https://market.android.com/details?id=uk.co.opeso.android.colorscheme)

</comment><comment><USER22> It would be interesting to see if you could use a system like this to automagically derive background colors for image display, picking muted complimentary colors to make the dominant color in an image "pop", so to speak.

<comment><USER23> You can do this by representing the colors into HSV and then modifying the hue by 180%, desaturating them and then adjusting the brightness (v). Most color libraries will let you deal in RGB(A) or HSV so this is pretty straightforward to DIY.

</comment></comment><comment><USER24> That is rather nice! <http://www.colr.org> has something similar, but it relies on server-side code. An all-javascript solution is a cool thing.

</comment><comment><USER25> FYI Mobile Safari on iPhone 4 and iPad 1 crashes every time for me.

<comment><USER26> Have you reported it to Apple?

</comment></comment><comment><USER27> <http://colorapi.com/> is similar but better suited for finding color theme inspiration.

</comment><comment><USER28> what does it mean when the colours get bigger as i mouse over them? is something else meant to happen? i guess that should mean they are clickable, or that something else is changing, but i don't see anything else change and nothing happens when i click. is something broken (chrome latest stable, linux)?

[edit: and if not, please don't do that.]

<comment><USER29> I think if the users can copy the Hex codes by clicking on individual colors, it can be a useful service.

</comment></comment><comment><USER30> I always wanted something like this and I even thought of building it, but I didn't have time to learn javascript.

</comment><comment><USER31> This kind of reminds me of <http://www.pictaculous.com/>

</comment><comment><USER32> Is white not considered for dominant colour? In the apple logo pic, white is clearly the dominant...

<comment><USER33> Looks that way:

[https://github.com/lokesh/color- thief/blob/master/js/color-t...](https://github.com/lokesh/color- thief/blob/master/js/color-thief.js#L84)

</comment></comment><comment><USER34> I wonder if it works with cross-domain images. I could not see in the documentation if it does.

<comment><USER35> I haven't tested it, but I would expect so. There aren't many cross-domain restrictions in browsers for images.

<comment><USER36> Yes, there are. That is why this thing exist: <http://www.maxnov.com/getimagedata/>

So probably it doesn't work for cross-domain images.

</comment></comment></comment><comment><USER37> Adobe Kuler also has this feature.</comment>


Woman Strips Down to Her Lingerie, Gets TSA Pat Down Anyway—Twice - <AUTHOR> http://gizmodo.com/5703878/

<comment><USER2> To be fair, if you go to the airport looking for attention, you will find it.

</comment><comment><USER3> She reminds me of this chick: <http://www.youtube.com/watch?v=XDAaTzccCik></comment>


Facebook pong easter egg - <AUTHOR> http://28.12.channel.facebook.com/pong

<comment><AUTHOR> and if you do this: <http://28.12.channel.facebook.com/p>

yet more strangeness.

direct link to the first: <http://28.12.channel.facebook.com/pong>

you can change the first 2 numbers to almost anything and it owrks... i just picked my birthdate ;)

</comment><comment><USER2> To be fair, not really an easter egg.

<comment><AUTHOR> for lack of a better name... in any case - Im just curious to understand why it happens</comment></comment>


Allen Hills Mars Meteorite: 20 Years Later, Debate Continues - <AUTHOR> https://www.space.com/33690-allen-hills-mars-meteorite-alien-life-20-years.html?20 <comment><USER2> _" …we really do not have a good working definition of what constitutes life. At the most fundamental level, we still do not know whether the difference between animate and inanimate is simply a difference in kind or degree."_

What would a "difference in kind" look like?

I thought it was pretty well established at this point there's no magic, that "life" is just another layer in the layers of natural complexity, how matter organizes itself, from physics to chemistry to biology to sociology and ecology.

Why would there be an expectation of a sharp (non-linguistic) boundary between "life" and "non-life", animate and inanimate, physics and chemistry, chemistry and biology?

<comment><USER3> There are sharp phase transitions between different phases of matter. It's a difference in kind, but it's all made out of atoms.

<comment><USER4> Except when there isn't. [https://en.wikipedia.org/wiki/Critical_point_(thermodynamics...](https://en.wikipedia.org/wiki/Critical_point_\(thermodynamics\))

> "Above the critical point there exists a state of matter that is > continuously connected with (can be transformed without phase transition > into) both the liquid and the gaseous state. It is called supercritical > fluid."

<comment><USER3> I'm well aware of this and it doesn't change my point.

<comment><USER4> Which is that we should expect a sharp transition between life and not-life because there are sharp transitions between some phases?

I don't see how that fits. Even if you think of "life" as a phase transition, why does it have to be a sharp transition?

<comment><USER3> It doesn't have to be. I'm just rebutting Radim's suggestion that since living things and non-living things are both made out of atoms we should expect that the difference between living and non-living is only of degree rather than kind.

<comment><USER4> The phrase was "Why would there be an expectation of a sharp ..."

I agree - why would we expect a sharp transition?

When a radioactive particle decays, it too is a sharp transition. (By which I mean the time scale is very short. At sufficiently fine resolution, it's hard to tell exactly when an atom of element X becomes something other than an atom of element X.)

That there are examples of very fast transformation doesn't mean that Radim is incorrect in asking why we would expect the difference between life and non- life to be one of those sharp transitions.

In a tabula rasa sense, it might be a good cautionary reminder to consider exceptions like what you pointed out. But it's "pretty well established" (to quote Radim) that the boundary between life and non-life is not distinct. The standard question is, are viruses alive? The usual answer is "no", but [https://en.wikipedia.org/wiki/Mimivirus#Implications_for_def...](https://en.wikipedia.org/wiki/Mimivirus#Implications_for_defining_%22life%22) pushes it close to yes.

Humans have created an organism with a synthetic genome. Give another decade or two and biologists will be creating semi-alive forms to explore what the boundary means.

Further, there are definitions beyond the strictly biological one: [https://en.wikipedia.org/wiki/Life#Definitions](https://en.wikipedia.org/wiki/Life#Definitions) .

<comment><USER3> > it might be a good cautionary reminder to consider exceptions like what you > pointed out.

It's not just a single exception, it's a huge class of counterexamples which is the focus of study for much of the field of condensed matter physics.

<comment><USER4> And there is an much larger class of examples.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> The scientist stated this fact:

> Magnetite crystals from Mars might more persuasively indicate life if > they're found in string-of-pearl formations, as they are when created by > bacteria on Earth, Chris McKay said. "If we find these chains on Mars, it > would be compelling evidence of past magnetotatic bacteria," he said. > However, so far researchers have not detected such strings, he added.

And later in the article it goes on to say:

> "The Mars Exploration Rovers, Mars Reconnaissance Orbiter, Phoenix lander, > Curiosity rover, Mars Express mission with its Beagle 2 Mars lander — all > can be traced to the ALH 84001 research.

So they have not been able to find any other examples of magnetite? I'm curious why not? Is it something that will be rare if it does exist? Or given that the original sample was billions of years old that any surface level digging by a rover would likely have eroded it away?

<comment><USER6> Are the rovers even equipped with anything that could detect microscopic magnetite string-of-pearl formations?

<comment><USER7> I think they meant the rock spurred _general_ interest in further exploration, not just magnetite research.</comment></comment></comment>


Fiat currencies not as Centralized, Bitcoin not as Decentralized, as you think - <AUTHOR> https://medium.com/metacurrency-project/national-currencies-arent-as-centralized-and-bitcoin-isn-t-as-decentralized-as-you-think-fa2afa022a2b#.b00mjlbj9 <comment><USER2> This is ridiculous. Anyone can go buy the hardware and actively be involved in the hashing and therefore production/operation of Bitcoin. What happens if I go set up a printer for USD? the two systems are absolutely fundamentally different. If centralized bank money is so great then why does it take 585 of the US ones to buy a single bitcoin?</comment>


Bypassing Antivirus with Ten Lines of Code - <AUTHOR> http://www.attactics.org/2016/03/bypassing-antivirus-with-10-lines-of.html <comment><USER2> That's a common mistake of people that do not understand how AV's work and how virustotal work. From their own FAQ:

At VirusTotal we are tired of repeating that the service was not designed as a tool to perform antivirus comparative analyses, but as a tool that checks suspicious samples with several antivirus solutions and helps antivirus labs by forwarding them the malware they fail to detect. Those who use VirusTotal to perform antivirus comparative analyses should know that they are making many implicit errors in their methodology, the most obvious being:

-VirusTotal's antivirus engines are commandline versions, so depending on the product, they will not behave exactly the same as the desktop versions: for instance, desktop solutions may use techniques based on behavioural analysis and count with personal firewalls that may decrease entry points and mitigate propagation, etc.

-In VirusTotal desktop-oriented solutions coexist with perimeter-oriented solutions; heuristics in this latter group may be more aggressive and paranoid, since the impact of false positives is less visible in the perimeter. It is simply not fair to compare both groups.

-Some of the solutions included in VirusTotal are parametrized (in coherence with the developer company's desire) with a different heuristic/agressiveness level than the official end-user default configuration.

thanks,

<comment><USER3> I do not understand how this is a response to the article, which doesn't make a comparison so much as claim that all AV suck.

<comment><USER4> It seems to say that just because something passes virustotal with 0/56 doesn't necessarily mean it would pass every AV in a real attack.

<comment><USER3> Does the comment actually say that, though? It would make sense if the comment said that, but e.g.

> perform antivirus comparative analyses

is not what is happening here.

<comment><USER5> > -VirusTotal's antivirus engines are commandline versions, so depending on > the product, they will not behave exactly the same as the desktop versions: > for instance, desktop solutions may use techniques based on behavioural > analysis and count with personal firewalls that may decrease entry points > and mitigate propagation, etc.

It says right in the parent comment that the VT results may not actually reflect how the product performs in the real world.

</comment></comment></comment></comment></comment><comment><USER6> This is interesting for signature based AVs. More interestingly bypassing dynamic AV engines that execute code in a sandbox seems to be fairly trivial as well. For example allocating 100mb or memory, running a few million iterations in a loop during startup will cause most av engines to stop executing the code due to resource constraints. This paper is a really interesting read on the topic[0]

0: [http://www.sevagas.com/IMG/pdf/BypassAVDynamics.pdf](http://www.sevagas.com/IMG/pdf/BypassAVDynamics.pdf)

<comment><USER7> I guess you could only fix this with a whitelist instead of a blacklist approach.

<comment><USER8> A whitelist might not even work. In practice whitelists end up with everything in the world on them. It might not be that hard to find a whitelisted program that would let you do the thing in the article just by passing it the right arguments. Certainly anything with a buffer overflow in it would work and there are probably a hundred other ways to do it too.

<comment><USER9> Yep, one place where I used to work, the owning company's policy had "Linux" on its whitelist, but not "Wireshark". Even though the company was developing software which communicated over the network in various protocols.

</comment></comment></comment></comment><comment><USER10> I once wrote a kernel extension that intercepted any and all file open()'s on OS X. If the application in question was opening a file that it was not whitelisted to do so, it would bring up a modal dialog box asking whether or not this application should be allowed to open this file.

It was basically a firewall on the kernel level.

It worked splendidly, however, I was never able to gain any traction in marketing it. That was back at around OS X 10.4 now. I've been waiting for another company to come along with something similar - since it really does seem a comprehensive way of blocking viruses (albeit more suited to more advance uses). I'm still waiting for something like that.

<comment><USER11> Neat, but most people will just click "yes."

See Windows Vista.

<comment><USER12> That's what happens when the answer is "yes" 99.9999% of the time. Have you ever seen a case where "no" is appropriate?

</comment></comment></comment><comment><USER13> It's initially worrying, but it makes a certain amount of sense when you think that an antivirus is a blacklist (of _known vulnerabilities_ ), not a whitelist.

Somewhat related: [http://security.stackexchange.com/a/117312](http://security.stackexchange.com/a/117312)

</comment><comment><USER14> A couple of things:

\- This wouldn't work for larger payloads. AVs flag binary looking data that is larger than a certain size and that is later processed or assigned to a variable.

\- Their veil project has some problems. Py2EXE gets marked as malware by some AVs in many cases just because it is Py2EXE. Same thing with non-commonly used obfuscators. Basically, they just pick up on the fact that something is obfuscated. If that obfuscator is not commonly used in goodware programs, it is marked as malware. This is kind of a dumb strategy on the part of AV engines, but it works okay.

You're not going to catch new malware with static (or dynamic for that matter) analysis anyway. Thing is, the problem is ill-defined.

What is malware?

Is it a program that does something a user doesn't want? If users knew what regular programs do, they wouldn't be okay with most of it either.

Is it a program that does some obfuscating tricks and exploits undocumented functionality in the system? Plenty of legit programs including a lot of AV engines do that as well.

The only usable definition in my opinion is that it is a program that makes the user unhappy with no easily accessible way of removing it completely.

This is why the only solution seems to be to only allow installation from a trusted repository. I am still not sure why Windows/Apple OSX haven't adopted such a strategy (with a developer mode override option for some advanced users).

<comment><USER15> > What is malware? Is it a program that does something a user doesn't want?

OS X Sandboxing seems to have the right idea: Instead of worrying about what the user _doesn 't_ want, _do only what the user WANTS._

Basically, sandboxed apps don't have access to files and folders other than the ones that the user explicitly chooses in an Open/Save dialog. It's a surprisingly nag-free opt-in mechanism that "just works."

After that, automatic backups will let users revert any undesirable changes to their data, whether they were made by their own selves or by malware.

I think operating systems should just do a better job of making the user more aware of all recently-modified files, especially if a process has been modifying a large number of them in a short time (the recent ransomware comes to mind) or if a third-party background process has been generating an uncanny amount of network traffic.

Seeing something like "1,590 files modified" on log-on or in a notification, is way more alarming and would make users take immediate action, compared to all the usual OS or antivirus nags that we are all accustomed to subconsciously agreeing to.

<comment><USER16> Re: OS nags...Windows is the worst IMO since UAC basically just asks ~it looks like you're trying to do a thing, are you sure you want to do that thing? The "open this program FROM THE INTERNET????" nag on OSX and Win are also pretty pointless. I've never not clicked yes, but there's not an easy way to disable this. It looks like it CAN be disabled with registry tweaks or GPEdit: [http://www.sevenforums.com/tutorials/182353-open-file- securi...](http://www.sevenforums.com/tutorials/182353-open-file-security- warning-enable-disable.html) but kind of ridiculous that it's not just an option, given how little it increases security (how many people don't just blindly click "open"?)

<comment><USER14> What if it's a browser exploit that basically got the browser to "download and open"? Then this would be the second line of defense - something is about to run even though you didn't know expect it to run.

<comment><USER17> The "execute downloaded executable" protection predates UAC. It was introduced in Windows XP SP2 and UAC implemented its own version of it, which created double prompts:

[https://blogs.msdn.microsoft.com/e7/2008/10/08/user- account-...](https://blogs.msdn.microsoft.com/e7/2008/10/08/user-account- control/)

That being said, the "execute downloaded executable" protection only applies when the binaries are marked as being from the internet, which is easy to bypass. You just need to download the software in a way that the mark is not applied (e.g. not through a software mechanism designed to apply it). Furthermore, attacks need not rely on "execute downloaded executable". They just need to achieve code injection, which bypasses the need to run through the "execute downloaded executable" process. However, any injected code would be able to download and open without triggering UAC by not marking the file as having been downloaded from the internet. Such file marking is entirely voluntary and malicious code would likely never voluntarily do it.

Protection against inadvertent execution of software downloaded from the internet is the only function in UAC that is useful. The other functions are designed to operate on already executing software. Since already executing software can gain system privileges (above administrator privileges) via vulnerabilities the that Microsoft refused to fix, it can do basically anything it wants. UAC is fairly useless against it because anything it wants includes turning off UAC. I know enough about Windows security that I stopped using Windows years ago, so I do not know whether UAC would require malicious software that has gained system privileges to turn it off. If it does, it should be a simple matter. Anyway, the hot potato proof of concept code demonstrates gaining system privileges by exploiting such vulnerabilities:

[http://foxglovesecurity.com/2016/01/16/hot- potato/](http://foxglovesecurity.com/2016/01/16/hot-potato/) [https://youtu.be/Nd6f5P3LSNM](https://youtu.be/Nd6f5P3LSNM)

The video of it running on Windows 7 uses the system privileges to give a regular user administrator privileges. There is a Windows firewall prompt that appears in the video, but it does not stop the exploit. The appearance of the prompt ought to be avoidable because whatever triggered the Windows firewall prompt was not necessary for the exploit and could be removed.

</comment></comment></comment></comment></comment><comment><USER18> Is this an oversight of the AV software companies? Did no one come up with this before? Could it be that if people did come up with this before that a lot of Windows computers have viruses without them knowing? Is their virus detection scheme fundamentally flawed?

Should I be shocked? Shouldn't I be? I'm currently shocked but I don't know if it's justified, not an expert in the field.

<comment><USER19> No, this is not surprising at all.

It's impossible to determine whether software is malicious or not (Rice's theorem).

Antivirus software only reliably detects code that is identical to known malicious software.

<comment> V-2 That's not true - there's heurystic analysis techniques, generic signature detection etc. Of course they may not meet your definition of "reliably"

<comment><USER17> > That's not true - there's heurystic analysis techniques, generic signature > detection etc. Of course they may not meet your definition of "reliably"

The only reliable things in security are the things that an attacker cannot bypass even when knowing that they in use (e.g. RSA). The premise of the article being discussed is that heuristics are trivial to bypass.

</comment></comment></comment></comment><comment><USER20> You're not secure until you whitelist. And even that's not a guarantee; it's a necessary but sufficient condition. But systems which do not run signed, whitelisted code from boot time forward are as good as pwnt.

<comment><USER21> > You're not secure until you whitelist.

No, a whitelist isn't good enough. You can't anticipate an exhaustive list of the programs the user will want to run.

What you can do, however, is enforce a policy by which programs are required to provide machine-checkable evidence, also known as proof-carrying code [[https://en.wikipedia.org/wiki/Proof- carrying_code](https://en.wikipedia.org/wiki/Proof-carrying_code)], that they respect the system's safety policy.

> it's a necessary but sufficient condition

Perhaps you mean "not sufficient"?

<comment><USER20> Yes, that's what I meant -- not sufficient.

Whitelisting seems to be working out well for Apple. It's a big part of why they're the most secure smartphone platform.

</comment></comment></comment><comment><USER22> Laughable. Ages ago I realised you could just take an exploit, base64 the contents of the binary code, save it in a string. You could then unbase64 it and execute the binary in memory. Nothing seemed to catch it.

<comment><USER23> You'd still need a decoding stub which can be fingerprinted. An XOR "decoder" is far smaller in shellcode and can be custom written in asm to reduce time- to-first-signature.

<comment><USER24> You could just pepper the b64 with jmp statements and randomly intersperse your decoder code.

</comment></comment></comment><comment><USER25> We just need a better sandboxing environment and individual permissions per excitable i.e. "Can this excitable connect to blah blah up?" , "can this program read outside of its sandbox folder?"

When you don't root your android or iPhone they handle it a lot better than desktop operating systems.

<comment><USER7> Only in the newer Android releases.

</comment></comment><comment><USER26> I used to work at a large, well known AV company. While much of what other commenters have said is true, I will note that VT is less authoritative than some realize; the version of our product that VirusTotal was given access to was substantively different than our normal product; features such as sandboxing were removed.

I wouldn't be surprised if this was a common practice; we considered our product's detection capabilities to be proprietary.

</comment><comment><USER27> So after the program is actually compiled into binary code do the resulting instructions become so simple (and so fundamental to the operation of programs) that any attempt to write a heuristics rule to stop this technique would break thousands of programs or are heuristics just so inherently shitty that this technique works? Because I would still think that this line here:

void *exec = VirtualAlloc(0, sizeof c, MEM_COMMIT, PAGE_EXECUTE_READWRITE);

Would be a red flag. I don't know how many programs use that on the Windows side of things but I have almost never seen programs call this function that weren't "crypters."

I am honestly impressed how simple this program is (it's the most elegant crypter I've probably ever seen) but I am still wondering whether heuristics could be made to detect this (without also falsely detecting thousands of other programs?) What do you think OP? Not really my field but curious all the same.

<comment><USER28> I believe this is how JIT compilation works. So this would trigger a false positive on the Java VM, Javascript V8 interpreter, Spidermonkey, and C# runtime.

<comment><USER29> Also, most emulators of reasonably modern systems. Although JITs in VMs are a few orders of magnitude more common.

</comment></comment></comment><comment><USER30> It'll work until a virus scanner company distributes a virus definition file containing a signature of your offending code.

<comment><USER31> And then you change the code a tiny bit and the signature doesn't catch it anymore.

<comment><USER32> > change the code a tiny bit

How? That's the principle that everyone's missing! Sure, you can change the code a bit, but the millions of copies of your worm out there already can't magically change themselves after AV vendors update their blacklist. Sure, you can make a polymorphic worm, but even after mutation, your worm still probably has common patterns that blacklist makers can catch.

I'm a fan of proactive, not reactive security --- but let's not pretend that what AV vendors are doing is completely bogus.

You also have to keep in mind that blacklists are behavior heuristics are more suited to a bygone age --- one of low-bandwidth, sporadic connections and floppy disks that could be infected with trojans. Nowadays, we're more worried about drive-by 0day flash exploits from ad networks than about infected, self- propagating executables.

I've always found it fascinating that the human immune system has _both_ proactive and reactive security, just like our computers should. The innate immune system [1] is analogous to mandatory access control, OS file permissions, buffer hardening, and other non-specific security mechanisms. The adaptive immune system, on the other hand, works like a blacklist updated throughout your life (and propagated from mother to child!).

Both systems catch threats the other does not. There's still a lot to learn from biology.

[1] [https://en.wikipedia.org/wiki/Innate_immune_system](https://en.wikipedia.org/wiki/Innate_immune_system) [2] [https://en.wikipedia.org/wiki/Adaptive_immune_system](https://en.wikipedia.org/wiki/Adaptive_immune_system)

</comment></comment></comment><comment><USER27> So what's this line all about:

((void(*)())exec)();

Type cast a void pointer to a void pointer, execute the result, then execute the result of that? Or ... Can anyone explain what's going on here?

<comment><USER33> This is a common way of executing shellcode in a PoC.

Exec (before the cast) points to memory containing the shellcode data.

To actually start executing the shellcode, you just need to somehow cause the program counter to point to the address of the shellcode.

An easy way to change the program counter is by calling a function ... which is what this line does.

Read this as "cast exec to a pointer to a function that takes zero arguments and returns void and call the function with no arguments."

This is the same as:

       typedef void (*some_func)();
       some_func func = (some_func)exec;
       func();

To familiarize yourself with C syntax regarding pointers, read about the "right-left rule" [1]

[1] [http://ieng9.ucsd.edu/~cs30x/rt_lt.rule.html](http://ieng9.ucsd.edu/~cs30x/rt_lt.rule.html)

<comment><USER27> This is insanely clever. Thanks a lot for the explanation

</comment></comment></comment><comment><USER34> AV is a little and often useless supplement to security, nothing more. It has always been trivially easy for a script kiddie to write malicious software that passes all of VirusTotal. In the decade I've been using AV, I had less true positives than false positives (yes please auto-delete my patches, hacktools and software I've written myself, idiot AV program) and of course some false negatives that wrecked me because I executed them. Since I stopped using AV and became more careful (e.g. use VM for suspicious files) I was never infected again.

</comment><comment><USER35> Can do you one better.. bypass Sophos AV in a single byte.. [https://www.youtube.com/watch?v=4xEzqISAmKI](https://www.youtube.com/watch?v=4xEzqISAmKI)

</comment><comment><USER36> lets not forget that this isn't actually all that practical in practice.

It's not executing any old binary: the 'shellcode' has to be position- independent and can't rely on any normal PE features; it has to do all it's library loading on it's own etc.

yes, very possible to create. but if your malware/tool you want to run is large, it will take a fair amount of time to convert.

</comment><comment><USER37> Most Antivirus software makes your system work like it has a virus. Most of them will eventually let intruder in. Waste of money in my opinion.

</comment><comment><USER38> I've said it before but as hard as it might be to believe, I think smalltalk was 30 years ahead of its time when every program was packaged inside its own OS image.

Anti-virus bypasses and even exploits are extremely common. My current line of thinking is that the best way to take control of your computer is to use virtualization to run many separate OS images for different sets of uses.

<comment><USER39> Interesting, but would have obvious drawbacks when it comes to addressing any OS vulnerabilities or improvements - suddenly you have to reinstall ALL your applications!

This reminds me of Nintendo's approach to emulation for their Virtual Console, actually. Rather than having a standalone emulator that you download images for, they package the emulator with the game. This way they never have to worry about inadvertantly breaking anything with a future release, but on the other hand any subsequent emulator improvements do not retroactively apply.

<comment><USER38> I think you could save and fork OS images. At the same time I would think vulnerabilities wouldn't matter as much, but you are right that it adds complexity to managing a system.

</comment></comment></comment><comment><USER40> Try running the binary instead of just scanning it for signatures. It will be detected by the heuristics engines of most AVs. I'm not saying that AVs are good but articles like this that only focus on defeating signatures while leaving away heuristics (emulation, behavior detection, and even white listing to some extent) don't do the subject justice.

</comment><comment><USER41> How many times do you plan on posting this?

<comment><USER42> This has been postested 3 times. From the FAQ:

> _Are reposts ok?_

> _If a story has had significant attention in the last year or so, we kill > reposts as duplicates. If not, a small number of reposts is ok._

> _Please don 't delete and repost the same story, though. Accounts that do > that eventually lose submission privileges._

Looking at the public available data of the 3 submitters, they look like real independent user (not sockpuppets or something).

</comment></comment><comment><USER39> I suppose the value in AV is that you are like one of the proverbial guys being chased by the lion - you don't need to outrun the lion, only the other guy. In this case, you just hope any malware hits enough other poor suckers first that your AV of choice is updated in time to save you.

</comment><comment><USER43> Yes, this shows that antivirus is trivial to bypass. However, antivirus is not the last word in endpoint protection. While this method can be used to get otherwise ordinary payloads past antivirus, behavior-based detection and application whitelisting can be used to prevent many of these attacks.

<comment><USER17> Proper security hygiene can prevent most attacks, not better antivirus software. Proper security hygiene means:

* applying security updates

* enforcing least privilege

* reducing attack surface (e.g. Does your desktop really need open ports?)

* using decent passwords and two factor authentication when possible

* not reusing passwords in case a place where you used a password is compromised

* not executing code from untrusted sources

* checking whether code from a trusted source is vulnerable to a MITM attack before executing it

* saying no to prompts for elevated privileges unless you can prove to yourself that there is a good reason for them and finding out what caused a prompt for elevated privileges when you see no legitimate reason for it

* wiping a system should you think it might have been compromised and maybe also discarding the hardware just in case firmware was altered, which is what the US government tells US CEOs to do with things that they bring to China

* not providing confidential information (e.g. your password) just because someone claiming to be a trusted party such as IT called asking for it

That last one is how the NSA red team hacked the Pentagon's Joint Staff intelligence directorate when doing penetration testing as part of a "war game" in 1997:

[http://www.slate.com/articles/technology/future_tense/2016/0...](http://www.slate.com/articles/technology/future_tense/2016/03/inside_the_nsa_s_shockingly_successful_simulated_hack_of_the_u_s_military.html)

That said, there are likely more when thinking about confidentiality (the other half of security), but these are the ones that occur to me when I think about ensuring system integrity.

Anyway, antivirus does not save you if you fail to do any of those things. Anything that could get by all of that would be a zero-day attack where antivirus software is likely to be similarly useless. Not all zero-day attacks can get past all of that (minimal attack surface is awesome). If you are the principle target (like the Pentagon was for the NSA red team), antivirus software has no chance of saving you against a zero-day attack.

</comment></comment><comment><USER44> It appears that not much have changed in the last ~20 years, when looking at mainstream antivirus software... I used to be able to move around a few assembly instructions inside a virus to avoid detection while still maintaining 100% of the virus' features

</comment><comment><USER45> Read [https://www.virustotal.com/en/faq/#statistics](https://www.virustotal.com/en/faq/#statistics)</comment>


Alacritty gets scrollback, publishes benchmarks - <AUTHOR> https://jwilm.io/blog/alacritty-lands-scrollback/ <comment><USER2> These benchmarks miss the point in my opinion. When I'm choosing a terminal emulator, beyond features, the one "performance" metric that I'm interested in is latency; nothing else really affects my workflow (positively or negatively). In the benchmarks I've seen of alacritty wrt latency[1][2], it doesn't come out so well.

[1]: [https://danluu.com/term-latency/](https://danluu.com/term-latency/)

[2]: [https://lwn.net/Articles/751763/](https://lwn.net/Articles/751763/)

<comment><AUTHOR> "miss the point" feels a bit strong. Rather, I get the impression Alacritty's values don't match your own values in a terminal emulator, and that's totally OK. Historically, input latency hasn't been considered a big pain point by most users.

That said, we do have a plan[1] to address this issue and be both high throughput _and_ low-latency.

[1]: [https://github.com/jwilm/alacritty/issues/673](https://github.com/jwilm/alacritty/issues/673)

<comment><USER2> Fair enough. :) I guess my stronger wording is because I don't understand workflows where being able to dump vast quantities of text to the terminal quickly is important. In general, a terminal emulator is for use by a human, and humans can't really process info at the throughput rate of other terminal emulators, much less the faster alacritty.

All that said, I'm glad to hear there's a plan on the latency front.

<comment><USER3> It's not ideal, but one flow I end up using at some points is tmux-as-grep. Basically, something either gets dumped to terminal, and I use tmux's search. So then, for a combo of reasons (some good, some bad) I cat files to terminal on occasion, and I use tmux's search to find something in it.

The idea isn't that the I'm processing at the throughput rate of the emulator - it's more that a low throughput rate delays when I can start actually looking for something useful.

<comment><USER4> I've never understood this mentality. If you can dump something to the terminal and use tmux search, you could just as easily use `less` which is pretty much purpose built for this.

</comment></comment></comment></comment></comment><comment><USER5> Regarding tabs, macOS has native tab support more or less built-in to the windowing system—you can see this in action if you create a document-based app in Xcode—but your app has to be structured in such as a way as to opt into it. Have you considered opting into this or would you consider it contrary to the project goals?

<comment><AUTHOR> The hardest part about supporting things like this on macOS is that they often require a lot of additional code or a certain design whereas on Linux, a lot of these features are provided by the window manager.

I don't consider it contrary to the project's goals if it's something that can be done unobtrusively. Given your description, it sounds like this may be something we could support easily. I filed #1544 to track this. Thanks for the suggestion!

#1544: [https://github.com/jwilm/alacritty/issues/1544](https://github.com/jwilm/alacritty/issues/1544)

<comment><USER6> This is actually enabled by default unless you disable it. No extra code needed, unless the macOS implementation conflicts with yours.

<comment><AUTHOR> Thanks for this clarification. I don't see any reason we couldn't support this given that info.

We haven't intentionally opted out of this that I know of, we just didn't start Alacritty from an XCode project on macOS.

<comment><USER6> I've noticed you don't have anything in the menu bar. The option normally appears under Window > Merge All Windows, which is inserted by default by the Xcode template.

<comment><AUTHOR> Thanks for this additional feedback. It sounds like we should create an XCode project from scratch to get many of the defaults and figure out how to bridge this with our current implementation.

</comment></comment></comment></comment></comment></comment><comment><USER7> Has the input latency improved any? I liked alacritty, but I switched to mlterm after a while because the input latency was really bugging me.

<comment><AUTHOR> Thanks for this feedback! We haven't heard a ton of complaints about the input latency, and comments like this help us to prioritize issues. This has been mentioned in two discussions today, so perhaps it's time to address this.

If you want notifications on any developments here, please subscribe to this GitHub issue: [https://github.com/jwilm/alacritty/issues/673](https://github.com/jwilm/alacritty/issues/673)

<comment><USER7> Thanks, I'll definitely be looking forward to any improvements on that!

</comment></comment></comment><comment><USER8> I'm really happy to see all the hard work come together on this. I think the project has a great new feature.

</comment><comment><AUTHOR> Author here; I'm happy to answer any questions!

<comment><USER9> How did you choose the terminals to benchmark against? What about xterm or gnome-terminal or powershell or whatever?

<comment><AUTHOR> We tried to strike a balance between "commonly accepted as fast" terminal emulators and coverage of "commonly used" terminal emulators. Termite gets us libvte-based terminals (like gnome-terminal), urxvt is generally considered as one of the fastest, and Kitty is another well-regarded GPU-accelerated terminal emulator. On macOS, there's not nearly as many choices.

Ultimately, it would be great if we could benchmark against every terminal emulator, but that can become a very time-consuming task. If there's another emulator you feel should be included, we can consider it for future updates/benchmarks.

<comment><USER10> You could try Terminology (I'm the main dev). It can be GPU-accelerated or not.

<comment><AUTHOR> I didn't know that Terminology has GPU acceleration. I would be happy to include this in the future!

</comment></comment></comment></comment></comment><comment><USER11> This sounds great, especially the fact that v0.2 has much improved throughput over v0.1 in most cases. I must however echo some other posters' sentiments: latency is even more important than throughput, and alacritty's input latency wasn't great last time I tested it (quite a while ago). Maybe it's been improved too, though no mention of it in the post.

</comment><comment><USER12> What possible use cases where such high scrollback speeds could be useful?

<comment><USER13> Performance is never a bad thing, even if I don't always need my terminal emulator to be able to handle large amounts of text, there is nothing wrong with having a light weight high performance (eventually cross platform, hopefully) modern terminal emulator. I read a while back that implementing the same in iterm2 (a popular macOS only terminal emulator) is challenging due to past design decisions [0]. Certainly in a time where hyper terminal[1] (based on electron) exists, I see no downside to having the option of Alacritty.

[0] [https://news.ycombinator.com/item?id=17635869](https://news.ycombinator.com/item?id=17635869) [1] [https://hyper.is/](https://hyper.is/)

<comment><USER14> > Performance is never a bad thing

Depends on your priorities and what you are willing to sacrifice for it. High bandwidth terminal emulators will cost CPU and battery life. I'd opt for longer battery life.

The choice is nice to have though.

<comment><USER13> I agree! currently I run windows as my main OS precisely due to inferior battery life on Linux. I only use Alacritty in minimal Linux VMs when they are not running headless(as pseudo containers, I will be migrating fully to Linux + Docker soon). I normally only work with the VMs while plugged-in and because Alacritty is not available for windows yet, have not used it much on battery therefore I am not in a position to say anything about the battery impact. One thing I have noticed is that it is noticeably slower to start on the same VM as compared to XTerm but the VIM experience was better in Alacritty.</comment></comment></comment></comment>


Show HN: Emulos records how people use your app - <AUTHOR> http://www.emulos.com

<comment><USER2> Bonus points from me for not using aggregated data from users for your own purposes apart from with regards to the use of your own services. Most of the the other mobile analytics services seem to require that they get access to general aggregated data.

I wouldn't record the video though except in a special mode where someone has temporarily opted in to show me how something doesn't work though. It is a bit intrusive if the user inputs any data (even what they are reading and for how long seems a bit intrusive to me but lots of web analytics do this).

</comment><comment><USER3> Awesome concept. Couple of issues/questions:

-The gray text is hard to read on my screen. It does make the black text stand out more, but it's at the expense of readability for the rest of the content.

-This looks like it's solely for mobile, but you don't mention that anywhere on the page (I suppose "app" sort of implies that).

-Is this opt-in for users? Is it even opt-out? That's a big question I would want the answer to, and the "See how Emulos protects users' privacy" link appears to be broken.

</comment><comment><USER4> Continuous video streaming in the background on a mobile device? Say goodbye to bandwidth and battery life!

Regardless, myself and many other app developers will still use this due to the tragedy of the commons.

</comment><comment><USER5> Thought it was similar to Mixpanel, but then I watched the demos. You went full NSA! You never go full NSA. You're basically streaming a full screen recording of user's activity on their phones.

</comment><comment><USER6> Seems very easy to just drop into your app. Although I'm worried about data usage. Doesn't streaming video use a lot of bandwidth?

<comment><AUTHOR> Customers had those concerns, so the video is super low bitrate, ~800 kb per minute.

</comment></comment><comment><USER7> Interesting concept. Is this iOS or Android?

</comment><comment><USER8> This is sweet!</comment>


Headless Chrome support in Cloud Functions and App Engine - <AUTHOR> https://cloud.google.com/blog/products/gcp/introducing-headless-chrome-support-in-cloud-functions-and-app-engine <comment><USER4> I wrote a collection of Dockerfiles for images running Python 2.7 or Python 3.6 + Selenium with either Chrome or Firefox and using Xvfb for the X display (necessary for running Selenium headlessly).

[https://github.com/seanpianka/docker-python-xvfb-selenium- ch...](https://github.com/seanpianka/docker-python-xvfb-selenium-chrome- firefox)

Using this, in conjunction with AWS Step Functions, Lambda, and ECS, it became merely cents a month to run a headless scraper task in the cloud.

<comment><USER5> Can you elaborate a bit? Sounds interesting. I had never heard of AWS Step Functions before.

What does your workflow look like?

<comment><USER6> Not OP, but to elaborate on AWS Step Functionss:

In short - this gives you the ability to pass the output of one lambda function to the input of another lambda function.

An example of one that I've written to regularly create a new copy of our Production RDS database in Ireland as a Staging RDS database in Oregon.

1\. Cloudwatch Event starts Step Function on the 15th

2\. Copy last Production snapshot from Ireland to Oregon

3\. Restore this snapshot as a new RDS instance (It will fail until the snapshot is available and retry with exponential backoff - this is a step function feature)

4\. In parallel:

      - Add tags to the instance (Once it's available)

      - Delete the snapshot copy (When finished restoring)

      - Modify the new instance with security groups and subnets

         - In parallel:

            - Run a SQL query to anonymize all of PII columns for GDPR compliance as data has now left the EU.

            - Call out to the Cloudflare API to update our DNS entry with the new RDS endpoint.

               - Delete the old Staging database instance

<comment><USER7> Do you run into problems with Lambda's 5-minute maximum execution time for those kinds of operations? I'd like to do something similar to this for both RDS and DynamoDB, but the execution time will often surpass 5 minutes, meaning I'd have to run a Step Functions worker on EC2 or ECS. That opens up a whole bunch of complexity with managing the worker code and its deployment, which I'd rather avoid if possible.

<comment><USER6> With the current implementation; no problems hitting the limit. As mentioned in my below comment, our query for anonymization would be the heaviest - <USER2> it's designed to be quick as we don't care about unique values for most data.

If we did though - Fargate is a great solution for it, but you wouldn't be able to feed data back into the next step without some additional complexity - Maybe have the next step pull an SQS queue, or an S3 file, or look for a database entry, etc. as it's next bit of data that it needs - and just fail until it finds it, and once the Fargate (Or whatever) has done it's job and placed it in your method of choice, then it could continue.

</comment></comment></comment></comment></comment><comment><USER8> I tried to use Headless Chrome on Cloud Functions for a project I'm working on, but even on the fastest instances loading pages was sometimes really slow (pages timing out after waiting for 60s).

It seems sometimes JS execution was taking a long time, so I guess that was preventing requests from being made. In a single CPU cloud function you have network requests, JavaScript execution, rendering, and the Node process controlling the browser all competing for resources.

That being said, it was super simple to get started!

<comment><USER9> Do simple hello-world HTML pages render ok? I found that rendering was slow- ish but totally acceptable for reasonably heavy HTML pages, so long as we weren't flooding page re-renders (eg by using React without any optimisation of render calls)

<comment><USER8> Yes those were fine.

</comment></comment></comment><comment><USER10> Among other things with puppeteer we do screenshot generation using GKE on Google Cloud @ [https://screenshots.cloud/](https://screenshots.cloud/) scaling up and down running instances depending on demand. We keep browser instances running constantly as the startup time is significant. I will be interested to see what the startup time is for puppeteer on this, will definitely be giving it a try.

<comment><USER11> Nice reference immediately above "used by us" :)

One completely unrelated thing. On Chrome 68.0.3440.84, I noticed the large icons (particularly the Kubernetes one) looked "weird", with jagged edges that didn't make any sense. Some poking with the devtools revealed that 'backface- visibility: hidden' seems to be disabling antialiasing.

Suggest opening the following in new tabs so you can flip back and forth between them:

\- As is right now: [https://i.imgur.com/nYzsukI.jpg](https://i.imgur.com/nYzsukI.jpg)

\- Nicer-looking: [https://i.imgur.com/GNlvx7Z.jpg](https://i.imgur.com/GNlvx7Z.jpg)

I noticed disabling this has an effect on the animation at the top (the edges of the moving webpage slides don't have constantly-moving jaggies).

There may well be a valid reason you have this enabled, perhaps for added performance. Or perhaps React added it in for you? :P

<comment><USER10> Thanks for spending time to let me know because I don't think I would have noticed it otherwise! I can't see it on retina but I can on my non retina display. We'll have to make the CSS rule more specific. As to why it's there I believe Firefox 57 or around that version had an issue with the sliding animation on the top of the page causing images to tear or not render at all when they scrolled in. This bug must have been solved recently because disabling backface-visibility on the image doesn't cause the same tearing.

<comment><USER11> I was very curious what was causing the non-antialiasing, it was fun.

And you can repro :) cool. Makes a lot of sense you can't see it on retina.

Interesting FF bug you hit. CSS3 GPU-accelerated animations are incredibly complex... heh, adding the rule fixed Firefox ~57, but now Chrome 68 is glitching out _because_ the rule is there. I wonder if Google realizes yet. _Ponders complexity of creating minimal testcase, versus waiting for someone else to notice :P_

</comment></comment></comment></comment><comment><USER12> Nice to see this concept get into the big cloud platforms. We built something similar couple of years ago, primarily to get a sandbox for some compute jobs we were running on Heroku:

[https://github.com/flowhub/jsjob](https://github.com/flowhub/jsjob)

</comment><comment><USER13> Does this mean that HC is preinstalled in the runtime?

Becauee as far as I know you can already run HC with other FaaS solutions, but having this out of the box would be really nice.

<comment><USER14> Running Headless Chrome / Chromium is a bit of a hassle on AWS Lambda and other FAAS providers. Chrome requires some specific bindings/binaries to work. I think the Chrome guys and girls convinced their Cloud coworkers to provides these in the underlying Linux machines that run the Cloud Functions.

<comment><USER15> exactly this. The base operating system comes with the system libraries necessary to support headless chrome out of the box.

(disclaimer: I work for Google Cloud)

<comment><USER16> What's to really stop other providers from doing the same thing though? :)

<comment><USER17> Nothing. It's about developer convenience, not technical possibility.

</comment></comment></comment></comment></comment><comment><USER18> If Google cloud ain't your jam then checkout browserless ([https://browserless.io/](https://browserless.io/)). It can be considerably cheaper under certain situations, and we've been up and running for almost a year. Happy to answer questions if anyone has any.

EDIT: We've got stuff on GH: [https://github.com/joelgriffith/browserless](https://github.com/joelgriffith/browserless), and startup is under 100ms most of the time. Fonts and other things "just work" as well, plus there's a slew of REST APIs for common stuff as well. Selenium webdriver support landing soon!

<comment><USER19> ...and if browserless ain't your jam, checkout [https://www.prerender.cloud/](https://www.prerender.cloud/)

cheap, because we optimized for just 3 things:

pre-rendering, screenshots, or PDFs for $0.000365 per API request

       curl https://service.prerender.cloud/screenshot/https://google.com/ > out.jpg

       curl https://service.prerender.cloud/pdf/https://google.com/ > out.pdf

       curl https://service.prerender.cloud/https://google.com/ > out.html

<comment><USER20> Pricing is in tiers, not per request as you seem to imply?

<comment><USER19> Correct, only the final tier is variable rate; apologies for the confusion.

Under 20,000 monthly requests = $9 flat rate ($0.00045)

Under 100,000 monthly requests = $40 flat rate ($0.0004)

>= 100,000 monthly requests = variable rate @ $0.00036/req

<comment><USER20> Yeah, that's completely different than what you were trying to imply. Still a cool service, though.

</comment></comment></comment></comment></comment><comment><USER21> I wish Google would support Ruby!

If you do too checkout the petition over at [https://www.serverless- ruby.org](https://www.serverless-ruby.org)

<comment><USER18> Something somewhat related: I think more Cloud providers need to start doing Docker-functions. It's always going to be a waiting game for runtimes and upgrades. Docker is portable and can run just about anything, so why not support that?

[https://zeit.co/blog/serverless-docker](https://zeit.co/blog/serverless- docker) is an example of what I mean

<comment><USER22> See the "Serverless Containers" section on [https://www.google.com/amp/s/gweb- cloudblog-publish.appspot....](https://www.google.com/amp/s/gweb-cloudblog- publish.appspot.com/products/gcp/cloud-functions-serverless-platform-is- generally-available/amp/)

It's still in alpha, so Google is not advertising it wide. It was announced about a week before Zeit's Serverless docker at GCP Next.

Disclaimer: I work at Google and I've had some involvement in the tech behind this.

<comment><USER11> That's an interesting domain: '[http://gweb-cloudblog- publish.appspot.com/'](http://gweb-cloudblog-publish.appspot.com/')

(it just redirects to cloud.google.com)

</comment></comment></comment></comment><comment><USER23> I think it would be pretty cool to use Cloud Functions as a Selenium Grid, sort of like Zalenium ([https://github.com/zalando/zalenium](https://github.com/zalando/zalenium)) does with Kubernetes. If you could parallelize end-to-end tests enough, you could get massive burstable capacity to run parallel tests.

</comment><comment><USER24> Not perfect but HtmlUnit anyone? I used it for scraping in the past with mixed experiences.

<comment><USER25> HtmlUnit API is really cool. It's fine for most use cases. But obviously the Javascript support is not perfect. [Shameless plug] I wrote a book about web scraping where I talk about HtmlUnit & headless chrome with Java: [https://www.javawebscrapinghandbook.com](https://www.javawebscrapinghandbook.com)

</comment></comment><comment><USER14> I've been screwing around with running Headless Chrome & Puppeteer on Lambda/Serverless/FAAS solutions. It's all a bit of a mixed bag. You CAN run Headless Chrome on AWS Lambda, but the cost involved is pretty crazy as you need ~1500Mb in RAM to comfortably run any code with Chrome.

Google Cloud of course has "inside knowledge" and I would love to switch to them for my SaaS [https://checklyhq.com](https://checklyhq.com), were it not that Google Cloud Functions is just offered in four (!) regions...

<comment><USER18> Great to see you here Tim, love that chrome extension! We should chat a bit more sometime. I'd love to back checkly's infrastructure.

</comment></comment><comment><USER26> that's it...im moving to GCP

sorry but Rekognition rekt it for any type of computer vision on AWS.

Great infrastructure...after all I do have an AWS Solution Architect Associate certification....which means jack shit

Great move by GCP, I'm also very pleased with Firebase and it's integration with cloud functions....

BUT my biggest reservation still in 2018 when it comes to serverless is the cold start up time...

I built a token based API on AWS Lambda and registering, signing up took forever when the app was not at peak. that was 2014 tho.

<comment><USER27> We use GCP heavily. It's really great.

Latency between Google Cloud Functions and other GCP products improved significantly in the past week, as did start up and execution time.

However we only use GCFs for background tasks and not for any web API/micro services, etc. Still too much latency for that use case.

</comment></comment><comment><USER28> Ok, I was just looking into this 1 week ago and was gonna spin up a VM to do headless. Now I get to keep my firebase project in cloud functions only. Much cleaner architecture.

</comment><comment><USER29> My company provides a similar service, with both Chrome and FireFox headless support for Automated Testing/Screenshots: [https://testingbot.com/support/getting- started/headless.html](https://testingbot.com/support/getting- started/headless.html)

We run each test in a new VM, running on our own private cloud (dedicated servers).

Note: we use the Selenium protocol for this, not yet Puppeteer.

</comment><comment><USER30> I'm currently using a headless Chrome for my latest project www.blockedby.com (still in alpha stage, looking for feedback)

I've been looking at a non-local solution. I'm using Python and this article hints that Puppeteer is not the only way to invoke this. But I don't see any documentation on the DevTools protocol.

Anyone knows if it's supported ? Or any providers that do ?

<comment><USER31> You should just need to get a handle on the host and port to use Devtools protocol to talk to the launcher headless chrome instance. If you use node.js to start it, you can get the port like this: [https://github.com/GoogleChrome/puppeteer/blob/master/docs/a...](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#browserwsendpoint)

Then you can use PyChromeDevtools to connect to that host/port:

[https://github.com/marty90/PyChromeDevTools/blob/master/READ...](https://github.com/marty90/PyChromeDevTools/blob/master/README.md)

</comment></comment><comment><USER32> I've tested HC on GCF and GAE standard with the IO launch. Sadly they're 3-10x slower than App Engine Flex (same vm size on App Engine Flex vs Standard). Even a screenshot of google.com takes 6+ seconds on GCF / GAE Standard vs 2 seconds for Flex. I hope they fix this as spinning to zero is important for me but the latency is too high right now.

</comment><comment><USER33> I don't know what this means for browserless.io but I hope he still retains a strong niche and has a desirable product.

</comment><comment><USER34> Tried HC just yesterday on cloud functions. For some reason, it runs extremely slow. Did some comparisons to AWS lambda with similar memory / cpu sizes and basic "load page and screenshot" jobs would take 2x - 3x more time on google cloud functions.

I'll dig deeper soon but this is a bad start.

<comment><USER35> Is it still within say 5 seconds?

Do you get the impression there's some config tuning to do or something you can't control?

</comment></comment><comment><USER36> Can someone please explain what the advantage of running a snapshot service via GCF would be vs. AppEngine Standard (w/ node)?

<comment><USER37> Lower costs. With AppEngine you pay for a whole instance, regardless of how much utilisation it gets. With cloud functions you only pay for the time the function is being executed. If your code isn't being executed often, GCF are much cheaper.

<comment><USER36> Oh I see... AppEngine is also "pay for what you use" but it seems it's rounded to 15 mins rather than CloudFunctions which is 100ms. Thanks!

</comment></comment></comment><comment><USER38> I wonder how feature/pricing compete with Browserless?

<comment><USER18> Our small instance ($30/month) is roughly similar to their $44.38 instance on App Engine. If you were to run a full 10 concurrent sessions constantly, which a small browserless instance can max-out at, this would cost roughly $427 a month in Functions. So depends on your use-case

<comment><USER39> For reference, how did you get $427? Thanks in advance

<comment><USER18> Running 10 concurrent Google Functions at 1GB/1CPU. Assumes 30 days in a month

<comment><USER39> I am getting a different price from GCP calculator. Almost $2000. In your example, 10functions por Second, so each function is taking 1000ms to finish. 2592000s in a month * 10invocation = 25920000 invocations month, running with a 1GB function and 500kb of network bandwidth (out). [https://cloud.google.com/products/calculator/#id=555e1af7-c8...](https://cloud.google.com/products/calculator/#id=555e1af7-c858-4573-9d8a-a7d87475c236)

<comment><USER18> Ah, I wasn't calculating their other fees like invocations and networking costs. Crazy it's almost an order-of-magnitude

</comment></comment></comment></comment></comment></comment><comment><USER40> How Google is behaving recently... I can feel it becoming Oracle. I want to stay 10 miles far from it. Learned my lesson with Google maps.

</comment><comment><USER41> I'm not buying it, Google Cloud just moved to node 8 earlier this year as the post says, but now it's node 10. It's just not good tech, it's unnecessary lock-in. Docker on anything is better, this is similar: [https://zeit.co/blog/serverless-docker](https://zeit.co/blog/serverless- docker)

<comment><USER42> It's wonderful tech that has saved us thousands a month. The lock in is very little as it's running a JS function. We've written our functions so that the export to the GC function merely passes in arguments to another function that is required. We could move to AWS, Zeit, or anywhere else with little friction.

<comment><USER41> Please do, then, it's better for everyone in the node ecosystem if people aren't running node 4.x in 2018.

<comment><USER43> Would you be happier if people switched to some other language for their cloud functions? Node is not that important.

</comment></comment></comment></comment><comment><USER44> It's nice to see serverless platforms adding support for headless Chrome. But there's still one problem with AWS Lambda / Cloud Functions / Zeit Now - <USER3> run time is limited to a few minutes. If you want to run any longer job, e.g. a web crawler, you need to either spin up the instances yourself or use platform like Apify, which allows running arbitrary-long jobs, provides pre- built Docker images for headless Chrome or XVFB, and provides SDK to simplify state persistence, access to proxies etc.

For example, a simple actor to convert HTML to PDF looks like this:

[https://www.apify.com/jancurn/url-to-pdf](https://www.apify.com/jancurn/url- to-pdf)

More info:

[https://www.apify.com/docs/actor](https://www.apify.com/docs/actor)

[https://www.apify.com/docs/sdk/apify-runtime- js/latest](https://www.apify.com/docs/sdk/apify-runtime-js/latest)

[https://www.apify.com/library?type=acts](https://www.apify.com/library?type=acts)

Disclaimer: I'm a co-founder of Apify</comment>