Python JITs are coming - <AUTHOR> https://lwn.net/Articles/691070/ <comment><USER2> > If it needs to be fast for CPython, it has to be written in C, but if it > needs to be fast for a JIT, you cannot use C. He showed a simple mysum() > function that totaled up the elements in an iterable. If it is passed a > Python object like list(range(N)), the JIT knows what it is and can do lots > of optimizations. But if it is passed a NumPy array, which is "opaque C > stuff", the JIT doesn't understand it, so it will have trouble even > achieving the performance of a non-NumPy version on a JIT-less Python.

There's a third way here - run the C code using the same JIT as the Python code, instead of compiling it natively.

That might sound mind-bending, but C is just a language like any other. It's actually fairly simple and consistent to implement compared to something like Python. You can interpret and JIT compile it if you want to - there's no major magic to that.

Then you can optimise the C code and the Python code at the same time, inline the two, do the same optimisations as you do on Python code, etc. We're using this technique to run Ruby C extensions in JRuby and the results so far are great - running real C extensions faster than native code because we can optimise both the Ruby and the C at the same time.

[http://chrisseaton.com/rubytruffle/cext/](http://chrisseaton.com/rubytruffle/cext/)

<comment><USER3> Very interesting point of view.

I see a fourth option:

\- compile the scripting language to C (like Crystal for Ruby)

<comment><USER4> Sure, but then you lose any advantage using a scripting language gives you.

<comment><USER5> That and you lose the advantage of what JIT compiling can get you.

</comment></comment></comment></comment><comment><USER6> The Truffle implementation of Ruby can be plugged into a Truffle implementation of C, in order to maintain compatibility with C extensions

[http://chrisseaton.com/rubytruffle/cext/](http://chrisseaton.com/rubytruffle/cext/)

Has anyone in the Python community thought of doing something similar, e.g. using PyPy to build a C interpreter, maybe re-using front-end components (pre- processor, parser, type-checker, etc.) from an existing C compiler? In fact, it might be useful to build an interpreter for something like LLVM IR, or even x86 machine code, in order to gain access to a bunch of existing languages.

Once access is gained at that low (ABI?) level, abstractions and interfaces can be built to hide the horribleness. The performance would initially be terrible, but some targetted, profiling-guided optimisation might get it down to reasonable levels; in a similar way to a JS interpreter adding specific optimisations to make asm.js code run fast.

<comment><USER7> Something like Sulong¹ in RPython would be really cool and probably entirely feasible—I too am curious if that's on the PyPy team's radar.

¹ [https://github.com/graalvm/sulong](https://github.com/graalvm/sulong)

<comment><USER8> Sulong is a really good idea -- I wish I'd thought of it first! We've had a student do a small project looking at something equivalent for RPython. There are, as expected, no show-stoppers yet, but I have no idea how far we'll be able to go with the limited resources we have to throw at the problem.

</comment></comment></comment><comment><USER9> I must say I'm always a little annoyed by the split in Python community around the scientific stack vs the rest. There are two ecosystems, two packaging tools etc (which is fine), but the insistence that there are no other python users that are worth considering (both sides are guilty) is really frustrating.

<comment><USER10> Former Cython developer here.

To get some perspective to this, consider the alternatives for a scientific programmer: MATLAB, R, FORTRAN, Mathematica, or if you're hip, Julia -- all specifically made for scientific programming with 0% general purpose/web/etc. development going on.

So I would say that the scientific Python community has been doing extremely well in terms of even using a language that isn't designed ground up for scientific computing.

I could write a lot about why that is (and how some of the CS and IT crowd doesn't "get" scientific computing..) -- I'll refrain, I just wanted to say that were you see something and get frustrated, I see the same picture and think it's actually an incredible success, to bring so many scientists onto at least the same ball park as other programmers, even if they are still playing their own game.

<comment><USER11> > I could write a lot about why that is (and how some of the CS and IT crowd > doesn't "get" scientific computing..)

Please do! I'm a PhD student in CS, and I don't think I "get" scientific computing (I'm in compilers myself).

<comment><USER12> I am a PhD in CS, specifically in Programming Languages and Parallel Programming and I believe I do get scientific computing.

Most people doing it did not have a formal CS education. They are biology, physics, mathematics or chemistry majors that have had one or two courses on programming, from other scientific programmers.

There are two main families, one that comes from the Fortran background, which still writes programs like they did in the 80s, with almost no new tooling. Programs are written for some time, and then they are scheduled for clusters that spend months calculating whatever it is.

The other family of scientific programmers, which I believe is the majority, uses a tool like Matlab, or more recently R, to dynamically inspect and modify data (RStudio is a Matlab/Mathematica-like friendly environment for this task) and use libraries written by more proficient programmers to perform some kind of analysis (either machine learning, DNA segmentation, plotting or just basic statistics).

Most of these programmers know 1 or 2 languages (maybe plus python and bash for basic scripting). They write programs that are relatively small and the chances of someone else using that code is low. Thus, the deadline pressure is high and code maintainability is not a priority.

For a non-CS programmer, learning a new programming language is almost impossible, because they are used to that way of doing things, and those libraries. They take much more time to adjust to new languages because they do not see the language logically, like anyone who had a basic compiler course.

Given this context, web apps, rest APIs and all the other trending tech in IT are not commonly used in scientific programming, because they typically do not need it (when they do, they learn it). Datasets are retrieved and stored in CSV and processed in one of those environments (or even in julia or python- pandas).

<comment><USER13> You're painting an awfully dark picture of scientists' skills. Having been on both sides, I believe the deciding factor is simply the availability of libraries.

If you're doing web development you have an insane amount of languages to chose from because after String, Array, and File are implemented, HTTP is next. Having done a bit of web development, I'd also say a typical project only uses a subset of libraries that is surprisingly small.

Scientific computing is quite different: a paper in structural biology (my former stomping grounds) can easily require a few dozen algorithms that each once filled a 10-page paper. These could easily be packaged as libraries, but it's a niche so it rarely happens. Newer language quite often don't even have a robust numeric library. Leave the beaten tracks and your workload just increased by a magnitude.

That's also why science, unlike "general purpose" programming, often uses a workflow that connects five or more languages or so: a java GUI, python for network/string/fileIO, maybe R for larger computations, all held together by a (typically too long) shell script.

But these workflows are getting better. There's a build tool that formalizes the pipeline somewhat (I forgot the name) and APIs are surprisingly common. The reason why csv will never die is that the data fetched from APIs is usually more static than it is in a typical web app (-> local cache needed) and that scientists often work with data that just isn't a good fit for a database. Postgres just doesn't offer anything that enriches a 15MB gene sequence.

<comment><USER14> I worked in the academia for a few years about a decade ago and nowadays interact with biology research in the industry for the last couple of years.

The way he painted the scientists skills matches my experience thus far.

<comment><USER10> Yes, scientists programming skills (as averaged over population) suck. Factor 1: Programming not credited in itself or reviewed in publishing process. Factor 2: Often little education in or focus on programming, relative to wall clock time spent doing it.

But I don't think that is only fixed by more education and making scientists behave more like programmers. I think that to change things one also needs far better alternatives than the options available today, so that people are really encouraged to switch. Somehow, these must be written by people who know their CS and can write compilers, yet engage with the _why_ scientific computing is a mess on the tool side too, not dismiss it as laziness.

I started out as a programmer, I have contributed to Cython, past two years have been pure web development in a startup. So I know very well why MATLAB sucks. Yet, the best tool I have found myself for doing numerical computing is a cobbled mess of Fortran, pure C, C/assembly code generated by Python/Jinja templates, Python/NumPy/Theano...

The scientific Python community and Julia community has been making great progress, but oh how far there is left to go.

<comment><USER14> I agree, this is also one of the things that drives me against C and more into saner programming languages.

Because the majority of programmers in areas where software isn't the core product being sold, don't spend one second thinking about code quality.

As such tooling that on one side is more forgiving while allowing for fast prototyping, but at the same time enforces some kind of guidelines is probably the way to improve the current workflows.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> Total tangent: Anyone doing scientific computing work on the JVM may be interested in:

ND4J: N-dimensional arrays for the JVM [http://nd4j.org/](http://nd4j.org/)

Libnd4j: The C++ engine powering the above [https://github.com/deeplearning4j/libnd4j](https://github.com/deeplearning4j/libnd4j)

JavaCPP: The bridge between Java and C++ (Cython for Java) [https://github.com/bytedeco/javacpp](https://github.com/bytedeco/javacpp)

Fwiw, all that works on Spark with multi-GPUs.

[http://deeplearning4j.org/spark](http://deeplearning4j.org/spark)

[http://deeplearning4j.org/gpu](http://deeplearning4j.org/gpu)

<comment><USER16> Wow, how did I miss javacpp? Looks featureful and actively maintained to boot. Thank you for these recommendations.

<comment><USER15> Glad you like the looks of it. It was built and is maintained by a Skymind engineer. All of us are on this Gitter channel if you have questions: [https://gitter.im/deeplearning4j/deeplearning4j](https://gitter.im/deeplearning4j/deeplearning4j)

</comment></comment></comment><comment><USER17> An example of how, for certain types of code, more than an order-of-magnitude performance improvement can be obtained with a good JIT compiler. I wrote a decompression routine (for a simple compression format) in several languages - it took the following lengths of time to run (normalized to C++ = 1 second):

* _CPython_ , Jython, Lua, MicroPython, Ruby (1.8): >100 seconds

* Cython (naive), IronPython, Ruby (2.3): 50 - 80 seconds

* _LuaJIT_ , _PyPy_ , Cython (with type hints): ~4 seconds

* C#, Go, _JavaScript (V8)_ : ~1.5 seconds

<comment><USER18> can you try it in julia?

<comment><USER17> I actually wrote a Julia version at the time, but it was difficult to accurately benchmark because the total runtime was significantly affected by a long (~5 seconds) delay at startup.

Recently, however, I came across [https://www.reddit.com/r/Julia/comments/4c09m1/](https://www.reddit.com/r/Julia/comments/4c09m1/), which suggests running with `--precompiled=yes` to reduce startup time. I have now run the Julia version with that command-line option, and the runtime is impressive - comparable to C#/Go/V8.

<comment><USER19> Most Julia users run code more often in an interactive REPL session rather than as a script to do a single thing. The precompiled code loading is now on by default on Windows in Julia 0.5 since the backtrace issue was fixed, but there's still a 0.3-0.5 second delay on startup at the moment.

</comment></comment></comment></comment><comment><USER20> I can't help thinking that we need a new scientific programming language that is properly vectorised and directly targets GPUs, as opposed to all these hacks that unroll loops. Now that a large proportion of data science and scientfic computing is GPU centric, aren't there new languages designed with these parallel architectures _foremost_ in mind? Doesn't functional programming map very well to this?

It almost seems to me that just as we're about to get c-speed in Python via JIT tech, so it'll almost immediately be left behind because the GPU is where it's at.

<comment><USER11> One problem is that functional programming style is often focused on recursion - not just in its functions, but also its data structures. Trees, linked lists and graphs are GPU poison, while arrays map very well to the hardware. However, arrays don't have the pleasant recursive structure that allows you to encode nifty properties (and do pattern matching) the way that is popular in functional programming.

However, array programming is basically functional programming, and _does_ map very well to parallel execution in general. While most array programming is nowadays of the fairly simple form you see in libraries like Numpy, older languages like APL show how far you can go. It's a somewhat alien style of programming to most modern programmers, though.

My rule of thumb: parallel programming is hard. Functional programming makes it trivial to make parallel programs that are correct, but they still might not be fast.

<comment><USER20> Is Rust the right way forward then? Functional enough without obsessing about linked-memory structures / recursion etc?

<comment><USER11> I don't think so. Rust has excellent support for concurrency, but does not have any interesting parallel constructs. Which means that while it can probably be fine for coarse-grained multicore parallelism, it does not provide the fine-grain parallelism you need on something like a GPU.

<comment><USER21> Not built in, but there are libraries, like rayon or crossbeam.

</comment></comment></comment></comment></comment><comment><USER22> Why not a Python to machine code compiler like Cython, but much faster and with all Python 3 capabilities? (e.g. as GNU gcj for Java)

Getting 30x speed up with compiled code using optimized data structures (e.g. pointer compression for better data cache usage, etc.), although not reaching performance of best JIT implementations, it could be good enough.

</comment><comment><USER23> The best way forward seems to be to continue to fund these existing JITs. Maybe once one is ready it can replace cpython.

Also pypy not supporting the latest python 3.5 is kind of a big deal for adoption. I would rather have 3.5 support than numpy/cpython compat but I don't do scientific computing :)

<comment><USER24> You ask for it, you got it!

[https://morepypy.blogspot.com/2016/08/pypy-gets-funding- from...](https://morepypy.blogspot.com/2016/08/pypy-gets-funding-from-mozilla- for.html)

<comment><USER23> That is awesome! Python 3.5 all the things :)

</comment></comment></comment><comment><USER25> Compared this to the Ruby, which is NOT getting much love from the compiler / JIT community.

While one can argue Ruby gets the best and leading edge JIT from Jruby / Graal / Truffle. I dont think it is used much in production. It seems the majority of Python and Ruby has stuck to the default CPython and CRuby runtimes. Both PyPy and JRuby seems to be rather small in usage.

Why may that be?

</comment><comment><USER26> >LLVM or libraries from Microsoft and IBM can be used to ease the building of JITs.

Which libraries are they talking about? I tried searching but the only mentions of JITs I found on Google were IBM's JVM, and Microsoft's CLR.

<comment><USER27> Oracle has GraalVM

[http://www.oracle.com/technetwork/oracle-labs/program- langua...](http://www.oracle.com/technetwork/oracle-labs/program- languages/overview/index-2301583.html)

</comment></comment><comment><USER28> Google tried to speed up CPython with a JIT, and failed. Remember Unladen Swallow?

<comment><USER29> IIRC, Unladen Swallow aimed to remove the GIL, which is a different can of worms to open.

<comment><USER30> Actually Unladen Swallow's aim was simply to be fast. :) They attempted to do that through using LLVM's JIT but ran into a lot of bugs in LLVM and had to spend their time fixing rather than on Python itself. There was no specific work to deal with the GIL.

</comment></comment></comment><comment><USER31> Are any of the JIT strategies able to create something that can be invoked as a raw function pointer from C/C++ without any dependency on the GIL?

<comment><USER32> Not Python, but you might like [http://terralang.org/](http://terralang.org/)

</comment></comment><comment><USER33> > Jupyter (formerly IPython) is "an overgrown REPL"

They say that like it's a _bad_ thing.

</comment><comment><USER34> How about add JIT and remove GIL at the same time?

<comment><USER35> Interesting talk to answer part of that question: [https://www.youtube.com/watch?v=P3AyI_u66Bw](https://www.youtube.com/watch?v=P3AyI_u66Bw)</comment></comment>


Ask HN: Historical Web Traffic data - <AUTHOR>

I am trying to do some analysis of a bunch of company websites (like Shutterstock, RingCentral etc) and am trying to find traffic data for these companies since 2000.  Does anyone know of  any such data sources (free or paid)? <comment><USER2> Have you looked at [http://www.alexa.com/](http://www.alexa.com/)

This gives other websites with similar statistic providers [http://www.alexa.com/topsites/category/Computers/Internet/St...](http://www.alexa.com/topsites/category/Computers/Internet/Statistics_and_Demographics)

Look at pricing here [http://www.alexa.com/plans](http://www.alexa.com/plans)

<comment><AUTHOR> Thanks.. I have. They only provide ranks and not raw traffic numbers.</comment></comment>


Zenefits CEO responds to rumors of company's struggles - <AUTHOR> http://www.businessinsider.com/zenefits-ceo-refutes-the-naysayers-2015-11 <comment><USER2> That image in the article makes it look like a terrible place to work: [http://static6.businessinsider.com/image/56495c0e112314ef038...](http://static6.businessinsider.com/image/56495c0e112314ef038b5496-1228-921/zenefits%20offices.jpg)

<comment><USER3> That's literally the scene of almost any SV startup. They don't "do" remote. Nor work life balance. It's all about asses in seats. Stay very clear of a work environment like that.

<comment><USER4> We are a remote team. I talked with a VERY PROMINENT VC not long ago and he had this to say about it:

 _It's probably also worth mentioning that we have a bit of a bias against distributed teams, based on previous investment experience._

<comment><USER5> What was their previous investment experience with remote teams?

<comment><USER6> [http://www.bothsidesofthetable.com/2010/07/05/the-power- of-i...](http://www.bothsidesofthetable.com/2010/07/05/the-power-of-in-person- why-distributed-teams-are-less-effective/)

</comment></comment></comment></comment></comment><comment><USER7> 1600 employees and only $80 million in revenue? That's only $50k per employee, which seems very low for a company hiring that aggressively (+25%/quarter).

<comment><USER8> I wonder why they really need 1600 employees...to be quite honest, and I have a feeling half of them are contractors / consultants.

<comment><USER9> Commissioned salespeople for sure.

</comment></comment></comment><comment><USER10> He'd rather respond to us disgruntled customers instead of doing PR. I'm very concerned about the future of this company. [https://news.ycombinator.com/item?id=10578397](https://news.ycombinator.com/item?id=10578397)

</comment><comment><USER11> I expect him to say that his company is "doing great," but he doesn't address why Fidelity cut its outlook on Zenefits (especially if everything is rainbows and butterflies).

<comment><USER12> It went from a 4.5B company in their estimation to a 2.5B company in their estimation, still less than 3 years after it was founded. All projections were so far into the future already that they could swing massive %'s based on partner activity, even small revenue misses (ever changed your mo/mo growth rate in a spreadsheet by 1%?), or competitive activity. But going from a 4.5B company to a 2.5B company while still less than 3 years old isn't a sign of the apocalypse in the company.

</comment></comment><comment><USER13> Assuming the figures that Conrad is saying are correct, it is still an amazing company that is growing quickly. Just investors are starting to take a more realistic view on valuations, and readjusting those valuations in their books.

Just because super ambitious growth targets aren't met doesn't mean the company is about to go under...</comment>


Mapping migration worldwide: India vs China - <AUTHOR> http://www.economist.com/blogs/dailychart/2011/11/diasporas

<comment><USER2> The indian descents in mauritius, carribean and south africa migrated some 200 years ago. Counting them as indian diaspora is the same as counting the brazilians as the portuguese diaspora!</comment>


Web Development Recipes Now in Beta - <AUTHOR> http://pragprog.com/book/wbdev/web-development-recipes

<comment><USER2> No knowledge of RoR required? I ask because a couple of the authors list RoR in their bio. I love the book idea though. Will check it out if it's just HTML/CSS/JS (which it sounds like it is).

<comment><USER3> I'm one of the authors of the book.

You don't need to know any RoR. There are a couple chapters where we use some Ruby (Jekyll and Cucubmer-driven Selenium Testing), but we walk you through getting things set up properly if you want to use them.

<comment><USER4> Does that mean that whenever you recommend server-side solutions that the server must have Ruby set up?

[OT question that has blocked this PHP-hobbyist from dabbling in RoR: do professionals ever add a new functionality as _part_ of a web app via RoR when the bulk of project has already been built with something like PHP? (looking for motivation to start some experimentation in Ruby)]

<comment><USER3> Nope, we don't only talk about Ruby. We have a few PHP examples as well, but there's a lot more on CSS, JS, Git, server administration, etc.

There's not a lot of server-side code in the book, to be honest. Chapters like Creating a Widget focus on the client-side code - loading content in to your page and providing an example for what the data looks like, without getting in to "here's how you'd' do this in Ruby or PHP or .NET".

</comment></comment></comment></comment><comment><USER5> I know its still under development, but are all the chapters in at this time? In other words, if I buy it now will I only be able to read 4 chapters (for example) until others are added, or are they all there in some rough form?

<comment><USER3> The beta currently contains about half of the chapters. More chapters will be added in the coming weeks as they work their through the review process, but they're not in the beta copy that you'll get today.

<comment><USER5> Thanks. By the way, I've been a back end developer for many years and have only dabbled in the front. The table of contents alone make we want to buy this book (probably will) as it appears to run the entire gamut of front end development and appears to be very up to date with what people are doing these day. For me it would be a "survey of front end best practices" that I can use to dig in deeper.</comment></comment></comment>


Ancient Maya cities found in jungle - <AUTHOR> http://news.discovery.com/history/archaeology/three-ancient-maya-cities-found-in-jungle-140815.htm

<comment><USER2> The title and article are a little sensationalized. I grew up in Guatemala, and the entire Yucatan peninsula, along with Guatemala, Belize and Honduras, are covered in "undiscovered" cities. I'd regularly dig up pottery and carved stones in fields and garden beds.

The Calakmul area is especially dense in ruins. Most xateros (rubber tappers) know them all and can guide you to them for a couple bucks. Very few of them are surveyed or even mentioned in literature.

</comment><comment><USER3> This hit front page on reddit today, much more interesting comments. Mostly noting that there are lots and lots of "undiscovered" cities because it's extremely expensive to unearth and preserve them. The yucatan largely is flat, not unlike Florida. Most hills in the jungle are sites. "Undiscovered" sites.

[http://www.reddit.com/r/worldnews/comments/2e2il8/maya_citie...](http://www.reddit.com/r/worldnews/comments/2e2il8/maya_cities_found_in_yucatan2_massive_cities/)

</comment><comment><USER4> That's got to suck finding an undiscovered Mayan city, and then it turns out to actually be a discovered-but-lost Mayan city.

Sure makes you appreciate GPS though.

</comment><comment><USER5> I grew up reading stories of lost cities of gold and all that, so when satellite maps became more common in the 2000s I thought for sure we would find tons of lost cities. Unfortunately you can't get much off of even infrared scans of these areas as the buildings are rock and wouldn't have heat signatures any different from their surroundings.

One day I bet we will see more emerge from faster scanning tech.

<comment><USER6> Back when I was remote sensing, I remember some talk of people using SAR images (Synthetic Aperture Radar??) to search for Maya roads. I imagine something was found since then but I didn't kept up with the literature on the subject.

</comment></comment><comment><USER7> This is great and very historical, I am very interesting with this article. I love historic place.

<comment><USER8> If you check this one, kinda interesting to read which is one of the cities featured an extraordinary facade with an entrance representing the open jaws of an earth monster.

</comment></comment><comment><USER9> This bring memories:

[https://www.youtube.com/watch?v=kq5Y_ogiyi0](https://www.youtube.com/watch?v=kq5Y_ogiyi0)

</comment><comment><USER10> When I saw the title, the first thing to come in to my mind was: Dun da duh daaaa, dun da daaaaaaa....

</comment><comment><USER11> By the way, the "Maya" in the article's original title is correct. There's no reason to change it to the incorrect "Mayan".

<comment><USER12> Thanks. We missed that one.

(If anyone is wondering, as I was, the convention is to use "Mayan" for the language and "Maya" for the people.)

<comment> ar-jan Yes - but note that there are about 30 Mayan languages, not just one.

<comment><USER13> How is this measured? For calibration, how many English languages are there now?

<comment><USER14> Well there isn't a nice and easy test, but in linguistics it tends to come down to some measure of mutual intelligibility. How much meaning can a speaker of Variety A garner from a speaker of Variety B? And vice versa. For tricky distinctions, throw in some historical and social analysis to taste.

I don't know much about Mesoamerica but I believe the "30 languages" claim refers to the count of current languages descended from a single ancestor, not the languages of Classic Maya. (EDITED to avoid implying "Proto" and "Classic" Maya were one in the same--they were separated by millenia!)

English is a particularly tricky issue because there are so many "standard" Englishes understood by many but natively spoken by none, and because lingua francas tend to become such a smooth continuum between their different lects. However for the purposes of the mutual intelligibility tests, the vast majority of world Englishes would still fall under one "language". Many that don't are already labelled distinctly, either as creoles/pidgins, or from historical divergence (e.g Scots, Frisian). English' divergence is little compared to that of, say, Arabic, and I suspect any two random English speakers in the world would understand each other much much more easily than any two Mayan speakers.

<comment><USER15> I believe you are correct about the 30 (not 100% sure on the number) Mayan languages being ancestor of Proto-Maya. Classical Maya belongs to the Ch'ol branch and is mostly likely a written form of the parent language of Ch'olti' (extinct) and Ch'orti'.

Here are some random facts about the Mayan languages and Maya people:

Yucatec is the most spoken of the Mayan languages, located in the Yucatan peninsula. Interesting story about Korean immigration to Mexico (Spanish only, some early immigrants ended up in Yucatan and one lady mentions that many learned Yucateco before Spanish because the worked with they Mayas): [https://www.youtube.com/watch?v=AmDTrvddD_8](https://www.youtube.com/watch?v=AmDTrvddD_8)

Popul Wuj is written in K'iche'. Trippy videos about the Popul Wuj: [https://www.youtube.com/watch?v=aTeOYNsBHvM](https://www.youtube.com/watch?v=aTeOYNsBHvM) [https://www.youtube.com/watch?v=TDt9rQrGp1I](https://www.youtube.com/watch?v=TDt9rQrGp1I)

Lacondon people managed to escape Spanish control: [https://www.youtube.com/watch?v=6nSb36zgGzo](https://www.youtube.com/watch?v=6nSb36zgGzo)

</comment></comment></comment></comment></comment></comment><comment><USER16> [https://www.youtube.com/watch?v=xb4n8Cl3leg](https://www.youtube.com/watch?v=xb4n8Cl3leg)</comment>


Ask HN: Is Paul Graham still doing Y Combinator interviews? - <AUTHOR> I know he is no longer involved in the day-to-day YC operation, but is he still on one of the interview "tracks" himself? How much does he still participate in the Startup selection process himself?
Just Curious.
BTW, We've been selected and will be interviewing on May 4th (it's a late interview).
If any YC alumni or any founders who've interviewed for YC before would like to chat, network, help, etc, I'd love to. We're from Brazil and will be in the Bay Area until May 5th.
My email is bernardo AT querobolsa DOT com DOT br <comment><USER2> From [http://paulgraham.com/int15.html](http://paulgraham.com/int15.html)

 _I 'm not involved with selecting startups for Y Combinator anymore, but that is certainly what they are trying to find._

</comment><comment><USER3> Congrats on getting an interview. I hope you ace it, and get in.

I'm a late applicant myself, and is still awaiting a response.

Is there anything that you want to share about your startup, that the community can help with?

<comment><AUTHOR> Thanks! We didn't apply late, but had to take a late interview to be able to get a Visa on time for a co-founder that didn't have one.

Sure, we're like a "Booking.com for Colleges". We help students search for programs in enroll in College in Brazil, saving money on tuition. The company is growing a lot and quite big now (almost 100 employees). We're contemplation YC because fundraising has never been easy for us, investors don't get us because we are not a copycat from the US.

<comment><AUTHOR> BTW, the website (in portuguese) is [http://querobolsa.com.br](http://querobolsa.com.br)

</comment></comment></comment><comment><USER4> Good luck and good news that at least some South American startups are being considered :)

<comment><AUTHOR> Thanks! It was not easy for us, it's actually the 3rd time we apply and we have a LOT of traction... First time we're invited to the interview. Most south americans I see being selected to the interview are focusing on the American or international markets (we're not).

Glio (the only company tackling the Brazilian market that ever got in, AFAIK) had to apply 4 times before getting in.

</comment></comment><comment><USER5> Hasn't he automated it ;)

<comment><AUTHOR> I think he productized it... :)</comment></comment>


Neoliberalism: the idea that swallowed the world - <AUTHOR> https://www.theguardian.com/news/2017/aug/18/neoliberalism-the-idea-that-changed-the-world <comment><USER3> (The following is not arguing against the article, but just providing some context. It's about time we see more arguments against neoliberalism, which serves only a very tiny portion of people at the expense of large masses of others.)

For those in the US who are used to non-standard definitions of "liberal", the word "neoliberal" refers to people who support unregulated markets, austerity, and free trade. Reagan is perhaps the most iconic neoliberal in US history.

Confusingly, many self-professed "liberals" (US definition) are accused of being neoliberals. Examples include Bill and Hillary Clinton.

Even more confusingly, the word is now used not to describe people who have any relationship to classical liberals (what libertarians aspire to be), but rather people who support a strong government that serves the wealthy -- anti- labor, anti-market, elitism tending toward plutocracy.

<comment><USER4> There are two related but distinct meanings of neoliberalism. The meanings come from different strains of scholarship.

The meaning that encompasses Bill Clinton is one that appeared (AFAICT) in the 1990s in American political economy circles. It basically describes Tony Blair's so-called Third Way. The phrases describe a near simultaneous shift by the Labor and Democratic parties toward classically liberal economic theory. This was not a coincidence. Both Bill Clinton and Tony Blair effectively succeeded Reagan and Thatcher, who had completely transformed the political economy landscape, successfully selling a free market platform to the public. (Bush and Majors were interim conservative leaders, but nobody talks about them anymore. Their legacy is overshadowed by their predecessors and successors.)

Neoliberal and Third Way describe an approach where the government no longer directly intervenes in the market, but instead rely on muscular but indirect regulatory regimes.

The older meaning of neoliberal comes from, I think, the 1960s or 1970s to describe then-radical economists like Milton Friedman who rejected almost all government intervention, however indirect, in contrast to traditional liberal economic theory (espoused by the Republican and Conservative parties) which accepted in theory some significant regulatory role. This older meaning is the one more widely understood today. But when I was in college in the late 1990s and early 2000s reading political science, I was most familiar with the newer meaning of neoliberal.

Third Way is less ambiguous, but Tony Blair took office when Bill Clinton was beginning his 2nd term. American pundits were, I think, already using the term neoliberalism before Third Way developed any significant cachet this side of the pond.

EDIT: So the article says neoliberal was first coined in the 1930s. That makes sense. The more I think about it, there's really a third meaning to neoliberal that is probably what most people understand today: an epithet to describe policies that promote free trade and open markets to the detriment of working- class whites.

EDIT EDIT: Okay... I should have just read the article first ;) Which seems to agree with my sentiment.

</comment></comment><comment><USER5> The word _neoliberalism_ is dead. It has so many prejudiced interpretations amongst its opponents, that it ceases to function as a neutral unbiased concept in a rational discussion. Its enemies deliberately conflate it with financialization, inequality, rent-seeking, corporatism, cartels, oligopolies, corruption and cronyism. It is now used mostly as a pejorative term.

In fact, almost all that baggage is completely antithetical to the original neoliberal ideals. If you add neoconservatism, the military-industrial complex and militarization of law enforcemnt (esp. in the US), you get something that should be called what it is: _fascism_ , not neoliberalism.

What has really swallowed the world is social democracy (voting for deficits), hence an explosion of debt and govt control of monetary conditions:

\- Huge bloated governments (look at govt spending as a proportion of GDP, and public sector employment, in all westernized nations).

\- High levels of debt in all sectors: personal, corporate and govt.

\- Explicit control of short term interest rates and manipulation of longer term bond yields through QE.

\- Monetization of govt debt (QE + rollover at maturity + refund of coupon interest to treasuries).

The distortion and corruption is astonishing. With the Fed owning $1.8tn of US real estate (MBS) from QE, there's a good chance the Fed is your landlord! In Europe, companies issue bonds specifically for the ECB to buy at a premium, using printed money (EUR60bn/month). Then companies use the money to buy back their own stock, hence increase their share price, and allow execs to earn handsome profits on stock options - printing money into exec's pockets! The BoJ prints Yen to buy Japanese shares. The SNB prints CHF to buy US stocks and now owns $1.7bn of AAPL! Milton Friedman must be spinning in his grave.

Neoliberalism favors small responsible government, with low debts, allowing markets to set prices, including the interest rate, which is the price of money. The _neo-_ in the name indicates its deviation from the classic liberal _laisser faire_ economics of unfettered free markets, because the natural tendency is to reduce competition (see P.Thiel). Neoliberallism encourages govt intervention to keep a level playing field and protect fair competition in the open market, especially to prevent the distortions of asymmetric information, rent-seeking, cartels, oligopolies and monopolies.

The only minor victory for neoliberalism was the freedom from capital controls, which is now being rapidly reversed with financial restrictions enacted in the name of the fake wars on drugs and terrorism (KYC, AML, FATCA, FBAR...). Next is the war on cash, which will bring a loss of freedom, privacy and ownership of your money. We already have NIRP, bailins, civil asset forfeiture, limits on cash transactions and phasing out of large denomination notes (e.g. EUR 500).

<comment><USER5> More commentary on the rise of national socialism, involving monetization of govt debt and stealth nationalization of stock markets by central banks:

 _Central banks hold a fifth of their governments' debt_ (FT - <USER2> required) [http://www.ft.com/content/ae19e60e-81b0-11e7-94e2-c5b903247a...](http://www.ft.com/content/ae19e60e-81b0-11e7-94e2-c5b903247afd)

Unpaywalled commentary from the Mises Institute, so you can see what the (neoliberal) Austrian School really thinks about current monetary policy:

[http://mises.org/library/are-central-banks-nationalising- eco...](http://mises.org/library/are-central-banks-nationalising-economy)

And yet more fascism jackbooting over the US constitution:

 _Congress Quietly Passed a Bill Allowing Warrantless Searches of Homes_ [http://thefreethoughtproject.com/congress-passes-bill- allowi...](http://thefreethoughtproject.com/congress-passes-bill-allowing- homes-searched-without-warrant/)</comment></comment>


Amazon Targets YouTube with New Online Video Posting Service - <AUTHOR> http://www.bloomberg.com/news/articles/2016-05-10/amazon-targets-youtube-with-new-online-video-posting-service <comment><USER2> I run a popular YouTube educational channel (in the top 10 for it's niche) and have had some serious issues with YouTube that have nearly made me quit.

I had a young (pre-teen) boy who filed copyright claims on several of my tutorials, and they where taken down instantly and took months to resolve via email. I lost about $1500 in revenues which was never returned.

Beyond that, YouTube is taking a ~50% cut of your ad revenue AND any donation revenue on the site which seems pretty extreme. They also don't do good when it comes to new content discovery, as the YT search promotes mostly popular content (basically, bad for new creators).

<comment><USER3> If YouTube is such a crap fest, why not use an alternative like Vimeo?

<comment><USER4> Vimeo has a reverse business model from YouTube, creators must pay Vimeo to host their videos. They also do not have an ad-revenue sharing program for creators to make money back from.

Brady Haran and CGP Grey discuss their YouTube business and their gripes with the platform on Hello Internet. [http://www.hellointernet.fm/podcast/58](http://www.hellointernet.fm/podcast/58)

</comment></comment></comment><comment><USER5> It is prime time for competitors of YouTube to come out of the woodwork. Everyone is incredibly tired of YouTube's bonkers copyright complaints system which has already resulted in many people moving to Twitch.

To beat YouTube you essentially need to reproduce YouTube then create a more balanced playing field between content creators and copyright owners. At the moment the content creators exist in a "guilty until proven innocent world" where the copyright owner are the judge and jury too! It is bonkers, fair usage is dead on YouTube.

<comment><USER6> The real question is how would a competitor deal with the immense pressure from the music and film industry bodies? I mean, YouTube has nothing to gain from making life difficult for content creators - they are acting in this way because they have essentially been compelled to. If a competitor became popular enough, the same thing would happen.

<comment><USER5> For one thing they could require people to submit real DMCA notices. YouTube's system isn't DMCA-based, and as much as I dislike the DMCA, it still provides greater protections than YouTube's system does.

That's how bad YouTube's draconian copyright system is, the DMCA is an "upgrade."

<comment><USER7> how does something like that scale to the size of youtube?

<comment><USER8> It doesn't, not without lots of labor, and that's sort of the crux of the issue...

A YouTube replacement will be better in this respect only as long as they remain small and under the radar of the movie/record industry. :/

</comment></comment></comment></comment></comment><comment><USER9> Amazon has a very bad track record of integrating with other services: do you think you'll be able to watch this on a normal Android phones? or an Apple TV? or a Chromecast? will they let other places on the internet embed videos easily? how will they be substantially better on copyright/DMCA complaints?

On top of all that, I think this needs a nice clean discoverable UI for this type of thing to be successful and they've really struggled with that in the past.

I'm also sad about what keeps happening to Prime. This has a benefit that I wouldn't consider strong enough to sign up for while the rest of the business continues to sneakily weaken the shipping benefit I _did_ sign up for.

<comment><USER10> Back when Kindle launched, it was very common to see the story of Kodak being floated around (film vs digital) to claim how Amazon is flexible. Fast forward a few years, Amazon video and music services are the worst in terms of cross platform support. The primary reason I don't buy/rent anything from Amazon Digital is because I can't stream it to my Chromecast. Considering the adoption for Chromecast I would have expected Amazon to support. Instead they ban it from Amazon.com.

<comment><USER11> Well I think Amazon just played the tit for tat card here. Historically, Google took the first slab by banning Amazon from it's app store: [http://techcrunch.com/2014/12/11/google-removes-amazons- app-...](http://techcrunch.com/2014/12/11/google-removes-amazons-app-listing- from-google-play-search-following-addition-of-appstore-instant-video- integrations/)

No wonder Amazon is now reciprocating by explicitly disallowing chromecast devices on its own site.

tl;dr; Google ain't a saint/good citizen either. No one is.

<comment><USER9> Google removed Amazon's app for violating the ToS, which clearly states that you may not sell your own app store through Google's.

This is still entirely on Amazon's bad behavior.

Also, your theory sort of excuses Amazon but does not explain why they also banned Apple TV.

</comment></comment></comment></comment><comment><USER12> I love Amazon, but I wish they would integrate more with other companies. Did they really need to make their own app store?

I own several kindles, and I was hoping to play a few learning videos for my child on the kindle fire. Well, the Youtube support is quite lacking.

I am not sure if this new video venture will work out. Youtube has quite a bit of momentum, and it has far more support across the mainstream devices than anyone else.

Vimeo is second place in my mind, but I have not watched a video on there in ages.

<comment><USER13> "Did they really need to make their own app store?"

Yes, they did. Not for your benefit, but for theirs. They're a retailer, and they want to control their own ecosystem. Does Walmart let Target have a corner of their stores to sell stuff in? No,they don't.

<comment><USER14> There are few real-world analogies for the vertical that is Amazon. We don't have brick and mortar construction companies that also are a retailer (as far as I know). Amazon is like a company that builds houses, and will only allow Amazon appliances inside the house, even though other appliances generally work fine. Your buddy across the street has a house made by Google, and their house can have any appliance, which kinda irks you.

<comment><USER13> Well, not quite. The point is that you can't get anything for your Amazon "house" that Amazon isn't taking a cut from the sale.

(And in ye olde days, there was a time when you could open the Sears, Roebuck catalog, and order, literally, a whole house and everything that needed to go inside it, and have it delivered to you.)

</comment></comment></comment></comment><comment><USER15> I am reminded of an old Hypercritical podcast: "Everybody Wants to Rule the World" ([http://5by5.tv/hypercritical/74](http://5by5.tv/hypercritical/74)). Everyone wants their own stack to have full control over. So now we've got multiple gardens (of differing openness) tended by cathedrals.

</comment><comment><USER16> This is HUGE for the independent film world.

It's currently incredibly hard to sell an independent film online via a well- known and trusted channel that people are used to buying through. The nearest equivalent before this was getting onto iTunes through an aggregator, which costs $1500 or so per film.

I'm tremendously happy Amazon have taken this step: it'll be a huge benefit for indie filmmakers.

<comment><USER17> Interesting. Isn't vimeo an avenue? [https://vimeo.com/ondemand/startselling](https://vimeo.com/ondemand/startselling)

<comment><USER16> Last time I checked they required you to create an account before buying a film. That's an absolute conversion-killer.

Also, there's a huge difference between most people's comfort level in buying from Amazon and buying from Some Random Site - and Vimeo still counts as Some Random Site for most people. (As does Gumroad and similar.)

Finally, the discovery aspect is significant. People are searching Amazon for films to buy and watch. They, by and large, aren't searching Vimeo for that.

</comment></comment></comment><comment><USER18> Presumably I won't be able to watch the videos using my Chromecast.

<comment><USER19> While Amazon Prime does not currently support the Chromecast, you should be able to at least cast the browser tab.

<comment><USER9> That is not an acceptable alternative, sadly. :(

<comment><USER20> You would unfortunately have to take that up with Amazon

<comment><USER9> Yep, this is totally on them. It's led me to stop using their video products, and we're not renewing Prime after this period expires.

<comment><USER21> I agree it sucks, but you could have bought a Roku or something you knew _already supported Amazon Prime_ in the first place instead of a Chromecast. It's not like there's never been competition in that space; Chromecast is a super-late-comer.

<comment><USER9> I could have, although I've owned a few Roku boxes and they've all been slow, unresponsive and crash-prone. We actually upgraded from Roku to Chromecast.

Amazon would prefer I buy their Fire TV stick thing, which they would then use as a hook to sell me more Amazon stuff. Given Amazon's attitude towards Apple and Google's streaming devices, chances seem decent to me that if Roku ever increased its market share then Amazon might cut them off.

I'd rather just use another streaming video store.

</comment></comment></comment></comment></comment></comment></comment><comment><USER22> Aside from Youtube's copyright system, I also hope something like this works out because of their UI. I mean, let's face it. Youtube just seems to change their layout and features every few months or so, regardless of whether they make the site easier to use or not and regardless of what anyone actually wants said changes.

If Amazon can come up with a good, user friendly UI and then not screw it up by making constant changes, they could do quite well here.

<comment><USER23> As a light user it's relatively the same for me..

Upvote on purpose :D

</comment></comment><comment><USER24> Something like two months later, I still have not managed to get my custom YouTube URL. Everything to do with YouTube (which is completely filtered into G+) is a miserable experience, and I can't wait for more competition.

<comment><USER23> Implying you've met standard requirements, correct? Did you try "advanced" method of claiming ownership of URL to use name?

<comment><USER24> I've tried just about anything, including contacting support who never got back.

They've got this dumb system where you can claim a custom link, but it's

       youtube.com/c/TaylorGood[]

where you have to fill in [] with something. It makes no goddamn sense, and if I do it, that's going to be the new permanent url, which I clearly don't want.

Google are so clueless sometimes.

</comment></comment></comment><comment><USER25> This is huge. Amazon has the firepower to take on youtube. Hopefully Amazon can come up with a clean UI that makes people WANT to use their service unlike amazons current video streaming service.

<comment><USER26> Funny, I was thinking the opposite... I was wondering if people would actually use this system, and how Amazon would distinguish the naming... part of why "YouTube" is successful, is it stayed "YouTube" instead of "Google Video".

A separate brand for a separate product is important, as anything else dilutes the brand, which is already doing that to itself pretty well. Google has been better than many companies with this imo. Apple, better still.

It will be interesting in how this works out and ties into Twitch, as I think that will be the more natural conduit for integration. As it stands, it's just another place for clickbait sites to embed video from.

</comment></comment><comment><USER27> Is this going to be tied to Amazon's crappy app store as well so regular Android users won't use it?

<comment><USER28> maybe they'll make it available via sideloading from the web directly.

</comment></comment><comment><USER28> In the article they mention 15cents/6cents per hour. Is it considered a lot compared to ad revenue ?

<comment><USER19> I have a video that got 17600 views and 25000 minutes watched according to youtube analytics. It made $22.50 from ads. So that comes out to 5.4 cents/hour.

</comment></comment><comment><USER29> Total aside, but now that they are encroaching on Google's territory in video, I would hope they move to create a free mapping service for AWS users. Startups that try to generate revenue are so hamstrung by the baseline $10,000 maps for work license.

</comment><comment><USER30> The Amazon service requires close-captioning. Is that hard for video producers to set up?

<comment><USER19> There's free software that allows you to create subtitles/captions. For your average consumer who just wants to shoot a family video and upload it, that is a very large barrier to entry compared to youtube. For a high quality video creator who puts a lot of effort into editing videos, it's not much extra effort. This would be one way to ensure most videos are high quality.

<comment><USER14> > _This would be one way to ensure most videos are high quality._

I'm not so sure of that - Amazon can't possibly police every single video to ensure correct closed-captioning. There are likely tools that can take a video, rip the audio stream and machine generate a subtitle file for you. If you have a person uploading a family video this will pass the "good enough" bar for you. You'd get something close to Youtube's automatic closed-captions.

</comment></comment></comment><comment><USER31> If I'm a big video content producer what incentive do I have to remove myself from the user base of youtube?

</comment><comment><USER17> I have pocket recorded (imagine the quality) a Les Miserables show in 2011 and uploaded to YouTube at the end of 2012 when it became clear a legal CD won't happen (it should've!). It was taken offline and a copyright strike lodged against me last week. Do they own the copyright? Sure! But there are no legal recordings of that particular cast. So tell me how was their business affected by my upload? Mind boggling.

<comment><USER32> A free version online would inhibit sales of any future releases if they choose to do so. Just because you can't legally purchase it today doesn't allow you to harm the future sales of tomorrow.

<comment><USER17> People miss completely miss my point. There is a demand for this, there are forums, there are forum trackers where people trade bootlegs, illegal recordings from shows. Why are these not released? Just put it out as MP3, it doesn't need to be anything spectacular, if you do not want do not even spend money on editing just put the soundboard recording out. There will be people who would _die_ to pay for these, I know. There is nothing to lose -- wait until the cast changes (perhaps even just the lead) and then upload the previous cast recording. Done! Why, why, why this is not done? I need to hunt down musical shows from 16 years ago, why I can't just buy them? Who loses a single cent from me copying a 16 year old show which was not released legally? Piracy is a service problem doubly, triply so in the musical world because there is no legal supply and there is demand.

<comment><USER33> dura lex, sed lex.

"The law is harsh but it is the law."

<comment><USER17> Ha! I presume I deserve that. But what I am asking is -- why is the law this way? And, of course, the next step is, how do I change it? Or much rather: how could I convince the rightsholders to serve us better? For many years I have been yearning to be able to speak up and be heard in copyright-related matters but I can't for my life figure out how.

<comment><USER32> Such things have been done. It's a matter of researching the people responsible and trying to figure out what they would respond to. Bring enough folks to the public outcry and maybe they will see your point. There are several movies that exist almost solely because of public outcry, so I would think it might be possible to get an official recording done.

Or run for office and work on legislation to change the law. But I seriously doubt you would make many friends in the entertainment industry if you try to change copyright law too much.</comment></comment></comment></comment></comment></comment>


What was once a bug is now a widely used CSS property - <AUTHOR> https://en.wikipedia.org/wiki/Internet_Explorer_box_model_bug <comment><USER2> It was a bug because IE did not implement the spec. However, the spec was unintuitive and what IE did made more sense than the W3C box model.

<comment><AUTHOR> Exactly, they didn't completely implement the w3c spec.</comment></comment>


Version 2.084.0 of DMD, the D reference compiler, has arrived - <AUTHOR> https://dlang.org/blog/2019/01/05/dmd-2-084-0-has-arrived/ <comment><USER2> Congrats to the D dev teams. It is so sad that D is so underrated and was eclipsed by other languages such as Go and Rust. D is definitely a diamond in the rough</comment>


Reducing the size of iOS app binaries - <AUTHOR> https://blog.halide.cam/one-weird-trick-to-lose-size-c0a4013de331 <comment><USER2> Circa 2014(15?), I spent a couple of weeks working on reducing a Universal iOS App's(Trapster) payload size. This was before Asset Catelogs, so everything was in a fat binary. We converted the audio to be monoaural. We did PNG-crush. Gzipped embedded dynamic data that was preloaded. Optimized SVGs. All the hoops, they were jumped through. It only reduced binary size from 60Mb to 50Mb.

Taking another look at the payload showed that the otherwise innocuous launch images were 32ish Mb in size. The background of the images wasn't a solid colour. The background pattern was more of grayscale static similar to an unwatchable terrestrial TV feed. It was intended to resemble the initial sound of a CB radio used by truckers. (The "Chshhh" in "Chshhh 10-4 Good Buddy")

Since "staticy" audio/imagery like that is inherently random, each launch image(1 per device resolution) was composed of primarily uncompressable data. Switching out of the "staticy" background for a plain gray reduced the payload size to ~20M, a savings of ~30Mb. That change took approximately an hour.

The takeaway here is to take a good inventory of the payload before tackling size reduction. Otherwise you end up fighting the hard fight just to be pennywise

Also, uncompressable launch images for iPad Retina displays are stupid huge.

<comment><USER3> _Switching out of the "staticy" background for a plain gray reduced the payload size to ~20M, a savings of ~30Mb._

I know this might sound somewhat obvious in retrospect, but you could've retained something similar to the original background _and_ reduced the size significantly by generating it at runtime --- the code to create a random- looking background may only be not more than a few dozen bytes, and can be the same for multiple resolutions.

For more examples of this technique, see demoscene productions --- the following video was generated by a binary of less than _4096 bytes_ :

[https://www.youtube.com/watch?v=jB0vBmiTr6o](https://www.youtube.com/watch?v=jB0vBmiTr6o)

Source code: [https://github.com/in4k/rgba_tbc_elevated_source](https://github.com/in4k/rgba_tbc_elevated_source)

Explanation: [http://www.iquilezles.org/www/material/function2009/function...](http://www.iquilezles.org/www/material/function2009/function2009.pdf)

<comment><USER4> You can't programmatically generate the launch image on iOS.

<comment><USER3> I've never worked with iOS but given what I've heard about the extensive restrictions on what apps can do, I'm not surprised. I guess this is more like an icon which the OS retrieves from a known place and displays, _before_ the app even runs?

Then again, like most demosceners, I see "you can't" as a challenge... and immediately thought of whether exploiting the image format/whatever loads it to run the necessary code would do the trick. Probably wouldn't please Apple for sure. ;-)

<comment><USER5> Use an obscure format with a code execution vulnerability in imagemagick to run the generator.

<comment><USER4> PNG only, and I think certain types of PNG as well (e.g. you can't use the alpha channel).

</comment></comment></comment></comment></comment></comment><comment><USER6> Reducing the size of _anything_ : websites, apps, you name it: just _think_ about it, pay attention to shrinking easily-shrunk things, and avoid including unnecessary stuff.

Improving the performance of _anything_ : much the same. (Caveats apply, of course. Sometimes using such-and-such a library can genuinely make things faster than hand-rolling an equivalent.)

Improving _anything_ : actually _think_ about things. _Care_ about them.

(There are trade-offs with everything, of course. And time is often a major one. When you see something you want to do but don't have time for it—"Alas for the wailing of the gulls!")

<comment><USER5> "Actually think about it" is not a very actionable recommendation though. If I have a big application and I want to make it smaller, it'd be more helpful if you gave me tips how to find spots where I should focus my attention and some hints about how to avoid "unnecessary" stuff. Developers don't include things just because it looks nice in the makefile. Everything solved some problem when it was included.

<comment><USER7> " Developers don't include things just because it looks nice in the makefile"

This is actually surprisingly common, much more so these days with cookie cutter repositories.

There are large communities of developers who feel better with a large toolkit of functionality in a build environment, where a typical project may only benefit from a fraction of the included libraries

My limited experience of iOS dev suggests this is less of a problem Here, but I'm sure still is a factor

<comment><USER3> _There are large communities of developers who feel better with a large toolkit of functionality in a build environment, where a typical project may only benefit from a fraction of the included libraries_

In my experience, these tend to be the same type of developers who tend to copy-paste code extensively and not really understand what they're doing. They may be really good at finding these "cookie cutter" solutions and following instructions, but when asked to actually explain, they're unable to. This is particularly problematic for debugging, when you ask one of these developers about the reason for the way something is written and he doesn't know beyond the fact that he saw and copied it from somewhere.

</comment></comment></comment></comment><comment><USER8> Not mentioned: their product is incomprehensibly simpler than FB. Besides having 100 times less features, they probably skipped all the a11n, i18n, multitasking, etc...

They mention not doing A/B testing and that sort of things because it's "their philosophy". Obviously for their size it doesn't make sense to spend a week A/B testing to possibly increase usage by 0.5%, but for FB any change like this represents millions of USD.

<comment> GuB-42 Still, 400MB is just huge. To put it into perspective, the entire storage of the Nexus One is 512MB. The current Facebook iOS app is larger than an entire smartphone OS with the 2011 Facebook app installed.

People don't seem to realize how big a megabyte actually is. Just look at the size of oldschool games (ex : 8MB for Super Mario 64), or what size coders manage to archive now (see demoscene 64k/4k intros).

And you might say that it was the past, things are much more complex now, blah blah blah... Well, it's partially true but modern platforms allow savings that we couldn't have back then. Realtime vector graphics, better compression algorithms. And considering the trend towards minimalist graphics as opposed to real-life images, assets shouldn't exceed a few kilobytes.

Libraries? The OS has plenty of libraries and assets you can use for free, this should be another size saving compared to the old times. And if you insist on packing your own libraries, I suspect sizes would be much smaller if development tools were smart enough to get rid of everything not needed.

Essentially, the real reason apps are so huge nowadays is because we have enough storage space to be lazy.

<comment><USER9> >> Essentially, the real reason apps are so huge nowadays is because we have enough storage space to be lazy.

Exactly. In the early days of gaming, if you came up with creative ways to pack more levels, graphics, or characters into that same 64k of RAM or 2MB of ROM, you'd likely end up with the more advanced game and sell more copies. Pushing hardware to it's limits was the pride and joy of programmers back then.

The barrier to entry is so much lower now. Anyone can create a phone app. All you need is a Mac and watch some YouTube tutorials, that's why 90% of the App store is garbage.

In the old days you have to have development kits and licenses and you would need to have an actual idea/product, expensive hardware to test and develop on, etc. And I'm really doubting that there were many arcade or console programmers that weren't "clever". Sure, there were bad games back then but mostly do to bad design, graphics, horrible movie license tie-ins...

</comment></comment></comment><comment><USER10> Slightly off topic but I wonder if the name is likely to cause issues given that there's another, possibly older project named Halide in the image processing software world: [http://halide-lang.org/](http://halide-lang.org/) (~6 yrs old according to Github commit history).

<comment><USER11> Also used heavily in mobile photography, by Google and others

</comment></comment><comment><USER12> Relevant: The Size of iPhone's Top Apps Has Increased by 1,000% in Four Years

[https://news.ycombinator.com/item?id=14616536](https://news.ycombinator.com/item?id=14616536)

<comment><USER13> That makes me think if all these stuff happens because of developers/requirements, or because of Apple/XCode build routine.

</comment></comment><comment><USER14> > It's just our philosophy around products. Knowing too much data warps your > mind. You find yourself optimizing for a local maximum, instead of making > big bets that really move the needle.

Talk about a false dichotomy around A/B tests. You can do big things _and_ do small things.

But I guess a lot of people like having the moral high ground of not following trends... they could really just say "we don't feel the need for A/B tests".

<comment><USER15> Not a false dichotomy at all. Ignoring that the size of the team is exactly two (and there's only so much you can keep in your head), when you have a metric, you optimize for it. It's not even a conscious process.

</comment></comment><comment><USER16> I have to update the screenshots for my iOS app, but it seems that I should submit a new binary, which in turn pushes it all my users.

I wonder if it would help to have a checkbox "This update is not worth pushing to already installed devices."

<comment> yoz-y At some point it was possible to update screenshots without uploading a new app. Apple changed it because it was abused by apps that uploaded a crappy app and changed the screenshots after review.

At one wwdc delta updates were mentioned (this would solve this issue) but it seems that it went to the pile with open source FaceTime.

<comment><USER17> Delta updates are for users, not developers. It means that they don't have to download the full app when updating.

<comment> yoz-y Well yes. But in this case that is the problem no? As a developer I have to upload new binary when changing screenshots, thus forcing my users to redownload a new version. A diff of an identical binary would be zero (or something very small as at least the build number would have to be updated). For developers it would be better too but Apple has always privileged user comfort over developers.

</comment></comment></comment></comment><comment><USER18> For some non-obvious on-point tips regarding binary size and (also importantly) startup speed in complex apps, see 'Swift with a hundred engineers' (2016) where Tuomas Artman shared what Uber engineering's learned from their iOS rewrite[0].

Nit: advocating AutoLayout to me doesn't seem to compute with avoiding libraries. Is it actually feasible to use AutoLayout in a larger-scale app without something like Cartography[1]? Interface Builder still feels quite buggy in this regard.

[0] HN discussion: [https://news.ycombinator.com/item?id=14207752](https://news.ycombinator.com/item?id=14207752)

[1] [https://github.com/robb/Cartography](https://github.com/robb/Cartography)

<comment><USER19> > Is it actually feasible to use AutoLayout in a larger-scale app without > something like Cartography?

You can now do this: addConstraint(button1.rightAnchor.constraint(equalTo: button2.leftAnchor, constant: -12.0))

I think this is pretty compact and readable as it is. But it's only supported in iOS 9 and macOS 10.11 (right now, only one version back).

</comment></comment><comment><USER20> This is a quite opinionated piece, and of course not for everyone. I want 100% of all crashes reported, so I need Fabric or similar. And I write way better and more stable code with an RX framework, so I'll use RxSwift or ReactiveSwift/Cocoa. And since I do have to interface with JSON, I will include ObjectMapper or similar (until I can have iOS11 as min target). And sure, I could go by with just NSURLSession, but Alamofire's footprint is not that big, so I'll add that too. Why should my app be smaller than 20-30MB if Facebook can get away with 400MB? There will be 1% of users who are delighted by a small binary footprint, but the rest will not even notice.

BTW there's a totally different story with Android apps that will be installed in developing countries, by users with cheap devices and low connectivity. In that case I would scrape every last bit off. On an iPhone I do expect most users to have a fairly decent connection because you won't get very far without it.

<comment><USER21> _Why should my app be smaller than 20-30MB if Facebook can get away with 400MB?_

Facebook can get away with it due to being so central to people's lives. Even though 32GB is the minimum storage on new iOS devices now, there are still a lot of people with 16GB iPhones and iPads. Managing storage on these devices is quite painful, with a lot of apps now taking between 50 and 500MB. When I still had a 16GB iPhone, I had to remove many applications because I ran out of space all the time since about a year ago.

We should feel collectively responsible for all this bloat. Sure, we don't have to fit a full Pascal IDE and compiler on a 1.44MB disk anymore. But it is certainly good to keep some reference points in mind:

\- Turbo Pascal was a full IDE and was 2.29MB for all the bits. They didn't use an existing GUI framework either.

\- QNX fit a kernel + graphical environment on a 1.44MB 3.5" disk, with a network stack and web browser.

213MB for a chat app is outrageous, no matter how you look at it [1].

[1] [https://itunes.apple.com/us/app/snapchat/id447188370?mt=8](https://itunes.apple.com/us/app/snapchat/id447188370?mt=8)

<comment><USER22> > Turbo Pascal was a full IDE and was 2.29MB for all the bits. They didn't use > an existing GUI framework either.

Forgetting about Turbo Vision? :)

I used to carry MS-DOS 3.3 and Turbo Pascal on a 1.44MB 3.5" disk, back on those days.

Don't remember if it was TP 3, or if I tailored it to my purposes though.

</comment></comment></comment><comment><USER23> As the article points out its mostly a review of [https://developer.apple.com/library/content/qa/qa1795/_index...](https://developer.apple.com/library/content/qa/qa1795/_index.html)

<comment><USER22> Still relevant, as it appears many developers don't pay attention to what happens at WWDC, IO or BUILD sessions, where these type of advices are repeated over and over.

</comment></comment><comment><USER24> Does iOS not have incremental app updates? One time big download on Wifi followed by smaller incremental updates is not a big problem when devices are starting to have doubled storage capacities and fast transfer speeds.

<comment><USER25> Yeah it kinda sucks that Apple makes you re-download the app essentially. Its worse for iOS updates though. A full GB just for a point release?! I haven't use a Mac in a few years, so I'm not sure if its the same on the OSX App Store. Few years back I was aghast at having to re-download XCode for updates, plus it was super buggy then. Maybe its changed now.

<comment><USER26> Over-the-air updates have been very small since iOS 9. Last one was about 32MB or so.

<comment><USER25> Hmm, I use itunes to update. Not sure why I'm forced to download such giant files if they already have tech for differential updates.

</comment></comment></comment></comment><comment><USER27> > "Unfortunately, tech CEOs [...] don't live in areas with shoddy speeds"

Even Linux distro devs don't : Fedora's DNF package manager systematically downloads a file of 30 MB that (I assume) contains the signature of all packages to compare it to my local packages. Even if there are no updates, it still takes 10 min and slows my connection speed.

This boggle my mind, I would have thought from all OS Linux would be the one to save bandwidth.

<comment><USER28> On my Arch Linux box, package lists are about 5.8 MiB total. (And yes, those contain basically a list of package metadata and signatures in the repo.) I'm going to assume that Fedora carries more packages (do you have rpmfusion enabled?), so at least the magnitude looks okay.

One thing is worth mentioning here: The package manager _could_ easily save bandwidth if it didn't download the whole package list and instead queried the server only for updates for the set of packages that are actually installed (which is usually an order of magnitude less than what's in the repos). However, this approach is not feasible for most Linux distributions since they rely on mirrors to distribute packages, and mirrors only serve static content.

<comment><USER29> There are several very clever ways around this, with continual rolling updates in a general fashion with only static files.

Not static per user, but basically giving (source + diff + diff) on a daily / weekly / monthly basis allows you to only download a diff based on the frequency which you connect.

Hell, at this point, just git-clone a repo/data structure (potentially with an initial shallow clone since you may not care about history) and roll forward "as good as git"

</comment></comment></comment><comment><USER30> The points regarding AB tests and analytics simply isn't applicable to most commercial apps.

Understanding how your features/changes affect key metrics and understanding how your users are interacting with your app is so crucial when you are trying to generate revenue.

I would argue Apples analytics are no where near enough to accomplish this.

</comment><comment><USER31> Facebook's app is so bloated because of one thing: Apache Thrift. Last I checked it came with over 16,000 stubs and serialisers for thousands of different data types.

</comment><comment><USER32> Everyone should throw their whole source trees at ImageOptim:

[https://imageoptim.com/mac](https://imageoptim.com/mac)

</comment><comment><USER33> What a luxury to have an app that is only 15kloc. That is why it's small.

If he had a 20kloc objective-c app, how much smaller would of it been ;)

</comment><comment><USER34> Wow, I thought I was going to read an article about optimizing your code for size, finding ways via assembly to beat your compiler, etc. What's actually in the article are basic "best practice" things that every developer should be doing anyway. I'm glad the article exists, but as a mobile developer, if this stuff is news to you, you might need a wake-up call.

<comment><USER35> It's a best practice to dump cocoapods?

<comment><USER34> I've never measured whether using cocoapods to import dependencies produces larger or smaller binaries than not using it. I would imagine it makes no difference, but it's something worth measuring if you are considering using cocoapods and are worried about binary size.

Personally I don't like cocoapods and if it were my project I would remove it (and/or not use it in the first place). This is more of a personal preference though, and not necessarily the right thing for other people's projects.

</comment></comment></comment><comment><USER36> I do not think so

</comment><comment><USER37> I am building an app, right now it's about 600MB exploded. My backup, with reduced image quality (winzip for mac does this), drops to 30MB (git is good but 3 backups per day on a cloud is better).

I will ask pngcrush out on a date, take it out to a nice restaurant and then marry it!!

Thank you for the great tips!

</comment><comment><USER38> The vast majority of people don't care about the app size unless it's blown out of proportion.

If Apple started to charge for bandwidth transparently at market price, that would make a lot of difference. But right now if an app is within the Over- the-Air limit of 100MB, developers don't really care.

Look at Apple's own best practices. No incremental updates, zip compression when brotli and zstd already there, forcing its own pngcrush which just hurts compression without any real benefits for loading time and now it's a mandatory for Asset Catalogs and App Thinning.

<comment><USER39> I disagree. My phone is completely full, every new app I install requires going through and deciding which apps I need to delete to make space for it. Large apps get chopped pretty much straight away unless they are exceptional.

<comment><USER38> If the vast majority did care, there would not be a 419 MB Facebook app.

I personally analyzed a number of apps and found image assets can be reduced by 30-50%. Not only caring, working on the solution.

<comment><USER22> People do care, but there is a big difference between Facebook and unknown developer X.

<comment><USER38> Then why do I get all these ignorance and downvotes by taking action?</comment></comment></comment></comment></comment>


Voronoi Fracture and Shatter Lab Tests - Blender & Bullet Physics - <AUTHOR> http://www.youtube.com/watch?v=FIPu9_OGFgc <comment><USER2> Do you have any written material (blog posts, tutorials, books ...) about the theory behind this ?

<comment><AUTHOR> The author gives some info[1] on his former video[2], which includes source code[3]. Other types of this video: <http://www.rtbot.net/voronoi_fracture>

This paper may give some insights and even more digging: [http://dilbert.engr.ucdavis.edu/~suku/nem/papers/voronoifrac...](http://dilbert.engr.ucdavis.edu/~suku/nem/papers/voronoifracture.pdf)

[1] [http://bulletphysics.org/Bullet/phpBB3/viewtopic.php?f=17...](http://bulletphysics.org/Bullet/phpBB3/viewtopic.php?f=17&t=7707)

[2] <http://www.youtube.com/watch?v=0_qVjLGuT6E>

[3] [http://bulletphysics.org/Bullet/phpBB3/download/file.php?id=...](http://bulletphysics.org/Bullet/phpBB3/download/file.php?id=869&sid=5f37ed2c369a14dbf3df2a9c88760ed8)</comment></comment>


The Superbook: Turn your smartphone into a laptop for $99 - <AUTHOR> https://www.kickstarter.com/projects/andromium/the-superbook-turn-your-smartphone-into-a-laptop-f?ref=19ekwh <comment><USER2> It's kickstarter which means it won't actually ship until at least a year after it claims. And there's no way they're going to provide a decent keyboard, touchpad and screen for $99 and still make a profit.

It'll ship late and be probably 2x that cost at which point you should just get a chromebook. Or, ya know, use the laptop you already have.

<comment><USER3> Very sad that Kickstarter has earned such a reputation.

<comment><USER4> True, but sadly he's right. I still get emails from projects I funded YEARS ago that have still yet to ship a single product and they're still trying to make promises.

I'd argue that these platforms create a perverse incentive for people to exaggerate their experience and/or product to appeal to the clueless masses but I guess that's the same with most kinds of marketing ...

<comment><USER3> The same perverse incentive underlies all marketing. If you tell the truth and make sane claims you lose to flash, grandiosity, and viral gimmicks. Over time the whole system runs over a cliff. It's sort of a game theory meltdown thing.

It is eventually self correcting, but that usually involves a market crash and a Game of Thrones style winter where the junk is purged.

</comment></comment></comment></comment><comment><USER5> Chromebooks start at $149. This is just a Kickstarter price claim, not a real price. It will probably increase after launch, after which it will no longer be competitive.

<comment><USER6> I am also wondering how this can compete with something like the Asus FlipBook Chromebook that not only nativly supports Android but also has a 11" touch screen and a unibody aluminium design.

</comment></comment><comment><USER7> Didn't a number of major vendors (Motorola, Microsoft..) already do this and failed to find a market for it?

<comment><USER8> There's been a few all right. I remember the Celio Redfly for one. None of them worked so well.

That said, tablets didn't work so well when Microsoft tried them out 15 years ago, but they blew up later. Same with a number of other techs that weren't ready. Is this the time for smartphone computers? Nah, but keep an open mind!

<comment><USER9> > That said, tablets didn't work so well when Microsoft tried them out 15 > years ago

Or when Eo/GO tried it 10 years before that!

[https://en.wikipedia.org/wiki/EO_Personal_Communicator](https://en.wikipedia.org/wiki/EO_Personal_Communicator)

</comment></comment></comment><comment><USER10> I'd rather connect a display, keyboard and mouse to my phone. But the UI needs a cursor, and a file system.

If I recall correctly Ubuntu wanted to try that, as did Microsoft. Too early to market perhaps.

<comment><USER11> Android has a cursor. If you connect a mouse using a USB On-The-Go cable (essentially a micro to full-size USB adapter) you'll see it appear. Keyboards and game pads work too.

<comment><USER12> Can use bluetooth as well. I've helped mates replace their windows laptops with android tablets for (non-STEM) uni coursework. Pick up a cheap bluetooth keyboard and mouse, a tablet, and you're good to go. The keyboard and mouse will last you a decade, easy, and buying US$100 tablets every 2-3 years, you're looking at nearly a decade before it would cost the same as a laptop.

</comment></comment></comment><comment><USER13> Well this is very similar to what Ubuntu Phone is providing, and Ubuntu does have a rich rich set of linux applications but not as rich as Android.

</comment><comment><USER4> Wouldn't calling your company "Andromium" infringe on two trademarks simultaneously?

<comment><USER14> Shouldn't - to be infringing it has to be in the same industry (check), and substantially similar in sound, graphic design or spelling to the point that it would confuse the proverbial "average person". Is "Andromium" and more confusing than "Andromeda" to someone who wants to buy an Android phone?

It does reminds me of the el-cheapo Panashiba stereo I had as a student, though.

<comment><USER15> I think OC meant "android + chromium".

<comment><USER14> yes. Is anyone who wants to buy an "android" and a "chromium" be confused by andromium? That's the question for establishing trademark infringement.

And I mentioned my Panashiba stereo because it was an obvious mix of Panasonic and Toshiba, but was not a trademark infringement of either.

</comment></comment></comment></comment><comment><USER16> Middle of the page, there's an image below the caption, "Multi-Touch Trackpad & Keyboard - With Android Navigation Keys". What is the mechanical pencil shown in that picture?

<comment><USER17> Rotring 800 Series

<comment><USER16> Nice! Thanks!

</comment></comment></comment><comment><USER18> Wow, people invest money even when there is NO working prototype.

<comment><USER19> We're in SF. Come by our offices and play with the complete, working prototype. email@example.com

<comment><USER20> Don't invite people on the internet to make a trip to your office. You know perfectly well that the vast majority of people reading here aren't even anywhere close to you.

Show video of a person actually using it. Unedited, continuous shot. Right now it looks like you're just plugging a phone into a macbook and then have the macbook fake up some stuff in the video.

</comment></comment></comment><comment><USER21> But I already have a laptop..

<comment><AUTHOR> This product is best for people who have something like the OnePlus 3 but a laptop with only 2 gigs of ram.

If you have a MBP 16gb or a Surface Pro 4 (or.. next year.. a SP5?) then this probably isn't going to be your every-day-carry.

</comment></comment><comment><USER22> Is Usb 2.0 fast enough for this kind of application?

<comment><USER23> From the FAQ:

Does the Superbook require MHL? Nope. The Superbook doesn't use MHL technology --instead, it uses Displaylink technology

Check out [http://displaylink.com/downloads/android](http://displaylink.com/downloads/android) for more info!

I haven't personally tried a Displaylink device yet but It looks like it works well with USB 2.0.

<comment><USER22> Ok, thanks, that makes sense. It's actually just a connection for the display and the keyboard, after all.

</comment></comment></comment><comment><USER24> [http://liliputing.com/2013/07/ubuntu-edge-canonical-wants- to...](http://liliputing.com/2013/07/ubuntu-edge-canonical-wants-to-build-a- crowd-funded-linuxandroid-phone-needs-32-million-in-31-days-to-do-it.html)

Crazy that Ubuntu needed $32m, while this kickstarter has only asked $50k.

<comment><USER25> Ubuntu wanted to kickstart a phone. This device doesn't even have a CPU.</comment></comment>


Static Site Generators compared. - <AUTHOR> http://asselinpaul.com/

<comment><USER2> hakyll ( <http://jaspervdj.be/hakyll/> ) is a nice alternative for those of us who like haskell or just functional programming in general.

It features a DSL and integration with the awesome pandoc (<http://johnmacfarlane.net/pandoc/>) project.

<comment><USER3> Hakyll is pretty widely used, as well (at least it seems that way to me, maybe that's just my confirmation bias). I was surprised not to see it on the list in the OP.

</comment></comment><comment><USER4> Right now, I'm giving nanoc (<http://nanoc.stoneship.org/>) a try. My goal is to leave WP behind.

But there are some things, that boggle me. But, i do this to learn and so it is quite interessting to research my problems, till I find a solution. And if I ever get stuck, the help on the google group is quite great.

So as it is not mentioned in the original posting, I thought I'd add it.

<comment><USER5> Check out their IRC channel. The group in there are quite helpeful. The main author is always willing to help.

</comment></comment><comment><USER6> I wrote a very simple one called MCMS that runs on Node.js and RingoJS (Rhino on the JVM): <https://github.com/olegp/mcms>

</comment><comment><USER7> I can't tell you how many blogging platforms I've hopped through. I've got posts on a trillion domains and twice as many engines. Reading lists like these make me want to try them all now! What is it about site generators and blogging engines that makes people so fickle?</comment>