Simple Dockerfile examples are often broken by default - <AUTHOR> https://pythonspeed.com/articles/dockerizing-python-is-hard/ <comment><USER2> I have a mixed opinion about his first point.

There are two basic approaches to take with dependency management.

The first version is to lock down every dependency as tightly as you can to avoid accidentally breaking something. Which inevitably leads down the road to everything being locked to something archaic that can't be upgraded easily, and is incompatible with everything else. But with no idea what will break, or how to upgrade. I currently work at a company that went down that path and is now suffering for it.

The second version is upgrade early, upgrade often. This will occasionally lead to problems, but they tend to be temporary and easily fixed. And in the long run, your system will age better. Google is an excellent example of a company that does this.

The post assumes that the first version should be your model. But having seen both up close and personal, my sympathies actually lie with the second.

This is not to say that I'm against reproducible builds. I'm not. But if you want to lock down version numbers for a specific release, have an automated tool supply the right ones for you. And make it trivial to upgrade early, and upgrade often.

<comment><USER3> > The first version is to lock down every dependency as tightly as you can to > avoid accidentally breaking something...The second version is upgrade early, > upgrade often...Google is an excellent example of a company that does this.

This is misleading. My understanding of Google's internal build systems is that they _ruthlessly_ lock down the version of every single dependency, up to and including the compiler binary itself. They then provide tooling on top of that to make it easier to upgrade those locked down versions regularly.

The core problem is that when your codebase gets to the kind of scale that Google's has, if you can't reproduce the entire universe of your dependencies, there is no way any historical commit of anything will ever build. That makes it difficult to do basic things like maintain release branches or bisect bugs.

> if you want to lock down version numbers for a specific release, have an > automated tool supply the right ones for you. And make it trivial to upgrade > early, and upgrade often.

This part sounds like a more accurate description of what Google and others do, yes.

<comment><USER4> For an easy open source example of such tooling, see Pyup.

We use it to do exactly that: pin down every dependency to an exact version, but automatically build and test with newly released versions of each one. (And then merge the upgrade, after fixing any issue.)

<comment><USER5> Or the original ruby bundler, which locks down exact versions in a `Gemfile.lock`, but lets you easily update to latest version(s) with `bundle update`, which will update the `Gemfile.lock`.

Actually, it goes further, `bundle update` doesn't update just to "latest version", but to latest version allowed by your direct or transitive version restrictions.

I believe `yarn` ends up working similar in JS?

To me, this is definitely the best practice pattern for dependency management. You definitely need to ruthlessly lock down the exact versions used, in a file that's checked into the repo -- so all builds will use the exact same versions, whether deployment builds or CI builds or whatever. But you also need tooling that lets you easily update the versions, and change the file recording the exact versions that's in the repo.

I'm not sure how/if you can do that reliably and easily with the sorts of dependencies discussed in the OP or in Dockerfiles in general... but it seems clear to me it's the goal.

</comment></comment></comment></comment><comment><USER6> Good points, but it's amusing that his solution to #1 didn't lock down the patch version, nor the distro around it. I think that also makes a decent point for Nix[0], which solves #1-#3 by default (since choosing a particular version of Nixpkgs locks down the whole environment, and considers the build as a DAG of dependencies rather than a linear history). It also supports exporting Docker images, while preserving Nix's richer build caching.[1]

[0]: [https://nixos.org/nix/](https://nixos.org/nix/)

[1]: [https://grahamc.com/blog/nix-and-layered-docker- images](https://grahamc.com/blog/nix-and-layered-docker-images)

<comment><AUTHOR> Good point, will go fix that. My soon-to-be-ready attempt at a production- ready template ([https://pythonspeed.com/products/pythoncontainer/](https://pythonspeed.com/products/pythoncontainer/)) covers the tradeoff between point releases vs. not-point-releases, and it does pin the OS.

And yes, Nix fixes some of the problems of building a production-ready image, but only a subset.

<comment><USER7> Could you elaborate on the remaining problems with Nix for building Python images?

<comment><AUTHOR> Not an expert on Nix, but it's not so much that Nix has problems (though I'm sure it does, my initial research suggested it's not quite there yet for Python packages) but that there other things you need to get right.

For example:

1\. Signal handling (only one bit of [https://hynek.me/articles/docker- signals/](https://hynek.me/articles/docker-signals/) is Dockerfile specific, the rest still applies.)

2\. Configuring servers to run correctly in Docker environments (e.g. Gunicorn is broken by default, and some of these issues go beyond Gunicorn: [https://pythonspeed.com/articles/gunicorn-in- docker/](https://pythonspeed.com/articles/gunicorn-in-docker/)).

3\. Not running as root, and dropping capabilities.

4\. Building pinned dependencies for Python that you can feed to Nix.

5\. Having processes (human and automated) in place to ensure security updates happen.

6\. Knowing how to write shell scripts that aren't completely broken (either by not writing them at all and using better language, or by using bash strict mode: [http://redsymbol.net/articles/unofficial-bash-strict- mode/](http://redsymbol.net/articles/unofficial-bash-strict-mode/))

etc.

<comment><USER8> > though I'm sure it does, my initial research suggested it's not quite there > yet for Python packages)

Can you expand on what's missing? I've successfully used nix to cross-compile a pretty substantial python application (+ native extensions, hence the cross compilation), for embedded purposes, and it pretty much worked out of the box. Adding extra dependencies was straightforwards.

I think you can use pypi2nix for pinned dependencies, and you can run it periodically for security updates.

<comment><AUTHOR> Like I said, it was very preliminary research... I reached the bit where pypi2nix did "nix-env -if [https://github.com/garbas/pypi2nix/tarball/master"](https://github.com/garbas/pypi2nix/tarball/master") and wasn't super happy about the implications of "just use master" for production readiness.

If it works, though, that's great!

The more general point though is that in my experience no tool is perfect, or completely done, or without problems. E.g. the cited [https://grahamc.com/blog/nix-and-layered-docker- images](https://grahamc.com/blog/nix-and-layered-docker-images) suggests you need to spend some time manually thinking about how to create layers for caching? Again, very preliminary research—I know people are using it, I'm just skeptical it's a magic bullet because nothing tends to be a magic bullet.

<comment><USER6> Regarding layering, it used to be a completely manual process (just like with Dockerfiles), but the point of the blog post was that you can now use `buildLayeredImage` and correct layering will Just Happen.

<comment><AUTHOR> Ah, neat, hadn't realized that was an actual Nix feature now. The post made it sound like this was just something they were writing for themselves.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> I feel like programmers often fail to grasp the distinction between example code and production-ready code.

On one project, we made some example code available, and people would copy and paste it into their project, change a few lines, and launch it into production. Then they were surprised it didn't handle this or that situation or deal with this or that detail.

Yeah, no shit it doesn't handle those things, _it 's example code!_ You're supposed to read this code along with the documentation so you can get a gist of what the API is like. It's a learning aid, not a software deliverable. Your real code is going to be more complicated. This simplified code exists to get you past the "how the hell does all this fit together at a high level?" hurdle faster. Once you're over that hurdle, you can _start_ on the real implementation.

<comment><USER10> I wonder if people who complain about this have thought about what it would be like for beginners to only get to see production code. Sometimes the cow needs to be spherical.

<comment><USER11> This reminds me of that image of how to draw an owl. People forget that everyone starts somewhere and there is no magical jump from beginner level code to production code. Having good, well-explained examples of the beginner and intermediate were incredibly beneficial to me.

</comment></comment></comment><comment><USER12> I have spent a ridiculous time building this so I'll take the opportunity and share. It builds python wheel packages in a build container and installs them in an app container. Works great for cpython and pypy. Also allows to build for alpine and works for most other languages. We started to build basically everything that way.

[https://gist.github.com/tuco86/67d84dfb27268b1faf05d2dbb1acb...](https://gist.github.com/tuco86/67d84dfb27268b1faf05d2dbb1acb667)

Ok, I kind of cheated and added the user just now. Sue me. Also posted this in the other Docker related news. Sue me again.

<comment><USER13> Looks like the last line needs to be updated to have the server listen on 8080 instead of 80. (I'm guessing this is left over from before you added the non- root user.)

</comment></comment><comment><USER14> The problems described here are called 'release engineering.' Dockerfiles don't solve release engineering, they provide an abstraction for building a release candidate, putting it through a pipeline, and then tagging a successful build as your release. In other words, the end-container is the immutable object that should be deployed, not the Dockerfile.

If you are building the container in each stage of your CI/CD pipeline, you are doing it wrong.

</comment><comment><USER15> > A broken Docker image can lead to production outages, and building best- > practices images is a lot harder than it seems. So don't just copy the first > example you find on the web: do your research, and spend some time reading > about best practices.

While I may not agree with absolutely everything in the article, this final point is paramount. Please don't blindly use technology because you managed to find a copypasta config that runs. Running != good.

<comment><USER16> Definitely very true. I write more C++ than anything else, and the sheer number of online examples that start with

      using namespace std;

is just staggering. Sure, it works in a toy example posted to stackoverflow, but it will cause problems in larger projects. I think globally there needs to be better emphasis on using best-practices in tutorials and examples; I remember this particular pet-peeve of mine also being present in college textbooks. Especially for content aimed at newbies, it should be frowned upon to show the wrong way to do things, since then it gets harder to show how to do it the right way. I've had people who were surprised to find out that they could type:

      using std::chrono::duration;
      using std::cout;

instead of pulling in the entire std namespace; simply because they'd only ever seen examples that did it the lazy way.

edit: lack of semicolons strikes again!

<comment><USER17> While I agree with the general point of using best practices in code samples, the Cpp Core Guidelines actually encourage[0] using

      using namespace std;

for std specifically, giving the reasoning that:

> sometimes a namespace is so fundamental and prevalent in a code base, that > consistent qualification would be verbose and distracting.

I also work mainly in C++, and personally I prefer using it, together with -Wshadow to catch possible issues.

0: [https://github.com/isocpp/CppCoreGuidelines/blob/master/CppC...](https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#sf6-use- using-namespace-directives-for-transition-for-foundation-libraries-such-as- std-or-within-a-local-scope-only)

<comment><USER18> Problems arise when you upgrade your compiler and a new symbol was added to std::

</comment></comment></comment></comment><comment><USER7> For reproducible builds, `python:3.7` isn't specific enough. python:3.7.3-alpine3.9 is more specific, for example. There aren't supposed to be breaking changes in the bugfix releases, but they'll happen anyway.

<comment><USER19> And `python:3.7@sha256:35ff9f44818f8850f1d318aa69c2e7ba61d85e3b93283078c10e56e7d864c183` is even better.

</comment></comment><comment><USER20> This Dockerfile linter would warn you about multiple of these problems and more: [https://github.com/hadolint/hadolint](https://github.com/hadolint/hadolint)

</comment><comment><USER21> I am actually preparing my own article titled "Docker antipatterns" that will include many more points like this.

</comment><comment><USER22> "Broken" means "does not work". These examples _do_ work. I'm annoyed by this incongruity. "Not sustainable"/"Not Forward Compatible", etc. would have been preferable.

</comment><comment><USER23> This is an advertisement disguised as a technical post on container security.

<comment><USER24> Then I wish all advertising were like this. It's very informative and provides a solution rather than just pointing out the problem. I hope this page ends up ranking highly in search results because there are a lot of incomplete Dockerfiles employing questionable practices that sit at the top of search results and proliferate due to cargo culting.

</comment></comment><comment><USER25> I feel like the better solution to #4 is setting up UID namespacing for docker instead of (just) creating random users within the container. Even if you create a user, it's still going to run as whatever UID it has within the container (probably 1000 is most likely your UID if you're the only one using said system)

</comment><comment><USER26> Nice eBomb. Describes the problem, offers value, does it politely so it is HN (and other places) friendly, then "there are still problems..." and then the paid solution.

I've seen docker images that do a git clone from the master head to get the source, so basically if their Github account gets hacked. You're f'd.

</comment><comment><USER27> It seems like the folks making docker files could stand to learn package management with a mature package system before making docker files.

I wonder how many use docker after they learn ?

</comment><comment><USER28> why is running as root in the docker a problem? Isn't the whole point of containers to isolate the container? So what is the difference in a container running root or a user? If there is, wouldn't that be more of a docker bug?

<comment><USER29> The page the article is linking to [http://canihaznonprivilegedcontainers.info](http://canihaznonprivilegedcontainers.info) is mixing up running as root with running docker with --privileged. Latter one renders Docker security to zero, but is barely required.

I'm not saying "go run your all your Docker images as root", but this is clearly FUD.

Non-privileged containers are still having "root", just with way fewer capabilities (See Docker [0] docs).

I'm not an expert, but I guess depending what you are doing the most problematic capability might be AUDIT_WRITE, because it is not namespaced and could be abused for DOSing syslog. But you might require it for things like sshd, sudo, adduser, passwd, …

Depending on how you are holding it the NET_BIND_SERVICE and NET_RAW can be an issue (depends on how your docker network looks like), but the others appear not to be a security issue per-se.

This page [1] gives a good overview on default capabilities, though they are also confusing to the reader with "better disable this".

I've created an issue, not sure if I have resources to fix their page though. [2]

[0]: [https://docs.docker.com/engine/reference/run/#runtime- privil...](https://docs.docker.com/engine/reference/run/#runtime-privilege- and-linux-capabilities)

[1]: [https://www.redhat.com/en/blog/secure-your-containers-one- we...](https://www.redhat.com/en/blog/secure-your-containers-one-weird-trick)

[2]: [https://github.com/mhausenblas/canihaznonprivilegedcontainer...](https://github.com/mhausenblas/canihaznonprivilegedcontainers.info/issues/8)

<comment><AUTHOR> Non-privileged containers running as root are a definite security risk.

Real world example: CVE from February 2019 which allowed escalation to root on host. It's preventable by (among other things) "a low privileged user inside the container".

See [https://blog.dragonsector.pl/2019/02/cve-2019-5736-escape- fr...](https://blog.dragonsector.pl/2019/02/cve-2019-5736-escape-from-docker- and.html)

<comment><USER29> (Same discussion as on lobste.rs)

Thank you for this link, I've only seen the initial CVE announcement.

> This is not FUD [...]

The site is practicing FUD, it accomplishes communicating a message in an untruthful fashion by mixing two different things into one. (Just check out their stack overflow links, it is not clear if they are talking about root or `--privileged`)

People are confused wether `docker root == host system root` and this site doesn't help them to get a better understanding whether or not it is the case. (It isn't) Plus it misses what its main goal should be, running a Secure Docker environment.

You are talking about a previous exploit, not a permanent issue. Keeping your host system up to date and additional hardenings is always going to be necessary in exposed environments.

> Use Docker containers with SELinux enabled (--selinux-enabled). This > prevents processes inside the container from overwriting the host docker- > runc binary.

Authors recommendation is also using SELinux, this also helped with outer Docker/Kernel related vulnerabilities in the past. Why isn't the page even mentioning this?

\---

I think it is important to give a proper outlook on how problematic things are and not to confuse people with super high expectations. You often end up running containers that you have only little control about.

1\. Avoiding root in self-built containers is definitively the way to go, since it reduces (unnecessary) attack surface, but

       1. It requires some glue code

       2. Might slow down your builds (`Dockerfile` multistage `cp --from=0 /app /app` loses permissions, requires chown afterwards)

2\. Avoiding root in CI/CD is nearly impossible 1\. many package managers won't work

       2. some capabilities to test things (sshd for testing ansible scripts for example)

       3. can you use kaniko for building Docker images from within Docker without root?

3\. Harden your Docker host

       1. Use SELinux

       2. Use monitoring

       3. Drop capabilities that aren't necessary (NET_BIND_SERVICE, NET_RAW, ...)

       4. Use docker network separation

       5. Frequent system updates

4\. Keep yourself up-to-date, especially if you are running an exposed environment

Just googled and this is rather more helpful:

\- [https://dev.to/petermbenjamin/docker-security-best- practices...](https://dev.to/petermbenjamin/docker-security-best- practices-45ih)

\- [https://blog.aquasec.com/docker-security-best- practices](https://blog.aquasec.com/docker-security-best-practices)

\- [https://sysdig.com/blog/7-docker-security- vulnerabilities/](https://sysdig.com/blog/7-docker-security-vulnerabilities/)

\- [https://github.com/docker/docker-bench- security](https://github.com/docker/docker-bench-security)

</comment></comment></comment></comment><comment><USER30> I do not intend to play down the importance of using docker carefully.

But the reproducible build aspect of the critic seems unnecessary to me: Isn't that more a concern of the packaging system? (no python scripter)

If your packaging systems supports version selection/locking, then use your packaging system right. If your packaging system cannot pin a version, how should docker solve this?

<comment><USER31> Docker can't escape all the blame here - its layer caching mechanism is IMHO flawed. It's fine to say that a packaging system should offer reproducibility but Docker's layer caching design assumes that every RUN command produces reproducible results.

You could of course blame users for not making sure that all the commands they use in their Dockerfiles are actually reproducible but many/most examples even in the official documention are clearly not reproducible.

Therefore you end up with what is in my opinion a semi-broken system - building images seems to be reproducible (and fast) until you lose your layer cache or you spin up a new CI build agent or a new dev joins the team and tries to build the same image.

Not that I can think of an clean and performant solution to this problem.

<comment><USER32> We have been working on a simplified container build system which does away with layers altogether. [1]

The use of layers at the build stage adds a lot of needless complexity with very little benefits and users really need to step back and question the value they are getting from the use of layers. [2]

Words like 'immutability', 'declarative' and 'reproduciblity' are often used in ways that can lead to user misunderstanding and can be accomplished with simpler workflows. For instance immutability, reuse, composition do not require layers. There needs to be a lot more technical scrutiny to avoid confusion.

[1] [https://www.flockport.com/docs/containers#builds](https://www.flockport.com/docs/containers#builds)

[2] [https://www.flockport.com/guides/say-yes-to- containers.html](https://www.flockport.com/guides/say-yes-to-containers.html)

</comment></comment></comment><comment><USER33> I would move the requirements.txt and pip install to after the user creation seeing as you'll invalidate that cache if your requirements change.

Best part is that was brought up as an issue in the article only to do the same thing in an example

</comment><comment><USER34> Running pip with sudo doesn't seem a great idea either...

</comment><comment><USER35> (1) and (2) aren't really broken, IMHO. For most cases always using the most up to date version is better than having 100% reproducible builds. After all, you have the docker image that you can distribute if you really need to. Better to pick up security and performance patches as they become available. If those updates break something then you can make the decision to fix on a known good version.

<comment><USER36> If you always pin, you have history to tell you which versions were good. If you mostly don't, you have to start disassembling a bunch of old images just to figure out what they were built from.

</comment></comment><comment> jo-wol The final example in the article is broken. Python interpreter as PID 1 can't handle linux signals.

<comment><AUTHOR> This is why I have a caveat at the top of the article as well as right after the last example. This particular issue is fixable with `docker run --init`, so not strictly necessary to fix in images.

</comment></comment><comment><USER37> not everyone is on a upgrade-daily churn, and should not have to be ! if you are externally exposed, sure, because security .. but really, isn't there some room here for different life cycles ?

</comment><comment><USER38> In general, in Go, Java, and Python I've resorted to copying in the Gopkg files, pom.xml, and requirements.txt, and then running the requisite dependency installer for the language (dep, pip, mvn, etc...) and then just copying in the rest of the repo, relying on the .dockeringore with a default- ignore for everything and specifying the individual files/directories you may want to add, and in some cases a rootfs folder when necessary.

This seems to be the happy medium for me. I don't have very strong opinions on requirements.txt always being the pinned output from a pip freeze, and it seems like pipenv may actually die in a few years, and poetry will evolve to take the mantle, but I do lots of things with conda anyway.

</comment><comment><USER39> Isn't it ironic that he isn't pining down the docker version?

</comment><comment><USER40> Isn't the example on how to make the referenced file better another contribution to the pool of "broken by default" images? Either that or I don't get the argument.

</comment><comment><USER41> Not locking it to a specific version is better for security updates. Do you want it to run stable with vulnerabilities or to run secure and broken?

<comment><USER42> > Not locking it to a specific version is better for security updates.

The idea is that you should take responsibility for your containers and verify fixes and test your application.

> Do you want it to run stable with vulnerabilities or to run secure and > broken?

If these are your two choices, you have a staffing or a workflow problem.

</comment></comment><comment><USER43> Probably going to want to use tagged docker repos so that updating certain packages, no matter the language, don't suddenly break your images

</comment><comment><USER44> One more broken part is

        CMD [ "python", "./yourscript.py" ]

This breaks if you want to debug yourscript.py on startup. Better use a sh to wrap it.

<comment><USER45> Why/what do you think this breaks / what does wrapping it in a shell do for you?

E.g., for me, the following Dockerfile:

      FROM python:3

      RUN pip3 install ipdb

      COPY test.py /test.py
      CMD ["python", "test.py"]

where test.py is:

      import ipdb; ipdb.set_trace()
      print('Hello, World.')

run as `docker run -ti --rm $IMAGE_ID` works as expected:

      » docker run -ti --rm 52e98c118dc3
      > /test.py(2)<module>()
            1 import ipdb; ipdb.set_trace()
      ----> 2 print('Hello, World.')

      ipdb> p globals()
      {'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x7f809a8c8278>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, '__file__': 'test.py', '__cached__': None, 'ipdb': <module 'ipdb' from '/usr/local/lib/python3.7/site-packages/ipdb/__init__.py'>}
      ipdb> ^D

      Exiting Debugger.

      »

<comment><USER44> Can you add extra environ before python executes any code?

<comment><USER46> Did you even try ?

`docker exec -e foo=bar -it ....`

</comment></comment></comment></comment><comment><USER47> Very good clues!</comment>


Airbnb Releases Infinity.js: A Javascript Library for Smoother Scrolling - <AUTHOR> http://airbnb.github.com/infinity/ <comment><USER2> OT - I can't help but mentally shorten code snippets with underscore/whatever. For example, I see this -

for(var index = 0, length = listItems.length; index < length; index++) { listItems[index].remove(); }

and I think to myself -

_(listItems).invoke('remove')

<comment><USER3> that's funny, I'm the other way around.. whenever I see syntactic sugar implemented via javascript, I unroll it in my head and consider it overhead, compared to sending a bunch of more bytes down the wire which hardly matter after gzip.

<comment><USER4> TIL javascript functions are syntactic sugar.

<comment><USER3> And TIL stands for what, exactly? Sure you would not simply equate "syntactic sugar" with "syntactic sugar implemented in javascript".. so what are "TIL javascrip functions"? I tried searching, but no luck, woth none of the permutations. There are too many acronyms for TIL to just blindly guess.

My point basically was that

 __(listItems).invoke('remove')_

is actually short for

* [content of <http://underscorejs.org/underscore-min.js>] _(listItems).invoke('remove')*

and even if you removed all non-called parts from underscore, that's still a lot more complex than

 _for(var index = 0, length = listItems.length; index < length; index++) { listItems[index].remove(); }_

it's just hidden, out of sight and out of mind.

<comment><USER5> TIL stands for Today I Learned.

They were attempting to use a meme acronym sarcastically -- an example of content-free commenting that works well for karma elsewhere but hopefully doesn't gain a foothold here.

On topic, I agree with your amplification of your point.

<comment><USER6> If you use memes on HN, you're going to have a bad time.

(They tend to be downvoted to oblivion, 'round these parts.)

</comment></comment></comment></comment></comment></comment><comment><USER7> I tested infinite scrolling on a large site about a year ago and the results were decidedly negative in almost every significant respect according to our A/B testing. Does anybody actually have evidence that infinite scrolling is a good thing?

<comment><USER8> HN likes to pretend that infinite scrolling has no utility, but it's simply not true. For sites where you want to quickly scan a lot of content, infinite scrolling is vastly superior to static pagination. For sites where you will typically consume every item in the list carefully and want to refer back to specific points in the list, pagination is vastly superior to infinite scrolling. They both have their benefits.

<comment><USER9> It's a good feature but it is often done so horribly it becomes not liked. Rule number one: If you use infinite scrolling instead of pages, don't remove pages. Especially on pages where content is somehow ordered (shops, lists, ..).

<comment><USER8> I agree it should be done well, but I don't agree you _always_ need pages. For instance, when I use google images, I'm usually just looking for a few specific images in a sea of pictures and I don't care too much about keeping my place in that list. I care a lot more about being able to quickly scan through a very large grid of images, though, which infinite scrolling allows me to do.

<comment><USER10> The one caveat here is, if you're going to use infinite scrolling. Don't have links on the bottom of the page!

Sites like Facebook do this, and I have no idea why.

Want to "Create a page". Well there's a link at the bottom. But if you scroll to the bottom of the page, it's going to load more content and push the link down.

I just tried it, and the page reloaded 15 times before I gave up... never did get to the link.

<comment><USER11> On the right sidebar, under "Facebook (c) 2012" should be duplicates of all of the links at the bottom of the page. For the "Create a page" link, you have to click "More".

This is clearly unintuitive, but at least it exists.

</comment></comment></comment></comment></comment></comment><comment><USER12> Loading their demo linked from the word "on" froze my browser for about 20 seconds. Scrolling the page afterward was horrible, very jerky and laggy.

I have noscript installed, but all scripts on that page were allowed.

<comment><USER13> Same here, but I wouldn't expect much from a jQuery plugin that implements "infinite scroll". Like I need stuff to start loading by itself whenever I reach the bottom of the page.

Also, looking at their code, the first thing I laid eyes on was this statement:

      var infinity = window.infinity = {};

Which, in this context, would be the exact same as

      var infinity = {};

<comment> brown9-2 I believe they are doing that to override any functions/objects already bound to _window.infinity_ from other plugins/scripts. Look at the _infinity.noConflict()_ function at the bottom of the source, which seems to be inspired by the _jQuery.noConflict()_.

<comment><USER13> It doesn't matter. Both snippets do the exact same thing, except the first has two attributions, the second just one.

</comment></comment></comment></comment><comment><USER14> Note to all developers planning on implementing infinite scrolling:

Don't have a footer. Example of this error: LinkedIn. Try to get to their ToS or Help Center from the logged in home page. Good luck.

<comment><USER15> It can, it just needs to be position as fixed.

<comment><USER14> I stand corrected. I personally like what Facebook now does (they used to have the same problem). They put the same information in a tile at the bottom of their right pane, which is fixed.

</comment></comment></comment><comment><USER16> I tried both off and on demos, and all I got was a huge page loading slowly with scrolling causing flickering and stuttering. This is on Linux/Chrome. Maybe people shouldn't try to create large, image heavy pages in the first place, not to mention make them infinitely long.

<comment><USER17> Yeah, Mac/FireFox here and the same experience. both the on and off demos gave me a spinning beach ball for a minute while what seemed like the entirety of the content loaded. I thought that's what infinite scrolling was supposed to prevent?

</comment></comment><comment><USER18> UITableView was made for mobile devices not for infinite scrolling per se, but simply for large lists (such as address books) due to the memory limitations of iOS devices.

Those memory limitations are simply nonexistent for the web for almost all but the smallest number of cases (ie, infinite lists).

But lets pretend that this infinite scrolling is even a real problem (honestly it's probably not the best UI for large infinite lists of things on the web).

The current implementation clearly needs some work. It's trying to over- optimize for memory to the point of choking the scrolling.

This, however, can be fixed fairly trivially.

Allow the user to scroll anywhere (even past loaded content). If you can't eagerload elements to compensate for it, then lazyload after the user gets to that point. This gives users the illusion that the scrolling is truly smooth and responsive. The important thing is never allow scrolling to stop unless it's truly at a dead end and there is absolutely no more elements to load.

The above is how UITableView functions. Infinity.js is UITableView for loading elements, but does not function anything like UITableView from a user interaction standpoint.

</comment><comment><USER19> ...I don't think I understand the utility of this. Scrolling is already pretty smooth. (unless I'm wrong)

<comment><AUTHOR> Author here. The utility is just to improve scroll smoothness (aka repaint times) of very-expensive-to-render pages, like infinite feeds with nice CSS effects. If you check out the demos linked to on the page, the demo with Infinity turned off should be noticeably choppier than the demo with it on, even once it's completely loaded and the browser's stopped reflowing. Probably depends on your machine/OS/browser combination -- but it'll keep loading more pugs as you scroll down, and eventually the version with Infinity off will grind just about anything to a halt.

<comment><USER20> By repaint times, do you mean re-layout times? ie. time required for CSS/HTML/JS to determine _where to paint to_?

The demo is not any smoother on my 27" Apple display driven by a late 2010 model MacBook air. However, even low res hardware accelerated video renders choppy at large window sizes (although perf gets better when VLC or QuickTime is true fullscreen; presumably the compositor is off)

Faster JavaScript is unlikely to be effective at enhancing buffer flip times, which (for me) dominate repaint times.

<comment><AUTHOR> Both repaints and relayouts are improved, actually. The faster repaints are what improves scrolling performance (repaints are triggered every time you scroll, but relayouts are not) -- but because there are fewer elements in the DOM tree, relayouts are cheaper as well.

We experimented with using display:none instead of removing the elements; it increased scrolling performance just as much as actual element removal, but the relayouts when new content was appended got to be prohibitively expensive once a certain amount of content was loaded (literally pausing the browser for a noticeable amount of time), because the browser still had to traverse a giant tree of HTML entities during the relayout. So even if the buffer flip times are still too slow for you to get completely smooth scrolling in complex demos like this -- and that makes sense, given your large-screen setup -- you'll hopefully still benefit some from the faster, non-browser-freezing relayouts.

<comment><USER5> _Both repaints and relayouts are improved, actually._

When _actually_ is defined as _for some on certain machines_ , judging from the number of directly contradictory observations in this thread.

The word "actually" ought to be reserved for concrete facts with zero counter examples.

The word "hopefully" in your closing sentence seems more apt.

That said, very much appreciate the sharing of the hypothesis and implementation. It's great to see ideas from other areas (video game terrain feature pop in) leveraged to improve end users' browsing experience.

</comment></comment></comment></comment></comment><comment><USER21> Hm, it doesn't degrade very well without javascript on (pagination would be a neat fallback). I know it's not generally worthwhile to cater to such a limited user population, but what about search engines? Should this only be used for content you don't need indexed, or you have indexed a different way?

Or, since I think Google uses javascript in its crawlers, I wonder how it handles it. Presumably it would just keep scrolling until the content ran out or it gave up. But does it ever come back later? I know with pagination it could pick up on the links it left off on, but couldn't really do that here, right?

edit: I guess this is really a question of infinite scrolling more than this particular implementation. Kudos to Dropbox for releasing a well-done version of the technique.

</comment><comment><USER22> If this only works on jQuery objects, why not implement this as jQuery plugin?

      jQuery.fn.infinity = function (options) {...};
      $(el).infinity(options);

And so on.

<comment><USER23> I think it might work with other methods of selecting DOM elements. At least that's what the API suggests.

<comment><USER22> Maybe I'm not reading it right, but it says it depends on jQuery and accepts selectors or jQuery objects, not DOM elements.

</comment></comment></comment><comment><USER24> OP: Maybe you should change the links [on]/[off] to [demo on]/ [demo off].

</comment><comment><USER25> Thank you for using sane js code style.

</comment><comment><USER26> If you try to move really fast with Infinity on, the screen turns completely white for a split second: <http://airbnb.github.com/infinity/demo-on.html>

<comment><AUTHOR> True fact. That's just due to scrolling fast enough to get outside the buffer before the throttled scroll event fires -- if you want, you can turn the scroll throttle down to make sure that doesn't happen, at the expense of doing slightly more computation while scrolling. For Wish Lists, we optimize for people browsing through the feeds, which is generally not super fast. :)

<comment><USER27> Its much faster for me as well.

In order to calculate the size of each ListItem you add it to the dom briefly when its appended to the ListView. This seems to be a fast operation since you do it hundreds of times yet the page is loading quickly. Is there a trick there that is avoiding having to calculate a ton of layout, or is that just not a big concern compared to other work the browser is doing?

Thanks!

<comment><AUTHOR> Just one trick, although I use it in both the on and off demos to keep everything fair: I'm adding elements in smaller batches of 70 rows per tick of execution (arbitrary number), and then using a setTimeout to wait a few milliseconds before adding more. The demo loads a tooon of elements upfront -- much more than an ordinary site would -- and Firefox would sometimes complain if I loaded 400 rows of pugs at a single time. Chrome, of course, had no complaints, and IE was surprisingly fine as well.

<comment><USER27> And from the comment above:

"...but because there are fewer elements in the DOM tree, relayouts are cheaper as well"

If I understand, this also applies? When I add the 400th (or whatever) ListItem it does trigger a layout when its added to the dom (to calculate its dimensions) but its much faster due to only having the currently viewable ListItems contained in the ListView?

Thanks again!

</comment></comment></comment></comment></comment><comment><USER28> We (at our company) don't use pagination anywhere now. But we do have extensive search abilities. We have found, very few of our users just scroll. They use the search. But the "infinite" scrolling is there if they want it.

</comment><comment><USER29> Cool! So basically it's a lazy load

</comment><comment><USER30> Android 4.1.1 with or without Infinity: "The page has become unresponsive. Wait or kill?"

</comment><comment><USER31> Both demos are the same for me. Chrome, Windows 7 64bit.

</comment><comment><USER32> Awesome! This is going to help so much.

</comment><comment><USER33> I think Lue Links was doing this for several years. I'm glad to finally have a library for it.

</comment><comment><USER34> why it's based on jQuery?

<comment><USER35> Because re-inventing the wheel is _soooo_ 20th century.

<comment><USER36> what if I don't use jQuery in my project?

<comment><USER2> Just for you, I went through their source, and these are all the jquery specific calls -

.height() .width() .offset() .remove() .append() .prepend() .prependTo() .find() .parent() .on() .off() .scrollTop() .outerHeight() .length

It seems to me like all of these would be covered by zepto. Or qwery. Or you could make your own adapter with Prototype. Or Mootools. Or Ext. Or YUI. Heck, you could probably whip up your own little library based on sizzle, or even just querySelectorAll. Then just fork infinity to use your lib, and you're good to go.

I hope you're not confused any more!

[I'm not affiliated with airbnb, just thought I'd help you out]

<comment><USER36> ok, sounds like a lot of boilerplate code to write, because "Its _only_ dependency is on jQuery."

<comment><USER2> <script src="zepto.js"></script> <script> var jQuery = Zepto; </script> <script src="infinity.js"></script>

There.

</comment></comment></comment></comment></comment></comment><comment><USER37> off version actually worked better for me</comment>


Simple Apache Form Authentication: mod_auth_cookie - <AUTHOR>

This module seems to have fallen by the wayside: The previous homepage has gone dark, and I was luckily able to grab the source from archive.org.
It's a really great way to set up a simple login form and get rid of the ugly Basic Authentication prompt. You still use Basic Authentication (including a .htpasswd file or mod_auth_mysql, for example), but redirect using a custom 401 handler that shows a login form, that in turn sets the user:password combination as a cookie.
On the server, mod_auth_cookie then takes that cookie and fakes the Basic Authentication headers. It's literally so simple, it can be configured with one line on top of Basic Auth:
AuthCookieName CookieName
I've resurrected the module from the abyss, and hosted a shiny new homepage at http://modauthcookie.weebly.com/ <comment><USER2> For the curious, I've had the recent misfortune of working with JBOSS at work; this sounds identical to how it provides form-based authentication.

For the curious: [http://wiki.jboss.org/wiki/Wiki.jsp?page=SecureAWebApplicati...](http://wiki.jboss.org/wiki/Wiki.jsp?page=SecureAWebApplicationUsingACustomForm)

</comment><comment><USER3> Did it fall out of grace because it was vulnerable to cross-site attacks ? ( <http://en.wikipedia.org/wiki/Cross-site_request_forgery> )

<comment><AUTHOR> i believe it fell out of grace because the maintainer's page disappeared. and no, it shouldn't be any more vulnerable to CSRF than a simple session-cookie system, which is what this basically is.

</comment></comment><comment><USER4> Very cool, I will probably use this at some point soon.

IMO, this is a big part of what makes open source so valuable.</comment>


Google engineer ends push for crypto-only setting in Allo - <AUTHOR> http://arstechnica.com/security/2016/05/incensing-critics-google-engineer-ends-push-for-crypto-only-setting-in-allo/ <comment><USER2> It's very dishertening to see people get pushed around by their employers loke this.

Does censoring the opinions of your employees not count as evil?</comment>


Wharton Study Shows Shocking Result When Women, Minorities Email Professors - <AUTHOR> http://www.policymic.com/articles/88731/wharton-study-shows-the-shocking-result-when-women-and-minorities-email-their-professors

<comment><USER2> question : could this be the result of spambot detection criteria having a bias towards one race/gender or another?

as a video game player with a leaked email address due to the poor actions of video game companies, I routinely receive spam from gold-farmers with Asian names in the headers, or female names offering me pills or pornography..I hastily indicate that their spam.

although I wouldn't for a second doubt the existence of a real-life bias, I'd be curious if that kind of thing had been accounted for. I didn't notice it mentioning it in the article linked. I'd hate to attribute a spambot bias to the staff of the schools.</comment>


MediaMath's ground-breaking new paper on causal ad-effectiveness measurement - <AUTHOR> https://arxiv.org/abs/1705.00634
Abstract: The problem of measuring the true incremental effectiveness of a digital advertising campaign is of increasing importance to marketers. With a large and increasing percentage of digital advertising delivered via Demand-Side-Platforms (DSPs) executing campaigns via Real-Time-Bidding (RTB) auctions and programmatic approaches, a measurement solution that satisfies both advertiser concerns and the constraints of a DSP is of particular interest.
MediaMath (a DSP) has developed the first practical, statistically sound randomization-based methodology for causal ad effectiveness (or Ad Lift) measurement by a DSP (or similar digital advertising execution system that may not have full control over the advertising transaction mechanisms). We describe our solution and establish its soundness within the causal framework of counterfactuals and potential outcomes, and present a Gibbs-sampling procedure for estimating confidence intervals around the estimated Ad Lift. We also address practical complications (unique to the digital advertising setting) that stem from the fact that digital advertising is targeted and measured via identifiers (e.g., cookies, mobile advertising IDs) that may not be stable over time. One such complication is the repeated occurrence of identifiers, leading to interference among observations. Another is due to the possibility of multiple identifiers being associated with the same consumer, leading to "contamination" with some of their identifiers being assigned to the Treatment group and others to the Control group. Complications such as these have severely impaired previous efforts to derive accurate measurements of lift in practice.
In contrast to a few other papers on the subject, this paper has an expository aim as well, and provides a rigorous, self-contained, and readily-implementable treatment of all relevant concepts. <comment><USER2> An interesting article. On Hacker News, using the article title is preferred. _Counterfactual-based Incrementality Measurement in a Digital Ad-Buying Platform_ is more informative.

To submit a clickable link using the "submit" page, leave the the |text| box empty. After clicking the |submit| button, you will be redirected to the |new| page and will be able to view the comments page and add comments.

<comment><AUTHOR> Thanks for the advice, it was my first post :)</comment></comment>


RIAA foiled by "innocent infringement" defense - <AUTHOR> http://recordingindustryvspeople.blogspot.com/2008/08/innocent-infringement-defense-may.html

<comment><USER2> $200 per song instead of $750 doesn't really count as "foiled", if you ask me.

</comment><comment><USER3> How do these payments work, anyway? Say this kid has to pay for all the songs. She goes into debt, defaults on the loan, and all is forgiven; except now my savings account has a lower interest rate. What's the point?

<comment><USER4> They will probably never even bother to try collecting on the judgement. The girl has no obligation to get a loan, it is up to RIAA to collect it against a person who has little income and very likely no assets. They won't even try.

This was all to build up rock solid precident they can use in other courts but it didn't quite go to plan. So to RIAA,this is a huge failure.

</comment></comment><comment><USER5> I hate the RIAA as much as the next guy, but I have trouble buying that a 16 year old didn't understand copyright infringement.

That's a Junior in HS!</comment>


Why we don't use a CDN: A story about SPDY and SSL (2014) - <AUTHOR> https://thethemefoundry.com/blog/why-we-dont-use-a-cdn-spdy-ssl/ <comment><USER2> Is anybody else disgusted that 1.6 _seconds_ is considered an acceptable page load time? When did this happen? Whatever happened to 200ms being the benchmark?

Dear Lord, people, please stop loading 14 zillion different domains every time I hit a web page.

<comment><USER3> If you can tell me how to make a decent looking site with a few MB of images load in 200 ms, I'm all ears. The only method I can think of, is by making the website ugly (i.e. few images and other design assets).

<comment><USER2> 1) Put your assets on the same domain.

Did you see how much time was being used in multiple DNS requests?

To an end user, everything not on your domain is _CRAP_. It's not there for me, the end user. It's there for _you_ , the website owner so you can aggregate my eyeballs, pitch me something, use somebody's comment service, extract advertising revenue, push network bandwidth onto Google, jQuery, etc.

2) Why on Earth do you need megabytes of images?

Really? WTF? Just because you took the picture with a gigapixel camera doesn't mean you need to serve every pixel.

3) Minimize the Javascript

No, you _don 't_ need that Javascript framework. Shoot it.

Especially on tablet/mobile browsers, a very small number of pages cache before the browser has to reload and regenerate (and that's _REALLY_ slow).

The research is against you. Every extra 100ms is a significant drop in retention rate. Your website will stick out because everything is so snappy. Users can _feel_ this--especially on tablets and mobile.

<comment><USER4> Agree with the above comments about reducing bloat, and I'm surprised by the number of smart people who think nothing of setting up a Wordpress site with dozens of CSS and JS files.

I just spent 20 minutes trying to find a link to an Apache function I found recently, which will join all your CSS or JS files together with something like an @include server-side automatically. Can't find it now but as I have been doing this manually I hope I find it again. Anyone know what this is? I seem to recall it was a native Apache function, not part of mod_pagespeed.

<comment><USER5> Do you mean modconcat? [0] There is also a nginx version by Alibaba available. [1]

[0] [https://code.google.com/p/modconcat/](https://code.google.com/p/modconcat/)

[1] [https://github.com/alibaba/nginx-http- concat](https://github.com/alibaba/nginx-http-concat)

<comment><USER4> Nah, found that while searching for the other. I can probably use that, though. The one I'm thinking of had you putting a short instruction at the top of the file (like @include) if you wanted it to be part of the output file. Maybe I'm not remembering this correctly as I can find nothing on it now.

<comment><USER6> Server Side Includes? (mod_include for apache, mod_ssi for some others)? Not sure I'd recommend that as a general solution today -- for static sites I think you're better of building the static html, and serve that up. With varnish, you might want to look at edge-side includes.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> The big mistake here is using a split host name CDN (instead of an edge/origin CDN), and then putting the primary CSS on the split host name.

That means you have to wait for DNS resolution and the TCP connection roundtrips (or worse, unoptimised TLS roundtrips) just to get the CSS.

If they were on the same host name, and the CSS is the first thing in the HTML, it'll be downloaded immediately after the HTML.

(And with SPDY, the browser could start downloading the CSS on the same connection as soon as it parses the link element.)

<comment><USER8> Yeh would be interesting so what performance would be like with the origin served through the CDN or even using dns-prefetch or tcp-preconnect

I think all we can gather from this example is that MaxCDNs TLS implementation wasn't optimal and they were working on it

Would be interesting to see how this behaves for other CDNs or with link rel= optimisations in place

</comment></comment><comment><USER9> We optimise and host websites for our clients, use an https/spdy CDN exclusively, most sites we work on are dogs when it comes to performance, and aim for half second page load times on average. Anything over 1 second is unacceptable. I read this article before, I think it was posted to HN previously, and wasn't impressed at the time. Still not impressed now....

Long story short: Not using a CDN is stupid. Learn to CDN.

</comment><comment><USER10> See also "Using CloudFlare to mix domain sharding and SPDY": [https://blog.cloudflare.com/using-cloudflare-to-mix- domain-s...](https://blog.cloudflare.com/using-cloudflare-to-mix-domain- sharding-and-spdy/)

</comment><comment><USER11> Rather than copy their suggestions for SSL configuration, I think you can probably find better (and more well-maintained) advice on the Mozilla wiki:

[https://wiki.mozilla.org/Security/Server_Side_TLS](https://wiki.mozilla.org/Security/Server_Side_TLS)

</comment><comment><USER12> [http://www.webpagetest.org/result/150402_TV_f0fe942e49dc1191...](http://www.webpagetest.org/result/150402_TV_f0fe942e49dc1191e818c0cecba48d76/) shows you take 8 seconds to load in Australia. Using

[http://www.webpagetest.org/result/150402_22_0fa16aec7ebc410a...](http://www.webpagetest.org/result/150402_22_0fa16aec7ebc410a00737efcd1321d3e/) also shows 8 seconds from tokyo

[http://www.webpagetest.org/result/150402_AN_22f88770d4bab89c...](http://www.webpagetest.org/result/150402_AN_22f88770d4bab89c84bd986d75da50ae/) shows 8 seconds from India

[http://www.webpagetest.org/result/150402_CT_49277f5267e82b6b...](http://www.webpagetest.org/result/150402_CT_49277f5267e82b6b75a5f2bc9a3cf589/) shows 36 seconds from Amsetrdam

[http://www.webpagetest.org/result/150402_9G_0810eaa940dc2941...](http://www.webpagetest.org/result/150402_9G_0810eaa940dc2941d0ff83615913e959/) shows 12 seconds from Buenos Aires

Not so sure that you dont need a CDN based on this

</comment><comment><USER13> My experience: [https://medium.com/p/196b5024899c](https://medium.com/p/196b5024899c) Major browsers will not support h2c switching(switching from http, rather than TLS), and faster(but less secure) ciphers or null ciphers will not work for those browsers either, so speed in the HTTP/2 world will likely be something on top of SSL's latency.

</comment><comment><USER14> In the Australia comparison, the CDN version spends almost 1.5 second on DNS lookup for the main domain. In the non-CDN version this figure is much lower. It seems disingenuous to attribute this overhead to the CDN, no?

And as an aside - Maybe there are some low hanging fruits in tuning their DNS setup instead? 1.5s seems extremely high to me.

</comment><comment><USER15> Why we don't use a CDN: NIH.

<comment><USER16> In this case, that's an appropriate attitude. If your environment is a mess and you add complexity with a CDN, you've got a complicated mess. But a local optimization like this benefits your entire organization and puts off the need for a CDN (a solution to a problem that no longer exists). They may grow to a point where a real strain is being put on their systems, they can't make local improvements, and involving a CDN is the right choice.</comment></comment>


Vue CLI 3: A Game Changer for Front End Development? - <AUTHOR> https://vuejsdevelopers.com/2018/03/26/vue-cli-3/ <comment><USER2> A game changer would a Delphi like RAD tooling IDE, fully based on WebComponents, with backend integration and an eco-system of companies selling UI components, not yet another CLI tool.

<comment><USER3> Have you ever tried Adobes Flex Builder?

That was good.

Only drawback .. it was based on flash and got killed by Adobe.

<comment><USER4> Flex was better than anything in use for web front end to this day, imo. Aside from requiring Flash, but iirc targeting HTML/js as an output format was on the roadmap.

<comment><USER5> We have Apache Royale [https://royale.apache.org/](https://royale.apache.org/) for that. Compile your Flex MXML + ActionScript to HTML + JS and more.

</comment></comment></comment></comment><comment><USER6> Am I the only one who thinks Webpack is really not _that_ bad? I spent a day or two learning it back in 2015 and it's served me well ever since.

Not to mention tools like this end up increasing the net complexity, not reducing it. I recently looked at forking create-react-app to get it working in a custom environment, and wow. What a mess.

<comment><USER7> I think the main issue is the inconsistency in tutorials out there and the different ways of setting up webpack and using all the loaders. For example you can use loaders and specify them in config, or directly in an import/require statement.

<comment><USER6> If that's the case I think FB could've served the community better by just publishing recommended Webpack configs than cargo-culting everyone into CRA.

</comment></comment></comment><comment><USER8> Yo dawg, I heard you like configuration so I put Babel config in your Webpack config in your Vue3-CLI config so you can configure while you configure.

In all seriousness, I'm not seeing how yet another build system with a plugin architecture changes anything for frontend development. I'm also not seeing what this extra layer offers over using Webpack directly.

<comment><USER9> Welcome to DJS (Department of JavaScript), please take a number, stand in the line to your right, and migrate your code to the next coolest framework.

<comment><USER10> Phtt. Migrate? That's crazy talk. Rewrite!

On a more serious note: that's why I exited frontend dev 15 years ago and never looked back. I now do hard-ass C and C++, and once I write something it runs for many years with no changes, and dependencies are often a decade old or more.

</comment></comment></comment><comment><USER11> "You may be relieved when you install your first Vue CLI 3 project and see there is no webpack.config.js in the project root. This is because most project configuration for Vue CLI 3 is abstracted into plugins and is merged into the base configuration at runtime."

Well, I guess have fun untangling that knot whenever there's a problem with Webpack! We all know how easy and fun the last Webpack major version upgrade was to all the things out there designed to reduce the amount of interaction with it when bootstrapping a project.

Just freaking learn Webpack, dude. I agree that it sucks, but not everything about your job needs to be fun.

<comment><USER12> Yeah, that made me laugh. Is it really a relief to have a config file you _can 't see_ instead of one you can?

<comment><USER13> Given `vue inspect` or even a `console.log(config)` inside the `configureWebpack` function in vue.config.js, it's very much a config file you _can_ see.

<comment><USER12> Fair enough -- it's not like it's completely opaque. But when I start a project and see there is no webpack.config.js in the project root, I feel the opposite of relief.

For example, if there isn't a config file, it's usually because there are more than one. Or, even worse, the config is dynamically generated. A build process like that is much harder for me to fully understand than one with a simple webpack.config.js.

However, this may be biased, as it's coming from the perspective of someone who already "knows Webpack".

</comment></comment></comment></comment><comment><USER14> Not needing to compile Vue.js is what got me started with it.

Looking forward to pure ES6 module version of Vue.js as a standard build for download from vuejs.org/npm. There's a test build (or something) from Nov. 2017, but it's not advertised, and it's an older version.

But it would mean no CLI/build/config needed at all to implement Vue in more modern environments.

But any improvements in the ecosystem is better for everyone.

<comment><USER15> You should check out lit-html+LitElement.

It's standards Web Components from the get-go - you don't write to a non- standard API and then compile it out like with Vue, you just use the standard connectedCallback(), disconnectedCallback(), etc

lit-html is standard JS, with all the flexibility of a non-standard syntax extension like JSX, but without a compiler. Templates are expressions, can be passed to higher-order functions, and use JavaScript for control-flow.

It's all ES modules, published to npm as ES modules, and loadable directly from CDNs like unpkg.com into all modern browsers.

No CLI or custom tools needed, but also works with all the tools our there.

Edit, link: [https://github.com/Polymer/lit- element](https://github.com/Polymer/lit-element)

<comment><USER14> I don't trust anything Google builds anymore, they've thrown so many projects under the bus that I'm not willing to invest in anything they build.

Unless I am really misunderstanding the connection of Polymer and lit-html... But also JSX gives me an icky feeling, I'd rather use standards compliant code everywhere. (while I contradict this statement with my v-bind: stuff everywhere, lol)

Plus, I got up and running with Vue.js on a major project in a day, and had reproduced a major jquery/ssr app in a week. I don't have much motivation to change at this point.

Unless Vue.js esm build never materializes, but that is a ways off to be concerned about now.

Thanks for the suggestions, I will take a look at lit-html again when I get a break.

<comment><USER15> | I don't trust anything Google builds anymore, they've thrown so many projects under the bus that I'm not willing to invest in anything they build.

Google's a huge company that makes a huge number of things. Of course you're going to hear about canceled projects, but there are far more that aren't canceled. But trust what you will, I can't really argue for or against that.

| But also JSX gives me an icky feeling, I'd rather use standards compliant code everywhere

lit-html is standard JavaScript. It uses tagged template literals for markup rather than non-standard syntax. But because template are expressions, they're structurally very similar to JSX expressions, and there's basically a 1-1 relationship between JSX forms and lit-html. So if you like the power and flexibility of JSX, but not the non-standard syntax.

I only suggested LitElement because of your desire for something like Vue but in pure ES6 modules and without a compiler. Since Vue is heavily inspired by Web Components, and LitElement meets your other requirements, I thought you might find it interesting.

<comment><USER14> > _I only suggested LitElement because of your desire for something like Vue > but in pure ES6 modules and without a compiler._

Which I appreciate, thanks. I have looked into it, but it was so long ago I had forgotten about it. And to be frank, it hasn't gotten a lot of attention online.

That being said, I will wait a while to see if Vue.js comes around with an ESM build (I guess I could make one myself, but that seems to brittle). And if not, then I will have motivation to look into another library/framework.

</comment></comment></comment></comment></comment><comment><USER16> Reasons I instantly loved Vue when I came across it:

\- Single file include,

\- Included in-browser template compiler if you wanted it (plus the option to write view functions manually)

\- Comprehensive Documentation that you could read through in a hour

\- Reasonable embrace of HTML, CSS, and JS as separate tools. For example, most of the reactivity system is managed in JS, but the integration with HTML components is excellent -- nothing crazy like passing all your variables through a single `data` attribute on a <div>.

Vue seems to be working hard to build a moat of complexity around itself. I think it all started with the introduction of .vue files. I personally don't understand the need to put 3 different languages in the same file. Vue CLI is another step in the same direction of adding complexity. It _looks_ like simplicity because "oh it's so easy for new people to get started", but if this keeps going, you're going to get tons of people who don't learn the basics/don't understand how the underlying tools work/can't fix problems in their own codebase because they didn't write the first 20% of it.

I still love Vue but I'm very very skeptical that this complexity they're adding isn't going to snowball, and eventually vue-cli will be the only supported way of creating vue apps, with some of the features that keep Vue simple being removed because they "prevent" progress.

Also, Vue CLI is not a game changer. Frontend frameworks all seem to have their own CLIs now, which is baffling in-and-of itself on the face of it.

<comment><USER17> Mithril[0] has had those three points since the beginning.

Updates to their documentation will soon reference Parcel instead of Webpack, but neither are required. Just include the file in your html and the reactivity (and routing!) are included.

The reason I like it is because you have to understand the core concepts. But once you do, you can easily build on them to get what you need. And it remains fast even if you don't do things perfectly idiomatically. If that makes sense... So it's probably best for small teams that want to be very fast and effective.

[0]: [https://mithril.js.org](https://mithril.js.org)

<comment><USER16> Mithril is awesome! I really like it, it's one of the only micro frameworks that I feel is really underrated right now (I also took a look at rivets, and few others). I've actually _really_ been feeling mithril! I avoided mentioning it to seem like I was trying to funnel people to another framework.

I like mithril because it's even simpler, and it has a few more concepts built in that are almost always necessary (ajax, etc), and the reactivity system is almost ridiculously simple -- if you don't fire an event stuff won't happen.

I think what mithril lacks is the adoption, or even better some sort of centralized repository of components that people who are fans of mithril can use. That, and an integration with Nativescript, so it can get "mithril- native" for free...

</comment></comment></comment><comment><USER18> For me the biggest game changer with vue-cli 3 is the inbuilt multi page support (not sure if it was present before). Rarely have I encountered any apps with a single page and this used to require all sorts of hacks before. But now it is as simple as adding a 'pages' entry in vue.config.js

</comment><comment><USER19> Am I the only one that dislikes webpack? It's a lot slower than just good ol' browserify, when using watchify.

Even if you precompile libs

<comment><USER20> parcel-bundler bundles a modified CSS in half a second, quicker than alt tabbing back to browser and no config file.

<comment><USER19> node-sass is still faster, and no config

</comment></comment></comment><comment><USER21> I think it absolutely can be. Ember CLI and its addon exosystem is probably one of my favorite parts about Ember. It's great to see other front-end frameworks adopting similar patterns.

<comment><USER22> Finally other projects start to grow up a little bit to Ember.js, I still feel the most matured and powerful is Ember. Not hyped, because it is just works out of the box. Incredible. And it is already with us for years. ;)

</comment></comment><comment><USER23> I believe the new version of create-react-app (2.0) will have a bit more flexibility.

See: Roadmap for email@example.com [https://github.com/facebook/create-react- app/issues/3815](https://github.com/facebook/create-react-app/issues/3815)

<comment><USER24> I really wish they allowed for a TINY bit of configuration. I understand their need for batteries include, but having to eject to add `target: 'electron- renderer'` is pretty whack.

I wrote a thing that overwrites CRA's own webpack configs with one that loads said configs and passes them to a function you control. There's probably a package that does this, or a better way of doing it, but here's what I have: [https://gist.github.com/mikew/97e2be1d428aae8460a33dfbee0ca8...](https://gist.github.com/mikew/97e2be1d428aae8460a33dfbee0ca866)

<comment><USER25> Doesn't it allow some kind of custom scripts?

<comment><USER24> I'm not sure if it allows custom user-based stuff.

There's things like react-scripts-ts that do things like add TypeScript. But they are actually forks of react-scripts, a whole series of packages.

</comment></comment></comment></comment><comment><USER26> How does this compare to Ember CLI?

[https://ember-cli.com](https://ember-cli.com)

</comment><comment><USER27> Not gonna lie, one of my favorite things about vue is that cli.

Don't even know why since once it's run and the project is setup, it mostly stays out of the way, and I've learned to put all configuration in _package.json_.

Hmm, maybe that's why.</comment>


Elixir's package manager, Hex, gains private packages and organizations - <AUTHOR> https://hex.pm/blog/private-packages-and-organizations <comment><USER2> Is it possible at this point to configure mix to fetch dependencies from a self-hosted package manager repository? Similar for example to self-hosted sbt or maven repositories that can be configured in the build script.</comment>


What School Should Teach - <AUTHOR> https://docs.google.com/spreadsheets/d/1pe1YR-HssoaRWtNAtEBUmtrlD5J7eW_d624qZqDONxk/edit?usp=sharing <comment><AUTHOR> How can this be improved? Is continuing with this website a worthwhile endeavour? Is the idea of what I am trying to do clear?</comment>


Covid-19 Analyzer - <AUTHOR> https://github.com/m1ghtfr3e/COVID-19-Analyzation <comment><AUTHOR> Hey guys, I'm quite new to programming in Data Analyzing. Due to the worldwide corona virus situation right now, I decided to get some data and wrote a program with pandas to analize and visualize / make the data more accessible. What do you think about it?</comment>


Show HN: sleuth, an autodiscovery and communication library for Go HTTP services - <AUTHOR> https://github.com/ursiform/sleuth <comment><AUTHOR> I wrote an introduction and tutorial post, but linking the actual repo is more appropriate for this audience. Here's the post about it as well: [http://darian.af/post/master-less-peer-to-peer-micro- service...](http://darian.af/post/master-less-peer-to-peer-micro-service- autodiscovery-in-golang-with-sleuth/)</comment>


In B-School, Is That a Syllabus, or an Itinerary? - <AUTHOR> http://www.nytimes.com/2014/05/18/business/in-b-school-is-that-a-syllabus-or-an-itinerary.html?action=click&contentCollection=Your%20Money&region=Footer&module=MoreInSection&pgtype=article <comment><AUTHOR> When I was at B-School, I would call up an alumnus at a company I was interested in working for and say, "I am going to be in your city/town this Friday, any chance for a coffee?". They would almost always say yes ... and then I would have to quickly book travel. On more than one occasion I did this, I turned up and found that I had meetings with HR, senior managers (even a CEO) planned. My experience is there is no substitute for travel during a MBA.</comment>


Unionized video game voice actors are considering a strike - <AUTHOR> http://arstechnica.com/gaming/2015/09/unionized-video-game-voice-actors-are-considering-a-strike/ <comment><USER2> Seems ridiculous to pay "stunt wages."

<comment><USER3> It may seem that way at first, but quite honestly, some voices they do CAN damage your ability to speak/sing/etc. Under those circumstances it seems completely reasonable to pay someone more for taking a risk with their primary method of generating income.</comment></comment>


How to evaluate your Machine learning model like a pro - <AUTHOR> https://ranvir.xyz/blog/how-to-evaluate-your-machine-learning-model-like-a-pro-metrics/ <comment><AUTHOR> A few days ago, I was learning about evaluating my Machine Learning Model and I have compiled the things that I learned during the period. Do let me know how I did with this.

Also, Let me know, how do you like to evaluate your Machine Learning Model?

<comment><USER2> I think this is a concise and good overview of ML model evaluation. I was recently thinking along the same lines and posted it on my blog here: [https://scienceofdata.org/2020/02/23/ml-classifier- evaluatio...](https://scienceofdata.org/2020/02/23/ml-classifier-evaluation-a- first-look/)

You have nice graphs and cover regression models too. I use mostly these when evaluating models I've built, I also use the Kappa statistic which can be a useful summary.

<comment><AUTHOR> Good post! I really liked the concept of ROC curve in there. Will read more about it.</comment></comment></comment>


Twenty rules for good graphics - <AUTHOR> http://robjhyndman.com/researchtips/graphics/

<comment><USER2> Twenty rules for good graphics should be beaten over the head with twenty rules for good typography.

</comment><comment><USER3> PGFPlots is an excellent plotting package for Latex. Just include data from text files and define styles in Latex, and everything goes without headache.</comment>


Show HN: Zero-Config Documentation Websites for Python - <AUTHOR> https://timothycrosley.github.io/portray/ <comment><USER2> Tried this on our project, GitHub.com/hail-is/hail.

Sphinx is a persistent thorn in our side, but we've made it work. A couple things I had to do to get this working:

\- something thinks a filename with dots (e.g. example.8bits.bgen) means there is a python module that needs to be imported (e.g. example), and fails when that does not exist; we have files like this in a data directory

\- I could not quickly figure out how to ignore files, so I had to delete a Sphinx conf.py file.

\- Sphinx resolves the _templates folder relative to where it is called, not relative to the doc string's source file, so I had to add several symlinks (rather than manually edit every doc string).

\---

OK docs are generating. Quite slow, maybe a couple minutes. Looks like one core is at 100% during this process. We've got ~400 files. Maybe its parsing some data files? The whole directory is 318 MB.

\---

Now I'm looking at the docs.

First thing I notice is that Python type hint forward references break everything. Consider:

        class GroupedMatrixTable(
            parent: 'MatrixTable',
            ...

The generated doc page is missing the class name and the entire class definition is inlined as a preformatted block (with highlighting :shrug:).

All my python function def's seem to be missing the function name?

No arg functions look really weird

        def (
        )

So all our docs are in ReST (thanks Sphinx :|). This means they're not valid Markdown. It seems that invalid markdown in a class doc string can break the formatting of all the methods (presenting them again as one reformatted block).

\---

On the bright side, the mobile version looks great and the search works better than my experience with Sphinx.

EDIT: formatting

<comment><AUTHOR> Thanks so much for this great feedback! I haven't done any speed optimization on portray yet - as both at work and online I tend to organize my own projects as small self contained repositories. Clearly there is a lot of room for improvement there, I have a lot of ideas to make it faster.

You can manually define the modules for portray (which you probably figured out to get documentation rendering): [https://timothycrosley.github.io/portray/docs/quick_start/4....](https://timothycrosley.github.io/portray/docs/quick_start/4.-configuration/)

And, yes write now Markdown only. I will say - this was a week only project: [https://timothycrosley.com/project-2-portray](https://timothycrosley.com/project-2-portray) so I think there's a good chance I could improve these points with one more week of time spent :)

Thanks!

~Timothy

<comment><USER2> Yeah of course! In retrospect my initial post seems a bit neutral to negative. I'm really glad this project exists, I'm endlessly frustrated by Sphinx. I'll keep my eye on your project!

</comment></comment></comment><comment><USER3> My team at present use sphinx-doc[1] and generate both HTML and PDF. We write the documentation as part of the code using docstring and then other documents manually in rst. We use Sphinx plugin to automatically generate document from code.

I will give portray a try and see.

Is there any comparison between Sphinx-doc and portray?

[1] [http://www.sphinx-doc.org/en/master/](http://www.sphinx- doc.org/en/master/)

<comment><USER4> We've historically done this as well. One issue we perpetually run into is that the docstrings get out of sync with the actual signature / types of the methods/etc that they purport to document. I suspect there are plugins that generate documentation from types such that there is less to keep synchronized. It is also unfortunate that Python projects have to set up infrastructure to build and publish documentation packages. I really like the effortlessness that Go's ecosystem brings--[https://godoc.org](https://godoc.org) just reads your source code from your git repository, so there is are no explicit steps for building and publishing documentation.

<comment><USER5> > the docstrings get out of sync with the actual signature / types of the > methods/etc that they purport to document

But that's on the devs, not Sphinx, eh?

<comment><USER4> It's kind of on the whole dynamically typed model. If you can have static type annotations that are validated for correctness, then you can use those annotations to validate or generate your documentation. Python supports annotations and validation thereof via Mypy, and assuming the project takes advantage of these, then Sphinx could leverage them to validate/generate documentation.

</comment></comment></comment></comment><comment><USER6> This is great! You hit the need for configless docsites that many Pythonistas who are in data science and research have: we write a lot of experimental and prototype code. Docstring documentation in itself feels like a considerable investment in these contexts so anything that helps publication without any hassle is more than welcome.

Are you planning to support Restructured Text next to Markdown? I find ResT is used more often (in the machine learning ecosystem at least).

<comment><AUTHOR> Thanks!

> Are you planning to support Restructured Text next to Markdown?

I will look into supporting this, I agree that it definitely makes sense as a feature!

</comment></comment><comment><USER7> I just used this on a random project I happen to be building documentation for. I'm pretty in love with the minimalism of 'portray'. I wish there was a chance to get my team to use it, but I'll definitely be using it in my personal projects going forward!

</comment><comment><USER8> It's always kind of interesting to look at a document created by the tool it's describing.

<comment><USER9> I thought the GIF within a GIF was very clever. Did you notice? It's "playing" itself back.

<comment><USER10> I think the number of playbacks is the number of times they had to re-record the video. :)

<comment><AUTHOR> Ha, I wish it was accidental! In reality, I created the gif many more times than needed just because I wanted to create the effect.

</comment></comment></comment></comment><comment><USER11> Very neat, I especially like it's compability with mkdocs-themes. Will try this out on some project :)

</comment><comment><USER12> I am confused, all I get is "No Python project found in the given directory".

<comment><AUTHOR> portray requires your project to be packaged for it's auto discovery - if you get this error and your project has a setup.py or pyproject.yaml in the root (and this is where you ran portray from) let me know and I'll create a ticket! Otherwise - I highly encourage looking at using poetry to setup your pyproject.yaml file: [https://poetry.eustace.io/](https://poetry.eustace.io/)

<comment><USER12> my project doesn't have a setup.py because it is not intended for packaging and it doesn't have a pyproject.yaml because to be honest, this is the first time I heard about such a file. But then again, the title says "zero-config".

<comment><AUTHOR> portray no longer requires a config to be present, instead you can just specify modules directly from the command line with `-m`: [https://timothycrosley.github.io/portray/TROUBLESHOOTING/#no...](https://timothycrosley.github.io/portray/TROUBLESHOOTING/#noprojectfound)

</comment></comment></comment></comment><comment><USER13> Excited to try this. From a Perl background, used to POD, Python's doc systems have been a sewer.

</comment><comment><USER14> How is the search implemented?

<comment><AUTHOR> portray uses MkDocs default search plugin which uses lunr.js behind the scenes.

</comment></comment><comment><USER15> How easy is it to theme for when Material starts looking dated?

<comment><AUTHOR> [https://timothycrosley.github.io/portray/docs/quick_start/4....](https://timothycrosley.github.io/portray/docs/quick_start/4.-configuration/)

portray is compatible with all MkDocs themes and plugins out-of-the-box

<comment><USER15> Awesome, looking forward to trying this out on a new project.

</comment></comment></comment><comment><USER16> Kudos to using toml for configuration.

</comment><comment><USER17> I was super pumped by this, and took some time to look into pdoc3 too.

\- [https://github.com/pdoc3/pdoc/issues/87](https://github.com/pdoc3/pdoc/issues/87)

\- [https://github.com/pdoc3/pdoc/issues/64](https://github.com/pdoc3/pdoc/issues/64)

That's a hard pass from me. I'm not going to link my employer's name (much less my own name) to anything with a swastika on it.

<comment><USER18> There is no question that these are Buddhist swastikas; they are clockwise and also rotated 45 degrees compared to the most common nazi version.

It is difficult to appreciate how ubiquitous swastikas are in the Buddhist world until you go there and see them everywhere. I have no doubt that this was an innocent decision.

<comment><USER17> It stopped being an innocent decision once it was pointed out and he doubled down.

Put another way: I cannot put my or my employer's interests in Western markets at risk because something is acceptable in Eastern markets. The maintainer can do whatever he wants to do; I'm not questioning that. I'm just saying I won't be using it, even if it would save me from sphinx-autodoc fragility.

<comment><USER19> > I cannot put my or my employer's interests in Western markets at risk > because something is acceptable in Eastern markets

My employer works in both markets, so rejecting something because it's Eastern would look much worse than rejecting it because someone in one market might not know what it is.

Anyway, it's a tiny icon on the boiler plate of a dependency of a project used to generate documentation. It would be a huge stretch to say your employer supports Nazis because of this.

<comment><USER17> I highly doubt the absence of any religious iconography is going to harm my employer in eastern markets.</comment></comment></comment></comment></comment>


What Android Users Really Think of Apple. - <AUTHOR> http://qrious.ly/i5fBrM What 10,500 Android users really think about Apple. Note sentiment towards iPhone if it ran Android... <comment><USER2> The first question shows that the software/OS is more important than the actual hardware. At least for the android users. Nothing surprising here, that's how it should be. (And that's the reason traditional big players feared android - it turns them into low margin hardware manufacturers.)

</comment><comment><USER3> I'm glad I finally have some statistics to show my Apple fanboy friends. For far too long have I heard that Android apps suck. You have to be discerning, but the good apps are _really_ good.

</comment><comment><USER4> interesting that people think that apple without steve jobs will still work

</comment><comment><USER5> really? i whould never buy a iphone again - love my desire

</comment><comment><AUTHOR> the apps on the iphone are still way better, hands down.</comment>


Searching a position in the video game industry - <AUTHOR> Hello, I am a student at Gamagora, a video game school in Lyon. In order to graduate with a M2, I am looking for a first job or an internship in video game development (gameplay or engine).
I got experience with C++, C#, JavaScript and Rust (another system language).
I have worked on several projects such as Crazy Orbit (https://lnkd.in/gPQtaxH) for the Global Game Jam 2018, or Koda Kiyomori's Guardian in the scope of the eend of year project at Gamagora.
I am also working on a 3D game engine written in Rust (https://lnkd.in/gp_c2wK).
I am open to any proposition, so feel free to contact me if you are interested.
Here is my email: email@example.com
Best, Mathieu NIVOLEZ <comment><AUTHOR> Here is my resume: [https://www.mathieu- nivoliez.com/documents/resume.pdf](https://www.mathieu- nivoliez.com/documents/resume.pdf)

</comment><comment><USER2> Where are you interested in working (location)?

<comment><AUTHOR> Mostly northen country (Swenden, Norway, Canada), but I am open to any possibilities. Remote is also an option for me.</comment></comment>


FAQ on Pi-Calculus (2002) [pdf] - <AUTHOR> http://www.cs.cmu.edu/~wing/publications/Wing02a.pdf

<comment><USER2> There has been tons of research on the pi-calculus. But very little has been lifted to the "real world". Maybe because the pi-calculus model isn't suited for programming (unlike the lambda calculus that led to Scheme, ML, Coq...).

I'd love to be proven wrong, but it seems to be a niche for a community of researchers to publish useless technical papers. At least, they should devote their energy to try to apply their results to solve real problems.

<comment><USER3> > ... they should devote their energy to try to apply their results to solve > real problems.

You might instead have said this about any researcher in abstract mathematics, computer science, and even theoretical physics.

Some people enjoy working directly on "real problems", and that is fine. But plenty of other people enjoy working simply to push the frontier of our knowledge, and that's also fine.

pi-calculus and lambda calculus are both theoretical frameworks, engineered for clarity of notation and thought, in order to reason better about theoretical problems in computation. Neither of them were designed to be practical programming tools, and their merit does not rest on whether they happen to inspire such tools.

<comment><USER2> One thing that bothers me a little with the pi-calculus, is that unlike some abstract maths or theoretical CS, many researchers sell their results with potential real-life application arguments. After several decades of the same papers studying very similar type systems and equivalence with no application, one can legitimately starts to wonder if this is worth the effort.</comment></comment></comment>