Hypermodern Python - <AUTHOR> https://cjolowicz.github.io/posts/hypermodern-python-01-setup/ <comment><USER3> This helps highlights the main issue I have with python today, and that's running python apps.

Having to ship a compiler to a host|container to build python with pyenv, needing all kinds of development headers like libffi for poetry. The hoopla around poetry init (or pipenv's equivalent) to get a deterministic result of the environment and all packages. Or you use requirements files, and don't get deterministic results.

Or you use effectively an entire operating system on top of your OS to use a conda derivative.

And we still haven't executed a lick of python.

Then there's the rigmarole around getting one of these environments to play nice with cron, having to manipulate your PATH so you can manipulate your PATH further to make the call.

It's really gotten me started questioning assumptions on what language "quick wins" should be written in.

<comment><USER4> I decided to drag myself kicking-and-screaming to the 21st century and start writing my handy-dandy utility scripts in python instead of bash. All was well and good until I made them available to the rest of my team, and suddenly I'm in python dependency hell. I search the internet and there are a lot of different solutions but all have their problems and there's no standard answer.

I decided "to heck with it" and went back to bash. There's no built-in JSON parser but I can use 'grep' and 'cut' as well as anyone so the end result is the same. I push it to our repo, I tell coworkers to run it, and I wash my hands of the thing.

<comment><USER5> jq has been a lifesaver for me parsing json in bash. Of course, it's an external utility not present by default in most systems.

Another thing to consider is more of a middle-ground approach. Most systems do have a python interpreter, so you can use a lot of base python without worrying about dependency hell. I use inline python in bash all the time, e.g.

      ls | python -c 'import sys,json;lines=sys.stdin.read();print(json.dumps(list(filter(bool,lines.split("\n"))),sort_keys=True,indent=2))'

You can even use variable substitution, if you surround the python code in double quotes. Even mix f-strings and bash substitution

      python -c "print(f'Congrats, ${USER}, you are visitor number ${RANDOM}. This is {__name__}, running in $(pwd)')"

<comment><USER6> Great trick with using the python standard lib! Thanks for posting that.

edit: You probably already know this, but for anyone reading along, piping `ls` is unsafe if you plan to use the paths for anything except for printing them out. A path on linux can contain any byte except for NULL, so when `ls` prints them out, you can get broken behavior if you try to break on newlines.

</comment></comment></comment></comment><comment><USER7> It's evident from reading the OP and previous similar posts on HN that many developers find it difficult to specify and replicate deterministic Python environments for their applications. Personally, I have found it best to use (a) a virtualenv or conda environment, with (b) a requirements file that specifies fixed version numbers for packages (e.g., `pandas==1.0.3`). Only very rarely have I run into issues doing this; it works quite well for me.

\--

That said, from a security standpoint, I'm not sure it's a good idea to run a script downloaded from the web, without verification, on your _local_ command line:

      curl https://pyenv.run | bash

If that URL ever gets hijacked, you would be running malicious code. At a minimum, you may want to take a look at the script before running it, or otherwise verify that you're downloading what you actually want to run.

<comment><USER8> I completely agree. I have no idea why many people seem to be fan of this `curl x | bash` approach.

For pyenv all is needed is to clone their git repository and add a couple of lines to your .bashrc.

<comment><USER9> I think `curl | bash` is treated unfairly. Whether you `git clone` or `curl` a script, you are fundamentally doing the same thing: downloading and executing code from the internet. `git clone` just feels safer because it is hiding that fact under layers of abstraction.

If I want to run pip, I need to trust PYPA. It's their code I want to run, and I need to download it one way or another. If I don't trust them to keep their domain secure, I don't see why I would trust them to keep their github repo secure.

And the whole point of pip is to download code from PYPI and run it. pip, git, curl|bash, all do the same exact thing in this case. curl|bash just smells funny because it makes it more plainly obvious what is going on.

<comment><USER10> git clone will not execute arbitrary code from the internet without inspection.

<comment><USER11> I assume the cloned git repo includes arbitrary code that will be executed with the same security profile as a downloaded shell script.

</comment></comment></comment></comment></comment><comment><USER12> I think that at this point all the ceremony around setting up and deploying a python project outweighs it's 'easy to read and use' aspects. Unless there is a library you can't live without or rewrite, it seems like a language with better tooling and real benefits of a type system is a better choice.

If you're mainstream: Go or Java. If you're edgy: Nim, Scala, or Crystal

All of those have much more sane type, build, and packaging systems.

@perl-people, was this a solved problem when Perl was big? Or is python walking the same roads?

<comment><USER13> It most definitely was not a solved problem when Perl was big, as far as I know Ruby is the language that finally solved it with bundler, which was released after Rails was, so that's like 2007 or something.

The recipe for the 'solved' deployment:

      - a compiler that ships with a build tool (so building is homogenous in the community)
      - a centralized or at least uniformly accessible package repository (so dependency acquisition is homogenous in the community)
      - a common file format for describing dependencies (so dependency resolution is homogenous in the community)
      - a common file format for locking dependency versions (so deployment can be done reliably without vendoring dependencies)
      - optional but very nice: a tool for managing compiler versions so it's easy to switch/upgrade projects

Any programming language that has all of these boxes ticked is a modern programming language in my book. As far as I know Ruby is the first that ticked all of them, but other ones I've used that have this: Node.js, Go, Rust, Haskell, Python (though it's a bit messy). I'm pretty sure C# checks them as well nowadays, but I haven't used it in over 5 years so I'm not sure. Same for Java.

<comment><USER14> Somebody needs to tell the Racket people about this. Dependency management in Racket is still C-style no-version-pinning-anything-goes. The third party dependency managers (see below) are all primitive and do not support multiple versions of the same libraries which is a standard feature in Go/Node.js/Rust/Java class loaders

[https://github.com/Bogdanp/racksnaps](https://github.com/Bogdanp/racksnaps)

</comment></comment></comment><comment><USER15> I have to say, at first blush I would not choose click over argparse. I do have to look up the docs of argparse every time I use it, but I like that it's just gives me the args and lets me structure the program flow how I want to, which I think is more natural.

And then I can do things like import a big package (pandas) only after parsing the args, which is highly convenient to users that want to check the argument options without a five second lag.

<comment><USER16> I recently wrote this ([https://github.com/jamesob/clii](https://github.com/jamesob/clii)) because I can't stand click and got sick of having to check the argparse docs every time I wanted to write a CLI. I guarantee you'll spend a tenth of the time trying to figure out how to use this thing, it has no dependencies, and is implemented in a single vendor-friendly file.

<comment><USER17> There is also Typer[1]

[1] [https://github.com/tiangolo/typer](https://github.com/tiangolo/typer)

<comment><USER16> Similar interface, different design goals. This lib has 6x the code, dependencies, and isn't as easy to vendor/audit.

<comment><USER17> Yep, good work! Though I do think the name "clii" could generate some confusion. Maybe not though.

<comment><USER16> Thanks!

</comment></comment></comment></comment></comment></comment><comment><USER18> I have a really high usefulness threshold for adding external dependencies to Python projects. If you can get away with never getting into the virtual environment mess, distribution/installation/development becomes _so_ much simpler.

Of course, sometimes you can't avoid external dependencies (personally, this often involves pandas). But the standard library gets you really far. And even though urllib.request is clunky, I will only use Requests if something else already is forcing me to add external dependencies.

<comment><USER9> I definitely agree for production-grade applications. Most of the time I'm using python, though, its to script a task I need to do or throw together a small app on a raspberry pi at home. In those cases, I have my own "standard library" of packages I always have installed system-wide, like requests.

Unless I am making a "real" application, I do my best to avoid virtualenvs altogether.

<comment><USER19> You don't have to. Use zipapps, this allows you to create a project with as many deps as you want, but use it as if it was a single python module.

It's an official feature ([https://docs.python.org/fr/3/library/zipapp.html](https://docs.python.org/fr/3/library/zipapp.html)) that lets Python execute an entire project bundled into a zip file.

If you don't want to create it by hand, use shiv: [https://pypi.org/project/shiv/](https://pypi.org/project/shiv/)

It's the best of both worlds.

<comment><USER9> That's really cool! I can't believe I've never come across this, thanks for sharing!

<comment><USER19> You are not alone. __main__.py in a zip has been supported litterally since Python 2.6 but very few people knew about it. Eventually in 3.5, the zipapp module has been released to make the feature more discoverable, but again it has been mostly ignored.

I think the reason is that something like shiv was missing: it streamlines the process, shows a finish product instead of just telling people "all the thing they can do", and the automatic unzipping solved plenty of problems that you had with alternative like Pex, espacially on windows or with static resources.

So now you have it. Spread the word.

</comment></comment></comment></comment></comment><comment><USER20> Hypermodern Python seems to have a lot more steps to it than Ye Olde Python, where you put a hello.py file somewhere and did 'python hello.py'.

Yes, yes, I know. But...

<comment><USER21> I genuinely thought that was a satire post about over-engineering Python projects

<comment><USER22> I think it's a very enthusiastic post. Anything can be over engineered - <USER2> generally young enthusiastic programmers are keen to learn about the options.

</comment></comment></comment><comment><USER23> I am not into Python, but the article lost me before it even started: you are required to install a bunch of compiler tools on your device to be able to proceed. Excuse me? Is this really what hypermodern Python looks like?

<comment><USER8> Those are needed to compile different versions of Python with pyenv.

Usually distributions bundle one or two specific versions of Python. Pyenv makes it super easy to install and use all the versions of Python that you want.

Even though for most people it might be enough to just use whatever version of Python comes installed with your system, for a team it might be important that everyone has the exact same version.

Moreover, pyenv-virtualenv makes it painless to use virtualenvs and so I recommend you give pyenv a try even if you do not need additional Python versions.

<comment><USER24> Again, do these tools really require a C compiler toolchain? If so, they are completely out of the question - they add much more complexity then they could possibly fix.

Why not just install the required version of Python, maybe from a 3rd party repo if not available in the main repos? Why would I ever want to get a compiler toolchain to get my Python interpreter?

<comment><USER8> Could you explain why it is such a deal-breaker in your opinion?

It is a single `apt-get install` (which probably downloads less MBs than our typical `npm/yarn` install).

Consider that if pyenv came pre-packaged for ubuntu/debian, those packages would be runtime dependencies and then you would just need to `apt-get install pyenv`.

<comment><USER24> What if I already have a different C toolchain installed? Would I now need "cenv" to keep the pyenv C toolchain and my other C toolchain separate?

</comment></comment></comment></comment></comment><comment><USER25> Great read! I see a few approaches I'd be interested in adopting. Nox sounds particularly useful for tooling consistency across development environments.

It's worth noting that this a rather opinionated toolset, and that we shouldn't mistake opinionated for hypermodern. You could replace poetry for pipenv (particularly now that it's being maintained again...), and I tend to prefer unittest to pytest. The further into it you get, the more opinionated it is -- for instance, code coverage support and CI/CD is not one-size-fits- all, but the choices of CodeCov and GitHub Actions are nicely illustrative.

<comment><USER17> Please no pipenv. A horribly managed project with worse design choices than Poetry. And a confusing name to boot (it makes it sound like a clean bridge between pip and pyenv, but it's definitely not). On top of everything you can throw in Ken Reitz's ego. Literally the only reason pipenv gained traction is because there are still an unnerving number of people in the Python community that think Ken Reitz shits gold, when he is clearly more of a one-hit-wonder (love requests btw).

Even the testimonials are self-important.

> Pipenv is finally an abstraction meant to engage the mind instead of merely > the filesystem.

Like, really? I want to interpret that as a joke, but I don't think it is.

Poetry should be the future of Python dependency management. I'd like to forget pipenv ever happened. Unfortunately since PyPA took pipenv under its wing, that might not happen.

</comment></comment><comment><USER26> Wouldn't hyper modern Python rely on containerization for isolation and portability? Poetry and pyenv seem like incremental improvements rather than a qualitative leap forward.

<comment><USER27> Heh, but that's one modern thing I don't like. Someone complained about click adding to startup time, running a new container is certainly "modern" but tiring.

<comment><USER26> It's certainly overkill for a one man side project. But if you have a team and deploy to clustered servers, virtual environments sure start looking like a horse and buggy.

</comment></comment></comment><comment><USER28> This feels like what will happen when the Node.js world gets bored with JavaScript and sets their sights on Python.

<comment><USER29> Yes, this! I'm not against the language evolving, but why do so many people seem to want to turn it into JavaScript? I've been using Python for the last 10 years, and when learning it went out of my way to make sure I was using the idioms of the language and not just use it as a different language with a Python syntax.

</comment></comment><comment><USER19> I don't see what's modern about it.

Those tools have been existing for years, and as always with Python, they serve only a subset of the Python users.

Not that it's a bad article. If you don't know about them, give it a read, it's good to know it exists and the post is well written.

But don't think it's the ultimate stack or whatever.

And if you learn Python, you should always strive to master the basics first:

\- installing python.org for windows and mac, or apt/yum on Linux (those ones are tricky, it's more than just the "python" package to get pip and venv)

\- don't try to get the latest and greatest version (which is 3.8, or 3.9 alpha today). 3.6 is great already and is wildly available. I personnally strive for 3.7 now, but I'm happy with 3.6 if I don't have to use asyncio. So get the most modern version that is easy to install for you.

\- be comfortable with "-m", the "py" command on windows, "pip" (-m, --user, install, install -r, freeze), and "-m venv".

\- make sure you understand what the PYTHON PATH is and how it works regarding the import system

\- know how to configure your favorite editor to work with those

Once you have those basics, you can go and try whatever have you: peotry, pyenv, pew, virtualenvwrapper, pipenv and so on. It will be easy because you will have a solid understanding of where it comes from and how it plugs in into the ecosystem. You will be able to chose if they are worth it or not for you. And more importantly, when you will have to move away from them (because of work, because they don't suit you, because they are abandonned...), you will always be able to fall back to basics.

But franckly, there are so many things that you should be learning before those extras: pdb, black, pylint, zipapps... Or even concepts like generators, decorators, unpacking, etc. They all have a better value for the time you spend on them than learning a new "modern" stack.

Not that it's not useful to have a modern stack. I update my stack all the time.

I'm currently using pip + pew + doit + nox + pytest, and I'm experimenting with dephell.

But it's my full time job. I'm an expert.

Most Python devs have a much limited amount of resource to spent in learning stuff. They have deadlines, and other stuff to care about than Python.

I know because I go from company to administrations to students to train them, and it's always the same: they chose Python because it's an efficient use of their time.

<comment><USER30> Context is important. Developing applications vs libraries vs scripts is different, developing internal company software vs external software is different.

If you're shipping a package on pypi, you will want to develop and automatically test using multiple Python versions and maybe a range of versions of your dependencies. Having good tools to manage multiple sets of Python versions and dependencies and parallel automated testing is a godsend.

If you're writing an internal or web application that will run only in a single well-known environment, pip + venv + pytest might well be good enough.

<comment><USER19> Sure, but again, there is nothing modern about this. And certainly not, hyper modern.

Besides, if you want to develop and automatically test using multiple Python versions and a range of versions of your dependencies, none of the tools from the article will do it.

At best they are one of the ways to get dependancies before you use a tool to do it. You can do that with raw pip and venv as well.

What you need though, is something like tox, which has existed for years.

As mentioned in my comment, I personnally use nox for this, as it is, ironically, more modern to my taste.

Nox doesn't care if you use poetry, pew, virtualenv wrapper or venv manually.

I would certainly advice to learn something like nox or doit before learning poetry, for example.

Now I don't advice against learning poetry, mind you. It's a good tool, well written.

Pyenv is another matter entirely. The day one really need what pyenv gives you to the point it's worth investing in it, they can discard all my advices entirely, as they would have the skill and experience to not need my explanations.

<comment><USER30> Not sure if you've read just the first article, it's a whole series, they cover nox later on.

I don't see how this stack is "hypermodern" either, mind you.

My point was just that these articles and other like it tend to omit context completely. What is this stack good for? Who should use it? Who shouldn't? Without answering these questions the articles are not all that useful, or even could be harmful, if a dev who doesn't know better apes a complicated stack for no good reason.

<comment><USER19> I missed that it was serie, indeed.

</comment></comment></comment></comment></comment><comment><USER31> My team has struggled a lot with how to do python deployment.

We know we can isolate the interpreter and modules via venv/docker/etc. and get repeatable and reliable deployments. However, because we're a utility module, we like to allow users to import our code freely, and that's a lot harder when that code is isolated and bundled with very specific requirements.

Seems like the only perfect solution is to just support every conceivable version of python and our required modules. Which is of course very hard. It would probably require greatly reducing our usage of tensorflow and some other packages which change a lot and are trickier to use on Windows.

Good old dependency hell.

</comment><comment><USER32> I think poetry is great and I hope it becomes more widespread.

<comment><USER33> Totally agree. Poetry is a joy to work with

</comment></comment><comment><USER34> I wrote a tutorial the other day on using Nix to handle the Python package situation. No need to install Python or even touch pip:

[https://thomazleite.com/posts/development-with-nix- python/](https://thomazleite.com/posts/development-with-nix-python/)

<comment><USER35> +1 for nix. Nothing comes close. Pain to first learn it though.

But now you have truly deterministic builds.

Well, almost. Don't forget to pin channels.

</comment></comment><comment><USER36> What's the benefit of using pyenv? I've been using `python -m venv .venv` to create virtualenv within the project directory for a while now, and VS Code recognize the .venv directory as python virtualenv and apply it to the workspace automatically. So far it's been quite painless for me.

<comment><USER37> They serve a different purpose. Pyenv enables you to maintain several Python versions. You can that use the venv module of each respective Python to create a virtual environment with a Python at a given version.

<comment><USER36> I see, I think I confuse it with pyenv-virtualenv. I never need to use multiple versions of python so I never try it so far. I always install the latest version globally and use python 3 docker image as the base deployment image.

<comment><USER38> If you have multiple projects, it may be undesirable that you're forced to have them all on the same Python version. Of course it helps that Python has fairly good backward compatibility, and even using a newer version to develop and test than the one you'll deploy with is not the end of the world, but pyenv is easy enough that I see no reason not to do it properly.

</comment></comment></comment></comment><comment><USER39> I guess that there are many tools for environments now... my personal choice is using conda (I actually start with miniconda and download what I need from conda-forge).

It provides the needed isolation, you can get lots of packages without having to compile from conda-forge and it still works with pip for the odd cases where a given package is not directly available (as a bonus, it works for other native toolchains, not only Python, which is a huge plus for me).

As a note, I personally use it with `conda devenv` to be able to replicate installs for a given project -- i.e.: [https://conda- devenv.readthedocs.io/en/latest/](https://conda- devenv.readthedocs.io/en/latest/).

<comment><USER8> The only thing I never understood about conda, is why it does not support packages from pypi.

I mean, the Python community has a standard repository for packages (pypi). You cannot even name a library $something if $something is not available on pypi. Why would I use something hosted by a private company that does not even interface with the rest of the community?

This is not an attack on conda. I just cannot understand its rationale.

<comment><USER39> The real problem (which is solved by conda) is that pypi doesn't solve dealing with non-python packages well (say, compile scipy and make all related native packages communicate well). This is a huge issue for people dealing with scientific packages (and the main pain point which pypi being just python- focused on installing on site-packages doesn't solve well enough IMHO).

Also, while it was initially done by a private company, I'd say it's definitely a community effort right now (it's also the reason I tend to use [https://conda-forge.org/](https://conda-forge.org/), which is community driven and not anaconda).

As for dealing with pypi, it does integrate well enough for me (given that you can just pip install packages on the python for which conda is managing the env), but yes, the other way around isn't true (conda solves a bigger problem than pypi up to the point that it's possible to even have non-python tools available -- one real use case example I have here is having innosetup binaries as a tool in the PATH in some conda env for doing builds).

Note: I don't have any affiliation with any of that, these are just my preferences for managing python envs (when I'm developing pydevd, which is the debugger engine used in pydev/pycharm/vscode, many times I need to reproduce some weird env and before using conda that was pretty annoying).

</comment></comment></comment><comment><USER40> This is so painful to me. So much of a reinventing the wheel for things that are not broken. :(

</comment><comment><USER41> Ok talks about some alternatives like Poetry and pyenv, which have their own issues.

pyenv is a poor cousin of rbenv, has such a poor cli api and a leaky abstraction over virtualenv (with its plugin for virtualenv)

<comment><USER42> it's not an abstraction over virtualenv _at all_ ; pyenv installs python interpreters, and those interpreters should use _-m venv_ instead of _virtualenv_ anyway.

did you mean pipenv instead? we suck at naming things (and pipenv has its share of issues, no arguments there).

<comment><USER41> No I meant the pyenv virtualenv plugin. Have been using it for a long time that forgot that there's a plugin involved.

<comment><USER42> oh! Yeah, I don't use it at all. pyenv-installer drags that plugin in, and i cleverly fail to source it since it's totally useless for my purposes.

</comment></comment></comment></comment><comment><USER43> This post is a snapshot of one webdev's toolbox. It certainly is indicative of webdev's predilection for increasingly complex and opaque tooling, mixed- markup formats, and over-engineering (if it can even be called engineering).

You don't really need any of that stuff. That whole tooling ecosystem undergoes a rewrite every couple years, anyways, so I predict this post will not age well.

Just stick to setup.py, pip, and virtualenv. It's sufficient into all thy needs.

<comment><USER44> I was with you up until your last sentence. pip and virtualenv are hacky kludges that grew legs and lungs, that's how old python's issues around deployment are, there's an era before pip and virtualenv.

There should be an abstraction for it (deployment, versions, import hacking) like what pathlib did for file path manipulation.

We have a classic _lava-flow_ pattern. [http://mikehadlow.blogspot.com/2014/12/the-lava-layer- anti-p...](http://mikehadlow.blogspot.com/2014/12/the-lava-layer-anti- pattern.html)

<comment><USER45> What makes you think the newer tools are better than pip and virtualenv?

(They most likely use some of those in the background)

Edit: the link you quoted is very relevant

> We developers should recognise that we suffer from a number of quite harmful > pathologies when dealing with legacy code:

> We are highly (and often overly) critical of older patterns and > technologies.

> We think that the current shiny best way is the end of history; that it will > never be superseded or seen to be suspect with hindsight

<comment><USER44> > What makes you think the newer tools are better than pip and virtualenv?

I don't. Heck, maybe they are. I don't use them.

My point is that rather than being the beneficiary of Python's excellent abstraction powers, distribution et. al. is and has been a mess for decades.

Here's a history of the sordid mess starting from 1998: "The distutils-sig dicussion list was created to discuss the development of distutils." [https://www.pypa.io/en/latest/history/](https://www.pypa.io/en/latest/history/)

FWIW, Since pip and virtualenv became relatively stable and "blessed" and PyPI has matured the only new thing I've tried is Anaconda.

Unfortunately my main Python project right now uses Tkinter and the folks who make Anaconda have a circular dependency in their build system such that you need python to build Freetype so their Python/Tkinter/TCL/Tk has gruesomely- bad support for fonts, so my project looks like a potato. [https://github.com/ContinuumIO/anaconda- issues/issues/6833](https://github.com/ContinuumIO/anaconda- issues/issues/6833) Someone has put in a PR to hopefully fix it, and-- ah! --it seems to have gotten some attention earlier today: [https://github.com/conda-forge/tk- feedstock/pull/40](https://github.com/conda-forge/tk-feedstock/pull/40) Fingers crossed for great good!

Anyhow, things are rough all over, eh?

</comment></comment></comment></comment><comment><USER46> This is why I gave up on Python.

<comment><USER47> What do you use now, and how is it better?

<comment><USER46> I work mostly in Go but personal stuff is a mix of Go, Java, or Kotlin.

Go's only flaw is the module system sucks. Java is rock solid and rolls up nicely into a fat jar.

</comment></comment></comment><comment><USER48> All of these virtual environment managers over complicate dependencies with Python. Using a simple virtual environment is quite easy. All you need is Python, its built-in _venv_ module, _setuptools_ and _pip_. I've found _pip- tools_ helps with generating/merging comprehensive _requirements.txt_ files.

</comment><comment><USER49> > This article series is a guide to modern Python tooling with a focus on > simplicity and minimalism

I'm not sure using pyenv + poetry + click qualifies as a minimalist setup. I have never had to use pyenv as I figured out it was more robust to install the python versions directly from python.org and create the appropriate symlinks. Then using the venv module which has been included in python since 3.3. The argparser module of python is not that bad once you are used to it.

I'm not saying the tools and libs mentioned in the article shouldn't be used, it's just that they're not mandatory for whoever wants to stay close to the bare minimum.

<comment><USER50> I would use pyenv locally so I can easily switch versions but dev and prod is a Docker container with one python version, then poetry install the virtual environment. You don't even really need to install into a venv, since you're already in an isolated container, but going straight for a venv is kind of a reflex for most Python devs

</comment></comment><comment><USER51> This is exactly what I needed as a somewhat on/off longtime user of Python.

It basically boils down to using pyenv, poetry and setting up the pyproject.toml and project src layout.

I read the comments before the article and was expecting a MUCH more complicated content and intricate bash, tool or config setups but it seems straightforward.

I feel everything else is part of getting started into professional software development (git, modern OS, env vars setup, bash or shell scripts, etc) which is it's own right complicated for newcomers.

I think deployment and packaging vary a lot depending on the target deploy env which is why I'm fine with it being left out of the article.

</comment><comment><USER52> I've moved away from requests to httpx.

<comment><USER53> I've had my eye on httpx for a while. Could you elaborate on your reasons for changing and your experience with it?

<comment><USER52> Primarily async and it works extremely well alongside FastAPI.

</comment></comment></comment><comment><USER54> One thing that I haven't been able to figure out with the 20 minutes or so of reading about poetry that I just did -- does poetry support editable installs akin to pip install -e .?

<comment><USER55> So you shouldn't need to use pip install -e . as poetry already configures your package to be installed as editable.

For adding a separate package in your filesystem, I believe using [https://python-poetry.org/docs/dependency- specification/#pat...](https://python-poetry.org/docs/dependency- specification/#path-dependencies) is what you are looking for

``` poetry add ../path/to/package ```

</comment></comment><comment><USER56> I loved the wonderful Victorian drawings of air-fishes with people in them. Who did those and where can I find out more about these?

<comment><USER57> Google Reverse Image search gave me [https://publicdomainreview.org/collection/leaving-the- opera-...](https://publicdomainreview.org/collection/leaving-the-opera-in-the- year-2000)

</comment></comment><comment><USER58> Reading this article (which asks me to install 18 apt packages on my system) and the top comments here (complaints about developing, running and deploying apps, replicating deterministic Python environments and so on) makes me wonder why Python developers don't use Docker.

</comment><comment><USER59> Great resource, I'd add for the typing section, the usage of `py.typed` file.

It'd be worth sharing the alternative minimalistic approach

``` python3 -m venv .venv

. .venv/bin/activate

pip install -e . ```

But is good for people who are willing to install those tools to simplify their work. I use pyenv and poetry when I can.

</comment><comment><USER60> Poetry looks good, I haven't used it. It reminds me of the way Quicklisp really revolutionized how to maintain and use packages in Common Lisp.

I don't do too much Python now, but I bookmarked this article to possibly use as a future reference.

</comment><comment><USER61> > This article series is a guide to modern Python tooling with a focus on > simplicity and minimalism

No it isn't, it introduces at least two external tools of questionable necessity - pyenv and poetry when python comes with venv and pip natively.

<comment><USER27> Yeah, but pip and venv don't leave me impressed, so please elaborate why those are better.

<comment><USER62> I think that given pip and venv are bundled with Python, then anyone who wants to introduce new dependencies to replace them needs to justify themselves. I haven't used either pyenv or poetry so I have no opinion on them.

<comment><USER63> The benefit of pyenv seems clear to me - it's a simpler way to have multiple Python installs and manage which one is your "default" Python at a project, directory, user, system, and global level (they can even all be different).

Poetry...not so much. requirements.txt is good enough for me most of the time.

<comment><USER64> Poetry gives you exactly what pip and venv give you, with the two pretty much perfectly integrated, in a way that is pleasant to use and not something that you'll grudgingly migrate your project to after your fifth dependency.

</comment></comment></comment></comment></comment><comment><USER65> Seems overly complicated. I'll just stick with pip and virtualenv for now.

</comment><comment><USER66> If it would be really hypermodern, it would use httpx instead of requests: [https://github.com/encode/httpx](https://github.com/encode/httpx)

</comment><comment><USER67> The problem with this article is, it will take me a day or probably more to get started with my application.

A newcomer will give up before the "hello world" program.

<comment><USER68> I don't think this is for a newcomer. Newcomer, here you go:

print("Hello World!")

This is for someone who has done Python for many years, but might have fallen behind on the latest and greatest trends. It collects a series of promising, bleeding-edge tools and gives an overview of what makes them clever and how to use them.

<comment><USER69> > It collects a series of promising, bleeding-edge tools and gives an overview > of what makes them clever and how to use them.

"Promising" being the operative word here

</comment></comment></comment><comment><USER70> With setup.cfg supporting nearly everything setup.py needed to include, I fail to see a reason to use poetry's own configuration file.

</comment><comment><USER71> Excited for 2030 when this is reduced to one click

</comment><comment><USER72> This is hypermodern, but a security nightmare.

curl someting | bash

<comment><USER73> This pops up occasionally on HN but I don't get it. Are you reading every line of every installation script you run normally? What about the things they download and execute? Are you never running a binary installer where you don't have access to the code?

Genuinely curious what the nightmare is.

</comment></comment><comment><USER22> Hypermodern? is that like the bleeding edge?

</comment><comment><USER74> Sad thing is `poetry init --no-interaction` fails if your name happens to have non-ascii characters :(

</comment><comment><USER75> This is great! Is there something like this for Typescript?

</comment><comment><USER76> s/hypermodern/annotations

</comment><comment><USER77> hm

</comment><comment><USER78> Want a real hyper modern Python? Try Haskell. (As someone who spent 10y with Python)

<comment><USER79> Nim feels more like hyper modern python to me. Haskell is an entirely different paradigm.</comment></comment>


How to get a data science job: A ridiculously specific guide - <AUTHOR> http://brohrer.github.io/get_data_science_job.html <comment><USER2> I don't understand why decline the offer if you get it?

<comment><USER3> "Eventually you'll reach a point, where you can't quite bring yourself to follow through on step five [Decline the offer]. The position isn't exactly what you were looking for but it feels so right, you can't let it go. Then you're done."</comment></comment>


Show HN: Unlimited, free WordPress installs - <AUTHOR> http://poopy.life <comment><USER2> Can anyone explain what this is? Is it just like a temporary Wordpress installation?

<comment><AUTHOR> That's exactly what it is. We use this internally for customer support, product demos, and testing/debugging. Now we're opening it up to the public.

The ridiculous domain name is to prevent people from using it in their own product demos/support channels. Soon we'll have support for custom URLs and a real landing page.

</comment></comment><comment><USER3> Looks great. The template feature will come in handy.</comment>


Ask HN: HN-like site but without the tech content? - <AUTHOR> I know it's primarily a tech community, but after a couple of years reading HN, I realised I don't enjoy reading tech stuff anymore. I'm here for non-tech articles and question threads. What HNers usually upvote to the front page apart from tech is a well-filtered mixture of literature/journalism/culture/nature/craftsmanship etc. Is there any other place like this on the internet, or I still have to filter HN myself? <comment><USER2> Sivv (www.sivv.io) is a forum for sharing / discovering bitesize nuggets of wisdom and insights covering business, wellbeing, personal & professional development

</comment><comment><USER3> [https://en.wikipedia.org/wiki/News_aggregator](https://en.wikipedia.org/wiki/News_aggregator)

<comment><AUTHOR> Yeah, but it doesn't come with upvotes

</comment></comment><comment><USER4> I mean, there's reddit, but again the subreddit determines the level of filtering and quality.

</comment><comment><USER5> Metafilter

</comment><comment><USER6> reddit.com/r/slatestarcodex/

Believe me, it's just what you're looking for.

<comment><AUTHOR> This might be it, thanks!</comment></comment>


The nationwide roll-out of the 911 system was a difficult endeavor - <AUTHOR> http://tedium.co/2017/06/20/911-emergency-system-history/ <comment><USER2> The original 911 system was rolled out when many phone switches were electromechanical. Step-by-step and crossbar offices didn't log calls, didn't pass line identification data with the call, and couldn't route based on the source of the call. But they had hard-wired relay logic in the originating registers (the units that receive dial digits) to recognize area codes (second digit 1 or 0 back then) and service numbers (second and third digits 1). Those were forwarded to a toll office, along with automatic line identification info. So 911 was forwarded to a toll office, like a long distance call. "999" was considered as the emergency number; the UK used that. But US switches didn't have a special case for 999, so that was hard to retrofit. Hence 911.

Toll offices, which handled long distance calls, were computerized by then. They could be upgraded to deal with 911 routing. This requires a database to map phone numbers to the appropriate Public Safety Answering Point. The idea of having a computer with a disk drive in the call chain was radical at the time, but it was made to work. The original computers for this were Western Electric 3B machines, set up as duplex pairs.

The toll office 911 system became a problem with the breakup of AT&T. AT&T retained most of the toll switches, while the local exchange carriers got the local offices. (Now, of course, with weak antitrust enforcement, they've all merged back together again.)

</comment><comment><USER3> My job is upgrading and maintaining 911 and other life safety systems.

The article is correct, advancing technologies are not reflected by additional capabilities by PSAPs, DISA's or ILEC's (The ppl that foward your call to a responder). It was only recently that IP based 911 sending of ANI/ALI (Telephone/Location data) data has been implemented on a large scale. In Canada, it ran on an old packet switched network for ages!

Their are different flavors of 911. E911 (Enhanced), V911 for VoIP phones, and recently the addition of Wifi based calling among others. The original 911 systems was designed when phones were static, they didn't move. Nowadays, with cell, voip and now, wifi devices, theirs no telling where the call (device) originates from. Yes, most send long/lat data, but that is based on triangulation of cell towers and not accurate enough. To further complicate things, long/lat doesn't take into account altitude. In a urban setting, the responders maybe at ground level wondering just which building and floor originated the call (yes, their systems to deal with this but not widely implemented, and, failing that, if the caller is unable to speak, you have a larger issue).

Most responders don't even have the ability to map a long/lat. So at this point, the accuracy is moot.

All that being said. The E911 system works well. Cell/VoIP/Wifi, not so much. The call will generate a response, but it's hardly efficient.

<comment><USER4> Then there's the idea that they'll go totally internet based, allowing forwarding or conferencing in multiple psaps, translators, etc. Somehow, they're supposed to secure all this, but I'm doubtful. It'd be one of the largest secure federated networks in the world I think. And they aren't exactly tech savvy. Doing one of the first wide-scale VoIP 911 services, we had psaps call us to say they wouldn't take phone calls from VoIP users, even on their landlines, because "they might send us viruses".

NENA folks also wanted the FCC to mandate an IP location system to be available on every Internet connection - what a privacy mess that'd be, when any app with IP access could get your exact location down to the apt number.

I trie experimenting with what I named "Advanced 911". For calls to places where we couldn't deliver location info, we'd prompt the answer to press a key to have us speak the info. But getting adoption for that was super difficult.

Also, the stress involved in those jobs, wow. I thought I could handle stuff, but auditing problem calls where people were dying, panicking, and didn't know where they were - that's traumatic. Also made me really dislike how cavalier 911 is handled by the telecos.

<comment><USER3> We have dedicated staff whose job it is to audit calls. ALL calls are recorded and we do screen snaps every 5 seconds, or, anytime the screen changes. Were talking terabytes of data. We keep it going back many years.

I'll vouch for the stress of the job. If I f'up. Somebody can die. Same thing for the operators. They get 3 months of training before handling calls. Most wash out before their training is completed. Most, shortly after that. The churn is unbelievable. We hire constantly. All operators max out their sick/vacation days. It's an unpleasant, boring, terrifying job.

<comment><USER5> This sounds like an interesting technical problem. If you're up for chatting about it further, I'd be interested in hearing more about it--my email's in my profile.

<comment><USER3> It's not really a technical problem, it's an adoption/money problem.

</comment></comment></comment></comment></comment><comment><USER6> There's some interesting technical history behind some of the emergency number choices:

999 in the UK (from 1937) was chosen because public payphone could be easily modified to make it a free call.

111 in New Zealand (from 1958) was chosen because the system was implemented using British Post Office equipment that already supported 999. But New Zealand phones pulsed in reverse so 111 on a New Zealand phone produced the same pulse as 999 on a British phone.

000 in Australia (from 1961) was chosen because 0 was already used for trunk access. On an automated rural exchange, 0 would connect you to a main centre. In remote communities it was 00. This meant that dialing 000 through an existing remote exchange would at least connect you to an operator in a main centre.

</comment><comment><USER7> I once worked at a bourgeoning hosting provider who had one of the higher floors in a building that was condemned, but couldn't be torn down because it was home to a phone switching room that had been a part of the original 911, and the designers of technology then _simply did not_ even imagine their systems being EOLed, so it turns out, no living or dead being knows how. I'm curious to know if that's still standing.

The freight elevator shaft had a shattered wooden freight elevator carriage at the bottom of it. We took the stairs.

<comment><USER8> Those kind of buildings are all around the country and serve various purposes - some of which specifically manage 911 calls. I can't imagine ever decommissioning one. Sometimes even gaining access takes something just shy of an act of god. While AT&T has re-consolidated in a major way, there are still a lot of external concerns to deal with.

Somebody would have to be very motivated to push through turning an interconnection point like that off. The politics involved in doing so would be extraordinary - even if there was common agreement from the start that it was time to let it go. I don't even know that you really could initiate a project like that from within a Telco.

It is a bit surprising that it was common knowledge that part of 911 ran out of there. While it's not "secret" information, the purpose and location of most similar properties are typically not openly advertised. Most people don't even know where their local CO actually is.

</comment></comment><comment><USER9> If you want to see an emerging country try it nowadays, look at the Philippines.

They basically gave up. Though the telecoms are a duopoly. Only working 911 system is in Davao.

Everywhere else you have independent short codes like 117 which may not be answered. Ironic for most of the BPO is outsourced call center agents.

Then the second issue are jokes,

[http://news.abs-cbn.com/news/08/01/16/dial-8888-911-govt- ope...](http://news.abs-cbn.com/news/08/01/16/dial-8888-911-govt-opens- complaints-emergency-hotlines)

"Hotline 911 received 2,475 recorded calls, between 12 a.m. to 7 a.m. Monday. Out of the total, only 75 calls were legitimate while the 1,119 were dropped and 1,356 were prank calls."

In other news, I've not yet needed it.

</comment><comment><USER10> An example of the drawbacks of decentralization - meanwhile in Sweden, 98% of phones were already covered by the "90 000" emergency number by 1965.

<comment><USER11> Wow, that must've taken forever to dial on a rotary phone.

<comment><USER10> On Swedish rotary phones the 0 was on the other side (one pulse)

<comment><USER12> What?! What happened when you made an international call, did you have to subtract 1 from all the digits?

<comment><USER13> In 1965 there wasn't much international direct dialling yet; mainly to neighbouring countries. The rest was done via an operator. International exchanges tended to buffer the entire called phone number anyway, even back in electromechanical times, which meant they could also translate the pulses if that would have been necessary.

</comment></comment></comment></comment></comment><comment><USER14> I'd like to see one on the roll-out of Japans Earthquake Early Warning System.

<comment><USER15> Totally agree. I'd really love to see Japan's ETWS system rolled out on the US west coast.

I was in Japan earlier this year, and happened to be watching TV when the earthquake warning system cut in. It gave about 5-10 seconds of advance warning before an earthquake hit. Enough time to move out of the way of anything heavy. It's quite impressive.

If anyone's curious, this is what it looks like: [https://www.youtube.com/watch?v=lAxxZkpV0HI](https://www.youtube.com/watch?v=lAxxZkpV0HI)

(The earthquake early warning cuts in at 0:15, and a tsunami warning automatically cuts in at 2:34... though most of the tsunami warning is cut off at the end. This is what the full earthquake+tsunami warning sequence looks like: [https://www.youtube.com/watch?v=qMNu8Y1jIlU](https://www.youtube.com/watch?v=qMNu8Y1jIlU))

<comment><USER16> Thanks for posting - I will be re-using those video clips at an appropriate time to warn of workplace upheavals etc.

</comment></comment></comment><comment><USER17> Article misses that little ol' Haleyville, Alabama was the first city to roll- out 911.

<comment><USER18> It was the first city to use those particular numbers, but the service existed as 999 elsewhere before it was introduced to the USA.

<comment><USER19> 999 on a rotary dial phone? Dear heavens.

<comment><USER13> That was, and still is, in the UK. Efficiency is apparently not their strong point.

<comment><USER20> It's a balance of efficiencies. On the one hand, false "1"'s can be generated by wind, etc, on the line. "111" calls can butt-dialled by sagging trees. With the rotary, "9"'s are nice and distinct.

On the other side of the balance, users perceived "9" as taking _aaaaages_ , partly because they're bloody users and never satisfied, and partly because in emergencies perception of time gets tweaked. Anyway, 999 it was, and remains.

<comment><USER13> [https://en.wikipedia.org/wiki/999_(emergency_telephone_numbe...](https://en.wikipedia.org/wiki/999_\(emergency_telephone_number\)#History) gives the, eminently reasonable, reason why this number was chosen:

 _> The 9-9-9 format was chosen based on the 'button A' and 'button B' design of pre-payment coin-operated public payphones in wide use (first introduced in 1925) which could be easily modified to allow free use of the 9 digit on the rotary dial in addition to the 0 digit (then used to call the operator), without allowing free use of numbers involving other digits; other combinations of free call 9 and 0 were later used for more purposes, including multiples of 9 (to access exchanges before STD came into use) as a fail-safe for attempted emergency calls, e.g. 9 or 99, reaching at least an operator._

</comment></comment></comment></comment></comment></comment><comment><USER21> That font is so horrible. :/

</comment><comment><USER22> while I want to read this article, that font makes it impossible.

<comment><USER23> I think you're being a bit overdramatic, or there's a difference between your configuration and mine (FireFox Desktop). I find it quite readable, pleasantly so even.

<comment><USER24> The color of the body text is (102,102,102). That cuts your contrast just about in half.

<comment><USER25> Which is awesome if you have a monitor with 1000 nits or more, as (0,0,0) to (255,255,255) has a blinding contrast on them.

The basic issue is that CSS colors are defined in terms of relative brightness of the monitor, and not within of a proper colorspace. Most browsers assume sRGB, but many users don't even have that.

In sRGB, a contrast of (102,102,102) to (255,255,255) is a very pleasant, and perfectly readable contrast.

But many users are on shitty 6-bit color depth 100 nits or less displays.

<comment><USER24> So you intentionally have your monitor set uncomfortably bright, but it's everyone else's fault? And the solution is for everyone else to spend money to make your site look better? How about no.

<comment><USER25> Not "uncomfortably bright". Those brightnesses are required for, for example, photo editing, and are useful when watching movies or gaming.

Changing contrast in hardware down is always a shitty option.

The solution is to handle this in hardware — artifically increase contrast and destroy image quality for people on shitty systems.

<comment><USER24> It may be a shitty option but at least you have the option. Unless you want your site to only be legible to photo editors with some money, you need it to have lots of contrast.

<comment><USER25> > Unless you want your site to only be legible to photo editors with some > money, you need it to have lots of contrast.

Or we simply fix browsers.

Because if it's a choice between blinding users on most mobile devices, and most higher end desktop computers, or blinding those who have cheap hardware, most developers will choose by who is actually willing to pay.</comment></comment></comment></comment></comment></comment></comment></comment>