Kubernetes at GitHub - <AUTHOR> https://githubengineering.com/kubernetes-at-github/ <comment><USER2> Love to see more Kubernetes success stories.

I work for an ISP and we are trying to write another success story ;) As an ISP, we have tons of constraints in terms of infrastructure. We're not allowed to use any public cloud services. At the same time, the in-house infrastructure is either too limited, or managed via spreadsheets by a bunch of dysfunctional teams.

For my team, Kubernetes has been truly a life saver when it comes to deploying applications. We're still working on making our cluster production-ready, but we're getting there very fast. Some people are already queuing up to get to deploy their applications on Kubernetes :D

What I especially love about Kubernetes is how solid the different concepts are and how they make you think differently about (distributed) systems.

It sure takes a lot of time to truly grasp it, and even more so to be confident managing and deploying it as Ops / SRE. But once you get it, it starts to feel like second nature.

Plus the benefits, in almost any possible way, are huge.

<comment><USER3> Red Hat's OpenShift makes it a lot easier by providing all of the infrastructure around it (docker registry, docker build from Git, Ansible integration and so on).

Best docs of all open source projects I've seen.

<comment><USER4> I second this. Have been PoCing OpenShift for a couple of months now and it's been a joy to use.

<comment><USER5> 3rd for sure. We're a RH partner and specialize in OpenShift work. Tons of excitement with customers on using OpenShift. I love it.

</comment></comment></comment></comment><comment><USER6> > During this migration, we encountered an issue that persists to this day: > during times of high load and/or high rates of container churn, some of our > Kubernetes nodes will kernel panic and reboot.

Considering that Kubernetes doesn't modify the kernel, this issue sounds like is present in mainline and kernel devs should be involved.

<comment><USER7> I would be interested to know what storage driver they're using for their nodes. High container churn puts a lot of stress on the VFS subsystem of Linux, and we've seen cases where customers have trigger lots of mount/umounts which results in filesystems causing panics. At SUSE, we do have some kernel devs debugging the issues, but the workaround is almost always "rate limit all the things". There are a few other kernel areas that are stressed with high container churn (like networking), but VFS is the most likely candidate from my experience.

While on paper containers are very lightweight, spawning a lot of them exercises kernel codepaths that probably haven't been exerted to that type of stress during development.

<comment><USER8> Hey, this is Aaron from GitHub. We're using devicemapper w/ LVM backed pools. Would love to hear about your experience there. We definitely see this problem during periods of high container churn.

<comment><USER9> My schroot tool used for building Debian packages could panic a kernel in under five minutes reliably, when it was rapidly creating and destroying LVM snapshots in parallel (24 parallel jobs, with lifetimes ranging from seconds to hours, median a minute or so).

This was due to udev races in part (it likes to open and poke around with LVs in response to a trigger on creation, which races with deletion if it's very quick). I've seen undeletable LVs and snapshots, oopses and full lockups of the kernel with no panic. This stuff appears not to have been stress tested.

I switched to Btrfs snapshots which were more reliable but the rapid snapshot churn would unbalance it to read only state in just 18 hours or so. Overlays worked but with caveats. We ended up going back to unpacking tarballs for reliability. Currently writing ZFS snapshot support; should have done it years ago instead of bothering with Btrfs.

<comment><USER10> In my work identity, we saw a similar problem in our testing, where blkid would cause undesired IO on fresh devices. Eventually, we disabled blkid scanning our device mapper devices upon state changes with a file /etc/udev/59-no-scanning-our-devices.rules containing: ENV{DM_NAME}==" _ourdevice_ ", OPTIONS:="nowatch"

Alternately, you could call 'udevadm settle' after device creation before doing anything else, which will let blkid get its desired IO done, I think.

<comment><USER9> Yes, we did something similar to disable the triggers. Unfortunately, while this resolved some issues such as being unable to delete LVs which were erroneously in use, it didn't resolve the oopses and kernel freezes which were presumably locking problems or similar inside the kernel.

</comment></comment></comment></comment></comment></comment><comment><USER11> If you're running Kube on AWS, make sure you install the proper drivers! For Ubuntu, that's the `linux-aws` apt package.

[https://github.com/kubernetes/kops/issues/1558](https://github.com/kubernetes/kops/issues/1558)

Missing ENA and ixgbevf can be a real performance killer!

<comment><USER12> Is this used by vanilla docker / ECS, or just k8s?

<comment><USER13> that advice holds regardless what software you're using, so long as the software does network traffic.

It holds for just running plain old nginx websites.

It doesn't really matter for small instance sizes where your networking is already rate-limited by amazon so much that ENA drivers won't matter, but on beefy instances it's always good advice to make sure you're using ENA supported driverse.

See [https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced...](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced- networking-ena.html#test-enhanced-networking-ena)

</comment></comment></comment><comment><USER14> > Several qualities of Kubernetes stood out from the other platforms we > evaluated: the vibrant open source community supporting the project, the > first run experience (which allowed us to deploy a small cluster and an > application in the first few hours of our initial experiment), and a wealth > of information available about the experience that motivated its design.

It's interesting that the reasons they cite for choosing Kubernetes over alternatives are entirely driven by 'developer experience' and not at all technical. It shows how critical community development, good documentation, and marketing are to building a successful open source project.

<comment><USER15> I believe developer experience on being introduced to a tool is paramount to its success. It gives a lot of confidence in what you're doing and keeps things moving forward. To me appear that the application is built on a solid simple concept instead of a convoluted complex architecture. Some tools sin on the opposite though, very simple to setup but very complicated to understand how to scale.

</comment></comment><comment><USER16> Really exciting stuff, happy to see the Github team launch this.

Kubernetes is becoming the goto for folks needing both their own physical metal presence and cloud footprint too. And the magic of Kubernetes is that it has APIs that can actually give teams the confidence to run and reuse deployment strategies in all environments. Even across clouds.

If you are like Github and want to use Kubernetes across clouds (AWS, Azure, etc) & bare metal and do deploy/customize that infra using Terraform checkout CoreOS Tectonic[1]. It also tackles more of the subtle things that aren't covered in this article like cluster management, LDAP/SAML authentication, user authorization, etc.

[1] [https://coreos.com/tectonic](https://coreos.com/tectonic)

<comment><USER17> I'm still utterly perplexed as to what Tectonic actually -is-. I kinda get that it's a kubernetes setup, but is it a GUI over the top of it? The website is pretty confusing and I think I gave up really quickly when trying to set it up.

<comment><USER16> Tectonic is Enterprise Kubernetes. We start with pure upstream Kubernetes at the core and install it in a production ready setup with the Tectonic Installer[1] across clouds or bare metal. On top of those basics Tectonic provides things most organizations need:

\- Authentication backed by LDAP/SAML/etc

\- One-click automated updates of the entire cluster

\- Pre-configured cluster monitoring/alerting

There is a bunch more in there and in the roadmap too but that gives you a taste.

The other thing is that we provide professional services, training, and support to customers on the whole stack from the VM or machine on up to the Kubernetes API. We have done neat collaborations with customers like the ALB Ingress Controller[2] too.

[1] [https://github.com/coreos/tectonic- installer](https://github.com/coreos/tectonic-installer) [2] [https://github.com/coreos/alb-ingress- controller](https://github.com/coreos/alb-ingress-controller)

<comment><USER18> I'm currently deploying Tectonic flavoured Kubernetes at a large organisation and I can vouch for how great you guys are at supporting users (who are not yet customers) at any stage of the process - love that, and can't recommend you guys for that more. However, as the comment above says, Tectonic (and Quay for that matter) documentation is... just horrible and if not the engineers support, I'd be pretty much stuck on quite few things. Why don't you push your docs to a public repo, so I could do some writing and send some PR's? ;)

<comment><USER19> Happy to report that the Tectonic docs are open source and we would love to review your PRs: [https://github.com/coreos/tectonic- installer/tree/master/Doc...](https://github.com/coreos/tectonic- installer/tree/master/Documentation)

Any topics that stick out as needing the most attention? Glad you're enjoying your interaction with our engineers :)

(Product manager for Tectonic)

<comment><USER20> The bare-metal scaling documentation needs to be scrapped and rewritten from scratch. With very limited knowledge about terraform, it's faster to start over with nothing than trying to get terraform (apparently you need the installer bundled one) to work with your assets.zip (which is not even mentioned in the installation documentation).

</comment></comment></comment></comment></comment></comment><comment><USER21> We're currently looking at moving our applications to k8s, and was wondering what deployment tools people are using? This week we are evaluating spinnaker, helm and bash wrappers for kubectl. There is concern over adding too many layers of abstraction and that KISS is the best approach.

<comment><USER22> We also did some evaluation and then decided to stick to KISS an chose kubectl commands combined with cat and kexpand. Really simple approach to allow dynamic kubernetes deployments.

Example command can be

       cat service.yml | kexpand expand -v image-tag=git-135afed4 | kubectl apply -f -

The service.yml contains the full deployment configuration, service definition and ingress rules. So this works without preconfiguring anything in kubernetes when deploying an new service.

An engineer only has to create the service.yml and Jenkins does deploy it automatically on every master build.

*kexpand is a small tool which does something similar to sed, but in a simpler and less powerful way (keep it simple): [https://github.com/kopeio/kexpand](https://github.com/kopeio/kexpand)

<comment><USER23> I do something similar, but envsubst does the job.

<comment><USER24> +1 on envsubst, it's the minimal solution to the problem of templating kubernetes (of course YMMV, we are a small team and don't need more complex stuff)

</comment></comment></comment></comment><comment><USER25> > _We enhanced GLB, our internal load balancing service, to support Kubernetes > NodePort Services._

Everyone does this - because Kubernetes Achilles heel is its ingress. It is still built philosophically as a post-loadbalancing system .

This is the single biggest reason why using Docker Swarm is so pleasant.

<comment><USER26> Any load balancer can be configured or modified to target routable Pod IP addresses and skip node ports altogether. You'll have to integrate with the Kubernetes Endpoints API[1] and support dynamic backends. Another option would be to leverage Kubernetes' DNS and the SRV records[2] backing each service.

The reason node ports are used in the Cloud today is because most Cloud load balancing solutions only target VMs, not arbitrary endpoints such as containers, a limitation that will go away over time.

[1] Envoy with Kubernetes Endpoints integration: [https://github.com/kelseyhightower/kubernetes-envoy- sds](https://github.com/kelseyhightower/kubernetes-envoy-sds)

[2] [https://kubernetes.io/docs/concepts/services- networking/dns-...](https://kubernetes.io/docs/concepts/services- networking/dns-pod-service/)

<comment><USER25> Hi Kelsey, thanks for replying. You are definitely k8s secret weapon ;)

I know about this. However, ultimately the question is that Kubernetes is not a gradual scale up solution for most people. I have to be prepared to deal with building my own load balancer.

Basically, I cannot do an on-metal deployment very easily. Most of the questions on k8s slack for metal deployments were - how do I set this up with few tweaks like ssl pass through and source ip preservation.

It is not easy.

Either you build your own load balancer or you use a cloud provided one. Now, Ingresses are not pleasant. I'm not sure about the state of source ip preservation, but last I remember that the nginx ingress had still not surfaced ssl_preread_server_name to the ingress configuration.

Now, what would have been nice is if it was ingress-all-the-way-down : ingress with something like istio/linkerd, maybe it is possible.

Tl;Dr - I'm not github. I can't build my own load balancer. Give me something that works out of the box. Yes, I know it may go down - I'll survive. Docker Swarm does this.

<comment><USER27> If you use a statefulset, there is no load balancing, regardless of bare metal or cloud. Every pod has a DNS record you can use to address all other pods, and it's carried over in case of a pod failure. Are you looking for a load balancer or not? If you are, as the other person mentioned, you can use the nginx one.

</comment></comment></comment></comment><comment><USER28> Any favorted training for learning Kubernetes?

I found this one so far: [https://classroom.udacity.com/courses/ud615](https://classroom.udacity.com/courses/ud615)

But any extra courses/trainings is always appreciated

<comment><USER29> Udemy: [https://www.udemy.com/learn-devops-the-complete- kubernetes-c...](https://www.udemy.com/learn-devops-the-complete-kubernetes- course/)

Pluralsight: [https://www.pluralsight.com/courses/getting-started- kubernet...](https://www.pluralsight.com/courses/getting-started-kubernetes)

<comment><USER30> I'm got that Pluralsight one earmarked for 'soon'. I thought Nigel Poulton's Docker courses were excellent, so looking forward to it...

</comment></comment></comment><comment><USER31> How hard (and how realistic) it is to actually get a reasonable understanding (and then stay up-to-date) with Kubernetes internals? Is there any go-to reading material?

We had ran another large-footprint container management system (not K8s, but also popular), and when its DNS component started to eat all the CPU on all nodes, best I was able to do fast,was just scrapping the whole thing and quickly replacing it with some quick-and-dirty Compose files and manual networking. At least, we were back to normal in an hour or so. Obvious steps (recreating nodes) failed, logs looked perfectly normal, quick strace/ltrace gave no insights, and trying to debug the problem in detail would've taken more time.

But that was only possible because all we ran was small 2.5-node system, not even a proper full HA or anything. And it had resembled Compose close enough.

Since then I'm really wary about using larger black boxes for critical parts. Just Linux kernel and Docker can bring enough headache, and K8s on top of this looks terrifying. Simplicity has value. GitHub can afford to deal with a lot of complexity, but a tiny startup probably can't.

Or am I just unnecessarily scaring myself?

<comment><USER32> I wouldn't say that you're unnecessarily scaring yourself at all. Kubernetes is extremely complex. I've been running it for a few months and I'm just starting to get my hands around it. Things will just stop working for what seems like no reason, and there are so many places to investigate you can easily burn most of a day troubleshooting.

It's a great system, but it's also relatively new, and most issues aren't well documented. You'll spend a lot of time in github issues or asking for help in the (very active, and often very helpful) community.

If you have a valid use case, I wouldn't steer you away from it, but your fears are well founded.

</comment></comment><comment><USER33> > Enhancements to our internal deployment application to support deploying > Kubernetes resources from a repository into a Kubernetes namespace, as well > as the creation of Kubernetes secrets from our internal secret store.

Would love to hear more about this was accomplished. I'm currently exploring a similar issue (pulling per-namespace Vault secrets into a cluster). From what I've found, it looks like more robust secrets management is scheduled for the next few k8s releases, but in the meantime have been thinking about a custom solution that would poll Vault and update secrets in k8s when necessary.

</comment><comment><USER34> One thing I would have liked to have seen addressed in the article is whether the new architecture requires additional hardware (presumably) to operate and if so how much more.

<comment><USER35> I've only dabbled in K8s and it strikes me that using it in production is a long term investment and, as it stands currently, a long term project to implement properly. You'll want to do exactly what Github did: setup a "review lab" or similarly comprehensive dev and test environment until you are absolutely comfortable with it in production. This will lead to the provisioning (and cost) of quite a bit of hardware - and when it is finally in production it'll likely be over-provisioned for quite some time until norms can be established and excess cut.

So basically its a traditional devops migration. But you get quite a few goodies and arguably much better practices at the end of it.

<comment><USER36> I agree very much, and I'd like to add one point: When you build a lab environment for testing Kubernetes deployments (and verifying Kubernetes upgrades), make sure it's on the same hardware as your production environment.

When my team did the first Kubernetes deployment, we made the mistake of building a lab environment that did not match the anticipated production environment. (Two reasons: The BOM for the production environment was not yet decided upon at that time, and the lab was frankensteined together by taking hardware out of existing labs.) We learned the hard way that, just because the Kubernetes upgrade worked in the lab, it need not work on the production hardware.

Right now, we're stuck on last year's (i.e., ancient) Kubernetes 1.4 release because no one dares to upgrade production. (There's light at the end of the tunnel, though. A new lab is being built up in the datacenter around now.)

</comment></comment></comment><comment><USER37> I'd be interested in hearing what kind of autoscaling system they use for their Ruby pods.

We're running a few (legacy — we're moving to Go) Ruby apps in production on Kubernetes. We're using Puma, which is very similar to Unicorn, and it's unclear what the optimal strategy here is. I've not benchmarked this in any systematic way.

For example, in theory you could make a single deployment run a single Unicorn worker, then set resources:requests:cpu and resources:limits:cpu both to 1.0, and then add a horizontal pod autoscaler that's set to scale the deployment up on, say, 80% CPU.

But that gives you terrible request rates, and will be choking long before it's reaching 80% CPU. So it's better to give it, say, 4 workers. At the same time, it's counter-productive to allocate it 4 CPUs, because Ruby will generally not be able to utilize them fully. At the same time, more workers mean a lot more memory usage, obviously.

I did some quick benchmarking, and found I could give them 4 workers but still constrain to 1 CPU, and that would still give me a decent qps.

<comment><USER38> At GitLab we recommend to use CPU cores + 1 as the number of unicorn workers [https://docs.gitlab.com/ce/install/requirements.html#unicorn...](https://docs.gitlab.com/ce/install/requirements.html#unicorn- workers)

<comment><USER37> How do you configure that? A pod doesn't know what machine it's running on ahead of time. You can create nodepools and use node selectors to pin the pod to that nodepool, but I'm not sure I love the idea.

<comment><USER39> Our entrypoint configures the unicorn workers before starting using a chef omnibus call. So we grab the memory and cpus using Ohai: [https://gitlab.com/gitlab-org/omnibus- gitlab/blob/master/fil...](https://gitlab.com/gitlab-org/omnibus- gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb#L303) and do cpus+1

^This is pretty much we do by default for all our regular package installs, but in some of our Kubernetes Helm charts we instead statically configure the pod resources and unicorn workers. (Defaulting to 1cpu - 2workers per front- end pod) eg: [https://gitlab.com/charts/charts.gitlab.io/blob/master/chart...](https://gitlab.com/charts/charts.gitlab.io/blob/master/charts/gitlab/values.yaml#L198)

As someone mentioned in this thread, using the downward api might be a cool way to configure the workers.

</comment></comment></comment></comment><comment><USER40> Curious, being a RoR app, did github ever run on Heroku? (Obviously googling "github heroku" is just a million tutorials on how to integrate.)

<comment><USER41> EY, Rackspace, and then our own metal.

</comment></comment><comment><USER42> hey! we at bitmovin have been using k8s for quite a while for our infrastructure and on premise deployments. In case you're interested in how we do multi stage canary deployments, check out: [http://blog.kubernetes.io/2017/04/multi-stage-canary- deploym...](http://blog.kubernetes.io/2017/04/multi-stage-canary-deployments- with-kubernetes-in-the-cloud-onprem.html)

</comment><comment><USER43> I'm curious to what this means for the existing puppet code base, is it now irrelevant, or are there still usages for it in the k8s world?

<comment><USER44> They could easily still use standalone puppet to handle the config management for individual container images. I currently do this with salt-minion. It reduces the burden on the Dockerfile itself, and lets you embrace a declarative configuration state at build time.

<comment><USER45> It definitely seems like the wrong approach to me to have puppet manage your base images. They're not VM's, they shouldn't have multiple services, they shouldn't require any complex configuration management, they should just be the minimum requirements to support your application's local runtime dependencies, and that's it.

From previous experience migrating from a puppet setup to one that used containers, puppet's vestigial use case ends up being to get the orchestration control plane itself setup (ie. kubernetes, networking configs, etc) and that's about it.

<comment><USER46> There's nothing inherently about Puppet that means it has to manage multi- service "whole OS"-like installations. It can just as easily be put to the task of a Dockerfile: install dependencies and deployables for a single application. Its robust ability to manage things like user accounts, packages, scheduled jobs (e.g. for alerting, though you would have to install at least _a_ second service for this: _crond) and the like makes it vastly superior to Dockerfile shell scripts for complex tasks.

Think of puppet more like a way of simplifying your Dockerfiles to have fewer crazy shell commands _in total_ , rather than hiding the craziness in layers and hoping it all composes properly. If you do use lots of layers, Puppet can make your life much easier, since it can be better at detecting previous layers' changes and working around them (think redundant package install commands. Even the no-op "already installed!" command takes time; if you're installing hundreds of packages--many people are, for better or worse--that can eat up build time).

Puppet isn't a VM provisioner; it can also be used as a replacement for large parts of your Dockerfile, or a better autoconf to set up the environment/deps for your application to run in.

Edit: syntax.

<comment><USER44> The point about layer complexity is a great one I didn't even consider. Your "config" step is no longer a mish mash of dozens COPY/RUN/etc directives (resulting in N new intermediate image layers), it just results in a single atomic layer where you run the Puppet bootstrap.

Obviously you could accomplish this with shell scripts as well to constrain your config step into one docker RUN directive, but I prefer the declarative state approach to the imperative one in this case.</comment></comment></comment></comment></comment>


Rents are dropping across the US - <AUTHOR> https://edition.cnn.com/2020/06/16/success/rents-are-dropping-us-cities-coronavirus/index.html <comment><USER2> "across the US" as in "in expensive coastal cities."

I don't know the broader trend but, anecdotally, I know people who have/are in the process of moving out of cities for more rural locations. One couple I know just moved out of a major Northeast city to the coast of Maine. Others are seriously considering vacating NYC etc.

With many tech industry jobs mostly WFH until the end of the year and the future in general uncertain, a lot of people, especially those with leases coming up for renewal, are thinking this is a good time to make a move.

ADDED: I suppose this is good news for people looking for cheaper city housing prices. Maybe not so good for those who were especially attached to either the vibrant city life or busy office life.

<comment><USER3> > Maybe not so good for those who were especially attached to either the > vibrant city life or busy office life.

I don't understand this reasoning. Was city life not vibrant when rents were 1/3rd the price?

If anything, "hypergentrification" tends to have a sterilizing effect on city life.

To be honest I'm kinda looking forward to it. Nothing against HENRYs (High earners, not rich yet)... but their "cultural tastes" are both expensive and not for everyone.

<comment><USER4> Maybe this sounds strange, but I think there is also a category of people who prefer suburbs or exurbs and are really bitter that the pendulum has swung back to cities in the last ~20 years.

They are really eager to see a trend out of small amounts of data. Or proclaim urban life dead, now and forever.

I personally feel strongly in the other direction. We had the infamous "white flight" and car-centric communities and life. We had the peak of urban crime in the 80s and 90s. Then we had a reversal of those trends. Yes gentrification is tough on many, yes the costs are out of control for many, but on the balance I think denser living is the right call.

<comment><USER5> Kids change mindset.

Kids love yards. They love the little creek that runs past.

I love the safety. Small neighborhood, we all know each other and look out for the kids that run around playing.

<comment><USER6> I bought into this mindset, and regret it. That's what you do when you have kids, move to the 'burbs, right?

We moved from a big city to suburban hell a few years after having a kid. In those first years, we took our kid to museums, parks, playgrounds, etc. all over the city. Now there is jack shit to do in the suburb we're in except going to the mall, cannot walk anywhere, have to use the car 100% of the time to even go to a shop. A backyard is a sad place compared to something like Central Park. I feel so sad for my kid. If the prices keep going lower in cities and remote work becomes the norm, we're moving back. (Un)fortunately my wife hates it here too, and we've often talked about moving back to a city.

<comment><USER7> With you on this. There was never a lack of things to do in new York and with a tiny apartment little incentive to stay home.

Back in the suburbs now and the kid would spend his Saturdays watching TV if I didn't stop him because the alternative is ... the one playground in driving distance? The movies? Target?

People really underestimate the volume of activity you can put in front of a young kid in a city.

<comment><USER8> Some things I've observed about families I know who seem to have suburban life figured out:

\- regular (meaning periodic) physical activity is essential, for the adults and kids. Sports leagues, gym classes, whatever. If you're not into sportsy physical activity, you need a physical hobby like woodworking, gardening, or literally anything to get your blood flowing

\- your weekly schedule looks totally different as the seasons change. Skiing in winter, hiking and camping in the fall, beach vacationing in the summer, etc

\- regular domestic vacations to escape the boredom, and to make you long for home.

\- regular home improvement, to make your life in your house 1% better every month

\- regular visits with nearby close friends and family. At least one visit to a close relation every two weeks. Non negotiable, and ideally tied to holidays

\- overdo your holidays. Birthdays are huge, winter holiday (Christmas etc) is huge. Thanksgiving is huge. It has nothing to do with the meaning of the holiday and everything to do with getting people together.

These kinds of things are less achievable in the city because you have less money, less space, less autonomy. Just my observations from happy suburban families, I'm curious to see others' notes.

<comment><USER9> What you've described sounds pretty suffocating to a kid. Growing up in the city my parents didn't have to force me to do a bunch of mundane bullshit, there was so much to do I was able to find cool stuff on my own.

<comment><USER8> I mean yeah, if you force your kids to do this stuff they're going to hate it? I think you might be discounting the fact that many kids love to do these kinds of things, and also the role the kids play in deciding what to do.

The idea is to expose your kids to a wide range of activities, and support them doubling down on an evolving handful of them. Avoid setting super specific constraints (you must play a team sport), and instead try setting goals (find a physical activity you enjoy and find motivating). From my perspective, parenting is about exposing children to their options, and making some choice recommendations where you have experience.

Parents can do that anywhere, and fail to do that anywhere. I do agree that if your parents aren't doing a good job of that in a walkable city, kids are more likely to discover those options on their own.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> If I were a renter right now and in an urban area with a lease renewal coming due, I'd 1000% call the mgmt co's bluff.

Context: my firm provides back office services to 50k+ rental units across the nation and unless you're in some oddly hot market, most mgmt companies currently are holding their breath that their renters don't say anything and renew at the same price.

If you're reasonable, you can easily negotiate a price drop and the mgmt co will most likely accept it begrudgingly because they do not have the data to call YOUR bluff. every mgmt co right now is taking whatever they can get and locking it down for as long as they can.

happy to respond to individual folks questions but keep in mind that any advice I may provide is not legal nor financial and I guess I should say my posts/comments are my own and do not represent my company's. =)

<comment><USER11> We just canceled our lease two weeks ago. We had a large apartment in one of the new high-rises in Downtown LA.

The last offer was 3 months free and 2 parking spaces free if we stay for 12 month lease, which comes to 34% discount on the rent including parking.

Other buildings are offering by default two months rent-free and they will pay the movers if you decide to move in.

DTLA is a special place as it has plenty of new supply, but I can imagine that it's not too different in other places across LA.

<comment><USER12> I'm guessing the freebies are to avoid lowering the actual rent due to Prop 13?

<comment><USER13> I think you mean rent control, not prop 13.

Anyhow this commenter sounds like they know what they're talking about and are saying that landlords in SF _cannot_ beat rent control in this manner: [https://socketsite.com/archives/2020/06/complimentary- rent-o...](https://socketsite.com/archives/2020/06/complimentary-rent-on-the- rise-in-san-francisco.html#comment-394729)

<comment><USER12> Ack, you're right, I did!

</comment></comment></comment></comment></comment><comment><USER14> > She and her roommate signed a lease on a two-bedroom apartment in the West > Village for $4,995 a month

Holy cow, I knew NY rent way expensive, but wow I didn't know it was that bad. Growing up in upstate NY, you could be a nice family ready home in my suburban home town for ~100K. Of course there were houses that went up to ~150K or even 200K, but 100K was standard for purchasing a home. I even though that that was a lot.

I'm working in RI right now, renting a room outside the city, and I pay ~$500 a month for a room and all utilities. Have I just lived in very cheap places and the rest of this country is very expensive, or is it really just the big cities that have extremely expensive housing?

<comment><USER15> It's not just "big" cities, it's all cities at least according to this source [1]. I grew up in RI but I moved to Seattle after graduating college because I didn't want to live in RI anymore but job prospects for tech increase when you're in a city.

I wonder what the cycle to expensive rents is? Wages from jobs start to increase in an area, some housing development happens, rents go up on those new developments to cover costs but also because their renters can afford it, repeat?

1: [https://www.zumper.com/blog/rental-price- data/](https://www.zumper.com/blog/rental-price-data/)

<comment><USER16> > Wages from jobs start to increase in an area, some housing development > happens, rents go up on those new developments to cover costs but also > because their renters can afford it, repeat?

The increased wages (more jobs + better jobs) leads directly to both new housing development AND increased demand for whatever housing currently exists. The latter is more impactful for higher prices than the former.

> but also because their renters can afford it, repeat?

This isn't quite how things work. Most landlords are always going to raise the rent to whatever the market can bear. Of course, how much people can afford does factor into that, but it's not everything.

I can _afford_ to buy milk that costs 10x as it does right now, but I wouldn't actually pay that because I have no reason to when cheap milk still exists. Landlords are able to raise rents when the demand for housing goes up faster than the supply, such that renters don't have better options available.

<comment><USER17> >>> I can afford to buy milk that costs 10x as it does right now, but I wouldn't actually pay that because I have no reason to when cheap milk still exists.

You can afford to not drink milk but you can't afford to not have a roof over your head.

Becoming homeless is not an option so people get pushed to living in older smaller flat away from the city or sharing with strangers.

<comment><USER16> Okay fine, I could say, "I could afford to pay 2x for groceries in general, but I won't, because why would I when I don't have to?"

Same principle for rent. It's not quite about what people can afford, it's about what they feel they have to pay (and how much they can reasonably afford is one factor in that).

<comment><USER17> Rent is precisely about what people can afford. It's no accident that rent is around 50% of local income, in all cities across the developed world.

<comment><USER16> No, rent is about supply and demand. What people can afford -- how much money they have -- is a component of demand, but supply also matters.

If you look at countries that restrict supply less than the US, you'll notice that rent doesn't go to such high extremes in booming cities. For example, in Germany, the most expensive cities, the ones that have been booming, have average rents that are around 40-50% higher than the national average. In contrast, in the US, the most expensive cities have average rents easily double or more the national average. Sometimes even getting close to 3x.

You could also look at NYC vs Tokyo. Japan has much looser restrictions on housing supply and it shows here. NYC has 50% higher salaries, but rent isn't 50% higher, instead it's nearly 3x as high: [https://versus.com/en/new-york- vs-tokyo](https://versus.com/en/new-york-vs-tokyo)

<comment><USER17> Agreed on supply. Might want to look at the 75% of household disposable income or exclude non-workers. Only the best earners get to live in the city.

Germany is a relatively equal country. A worker in the capital doesn't get double salary as the same position in the countryside. In fact there is hardly anybody who earns triple as of anybody else in the country.

The US is extremely unequal. A position in NYC can pay numerous times than a similar position in the countryside, then even for a given position in NYC two companies across the same street can vary from simple to double. Rent in NYC is naturally triple to reflect the triple in income.

<comment><USER16> > In fact there is hardly anybody who earns triple as of anybody else in the > country.

Bullshit. I live in Germany, there's somewhat less income inequality than the US, but this is a gross exaggeration. A programmer in Munich easily earns triple or more than what someone gets at a random McJob like at Starbucks, also in Munich.

The bigger reason for why German rents are lower is simply that they allow for way more housing supply. In the US, even in major cities it's common to have wide swathes of land zoned for mandatory single family homes on large lots. In contrast, Germany doesn't have any such zoning _in the entire country_. You're always allowed to make at least some level of multi-family housing.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER18> Seems silly to discuss the falling rental prices in prestigious cities without even a nod to AirBNB.

AirBNB rentals have _collapsed_ , and so a lot of owners have moved their properties into conventional long-term rental. This has created a glut in the rental market, and is driving prices down.

[https://www.straight.com/news/some-vancouver-airbnb-units- ar...](https://www.straight.com/news/some-vancouver-airbnb-units-are-being- converted-into-long-term-rental-dwellings)

<comment><USER19> A report came out that showed the effect is super weak: [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3006832](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3006832)

"1% increase in Airbnb listings is causally associated with a 0.018% increase in rental rates and a 0.026%"

This is about a fifth of the overall increase. We're talking about a $9-10 a month difference in rent in most urban areas. Not great, but clearly Airbnb is a small part of the problem.

<comment><USER20> You seem to be misinterpreting the study. The $9-10 a month number is for the median zipcode in the US, which is unlikely to be an urban area. And it represents the difference for each 1% increase in Airbnb listings. So a median zipcode with 10% more Airbnb listings would see a ~$100 difference in rent from a similar median zipcode.

Assuming that 'a fifth of the overall increase' is correct, that means that a place like SF can attribute at least a few hundred dollars a month in rent increases to AirBnB, as the median rent has increased by almost ~$2k since AirBNB came into existence.

</comment></comment></comment><comment><USER21> A lot of engineers in my group as well as a few friends are leaving the SF bay area. Now that working there is no longer a requirement and everything is shut down, why live there? Oregon, Nevada, Colo, (cheaper areas of CA), all are places that they are heading to. Main reasons are cost, crowds(lack thereof) and overall better quality of life. Expect this trend to accelerate once more leases end.

<comment><USER22> And in my circle - none are leaving.

We all recognize that leaving the area means leaving job opportunities behind too. We've seen what happens to engineers career trajectory who are remote at an in-person company.

<comment><USER23> So we can expect the population of the valley to shift towards even more career minded over achievers while normal folks flee. Great.

<comment><USER22> I mean - did we ever expect the trend to reverse? This region is built on the idea of professional opportunities (FAANG, bleeding edge tech, etc.) and the possibility of getting rich if you join the right startup (aka - buy the right lottery tickets/exercise-options).

I've only seen it get worse as I've spent my time here. Has it ever been worse in terms of wealth disparity? I think it will continue to get worse short of radical change. (Prop 13 dying, rezoning w/ hundreds of thousands of apartments built, radical public transit, etc.)

<comment><USER23> > This region is built on the idea of professional opportunities

I'm pretty sure the region existed before tech blossomed half a decade ago. It was definitely a different place even 20 years ago. Also, like 99+% of folks here don't work for a startup.

The people with the money don't seem to care about fixing the region's problems. The other people can't afford to. This area seems like it will just continue devolving into some sort of dystopian, cyberpunk future.

</comment></comment></comment></comment></comment><comment><USER24> I don't think TFA is supported by the Zumper data

Looking at the linked Zumper[1] data it doesn't seem that dramatic outside of SF's 9.2% drop. NYC 1 bedroom is down 2% Boston 1% and SJ 0.4%. Interestingly Oakland is UP 4.9%. So the story is different than the one they seem to want to tell.

[1] [https://www.zumper.com/blog/rental-price- data/](https://www.zumper.com/blog/rental-price-data/)

</comment><comment><USER25> That hasn't been my experience. My lease expires mid August, and I already got the lease renewal. The same level as the current one, not a dollar less. We started looking around. Everywhere the same, same level as the current one. I didn't get to ask about free months yet, maybe management companies will offer one-two months. But the actual rent per month is not going down, despite a significant number of vacancies.

<comment><USER26> Negotiate. Tell the manager they've been great, but ultimately it comes down to dollar price for you. Let them know you're starting to look elsewhere unless they can lower the rate. When they reply, even if you like the rate, don't tell them that. Tell them it's not what you are looking for; that because of work and expenses, that rate isn't as affordable. Give them another chance to offer another lower rate or add a perk such as a free month or two.

</comment></comment><comment><USER27> >At the end of May, Ilana Freund landed a deal on an apartment in New York City where she will be attending graduate school. She and her roommate signed a lease on a two-bedroom apartment in the West Village for $4,995 a month. It was a significant discount: Similar apartments in the building were going for around $5,300 before the pandemic took hold, according to the listing agent.

wow a whole 6% reduction. Odds are next year it will be back at $5,300. One way to offset this reduction is to increase the mandatory down-payment or increase min. duration of the lease.

<comment><USER28> it's primarily a big deal because this is the end of YOY increases that lasted for more than five years

</comment></comment><comment><USER29> I for one am ready and rarin' to go for a housing market crash (or "correction" if you wanna call it that) here in NYC, and strongly encourage everyone who doesn't feel like living here to strike out for the heartland. It'll be a better city all around with fewer people and lower rents.

<comment><USER30> I've been thinking about this and the problem I keep circling around is that basically everyone I know says the exact same thing (waiting for a housing market drop so they can buy in). That implies that the latent demand in the market is _huge_ and any price drops will immediately be gobbled up by an army of millennials who've been waiting for their chance. The question to me is if there are more homes existing owners can't afford anymore or more buyers with enough money to pick them up when they drop a few percent.

<comment><USER29> This is an annoyingly good point, honestly.

I'm mostly hoping that a lot of the other millennials who have been waiting their chance are planning to skip town for some cheaper suburbs.

</comment></comment></comment><comment><USER31> That doesn't sound like much of a discount, what, 5%?

Without moving, I got 40% off my rent for six months in the middle of London. I'd encourage tenants who haven't asked for a discount yet to do so.

<comment><USER32> What do you say?

'Give me a discount or...' or what? You'll move out? Sounds like moving would out be an absolute nightmare at the moment. Are you going to go flat-hunting in the middle of a pandemic?

<comment><USER33> I've thought about reaching out to my landlords and ask if they would "consider aligning with the current market".

I haven't done it because they've been good landlord, but at the same time I'm paying an insane rent (I live in SF) for a pretty average 1 bedroom...

But what if rent continues to decrease, can't ask twice.

</comment></comment></comment><comment><USER34> This seems more like a market correction than an actual rent drop.

> She and her roommate signed a lease on a two-bedroom apartment in the West > Village for $4,995 a month.

I struggle to believe that there are two bedroom apartments that have that kind of demand/value outside of an inflated circumstance.

<comment><USER35> > This seems more like a market correction than an actual rent drop.

An "actual rent drop" _is_ a market correction.

</comment></comment><comment><USER36> If we needed any more evidence the way to lower costs in the long term is building more housing. Supply and demand works people!

<comment><USER37> Except this proves the opposite. When we built, prices skyrocketed. When we _stopped building_ (due to COVID), the prices began dropping.

It's a combination of AirBNB being unable to illegally occupy housing, and the ultra-wealthy having a fear of urban density contagion. Demand is basically unchanged over the last 6 months, and the housing that exists is basically unchanged over the last 6 months, and yet prices are finally falling.

It's got basically nothing to do with "supply and demand" for cities (unless your definition of "demand" is so loose as to include literally anything that might impact a pricetag).

<comment><USER35> > Demand is basically unchanged over the last 6 months

I'm calling major BS unless you can back that up with a source. This article talks about the major exodus in NYC: [https://therealdeal.com/2020/06/02/new- yorkers-exodus-could-...](https://therealdeal.com/2020/06/02/new-yorkers- exodus-could-unravel-rent-regulation/)

</comment></comment></comment><comment><USER38> Not only are they dropping, the incentives being offered around NYC are pretty nice. 50% off security deposit and 1 month free are what I've recently received. I checked back to see a week or two after and they were offering 2 months for free with 50% off the deposit.

<comment><USER39> Are they still doing salary requirements of 42x monthly rent unless you have a guarantor?

<comment><USER38> Yeah I don't think they've budged on that in NYC much yet unfortunately.

<comment><USER39> It's quite frustrating how hard it is to get an apartment in NYC without a proper job, e.g. as a freelancer. If they even answer you, they want you to put a ton of money down up front.

</comment></comment></comment></comment><comment><USER40> There's a couple of things worth at least considering. The article states:

>"The two roommates were also given one month's rent free and did not have to pay a broker fee, which typically would have cost them 12% to 15% of the annual rent."

There was actually legislation passed back in the Winter that effectively did away with the whole broker fee system.[1] I know a judge temporarily blocked this as per the article but that was only until June.

>'"But, he said, more apartments are also available because people just up and left their leases. While many left when the pandemic shut down the city, others left later when the protests over the killing of George Floyd began, Gavzie said.'

Generally you are legally obligated for the rent for the duration of your lease. So while a renter can "just up and leave", if they don't fulfill their obligations they can be sued. So if by upping and leaving a tenants pay's the remainder of their financial obligation then it would make sense that a landlord would give them a discount since they may be collecting twice on the same place for some period time.

[1] [https://ny.curbed.com/2020/2/11/21132836/nyc-broker-fees- law...](https://ny.curbed.com/2020/2/11/21132836/nyc-broker-fees-law-real- estate-tenants-rights)

</comment><comment><USER41> > 'Pandemic pricing' is here. Rents are dropping across the US

I don't understand the negativity, are most people here are rejoicing that they get cheaper rents in major cities?

<comment><USER42> CNN is acting as a mouthpiece here for landlords. Landlords don't want rents to go down, so they are reacting negatively to this news.

</comment></comment><comment><USER43> LOL say's California! I have seen steady rent prices and decreasing housing prices.

<comment><USER44> Even within California it's mostly just a shift from urban areas to suburban and rural. For instance, leaving San Francisco and moving to Sacramento or Tahoe or Santa Cruz.

</comment></comment><comment><USER45> One dark thing the article doesn't seem to mention: what about COVID-19 evictions?

Have to get new tenants when the old ones are being kicked out or going back home to live with their parents.

<comment><USER46> You're correct, it's dark thing and that no one is talking about it.

How do courts open back up when 30%+ rentals are overdue on rent?

</comment></comment><comment><USER47> "Conveniently" just got a notice from my property to renew my lease with a slight increase in rent. July 4th seems like a pretty good time to look for alternatives!

</comment><comment><USER48> Rents, and house prices, are going _up_ where I live, reportedly as a result of all of the people leaving California cities and moving here instead.

</comment><comment><USER49> This articles takes 4 pieces of anecdotal evidence and concludes rents are dropping; there is no study, no calling around, no use of modern technology. Just 4 anecdotal pieces of evidence.

Then everyone on HN is going to start talking about their anecdotal evidence which does exactly nothing for anyone.

This is exactly the kind of drivvel and BS that does not belong on HN or frankly, published in 2020.

</comment><comment><USER50> [https://xx.reddit.com/r/AskNYC/comments/h9nzrg/my_landlord_i...](https://xx.reddit.com/r/AskNYC/comments/h9nzrg/my_landlord_is_dropping_my_rent_200_when_i_reup/)

</comment><comment><USER51> It's really not a big change. The rent on the market is the visible marginal rate and skews upwards because of locations that are way above their market rate being not taken for a long time.

</comment><comment><USER52> One wonders if this is, perhaps, explained by a lot of people holding on to rental properties as airbnb units had to give them up, so there are lots of units available and prices are coming down.

</comment><comment><USER53> Incidentally, I just opened a large urban apartment community in downtown Memphis, and we are leasing up at 2x the rate we planned on, for higher rents than we planned on.

</comment><comment><USER54> Supply & demand actually works, even for real estate.

</comment><comment><USER55> Nearly $8000 for luxury one bedroom apartment. Money to burn. My house payment is about $1200 per month for 2800 sqft in one of the nations largest cities and that includes property tax. Plus I have equity on the house which has doubled in value over the last 10 years. Financially I really do not understand dumping cash into a lease without equity.

</comment><comment><USER56> I have a feeling commercial real estate is next in about 3-6 months

</comment><comment><USER57> > "It has been crazy," Freund said, "but we definitely got a very good deal."

This reads like a parody to me. Paying 5000 dollars per month in rent and calling it a good deal. That's more than what many people even make in salary.

<comment><USER58> I'm not sure why that's a contradiction. If you could buy a nice new private jet for $100,000 it would be reasonable to call that a good deal, even if it's more than what most people make in a year.

<comment><USER37> It's a contradiction because a regular 2-bed apartment is a basic essential necessity for human life, and not an extravagant ultra-luxury like a nice new private jet is.

<comment><USER51> I've been living in a 1-bed apartment for most of my adult life. Hasn't felt like I've been missing a necessity.

</comment></comment></comment></comment><comment><USER59> This is interesting, immediately before Covid-19 hit NYC I moved to mid-town Manhattan in early March.

Compared to Boston (where I'd previously lived for about 7 years) I was stunned how affordable (with roommates) luxury housing was, at least during "off-season".

For the master suite in a unit with a clear view of the Hudson and the USS Intrepid (I actually had a great view of the USS Comfort while it was in port) I only pay $1900/mo. with free gym perks, laundry in unit, AC, with brand new construction standards (building was constructed in 2018). It's not cheap, but by NYC standards when correcting for the quality of the building / building's I've lived in around Boston it's unparalleled.

Even if I moved to Austin TX, I'd essentially only be saving about $400/mo. to have a decent apt to myself but also be getting paid substantially less. It's going to be very interesting in a year whether I negotiate my rent down, or look for a new place to live and bargain HARD.

That said, I'm an NYC rental noob, very open to what kind of luxury rents people are seeing now in a post-covid rental environment?

<comment><USER60> I vouched for you to get this comment out of dead. I think you may be shadowbanned for your first comment on your account? Talk to the mods.

<comment><USER17> Might have got downvoted for the nonsense of "luxury with roommates". There is no luxury in having to share living space with 3 strangers. This resonates very strong with members over 30 and couples, it's the sign that having a roof for yourself let alone your family is totally unaffordable.

</comment></comment></comment><comment><USER61> On a related topic, for those of you who maybe unemployed or know someone who is unemployed.

Beware of rumor spreading when you are looking for work there are companies that are setup for a single purpose. They keep track of the recently unemployed and specialize in spreading rumors at every and any company that you interview at (They masquerade as in-person advertisers or MLM companies). They do this to keep you unemployed long enough so that they can force you into certain places at ridiculously low salaries, they then keep you in debt slavery to force you to purchase from certain places that landlords or shop owners benefit from. This happens all over the country and these vultures are taking advantage of people all over the place.

<comment><USER62> Citation?</comment></comment>


Trump Exec Order Could Block 500K Legal Residents from Returning to America - <AUTHOR> https://www.propublica.org/article/trump-executive-order-could-block-legal-residents-from-returning-to-america <comment><USER2> What the hell is up with the article's title? 500,000 people from these countries have received green cards in the past decade.

That doesn't mean that all 500,000 are currently out of the country and unable to return.

Seems like excessive hyperbole to me.

<comment><USER3> The title of the article says "... from trips". It's the HN title That's stupid.

</comment></comment><comment><USER4> So I suppose this is out now:

"Give me your tired, your poor, Your huddled masses yearning to breathe free, The wretched refuse of your teeming shore. Send these, the homeless, tempest- tossed to me, I lift my lamp beside the golden door!"

</comment><comment><USER5> Article title: "Trump Executive Order Could Block 500,000 Legal U.S. Residents From Returning to America From Trips"

Suggested alternate: "Executive Order Could Block 500K US Residents From Returning to US From Trips"</comment>


Python string formatting and UTF-8 problems workaround - <AUTHOR> http://blog.endpoint.com/2015/07/python-string-formatting-and-utf-8.html <comment><USER2> Well... That, or you just use Python 3, which already supports UTF-8 as default encoding for strings...

<comment><USER3> I'm pretty sure that is not true. I don't think that py3k strings even have a default encoding per se.

<comment><USER4> Text mode file IO defaults to a platform specific encoding:

[https://docs.python.org/3/library/functions.html#open](https://docs.python.org/3/library/functions.html#open)

(which will probably be UTF-8 on recent systems that are not Windows)

</comment></comment></comment><comment><USER5> "workaround" is the operative word here, rather than fix the issue of their program being unicode-unaware (despite getting unicode data from their XML API as should be) they just sprinkled .encode calls until the issue disappeared, then fabulated something to explain it.

</comment><comment><USER6> Okay, someone with a blog just learned he had to use Unicode because not everyone speaks English and not all text is ASCII.

Why is this an interesting link?

<comment><USER7> It's not and is actually misleading as people who do not already know this are not gaining any concrete information, just an anecdote.

</comment></comment><comment><USER8> For those interested in Python 2 string encoding, I've spent some time writing about it, here:

> > [http://stackoverflow.com/a/6539952/19212](http://stackoverflow.com/a/6539952/19212)

</comment><comment><USER9> The fix isn't quite right. It may technically produce correct output _now_ , but it's sloppy. The sloppy code is brittle and dangerous and perfect food for bugs, but that's a minor problem. After all, a single mistake or a small bit of sloppy code can only cause a few bugs at most. The major problem is that the sloppiness indicates a possible lack of understanding. A misunderstanding can continue to produce bugs and brittle code indefinitely. Misunderstandings are the devil!

The symptom is that the .encode() comes too early. The general principle is to .decode() as early as possible and to .encode() as late as possible. The results of .encode() should be as temporary as possible -- preferably never even assigned to a variable.

Seeing the encode in the wrong place leads me to suspect that the author is confusing byte arrays and strings. These are two distinct things, but most documentation makes that distinction clear as mud.

The key thing to realize is that strings are not bytes, and bytes are neither characters nor strings. Think of strings as abstract data structures, like hash tables or linked lists. Bytes are binary integers. On the surface, byte arrays are integer arrays, not strings nor hash tables nor lists of objects.

Programs interface with the world via bytes. Files are bytes. The ntetwork is bytes. Everything is bytes. Bytes are not strings. A byte is an 8-bit integer. When you do I/O and get bytes from the world, you must deserialize them into whatever abstract data structure they represent. Ignore the C language and it's misnamed "char" type. A string is an abstract data structure, as is a hash table, or a list. In some sense even binary integers are abstract data structures that need to be serialized. String serializations are called "encodings". Binary integers are serialized by choosing a byte order (big or little endian). There are various standard ways to serialize hash tables and lists, like json, various XML formats, python's "pickle" and "shelve", etc.

When you get bytes from the network or a file and those bytes are supposed to represent a string, you must deserialize the bytes into a string object. This is called _decoding_. Often you're using a web framework or other library that does this for you. Python 3's file objects do it. If it's not done automatically, then you must do it yourself. You or your framework should decode bytes into a unicode string object as soon as possible. You should do this everywhere that you do input, and then leave your strings as strings for as long as possible. Do all of your operations on strings ("unicodes"), not bytes. You parse strings, join strings, replace characters in strings, trim, find lengths and match regexes on strings ("unicodes"). Doing any of those operations on byte arrays is nonsensical and will lead to bugs. Only when you have your final string completely ready to go should you worry about serializing it for printing or to write to a file or the network. Only then, at the last possible moment, should thoughts like utf-8 or ascii enter your mind.

As written, it's unclear whether "freqs" containing byte arrays or unicode strings. Getting that mixed up can result in failing to find and item which really is in the dict, or miscounting frequencies, or it can even cause more UnicodeEncode/DecodeErrors. By decoding as early as possible and encoding as late as possible, such sneaky bugs are much less likely to occur.

In Python 2, I would have fixed the problem like this:

      for e in results:
          simple_author=e['author'].split('(')[1][:-1].strip()
          if freqs.get(simple_author,0) < 1:
              print ("%(date)s -- %(author)s -- %(title)s" % {
                  'date':   parse(e['published']).strftime("%Y-%m-%d"),
                  'author': simple_author,
                  'title':  e['title'],
              }).encode('utf-8')

You might dislike my multi-line print and would prefer to .join() a list, possibly with a temporary variable. Or, maybe you'd prefer the newer .format(). Regardless, the important point is that the .encode() should happen later than it does in the article.</comment>


Silicon Valley Has a "Problem" Problem - <AUTHOR> https://medium.com/life-learning/silicon-valley-has-a-problem-problem-b34437a57e99 <comment><USER2> Slow clap. Yes. Let's keep it all in perspective, and maybe even work on things that aren't super profitable in financial terms.</comment>


Sparklines for members of online communities - <AUTHOR> http://www.tipjoy.com/our2cents/2007/08/sparklines_for_members_of_onli.html

<comment><USER2> I like this idea. For the purposes of a forum, you might want it to be more iconic (read: smaller), so maybe a really rough bar graph of like one bar per week over the past month or so. At that rough of a level, you can also easily push the generation to a weekly cron job (rather than trying to keep up with every new post).

<comment><AUTHOR> Thanks. I've been thinking about your advice and I made a quick mockup of sparklines which show 4 bars, each representing a week, where height is average popularity over that week, and darkness of the bar is how many posts there were that week: [http://www.tipjoy.com/our2cents/2007/09/sparkline_update.htm...](http://www.tipjoy.com/our2cents/2007/09/sparkline_update.html)</comment></comment>


Papers 3 for iOS is now free - <AUTHOR> http://blog.papersapp.com/papers-for-iphone-and-ipad-now-free/ <comment><USER2> Absolutely love Papers. I think anyone doing serious work with research papers/citations will need the tools from the desktop app, but for reading/organizing papers the iOS app is great on is own.

</comment><comment><AUTHOR> The Mac app is still a paid app, but given the iOS ecosystem, it seems they decided it was not worth trying to charge for it. It's a bit sad but a great deal!</comment>


Apple Schedules Event For September 12 - <AUTHOR> http://techcrunch.com/2012/09/04/apple-schedules-special-event-for-september-12/ <comment><USER2> I like how they work with shadows in the poster. I wonder what is the direction of the light to create that shadow.

<comment><USER3> The direction of light is "photoshop" :)</comment></comment>


Show HN: Updated Hacker News Chrome Extension - <AUTHOR> http://blog.tommoor.com/post/45072871249/hacker-new <comment><USER2> One of the features the Reddit Enhancement Suite provides is the ability to collapse _all_ child comments for an entire thread, and I find it immensely useful. In fact, that and the "view all pictures" features are really the only two I find useful in that extension. OP should add the former to this plugin.

<comment><AUTHOR> I could definitely provide this, if you wanted to open an issue i'll get round to it soon enough...

</comment></comment><comment><USER3> For anyone interested, here's a version that highlights the original poster in comment threads: <https://github.com/killerbat00/HackerNew>

</comment><comment><USER4> I really like the extension but what I'm missing is auto refresh and opening links in new tabs (both comments and articles). That functionality is offered by another extension ([https://chrome.google.com/webstore/detail/hackemup/abjhkncpe...](https://chrome.google.com/webstore/detail/hackemup/abjhkncpebiaepcpeopknaghepekbnpm)) but sadly both extensions seem to be incompatible (after browser restart).

</comment><comment><USER5> Really digging this. For those who prefer light on dark (such as myself), I made a few minor tweaks. It's not perfect, but mostly usable. You can get a copy at <http://github.com/michaelmior/HackerNew>. Just clone somewhere and go to chrome://extensions, click "Load unpacked extension" and select the folder you cloned to.

<comment><AUTHOR> nice, glad you could tweak to your liking!

</comment></comment><comment><USER6> Very cool. I like some of the stuff you've done but the css conflicts with another HN extension that I enjoy ([https://chrome.google.com/webstore/detail/readable- hackernew...](https://chrome.google.com/webstore/detail/readable- hackernews/jnnojalnifmaekkfknejbppafnaiepjl)) I think their css is easier to read but I really like what you're doing in the menus.

</comment><comment><USER7> Nice to see progess! I used this extension for over a year.

I've since switched to HN Enhancement Suite because it fixes my biggest gripe: horizontal scrolling in comment threads. I just tried this out and it doesn't fix this. I'd love to switch back, perhaps if I get time I'll submit a pull request.

</comment><comment><USER8> FWIW, when I try installing the extension, it says "Manifest file is invalid" and the installation fails.

<comment><AUTHOR> Strange, it's definitely valid according to the docs and Google also processes the manifest on upload - are you all on Linux?

<comment><USER8> Chrome 25.0.1364.160 on OS X.

I think I also tried installing it on a Windows computer, too, though, and got the same error.

</comment></comment></comment><comment><USER9> I like the functions you added in. Here's a few constructive criticisms:

1\. I don't want to use this because it doesn't _feel_ like I'm on Hacker News. I miss the orange. This is obviously subjective.

2\. It's mildly annoying to have a hover bubble crop up when I trail the mouse across a user name without meaning to. It's distracting. Maybe you could add a higher threshold for registering the hover? This way it's deliberate most of the time.

3\. The endless scrolling isn't as seamless as I would hope. I find this is a common problem with "endless" pages - it never loads before you reach the end of the page, which gives a slight refractory period where you've very aware you've reached the "end" of the page.

I mean overall I'd use it if it still fit the color scheme. Just some food for thought. Good job :)

<comment><USER10> I would like to see the orange in the extension too. Other than that I have no complaints.

<comment><AUTHOR> People you follow are highlighted orange, as the only orange is your own stuff, the logo and these people it makes sure they stand out :-)

</comment></comment></comment><comment><USER11> I just installed this for the first time and my favorite feature is relative font-sizes, hands down. I have a larger default font-size on my laptop and I didn't realize how many sites don't respect that until I switched.

</comment><comment><USER12> I really like this but the hover actions make it pretty annoying to casually browse. After just a few minutes of use, I accidentally triggered the user hover actions a few times.

<comment><AUTHOR> That's a shame... I can't say it happens for me, perhaps I've subconsciously learnt not to scroll in the vertical band where the usernames live! I'm using a hoverIntent plugin so you do need to hover for a few hundred milliseconds before it's triggered too.

<comment><USER13> I'm in the same camp as nicksergeant. I also don't like having to click outside the tooltip to dismiss it.

Nice work regardless, thanks for building it.

</comment></comment></comment><comment><USER14> Is there a reason recent Chrome extensions never have desktop notifications? Is non-user-initiated scraping banned or discouraged?

</comment><comment><USER15> Nice work! As said by others; would be nice to be able to switch between dark and light themed CSS though.

</comment><comment><USER16> I would like you to add a facility to let us see the linked web page on the right hand side.

</comment><comment><USER17> I use Hacker News <I>because</I> it doesn't have those features...

<comment><AUTHOR> It doesn't have html comments either ;-)

<comment><USER18> And I guess he probably wanted to underline "because" instead.

</comment></comment></comment><comment><USER19> No love for Firefox? Could someone repackage this to work on Firefox? Please!

</comment><comment><USER20> Would it be possible to add a retina favicon as well?</comment>