Oh shit, git: Getting myself out of bad situations - <AUTHOR> http://ohshitgit.com/ <comment><USER2> Adopted a git GUI years ago and haven't looked back. I get looks sometimes, but I can't help but gloat when I can stage and unstage individual lines in less than a second.

I think anyone who uses the CLI is either trying too hard or hasn't realized the beauty of a git GUI.

Takeaways:

\- My commit time is usually much faster than coworkers, with higher accuracy (less frequent accidental commits, etc.)

\- I don't remember the last time I made an irreversible change to the repo, or had an "oh shit" moment. And that's despite using some interesting git features.

\- Staging individual files, folders, lines of code, or hunks is easy. Makes maintaining multiple trains of though / addressing bugs while working on other code a non-issue.

\- It's easy to keep files uncommitted for long periods of time intentionally, without slowing down my workflow.

\- It's much easier to get an overview of the changes I'm making.

<comment><USER3> What GUI do you recommend? My experience has been that GUIs are the easiest and fastest way to make a mess that can't be corrected without dropping to CLI or re-cloning. (I'm looking at you SourceTree). I've long recommended that everyone who uses git know how to use the CLI even if they don't use it regularly.

<comment><USER4> If you're on a mac, I can't recommend Tower enough. It's a paid app but well worth the money. Otherwise Git Kraken is a foss offering that is also excellent.

<comment><USER5> How is atlassian's Sourcetree?

<comment><USER6> I like it a lot, it make git as simple as mercurial. I don't think it is hiding complexity, but the visual arrangement makes sense. The staging area becomes actually useful, and I love that you can stage (or revert) pieces of a file, which is handy when you forgot to commit one thing before moving on to the next.

I've only had to merge a branch once, and here I was completely confused on what was going end. This may be SourceTree's fault or git's fault. Maybe my lack of understanding.

I hate git, I don't understand git, I don't want to have to understand git, and I think the command line interface makes every UI error in the book. But SourceTree has made me like git.

</comment></comment></comment></comment></comment><comment><USER7> A git off my lawn moment: Every time I see someone complaining because they have to dive into the reflog to fix _their own_ mistake, all I can hear is "I was operating my table saw without using a push stick and can't understand why I lost a thumb".

Friends don't let friends (especially those who don't learn how to use their tools) rewrite shared git history. If you don't understand rebase, amends, etc can do to your (and everyone else's) repo, DON'T DO IT.

<comment><USER8> rebase can be dangerous, but so productive. I think it's important to teach new devs how to properly use it. Especially if you're working in a continuous deployment environment, where you may need to quickly revert something.

<comment><USER7> Reverting something does not require a rebase. It requires an aptly named command "git revert". Additionally, in the event that a deployed feature needs to be reverted, any halfway decent change management policy will want a history of what you reverted and why; a git revert commit can show both very cleanly.

And, frankly, since all rebase does is re-write history, I fail to see how it's inherently productive, especially in the context of re-writing the history of previously shared commits. Yes, it is capable of cleaning up and reducing the number of commits you have to read through when looking at history, but that's such a rare event that optimizing for it (especially by normalizing dangerous commands like `git push -f`) seems, well, premature.

<comment><USER9> Completely agree. Rebase is practically only safe on single developer feature branches which haven't been merged or completely isolated repos that produces patches only. Keeping track of all requirements in the former is hard, especially if the dev doesn't know git pitfalls. No one should default to rebase in a day to day workflow, but it is quite common.

</comment></comment></comment></comment><comment><USER10> Nice. For anyone who hasn't see it, Flight Rules for Git is even more comprehensive: [https://github.com/k88hudson/git-flight- rules](https://github.com/k88hudson/git-flight-rules)

<comment><USER11> I'm planning on making a hard push for git on the team I just joined (that isn't using any VCS). This and OP are going in my bookmarks.

<comment><USER12> I'm genuinely curious how they manage source without any VCS. Is it just a bunch of zip files for old versions?

No judgment, nobody is born knowing this stuff, just that I'm surprised to hear this is still out in the wild.

<comment><USER11> We SSH onto a server to do work and it seems we just make folders wherever for our individual projects. They do (nightly, I think) backups which is sort of a rudimentary version control. And while I'm told it exists, it seems the "dev" version of their website isn't used, so they just kind of add in their changes live; definitely a risky environment. They've had a small team for a while and most of their applications could run relatively in isolation from others, so they could get away with it, but they're looking to grow so I'm trying to push this stuff.

<comment><USER12> Yeah the "right way" to do VCS is definitely a lot of overhead for a team like that, but by the time you need to scale up (and by "up" I mean 3-6 people total who are committing code multiple times daily?) the business is likely so busy it's a HUGE uphill battle to implement something like that, especially since most of the people who let an environment like that happen will likely not be super familiar with Git/hg/TFS/whatever.

<comment><USER13> I dunno if I'd agree that it's a lot of overhead, for what you get back. Hell I'm a one-man team right now and I'm using source control. Some of my little experiments go horribly awry. Some of my deletions are a bit too aggressive/arrogant. Sometimes you need to diff shit. Version control frees me to be bold.

<comment><USER14> Agree with you here. I use git for literally every coding project, even my quick practice folders. I haven't had an "oh-shit" moment from losing anything in like 5 years, back when I was still learning git.

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> In three years of using git I believe there is a single bad command that I could not undo: `git checkout -- somefile` The second worst thing I did is losing a commit in a `git rebase -i` but I was able to find it back with `git reflog`. Which makes me think that git is really well designed.

<comment><USER16> If by "designed" you mean "picking a content-addressable DAG as an underlying data structure", then yes, it is really beautiful. If you mean "provide a sane level of abstraction over said data structure", then hell no!

<comment><USER17> I think git provides a sane level of abstraction.

I don't think it provides the most consistent UI or helpful help, though. Once you move away from "learning git commands" to "learning how git works" and kind of figuring out which parts the commands refer to has helped also. That's still terrible though.

<comment><USER18> >I don't think it provides the most consistent UI or helpful help, though.

Understatement! Git is one of the most user-hostile tools I've ever seen. And I've used Sendmail.

<comment><USER19> Just read the manual.

[https://git-scm.com](https://git-scm.com)

</comment></comment></comment></comment></comment><comment><USER20> You could also use an alternative CLI for git. My preferred one is hg-git.

I'm sort of kidding, but not really. By using it, I can operate on a git repo using the mercurial CLI, which I am more comfortable with and have built tooling around.

Once you have the extension installed, it's a matter of prepending 'git+' to the beginning of eg a github url to clone it, and then you can use native mercurial commands.

Obviously, this is an extra layer with all of the problems that involves (eg you're going to be managing git branches through an hg interface. Which works surprisingly well most of the time.) So this makes NO SENSE WHATSOEVER unless (1) you are comfortable with the hg interface, (2) you aren't that comfortable with the git interface, and (3) you don't need to do complex operations that would require thinking through the layers of what is happening.

Honestly, though, if I want to contribute to a github repo, this gives me everything I need 95% of the time. So far, the only time I ran into trouble was when the active branch was renamed in the github repo. Things got very tangled up, and I had to re-clone and import my patch stack. Judging by the number of people talking about making "backups" of git repos in this comment section, it doesn't sound like that's an unexpected operation in git-land. With mercurial's immutable and append-only nature, I very very rarely need to do this, so it felt kind of awful.

(Admittedly, my main hg repo that I work on is 6GB and I'm normally near the limit of my disk space, so my habits don't really match your typical megabyte- sized github repo in some noncompiled language.)

</comment><comment><USER21> Honestly, a lot of what's wrong in git is that people seem to mostly memorize or copy-paste a finite amount of commands, and when something goes wrong they are completely lost unless they can find a way to copy-paste a solution.

Instead of saving a 6 command list for some use cases, why not just get used to the simple but kinda unintuitive way of how revisions and branches work? If you know that, you can solve any problem with `commit`, `checkout`, `reset`, `reflog`, and occasionally `cherry-pick`.

<comment><USER22> Quick quiz, what does 'git checkout x/y' do? I am aware of users hitting at least 3 interpretations.

<comment><USER23> I'd say it changes your code to match that of a branch in your local copy of the repo.

<comment><USER22> It resets the file x/y, or checks out branch y of remote x, or checks out local branch x/y. I've had confused users manage to create al of these cases, and git is very unhelpful in explaining the difference between the three cases.

</comment></comment></comment></comment><comment> compiler-guy The fact web pages like this exist, and are popular, and make it to the top of Hacker News is all you need to know about git's ease-of-use and mental model.

That Perforce and SVN don't get such things should also tell you something.

(Which isn't to hate on git--it's a great tool.)

<comment><USER24> Threads like this are a nice reminder of how nice it is to use Perforce. There's no such thing as getting yourself into a bad situation. Not even for artists or designers.

Git has eleventy million blog posts trying to explain how simple it is to use. That's a clear sign that it is, in fact, not simple or easy.

<comment><USER25> Reminds me of all articles attempting to explain how simple monads are

</comment></comment></comment><comment><USER26> Good thing HN doesn't have memes, otherwise I would post one of those bears confessing "I'm a software engineer with 19 years of experience and I still don't really understand git".

Silliness aside, I guess Git suffers from the trait that it is very hard for many people to internalize and visualize how it works in their heads. Totally different but related: the Azure CLI is for some reason 10x easier to work with than the AWS CLI, while I'm 10x more experienced with AWS. I guess CLI design is also design.

</comment><comment><USER27> I like using IDEA, PyCharm's and Webstorm's "local history" feature to compensate for gaps in my git knowledge.

A constant, instant backup of all files, independent of the repo, has been my saving grace for small projects.

<comment><USER28> \+ it shows you a diff with the ability to revert individual changes in code

</comment></comment><comment><USER29> "Git gets easier once you get the basic idea that branches are homeomorphic endofunctors mapping submanifolds of a Hilbert space."

— Isaac Wolkerstorfer (@agnoster)

</comment><comment><USER30> My last git mistake was pretty terrifying. I decided to try and go back to an old commit on a project on my local machine after about a days work. Somehow I ended up making the commit I wanted to revert to a new branch, then somehow tagged that branch with the name of the commit making git get angry and decide that branch wasn't valid. Then continuing in my ignorance I reset to that branch and tried to checkout only to watch my source and resource files vanish one by one. Deciding the commandline had caused enough trouble I returned to a qgit window I had open, all the source files were still there and I could at least save them one by one. Better refresh the qgit window. Oh shit now qgit's mad at me too. Well there goes thousands of lines of code and about 3 months worth of work. Eventually,after reading gits cryptic error messages and a few google searches I figured out how to remove a tag from a branch and properly checkout an old commit. I was really happy when all those source files reappeared.

<comment><USER31> Git is not a backup! Had you had one, you could have had your mind at ease.

<comment><USER32> Pushing to a remote is a backup. Gitlab offers private repositories for free.

<comment><USER33> Until you force push ;)

<comment><USER32> And then you still have the commits, but no branch pointing to them. Don't worry, they're still there.

</comment></comment></comment></comment></comment><comment><USER34> > Oh shit, I accidentally committed to the wrong branch!

I find cherry-picking to be easier in this case. Just checkout the branch and cherry pick commits from 'wrong' branch.

[https://git-scm.com/docs/git-cherry-pick](https://git-scm.com/docs/git- cherry-pick)

<comment><USER35> But then those commits are still in the wrong branch. If I accidentally commit something to master instead of a development branch, I can't deploy master until my development branch is merged in, as the one commit isn't ready for live.

<comment><USER36> If you haven't pushed, it's as easy as resetting the branch to the last good commit. If you have pushed, well, it's the same, then push -f, and then making sure that all your teammates pull the fixed branch. If that's a situation you regularly find yourself in, protecting master from direct pushes and only doing PRs is a good solution (GitHub, BitBucket and GitLab all have tooling for this).

<comment><USER34> Maybe it's just me but I feel like if you work with multiple developers you should never, ever force push master. I'm wary of force pushing branches in general, unless that's a branch that only I work on.

<comment><USER36> It's not ideal, at all -- and if someone else has pushed in the meantime, they risk losing the commit. But if you're in a situation where doing this would be a significant problem, you should probably not be pushing directly to master in the first place, instead relying on a PR workflow.

<comment><USER37> Why not simply push a second commit that reverts the broken commit? This will avoid rewriting history and messing up the rest of your team.

<comment><USER36> Because that will pollute the history and diminish its value as a record of what happened in the code base. For instance, blame will no longer tell you when a line was changed, but rather point to the revert commit. Bisect breaks if it divides through the reverted commits.

Of course, rewriting history is only feasible immediately after the mistaken commits, before anyone builds on top of them. If they've lingered, reverting is the right way.

And again, if this is a repeating problem, fix it upstream (no pushing to master, only PRs).

</comment></comment></comment></comment></comment></comment></comment><comment><USER38> I gave up on rebasing or any form of history rewrite after going through the screw up phase, then watching every dev that came after me do the same at least once.

I realised that treating a git repo like an immutable collection, where existing commits are NEVER mutated makes it far easier to reason about history and nearly impossible to do serious damage.

Devs can do whatever they please with their own local pulls (such as squashing 50 rambling commits during dev), but once a commit is on remote, it never changes.

</comment><comment><USER39> On barrier to learning git is that people are understandably reluctant to try things out because they are working in real repositories.

This site makes a game out of learning git: [https://learngitbranching.js.org/](https://learngitbranching.js.org/)

It's a great way to learn without putting your real repository at risk.

<comment><USER40> Thanks, I am new to git and already made some dumb mistakes with my real repo and had to start over. Now I'm terrified to try anything past pulling and pushing to master. Hopefully this helps me understand branching so I don't feel like I will break it all again.

</comment></comment><comment><USER41> The very existence of such guides tells us a lot about how easy to use Git is :)

<comment><USER42> Name me any software that doesn't need a guide like this.

like, how do you even exit vim? You need a guide for that. Or to get it closer to git / this guide, how do you undo / redo?

<comment><USER43> You're comparing git to vim in terms of ease of use...

<comment><USER44> That's actually a rather good comparison. Both have learning cliffs of doom[1]but once you've spent a few months with them they're arguably more powerful than the other options. They're usually fast to use but very hard to learn. I wouldn't say either one is easy though.

[1] [https://dementiagaming.files.wordpress.com/2010/08/learning_...](https://dementiagaming.files.wordpress.com/2010/08/learning_curve_of_eve.jpg)

<comment><USER43> That's debatable. I don't think anyone has actually made any study which proves that git is superior to Mercurial or that vim is superior to Notepad++ or Sublime.

</comment></comment></comment></comment></comment><comment><USER45> I stubbornly stuck with Mercurial for a long time because of the complexity of git's UI. But when one moves beyond a few developers, the available tooling and extensive ecosystem for git makes it inevitable.

<comment><USER46> Still with hg. You've frightened me.

</comment></comment><comment><USER47> We pushed large binaries into our git in the past. This was fine-ish as long as Git was hosted inhouse, but now that it's SAASed out, they are a huge pain in the rear.

I've browsed through a few git guides, but can't seem to find anything that would let me:

1) Do something like "du -s *|sort -n" for the entire Git history

2) Let me "rm -rf --from-history-too", that would cause the remote repo to actually shrink in size.

<comment><USER48> I think you should be able to do something like this with

      git filter-branch

This won't be a terribly fun exercise, and could be very painful if your history contains a lot of merges. (should be easier with the cactus/rebase development model)

And of course everyone will have to hard-reset to the new branch.

I should mention I'm far from an expert on this. I've only ever used git filter branch on a handful of commits, and only based on examples provided by kind internet people. I certainly haven't done anything nearly as far-reaching as you're about to embark on.

<comment><USER42> Yeah, filter-branch is the way to go for this. Also to for example extract a folder into a new repository. I've used it in a few cases.

</comment></comment></comment><comment><USER49> I always fight for people really learning git, because that's when it finally starts to get good. And I always tell people that git is not the tool that everybody should use. Most people just need a simple data storage with diff management, like Dropbox or SVN.

But even after nearly 10 years, the pressure from the aint-nobody-got-time- for-that crowd is still there. I really, really hope that the git devs don't feel pushed to simplify and therefore depower git. Thinking turning VIM into the Windows text editor.

<comment><USER50> They can certainly provide much better porcelain out of the box. Mercurial is at least as powerful as git, but I always joked that the learning curve between the two is something like, you read the docs for 5 mins and you'll know what the next 50 commands to type into hg. Whereas git, you spend 50 minutes reading the Pro Git book, just so you know what the next 5 commands should be.

<comment><USER49> Sorry, but you don't know git if you think Mercurial is even coming close.

That is not trolling btw. It's really possible not to know what you can do with it if you haven't learned it in depth.

The "Git Book" is what you should read. And you read it once, you read it indepth, and then you're done. After that it's also only 5 minutes of googling, but you can do a lot more. At that point you can even program a simplified git if you want.

<comment><USER50> TBH, I've been an hg and git user for about 10 years now, I have only come across simple things that you'd expect to work in git but doesn't. For the longest time, you have to resort to contortions like this[1] so as to not lose commits after reverting a merge and then merge again.

[1]: [https://github.com/git/git/blob/master/Documentation/howto/r...](https://github.com/git/git/blob/master/Documentation/howto/revert- a-faulty-merge.txt)

What _can 't_ you do in Mercurial that you can in git these days?

BTW, I think we are talking about the same book.

<comment><USER49> I could name a few things, like interactive rebases, but the main thing is not something that can really be explained. Have you ever significantly become strong in some kind of contest, may it be sports, gaming, music or similar? There is this situation where one day you struggle with something and don't see an end, and the next day it finally clicks, and you can do things naturally that one day earlier where not even imaginable.

If you've experienced that once consciously, then you can regonize it and you know when you experience it you've hit someting really good in your life that moves you forward.

Sadly it seems you must take that hurdle in anything until a certain age or you will always believe this is impossible and therefore never invest the energy to reach it.

The thing is that this is not possible to achieve with any random software. For instance no matter how much you learn MS Word, you probably won't experience that. But when learning Vim or Emacs there's a chance you get there. Same is with git. And once you've achieved it once consciously, you will always want to be in that state in everything important you do.

That's why the really good stuff only has a few followers, most simply don't get the appeal because they never would invest the energy to get "there" even if they knew exactly how much it would take. But for those who have achieved it there is no going back. You cannot go back from controlling (almost) any bit of your repository to Mercurial.

But that's also why I think for most users something like Mercurial should be The VCS. Most people don't know the reward they are missing, so they don't feel the pain of missing it, and therefore have no logical reason to go through the pain of really learning git.

<comment><USER51> Long story short, you're using git because it makes you feel smart. Ironically, you even admit it - "therefore have no logical reason to go through the pain of really learning git". Correct, there is no logical reason to deal with a contorted DVCS when there are better alternatives. Your arguments are fully subjective.

<comment><USER49> If you tell a blind person, he should use the color red more when dressing, it might be a good advise, but for the blind person there is no logical reason to choose red over green.

What I'm saying is that most people are blind when it comes to skill, because they never make it over the hurdle once. But because most people don't have any skill, it is general consensus in our society to declare skills based on interest rather than the ability to solve hard problems or win competitions. I mean if one person is unable to judge if another person is good at solving merge conflicts, how could he criticize that person declaring himself as Git Expert Of The Team. And then again of course most people feel it is inappropriate to tell other people they are unskilled, so posts like this one get hated a lot on. But actually it's just calling out the facts. Sky is blue, grass is green, most people are too unskilled to even recognize what would enhance skills.

<comment><USER51> Unless I'm mistaken, people are paid to solve engineering/programming problems, not fight with a mess of a DVCS. And if I can use a Mercurial GUI and do in 15 seconds what would take 5+ minutes in the git command line, what's the point of being the Git Expert Of The Team? Is version control not used for saving work progress? A simple thing that 99.99+% of developers should be taking for granted in an ideal world, just like when pressing Ctrl+S in your IDE (hopefully) saves the file without asking what encoding you want to use and what partition you want it written on.

You seem to be under the impression that doing hard things has some intrinsic value, when it's not the case. You could be rolling rocks uphill all day and be the best in the world at it. So what? A keyboard going _clickety_ clack* for 5 minutes while typing git commands (and possibly stackoverflow queries) is the opposite of work, it's actually a waste of time. Sure, at the end of the day you solved the problem git presented you; you're smart, and feel satisfied with you amazing display of intelligence and skill. However, this ignores the fact that you should not have had to solve that problem to begin with, because your version control system should just magically work (unless you get a bonus for each git commit/merge).

> how could he criticize that person declaring himself as Git Expert Of The > Team

I have no problem with someone being a git expert - I actually will even admit that I am not a git expert and that there are people extremely skilled at git. The issue here is that there are some people really skilled at, for example, typing using the backs of their fingers. Potentially impressive, but entirely useless. Saying you're skilled at git is like saying you're skilled at using Jira or editing Wikipedia; good for you, but in all but most extreme cases this doesn't actually generate value and it also doesn't make you a better developer.

> one person is unable to judge if another person is good at solving merge > conflicts

Just out of curiosity, is this somehow relevant to git? Shouldn't solving a merge conflict be a simple matter of looking at a diff tool presenting you two inputs and one output screen? Or better yet, wouldn't it be better if you were automatically told you are editing a file someone else is working on, so that it prevents conflicts from happening?

<comment><USER49> feel downvoted.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER52> A collection of commonly used git tips and tricks:

[https://github.com/git-tips/tips](https://github.com/git-tips/tips)

<comment><USER52> Learn git one commit at a time:

[http://gitready.com](http://gitready.com)

</comment></comment><comment> trevor-e I think Git would be way more approachable if things were named better. A key part of Git is whether a file is untracked/unstaged/staged, yet the terminology around this is very confusing. Why aren't commands simply `git stage`, `git unstage`, `git add` (to track files), `git remove` (to untrack), `git undo`, etc? Not to mention the overloaded command names like `git checkout`, how is `git checkout -- someFile` intuitive at all?

<comment> satish-setty This is precisely one reason I like Mercurial over Git. Hg has commands which are more meaningful than those of git.

<comment><USER46> +1 hg. I see it mentioned so seldom here that I'm actually worried it may go away.

</comment></comment></comment><comment><USER53> 5+ years later and I still sometimes have to do the classic "repull the entire repo into a new folder and manually copy the wanted changes over"

</comment><comment><USER32> I think lists like these are part of the problem. You shouldn't rely on tricks to use git, you should understand how it works.

<comment><USER54> *this

</comment></comment><comment><USER55>
      git checkout name-of-the-correct-branch
      # grab the last commit to master
      git cherry-pick master
      # delete it from master
      git checkout master
      git reset HEAD~ --hard

The two lines in the middle can be

      git cherry-pick -
      git checkout -

, and voila, it suddenly is name insensitive.

</comment><comment><USER56> One can read any amount of similar articles, but if they don't understand the data model behind all this, git will remain a complex and fragile beast for them. The feature-set is huge to be able to just remember all possible commands for all possible scenarios. If you don't know what index is or what a branch is, this all will look like a bunch of nonsense. On the other hand - your ruby code won't seem too logical either if you don't know the language.

So if git is the tool you use to get your job done - don't hesitate to spend a day or 2 on reading how it works and how you're supposed to use it.

</comment><comment><USER57> The absolute worst situation I've gotten into was just the other day, when my team managed to check in two files with the same name, but different capitalization. The server (Bitbucket) is Linux-based, but all the workstations are Mac or PC, so the names collided on everyone's desktops. This prevented anyone from pulling or checking out a different branch, so even after master was fixed, everyone's environment stayed broken until someone figured out the right magic incantation (git rm --cached).

<comment><USER58> Not really an issue with git, but with OSX. In doing some research of git, I wanted to look at the first git repos. Those are torvalds/git.git and torvalds/linux.git. If you clone linux.git on OSX and run `git status`, you'll see this:

        $ git status --short
        ## master...origin/master
        M include/uapi/linux/netfilter/xt_connmark.h
        M include/uapi/linux/netfilter/xt_dscp.h
        M include/uapi/linux/netfilter/xt_mark.h
        M include/uapi/linux/netfilter/xt_rateest.h
        M include/uapi/linux/netfilter/xt_tcpmss.h
        M include/uapi/linux/netfilter_ipv4/ipt_ecn.h
        M include/uapi/linux/netfilter_ipv4/ipt_ttl.h
        M include/uapi/linux/netfilter_ipv6/ip6t_hl.h
        M net/netfilter/xt_dscp.c
        M net/netfilter/xt_hl.c
        M net/netfilter/xt_rateest.c
        M net/netfilter/xt_tcpmss.c

<comment><USER57> I disagree here. Case-insensitive file systems have existed for decades, and are part of the development landscape. Any tool that works across multiple machines needs to support them. In this case, I blame Bitbucket, for not providing any way to treat this as a merge conflict.

<comment><USER58> The tool that doesn't work with case insensitive filenames is OSX.

Edit: To add, how is git supposed to handle two files with the same name -- as is the case -- with a case insensitive file system? You can't have two files with the "same" name.

        git checkout file1

Is entirely different than

        git checkout FILE1

In any other platform ls file1 FILE1 would show two different files, but on OSX, it'll show the same file twice.

</comment></comment></comment></comment><comment><USER59> I've said this before, but the business leadership, and tech leadership, need to think carefully about whether or not they need all of the power of Git. This sums up my concerns:

\</comment><comment></comment><comment></comment><comment>-----

 _Here are some minor failure modes I've seen with Git:

1\. a branch that stays open for many months, perhaps even a year (for instance, at Maternity Neighborhood)

2\. data is erased for good because someone makes a mistake while using rebase

3\. a developer introduces a new bug while trying to resolve a merge conflict

4\. widespread but fine-grained cherry picking leaves the team unclear about what's been merged and what has not been merged

5\. a developer makes a change in the wrong branch because they forgot what branch they were in

6\. a developer is unable to recover a stash because they forgot where they were when they created the stash, or they simply forget that they have work in a stash

7\. developers confused by working in an unattached commit, after a botched attempt to revert

8\. a developer feels the need to delete the repo from their harddrive and clone it again, because the whole repo got into a state that they seemed unable to resolve

9\. the "blame" command is nearly useless — maybe its because we never know in which branch a given change was made, finding who made a mistake is very difficult

10\. developers get confused about which branch will be deployed (I see this especially in shops that have lots of repos for lots of small apps, and use different Git workflow strategies for the different repos, often because different programmers or different teams control the different repos)

11\. developers push their changes to "origin" but forget to push to "upstream" or vice versa.

But all of that stuff is trivial compared to the major flaw:

Graphic designers, writers, HTML/CSS frontenders, managers, data analysts and QA staff can't use Git, even though they all used Subversion._

[http://www.smashcompany.com/business/business- productivity-h...](http://www.smashcompany.com/business/business-productivity- has-been-undermined-by-the-hubris-and-power-grabbing-of-elite-computer- programmers)

<comment><USER32> > 2\. data is erased for good because someone makes a mistake while using > rebase

No. Just no. Please stop spreading FUD like it's candy. Git only deletes commits after a GC, which won't erase commits from reflog and will keep unreferenced commits for at least a month before deleting them. And rebasing generates new commits, leaving the old ones exactly how they were. If somebody lost a commit after a rebase, and nobody nearby could help them recover it, they should consider spending a few hours learning about git.

I've been using git for 4 years both at work (with a team of 40+ people) and at home, without ever having any of the problems listed here (except 3 which has nothing to do with git). It takes a few hours, maybe a few days to understand how git works and how to use it. Instead of blaming the tools, you (and your team) should probably learn how to use them.

<comment><USER60> >If somebody lost a commit after a rebase, and nobody nearby could help them recover it, they should consider spending a few hours learning about git.

GIT is the only source control system where you can completly loose commited data by actions (or commands) available the non-admin user. It's funny to see that this is considered a "feature" and not a design failure.

<comment><USER32> > completly loose commited data

Except the whole point is that you can't completely lose commited data, even if you tried really hard. And the solution to the problem was `git reflog` which shows the history of the HEAD.

</comment></comment></comment></comment><comment><USER61> This is why UI's were created. No productive person wants to fill there heads with a ton of commands and switches when you could just click a button.

<comment><USER62> Which button? What happens when I click it? Am I going to have to remember this later?

It's all "choose your abstraction", so you'll be filling your head with _something_ whether it's which button to push or what _git checkout_ does. GUI or not, no button alt text is going to tell you what _rebase_ means.

</comment></comment><comment><USER63> Most oh shit git moments are a result of "advanced" git users trying to force people who just want to get shit done to "keep the history clean" with rebases. If you stick with git push, pull, and merge you really can't lose work. And any "advanced" git user knows the right git log incantation to filter out merge commits and such to make the history look clean.

</comment><comment><USER64> If this is a guide for beginners, there should be (more?) warnings not to do history rewriting operations on commits that have already pushed.

<comment><USER65> From the disclaimer at the bottom:

> This site is not intended to be an exhaustive reference, nor is it a > beginner's tutorial.

<comment><USER66> It may not be intended to be a beginner's tutorial, but it is. Someone who needs to be told about `git commit --amend` and `git diff --staged` also needs to be warned about rewriting history that has been shared with others.

</comment></comment></comment><comment><USER67> I too felt this way once, and I still have to think (or even lookup) about what the conventions are occasionally, but there really is a point in learning Git at which you realize there really are not many 'oh shit' situations in which you can find yourself and not be able to escape unscathed.

But beyond learning Git itself, you need to develop suitable _habits_ for using Git that are appropriate for your development workflow.

One habit I've consciously developed is to use 'WIP' commits (i.e. a regular commit with "WIP: Rebase me!" on the first line of the commit message) instead of stashing. All too often a stash lives for much longer than you'd naively expect and stashes are much harder to work with than commits. Even if the modifications are applied right away, commits are nearly as easy to 'apply' as a stash anyways.

</comment><comment><USER68> It happens to everyone :)

I'd like to contribute my list of git booboos: [https://github.com/1337/yesterday-i- learned/blob/master/git....](https://github.com/1337/yesterday-i- learned/blob/master/git.md#booboos)

<comment><USER69> > I used git in the terminal and the diffs/patches/merges/pull requests don't > turn out right

Your answer there is just doing git pull --rebase by hand. I don't like that style at all, but if you must do it surely use the command that's designed for doing it.

> I pushed stupid things onto the remote server

Worth saying you can branch before fixing - that might be a little less intimidating than "access by commit ID"

> I already made my changes in multiple commits, but the repo owner wants me > to rebase it to a single commit

Disagree with "you should do it", though that's probably a political question.

<comment><USER68> Thanks for the review! I actually wrote that years ago (in frustration). Will give it a revision when I have time.

I think I would rebase my PR comment history away only if "the repo owner wants me to rebase it to a single commit". In that case, I'm just doing whatever the repo wants the repo to be structured.

</comment></comment></comment><comment><USER70> > Oh shit, I did something terribly wrong, please tell me git has a magic time > machine!?!

Shouldn't it be easy for Git to remove objects created after a given time, and then remove all objects that refer to those objects iteratively?

<comment><USER71> I think that's exactly what it does; the default is two weeks: [https://git- scm.com/docs/git-gc](https://git-scm.com/docs/git-gc)

</comment></comment><comment><USER72> One thing I do if I'm a little uncertain about my upcoming git actions is just move to a new branch: git checkout -b tryScaryGitStuff. That way if I ruin the commits, the original branch is still around as a back up.

</comment><comment><USER73> Is it just me or did [http://ohshitgit.com/#accidental-commit- master](http://ohshitgit.com/#accidental-commit-master) forget to checkout master before calling reset?

<comment><USER74> It's still on master because `git branch` just creates and doesn't do checkout.

(Which is why I've almost exclusively switched to `git checkout -b <newbranch>` for creating. How often do I want to create but not switch to the branch?)

<comment><USER75> Git branch is helpful if you just want to make a snapshot before doing something potentially harmful. If you hose up, just checkout that branch again.

</comment></comment></comment><comment><USER76> So I _just_ ran across this one: git diff --staged

I added a file and wanted to diff it, and this command helped! However, I made some changes in the file, and when I tried this command a second time, the changes don't show up :\ Only the original file that was added shows up in the diff. Now what? It's not the end of the world of course, as I can just look at the file in my editor, but I usually use diffs as a personal code-review before I commit.

<comment><USER77> There's three "versions" of the code at play, if you will:

* the most recent commit

* the staged files

* the working directory

`git commit`, as you might already know, takes "the staged files" and turns it into a commit, making that the latest commit. `git add` adds a _snapshot_ of a file in your working directory to the staged files. The important bit here is that the copy of the file in the staging area is _separate_ from the file in your working directory. So, if after `git add`ing a file you make more changes, you will need to `git add` those subsequent changes if you wish to commit. `git status` will tell you this:

      » git status
      On branch master
      Changes to be committed:
        (use "git reset HEAD <file>..." to unstage)

      	modified:   foo.txt

      Changes not staged for commit:
        (use "git add <file>..." to update what will be committed)
        (use "git checkout -- <file>..." to discard changes in working directory)

      	modified:   foo.txt

"Changes to be committed" is the staged files. "Changes not staged" is stuff that has been modified, but not `git add`'d. You can see here that I've changed foo.txt after git adding it; if I want those changes, I need to git add it again.

I can look at the diffs, too:

      # diff between the last commit, and the staged files
      # (i.e., what will be committed)
      git diff --staged
      # diff between the staged files and the working directory
      # (unstaged changes)
      git diff
      # diff of all changes since the last commit:
      # (stage+working dir, essentially)
      git diff HEAD

That should be all the various combinations.

I find that a lot of newcomers find the staging area weird, and usually ask some variant of "why would I _not_ want to commit all of the files I've changed?" The staging area, used effectively, can really help you break out things into logical chunks of changes that can then be adequately described with a message in a commit. This can help others later: if your change is a bug fix, and someone wants to cherry-pick it to production, they might not want your new feature, or your lint fixes: they want a minimally risky fix. To that end, the stage/working dir separation acts as a sieve of sorts, letting the stuff that's "ready to go" get filtered out into a commit.

I want to mention the extremely useful `git add -p`: this command will interactively prompt you, hunk by hunk, for whether or not you want to stage that hunk. It will even let you edit the hunks prior to staging. So, for example, if I run across a spelling error, or a minor bug, I can very quickly add it (and _just_ it) to the stage w/ `git add -p`, and then commit it, even if there are other modifications, even in the same file.

<comment><USER78> > There's three "versions" of the code at play, if you will:

> * the most recent commit

> * the staged files

> * the working directory

This is weird. The staging area is like a commit but not a commit. They're changes that git is aware of and has a record of but not quite a permanent record.

Why not just make it a commit? You can always keep editing commits, or throw them out, or whatever. That's what I do with Mercurial. I write a commit and I keep adding stuff to it if I think it needs more stuff (or less stuff).

Gregory Szorc has a more extensive analysis of the situation in first subsection here: [https://gregoryszorc.com/blog/2017/12/11/high-level- problems...](https://gregoryszorc.com/blog/2017/12/11/high-level-problems- with-git-and-how-to-fix-them/)

<comment><USER77> My best guess is that the commit metadata (particularly, message) is missing. You could always have it be "(uncommited, staged changes)" though, and that's probably descriptive enough. (I agree with you on the whole: having the staged data be a commit makes things conceptually much simpler.)

My other guess is that the "index" (the other name for the staging area) is also used for conflicts during merges & rebases, and that somehow plays into the problem of making it an actual commit. (But again, this comes across more as an excuse than a reason: I don't any viable reason why the staged changes can't still be an actual commit, and the merge conflict data just stored as a separate structure.)

That, or the person who added it just didn't think of it, or couldn't do it due to backwards compatibility.

</comment></comment></comment></comment><comment><USER79> For me the big thing was learning to 1/ not work on master unless I'm alone and 2/ commit all my work incrementally

        git pull —-rebase

&

        git log
        git rebase -i HEAD~n

So basically I want to make sure my work is pushed to remote as a separate commit before attempting to run any commands except the ones above. But you can do pretty much anything with the set above.

<comment><USER80> I generally replace `git pull` with a `git fetch $remote` + `git rebase $remote/branch-name`. It provides the added benefit that I can easily diff changes prior to the rebase so that I know what is going into my branch. The use of the `--rebase` flag isn't bad though and I may start doing that when I know I will not be doing any diff prior to a rebase.

I also use `git merge-base` quite a bit so that I do not have to count commits when working with really simple branches.

</comment></comment><comment><USER81> A missed opportunity to say gitting instead of getting.

</comment><comment><USER82> Geez, using `git add .` is the worst thing you can do. Never use this. Tutorials: stop suggesting this. It's just a recipe to getting yourself into a mess (plus, you should never work so blindly!).

Add individual files, and, if possible, use `git add -p`, which lets you review each chunk before adding it to the index. Otherwise, diff and add individual files.

<comment><USER83> Hell, even if you want to stage every change, what you really want is `git add -A`, which stages deletes too (`git add .` ignores any files you've deleted or renamed, it only stages new files and changes to existing files.)

</comment></comment><comment><USER84> Note that for fixing rebase mistakes, `git reflog` is not super user-friendly; each step in the rebase is included, including the ones that did not involve user intervention.

In those cases, it's better to use `git reflog $BRANCH_NAME`, which has the additional advantage of not including your switching from one branch to another.

</comment><comment><USER85> I've always found this flowchart[1] wonderful and concise for handling this problem (that is, how do I fix the problem I've made for myself with git)..

1: [http://justinhileman.info/article/git- pretty/](http://justinhileman.info/article/git-pretty/)

</comment><comment><USER86> Maybe it's not worth being in there because of how simple it is, but I guess it would be useful for newcomers:

        # Oh shit, I've changed 200 files and want to revert to the state of this morning
        $ git add . && git commit # often. Very often.
        $ git reset --hard commit_id

<comment><USER71> Wait, aren't you committing and immediately orphaning that commit? I think you should create a new branch or tag before the reset, otherwise how are going to get it back?

I guess you could rely on the reflog, but I don't think that's good practice.

<comment><USER86> lol, no, it's implicit there are many hours between the first command and the second one :)

It's purely commiting often to easily have checkpoints to revert to, nothing fancy (yet, incredibly useful).

</comment></comment></comment><comment><USER87> This is a great resource, but I wish the name was different. The oh-shit reference is a bit too much.

</comment><comment><USER88> Also useful for panicky moments, with handy workflow: [http://sethrobertson.github.io/GitFixUm/fixup.html](http://sethrobertson.github.io/GitFixUm/fixup.html)

I visit this page every few weeks. :)

</comment><comment><USER89> > Fuck this noise, I give up.

PLEASE add a note to check if you have anything interesting in stashes/branches before doing the rm -rf, I've lost some work so many times because of the "fuck this noise" approach.

</comment><comment><USER9> The site should clearly differ between local changes and pushed changes. Latter usually requires different strategies to recover, depending on repo policies.

Git golden rule: Never push -f if you don't know what you are doing.

</comment><comment><USER90> My oh shit moment is that `git add .` doesn't stage deleted files. I almost always use `git add -A` because that is typically what I meant, especially for someone coming from another Version Control system.

</comment><comment><USER91> In choose your own adventure style: [http://sethrobertson.github.io/GitFixUm/fixup.html](http://sethrobertson.github.io/GitFixUm/fixup.html)

</comment><comment><USER92> I ran into a situation the other day where the instructions for "Fuck this noise, I give up" didn't even work! Glad to hear others have found themselves in similar boats.

</comment><comment><USER93> Pro tip: -p

Interactive staging, eg git add -p or git checkout -p

You'll be asked, for each change, if you want to stage it. This helps tremendously in preventing commits with stray marks, console logs, etc. Try it!

</comment><comment><USER94> Thank you so much! These are pretty useful tricks. Just a question: What's the difference between reflog and using git log to later checkout to a given commit?

<comment><USER32> You almost never checkout to a commit. You checkout to a branch. A branch is a reference to a commit, and the HEAD is a reference to a branch. The HEAD is where you are right now. When you make the HEAD point to a specific commit, you're in a 'detached head' state.

`git log` tells you all the commits that are part of the branch. More generally, they are the commits who are children of the current commit (the commit your branch points to).

`git reflog` tells you where your HEAD has been. When you change for branch to branch, the HEAD moves around. That history is recorded in the reflog. If you botch a rebase, your HEAD will sit on top of the newly created (and botched) commits. You can still access your previous commit by looking where the HEAD was before moving to the new commits made by the rebase.

</comment></comment><comment><USER95> Last one in particular: [http://ohshitgit.com/#fuck-this- noise](http://ohshitgit.com/#fuck-this-noise)

</comment><comment><USER96> A better tool would be a remote repo clone where you have to resolve messed up situations and only pass if you push a fixed solution :)

</comment><comment><USER97> Haha. There should be an entry there for 'Oh shit - I just pushed our private keys to the public github repo'.

<comment><USER58> Just as an FYI, those will likely live in Github forever. Run `git ls-remote` where ever you have had this problem. You'll see that there are refs for every PR in the form `refs/pull/NNNN/head` and `refs/pull/NNNN/merge`; where `NNNN` is the PR number, `head` is the latest commit and `merge` is the final commit.

If any of those commits with confidential information is reachable from any of those commits, then they'll never be garbage collected from Github.

</comment></comment><comment><USER98> I use git in this particular way, and if you do anything different you are WRONG AND BAD or you have IMPURE MOTIVES!

</comment><comment><USER99> Git is easy once you understand it. What are people doing that is so difficult to understand?

<comment><USER54> Use without understand it

</comment></comment><comment><USER100> reminds me of [https://github.com/blog/2019-how-to-undo-almost-anything- wit...](https://github.com/blog/2019-how-to-undo-almost-anything-with-git)

</comment><comment><USER101> Pretty basic stuff (and very few of them), I expected more.

</comment><comment><USER102> Oh shit I give up could instead be `git clean -fdx`

</comment><comment><USER103> I admit to using this at least once a week.

</comment><comment><USER104> In short:

| Oh shit, I do before I think

<comment><USER42> Yeah that explains a lot of git mishaps - typos, a missed `--`, `master` vs `origin master` vs `origin/master`, `reset` vs `checkout`, that kinda thing. A lot of things that can go wrong with subtle mistakes, and a lot of things that can go wrong thanks to muscle memory.

</comment></comment><comment><USER102> "Oh shit I give up" should perhaps instead be `git clean -fdx`

</comment><comment><USER105> Nothing about detached heads?

</comment><comment><USER106> You mean, _gitting_ yourself out of bad situations? ( ͡° ͜ʖ ͡°)</comment>


Ask HN: startups working on this problem? - <AUTHOR>

A lot of time when I read articles there are mainly just few sentences I think are important/relevant/funny etc that I really want my friends to see.  However I have to just tweet the whole article, and chances are people won't see what I intended.  Are there any companies currently working on a solution to this? <comment><USER2> Yah, but I stopped development it... ScreenRack. Save snippets by highlighting text, archiving full articles and capturing screenshots of the original articles, etc.

Mobile / web app. Think Instapaper + Historious.

<http://screenrack.com/>

If there's interest, I'll push forward again.

</comment><comment><USER3> check out snip.ly -- the do a good job of emphasizing text you select but still let people get to the whole article.

<http://www.snip.ly/alpha>

</comment><comment><USER4> you can email your friends</comment>


Show HN: Plotting 3 years of hourly data in 150ms - <AUTHOR> https://leeoniya.github.io/uPlot/demos/time-periods.html <comment><AUTHOR> Unfortunately, I have to use Google Analytics and Google Ads at work every day, and these UIs have absolutely terrible performance :(

A small part of the problem is drawing the trend charts. So I decided to make uPlot [1] to see what was really possible.

[1] [https://github.com/leeoniya/uPlot](https://github.com/leeoniya/uPlot)

<comment><USER2> Honestly the title doesn't do it justice. For a plotting tool it's not an impossible task, but making a _webpage_ this responsive is incredibly hard, so well done there.

<comment><AUTHOR> it's an understatement for sure :)

a lot of time is spent on just DOM layout and JITing the JS, which get amortized with huge datasets. the 166k bench doesnt take any longer.

</comment></comment></comment><comment><USER3> The 150ms benchmark in the title is really selling this short, the performance is very impressive. The 150ms seem to refer to the time it takes to initialize the graph, and with a hot cache that is more like 50ms for me here. Redrawing seems much, much faster.

I have done some visualization with WebGL because I couldn't get it fast enough with just drawing lines. That was a while ago so I'm not sure about the details, but even a simple prototype just drawing a few tens of thousands to a hundred thousand lines using canvas was slower than this for me (subjectively).

I'll have to look at the code later, but I'm curious about where this library is getting the performance from. Before I saw this I would have said you can't do this at this speed without using previously downsampled version of the data points, I'm not entirely sure now.

I'm still looking at the performance tab in the browser dev tools and how impressively empty the main Javascript usage plot is.

<comment><USER4> It looks like the mouse lines and the selection highlight are just partially- transparent divs stacked on top of the canvas that get moved around, so nothing actually gets redrawn unless the date range changes (which is a pretty clever approach!).

<comment><USER3> That's something I expected, you really don't want to trigger redraws on mouse over. What surprised me was that I couldn't tell the the times where I zoomed in or out from the JS flame chart. Usually that is really obvious, but in this case zooming was so fast that you could hardly see it. And this graph has probably around ~194k data points (388k in the source data, I assume that's x and y). I'm not entirely sure about the number of points here, I'm taking that from the json delivered to the site.

The selection highlight is unusual, I admit. That's something I'd just skip if I were going for high performance.

<comment><AUTHOR> > And this graph has probably around ~194k data points

no, the top graph is pretty much what the title says: 3 * 365 * 24.

but 194k is also no problem :)

</comment></comment></comment></comment><comment><USER5> Very nice project, but I just want to randomly point out that 3 years of hourly data is really not as much, as it may sound to somebody. In fact, it's 3 times less than 24 hours of every-second of data, which is pretty common scale for all sorts of real-time monitoring tools we often use. These fine- sliced stats really pile up quickly...

<comment><AUTHOR> > but I just want to randomly point out that 3 years of hourly data is really > not as much, as it may sound to somebody.

you're right, it isn't. and yet many js charting libs struggle even with this.

on my i5 thinkpad with integrated gpu uPlot can render 600 series x 8,000 datapoints in ~2000ms [1]. and it finishes this job in ~7s on my 2015 sony z5 compact phone. so there's that :)

but really, pumping 4.8M datapoints into a browser is probably not a great idea. you're gonna want to aggregate on the server at some point. just sayin'.

[1] [https://leeoniya.github.io/uPlot/bench/uPlot-600-series.html](https://leeoniya.github.io/uPlot/bench/uPlot-600-series.html)

</comment></comment><comment><USER6> This looks really really neat!

Just want to ask, because I love talking about render performance: have you tried doing this using offfscreen canvas? That should allow you to move a lot of things to a worker, so you avoid blocking the rendering with js? It probably won't speed up the total time to finished render, but I assume it will lock the page for a shorter period of time?

<comment><USER7> If the rendering is this controlled (and sparse), blocking the thread with it isn't really a concern. And with this data size, serialization/deserialization between the main and worker thread would probably become nontrivial.

</comment></comment><comment><USER8> Super cool! It would be great if you could provide some insight into how you built this and the kind of tricks you had to use to make this possible. Looking forward to a blog post in the future :)

<comment><AUTHOR> a few things:

\- the data is flat arrays of numbers

\- there is no per-datapoint memory allocation beyond whatever is necessary for the browser to construct the canvas Path2D object. this keeps the memory pressure low and the GC nearly silent.

\- the amount of draw commands is reduced by accumulating the min/max data values per pixel

\- uPlot does not generate axis ticks by walking the data. it only uses min/max of the x and y ranges and finds the divisions from that.

\- there is no mass-creation of Date objects, data is kept in timestamp format except when hovered.

\- the date/time formatting is done by pre-compiling templates and not re- parsing them all the time.

\- the cursor interaction uses a binary search over the x array

<comment><AUTHOR> another one i forgot

\- drawing at exact aligned pixel boundaries to avoid or minimize antialiasing

this makes uPlot charts a bit rougher looking, but the perf impact is quite large.

<comment><USER9> How much of an impact does this have?

I prefer the look of things snapped to pixels; I didn't realize it also speed things up!

<comment><AUTHOR> hey adam :)

it's highly dependent on what's actually drawn. i regressed it by accident and didnt notice a huge difference until i randomly opened the stress test (which is also densely packed so probably a worst case for AA)[1]. it went up by a factor of 2-4. cant remember exactly.

[1] [https://github.com/leeoniya/uPlot/blob/master/bench/uPlot-60...](https://github.com/leeoniya/uPlot/blob/master/bench/uPlot-600-series.html)

</comment></comment></comment></comment></comment><comment><USER7> Let this be a lesson that JavaScript is not slow. The DOM can be slow, and downloading large bundles can be slow, and _any_ code can be made slow if you try hard enough to write it poorly. But JavaScript the language is not slow in 2020.

When we talk (or rant) about the performance of JS-based apps - running on the web or in Electron or otherwise - it's really important that the conversation focus on the factors that actually matter.

<comment><USER10> No, JS _is_ slow(ish). The GC pressure alone will kill you. This is fast despite being in JS by simply doing as little as possible (while still meeting the requirements on the end result, which is where the genius lies).

The fastest code is no code. No code is fast even with pathologically slow languages/runtimes.

<comment><USER7> It's no slower than Python. Probably faster unless you're staying within a native library like NumPy.

But that wasn't my main point: my main point is that people _love_ to rant about "JavaScript apps" while maintaining (willful?) ignorance about what the actual factors are that manifest as the slowness they experience. Sometimes it's poor usage of the DOM. Usually it's ads. It's almost never the unavoidable overhead of JS cycles.

<comment><USER11> > people love to rant about "JavaScript apps" while maintaining (willful?) > ignorance about what the actual factors are that manifest as the slowness > they experience.

Are you sure you got this right?

Just because JS is fast[0] doesn't mean we cannot complain about Javascript apps that should have been plain web pages?

It is clearly possible to create advanced Javascript apps that are enjoyable even to people like me.

For most pages I use as a consumer I don't see much value add because of frontend code: I see the value of autocomplete, drag-and-drop etc but I would much prefer if my CPU stopped chewing after web pages were loaded and rendered.

[0]: yes, I agree - in most cases the Javascript language or engine is not the cause of the performance problems on the web

<comment><USER7> Whether or not a given site should be client-side rendered is a valid discussion

Ad bloat is a valid discussion

Lazy development practices in modern sites/apps are a valid discussion

But in my experience, these as well as other less-legitimate issues all tend to get lumped under the banner of "JavaScript stuff == slow", without any nuance.

<comment><USER11> Can't say for sure but the way I read them most complaints about Javascript performance are about the results of the abuse of Javascript applications everywhere.

I guess Javascript just happens to be the common thing between a number of them, and people would be just as annoyed if the multi-megabyte, cpu-hogging, data-stealing monstrosities where hand-crafted in assembly ;-)

And: If someone complained about Javascript being slow for scientific calculations I'd probably call them out on it ;-)

</comment></comment></comment></comment></comment></comment><comment><USER12> This is really cool! Another great library for working with large time-series datasets is Bokeh ([https://docs.bokeh.org/en/latest/index.html](https://docs.bokeh.org/en/latest/index.html)).

<comment><AUTHOR> Bokeh looks cool, too.

</comment></comment><comment><USER13> This is awesome, thanks for sharing!

Question: in the demo you load papaparse but the data is delivered as JSON?

Would you mind sharing what your backend looks like for the project where you're using this, and how do you extract all of this data at once from storage?

<comment><AUTHOR> > Question: in the demo you load papaparse but the data is delivered as JSON?

good catch, leftovers from original PoC i made locally. the exported data was still in CSV back then. i should clean that up!

> Would you mind sharing what your backend looks like for the project where > you're using this, and how do you extract all of this data at once from > storage?

i just did a manual export from Analytics. how you get the data out is up to you, but a time-series or columnar database would probably be a good start :)

</comment></comment><comment><USER14> I clicked out of mild interest and am completely blown away. I wish this kind of optimization was more common. Well done!

</comment><comment><USER15> That looks amazing, nice job! I was trying to create a seb dashboard with several graphs each with a few hundred data points recently. None of the libraries felt "snappy" on a mobile device in terms of load time or responsivity. I ended up abandoning the proejct because of that, but if only I knew about this library at the time...

The only thing I wish µPlot had is support for tooltips rather than numbers in the legend, but that's a small price to pay for this level of performance.

<comment><AUTHOR> there are a couple tooltip demos here:

[https://leeoniya.github.io/uPlot/demos/tooltips.html](https://leeoniya.github.io/uPlot/demos/tooltips.html)

[https://leeoniya.github.io/uPlot/demos/cursor- tooltip.html](https://leeoniya.github.io/uPlot/demos/cursor-tooltip.html)

<comment><AUTHOR> oh, and another one is in here [1] which turns the legend into the tooltip :)

[1] [https://leeoniya.github.io/uPlot/demos/candlestick- ohlc.html](https://leeoniya.github.io/uPlot/demos/candlestick-ohlc.html)

</comment></comment></comment><comment><USER16> That's 26k points. I don't think it's very difficult to render that many points quickly unless you try and use SVG. How long does Dygraphs take?

<comment><AUTHOR> i don't have dygraphs demo for this, but you can see where it lands on the 166k bench: [https://github.com/leeoniya/uPlot#performance](https://github.com/leeoniya/uPlot#performance)

</comment></comment><comment><USER17> I didn't even read the title before clicking and was blown away at the speed. Honestly thought something acted up since you're not used to seeing things render this fast.

<comment><AUTHOR> interestingly, focusing on performance in web dev (and actually delivering on it) really presents new UI challenges, because users are not used to near instantaneous feedback in web-apps. you have to start adding complexity like css transitions, and UI delays just to reduce user confusion.

lol :(

</comment></comment><comment><USER18> I'm somewhat confused. Is 150ms considered low? That's less than 7fps.

3 _365_ 24=26280. Emitting 26k points is like child's play for any program, are we really so far down in bloat that this is considered impressive? Because this is not impressive. A 16MHz AVR could execute 91 instructions per sample in those 150ms, more than enough for plotting that data.

A 3GHz machine could execute 17123 instructions per sample, and it would probably need less instructions than the AVR would.

<comment><AUTHOR> FWIW, the majority of this 150ms is bootup time and includes initializing the DOM & canvas, JITing the js, downloading & parsing the dataset, and actually running the code, which includes data downsampling & gap detection/clipping.

try to get this perf on the web, and then you can re-asses your statement. if it was easy, then every other js charting lib would not be struggling to do it, right?

for native code (or webgl), obviously this is child's play. but webgl has significant trade-offs. e.g: [https://bugs.chromium.org/p/chromium/issues/detail?id=771792](https://bugs.chromium.org/p/chromium/issues/detail?id=771792)

finally, you cannot extrapolate from the 26k/150ms number. uPlot can draw 4.8M points in ~2000ms on an i5 with integrated gpu (after bootup amortization).

</comment></comment><comment><USER19> Looks really nice.

I was slightly disappointed when I zoomed in as I expected 3 years of data in 150ms increments (600 million measurements) but that's probably not possible

<comment><USER20> Assuming there is some culling, I've found myself somewhat obsessed about the _methods_ of dataset-culling. If there's a single outlier, for example, some methods would skip it, where it should really be highlighted.

<comment><USER21> [https://knowledge.ni.com/KnowledgeArticleDetails?id=kA00Z000...](https://knowledge.ni.com/KnowledgeArticleDetails?id=kA00Z0000019YLKSA2&l=en- US) is a good algorithm to use to keep peaks. At a maximum you need 4x the chart width data points. In practice, you can easily reduce that. I've implemented the algorithm before and it can be done in linear time.

</comment></comment></comment><comment><USER22> Slick! I too felt that many libraries were not slick/fast enough, but I eventually ended up using react-stockcharts (in canvas mode) for my use case and was quite satisfied with its performance.

Could you compare your performance against this one if possible?

[https://github.com/rrag/react-stockcharts](https://github.com/rrag/react- stockcharts)

</comment><comment><USER23> Minor suggestion: not really a bug, but inconvenient UX. When I move start the selection and I move my mouse outside the chart it stops selecting it. If you added the mouse movement listeners to the document/body then when I move my mouse outside the chart it would keep working.

Great job btw, really instant.

</comment><comment><USER24> For a moment thought here you were plotting 3 years of data collected every 150ms - wow! What you have here still very nice. I had to work with display/charting interval data in past and having it load quickly is very nice especially when toggling new dataset intervals.

</comment><comment><USER25> Would it be possible to allow for the cursor to still update when doing a horizontal scroll? When I move the mouse left and right it does that, but it doesn't move when I traverse the page by horizontally scrolling

Either way, just a bit of feedback on a beautiful page

<comment><AUTHOR> can you open an issue with a repro?

</comment></comment><comment><USER26> Beautiful demo, thank you for sharing.

I work with analytic tools to watch production systems and they boil my data down to a pathetic number of datapoints prior to rendering, when you _know_ computers — even in a browser — are vastly more capable.

<comment><USER27> I don't think systems like this have ever been really limited to the amount of data they can _present_. A bigger limiting factor is the volume of data they can ingest, at what cardinality (ideally arbitrary), and how quickly they can get it back to you, transformed in some way (e.g. "i want to see p95").

That said, things like SignalFx or Datadog are certainly slow at the presentation layer, too ;)

</comment></comment><comment><USER28> Wow this is amazing! It makes me really want to build a Python wrapper.

</comment><comment><USER29> I find it curious that the volume of users are similar for day of the month regardless of year. For instance the first of the month is not always the same day of the week.

<comment><USER30> Eyeballing the data, there doesn't seem to be a strong weekday effect. Not sure if this is real data (human activity data usually show weekday effects).

If there was a very pronounced weekday effect, a quick hack is to use a 364-day instead of full-year offset for comparison.

<comment><USER29> Agree. That's why i find it curious that it isn't representative of human activity but there definitely is seasonality.

</comment></comment></comment><comment><USER31> This looks nice. I'm currently using HighCharts and Anychart. Wish they'd included Anychart in their performance comparisons.

<comment><AUTHOR> AnyChart uses SVG, so it will be about the same place as Highcharts, but likely slower.

</comment></comment><comment><USER32> Congrats on the performance and thanks for posting. I especially appreciate how clean the source code is. Great work!

</comment><comment><USER33> Very fast indeed! How do I zoom out?

<comment><AUTHOR> drag. double-click to reset zoom.

</comment></comment><comment><USER34> Man, you forget how fast computers can be. This really opened my eyes.

</comment><comment><USER35> Cool!! I wonder how the performance compares to Highcharts?

<comment><USER36> For a dataset like this, Highcharts would be slower for sure since it will create DOM SVG components for each of the points. <canvas> (which this is built with) is much quicker at presenting tons and tons of data points like this.

<comment><USER37> Highcharts has a Boost module which uses WebGL.

Here's a sample chart with 1,000,000 points: [https://jsfiddle.net/5bvLgs5w/1/](https://jsfiddle.net/5bvLgs5w/1/)

<comment><AUTHOR> the bench code uses the boost module. so it's apples-to-apples:

[https://github.com/leeoniya/uPlot/blob/master/bench/Highchar...](https://github.com/leeoniya/uPlot/blob/master/bench/Highcharts.html#L12)

</comment></comment></comment></comment><comment><USER38> Excellent work - and a great Readme to boot too!

</comment><comment><USER39> Holy crap, this is bleeding fast. This is great!

</comment><comment><USER40> Dang, this renders FAST. Well done.

</comment><comment><USER41> that's really good..... but the bigger problem is getting all the data to the client.

</comment><comment><USER42> I wish Grafana was this quick

<comment><AUTHOR> there's this, but it looks stalled:

[https://github.com/grafana/grafana/pull/21835](https://github.com/grafana/grafana/pull/21835)

</comment></comment><comment><USER43> Super impressive, great job!

</comment><comment><USER44> So how many is the data?

</comment><comment><USER45> I feel like in general people underestimate just how fast computers are. Crazy to hear about libraries that choke on a couple thousand points when compared to [https://hackernoon.com/drawing-2-7-billion-points- in-10s-ecc...](https://hackernoon.com/drawing-2-7-billion-points- in-10s-ecc8c85ca8fa) (for instance)

<comment><AUTHOR> completely agree.

additionally, it's hard to overstate how impressive modern javascript JITs are, as well as GPUs.

every time i visit a web page that downloads 2MB of js and spins up my CPU fan, i feel ashamed that this is the industry (webdev) where i make my living.

at least with uPlot i show that all this waste is not necessary.

<comment><USER46> The problem with large websites is that there's a dozen layers of abstractions leading to large and inefficient code. But when it comes to just raw speed (iterating over a huge array of numbers in a tight loop, like drawing this chart), then it's blazing fast especially with JIT. That's specifically where a lot of people underestimate JS. I've seen a lot of people do things in the backend when they can just send all the data and do all the filtering/sorting locally leading to much faster UX.

</comment></comment></comment><comment><USER47> If you wanted to render these on a regular basis, WebGL is fairly straightforward, and works really well for this simple sort of rendering. You could do this with one polygon and a small fragment shader (treat the data as a texture, and use SDF to draw the line and fills), or use the actual geometry (render as a triangle strip; and separate the line if you want to do more interesting stuff in your fragment shader).

<comment><USER48> I probably wouldn't even generate an SDF, instead read the data directly from a 1D texture and fill the anti-aliased line directly in the fragment shader.

<comment><USER47> Yeah, I meant more in the abstract sense, rather than "create a two- dimensional texture of the distance field", which as you point out would be unnecessary.

You could probably sample all the 1D textures in one pass, and draw all the lines and fills there. One additional nice side effect of this is that you can easily have sequences at different resolutions.</comment></comment></comment>


Show HN: Huxley (HTTP JSON Proxy for UK Realtime Trains SOAP API) Staff Version - <AUTHOR> https://huxley.unop.uk/#staff-departures <comment><AUTHOR> You can click the example path links on the page to try this out using my demo server ([https://huxley.apphb.com](https://huxley.apphb.com)).

There's also a blog post with more details here: [https://unop.uk/huxley- staff-version](https://unop.uk/huxley-staff-version)

It's open source and the code is here: [https://github.com/jpsingleton/Huxley](https://github.com/jpsingleton/Huxley)</comment>


Wolfire open sourcing Aquaria, Gish, Lugaru HD, and Penumbra Overture - <AUTHOR> http://www.wolfire.com/humble?a=1

<comment><USER2> The title is misleading, Wolfire made Lugaru, not the other games. Those other developers have independently chosen to open source their games.

</comment><comment><USER3> Wow, this is incredible. I can't wait to see the source for Aquaria.

I guess that the humble indie bundle is over now that the projects are open sourced. Or maybe people will continue contributing just to be nice.

<comment><USER4> Only the code is being open sourced. I presume this means that the graphics, sounds and other resources will stay closed. Thus, if you want the games, you will need to buy them.

<comment><USER5> Well... for a penny, yes.

<comment><USER6> You are allowed pay just a penny, no one is stopping you, but don't you think it is a bit... not-nice? The payment services will take that penny in charges (maybe even more? Don't know who gets stuck with negative, but you are possibly causing indie devs and charities to LOSE money.), plus you are going to cost Wolfire some bandwidth on top of that.

</comment></comment></comment></comment><comment><USER7> I'm really, really glad that all of the games were available for Linux, and I'm glad the numbers show that Linux could be a viable gaming platform.

Massive kudos to all the developers!

<comment><USER8> I don't think the numbers mean much. After all, this stunt was based mainly on advertisement for the bundle page, and such advertisement seemed to be focused on techie websites (hacker news, slashdot, reddit). There is also the "finally, games for linux!" novelty that may or may not wear out quickly.

<comment><USER9> $250000+ just from Linux users is hardly meaningless. And even if it is just a matter of novelty, the results of this experiment could still influence other developers to start porting their games, and break the chicken-and-egg problem that prevents Linux from being a good and popular gaming platform.

</comment></comment></comment><comment><USER10> This whole phenomenon makes me happy. I already had 3 of the games, so the user set price hits my sweetspot.

The idea of open-sourcing the games after a set goal is achieved is interesting. Could there be some future business model there?

<comment><USER11> That was the "business model" used for Blender: The company developing it was bankrupt but the creditors allowed open-sourcing the software for a one-time fee of 100k.

</comment></comment><comment><USER12> Over a million dollars, wow, pretty amazing! According to the JSON file, each of the developers will be getting $130,000+, right? I'm sure they will be eating a nice success steak tonight... or a fatty veggie burrito for the vegetarians, heh.

<comment><USER13> And I'm sure a lot of entrepreneurial minded folks here on HN are taking note of this as well. A million dollars gross in 7 days for a bundle of indie games. Who knew? Let the gold rush begin, eh?

<comment><USER14> I was thinking about that.

On the one hand, it's not an easy stunt to reproduce. These are great games at a massive discount in a well-publicized event. Not surprising you can make money at that.

On the other hand, none of these games are very new. Games are usually like movies: a lot of sales in the opening weeks followed by a quick convergence to zero as they fall off the radar. A million dollars is not a _lot_ of money split seven ways and then split again among developers. But I'm sure it's great for games headed for the bargain bin.

Certainly there are a lot of games out there great enough in their day to be worth playing even now that technology has moved on. HOTU is evidence enough of that. A service that sold _only_ hand-picked classics, polished for modern platforms, at cheap prices? Yeah. I'd buy that.

<comment><USER15> "A service that sold only hand-picked classics, polished for modern platforms, at cheap prices? Yeah. I'd buy that."

<http://www.gog.com/>

<comment><USER14> Cool! Thanks!

</comment></comment></comment></comment></comment><comment><USER16> This is a really interesting approach. I've been reading <http://jeff- vogel.blogspot.com/> for a while now and the whole DRM argument there has been that there's realistically a threshold on how much you make on a game before it pretty much becomes readily available to pirates and/or the initial sales interest is just gone anyway and the rest is not a reliable source of income. Something like that. So an approach which provides an incentive to maximize early sales during a brief period with a statistical understanding of what the target sales are and a reward to the public for getting there is really cool. It sort of embraces the piracy constraint and optimizes for being profitable in light of that.

Imagine if the maker of some new, high-budget PC game called "DRM GoreShooter" decides that they will be in good shape if they make $5 million in the first three weeks of sales and past that point they won't really be profitable anyway (DRM or not). They want a profit better strategy than trying to use DRM to extend the window of pre-piracy sales out to week four or five, which might or might not work anyway. So they sell the game with something physically in the package that's really cool and you can only get in the first few weeks and they let you report in your product code, one time, for credit on Facebook or wherever for you having sponsored the development of the next game and been a generally cool person paying for quality entertainment. Would that be a better approach? It might ultimately result in the same money, but could be done as a way to cost less or at least provide a great marketing push and build good will in the fan-base.

My argument would work better if I had any idea how much it costs to develop, maintain and manage a per-title DRM strategy, I suppose.

</comment><comment><USER17> I bought the pack for $50, from them $25 for donations. I'm happy to see it open sourced. Great job! And great service to the community!

</comment><comment><USER5> I can see the Penumbra engine being used to build some 90's style point-and- click adventure games in first person 3d, if someone was so inclined. Maybe even something along the lines of Myst. However those genres are probably dead for a reason.</comment>