Some bad Git situations and how I got myself out of them - <AUTHOR> http://ohshitgit.com/ <comment><USER2> I can't believe no one has responded yet with "use a GUI". After gaining a basic understanding of how branches and merges work, and I do mean _basic_ , I've never been able to screw up a local repo with a GUI client enough that I haven't been able to recover with the same GUI tools.

I understand that people need to know how to use their tools, but for git most people can get away with the very basic usage that GUIs provide. If you've made some unrecoverable mistake with an important set of changes, you can always review the history in the same GUI and reimplement the important changes in a new branch.

<comment><USER3> I'm curious: What are the best free GUI for Git?

<comment><USER4> I'm surprised nobody mentioned Git Extensions yet. My absolute favorite Git UI, better than my previously favorite git-cola.

[https://gitextensions.github.io/](https://gitextensions.github.io/)

</comment></comment></comment><comment><USER5> One of the nice workflows that's already built in to the git command line tools is this one. When you're working on a branch and realise that a commit you made a few commits back has a mistake in it:

        # Make correcting change
        git commit --all --fixup=<ref of commit with mistake>
        # Continue working on branch, then at some point
        git rebase --interactive --autosquash

The --fixup option creates commits with subjects formatted like 'fixup! previous commit subject'. --autosquash uses these subjects when building the interactive rebase list.

Handy enough that I set rebase.autoSquash to true in my ~/.gitconfig so 'git rebase -i' always works like this.

<comment><USER6> Even better in combination with the `:/grep` syntax that allows you to specify the commit by grepping the commit messages, prefering more recent commits: `git commit --all --fixup ':/implement some feat'`.

<comment><USER5> Mmm - nice. Glad I posted it now, as I didn't know that variation!

</comment></comment></comment><comment><USER7> Somebody proposed to use a GUI. That doesn't solve the usability issues of Git. There's this triangle of what the user tries to do, what the commands and options are called and what they actually do. None of them really align, though with some careful use you can actually make Git do what you want - eventually.

I would like to understand what's the yearly damage of such an important tool being so difficult to use. People committing the wrong stuff, unmergeable messes, people not being able to correct their mistakes, there must be thousands of Git rookies fucking up their Git repo or throwing away their work just as I am writing this.

What would be the cost? Millions of Dollars? Perhaps even billions?

It's about as bad as 0 being a legit value for a regular pointer.

<comment><USER8> God, it's probably billions. I love git but its user interface is borderline criminal. The sad thing is mercurial has like 95% of git's power and is waaay easier to understand, but it never took off in a big way.

<comment><USER9>
      $ hg clone https://bitbucket.org/eigen/eigen/
      $ cd eigen
      $ time hg grep CUDA > /dev/null
      real	0m16.661s
      user	0m16.097s
      sys	0m0.531s

      $ git clone https://github.com/RLovelett/eigen.git
      $ cd eigen
      $ time git grep CUDA > /dev/null
      real	0m0.019s
      user	0m0.035s
      sys	0m0.057s

Never looked back.

<comment><USER10> Did you run hg grep once before to keep it in the disk cache?

<comment><USER9> It seems I cannot post a performance comparison without people asking about disk caches. :)

First of all, if I just cloned the repos, they should be in disk cache. My machine has 64GB of RAM.

Second of all, if they're not in disk cache, what on earth is hg doing that it takes 16 seconds to read them all in? Eigen is not that big, and I have a fast SSD.

Third of all, you can run the tests yourself -- I gave you the exact commands to try. "I tried it and my results are X" is a much better contribution to the discussion.

Fourth of all, yes, I ran the benchmarks multiple times in a row to check whether disk cache was at play. It was not. :)

<comment><USER10> > It seems I cannot post a performance comparison without people asking about > disk caches. :)

Sorry, the speedup was very great so it almost seemed like something else was causing it. Thanks for the benchmarks.

> Third of all, you can run the tests yourself -- I gave you the exact > commands to try. "I tried it and my results are X" is a much better > contribution to the discussion.

That's true; I'm lazy.

Actually, I was thinking more of the disk cache of loading hg, not the repo (Python programs tend to have a great speedup through the disk cache).

</comment></comment></comment></comment></comment></comment><comment><USER11> I absolutely love git now.

I'm still at uni (at a highly ranked but actually crap university where we don't learn git properly) and this year was my 'year in industry' as we call it in the UK, and my first proper experience with git, aside from `git init` at the end of my project and pushing it to a repo.

I've become so much more confident with git. Seriously, with one caveat (i.e., you haven't pushed your changes to a branch which other developers are working on), it is almost impossible to break irrevocably. Even if you do accidentally break master/develop/whatever, it only causes a bit of hassle and grumbling.

Highly recommend that everyone take a bit of time to learn about "undoing" git commands, whether that's through soft resets, hard resets to origin, or the reflog.

Reflog is also useful for figuring out how someone else broke something and explaining what they did wrong, since you can see what branch they were on at what commit and what commands they ran.

I think git's main problem is the somewhat arcane language it uses, and lack of understanding of what's actually happening _behind_ those words like "rebase", "commit", "patch", "reset" etc.

<comment><USER12> I would not expect a university to teach git. Maybe the theory of version control systems, their history, or a comparison of different version control systems.

But not how to use the tool.

<comment><USER11> Perhaps not specifically git, but ANY version control would have been good.

We were told to version control a group project, but all we were given was an SVN repo and told "if anything goes wrong, email this address".

Personally I would expect tuition through the use of a tool, that tool doesn't have to be git, it just so happens to be the tool I've had the most exposure to.

</comment></comment></comment><comment><USER13> A somewhat lesser known git trick that's pretty much a strict improvement - use `--force-with-lease` for force pushing instead of `--force`.

What this does is check what's on the remote branch and compare it with what you think is on the remote branch, and only do the force if they're the same thing. So if someone pushes a commit, the force push errors out instead of silently overwriting it.

Basically every single time you want to force push, you probably should be doing a `--force-with-lease` instead. I can't think of a situation where you'd want to silently lose commits you don't know about rather than get an error.

<comment><USER14> I also can't think of a situation where you'd want to force push to a branch that others might be committing to.

<comment><USER13> If they're not committing to it, --force-with-lease is the same thing as --force, so it doesn't cost you anything.

<comment><USER14> Costs ten keystrokes by my count. But it's not the keystrokes that trouble me, it's the idea that one of my co-collaborators might use this feature to "safely" rewrite history in a branch I'm also using. "Safely" being relative, since the rest of the collaborators will have to force pull that branch later, and rebase their pending changes.

By my reckoning, rewriting history just isn't a reasonable decision in shared branches.

</comment></comment></comment></comment><comment><USER15> One thing not covered very well was what to do if you push to origin. My favourite way to fix this: use git revert to create an exact opposite commit to your bad commit.

      git revert <bad commit>
      git push

It leaves a history of the mistake, for better or worse, but it does undo the mistake on origin.

<comment><USER16> Never do this on a merge commit however. Unless you are prepared: [https://git-scm.com/blog/2010/03/02/undoing-merges.html](https://git- scm.com/blog/2010/03/02/undoing-merges.html)

<comment><USER17> This is why rebase is so, so much better than merge. And it's infinitely better for bisecting, too.

Simple, completely linear history for origin/master is just so powerful in many that aspects that I'm constantly baffled why merging seems to be the flow mostly being talked about.

Now someone usually comes and says that merges are superior for long running branches. Which may be true in some aspects, but when you have more than one "running" branch and you got to try and find which commits have the code changes that only together seem to break and this is after 2 non-trivial merge commits, you really start to wish you'd gone with rebase.

<comment><USER18> For those of us who are uneducated / haven't seen this before: how do you use rebase to merge a separate branch? Are you suggesting you just `git rebase my- branch` from master / develop? How do you coordinate this with multiple team members. Merges appear to work better when you have multiple team members with separate feature branches that may have some overlap (conflicts may occur, but those can be handled as they arise).

<comment><USER19> First off, see [0] for a very helpful insight into the rebase command's syntax.

With multiple team members (each on their own feature branch), I encourage what I call the "pitchfork" approach [1], where the feature branches get stacked up into an integration branch to preview what the master branch would ultimately be when all the feature branches are merged in via pull requests.

A key part of building and recreating the pitchfork structure is git rerere [2].

[0] [http://matthew- brett.github.io/pydagogue/rebase_without_tear...](http://matthew- brett.github.io/pydagogue/rebase_without_tears.html)

[1] [https://gist.github.com/dkaminski/c8e59221bea74ab1fea615a468...](https://gist.github.com/dkaminski/c8e59221bea74ab1fea615a468e3f4cf)

[2] [https://git-scm.com/blog/2010/03/08/rerere.html](https://git- scm.com/blog/2010/03/08/rerere.html)

<comment><USER18> Hey @malingo, sorry I didn't reply to this immediately, but here are some of my initial thoughts. I apologize if this whole post sounds like a bit of a ramble, I'm writing this late in a train of thought. First thoughts:

1) git-rerere is really cool, and can probably make things very convenient, especially if you do several rebase operations to stack your integration branch. Without git-rerere, I can't imagine this workflow being remotely sane (resolving the same conflict every time you try to add an additional commit on one of the branches would get old fast).

2) This approach could probably help in "merging" (not git-merge, but you know) several feature branches with lots of conflicts, but does sort of rely on "As long as you make clean commits on the correct feature branches". While it's a laudable goal, how often can this really be the case, especially in projects where contributors are not composed of a single, core team (i.e. open source or research based projects)? I would think that ensuring clean commits is hard enough without a fixed team, so I am genuinely curious how hard it is to enforce this assumption.

See, here's the thing -> integration like this seems to work really well, especially if you're sharing branches across the repository with several teammates / coworkers. But what if you're not sharing branches? In the example you provided at [1], you have three branches: A, B, and C. Because you have access to all branches, it's easy to make an integration branch that does master -> Ai -> Bi -> Ci, and to handle all the conflicts with rerere. Then as A, B, C evolve and more commits are pushed, you could even have a build-bot of some sort assist in rebuilding the integration branch and checking for significant conflicts or errors. Of course, this all works great, but git is distributed.

What that means in practice is you may have master and branch A on your machine, and your two coworkers might have B and C on their machines, but none of you have rights to push these branches to the main repository location. This is common in BDFL-type projects, where one person or a handful of people are the only ones capable of pushing branches to the main repo. If your organization is split up like this, and individual developers tend to work on fixing single issues at a time, then you really never get the opportunity to use the integration branches, since you're only ever integrating your own (single) branch! If there's a bottleneck on upstream whatsoever, then the difference between this model and merging isn't really apparent, as you're not really getting any benefit either way (rebase -i non-withstanding). It would seem this would just kick the can down the road for the next developer to rebase off of the new master (with A rebased/merged in) and then solve their problems locally. This could be a significant duplication of effort if handled poorly.

I suppose you could have an intermediate "integration" repo where everyone pushes their branches, and build this sort of setup in assistance with some sort of CI, but I wonder how that scales if you have > 20 issue branches. The reason I ask this is because in the pitchfork workflow, how do you know which branch gets integrated first, second, etc.? I'm sure there's somewhat of a natural ordering in practice, as some issues or features probably take priority over others for some reason or another, but what if you don't know? What's the best way to keep the history? Certainly history isn't everything, but you may want to bisect down the road and A -> B -> C may cause more problems than B -> C -> A (you might not know at this point in time). Furthermore, do you not need someone / several people working together to resolve these conflicts? Is it sane to keep bothering your coworkers with "hey I just made a new integration branch, what do I do about conflict X", while they're still pushing new commits to B or C?

I think the pitchfork strategy for rebasing onto master is intriguing, and I'd have to play with it for an extended period of time to really get answers to some of these questions. Unfortunately, it seems that the bottleneck issue isn't something that can be easily solved for most projects. I don't come across many projects that allow every contributor to push branches, especially if they allow for pull-requests of any kind (even for one-off contributions). Nonetheless, I've definitely learned something here, and I think I understand better why one might want to rebase instead of merge. Then again, pull- requests are very heavily tied to merge-based flows, and if the project I'm contributing to wants to use them, then I'm probably hooped to begin with. Git is great for letting teams decide how they want to work, but it can be very painful if even one contribution doesn't follow guidelines (both to break a consistent history / commit graph, as well as if you ever have to git-bisect over it).

[1] [https://gist.github.com/dkaminski/c8e59221bea74ab1fea615a468...](https://gist.github.com/dkaminski/c8e59221bea74ab1fea615a468e3f4cf)

</comment></comment></comment></comment></comment></comment><comment><USER20> If you're concerned about not knowing how to do certain things with git, and understanding at a deeper level how git works, I highly recommend reading Scott Chacon's "Pro Git" book:

[https://progit.org/](https://progit.org/)

<comment><USER21> I second your recommendation, this book has saved my bacon multiple times.

But does anyone know why this project has two homes?

[https://git-scm.com/book/en/v2](https://git-scm.com/book/en/v2)

<comment><USER20> My understanding is that the dedicated site above highlights the book itself, while git-scm.com recommends/promotes the book.

</comment></comment></comment><comment><USER22> With all these recipies - one thing I do whenever I attempt some stunt in git: I assign temporary tags to every changeset that's important.

        git tag tmp
        git perform-stunt

This eases undoing the stunt without needing to find the "before" state from reflog. And if you use a graphical log viewer (I like SourceTree on Mac) you'll see the tagged state in the history view - which makes things a lot clearer.

And to be aware what happens, there one single explanation of git that helps a lot: [http://eagain.net/articles/git-for-computer- scientists/](http://eagain.net/articles/git-for-computer-scientists/)

As soon as you start viewing git as a graph of nodes with branches/tags just being "marked" nodes a lot of things make sense, and whatever "git perform- stunt" you attempt it's easy to explain within that mental model.

<comment><USER23> [http://www.robertames.com/blog.cgi/entries/git-in-two-ten- mi...](http://www.robertames.com/blog.cgi/entries/git-in-two-ten-minutes.html) ... feedback welcome, but I feel like I go into a bit more depth using similar metaphors.

</comment></comment><comment><USER24> The last addendum reminds me of this inexorably relevant xkcd entry: [https://xkcd.com/1597/](https://xkcd.com/1597/)

<comment><USER25> Shhhh ... we all have to pretend we do git properly.

</comment></comment><comment><USER26> > Oh shit, I accidentally committed to the wrong branch!

Other ways to do it (that don't require to retype the commit message): \- rebase onto the correct branch:

      git branch foo

      git reset --hard HEAD~

      git rebase --onto name-of-the-correct-branch HEAD foo

      git checkout name-of-the-correct-branch

      git merge foo

      git branch -D foo

\- cherry-pick

      git reset --hard HEAD~

      git checkout name-of-the-correct-branch

      git cherry-pick name-of-the-branch-you-mistakenly-committed-to@{1} (or git cherry-pick HEAD@{2})

> Oh shit, I tried to run a diff but nothing happened?!

You probably want to know `git diff HEAD` too.

Edit: formatting.

<comment><USER27> Or use git branch -m to rename master to something else, then checkout master anew.

<comment><USER28> That's clever, I'll remember that. But don't you lose all tracking information if you do that?

<comment><USER29> What tracking information are you referring to?

</comment></comment></comment></comment><comment><USER30> Surprised there was nothing on messed-up merges or rebases. They're some of the worst to get out of when you're not totally comfortable with git yet.

<comment><USER31> Any points on how to continue from there? Been in a few "I give up" conditions during rebases that I completely stopped trusting git it any way.

The thing is, some repositories on github _require_ you to commit only after proper rebasing. But I can never in my life remember how and need to google it...

<comment><USER32> To abort a screwed up merge:

        git merge --abort

To abort a screwed up rebase:

        git rebase --abort

Both of these will take you back to immediately before you began the operation.

</comment></comment></comment><comment><USER33> 'sudo rmdir'? I don't think that does what they think it does.

<comment><USER34> For posterity, `rmdir` will only remove empty directories. The author presumably should have used `rm -rf`.

<comment><USER35> I think the command we're looking for is 'rm -R name_of_directory'

<comment><USER36> Honestly, the proper command is not rm at all, because you can then have an "oh shit I forgot something" moment. I have learned (the hard way) to retain even an horribly broken repo, but shove it out of the way. Because disk is cheap, repeating work is expensive.

So that example should have been:

        cd ..
        mv fucking-git-repo-dir fucking-git-repo-dir.archived.$(date +%s)
        git clone https://some.github.url/fucking-git-repo-dir.git

<comment><USER37>
      git fetch origin master && git reset --hard origin/master

imo

</comment></comment></comment></comment></comment><comment><USER38> Based on the article, and many of the comments here, I didn't realise how comfortable I have become using git!

For example, the last "bad situation" I had to get myself out of involved unreadable .git contents caused by filesystem corruption. If you can "rm -rf fucking-git-repo-dir" then it's not too bad; when that fails with an IO error is when things get interesting!

</comment><comment><USER39> I really like this article but there is a problem with it: what happens if I use one of your techniques and I screw up? These steps you describe are a black box to someone who is no git savvy yet. While these definitely help but they propagate the "git is scary, cross your fingers" mentality. What I mean by this is that the reader won't be any wiser after reading

> git reset HEAD~ --hard

What is ~ after HEAD? What is --hard? Is there a --deep option as well?

So I think that you could upgrade this with some annotations over the cryptic parts with a little explanation. What do you think?

<comment><USER40> On the other hand all of these options are well documented in the git book and manual, so it's quite easy to check the details you're not sure about (and learn something new in the process).

</comment></comment><comment><USER41> Doing mumbo-jumbo between branches with `git stash` is way to hell. Don't do it, you lose data. This fucker will unstash changes until first conflict, then it stop and present you with <<<<<<<ID and >>>>>>>ID which nobody understand. Well I understand it, but never know which is which (theirs/ours label don't help here). You try to undo everything, but then you're fucked - all unstashed changes are removed from stash while conflicts are still there. You must be very careful now not to lose changes. You won't succeed!

That is I believe `git stash` should be removed from git as evil data loosing feature, not needed. Instead just make an alias `git save <TEMP_BRANCH_NAME>` which saves your temporary work to the branch:

`save = !sh -c 'export PREV=$(git symbolic-ref HEAD|cut -d/ -f3-) && git checkout -b "$1" && git commit -am "$1" && git checkout "$PREV"' -`

<comment><USER42> You can use `git stash apply` to apply the stashed changes without removing them from the stash.

</comment></comment><comment><USER43> Actually the easiest thing is simply not to care about how your log looks.

If you don't then there are ry only two things you need to know how to do:

If you didn't push to origin do an ammend. If you did, revert soft and commit the previous code to revert it (you can also put a stash or patch to apply it back).

Which frankly is what the article does, basically.

<comment><USER44> > Actually the easiest thing is simply not to care about how your log looks.

You need to expand on this because i don't quite understand what you mean, are you talking about "fix typo" commits or are we getting to the level of just committing away until things work and then not cleaning up the work later on? The linked article doesn't cover rebasing or squashing commits, which can be pretty powerful when used correctly.

Your commit log is the _one_ thing that is _immutably_ linked to your code changes. Your documentation isn't, the comments in your code aren't, you will forget why you made a change, anyone reviewing your logs needs to understand your intention, your bug tracker will probably change several times over the life of a project, and so on. So, make an effort to have a commit log that is clean and comprehensive and commits that don't break tests.

I go into this in more detail in a talk i gave last week: [https://www.youtube.com/watch?v=9OHAq8dCoS4](https://www.youtube.com/watch?v=9OHAq8dCoS4)

<comment><USER45> Develop on feature branches, merge with master only when the feature is done. Always merge with --no-ff. Then, when reviewing history, on the master branch you can use either "git log" or "git log --first-parent master". The first log is the complete log, with exploratory development, mistakes, backtracks and such. The second is the clean log people keep rewriting history to obtain.

You get your cake and you can eat it too.

<comment><USER44> This is pretty much the reason i don't understand people who don't want merge commits (the use case for rebase in the video is modifying commits in a feature branch BTW, _not_ merging to master). You can bend git logs to your will with the various options so there's no problem in being as verbose as possible in commit messages and the concept of a "linear history" is a moot point because, no matter how messy your history actual is, there's a command line option to clear it up.

Be verbose in your commit messages and don't worry about merge commits, it will pay dividends later.

</comment></comment></comment></comment><comment><USER46> lots of these are unnecessarily complicated:

> Oh shit, I accidentally committed something to master that should have been > on a brand new branch!

        # disappear the last commit and all changes from it
        git reset --hard HEAD^
        # make a new branch using the last commit
        git checkout -b new-branch HEAD@{1}

> Oh shit, I accidentally committed to the wrong branch!

first, you don't need to git-add before _and_ after stash, stash will save the working directory and the index (as documented in the DESCRIPTION of git- stash(1)). but for a more logical way:

        # disappear the last commit and all changes from it
        git reset --hard HEAD^
        # get onto the new branch
        git checkout new-branch
        # grab the stuff from what was on the old branch
        git cherry-pick old-branch@{1}

> Oh shit, I tried to run a diff but nothing happened?!

        git diff --cached

recommended reading for intermediate git users: the DESCRIPTIONs of all of these commands (git-reset(1), git-checkout(1), git-cherry-pick(1), git- diff(1)), and the entirety of gitrevisions(7).

<comment><USER47> I personally prefer this because old-branch@{1} is a little weird..

      git checkout new-branch
      git cherry-pick old-branch
      git checkout old-branch
      git reset --hard HEAD^

</comment></comment><comment><USER48> Getting to "Fuck this noise, I give up." is a very clear indication that you aren't competent enough and you should take a GOOD course about git as soon as humanly possible.

Shameless plug: [http://engineering.hipolabs.com/how-to-work-in-a-team- versio...](http://engineering.hipolabs.com/how-to-work-in-a-team-version- control-and-git/)

<comment><USER49> > Getting to "Fuck this noise, I give up." is a very clear indication that you > aren't competent enough

I think that's a tad judgemental for two reasons:

1- because git is legitimately hard to learn. Personally I suspect git is unnecessarily hard to learn, that command names and the concepts and workflows I need are possible to learn and use with less effort. For many years people around me have been asking for git help because I know how to recover from lost stashes and use the reflog, etc., and yet I still have to google the magic incantations for commands I use regularly because they're impossible to remember.

2- There is actually a lot of value in being able to spin up a new repo instantly, in knowing that you can, and in practicing it often. Not unlike the move to VMs for development environments. Plus, there are definitely bad situations where a fresh git clone is the simpler way to go -- just not in this article. ;)

Anyway, I also agree with you because this blog post doesn't describe any truly bad situations, and because for years I've seen people blowing away their repos and starting over, and always thought to myself it was funny. It's a drastic action that takes more work than a rebase or reflog or whatever the problem was, and doesn't work well if you've made changes.

<comment><USER28> 1- That's certainly true, git is really hard to learn. But if you're a developer it's worth getting comfortable with it, which means not going for the "nuclear option" every time something weird happens. Someone on StackOverflow has had your problem, and has the perfect fix, look that up instead.

2- Yes, absolutely, it's good to remember that it's a fairly painless and easy process to spin up fresh clones. Though, if all you want is a fresh copy, in 99% of cases a "git clean -xfd" will do that for you (read the man page to find out what the options mean! man pages are your friend!). Though that one is generally a "pull ripcord in case of emergency" type of command, "git stash" generally suffices and avoids risk of losing data.

<comment><USER49> I totally agree, git is worth learning.

I don't recommend git stash, it frequently causes bad situations. Just branch instead, branching is safer and just as easy as stashing. Stash is not as safe as other git commands. From the man page: "If you mistakenly drop or clear stashes, they cannot be recovered through the normal safety mechanisms."

</comment></comment></comment></comment><comment><USER50> This doesn't even cover half of the bad situations I've gotten myself in over the last few years :D

Long term, it's best to thouroughly read the man pages, e.g. nicely formatted here: [https://git-scm.com/docs](https://git-scm.com/docs)

</comment><comment><USER51> Another option is to use Mercurial with hg-git to GitHub for network effects.

I've been doing that for a while for dropbear ssh, it does hit occasional problems but is overall more pleasant than straight git.

<comment><USER52> To each his own, of course. As a Git aficionado, I find Mercurial utterly baffling. Branches are permanent and global? You have to clone the repo to make a new thing I would call a branch? "hg log" shows you things that are not in the history of your current source tree? Mind boggling.

:)

<comment><USER53> What git calls branches is most similar to what Mercurial calls tags.

<comment><USER54> You mean Mercurial bookmarks.

</comment></comment></comment></comment><comment><USER55> There is no substitute for understanding what's going on, especially using a power tool like Git.

It's a cute website, and useful, I really like it. This sentence,

        Bizarrely, git won't do a dif of files that have
        been add-ed to your staging area without this flag.
        File under ¯\_(ツ)_/¯"

just screams to me (a professional Git trainer), "I don't understand the Git staging area! I don't know my Git fundamentals! Train me!"

<comment><USER56> Any resources you recommend for learning git in more detail?

<comment><USER55> 1\. Our "Git Fundamentals: Basic Concepts and Definitions" webinar, [http://www.verticalsysadmin.com/git/](http://www.verticalsysadmin.com/git/) (you have to register) lays the groundwork you need to start studying Git.

If you have a team that needs Git training, email me and we can deliver a instance of this webinar (complete with Q&A session) free of charge (as an introductory service).

2\. For someone just starting out, I also recommend "Learn Enough Git to be Dangerous", [https://www.learnenough.com/git- tutorial](https://www.learnenough.com/git-tutorial) (HTML version available for free), from @mhartl, author of the popular RailsTutorial.org

3\. The "Pro Git" book is great. [https://git-scm.com/book/en/v2](https://git- scm.com/book/en/v2) A comprehensive resource.

</comment></comment></comment><comment><USER57> I don't know if this post was intended as humour or a way to vent out some frustration but in my experience, this path of treating git as "spell X solves problem Y" will always break down.

Version control systems are an important part of the programmers toolkit and it's worth investing a little time to get the fundamentals right.

Sure git is not the friendliest of beasts but what it lacks in interface, it more than makes up in internal consistency and trying to learn it "inside out" is a better long term investment than having a list of ways to solve "git problems".

<comment><USER58> "Not the friendliest of beasts" is putting it mildly.

Git is basically Linus in a nutshell: abrasive, unforgiving, and behaving like an absolute asshole to any non-expert struggling user.

Sure, engineers should probably get to know its quirks and learn to work around them because it's now ubiquitous in the field, but let's not pretend that there's something virtuous about it. This is a piece of truly terrible software with an even worse UX that just happened to win the PR battle against its superior foes, because...Idunno, Linux, I guess?

<comment><USER59> What are the superior foes in your opinion ?

I don't know any of the other distributed version control systems. The only other version control system I have much (too much) experience with is Subversion and it can't hold a candle against Git.

In my opinion Git is the C language of version control systems. If you are careless it's not the tool for you. Otherwise you have a really great tool with lots of power.

<comment><USER60> Can you justify git>subversion or are you just following the sheep?

Not saying I disagree... but subversion is still king in the corporate- engineering companies and it tends to work well in those types of setups.

Just beware of having unjustified-opinions. SVN may not be trendy... but neither will git in a few years time.

BTW: Mercurial still beats them all.

<comment><USER61> >subversion is still king in the corporate-engineering companies and it tends to work well in those types of setups.

For some values of "work". We're using SVN at my new job, with a single trunk branch. Not having the ability to commit locally, and mess around with local branches is a pain in the ass. I found a solution though. I cloned SVN repo using git-svn and now I can make local commits like a pro, and use my beloved magit (Emacs git interface).

<comment><USER62> Yes, git-svn is probably _the_ reason why the mutiny didn't happen yet in some departments. You can identify their users by having several SVN commits in as many seconds.

(I used git in an cvs-to-svn migration, and stayed since.)

</comment></comment></comment></comment></comment></comment><comment><USER63> One of my SO questions could almost fit in there, somewhere before "Fuck this noise...":

[http://stackoverflow.com/questions/28759887/git-gc-no- space-...](http://stackoverflow.com/questions/28759887/git-gc-no-space-left- on-device-even-though-3gb-available-and-tmp-pack-only-16m)

</comment><comment> Anthony-G From the section on using `git commit --amend`:

> This usually happens to me if I merge to master, then run tests/linters.

If this happens on more than one occasion, I'd strongly consider creating a pre-commit hook to run tests and/or lint the changed files, e.g., I run `checkbashisms` and `shellcheck` as a pre-commit hook when working on shell scripts.

</comment><comment><USER64> I constantly wonder if there's a "better CLI UI" that can be made for git, even if simply a wrapper around git.

Ie, if the implementation of Git is right, but only the cli commands/etc are wrong.. what would the right UX be? What would a friendly UX look like?

Seems like something a lot of people could love - even if blasphemy to Git purists.

</comment><comment><USER65> Great article. A couple notes

Just git reset <ref> should do. The --soft flag is implicit.

Amending and rebasing is something you should be careful with. If you've already pushed, you'll now need to force push. If another person has put new commits upstream, they will be wiped out irreparably. Not saying don't do it, just that it's very risky.

Instead of deleting and recloning the repo at the end, if you're really at that point, just doing git reset --hard origin/master should be equally as effective with fewer steps and less time.

Pulling down the repo a second time, however, can be more useful for just having a snapshot of the code in a second location that is totally independent of git traffic (don't pull to it very often). Say someone force pushes something that removes code. Your snapshot or someone else's non-pulled repo is your only hope of getting it back.

<comment><USER66> Just a tiny correction: git reset <ref> defaults to --mixed, not --soft. The only difference is whether or not files are staged. Since the OP used --soft, I believe the subsequent `git add .` is unnecessary since the files should already be staged when --soft is used.

Regarding the case of an erroneous force push removing code, usually things can be recovered by looking through the reflog and doing some surgery via the lower-level plumbing commands... it's not exactly a fun process though :)

</comment></comment><comment><USER67> I've heard advice from #git on freenode not to use "git commit --amend", especially on shared repos.

I wish I still had a log of the conversation or remembered the exact problem that led up to it, but it involved a simple amend totally screwing up my repo, and I've avoided it since.

<comment><USER68> The problem occurs when you amend a commit that someone else is already using. That is rewriting the shared history. A merge commit is required to get the branches in sync again.

Amending your commit in your own feature branch or fork isn't a problem. I often amend commits to make a cleaner git history to ease PR/code review.

        git checkout master
        git checkout -b 10-new-feature
        # Make changes
        git add example.code
        git commit
        git push origin 10-new-feature
        # Oops... left a debug message, fix it.
        git add example.code
        git commit --amend
        git push -f # Rewrites the remote history with your local modified version

Consider --amend as the "Just the last commit" form of rebasing.

</comment></comment><comment><USER64> I've been debating starting a section of my personal site for stuff like this. Unfortunately it's a bit embarrassing, but i figure anything i have to google to learn, it would be beneficial to help others learn it as well. Everything from programming languages (lots of Rust errors are foreign to me, for example), to git issues.

I've often had the thought that if everyone did this it could have potential to be quite the open-sourced collection of material - a distributed self- answered Stack Overflow perhaps.

Is there any sanity in this? Or would posting everything as self-answers to Stackoverflow be more welcome to the average Googler? _(higher ranking, better meta, more likely for the user to see it and community features such as commenting /voting/editing)_

<comment><USER69> Please file bugs for bad errors in Rust. We have an entire tag dedicated to them, and have been doing a lot of work to make them better. A non- understandable error is a bug.

</comment></comment><comment><USER70> I just uploaded a presentation I gave last February called "Git back on your feet":

[http://www.slideshare.net/ChristianCouder/git-back- onyourfee...](http://www.slideshare.net/ChristianCouder/git-back- onyourfeet-65866467)

</comment><comment><USER49> Great idea! We need more basic git workflows described in plain English.

I was expecting some actual "bad" situations based on the title, and to be fair these were bad to me once and are bad for people new to git, but I'd love to see the level 2,3,etc. version of this article.

<comment><USER37> Here's my standard recommendations:

* Learn Git Branching (gentle introduction to git's UI, with a visualization of the DAG) [0]

* Git for ages 4 and up (beginner-oriented, teaches you how to model git's internal concepts, 90m video) [1]

* Git from the bottom-up (high-level overview of git's internal data structures) [2]

* My Git Habits (sophisticated, you will have to study the man pages used to completely internalize the workflow) [3]

[0]: [http://pcottle.github.io/learnGitBranching/](http://pcottle.github.io/learnGitBranching/)

[1]: [https://youtu.be/1ffBJ4sVUb4](https://youtu.be/1ffBJ4sVUb4)

[2]: [http://ftp.newartisans.com/pub/git.from.bottom.up.pdf](http://ftp.newartisans.com/pub/git.from.bottom.up.pdf)

[3]: [http://blog.plover.com/prog/git- habits.html](http://blog.plover.com/prog/git-habits.html)

</comment></comment><comment><USER71> You know, a few years back I had exactly the same attitude towards Git. I hated it, tried to approach multiple times and still hated it. But after working with Git on multiple projects, and having developed some deeper understanding of how things work there, I honestly think now that Git is one of the best tools that have been invented over last 20 years for improving the development processes and engineering excellence. I don't really want to defend it, the learning curve is apparently pretty steep (at least it was for me), but I would just recommend to not give up and keep trying. There are some good tutorials how to handle it and why it's helpful (Atlassian's Git book, for instance).

</comment><comment><USER72> Oh shit, I commited a binary file larger than 100 mb and now i can't push to github.com. Solution: [https://rtyley.github.io/bfg-repo- cleaner/](https://rtyley.github.io/bfg-repo-cleaner/)

<comment><USER73> Is there some tool that runs a full internal consistency check on a git repo?

Because otherwise, how can I be sure that some tool didn't mess up my repo, and one day I'll be getting error messages when checking out files, etcetera?

<comment><USER74> git fsck

<comment><USER75> This. Also, hold on to the commit before you started refactoring (preferably by placing a temporary branch there, in order to avoid premature garbage collection). Then after your refactoring is done, `git fsck` and `git diff --stat $old_branch..HEAD` to verify that you did not lose any important content during the refactoring.

<comment><USER76> That is true in general. However, in the context of bfg-repo-cleaner, that doesn't help, because the purpose of the bfg-repo-cleaner is to completely remove commits that match the criteria you give it. I say this not to "correct" you per se, because you are correct, but to ensure that people using bfg-repo-cleaner do not believe that this will help them.

The "correct" course of action prior to using the bfg-repo-cleaner is to make a complete copy of the target .git directory. I've used it many times and it seems quite reliable, but if nothing else, that's a good defense against terminating it halfway through accidentally or something. (Of course even without that, you shouldn't be operating on the only copy of your .git repo anyhow since it is almost certainly somewhere else, too, or you wouldn't be having the problem bfg-repo-cleaner is designed to solve in the first place. But losing your own local branches and work is still a pain.)

</comment></comment></comment></comment></comment><comment><USER77>
        git add .

Ugh, no, never do this, never recommend to users to BLINDLY ADD ALL THE CHANGES FROM THE WORKING COPY. I honestly can't think of any worse git usage than this.

Add the single change you missed, or even better, `git add -p`, to add chunks manually.

</comment><comment><USER78> The screwed up and committed to master should end with:

         git reset --hard origin/master

(assuming that the remote is called "origin") With the example in the text, you have to know the number of commits you've made to master.

<comment><USER79> It was also a bit pointless to checkout a new branch just to switch back. The whole thing should be

        git branch some-new-branch-name
        git reset --hard origin/master
        git checkout some-new-branch-name

<comment><USER79> The "committed to wrong branch" case is also a little overly complex. I'd just do:

        git checkout correct-branch
        git cherry-pick wrong-branch
        git checkout wrong-branch
        git reset --hard HEAD~

<comment><USER80> This is less safe than the branching alternatives. Instead of "moving" the original commit, this copies the original commit to a new one and leaves the original one dangling and waiting to be garbage collected.

Also, if you have more than one commit before you noticed you were on the wrong branch, this only grabs the one commit.

</comment></comment></comment></comment><comment><USER81> > Bizarrely, git won't do a dif of files that have been add-ed to your staging > area without this flag

Not sure what's bizarre about that. Doing so helps keep your commits clean and helps git tools (diff, git-gutter, etc) by ignoring things you've already stated are complete.

For example, I quickly fix a bug. The code is ugly and I don't want to commit it yet. So I add the bugfix files. Then I clean up the code (before commiting). Now I can do another diff between the staged and unstaged files, checking that it looks better than before, and still works. This way there is 1 clean commit "bugix" and not 2 commits "bugfix" and "bugfix code cleanup"

</comment><comment><USER82> I managed to "rm -rf .git" at one point. Took me about a minute to realize and -surprisingly after <c-c>-ing i lost nothing (as far as i was aware). Git is freaking hard to break. Also always remember git-reflog, it safes lives.

<comment><USER83> What do you mean by <c-c>-ing?

<comment><USER82> Control C to cancel

</comment></comment></comment><comment><USER84> Also great for getting out of Git messes: [http://justinhileman.info/article/git- pretty/](http://justinhileman.info/article/git-pretty/)

</comment><comment><USER85> I prefer stackoverflow for things like this as I can see from comments and up votes whether the command does what the poster claims, or whether it is going to make things worse for me.

<comment><USER52> Spend a few hours and really learn how Git works! Then you'll understand what those commands actually do and be able to judge for yourself. Great resources are the ProGit book, gittutorial(7) and eventually gitcore-tutorial(7) when you want to go deeper.

</comment></comment><comment><USER86> Nobody likes to read manuals or books, due to which one can see FAQs being posted on Q&A sites.

[http://gitready.com/](http://gitready.com/) contains a number of small articles categorized by beginner, intermediate and advanced that might be helpful.

Another resource for commonly used git tips and tricks: [https://github.com/git-tips/tips](https://github.com/git-tips/tips)

</comment><comment><USER35> I moved to git at the begining of this year and I must say that I miss SVN. But everyone keeps telling me that git is better so I'm sticking to it.

</comment><comment><USER87> Well, there are many more situations you can get into.

Like cherry picking, force pushing, merge --no-commit, rebasing... almost any operation can end up going wrong.

Just pay attention.

<comment><USER88> But he has the final solution to any git problem:

      cd ..
      sudo rm -r fucking-git-repo-dir
      git clone https://some.github.url/fucking-git-repo-dir.git

<comment><USER87> He needs to create a hook that deletes the repo the next time they do anything with the repo.

</comment></comment></comment><comment><USER89>
        # create a new branch from the current state of master
        git checkout -b some-new-branch-name
        # remove the commit from the master branch
        git checkout master

Or just `git branch some-new-branch-name`...

        cd ..
        sudo rmdir nsfw-git-repo-dir

That will only remove it if it's empty? Which it never will be, because there's at least `.git/*`...

Still, amusing :)

</comment><comment><USER90> Fucking love it.

alias gitshit="open [http://ohshitgit.com/"](http://ohshitgit.com/")

</comment><comment><USER91> It's probably a good time to check if you have some safety against `rm -fr `.

Two days ago, I wanted to delete .git only, but accidentally as my fingers were accustomed with -fr , the command was `rm -fr * .git`. Rails server was running and some hope arose at the moment to just `lsof | grep` .. unfortunately that didn't work with me !

Ironically, all dot files have stuck as obvious :)

<comment><USER86> > Ironically, all dot files have stuck as obvious :)

Those wouldn't have if you had set the following:

      shopt -s dotglob

</comment></comment><comment><USER92> git to me is a work of art. There is a lot of complexity underneath but the end user sees something that is simple, fast and easy to use. It scales depending on user needs and It's easy to reason about.

This is a feat of engineering, to take something complex and make it easy for anyone to undestand and use. It shows real expertise and deep understanding of the area.

In many ways it's a shining example against the 'culture of complexity' that we increasingly find ourselves in. Here rather than simplying the objective is to be to make thing as complex as possible, usually in pursuit of extremely niche use cases or because either the expertise or the interest to simplify is not there. If git was designed in this culture it would be fragile, full of buzz words, poorly documented and prone to failure, and something only a few self appointed experts could reason about and use properly.

</comment><comment><USER93> > _This usually happens to me if I merge to master, then run tests /linters... > and FML, I didn't put a space after the equals sign._

Am I the only one that runs my tests before committing, let alone merging to master?

<comment><USER94> Yeah, pushing untested code to master is not such a great way to work. Push to a remote branch other than master, test it there, then merge it into master.

That's pretty much git 101

<comment><USER93> Or test locally? I work in government contracting so everything is .Net and TFS, but even in this environment it's trivial to make sure your code isn't objectively broken:

      1. Click "Run all tests"

On your local machine even thousands of tests only takes a few seconds including the build process. We've got automated testing in the dev and test environments as well but if you're failing those there's a failure in workflow somewhere. There's very little reason to fail a test or to have a broken build in source control.

</comment></comment></comment><comment><USER95> > Oh shit, I committed and immediately realized I need to make one small > change!

If you don't want to change the commit message, in addition to --amend:

      git commit --amend --no-edit

</comment><comment><USER96> The post is missing the most important command : git --help

</comment><comment><USER97> It would be nice to add these two:

Oh shit, someone checked-in huge file that shouldn't be in repo.

Oh shit, this folder I had been working on should have been in its own repo.

</comment><comment><USER98> for the rest of it there is [http://git.io/git-tips](http://git.io/git-tips)

</comment><comment><USER99> why do you run git add . before git stash?

<comment><USER100> `git stash` will not add new files to the stash by default, so adding them first makes sure it does

<comment><USER99> But `git stash --include-untracked` will

</comment></comment></comment><comment><USER101> The last rmdir example should be rm -rf.

<comment><USER102> Maybe he has an alias named "rmdir" that executes "rm -rf"

</comment></comment><comment><USER103> I fear Git so much that I make zip packages of the repo before potentially destroying operations.

</comment><comment><USER104> One of my favorite teachers in school was a dude-bro programmer who pretty sure as younger than me. He'd spent a summer at Google and made us use git and gerrit. I'm honestly a much better programmer thanks to him. I'm still using the git cli to this day.

I also still say "new up" an object thanks to him. I'm not so proud of that one.

</comment><comment><USER105> Most of the time, stay calm. Do not `git push` hastily, and check `git status` if you can :)

</comment><comment><USER106> Surprised he finds git to be complicated. It probably is deep down, but for day-to-day use, compare it to SVN.

Until I switched, there was always a panic when branching or merging. With git, I can branch like a nutter and things seem to still work out in the end.

Not sure why, perhaps someone else has a perspective on it.

<comment><USER107> In my experience it's all fine and dandy until you hit a rebase or merge that needs manual attention.

<comment><USER62> In mine, not quite. All forms of merges are simple: resolve, add, commit. Inside a rebase I find it harder to memorize whether I need to to the add and/or commit or rebase --continue does it for me.

And conflicts from 'stash pop' are somewhat counterintuitive to handle.

</comment></comment></comment><comment><USER108> All of those things and more are way easier to do with gitk and git gui.

</comment><comment><USER109> If the owner is here: with javascript disabled the code is unreadable.

<comment><USER110> I'm curious, what would you say the % of sites you visit is where they're still useful without JS?

<comment><USER75> That question is not very useful the way it's written down. Some sites legimitately require JS (everything in the "web apps" territory; just try to imagine Grafana without JS), so I would ignore these (which I hope is in line with your intentions).

Of the sites that are mostly static content, I would estimate around 60%. There's a small chunk of sites that are just blank with JS disabled, and there's a larger chunk that will load images only when JS is enabled.

<comment><USER110> Yeah that was a poorly worded question, sorry. I'm interested in how far one can get without JavaScript on today's Internet. But it sounds like you don't disable it fully, just only on untrusted sites?

60% is better than I'd expect for sites you expect to work without JS. =)

<comment><USER75> I use uMatrix and enable first-party JS by default. That allows me to get by on most sites. Sometimes I have to enable additional domains, mostly:

\- ajax.googleapis.com and the like

\- somethingsomething.cloudfront.com

\- variations of the same domain (for example, hn.algolia.com queries algolia.net and algolianet.com)

By the way, it's a very valid question whether my model is actually more secure than just uBlock, but I like it on an emotional level because I feel in control.

</comment></comment></comment></comment></comment><comment><USER111> maybe also useful [https://gitlab.com/esr/reposurgeon](https://gitlab.com/esr/reposurgeon)

</comment><comment><USER107> The last one is my usual tactic.

</comment><comment><USER112> It is not the fault of Git if to use it you have to know it!

However would appreciate a quick and dirty handbook.

</comment><comment><USER113> LOL. Bookmarking along with my other Git reference sites...

</comment><comment><USER114> Not quite in the spirit of this article, but "I just want this /one/ file without the rest of the changes from branch foo" is something I use all the time

        git checkout otherbranch
        git checkout firstbranch -- fileIwant maybe1more
        git commit -m "brought over files"

</comment><comment><USER115> Git is complex and nuanced, and short term purple think it's faster to memorize some commands instead of understanding the fundamentals.

I kept having problems with git, so I read a fucking book on it [https://git- scm.com/book/en/v2](https://git-scm.com/book/en/v2)

I'm not saying I never get into situations I can't get myself out of, but the examples in the oh shit website now look like obviously trivialities.

<comment><USER116> It's easier to figure out the various workflows you need (most teams need very few - work on feature, sync code, commit/push/issue PR, release) and then scripting them.

I noticed that git errors on my team were almost eliminated when we started doing that - especially once I started adding sanity checks to the git workflows. It also allowed members of the team who previously struggled with git to contribute much more effectively - they'd be rebasing without even necessarily knowing what rebasing was, for instance.

It also meant that the git workflow, if it was version controlled, could also be amended.

This worked out much better than the git course people were sent on.

<comment><USER115> > It's easier to figure out the various workflows you need

Fine, that might be cheaper than properly training your employees. Of course, if your employees aren't training themselves out of genuine interest that's really the only option.

But then once something goes wrong you'll have to resort to "oh shit" websites and hope that you can the exact issue that you're having, since you really don't understand what's going on anyway.

> This worked out much better than the git course people were sent on.

I submit this must be because people came out of the course without really understanding the tool. Why they didn't understand I won't speculate, but I submit that they didn't.

EDIT: I think that which one is more appropriate depends on where it's used. I a big structured company I can believe that yours is better, or even crucial. But on a scenario where I'm starting a company with two other guys on a garage, I wouldn't want the guys to be the type of people who can only use a workflow that someone designed for them and never had the interest to dig under.

<comment><USER116> I tended to find that a few things were likely to go wrong and simply changing the git workflow scripts to account for those things was enough.

If things went wrong, people could come to me. That often resulted in me fixing their problem and fixing the script to ensure that kind of problem didn't happen again - after which people stopped coming to me because they didn't have problems.

Basically it was like releasing software.

>I submit this must be because people came out of the course without really understanding the tool.

I submit that not everybody needs to be an expert in the tool. Non-programmers can actually help a lot with writing stuff that needs to be version controlled with code - e.g. writing test scripts, updating translations, various configurations, etc.

</comment></comment></comment></comment><comment><USER117> Is the documentation really that bad? Would it benefit from a technical writer going over it? Is the project open for discussion on changes to the documentation?

<comment><USER118> Git is like Perl. And not in any of the good ways.

Perl 5 -- I don't know enough about 6 to know whether this is still the case -- involved a couple of design decisions which worked well for one subset of people and did whatever the complete polar opposite of "works" is for a different subset of people.

One of those design decisions is the famous "there's more than one way to do it". Ask five Perl programmers to solve a simple problem in the language and you'll get fourteen solutions, and no way of knowing which, if any, is to be preferred in real-world use. Git similarly tends to have multiple ways of achieving a given goal, and endless words have been devoted to documenting them; nobody has yet settled which of them is or should be generally recommended, or if there even can be such things as generally-recommended ways to use git.

Perl also involves an incredible amount of memorization. The large collection of "magic" variables which can be read or set to determine language and local- scope behavior are daunting and can't be mastered through any process other than rote memorization. And the differing behavior of basic operations depending on context produces a combinatorial explosion of possibilities which, again, are not amenable to any technique short of rote memorization. The same is true in git: though there are underlying abstractions, the interface provided to them is inconsistent, varies depending on sometimes- invisible contextual factors, and ends up requiring rote memorization of commands and options and how they behave in any given situation.

Unfortunately, no amount of documentation or review by technical writers can fix this. A bad API can't be made good through better documentation, it can only be made good through being replaced with a better API.

<comment><USER119> I am a long-time Perl user and also a long-time Git user.

Git is worse than Perl. Perl is just opinionated: despite what you are saying about it is true, you can write working programs and have fun at all levels of Perl mastery. There's a lot to memorize, but you can start coding immediately and do something useful. Also, Perl community is incredibly friendly and helpful.

Not so with Git. If you start without thorough understanding how Git internals work, you will get burned really soon. And more often than not, the only response you'll get is the arrogant RTFM or the link to Pro Git book.

</comment></comment></comment><comment><USER120> Yep, git sucks but it's all the rage now. Mercurial is 100x nicer to use and logical, but since it's written in Python it's slow as molasses, especially with large binary files.

Next on the list: Larry McVoy's Bitkeeper promises to be everything git and Mercurial aren't. (git is "inspired" (read: copycat) by Bitkeeper).

It's funny how _Sun Microsystems_ influenced the industry in so many ways, isn't it?

<comment><USER121> For anyone who wasn't aware, it's also open source these days - [http://www.bitkeeper.org/](http://www.bitkeeper.org/)</comment></comment>


How to use a hackathon to make contacts within your target market   - <AUTHOR> http://startupjuncture.com/2013/06/26/hackathon/

<comment><USER2> Piotr, how would this hackathon have panned out in case you wouldnt have won? (which was the case for the other 95% of the participants ;)) Or worded differently: would it have been worth it if you wouldnt have won?

</comment><comment><USER3> good tip, but he makes winning a hackathon sound easy. I am sure it isn't for everyone :-)</comment>


YOLO: Real-Time Object Detection - <AUTHOR> https://pjreddie.com/darknet/yolo/ <comment><USER2> Hypothesis: All demos are improved with the addition of dragonforce.

<comment><USER3> Discovery: Blaring music of any kind, including my preferred genres, is unpleasant when trying to watch a demo.

<comment><USER4> Doesn't hold up in independent tests.

</comment></comment></comment><comment><USER5> Nice! Btw this guy's CV is awesome :) : [https://pjreddie.com/resume/](https://pjreddie.com/resume/)

<comment><USER6> I wouldn't hire someone who formats their CV like that. Employment is a serious proposition, and ... Ah just kidding. Nice CV.

<comment><USER7> It's the kind of thing that you can only get away with if you're very, very good. At that point it becomes a kind of power move, like showing up to a high end interview in jeans and a t-shirt.

<comment><USER8> Or if you pretend that you are very good. If you show up to an interview in jeans and a t-shirt, people will think you are very good.

<comment><USER7> True, it can also be a form of peacocking. But in an industry where results are easily tested it's much harder to get away with that.

<comment><USER9> In this case, the guy has been spearheading a very high performance, innovative methodology for real-time object detection. Companies interested in that task should be falling over themselves to hire him.

</comment></comment></comment></comment></comment></comment><comment><USER10> TED talk: [https://www.ted.com/talks/joseph_redmon_how_a_computer_learn...](https://www.ted.com/talks/joseph_redmon_how_a_computer_learns_to_recognize_objects_instantly)

</comment><comment><USER11> This is cool but outdated, there're other better papers now.

<comment><USER5> What better papers would you recommend reading?

<comment><USER11> Arxiv is thou friend :)

[https://www.google.com/search?q=fast+object+detection+deep+l...](https://www.google.com/search?q=fast+object+detection+deep+learning+arxiv&oq=fast+object+detection+deep+learning+arxiv&gs_l=psy- ab.3...7628.7966.0.8024.5.4.0.0.0.0.102.315.3j1.4.0.foo%2Cnso-ehuqi%3D1%2Cnso- ehuui%3D1%2Cewh%3D0%2Cnso-mplt%3D2%2Cnso-enksa%3D0%2Cnso-enfk%3D1%2Cnso- usnt%3D1%2Cnso-qnt-npqp%3D0-1633%2Cnso-qnt-npdq%3D0-5608%2Cnso-qnt- npt%3D0-1229%2Cnso-qnt-ndc%3D2051%2Ccspa-dspm-nm-mnp%3D0-06145%2Ccspa-dspm-nm- mxp%3D0-153625%2Cnso-unt-npqp%3D0-1506%2Cnso-unt-npdq%3D0-4694%2Cnso-unt- npt%3D0-061%2Cnso-unt-ndc%3D300%2Ccspa-uipm-nm-mnp%3D0-007625%2Ccspa-uipm-nm- mxp%3D0-053375...0...1.1.64.psy-ab..2.0.0.Gv5RYBfHmfU)

and

[https://www.reddit.com/r/MachineLearning/search?q=object+det...](https://www.reddit.com/r/MachineLearning/search?q=object+detection&restrict_sr=on&sort=new&t=all)

There's been an absolute deluge of papers, I can't say I've kept up with them all. There was one interesting one in particular was able to learn object detection in an unsupervised manner in a novel improvement upon Bottou's "is object detection for free" paper.

<comment><USER12> To save someone googling, the referenced "Is object detection for free" paper is at [http://leon.bottou.org/publications/pdf/cvpr-2015.pdf](http://leon.bottou.org/publications/pdf/cvpr-2015.pdf)

I also just came across "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", which looks interesting and is at [https://arxiv.org/pdf/1506.01497.pdf](https://arxiv.org/pdf/1506.01497.pdf)

(I'll be reading these after work tonight.)

<comment><USER13> These are both older and not as good as YOLO9000.

<comment><USER11> I know, they are old references.

</comment></comment></comment></comment></comment></comment><comment><USER14> most of the top submissions to the last ImageNet comp. are tweaks on YOLO and Faster RCNN.

But why is this trending now?

<comment><USER15> could be because more people are getting into DL and image recognition? I am one of them so what might be old to some people is new info for beginners

</comment></comment><comment><USER16> For the threshold value, how is 0.25 determined to be the most optimal? is this value just specific to the datasets YOLO was trained on or is it something that is universal.

Also, this is really awesome!

<comment><USER13> Generally you learn a threshold value like that under cross validation.

</comment></comment><comment><USER17> If you look closely at the video, the quality of detection is not really great for anything besides `person`. I wonder if weighing objects by the rate they are being found over N frames, and conforming to a somewhat linear trajectory could improve results; feeding matches from previous frames could also help find the same object in subsequent ones, and maybe even work out temporary occlusion (thinking out loud here - probably way more complex than one would expect).

<comment><USER18> I'm not 100% sure about this but I think part of the reason why its called you only look once is that it is single frame object detection. I do know that there are other types of networks that can previous frames into account, but the models themselves are much larger (in terms of VRAM requirements for loading into memory) and more computationally intensive. This network is special because it can run with very little power. From what I remember it can run on a 10W board at 6fps which compared to networks only a few years ago is 10x decrease

<comment><USER19> It's clearly single-frame; you can watch recognition succeed and fail from frame to frame. The "person" recognizer misses some clear faces, so it's not heavily face-oriented. Since it's single-frame, it's not recognizing articulated motion.

Recognition seems to be limited to "person", "motorbike", "tie", "cell phone" (a gun, actually), "umbrella", "truck" (misidentified part of a train) "bench" (a railing) and "horse" (motorbike with duffel seen from rear). "Person", "umbrella", "tie", and "motorbike" seem to work; the others are kind of random.

The trouble with running recognizers on Hollywood movies is that they have many conventions of what appears on screen and how big it is on screen. Are they training on such data?

Good data sets would be side views from a moving vehicle, like Google StreetView data or just a GoPro pointed sideways while driving around.

</comment></comment></comment><comment><USER20> (a) very cool project

(b) ..interesting CV

</comment><comment><USER21> The tutorial says "Edit src/yolo.c, lines 54 and 55" to train for different classes, but I don't have any yolo.c file at src folder... any ideas?

[[https://github.com/pjreddie/darknet/wiki/YOLO:-Real-Time- Obj...](https://github.com/pjreddie/darknet/wiki/YOLO:-Real-Time-Object- Detection)]

</comment><comment><USER22> This is good background but modern architecture are significantly better:

Learning Transferable Architectures for Scalable Image Recognition

[https://arxiv.org/abs/1707.07012](https://arxiv.org/abs/1707.07012)

<comment><USER23> This is not "modern" though, the paper is quite old. He has improved it significantly since then: [https://arxiv.org/abs/1612.08242](https://arxiv.org/abs/1612.08242). Best paper honorable mention at CVPR2017.

<comment><USER22> I missed that the site was updated for YOLO9000, it is very impressive. The Google paper is recent though (July), maybe you're thinking of [https://arxiv.org/abs/1611.10012](https://arxiv.org/abs/1611.10012)? Anyway, what I meant to point out was that there has been a lot of work since on improving the efficiency of the architectures used for feature extraction (Xception, MobileNet, ShuffleNet, NASNet).

<comment><USER23> Sure, there's new work all the time. But I think you might be missing the forest for the trees here. The main efficiency gain here is not in how you extract and transform features (although those improvements would also work in YOLO, as far as I can tell). The novel part is in structuring the problem as a single forward pass regression. Further improvement in this year's paper is in a better loss function. Other possible efficiency improvements are orthogonal and complementary to that.

<comment><USER22> I agree that the single shot approach in YOLO was pretty clearly a big step forward. In the two years since though there much of the efficiency work has been in the underlying feature detector architecture, which as you point out should integrate well with the YOLO9000 training improvements. It's very cool to see this kind of capability get within range of phone-sized devices.

</comment></comment></comment></comment></comment><comment><USER24> Impressive. Does anyone actually know of real apps that make use of object detection? Apart from Google apps.

<comment><USER25> Not hotdog?

<comment><USER24> haha. Had no clue the guys at Silicon Valley built an actual app for that. Thanks for sharing.

</comment></comment></comment><comment><USER26> Impressive speed.</comment>


Deploy a rich communications app in 3 minutes with Ruby, Heroku, and Tropo - <AUTHOR> http://tropo.heroku.com

<comment><USER2> Compared to Twilio, Tropo has an _awful_ API. Also, a few features Tropo publicizes (esp. to do with IM) just don't work - you end up going to another Voxeo company to get semi-working functionality (IMified), but then some of _that_ IM functionality doesn't work either, and IMified support is non- existant.

Personally, I hope that Twilio goes all-in on the "communications" part of "cloud communications". I'd gladly pay for working and supported IM endpoints with a sane API.

<comment><AUTHOR> I'd love to hear what it is you don't like about Tropo's API.

Also, what IM issues did you have? IMified's tech is what's driving the Tropo IM integration, and I'm one of the IMified founders, so I can probably help you with you with that.

<comment><USER2> :) AIM is screwy on IMified - I've posted on the support site but haven't heard anything yet. Apparently I have an account online, but it's not connected to anything. See the post at: [http://help.imified.com/discussions/problems/326-unable- to-a...](http://help.imified.com/discussions/problems/326-unable-to-add-gtalk- or-aim-accounts)

MSN works on IMified but doesn't work on Tropo - that's just odd, as you yourself said that the IMified system powers Tropo's IM integration (and the UI at Tropo hints at that from a third party perspective).

The bot.im users still aren't appearing on Google Talk, and I can't seem to add an additional account through GTalk directly.

BTW, though I didn't mention this in our Twitter chat earlier, I far prefer the API that IMified offers to that of Tropo.

</comment></comment></comment><comment><USER3> Pretty cool how easy it is to deploy, but I wonder how much work went into the app first. Deploying with Heroku is always simple due to the pure git workflow (ie: git push and you're done).

<comment><AUTHOR> Total time to create the app itself was about 6 hours. We spent more time on the web site and the video than we did the app. :)

<comment><USER3> Wow, that is pretty awesome. Not bad.

</comment></comment></comment><comment><USER4> Who is the design/UI/UX guy at Heroku? His design sense is incredible.

</comment><comment><USER5> Bragger!</comment>


Getting started building desktop apps with HTML5, CSS and JS - <AUTHOR> http://media.bemyapp.com/getting-started-building-desktop-apps-html5-css-js/?utm_source=bma&utm_medium=ycombin&utm_content=&utm_campaign=media <comment><USER2> Yeah, don't do it.

</comment><comment><USER3> And when you're finished and ready to ship (and maybe even sell!) your app, I built [https://keygen.sh](https://keygen.sh) to make licensing your app easier. It's currently in closed beta, but I'm always down to get some additional beta users. Will be coming out with a few nifty tools soon to help with distribution and licensed auto-updates for Electron apps, as well as a Node client library _really soon_. :)

<comment><USER3> Hey everyone, sorry if this seems a little sales-ey – wasn't intending it to be. I built a product that helps with the pain points I mentioned above, so just trying to spread the word and maybe get a few more beta users. Wasn't trying to detract from the article or anything like that. Again, sorry if it was a bit too much – I would remove it if I could.</comment></comment>


Opinion Mining with Deep Recurrent Neural Networks - <AUTHOR> http://www.cs.cornell.edu/~oirsoy/drnt.htm <comment><USER2> Could someone explain the results of this, rather than the code and the theory?

<comment><USER3> This is from the "Conclusions" slide: • Deep recurrent nets perform better than their shallow counterparts of the same size on both DSE and ESE extraction. • Both shallow and deep RNNs capture aspects of subjectivity, but deep RNNs seem to better handle the expression boundaries. • Deep RNNs outperforms previous baselines CRF and semi-CRF without having access to the dependency or constituency trees, opinion lexicons or POS tags, even when (semi)CRF has access to word vectors.

tldr: Deep RNN's are better than shallow RNN's and CRF's for several NLP tasks.

</comment></comment><comment><USER4> OT: Am I the only one practically unable to read the abstract? Grey on white does not work well for my eyes.</comment>


NHS £13bn IT programme to be overhauled - good luck with that - <AUTHOR> http://www.computerworlduk.com/management/government-law/public-sector/news/index.cfm?newsid=21093&utm_source=ycombinator&utm_medium=sb&utm_content=anguyen&utm_campaign=sb

<comment><USER2> I wish these large IT projects would be done in an agile manner. Start in one hospital and one doctor's surgery and work up from there. Of course, that's politically not acceptable because what ministers need is a massive project to announce.

I bet a small group of motivated people could get the work done faster, cheaper and with greater user satisfaction if they built out from a small project to the wider NHS.

I happen to be in the weird position of knowing some of the people who are currently running the UK. I was at Oxford at the same time as Cameron etc. and actually know Michael Gove. I wrote time him just before the election about this very issue. Who knows if they'll listen.

<comment><USER3> More to the point, unlike bottom-up the top-down approach tends to be a solution in search of a problem.

<comment><USER4> And centralized financing and management of large segments of the economy tends to encourage top-down decision making.

I think one of the biggest problems with modern society is that due to the scale of institutions, there are way too many layers of abstraction and aggregation between decision-makers and the decisions they make. This breaks the feedback loop, since decision-makers can only operate against generalities and not actual knowledge, and it encourages a great deal of risk aversion that prevents the exploration and variation necessary to create real solutions.

</comment></comment></comment><comment><USER5> The impression I've got is that the problems with the project come down to a combination of firstly, as @jgrahamc rightly points out, the fact the project has been unleashed all at once across the country rather than one surgery at a time as would be sensible, and secondly (ok, I am risking downvotes here bigtime), I think, poor programmer and management quality - I'm sure there are good programmers on the project, but overall it seems to me the quality must be pretty low. The damage one bad apple can do in software is immense and entirely out of proportion to their position, magnified if that person is higher up in the hierarchy. Additionally it seems obvious that the project has been somewhat mismanaged.

I do understand the project is huge and really very complicated, and naturally a difficult problem probably more so than is immediately obvious, but it is that very fact that makes it easy for poor engineering decisions to have a big impact.

<comment><USER2> I agree with your on the programmer quality. I know people working on these type of projects: not the sharpest knives in the drawer.

<comment><USER6> I bet they are charged out at over £1000 a day though.

<comment><USER7> ranges between £400 to over a grand in my experience depending on what kind of skill sets.

Remember that scaling this kind of business is hard because you are selling peoples time - so for each salary you pay you can only sell it one time over. Compare and contrast with being a product company - I might spend a few weeks working on a product and then those few weeks of my salary can be sold over and over again.

The scaling of so many employees also brings massive overheads.

So yes - the day rates seem high but the margins can be pretty rubbish.

</comment></comment></comment></comment><comment><USER8> Can someone with knowledge or experience of extremely large projects such as this please explain what exactly are the costs? I cannot even begin to understand how you could ever arrive at £13bn, even if you had an army of systems engineers and buildings full of new equipment. What exactly is being paid for, and what is it billing at?

(I'm not looking for an answer like "They're gouging them because they can" - although I'm sure there is some of that going on - I'm actually interested in what are the purported costs, i.e. how could it even be seen to be the correct order of magnitude of expenditure)

<comment><USER9> It's a huge mess of contractors, sub-contractors and consultancies. Enormous companies or consortia bid for these projects, government ministers have no freaking idea about software or technology so hire consultants to consult about hiring these consultancies and get thoroughly shafted in the process.

For example, check out the UK e-Borders system [1]. The spec, according to the article is

 _"Under the e-Borders scheme, immigration and government security systems will be linked with transportation hubs to check and log every passenger travelling in and out of the country."_

On the face of it, it doesn't sound much more complicated than say the Diaspora project, and I'd fully expect a small group of YC-calibre hackers to put together a functioning prototype in a couple of months. It's obviously not _that_ simple, but what do you expect when you hire a consortium consisting of Raytheon, Serco, Accenture, Detica, QinetiQ, Steria and Capgemeni [2]? Christ.

And how do you think the existing security systems that need linking to have been developed? Do you think there's an api?

[1] [http://www.computing.co.uk/computing/news/2202536/borders- wi...](http://www.computing.co.uk/computing/news/2202536/borders-winner-set- sign-3607674)

[2] <http://trustedborders.com/about-us.shtml>

<comment><USER6> There will be no push back on requirements - nobody is going to say "do you really need that" as they will be reducing the scope of the project and therefore the amount of money made.

Implementations of complex systems in the commercial world (e.g. global ERP systems) often work best when the chance is taken to align the processes of the organization at the same time. I can't see that happening in the context of outsourced government projects - nobody has any incentive to reduce the amount of money spent.

<comment><USER7> The current cuts are providing some incentive but I think you are right but there is another part of the picture:

the lack of incentive to cut is one thing.

The ability to see things that _can_ be cut is the other piece of the jigsaw - and I guess this stems from exactly what you are talking about - its not been a necessary skill or a part of the process before.

In order to constrain costs both motive and ability to do so are essential.

</comment></comment></comment></comment><comment><USER10> £13bn? Are these people nuts? That's more than 10% of the total NHS budget.

<comment><USER11> yeah, the benefits afforded by IT aren't worth 10%.

<comment><USER12> Actually, in many use cases & for many organisations 10-20% of budget on IT is a very reasonable ratio. As hackers we may not like it, but that's how the real world functions.

<comment><USER11> I was being sarcastic. Evidently I wasn't clear enough.</comment></comment></comment></comment>


Free Blog, Forum, Website Platform and CMS  based on Node.js (Stored in JSON) - <AUTHOR> http://ourjs.org/

<comment><AUTHOR> It's a beta version. I will be very appreciated if you can help me testing the usability, performance, security, etc.

PS: English is a foreign language for me, if you can point out any grammatical errors, I will be very happy too.

</comment><comment><USER2> you means there's no db? everything get stored in json ?

<comment><AUTHOR> Yes, at first I use mongodb but I found the performance was not affected after I removed mongo, since I caching all the data in memory.

However the article ID is still created by mongodb-native module.

<comment><USER2> so these means for small websites only?

How would u scale it, expecting to cache everything in memory?

<comment><AUTHOR> Sorry the response may be very slow.

Yes. I plan to do this, cache everything.

I have carefully calculate the demand of memory and the cost.

1) As traffic grows, memory consumption does not growing, because of node.js?

I was running another site ourjs.com, sometime I saw 100~200 active users on site at the same time (from Google Analytic), but the memory used of ourjs thread is still less than 100Mb, almost the same when there is no traffic.

2) JSON Data doesn't consumption much memory, I post one article per day on my ourjs, now there are more than 450 articles, I think I can use this framework for 10 years.

3) There maybe memory limitation in V8 but I don't think the blog/website will exceed this limit, maybe I will split into several threads in future if I really run into this issue.

</comment></comment></comment></comment><comment><USER3> Looks good

<comment><AUTHOR> Thank you</comment></comment>


Why QR codes never took the world by storm - <AUTHOR> http://www.codinguser.com/2012/08/why-qr-2d-codes-never-took-the-world-by-storm/

<comment><USER2> Meh. No modern QR decoder operates as described.</comment>


MIT students helped WikiLeaks suspect, hacker says - <AUTHOR> http://www.cnn.com/2010/WORLD/asiapcf/08/01/afghanistan.wikileaks.suspect/index.html?hpt=T1

<comment><USER2> Hmm a tenuous bit of second hand information and it would be nice for the military to have someone outside their ranks to blame.

Is anyone else thinking Adrian Lamo is trolling for attention? Never hurts your career prospects to have your face plastered on the news repeatedly in this way.

<comment><USER3> Lamo is definitely trolling for attention.

</comment></comment><comment><USER4> Am I the only one who thinks Lamo is sort of a jerk in this whole ordeal. I feel he was covering his own butt out of fear, and did nothing patriotic in any way at all.

<comment><USER5> Treason charges are nothing to sneeze at. Someone like Lamo (with his history/record) has a huge incentive to stay squeaky clean during something like this.

<comment><USER6> The best way to stay out of this is by not talking to the media.

No comment is far more effective than any story.

</comment></comment></comment><comment><USER7> He felt threatened by 2 MIT students? Surely Lamo must have knew that bradass87 and his alleged "accomplices" were not mafia heavy hitters.

In all seriousness, this is piss poor reporting.

Like others have pointed out, this is second hand information that makes no substantive link between Manning and the 2 unnamed MIT students. Surely the reporter could have deduced from Lamo's clues who the two MIT students were and followed up for comments or quotes from them.

Even taking this story at face value there are lots of unanswered critical questions.

Was Manning taught to use the "encryption software" with the purpose of enabling him to hide stolen information?

What encryption software was sent?

The meaning of encryption software is left intentionally vague. Was the software something like truecrypt or perhaps the MIT masterminds just sent Manning a link to a firefox installer...

It's a shame that this passes for journalism.

</comment><comment><USER8> Adrian Lamo is a violent psychopath. His ex girlfriend got a restraining order on him because he had attacked her with a stun gun.

<comment><USER9> Any chance of come context to support your accusation and it's relevance to this situation?

<comment><USER10> It is on his wikipedia page, sourced from this 2004 Wired article: <http://www.wired.com/wired/archive/12.04/hacker_pr.html>

Lamo makes a point of refuting the accusation: <http://pax.vox.com/library/post/citability-is-important.html>

I found it interesting he only addressed the restraining order and not the general stalking behaviour or (alleged) stun gun use.

Edit: Not claiming this has any relevance to current events, I was just curious about the grandparent's claim.</comment></comment></comment>


New "Smart Windows" go from transparent to black in seconds, 5000+ cycles OK - <AUTHOR> http://www.sciencemag.org/news/2017/08/watch-window-turn-itself-black-less-minute <comment><AUTHOR> Full paper: "Dynamic Windows with Neutral Color, High Contrast, and Excellent Durability Using Reversible Metal Electrodeposition"

[http://www.cell.com/joule/fulltext/S2542-4351(17)30001-6](http://www.cell.com/joule/fulltext/S2542-4351\(17\)30001-6)</comment>


Ask HN: Is Revenge worth it? - <AUTHOR>

Have you ever been wronged in a relationship, job, etc. in a very unfair way that made you feel powerless? Did the system fail you?
What happened to you? Did you seek revenge?
I'm not really interested in a philosophical debate, more real life stories. <comment><USER2> Think of yourself as a marionette, a puppet controlled by strings. Do you control yourself, or are you letting other people pull your strings? If you react to a wrong, you are letting others pull your strings and control you. The other 'wins' because they are controlling you. Any reaction: yelling, fighting, tweeting, blogging, crying, means that they are winning, controlling you.

Fairness is a wishful concept, a myth that we want to believe in. Companies are not fair, they have to be profitable and feed the megalomania of the CEO. People change, and so do relationships. Shit happens.

Let it go, and move on to Plan B. You control your future, the past cannot be changed.

</comment><comment> noir-york You can either look at the past. Or spend your finite time and energy on working towards your (personal) goal.

Revenge is a dish best served cold. Nothing annoys the f __k out of people then succeeding despite their efforts to trip you up. It makes _them_ feel powerless.

<comment><AUTHOR> > Nothing annoys the fk out of people then succeeding despite their efforts to > trip you up.

I like it. I want to turn the tables.

</comment></comment><comment><USER3> No, it's not worth it. Turn your back and walk away.

I'm old enough that I've been crapped upon a few times by employers, governments, the law etc. It hurts and makes you angry but you never really feel like you "win" if you get revenge.

<comment><AUTHOR> Can you elaborate?

<comment><USER3> No, I really have moved on and don't want to go over those things again.

If it helps, I am in a significantly better situation than those who sought to gain at my expense. Exacting revenge on others only slowed _my_ progress through life.

<comment><AUTHOR> > I am in a significantly better situation than those who sought to gain at my > expense.

That feels therapeutic enough and I can relate.</comment></comment></comment></comment>


North Korea's Digital Underground - <AUTHOR> http://www.theatlantic.com/magazine/print/1969/12/north-korea-8217-s-digital-underground/8414/

<comment><USER2> Just last night I watch "The Vice Guide to North Korea". If you want to see video of NK from the inside I highly recommend it, really creepy.

[http://www.vbs.tv/watch/the-vice-guide-to-travel/vice- guide-...](http://www.vbs.tv/watch/the-vice-guide-to-travel/vice-guide-to- north-korea-1-of-3)

<comment><USER3> And if you ever want to see NK yourself from the inside, check out <http://www.koryogroup.com> \- took a tour with them a couple years ago and was an interesting experience.

<comment><USER4> This is really interesting. How much did it cost? What was the experience like?

<comment><USER5> Once you get to Beijing ($600-1400 from SFO), a 3-6 day trip costs $1-3k depending on EUR/USD, exact itinerary, and when in the year you go (winter trips are cheap). I know a few people who have gone and loved it, although I'd prefer an Antarctic cruise.

</comment></comment></comment></comment><comment><USER6> _Most computers, which generally run on pirated Microsoft software, come from China. The country's only computer-manufacturing company, Morning Panda, produces barely 10,000 a year. If computers are rare, printers are even more so. They are closely monitored because of their potential for spreading anti- regime documents._

I'd love to see a Morning Panda computer in action. Anyone have a picture?

<comment><USER7> A Russian blogger who visited NK last year bought a copy of the NK-produced "Red Star" operating system (in reality, a flavor of Linux), but he doesn't show the hardware: <http://ashen-rus.livejournal.com/4300.html>

</comment></comment><comment><USER8> I was surprised by the last scene in the video. In most countries no police officer on duty would tolerate being verbally and physically assaulted. Is he afraid (the woman being a party official), or is there another reason (e.g. cultural differences - with angry woman hitting man during conversation).

I'd love to see a documentary based on this underground material.

</comment><comment><USER9> North Korea seems to be among the trending, non-tech-related topics on HN. Really interesting stuff.

<comment><USER10> I think China's attitude has changed, so they are letting reports come out through the border.

Or the article could be right - the flood of second-hand electronics has changed some North Korean citizens' attitudes.

</comment></comment><comment><USER11> If you want to put a face to the suffering of north koreans, check out the young man in the middle of the embedded video. Quite unlike anything I've ever seen.

</comment><comment><USER12> those pirated copies of windows are probably so dangerous, who knows what kind of malware comes along for the ride, it would be like passing the same women around to millions of men...

<comment><USER13> Without internet access, does it really matter?

<comment><USER12> I pray you don't work in IT...

<comment><USER13> I get the impression there aren't many corporate IT departments in NK. Not sure what your point is.

In any case, most malware these days is designed to either steal information and phone home or make the infected computer part of a botnet or both. Without an internet connection (whether the PC is in a LAN or not), the practical risk is pretty minimal.

<comment><USER14> NK would still be vulnerable to something similar to Stuxnet. A highly- targeted sneakernet-based attack could be used to very, very slowly migrate information from flash drive to flash drive into and out of NK, or for sabotage.

<comment><USER13> Oh, absolutely. But a targeted attack with that level of sophistication is hardly going to let a few Windows updates get in its way.

</comment></comment></comment></comment></comment></comment><comment><USER15> Is this the North Korean people's chance to do the hump?</comment>


Telegram is the 'hot new thing among jihadists' - <AUTHOR> http://money.cnn.com/2015/11/17/technology/isis-telegram/index.html?iid=hp-toplead-dom <comment><AUTHOR> From the article:

 _" Encryption is one of many ways that an adversary, whether that's a criminal, a terrorist, a rogue nation, one of the many ways that they might use to hide their activities," former NSA Deputy Director Chris Inglis, told CNNMoney._

I recall Paul Graham's Submarine article...[http://paulgraham.com/submarine.html](http://paulgraham.com/submarine.html)

Did the NSA give CNN this story? If so, why? In order to retroactively justify its own attempts to route around encryption? In order to help generate support for some upcoming policy proposal or action?

Also for discussion -- how would you feel if you were the founder of Telegram, and this article is actually true? What would you do? I personally have had some ideas for secure email or chat services. But I really do not like the idea that it would be used by the worst-of-the-worst. I wonder if there is any way to reconcile the two goals of private and secure communication with avoiding being used as a tool of criminals.

<comment><USER2> At one time I was looking into starting a secure messaging business myself. Had I done so and then found myself in this situation, I would probably shut the business down. Ethically, that would probably be the wrong thing to do, since it's implicitly a vote in favor of NSA spying, which I am against. But the thought that I might be facilitating mass murder would be too much. Don't think I coukd sleep at night with that hanging over me.</comment></comment>


Apply HN: Statistics as a Service - <AUTHOR> Basic Concept: Provide simple-to-use, basic, automated statistics to a user based on an uploaded file (e.g. a CSV) to help them spot trends, data anomalies, etc. Deliverables would be either a PDF report or JSON according to user selection  (possibly with actual cleansed data used to generate the results).
This is in the idea stage.
Business Model: Freemium
Pricing tiers: 1. Free 2. Pro (additional, more advanced statistical smörgåsbord* to choose from, suggested stats to run, etc.) 3. API (programmatic access to FREE and PRO tier stats) 4. Custom (could be custom stats implementations w/ API access, more of a consulting project, or even, a statistician on call, etc.)
* Though, initially the options would start relatively small, and be added to as time progresses and it makes sense to do so.
Use Cases: 1. Knowledge worker at a SMB or local government entity with little access to data analytics or statisticians, but still need to make use of statistics. 2. Dev shops needing stats, but don't have the time, in-house resources, etc. to bother with it.
It is not an attempt ... * at big data analytics * at building an AI/ML service * to be a replacement to R, Python, Julia, et al. <comment><USER2> Somewhat related "Design and Implementation of CSV/Excel Upload for SaaS" [http://www.kalzumeus.com/2015/01/28/design-and- implementatio...](http://www.kalzumeus.com/2015/01/28/design-and- implementation-of-csvexcel-upload-for-saas/)

Perhaps you can find some ideas there.

<comment><AUTHOR> Thank you for the pointer. Problems such as CSV/Excel handling are non- trivial, but, thankfully, there are many OSS options out there to help with some of the edge cases.

</comment></comment><comment><USER3> I'm interested in this. Would you have a way for me to pipe you lots of data? Like way bigger than a CSV file upload?

<comment><AUTHOR> Possibly. The API would likely be the venue to handle this. What do you have in mind?

</comment></comment><comment><USER4> I think one of the big problems with statistics for most people, besides the complexity of the math, is understanding when to use which techniques. How will you help the user choose the correct way to analyze their data?

<comment><AUTHOR> Thank you for your reply. I agree with you.

For the Free Tier the idea is provide some basic descriptive statistics with maybe a histogram overlaid with the normal distribution curve on top. Some example values that might be calculated are the mean, median and mode along with several standard deviations. Down the road a bit, I could envision a feature to suggest further exploration, which would likely guide the user towards some of the Pro features. (Although, I would like to expand what is available in the free tier, where it makes sense.)

For the Pro Tier, they would of course, have access to the same results as the Free Tier, but I imagine a wizard like interface (which can be turned off) to help guide the user through choosing what they want done. Initially, the focus would be on helping spot basic trends and more obvious data anomalies. Further down the road, I could see the options expanding.

One of the challenges to be handled is the "width" of the data set. For example, if User A has a CSV with 100 columns, the basic calculations may not be a big deal, but do we present a 100 histograms? What if there are 1,000 columns. My current thinking is for the user to prioritize the columns they want analyzed if they exceed a certain threshold (say, 25 columns). The results would be done on the columns up to this threshold.

I am open to ideas and suggestions on this, and even on things such as which "basic" descriptive statistics are most important.

Edit: Clarification (last sentence)</comment></comment>