Rsync, GUIs, power, control, design, and decisions - <AUTHOR> https://bits.ashleyblewer.com/blog/2019/06/29/rsync-guis-power-control-design-and-decisions/ <comment><USER2> > Is it better to make up common rsync "recipes" for people to use, or is it > better to let folks have access to ALL of the flags [...] and pick'n'mix > what they want?

I think both. This is the hard part about GUI design -- it is pretty easy to make a list of 100 command-line options listed alphabetically on the man page, but for GUIs, people expect much more.

There should be some sort of logical grouping. One area of the screen would be options related to what is transferred, a second area is metadata, the third one is speed-only optimizations, fourth is logging and so on. Having "hardlinks" all the way on the left and "symlinks" all the way on the right does not make a good GUI.

There is a very wide variety of input elements, so checkboxes are are not the right solution all the time! For example, while having separate "\--verbose" and "\--quiet" options is fine for CLI, one would expect to see a drop-down or a slider in the GUI. This will decrease cognitive load because there are less things to read and there is no need to worry what happens if you check both.

Another great GUI feature is dynamicity -- take the metadata for example. A good design might have a dropdown: "everything", "nothing", "x-bit only", "custom". Most of the time, first three options would be fine, but "custom" will show entire new group of checkboxes.

... this was just my subjective design, based on how I use the program. If I were trying to do a popular product, I'd have to spend some time asking people / looking at the forums to determine which sets of features do people use. For all that I know, there is a super common case for "\--group" without "\--owner", and this is just me not knowing about it.

<comment><USER3> I would think there maybe a "command-line" version of GUI design: supply limited default buttons for common options, but allow user customization. For example, I would love to have the button from author's design but tune it specifically to my need including running a specific set of "command lines".

<comment><USER4> Note that even rsync itself has some limited support for "profiles", like --archive.

<comment><USER5> rsync do actually support profile via option aliasing. rsync uses popt(3) to parse its command line options and popt allows you to define aliases, so you can put something like this in ~/.popt:

        rsync alias --sync -rpcLDvz --chmod=D0755,F0644

A call "rsync --sync foo@bar:baz/ baz" will expand the command into "rsync -rpcLDvz --chmod=D0755,F0644 ...".

The "program alias newopt opt" syntax is actually popt's thing (see Option Aliasing[1]) and it works with everything that uses popt(3) for its command line parsing.

[1]: [https://linux.die.net/man/3/popt](https://linux.die.net/man/3/popt)

</comment></comment></comment></comment><comment><USER6> 'People complain about Electron a lot, but the primary complaint just seems to be that its bloated – carrying an entire browser engine along with something small. But like, all of our computers are already bloated, and there's at least a 40% chance that right now I am bloated, and it doesn't mean you should love me any less, and I think the above points I've raised override the lack of svelt-ness of the program. People don't care that an application is a streamlined 3MB or a big round 200MB boy – they care that it works for them and it makes them feel good about using it.'

I think this is true in some cases, but the sorts of folk who run rsync may be more interested in performant hardware that can run on older machines than average. Then again, they'd probably opt for sticking to the command-line tool. Still a very cool project!

<comment><USER7> I avoid electron apps because it is almost anti web.

I would use apps built more like electron if they simply removed the electron part and replaced the app with a name for a namespace and simply shipped tht js/html/images along with a api service.

Installing would be binding tht app to tht name space and using the browser of choose and navigating to namespace.localhost. In this case rsync.localhost.

The system would be a bit more complex for handling auth and a few things but let's not get bogged down in solvable details.

Then I could access my apps from my phone. Another system (if I chose to expose them publicly).

<comment><USER8> This is more or less how Jupyter notebooks work, and I agree this experience is pretty great. Make it a local web server and the browser becomes your UI.

<comment><USER7> Like maybe we can build this out. A common daemon for registering with.

The concept would be very simple to build. I might start a github project.

\- Global daemon to register apps with. Apps could be installed system wide or per use.

Think plopping a zip/tar/ or directory structure in something like /usr/local/webap.d/(namespace) -- might be abetter directory. Then a manifest file that describes some things about the service, such as how to start the API service, and the default entry point (index.html).

\- The global service is responsible for bringing apps on line by way of instance activation on say visiting a name space, or just keep them running.

\- Services once launched pick random port from range to bind to and system service acts as a reverse proxy and common auth for validating the local user. Maybe even ensure each of the API services are run per user and jailed to the home directory

IDK, just thinking off the top of my head, I have written bits and pieces of this for various work projects, but never glued the idea into one single system.

<comment><USER9> It's not that hard, actually, I've done this with Node. Since you can install Node anywhere, include a binary, or whatever, you can have Node run as a server and it can run with elevated privileges, if needed.

You have it scan a directory structure and auto generate a path that you can use in a browser, any browser, to view your newly install app.

You can do this with APIs too so you can have internal tools with APIs via a Node server that do all kinds of things.

Some simple port forwarding rules (ymmv) make this available to your LAN or to the WWW. Add a DNS layer on top and you have something really cool.

I do this for all kinds of stuff, mostly PWAs, but it lets me have my own notes, todo, systems management and more.

<comment><USER10> It is easy, till you get into the trouble of local Firewalls, Double use of port numbers and all those details, where the daemon runs unexpectedly or stops without good information to the user etc

<comment><USER7> In my minds eye we would give one dedicated port to the main service. The apps would pick a random unused port, and report back to the main service to which would then become a reverse proxy from the main services port to the bound port.

So if the rsync app bound to 5392, it would report back to the main service so. And it would ensure all (or some subset of request) to rsync.localhost were reverse proxied back to localhost:5392.

For the most part when I say some request I mean most if not all, but it might be nice to have a system level rsync.localhost/.info uri that would give system level information about the app.

<comment> yjftsjthsd-h If we're on the local system anyways, why not just use an actual socket in the file system?

<comment><USER10> Since requirement was "works with any browser"

<comment> yjftsjthsd-h Yeah, but we're just communicating with the local proxy, right? So it'd be browser > HTTP over TCP > proxy > socket > application.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> > People don't care that an application is a streamlined 3MB or a big round > 200MB boy

At the risk of beating this topic to death here on HN...Some people care; I know I do. I think the audience for rsync would care more than a nontechnical user, for sure. If I downloaded a program with an extremely basic UI that essentially is a string builder, I probably wouldn't bother if the final executable was over, say, 10MB. My preference would be to hack together a basic CLI menu that could do the same thing and I can stick it in a folder that contains a bunch of other scripts.

Am I the target audience, or even close? Probably not, but I don't know many people who want what rsync offers who aren't programmers.

From the linked ServerFault page asking why Windows users don't use rsync:

> I would say that rsync is just too freaking complicated. Anyone I know who > uses it regularly has a pre-set group of flags that generally does what they > want.

This is followed by more reasons Windows users can't handle or don't want a CLI, along with a few mentions of Robocopy on Windows. Which, honestly, is the reason I don't use rsync on Windows. Windows (both Desktop and Server) ships with Robocopy, not rsync. To run rsync, I'm (or rather, was until WSL) forced to use a Cygwin-like environment to run it, which is overkill unless you're already using an environment like that (i.e. not on a Windows Server instance, most likely).

</comment><comment><USER12> If ever there was a tool that didnt need a gui its rsync. Idea is to keep two folder structures in sync. Exactly the sort of thing you want to script, or trigger when one side changes, or cron.

I've use rsync all my computing life but i've rarely wanted to have to manually push a button to make it work.

A good cli interface is clearly the correct interface to rsync.

Embeding rsync in an app that does something else makes sense, eg edit locally auto push changes, but embeding rsync in an app that does nothing but rsync without any automation or integration options makes no sense to me.

You have to learn a new interface that you cant automate. Its dead end interface. The whole point of cli interfaces is that they are infinitly hackable and pluggable.

<comment><USER13> I think the foundation of this comment is incorrect. This helps you build up the rsync command and test it. It gives you the rsync command that it runs. You're supposed to take the command it generates and use another tool to automate it. You're not supposed to automate the interface. It fulfills the conditions you expect of it.

And personally, I loath constructing rsync commands. Every 2 months I have to parse that 2300 line rsync man page. And every 2 months google sees another search query "rsync GUI" Because I am sick of building these things.

<comment><USER14> I appreciate this comment, because at first I didn't understand why I would ever use this program, and now I want to use it. Rereading 'man rsync' for the 10th time is really is annoying.

But it just will never be worth a 200mb download. Gosh. It's like bloat on a whole 'nother level.

<comment><USER15> Interestingly, the useful parts (for you, and likely lots of others) are those that would be readily adapted to a regular web app/site.

</comment></comment></comment></comment><comment><USER16> > Successful UNIX tools (or any tools, for the most part) are the ones with a > simple concept manifested very thoroughly, and rsync is certainly that. > [...] There are over 100 flags in rsync that you can select to do different > things. So, this is what I mean by doing something simple, and doing it very > well and very thoroughly.

Rsync is certainly not simple. It certainly doesn't do one thing. It's "extraordinarily versatile" to the point of madness. Here's what it does:

\- Copies files from A to B

\- Mirrors directories

\- Computes diffs

\- Applies patches

\- Evaluates filters

\- Runs as a daemon

\- Runs over various remote shells

\- Applies filters to filenames

And then it doesn't give you a good progress bar or let you resume interrupted operations, or gather errors at the end. I think this is a good case where the tool may have grown too large and unwieldy, and it may be better to replace it with an API or a collection of small, UNIX-like tools. For example, if I just want to copy files from point A to B, I can do this:

        tar c . | ssh host tar xv

If I want a progress bar, I can use pv,

        tar c . | pv | ssh host tar xv

Usually I find this easier to use than rsync.

<comment><USER17> What is simple in rsync is the idea of synchronising stuff from A to B. That is all it does in all the cases you mentioned.

It is hard to emulate what rsync does with archive mode enabled, which speeds up transfers, so tar isn't exactly a replacement for rsync.

There are better ways to show progress, for example using the flag `--info=progress2`. It doesn't behave well in automated scripts though

<comment><USER18> fyi, combining something like _`--out-format= "file: %f %l"`_ with _`--progress`_ results in reasonably parse-able progress updates.

Keep in mind however that the format of --progress has changed in the past. Depending on your version of rsync it may or may not insert thousand seperating commas into the number of bytes written, e.g. 2,384,908 vs 2384908. The former can be supressed with --no-h, an option not available on at least some older versions of rsync that default to no commas.

</comment></comment></comment><comment><USER19> Nothing beats a good CLI with a proper man page in my book. I like having all the power at my fingertips, and a good way to search for what I want to do without using the internet.

<comment><USER20> I always end up having to google how to do stuff with CLI tools but GUIs let you discover how to do things intuitively. Tried using man pages but they're always slower to find stuff in and a stackoverflow answer is more likely to target the use case I'm going for.

<comment> F-0X > Tried using man pages but they're always slower to find stuff in

Man pages are presented through less, so you can begin a (regex) search with / and hit n to go to the next result or N to go back.

If you knew this, I am actually very surprised you don't find information near-instantly.

<comment><USER20> I'm aware, but this involves guessing the right word/phrase and sifting through multiple matches, and there often aren't quick examples given.

</comment></comment></comment></comment><comment><USER21> Very interested in this GUI for rsync that you wrote (for obvious reasons) ... one thing that jumps out at me right away is that none of your examples show a remote in the form of user@host:path ... I also did a text search of your page for "ssh" and it is not mentioned ...

Does your GUI have the ability to specify a remote in the form of user@host:path ?

This gets a bit complicated - not only do you need to allow password interaction with ssh but further, upon initial connection, you'll be presented with a host key dialog and need to confirm the key fingerprint, etc.

</comment><comment><USER22> > our computers are already bloated, therefore it's okay to make them more > bloated

No. Don't do this. The only reason it's not already a problem is that most apps still _aren 't_ Electron.

</comment><comment><USER17> If anybody is interested, I have written a simple python wrapper for rsync called sysrsync that abstracts the idiosyncrasies of using trailing slash.

It executes or compiles the rsync command if you want to proof it before executing

You can install it with

      pip install sysrsync

[https://pypi.org/project/sysrsync/](https://pypi.org/project/sysrsync/)

<comment><USER23> Yes, the trailing slash is key

</comment></comment><comment><USER24> I really wish there were more GUI wrappers that literally wrote the CLI commands out on screen as the GUI was modified, to help with bridging the GUI/CLI gap for people.

Also, I miss arRsync.

<comment> F-0X The problem with this is that it misses the point of most cli programs, which is that they are to be composed with other programs.

Besides, when colleagues of mine decided to demystify the command line and learn it proper, they're always surprised how much easier it is than they expected.

The GUI/CLI gap, as far as I have seen, is one of willingness to invest some time to learn.

<comment><USER25> > The problem with this is that it misses the point of most cli programs, > which is that they are to be composed with other programs.

That's not true, CLI programs would still exist even if they could not be composed. And I would guess that if you counted how many of our commands include 1, 2, or 3 executables, the 1 column would be the biggest by far.

One of the major headaches of CLI programs is that they're hard to learn interactively because there's no starting point. You have to read the docs. The point of a GUI command builder is that it gives you something interactive to play with to start to learn how the CLI commands work. The point is not to do everything through the builder, the point is to bootstrap people to the point where they no longer need it.

Also, there's no reason a GUI command builder couldn't also show you how to do composition.

</comment></comment></comment><comment><USER26> There is a gui for rsync that's quite popular. Its called Dropbox.

A lot of successful projects seem to be of the form 1) Consider old-school unix tool. 2) Aim for one aspect of its functionality that serves a mainstream need. 3) Put friendly face on it so less technical users can reap the benefits.

</comment><comment><USER27> I recommend also trying Grsync (released in 2005), which is mostly the same thing and is available with multiple languages support. [http://www.opbyte.it/grsync/](http://www.opbyte.it/grsync/)

</comment><comment><USER28> > People don't care that an application is a streamlined 3MB or a big round > 200MB boy

Assuming all else to be equal (UX, functionality, etc), people will absolutely gravitate towards the [5MB storage, 20MB RAM, 0.5% CPU] combination to electron's [300MB storage, 200MB RAM, 5% CPU], especially when they are running multiple such programs.

But, what electron offers is getting from zero to product across all desktop OS-es in arguably the least time possible. So, the real choice is between [PRODUCT_NO_SHIP] to [300MB storage, 200MB RAM, 5% CPU], and we all know what the world chose.

<comment><USER29> This is actually an issue in app development? I thought with SDL, QT, wxWidgets, et al., writing native apps was basically a covered topic...?

<comment><USER28> It is a problem. Here's [1] a talk from fossdem on the various issues, and a couple solutions.

[1]: [https://youtu.be/ANcU1g7ZWdU](https://youtu.be/ANcU1g7ZWdU)

<comment><USER29> It seems to me, that the issue here is mostly web developers who are used to coding for browsers, are slowly moving into the desktop application domain, and instead of them applying the traditional languages used there (C and C++), are circumventing these by creating frameworks that allow them to continue using their web langs in a desktop environment.

If that is the case... they'll never get rid of the megabyte bloats. That's basically just baseline overhead from a browser engine itself.

Crazy idea: Reverse the entire process? Use C for everything, including web tasks, i.e. FastCGI? Correct me if I'm wrong, but with WebAssembly around, is it possible that C programming might make a comeback into web development?

<comment><USER28> I think the central problem with native UI is how much better looking web- based UIs are, and the talk I cited spends time making this point. HTML/CSS/Js built UIs have much better, modern, reactive styling options & when done properly, look much better than, say GTK.

I'm personally a huge proponent of C and would love to have access to WebApis [1] via Webassembly. That would still necessitate a runtime capable of interpreting HTML/Css into views, but projects like webview [2] have demonstrated that this access needn't come at a huge cost. For instance, in [3], for a UI in HTML/CSS with business logic in C, system resources clock in at [RAM: 6MB, CPU: 1.3% Storage: 10MB], which is completely reasonable.

[1]: [https://github.com/WebAssembly/webidl- bindings/blob/master/p...](https://github.com/WebAssembly/webidl- bindings/blob/master/proposals/webidl-bindings/Explainer.md)

[2]: [https://github.com/zserge/webview](https://github.com/zserge/webview)

[3]: [https://fosdem.org/2019/schedule/event/godesktopapps/attachm...](https://fosdem.org/2019/schedule/event/godesktopapps/attachments/slides/2994/export/events/attachments/godesktopapps/slides/2994/slides.pdf#page=19)

<comment><USER29> Wow, thanks, those resources will hopefully keep me busy later... I'm building a cross-platform C engine, and had the usual suspects of "Windows, Linux, maybe Mac", but then recently also "discovered" this ancient FastCGI technology, and just day or two ago skimmed the WebAssembly wiki, and I'd like to explore possibilities of implementing my engine in web applications, too, afterwards.

But if I understand WebView correctly, wouldn't this be HTML in C/C++, i.e. is WebView... "just" a browser? Or... is it "just" spawning a minimal OS-Window on the target system with a system-built-in browser engine? For my intents and purposes, that's kinda the reverse of what I'm aiming for with my C project, which rather would be to bring sprites and raw data formats to your browser window.

In essence, what I meant with "reversing everything with C" is basically programming webpages like C programs. Taken to the extreme, what I mean is for example compiling Windows to run in a browser-context instead of a physical x86-PC.

</comment></comment></comment></comment></comment></comment><comment><USER30> electron gui for rsync when all one needs to do is read a very good man page for rsync. Sigh.

Yes, yes, it's not "intuitive". But maybe, just maybe, learning and/or teaching is just as important, and remembering "rsync -avuz [from] [to]" is really not that hard.

</comment><comment><USER31> There are reasons why it will be difficult to write a GUI for rsync:

1\. The 80% use case for rsync is a carefully crafted pre-set script that runs in a crontab, not as a one-off thing that you select options for and hit "run".

2\. There are so many switches and options that you're not going to be able to model them all in a coherent manner without overwhelming the user.

3\. The price of failure is data loss, which means that people will either be afraid to use it, or foolishly lose data and then be afraid to use it. A dry- run mode helps, but is not a panacea.

What you need is a multipart system:

1\. A set of vetted recipes with good, succinct descriptions of what they do, why they're useful, and where the cliffs are, and a program to apply them for the specific user.

2\. A design GUI that helps guide you through the process of building your own recipes.

3\. A runner for running your recipes from the command line, so you can crontab it (if the recipe system doesn't generate self-contained executables - scripts or whatever - that you can just call directly).

4\. A runner for running or testing your recipes in a GUI environment, for the 5% of times you actually need this.

Trying to mix more than one of these parts into a single program is a UX mistake.

</comment><comment><USER32> Great article.

Check out Unison.

[https://www.cis.upenn.edu/~bcpierce/unison/](https://www.cis.upenn.edu/~bcpierce/unison/)

> Unison is a file-synchronization tool for OSX, Unix, and Windows. It allows > two replicas of a collection of files and directories to be stored on > different hosts (or different disks on the same host), modified separately, > and then brought up to date by propagating the changes in each replica to > the other.

> Unison shares a number of features with tools such as configuration > management packages (CVS, PRCS, Subversion, BitKeeper, etc.), distributed > filesystems (Coda, etc.), uni-directional mirroring utilities (rsync, etc.), > and other synchronizers (Intellisync, Reconcile, etc). However, there are > several points where it differs: ...

<comment><USER33> Unison is terrible in that you must have exactly the same version (down to micro version) of unison on both sides. That makes it very hard to use in practice.

<comment><USER34> While that is true, it was so frigging handy that it was well worth setting this up to get it to work.

</comment></comment></comment><comment><USER35> Thanks for making one, I only recently learned about rsync and was wondering if there was a handy gui for it :), then it magically appears before me on HN.

I'm weary of this "electron" thing though. Do I have to download a 500mb package to run this?

</comment><comment><USER36> I wrote my own rsync gui utility recently as well. Here is a comparison:

[https://www.getlazarus.org/apps/backer/](https://www.getlazarus.org/apps/backer/)

</comment><comment><USER37> It seems like to write a tool that's genuinely helpful for non-experts, you need to become something of an expert in how people use the tool: what are they usually trying to accomplish? What are common mistakes?

It's similar knowledge to what you would need to write a book about rsync.

And the way to do that is with lots of experiments and user testing. A good start would be to find one person who really wants this and make sure it works for them. Designing things by yourself, even with lots of sympathy for the user, doesn't seem like enough?

</comment><comment><USER38> Tcl/Tk is much a better choice of a toolkit for this job.

</comment><comment><USER39> I agree with the other users who have a pre-set of options that work for them. I have used _-avzP_ for 99% of my Rsync uses over a decade and a half (exceptions being adding _-e ssh_ or refactoring for backup solutions). By putting all the options in an 'options menu', keeping the defaults sane, and check boxes off the screen, you end up with a much simpler and easier to use tool.

</comment><comment><USER40> I couldn't get over "add source" and "add destination", set would be a better verb... or just have no verb

</comment><comment><USER41> Slightly off topic, but is it possible to use Rsync with Dropbox (or Box, G Drive, OneDrive, pCloud, Mega, etc.)?

</comment><comment><USER42> Wait, this is a parody, right?

<comment><USER43> I'm not sure. Don't want to be mean, and I'm sure building this was a great learning experience, but it seems more like and excuse to practice Electron than make rsync more usable.</comment></comment>


Ghosts turns 3 years old - <AUTHOR> https://blog.ghost.org/year-3/ <comment><USER2> Wow, is it going to be the usual "we are better than WordPress..bla bla bla" ?</comment>


Programmers: Stop Whining, Be Productive - <AUTHOR> http://www.alexonsager.com/programmers-stop-whining-be-productive

<comment><USER2> I think this completely missed the point of the article, it wasn't that 'makers' were unwilling or unable to work under a managers schedule it was that 'makers' are more productive when they are able to work straight through their problems for long periods of time when they have gotten into a groove.

If say you are in a groove and really starting to understand and solve a problem and then you must take an hour or two hour break to listen to a meeting on productivity you can completely lose focus and have to start thinking anew when you go back to work.

</comment><comment><USER3> The example given to show that the "manager schedule" works isn't even about programming it is about making a wiki page.

I agee that the argument is somewhat BS (I use the excuses sometimes so I can work on something I would prefer to work on), but that doesn't mean that the "markers scheduler" is worse than the "manager schedule"

EDIT: I seem to have missed the point of Alex's article. The comments provide some clarity</comment>


The Airbnb for America's Extra Crap Is Here - <AUTHOR> https://www.citylab.com/life/2019/07/storage-unit-for-rent-shed-neighbor-app-side-hustle-airbnb/592630/ <comment><USER2> How about not buying so much crap in the first place? Make an app that uses AI to predict how many times you'll use the item you're considering purchase of. We all have apparel, tools, gadgets, exercise equipment etc that we purchased without adequate consideration. With my proposed app, problem solved!</comment>


Ask HN: Is asking obscure algorithms any test of programming ability? - <AUTHOR> A friend of mine is being interviewed for Big 4. In the first round of screening, (s)he was asked about a linear time algorithm to find all the maximum element in a sliding window in an array.<p>Now this is a simple problem for anyone who has taken any DS or Algo Undergraduate level class since it involves the important concept of the priority queue. However, it is my understanding that to do this in a linear time is neither trivial nor straightforward and unless that person is really gifted. (Its something called Monotone Priority Queue which is not present in any standard Algorithm textbook) So my question to the general populace is: Are knowing obscure algorithms like this any test of a person's programming ability ? <comment><USER2> Here's what happened to these companies:

\- They started asking algos and datastructs in the 00s because not too many people had the resources to study it. This served as a proxy for intelligence and was vaguely related as compared to other IQ tests or puzzles

\- In this decade, more and more CS graduates and bootcampers started studying the same algo and datastruct problems

\- Unable to reject anyone (because everyone could solve those problems), they started increasing the difficulty of the problems and started fretting over stupid things like variable names, arcane data structures and solutions, culture fit, ability to handle pressure etc.

\- Today, it has devolved into a nerd show off event where the entire goal of interview has become diluted.

The goal of the interview was to open a requisition, find a smart candidate to do the job, close the requisition.

Nowadays, they'll open a req/have a pipeline of candidate, grill them over unnecessary questions and keep interviewing candidates until they get bored/really need to close this req.

The whole point of the interview process is so lost. As a person who is employed in one of these companies, I hate to see what it has become. We regularly reject candidates who are clearly more passionate than their interviewers, have better experience and bring something new to the table.

But because these mediocre interviewers (my peers) interview candidates on some arcane crap (that they obviously can't solve in 30 mins) these smart guys get rejected and I'm relegated to working with these retards.

It is one thing to ask coding questions to filter out the complete losers. It is completely another thing to hire only people who can solve these stupid questions.

Sad!

<comment><USER3> I think you've missed an important step in how these systems are used at these companies and how their funnels work. For a comapny like Google, their funnel is so tight by the time a candidate is onsight it's known that they're capable.

Instead, the "nerd show off event" exists for one reason "see how much you want to work at that company". Since the realities of software development means the bulk of the engineers will probably be working on some small part of an unglamourous project, yet the "big 4" still want a monopoly on the best and the brightest, the interview process exists as a very targeted test to find people willing to jump through hoops. At this point the broad strokes of what is required is a solved problem, you just need to spend several weeks/months studying up on errata. The thinking is, any candidate willing to do that will be willing to spend several years working on CRUD interfaces for adsense or refactoring old php.

<comment><USER2> You are right.

In other words, they want people who'll work for them without complaining. Submissive people. People who won't think out of the box.

<comment><USER4> That's a scary thought to me. One that's probably correct, but scary, especially at companies who claim to champion "outside the box" thinking. It's almost like a sort of "Do as I say, not as I do."

<comment><USER3> I think it's just the reality of how software is made. There's room of innovative thinking in small startups, or at the highly technical specialized teams in mega corpX, but at the end of the day the grunt work has to be done by someone and that's going to fall to the line member.

</comment></comment></comment></comment></comment><comment><USER5> I think these questions usually focus on the wrong part of the equation. Is knowledge of obscure algorithms a good indicator of programming ability? Maybe, maybe not. Is the interviewer expecting you to know the Monotone Priority Queue algorithm, or are they evaluating your ability to approach and solve a problem? That's the question that matters, and there's no information about that here.

"Given an array, and a sliding window within that array, how would you go about finding the max" is a legitimate question with plausible real-world application. The fact that there's an obscure algorithm to optimally solve this doesn't mean it's not a good test of someone's ability to think through an algorithmic problem - as long as you don't make it an algorithm memorization test.

<comment><USER6> > The fact that there's an obscure algorithm to optimally solve this doesn't > mean it's not a good test of someone's ability to think through an > algorithmic problem - as long as you don't make it an algorithm memorization > test.

From my read of the description, the OP implies that the candidate provided (or at least, if OP were the candidate s/he would have provided) a O(n log n) solution using PQs. The task was to specifically to make the algorithm O(n), which required an obscure data structure discovered in the late 90s. So I think this is more of an algorithm memorization test.

There are very few practical problems where an O(n log n) solution is completely unacceptable, and an O(n) is essential. And in those situations, many engineers can do the relevant literature study and come up the MPQ solution.

</comment></comment><comment><USER6> > Are knowing obscure algorithms like this any test of a person's programming > ability ?

No, it is not. Knowing what a priority queue is is a reasonable expectation; knowing the monotone variant is not.

Unfortunately, far too many empathy-deaf programers get to interview candidates. They think that maintaining the company's reputation for having a hard interview process will work in their favor - no "undesirable" candidate get through under MY watch! - unaware of the long-term ill effects such stupid questions have on their profession.

"Hire programers smarter than you" seem to be interpreted as "ask more obscure things than what you were asked". :(

</comment><comment><USER7> It's a tough reality, but yes. Memorizing trivia is a useful skill to get past first-tier tech interviews at large firms.

As a developer, you have a few options. First, you can recognize this reality and simply spend the necessary time cramming useless algorithms for a few weeks before you go off interviewing.

Or you can declare all this stupidity as beneath you, go in to your Google interview cold, and get rejected based on trivia even though you're genuinely smart and good at what you do.

Or, you can spend your first several years building impressive artifacts and creating relationships that you can leverage when the time comes to bypass the entire Tech Interview/Resume Filter/HR step of the hiring process and skip straight to the part where you're having coffee with the guy with hiring authority.

I've personally landed every gig I've had in the last 15 years using that last trick. Which is a good thing, since despite having made a lot of software companies a lot of money, I can't recite so much as Bubble Sort on demand.

</comment><comment><USER8> Normally, you would interview someone on problems that you want to solve. You would want to hire someone that will help with business challenges by providing specific programming expertise. Requiters do keywords search for a reason.

Problem with Big 4 is that these problems are on such scale that you cannot possibly expect anyone knowing how to solve them. In the same sense your experience is irrelevant because these companies have internal tooling and frameworks that are mostly specific to problem domain.

Algorithms questions test two things: 1\. How you think under pressure. (culture fit) 2\. Dedication, because you need to waste weeks on preparations (corporate slave).

</comment><comment><USER9> A Monotone Priority Queue is neither sufficient nor needed to solve this problem (as it does not give a linear time solution). Instead the proper solution just needs a deque, see [http://techieme.in/maximum-element-sliding- window/](http://techieme.in/maximum-element-sliding-window/).

So yes as you can see knowing algorithms is important, as if you don't you will attempt to use an overly complicated data structure to come up with a suboptimal solution.

<comment><USER10> The solution using double ended queue is often referred to as monotonic queue, because queue content will be monotonic sequence.

<comment><USER11> The OP referred to a "Monotone Priority Queue" which seems to be something different: [https://en.wikipedia.org/wiki/Monotone_priority_queue](https://en.wikipedia.org/wiki/Monotone_priority_queue).

</comment></comment></comment><comment><USER12> The test isn't about whether you _know_ the algorithm – quite the opposite. The test is about seeing how you approach a problem you haven't (possibly) solved before.

<comment><AUTHOR> That's an interesting observation. So what you are saying is that if I make sufficient headway into the problem without providing the most efficient answer, it should be fine?

There are rumors that unless you are spot on perfect in Big 4 interviews, they wont advance you.

<comment><USER13> It all depends on the interviewer. Anyone claiming different is selling you bullshit. The purpose of the interview and how they are actually used in practice is all on the interviewer.

</comment></comment></comment><comment><USER14> Monotone Priority Queues were formulated _after_ I took an algo class. I've encountered them because I had exceptional need to understand priority queues, but have never needed to implement, describe, or recognize them.

If someone thinks that is a good way to judge hires they are wrong.

</comment><comment><USER15> I was recently asked to calculate the fibonacci sequence value in an O(log n) time complexity. I had no clue how to do it and explained that I can solve it in O(N) easily but don't even know how to approach the O(log n) method. I knew there must be some trick using factors but didn't even know how to start the problem. I looked it up later and one of the ways is using matrix multiplication (an implementation I still wouldn't be able to do without seeing pseudocode).

<comment><USER16> I wonder what the interviewer would have said if you'd written down Binet's formula :)

[http://mathworld.wolfram.com/BinetsFibonacciNumberFormula.ht...](http://mathworld.wolfram.com/BinetsFibonacciNumberFormula.html)

<comment><USER15> Maybe his response would have been "Thank you for memorizing obscure formulas. Unfortunately, we look for motivated employees who study relevant topics pertinent to work."

Like really I don't know what to expect. I wonder what % of programmers can actually solve the problem without studying the solution (I'm sure there are many out there but I would guess most are from a mathematical field/background).

</comment></comment></comment><comment><USER17> No it's stupid. Like asking an iOS engineer a question involving a tree data structure. It is stupid and irrelevant because at the end of the day our job is to build products/features. If you think at a student level you will never learn anything past college. Companies think that by default everyone can write code and build product. So they focus on stupid obscure stuff.

I believe it's the opposite. We all went to college and got the same freakin CS degree. We don't know anything about building products when we graduate. Building a product from the ground up without the help of a designer and PM is a tough exercise. Most engineers would fail. Guess what, that's what we do all day in the real world. 10 years later I still get algorithm questions as if my job is %100 about algorithm. That's %5 of my Master's program by the way.

<comment><USER18> Will an iOS engineer never consider a binary search tree to store sorted data?

<comment><USER17> An iOS engineer would look at things like CoreData or Sqlite :) certainly not a binary search tree.

</comment></comment></comment><comment><USER19> Yes, absolutely! A person who knows the answer to that question is definitely a strong developer. It does not mean someone who does not know the answer isn't as strong, but my bet would be on the one who knows it.

If I was asked the question I would probably use a set and mark the indices of the maximum elements. I think that should be O(n) but there is probably methods that are much faster.

I also really don't agree that it is the "thought process" that matters. What that means is that you should speak what your mind is thinking while you solve the problem and the interviewer should rate you based on this speech. Meaning those who like to think quietly are unfairly penalized.

Like who would really want to talk to the examiner while solving tough linear algebra problems? Math doesn't work that way and neither does programming.

</comment><comment><USER20> As you probably expected from your phrasing/question, no.

Ultimately there's often some frustration from both sides when hiring. Interviewers get a ton of under-qualified candidates and need a way to weed them out. A semi-technical conversation isn't always enough (lots of practically useless programmers can repeat dogma). Stumping someone _is_ a good way to assess their technical capabilities (you get a nice mix of analytical thinking and creativity to try to come up with a working solution even if it's not perfect).

I try to tie my technical questions to previous projects/areas of interest, with increasing difficulty/complexity, but sometimes this is difficult (particularly if the potential hire doesn't have any particular specialisations). If you start too low, the interviews go too long and you don't get a good feel for what the person can actually do. If you start too high, it comes across as demanding knowledge of obscure algorithms. I know that I've unintentionally asked questions that have come across this way, but in those cases you can't really expect a full and correct answer -- all you can expect is an appreciation of the components and some thought on how the person approaches problems.

That said, if you're being hired for your specific pre-existing knowledge, that's a different matter.

</comment><comment><USER21> Algorithms knowledge demonstrates competency in the small, but isn't good for demonstrating skill at programming in the large. The person passing such tests will be effective in developing _programs_ , but not _systems_ , and may not be effective in self-directed design of programs that exist within systems.

But the more obscure stuff doesn't demonstrate much, really, except that they know obscure stuff or are good at researching to find solutions. A more effective question might be: develop or use an algorithm (no constraint on performance like here) to solve problem X. Then follow it up with discussions on how their solution performs and see if it can be improved, and how well they handle being directed to a better--more efficient--solution if they didn't pick an optimal one already.

Of course, even that has problems (with the second part having an implicit bit of "culture fit" to it, and depending on the interviewer being a good critiquer and not merely a nitpicker). And it still only reveals the ability to program-in-the-small. So would only be useful for certain roles, or perhaps more widely for entry level employees.

</comment><comment><USER22> This seems to be a rather easy, no ? All you need to do is to keep the top-2 elements in every window (okay I guess that part can be done with queue), and update it as you move it to the right. This is trivially linear time since you only look at elements once.

Nah this is cool.

What you need to watch out for is obscure badly worded nonsense. I once had a interview where I was asked to "design a protocol". The guys asking the question were so vague that it essentially turned into a game where I was constantly throwing predicates at them to find out what they meant. In the end it turned out they wanted me to use a predefined alphabet S without an end symbol, and lift it to one with an end symbol. Obviously, the cartesian product of S does the trick. All I got out of the interviewers though was some verbose non-sense about the grand design of computing. What a waste of time that was (and what a relief to not get that job!).

</comment><comment><USER23> No, with the sole exception of interviewing for a research position where new algorithm variants on the sliding window might be developed.

What an interviewer focusing on specifics can pick up on is near term familiarity with a topic. But someone who can deliver value is also able to propel knowledge into novel areas, and can explain and debug the solution they came up with. They aren't there to be a student passing an exam.

Familiarity tests are often "weeder" filters that clear out candidates with no knowledge of the domain problem. This is a mixed blessing as it can find senior people if you drill deep enough with the right problems, but it can also make the overall skillset and wisdom of the team brittle if they are overfit.

</comment><comment><USER24> I think it depends on the interviewer intentions. If the intention is to judge whether the candidate knows such and such algorithm (aka they have used or read about it) then it is obviously stupid. On the other hand it could be about how you approach the problem, how you start with a simplest possible solution and then engage in a thought process (along with conversation about what you are thinking) to improve the solution. In this case the final solution doesn't matter, what matters is your thought process.

</comment><comment><USER25> If you believe some reports, it doesn't matter what is covered in an interview; it's no better than a lottery regardless.

<comment><USER25> Also, any test that you ace has not completely characterized your actual skill level; all it tells you is that you're above the skill range measured by the test. You might be a little bit above, or far above, that upper bound. Maybe they've decided to have one really hard question that it's unlikely anyone will answer correctly, but if they get someone who can answer it then they know they've found someone really suited for the job.

Or it could be a mistake, and it was just bad luck that your friend got an interviewer who doesn't know how to interview, and asked a question that about a subject that they're intimately familiar with, but is too hard compared to what is asked by other interviewers.

<comment><AUTHOR> One may argue that a Phd student working in genomics must know the Monotone Queue variant because it MAY come up as a research implementation in his/her work. But this doesn't hold true for general software engineering folks.

I am a Phd student in Computer Graphics and so it is a reasonable expectation that I should know more about linear algebra than my peers. But if you extrapolate it to ask extremely hard research level problem it kills the purpose.

</comment></comment></comment><comment><USER26> I recently did a test with a huge ( 200M MAU ) consumer company for a developer role ( back-end ), and the first test was extremely challenging. It wasn't algorithms per se, but maths. To solve this, you had to understand counting ( N choose K ) but to a level such that the way to solve these particular problems really only emerged in a paper a decade or so ago. So you had to go look up a paper, and implement. None of this was specified, I just had to figure it out. Another question also involved looking up a paper ( tho it probably could have been done another way as well ), and the easiest question was a relatively basic programming question. It was hard to see a rationale that linked the questions together, and linked the test to the role, but at least you could be sure that if people got through they were pretty clever and capable.</comment>


Swift 2 Apps in the App Store - <AUTHOR> https://developer.apple.com/swift/blog/?id=32 <comment><USER2> I ran the "Convert > To Latest Swift Syntax" on an app with about six view controllers, four views, a network service, and three cocoapods. It worked surprisingly well, most of the changes were really just suggestions ("this var something could be a let something"), small change to how NSData unwraps.

I did have to guess around an obscure compile issue (solution: "Clean Build Folder"), but after that it was clear sailing.

My overall Swift experience has been terrific. I'm not the first to say this, but to anyone who's wondering if it's "ready", yeah, it's ready.

<comment><USER3> +1 for mentioning the "Clean Build Folder" issue. I was stuck on that for about an hour yesterday, then I found this post:

[http://stackoverflow.com/questions/32586144/upgraded-to- swif...](http://stackoverflow.com/questions/32586144/upgraded-to-swift-2-and- cocoapods-38-2-now-getting-build-error-command-bin-s)

I didn't know about using the Alt key to reveal "Clean Build Folder", and I've been doing iOS dev for years. I'll add this to the list of unhelpful compile errors that are fixed by cleaning the build.

<comment><USER4> "Clean Build Folder" is basically the "turn it off, then turn it back on" of Xcode. Almost anytime you get a weird compiler error on a project that compiled fine moments ago, it's solved by "Clean Build Folder."

Definitely a good trick to know.

<comment><USER5> Or for those who prefer automating, add this to ~/.bash_profile:

       alias nuke="rm -rf ~/Library/Developer/Xcode/DerivedData/"

<comment><USER6> I always advice people to put their DerivedData and build folder somewhere accessible. You can change it settings, don't use the default setting.

<comment><USER7> Good tip. As someone with quite a small main HDD I'll do this ASAP.

</comment></comment></comment></comment></comment></comment><comment><USER8> If your Swift app is using SpriteKit it might take a little longer to get it ready for the App Store: [https://forums.developer.apple.com/thread/17463](https://forums.developer.apple.com/thread/17463)

Apple really screwed up SpriteKit badly in the new version. I'm experiencing almost all of the problems mentioned in the forum.

<comment><USER9> I scanned the post and it seemed the issues are regardless of the language.

<comment><USER8> That's true. I didn't want to badmouth swift, I quite enjoy developing with it. I guess I just needed to vent my anger about SpriteKit a little.

</comment></comment></comment><comment><USER10> I maintain a database of Swift resources: [http://www.h4labs.com/dev/ios/swift.html](http://www.h4labs.com/dev/ios/swift.html)

It appears that most iOS devs who blog are using Swift.

</comment><comment><USER11> Here's my prediction : apple is currently working on a replacement framework for cocoa (at least uikit), that will be writen for swift first, taking advantage of its great special features.

My other guess is that it will come in time for ios 10 and be cross-platform ( aka work on mac os as well as ios).

I have absolutely zero inside info, but the fact is cocoa felts great and elegant on ios3 , but with all the latest additions it now starts to feel old and convoluted. Objective-c has been patched incredibly well but can only go so far, and the convergence of hardware between devices makes the distinction between computers and tablets more and more obsolete.

<comment><USER12> Plus, OSX needs a UIKit-style revival, especially with all the recent grumbling about the Mac App Store. Having just written an OSX app after doing UIKit for several years, it feels incredibly old and clunky in comparison. Even simple layer transform animations are for the most part unsupported!

</comment></comment><comment><USER13> Ok cool, but where's the source code? It's been almost three months since the announcement and no ones even mentioning it anymore.

<comment><USER14> This was ask on Reddit recently, and Chris Lattner answered:

[https://www.reddit.com/r/swift/comments/3kbjjn/so_swift_isnt...](https://www.reddit.com/r/swift/comments/3kbjjn/so_swift_isnt_open_source_yet_then/)

We are still on track to open source Swift (including Linux support) "by the end of 2015" as promised, more details will come out when they can.

-Chris

</comment></comment><comment><USER15> Swift 2 is fantastic _except_ (har har) for exceptions. Optional<T> has been hanging out, monad-y and all, since v1. Why not Result<T, E>? You can't flatMap exceptions, and the Swift 2 implementation even throws away type data (or hides it, at least). Huh?

<comment><USER16> I guess you mean the new do-try-catch syntax? IMO it's quite lovely, and strikes exactly the right balance between the checked exception tyranny of Java, and the strap-in-and-good-luck wild west of C++.

My favorite feature of Swift's exceptions is the 'try' syntax required at the call site. This isn't like Java's try syntax: every call that may raise an exception needs a try, even if all you want to do is rethrow the exception. But the syntax is lightweight: just preface the call with 'try', it doesn't make a new scope. This makes the abnormal control flow explicit, addressing one of the strongest critiques of Java-style exceptions.

My second favorite features is the 'try!' syntax. This is the escape hatch: Swift's version of the empty-catch-block pattern of Java, except that it's 1. much more lightweight, because it doesn't introduce a new scope, and 2. the default behavior is to abort instead of swallow. It's perfect for non- production code like tests.

Regarding "you can't flatMap exceptions," I would respond that exceptions are meant to be exceptional, and this is what justifies the abnormal control flow. The do-try-catch syntax is an adequate substitute for flatMap, and more lightweight than requiring explicit unpacking of every return value ala Go.

But as you say, the hiding of type data is definitely weird. For those following along at home, Swift requires explicitly annotating functions that may throw errors, but not what those errors are, so you have to refer to the documentation. Also the compiler will complain if you don't have a backstop exception handler. Lastly, it introduces a magic variable 'error' that is defined within catch blocks: convenient but it feels ad-hoc.

<comment><USER15> Well, for example, try/catch don't work asynchronously, so you can't pass it to a callback (or use it in a SignalProducer in RAC). If failure/success is Just Another Value, it works everywhere that values work. That they already use an ADT for nullability it what confuses me - ".None, but with an error" isn't a far off concept from that.

antitypical/Result already provides Result.init(() throws -> T) -> Result<T, E>, so it's easy enough to work around, at least.

The magic variable stuff exists in didSet as well, weird and sketchy to use. Would prefer it to be explicit in all cases.</comment></comment></comment>


Show HN: Simple color comparison tool - <AUTHOR> http://amitzur.github.io/amitzur.ninja/color <comment><USER2> Autocomplete/suggestions for standard names might be a useful feature for many users.

Good luck.

<comment><AUTHOR> thanks!</comment></comment>


C++11 will be introduced by B.Stroustrup at Microsoft Event - <AUTHOR> http://channel9.msdn.com/Events/GoingNative/GoingNative-2012

<comment><USER2> I appreciate Microsoft's renewed commitment to native code but I'm rather disappointed at the limited C++11 support in the upcoming Visual Studio 2012 [http://blogs.msdn.com/b/vcblog/archive/2011/09/12/10209291.a...](http://blogs.msdn.com/b/vcblog/archive/2011/09/12/10209291.aspx) especially when compared with gcc and clang.

gcc: <http://gcc.gnu.org/projects/cxx0x.html>

clang: <http://clang.llvm.org/cxx_status.html>

To pick a few examples, range-based for, variadic templates, and non-static data member initializers are huge improvements to the language.

<comment><USER3> > I'm rather disappointed at the limited C++11 support in the upcoming Visual > Studio 2012

As a C user who's been waiting 12 years for MSVC to implement C99, I had to sardonically laugh when I read this. They don't even let you declare variables in the middle of a block!

Ironically the excuse they always give is that their customers want them to dedicate more of their time to C++11, not C: [http://connect.microsoft.com/VisualStudio/feedback/details/3...](http://connect.microsoft.com/VisualStudio/feedback/details/345360/visual- c-should-support-c99)

<comment><USER2> Honestly, that's probably right. I have no data, but I'd predict the number of C++11 programmers on Windows to dwarf C99 programmers.

<comment><USER3> I wasn't disputing that, I was just pointing out that OP was complaining about lack of C++11 support in Visual Studio compared to gcc and clang, but this is even more unfortunate considering the Visual Studio team has indicated that they prioritize C++11 support above other things.

</comment></comment></comment></comment><comment><USER4> Isn't this a little weird, given that MSFT wants people to use "managed code", the CLI, and all the attendant software?

Or is it telling us that people who get to choose (free software) aren't going to be using C++11?

I can't tell.

<comment><USER5> Windows 8 WinRT is unmanaged/native; this has caused a conspiracy theory blaming infighting between Microsoft teams:

Why is WinRT unmanaged? [http://stackoverflow.com/questions/7457371/why-is- winrt-unma...](http://stackoverflow.com/questions/7457371/why-is-winrt- unmanaged/7524710#7524710)

<comment> iso8859-1 Which of those answers do you prefer? Accepted one seems a bit far fetched and simplistic. As Tatiana notes: "SharePoint is .NET, and it's half of Office.". Also see Larry's comment.

<comment><USER5> I'll warn in advance that this is the jazz hands answer, because it's really impossible to tell. I doubt any one person decreed WinRT should be native; most likely it was the best of many compromises. Anyone willing to name names while documenting the internal politics that went along with the decision(s) could write quite the page-turner!

* I could see decisions being influenced by long-term career prospects, but believe that people would be smart enough to hide most of this by working harder to find evidence supporting the direction they needed the tech to go.

* According to my limited understanding, Microsoft long ago threw out a bunch of OS-level managed development pre-Vista (WinFS-type stuff) in part because of performance related issues.

* ASP.NET (SharePoint) / managed code server-side is not going away any time soon, if ever... it's just too easy (and server hardware can typically handle any overhead).

* .NET client-side could go either way; I think this is what the accepted answer is saying (eg. Silverlight as this decade's VB6 - convenient LOB tool killed with no direct transition to new tech)

* Mono/Unity 3d/etc. are wildcards here, particularly as they support C# for non-Microsoft platforms (especially iPhone/Android).

* Microsoft, or at least powerful people within Microsoft, love COM (one of my least favorites, but I don't deal with cross-language runtime/binary compatibility).

I found this StackOverflow link on a huge discussion of the death of Silverlight, saying it was the first casualty of a "death to managed code" campaign: <http://forums.silverlight.net/post/612643.aspx>

</comment></comment></comment></comment><comment><USER6> > _"C++11 feels like a new language" says C++ creator Bjarne Stroustrup_

Great. Finally a reason enough to go back to C :-)</comment>


Google Hacked? - <AUTHOR> http://www.googlewatchdog.info/2007/09/spam-and-virus-sites-infesting-google.html It seems the more popular the site, the more people are desperately trying to game the system. Has it reached the point where any startup looking to burn through a lot of hits had better think a bit about how to handle the attacks? You obviously can't spend a lot of time defending a site that isn't even popular yet, but you can't get caught blindsided either, right? I wonder where the balance is. <comment><USER2> The headline is a bit of a stretch; the author in the article is merely _speculating_ that their database has been hacked. I'd be pretty shocked if that were actually the case.

Instead, it looks like someone has found a flaw in the current ranking algorithm and is using that to their advantage.

<comment><AUTHOR> Apologies for using the word "hacked" -- I meant the system was hacked, not necessarily the database. Durn title was long enough already.

I wasn't trying to do a Digg, honest. My point was that as sites grow in popularity, there's this huge arms race that goes on. In addition, as the internet goes everywhere, you can pay a thousand people a buck a day to do the hacking for you. It's a huge problem, and some thought has to be given to it even as a startup, imo.

</comment></comment><comment><USER3> Lately my parents have been complaining about certain search results on Google linking to completely different pages than what they clicked on (like car part search results linking to travel websites). They are using a combination of Windows, IE, and AOL so who knows what the hell is going on. I've gone over it with them step-by-step on the phone and there is definitely something there. It must be some program running locally because the same search result on my mac goes to the right page.

To deal with the problem... my dad reinstalled Windows.

<comment><USER4> I think this is a nameserver problem. My girlfriend's parents' computer did this, too, and there was an entry in the registry which I needed to change in order to fix it. Obviously, the problem originated from some type of malware changing the registry, but after that, it doesn't require a program to continue running. What happens is that the request for a website is sent to the malicious nameserver, which returns its own malicious IP. The address is then scanned for certain information (e.g. Google search information), and if it finds anything relevant, it redirects the user to their own ad-riddled site; otherwise, it redirects them to the originally-intended site. Obviously, this is a security issue in that all information you are transmitting is going through a malicious site. I would recommend trying to do something about this problem ASAP, even if it involves re-installing the OS.

</comment></comment><comment><USER5> The author makes a claim but doesn't cite a single example of this. This is BS until I see proof.

<comment><USER6> I've seen it. Unfortunately, I can't remember the query phrase but I remember being confused about the pages and pages of .cn sites with no other results. It didn't reoccur on subsequent searches so I forgot about it.

</comment></comment><comment><USER7> It's incredibly frustrating to see how search quality is an arms race just like email spam. First there was keyword stuffing, then link trading, keyword stuffing on many other sites, blog spam, made for AdSense pages and now we don't even understand what is going on.

<comment><AUTHOR> There's a lot of ways you can trick or "hack" Google. I can think of 2 or 3 right now that might work, and I'm sure there are others.

This looks like something that involves a lot of resources. We may be seeing the beginning of a sort of economic Web war, where one side spends tens of millions to break into and use a trusted system and another side spends tens of millions to prevent it. We've seen a lot of little petty stuff with botnets and DOS attacks over the years, but it could be that the game is scaling up to a even higher level.</comment></comment>


Stanford Ph.D in statistics wins lottery four times - <AUTHOR> http://www.dailymail.co.uk/news/article-2023514/Joan-R-Ginther-won-lottery-4-times-Stanford-University-statistics-PhD.html

<comment><USER2> <https://news.ycombinator.com/item?id=4307662>

<https://news.ycombinator.com/item?id=2868747>

<https://news.ycombinator.com/item?id=2861390>

There are more - if you search for "lottery" in the title you find this story many, many times, and lots of others.

Added in edit: Actually, here's a list from a previous comment:

Related - breaking the Massachusetts State Lottery:

<http://news.ycombinator.com/item?id=2828122> <\- lots of comments

<http://news.ycombinator.com/item?id=2829953>

<http://news.ycombinator.com/item?id=2834002>

<http://news.ycombinator.com/item?id=2834122>

<http://news.ycombinator.com/item?id=2839674>

<http://news.ycombinator.com/item?id=2842018>

</comment><comment>==

Also related, breaking the scratch card lottery:

<http://news.ycombinator.com/item?id=2166555> <\- This has lots of comments

<http://news.ycombinator.com/item?id=2166829>

<http://news.ycombinator.com/item?id=2174333>

<http://news.ycombinator.com/item?id=2181729>

<http://news.ycombinator.com/item?id=2186178>

<http://news.ycombinator.com/item?id=2188198>

<http://news.ycombinator.com/item?id=2202232>

<http://news.ycombinator.com/item?id=2241306></comment>


GameTech lecture: Designing and Evaluating Reusable Components - <AUTHOR> http://mollyrocket.com/873

<comment><AUTHOR> For the quick summary, go to 42:00 on the audio, and slide 43.

</comment><comment><USER2> Awesome! A good insight into the complexities of writing reusable libraries. While he talks about game development, the problem seen in GD is much more intense than things seen in application development. However the same concepts apply and this definitely helps with the problems I've been having at work with API designs.

If you plan on writing a framework of any sort, listen to this :)</comment>


How Silicon Valley fell in love with sourdough - <AUTHOR> https://www.eater.com/2018/11/19/18099127/bread-silicon-valley-sourdough-tech-bros-tartine-chad-robertson <comment><USER2> I cannot stop laughing. Every now and then Silicon Valley discovers something (outside of tech) that we have been knowing for centuries.</comment>


Book Review: 'The Dart Programming Language' by Gilad Bracha - <AUTHOR> http://divingintodart.blogspot.com/2016/02/book-review-dart-programming-language.html <comment><AUTHOR> I took a look at Dart again after a couple of years and was pleasantly surprised at the improvements. I especially enjoyed the await async operation [1]. The Dart engineers have done some nice work.

[https://dartpad.dartlang.org/06e989a874864aff6195](https://dartpad.dartlang.org/06e989a874864aff6195)</comment>


How I Lost 20 Pounds in 20 Weeks With My iPhone (or: Data is King) - <AUTHOR> http://chadaustin.me/2009/06/how-i-lost-20-pounds-in-20-weeks/ <comment><USER2> Couldn't agree more. I tried losing weight without keeping track of my daily weight, and got nowhere. Then I happened upon Jeremy Zawodny's spreadsheet (<http://jeremy.zawodny.com/blog/archives/006851.html>).

Maintaining a daily record of my calorie intake as well as my weight has really helped me keep my eating in check. It has also helped me get motivated to keep going to the gym - everytime I see that graph flat lining, I kick it up a notch.

</comment><comment><USER3> Very similar to "The Hacker's Diet" by John Walker, author of AutoCAD. He wrote a book about how he lost a bunch of weight using an engineering mindset. He also includes some very cool tools (Palm, Excel, and web-based).

<http://www.fourmilab.ch/hackdiet/>

<comment><USER4> Hacker's Diet is a bit scary, with all the excels and formulas. The "short version" worked well for me: count all calories, and better fast then slow.

</comment></comment><comment><USER5> Pretty much exactly how I lost 18 pounds in 24 weeks. Budget 400 calories (technical kCal) for each of three meals, allow 400 kCal for snacks, fruit, biscuits in meetings, etc., and allow a blowout every week.

Works for me.

Sneaking the weight back on - three weeks visiting the folks back home will down that, but getting back on the plan again and looking forward to seeing the progress again.

<comment><USER4> Blowouts and budgeting for snacks is damn useful. I used to exercise on blowout days, and had them somewhat more often (every fourth day). Also made sure I went over 2000 calories - this way for a couple of days afterwards I actually feel like eating less.

</comment></comment><comment><USER6> I did something very very similar. I lost about 50lbs in about 2 months by very VERY closely tracking what I was eating.

I wrote a super simple little web form for myself to input what I had eaten, and how many calories were in it. There were no fancy graphs or anything like this appears to have, but it made me very aware how what was going into my body. What made it great was that it was accessible from my phone, from my friends phone, from my computer at work, etc. etc. etc.

If you care (although I suspect that anybody here could recreate this in about 10 seconds), you can check it out here: <http://www.gibsonandlily.com/getskinny>

</comment><comment><USER7> Interesting. I lost 20 lbs in about 20 weeks with my G1. ;)

I've been going out running for about 30-40 minutes 2 out of every 3 days, and have pretty much stopped eating crap.

The phone really didn't have _that_ much to do with it, though having an app that tracked my time and distance has made it something of a game to see if I can improve my time.

I got fed up with my gradually increasing weight, and started looking into what I was eating and the calorie cost. Granola will make you roly-poly! Who knew? Clearly I had some bad ideas on what to eat.

I'm really bad at tracking calories or keeping regular regular records of stuff. For me it's simpler to just not have something around to eat at all rather than try to limit consumption. Luckily, I have something of a short food memory so I can eat the same thing most days (oatmeal with blueberries for breakfast; raw vegetables with Wasa crackers for lunch) and not get sick of it.

A key factor was seeing real progress. I hate making myself get dressed and go out first thing in the morning, but losing belly fat and feeling (and looking) _lots_ better makes me want to keep it up.

I still eat junk now and then, and don't go crazy over it, since I know that on average I'm still getting fitter.

</comment><comment><USER8> _"If you vigorously lifted weights for an hour, you'd only burn ~400 calories, less than a single cheeseburger! You'd have to keep that up every single day without increasing your diet to lose a less than a pound per week. I decided it's easier to simply eat less."_

What he is missing here is that lifting weights is anabolic and re-building damaged muscles from weight lifting takes lots of calories. Therefore, your metabolic rate increases and your fat burning increases 24/7 when you are progressively challenging your muscles by lifting increasing amounts of weight. Therefore, burning 400kcal in one hour lifting weights isn't the extent of the effect of that workout! This guy doesn't seem to know very much about physiology or metabolism...

<comment><USER9> "This guy doesn't seem to know very much about physiology or metabolism..."

Exactly! With a bit of effort and really no impact to my life schedule, I reversed a weight gain trend. I very well could have lifted regularly or gone running, but I didn't want to.

</comment></comment><comment><USER10> <http://www.precisionnutrition.com>

These guys are fantastic if you want to change your lifestyle. In the past, I've done something like this program and I lost about 70 pounds (meanwhile becoming relatively buff and, if I might say so myself, rather sexy). I like this program because it's a laid back version of most ketogenic diets mixed with a whole food mentality (read: it's sustainable for life).

</comment><comment><USER11> Site was loading extremely slowly. So I have mirrored it here <http://www.manu-j.com/tmp/chad/>

</comment><comment><USER12> So calorie counting sounds great. Well not really. See I cook at home. By cook I dont mean "take out of box and put in microwave for specified time", I mean "chop, sautee, bake, stir, etc". If you have ever tried to figure calories from that, you understand why calorie counting sounds like 50-100x more trouble than it's worth.

<comment><USER9> You'd be surprised. After a couple weeks of obsessively counting calories and ingredients you get really good at estimating the caloric value of food. It's not hard to look at a sandwich and say "Oh, about 800." And if you're plus or minus 100, who cares? You don't have to be perfect, just show continued weight loss.

</comment></comment><comment><USER13> My personal trainer said that weight lifting is better than aerobics because it elevates your metabolism for 36 hours whereas running will only elevate it for 4 hours. Also after my weight training session on Thursday my heart rate monitor / calorie counter unit said I burned 800 calories in an hour so I believe the guy.

<comment><USER14> I don't really trust the calorie measurements from heart-rate monitors. I have mine "calibrated" appropriately and I sometimes hit 1300 calories in an hour and a half on my bike. Yeah right...

</comment></comment><comment><USER15> Why did you he say losing weight makes you smell bad? I've never heard of that. It sounds like a folk legend.

<comment><USER16> Fat is a storage device for lots of stuff. For example, THC (marijuana) builds up in fat cells. Losing weight releases some of that stored up junk. It's not out of the question that some of it would cause you to stink up a storm.

<comment><USER17> Also, LSD and other drugs. So it must be weird? awesome? for an ex-consumer to try to lose weight.

</comment></comment></comment><comment><USER18> The one thing that I wish Lose It! and similar weight-tracking apps did is report a moving average. This is crucial to Walker's "Hacker Diet"---otherwise progress looks random rather than, as a (say) ten-day moving average would reveal, steady and encouraging.

</comment><comment><USER19> How about counting how many calories you _buy_? I understand people eat out a lot, but the grocery store is a place where you can make it both precise and effortless. That's a business idea for you, or Kroger...

<comment><USER20> I thought about this, too.

</comment></comment><comment><USER21> Hrm, a title with a healthy and realistic weight loss rate. Highly suspicious.

</comment><comment><USER22> For Android a similar app is called Calorie Counter(free) and is also very good at making counting calories easier.

</comment><comment><USER23> Site not loading?

<comment><USER9> Ack, sorry. I'm in the process of moving to Dreamhost and my home DSL isn't very good at being a webhost... Sorry, this was bad timing. :)

</comment></comment><comment><USER24> It's great a calorie counting approach works for this guy, but it has a horrible clinical track record. Dozens of studies conclusively show it does not work in the long run absent focus on the composition of the calories. Even in clinical studies where they control everything the subjects eat it doesn't work.

There is good empirical data on how to eat and exercise to get and stay thin. There's no reason to rely on outlier anecdotes like this. The prescription is: minimize starch and sugar (hard to do without eating fat), eat infrequently (don't snack), brief bouts of difficult exercise (sprinting, lifting) on an empty stomach.

p.s. The headaches he describes are from very poor insulin sensitivity. He has a more serious health problem than body fat. The starchy/sugary diet he continues to eat leaves him at risk of disease. He says he still lives on junk fast food. What's the point of being thin but malnourished?

p.p.s. Caffeine is not an appetite suppressant.

<comment><USER4> Calorie counting always works - when you want to lose a lot of weight and lose it quickly. For maintaining long term you can't stick with it though, so you most likely want to make lifestyle changes, including exercise.

Exercise is fundamentally hard for weight loss. It takes a lot of time exercising in order to make a difference. Eating 600 calories less is not that hard - but from a bit of googling it looks like it would take 1.5 hours of continuous aerobics (plus time for changing and shower - that's over 2h every day). I'm not terribly out of shape, but I'm pretty sure I can't do 1.5h of aerobic and still stand. Also, I'd be famished afterwards.

If you have good data for exercise, please share. Otherwise it's still anecdote vs anecdote.

<comment><USER24> Calorie counting doesn't even work well in the short term. Metabolism drops dramatically at the _cellular_ level on restricted calories. Insulin levels are the key to fat catabolism/anabolism. Calories are secondary. You can get fatter in caloric deficit if you're just eating sugar, or injecting loads of insulin.

The value of exercise for weight loss has little to do with burning calories. It's about hormones. Brief, intense exercise boosts growth hormone and testosterone, which make you shed fat independent of calories burned. Your body reconfigures. Note that endurance exercise doesn't really have this effect.

Anaerobic exercise on an empty stomach directly burns fatty acids released from body fat. A quick lifting session (20m) while hungry followed by a one or two hour fast will burn a lot of fat. Aerobic exercise with a focus on calories is fairly useless, as you indicate. It simply burns down glycogen levels which are then replenished at the next meal with little impact on fat reserves, unless you go for a very long run or bike ride. But then this has the effect of dramatically boosting appetite, which can negate the effect.

<comment><USER25> As long as you're not doing anything drastic and your basal metabolism stays fairly constant, reducing your caloric intake works fine. Second law of thermodynamics.

<comment><USER26> "As long as you're not doing anything drastic and your basal metabolism stays fairly constant, reducing your caloric intake works fine. "

But that's the entire point. Reducing your caloric intake _causes_ your metabolism to drop, for reasons that ought to be fairly obvious if you think about it. If you think about it, it also should become obvious why that's the only possible direction of causality, too.

<comment><USER25> It doesn't necessarily cause your basal metabolism to drop. I just confirmed this with two physicians and a nutrition student.</comment></comment></comment></comment></comment></comment>


4Q: the final archive format - <AUTHOR> https://github.com/robey/4q <comment><USER2> Final? Really? What if I need xattrs and Posix ACLs (or I use Windows and want NT ACLs and streams; or forks under OSX; …)? Hard-coded encryption algorithms also don't seem particularly future safe.

<comment><USER3> No forward error correction either. It can't be considered "final" without some form of bitrot protection.

And no, storing it on a ZFS or BTRFS volume with error recovery enabled does not count. (They don't use FEC, they use 1960's triplicate storage. Hugely wasteful of space, does nothing to protect against transmission errors and can still be corrupted by two of the exact wrong bits being damaged.)

Storing it on a media that does use FEC also does not count. I want a per-file tunable FEC knob, not one vendor-determined setting. And as history has shown, it needs to be done through FOSS code and not trade secret firmware.

<comment><USER4> Is there any FS that implements FEC?

<comment><USER2> Not that I'm aware of. The device layer usually does it already and it's _supposed_ to be sufficient.

<comment><USER5> > it's supposed to be sufficient.

Supposed to. If you do any backups to DVD/BD and still want to get your data back in 5 or 10 years, though, you'd be well advised to do _some_ sort of FEC - burn multiple copies of each disc, generate a bunch of PAR2, whatever.

(You might want to do that for backups on hard drives too. Yeah, maybe the hard drive firmware is supposedly taking care of any errors below the block level and you're not too worried about bitflips, but that just means you'll lose entire blocks and files when you lose something.)

</comment></comment></comment></comment></comment><comment><USER6> The README implies that `tar` is "not streamable". Someone needs a history lesson on what it was originally used for...

<comment><USER7> Tar is not streamable _when compressed with gzip_. 4Q supports compression on a per-file basis and thus can support streaming and compression at the same time. Zip also supports compression on a per-file basis, but it requires an index and thus is not streamable.

<comment><USER8> Funny, I've done plenty of gzip compressed tar streams:

        nc -l 7000 | tar -xf -
        tar -czf - * | nc 18.104.22.168 7000

Files appear one by one on the remote end.

</comment></comment></comment><comment><USER9> Things it doesn't support: symlinks, posix acls (xattrs). The first one makes it a certain failure for archival use. The hardcoded link to an external crypo service keybase makes it a failure for long term use.

<comment><USER10> "The final archive format" is a very big promise that 4q doesn't keep right now. It falls short of 7z, RAR and tar.xz, and certainly isn't ready to replace them at the moment.

I'm not too familiar with Coffeescript, but it doesn't seem like a good choice of language to write an archiver. There's no actual draft file format spec I can see, either? But from a first pass, I have the following comments:

Crypto: Encrypted blocks: AES-256-CBC, random IV, with no MAC (!!!). You _need_ to look at that again: that could be a Problem. Hashed blocks: SHA-2-512. Maybe OK (how's length encoded? Look out for extension attacks). That crypto is 14 years old and missing a vital bit: not "modern". Modern choices would include CHACHA20_POLY1305 (faster, more secure, seekable if you do it right); hashes like BLAKE2b (as the new RAR already does); signing things with Ed25519. Look into that kind of thing. You need a crypto overhaul. The keybase.io integration is a nice thought for a UX - but is an online service in invite beta really ready for being baked into an archive format?

Packing: LZMA2 is pretty good: 7z and xz already use that. For a fast algorithm, Snappy is not as good as LZ4, I understand? Neither is the last word in compression. Text/HTML/source code packs much better with a PPM-type model, like PPMd (7z has that, too, as had RAR, but removed it recently), but you need to weigh up the decompression memory usage. ZPAQ's context model mixing can pack tighter, but that's much more intensive and while I like extensibility, I don't like the ZPAQ archive format having essentially executable bytecode.

Other missing features that other archivers have: Volume splitting? Erasure coding or some other FEC? Can you do deltas? (e.g. binary software updates)

You've got some pleasant UX ideas for a command-line archiver (compared to some other command-line archivers!), but sorry, I don't think you're ready for 1.0.

</comment></comment><comment><USER11> Somehow this makes me think of this xkcd comic [https://xkcd.com/927/](https://xkcd.com/927/)

<comment><USER12> I like the mouse over text: "Fortunately, the charging one has been solved now that we've all standardized on mini-USB. Or is it micro-USB? Shit."

Now we're going to transition to USB-C.

<comment><USER2> Unless we're going to transition to DockPort to tunnel USB over DisplayPort.

<comment><USER13> Fairly certain they dropped DockPort as USB Type-C covers all the functionality already.

<comment><USER2> I hope so. But when did that ever stop anyone?

</comment></comment></comment></comment></comment><comment><USER14> Why would you make an archive format depend on a third-party service (keybase)? That's just a terrible idea for data longevity.

</comment><comment><USER15> I really hope that this does not get mainstream or I will have to install yet another archiver tool...I really don't understand why people use 7zip for example when storage is cheaper than ever. Just use tar and get on with your life.

<comment><USER16> Maybe because it doesn't have unicode issues that zip has?

<comment><USER17> Put "cu=on" (without quotes) in the "Parameters:" box of 7-Zip's "Add to Archive" window, and all your Unicode issues are solved.

</comment></comment></comment><comment><USER18>
      "the final archive format"

[https://xkcd.com/927/](https://xkcd.com/927/)

;)

</comment><comment><USER19> How is that any better than modern archive formats like 7z?

</comment><comment><USER20> What are the advantages of this over tar?

<comment><USER21> The four things in the first paragraph of the link?

<comment><USER22> Tar already has the first two, and even POSIX xattrs (which this doesn't preserve), the third seems useless (seems being the key word here, some people might find it useful), and I'd rather just use a program that will encrypt the archive for me (i.e. have a .tar.xz.enc).

One advantage this could have over the above, is if you can open any file at random, as with the above scheme, you might have to linearly decrypt and decompress the entire archive up to that file.

<comment><USER23> Technically speaking then, the tar utility cannot compress or encrypt per- file, but the _tar format_ can be used for this, and since we're talking about format then the tar format _can_ accomodate the requirements. It's just that there's no tool doing it at the moment.

(A counter point: while each file could be compress and encrypted, there's nothing in the tar format that explicitly says so, meaning that each file would have to be probed to determine if it was compressed or encrypted)

</comment></comment></comment></comment><comment><USER24> It's in coffeescript! I wasn't expecting that, I was expecting C, Go, or Rust.

</comment><comment><USER25> If it's final I guess the TODO section should be empty ;-)

</comment><comment><USER26> Click the language bar of GitHub, see CoffeeScript, close the tab</comment>