Deep image prior 'learns' on just one image - <AUTHOR> https://dmitryulyanov.github.io/deep_image_prior <comment><USER2> Wow:

"In this work, we show that, contrary to expectations, a great deal of image statistics are captured by the _structure_ of a convolutional image generator rather than by any learned capability. This is particularly true for the statistics required to solve various image restoration problems, where the image prior is required to integrate information lost in the degradation processes.

To show this, we apply _untrained_ ConvNets to the solution of several such problems. Instead of following the common paradigm of training a ConvNet on a large dataset of example images, we fit a generator network to a single degraded image. In this scheme, the network weights serve as a parametrization of the restored image. The weights are randomly initialized and fitted to maximize their likelihood given a specific degraded image and a task-dependent observation model.

We show that this very simple formulation is very competitive for standard image processing problems such as denoising, inpainting and super-resolution. This is particularly remarkable because _no aspect of the network is learned from data_ ; instead, the weights of the network are always randomly initialized, so that the only prior information is in the structure of the network itself. To the best of our knowledge, this is the first study that directly investigates the prior captured by deep convolutional generative networks independently of learning the network parameters from images."

PS. This makes me wonder whether and to what degree the _structure_ of the brain's connectome might be a necessary prior for AGI.

<comment><USER3> Can someone explain this in a way so that an ordinary mortal computer scientist can understand it?

<comment><USER4> > Can someone explain this in a way so that an ordinary mortal computer > scientist can understand it?

I'll try.

Instead of the common approach that tries to search for image pixels to minimize e.g. a denoising objective, they realize that they can instead search for the weights of an image generator network such that the generated image matches the objective.

They argue that the structure of the network then constitutes some prior knowledge over what a natural image should look like.

My (probably wrong) interpretation: since a convolutional neural network essentially works by looking for some spatial patterns at different resolutions, their optimization process boils down to finding the high and mid-resolution patterns that best match the input image, and then re-using that information to fill in the missing information (or replacing "noise" that does not match the extracted pattern).

<comment><USER5> What do they mean by 'the structure of the network' is this the network topology?

</comment></comment></comment></comment><comment><USER6> Remember fractal image compression ?

[https://youtu.be/AjdogjBxfco?t=260](https://youtu.be/AjdogjBxfco?t=260)

<comment><USER7> This can also be looked at as the original source of patch based denoising, etc. In the end it's about capturing the scaling properties and self similarity of natural images. This is also, for example, why wavelets were so effective as a basis.

David Mumford particularly did some great work on this sort of thing a couple of decades ago, along with many others. I hope when people are rushing around trying to apply convolutional nets to everything they aren't losing these insights.

<comment><USER8> The benefit of CNNs is like the benefits of SVMs -- they generalize all the great old techniques so you don't have to understand them all, you just throw more CPU at the optimization problem.

<comment><USER7> I don't think that's true particularly in this case.

This paper is pointing out that you can encode a structural prior in a CNN - but knowing the "great old techniques" will help you design the right network architecture to do that.

SVMs were a surprise when they came out, no so much a generalization as a challenge (at least at first)

</comment></comment></comment></comment><comment><USER9> Huh. This seems to boil down to 'noise is higher information entropy than realistic content; partial learning will learn realistic content before learning noise' or something like that.

<comment><USER10> I think that can be mainly attributed to the fact that the last few deconvolutional features are overfitted to features in the image and are somewhat robust to noise. The network does not even learn features to produce e.g. white noise as output. This is probably much less magical than the paper makes it seem to be.

</comment></comment><comment><USER11> Red Dwarf - "Uncrop" \- [http://www.dailymotion.com/video/x2qlmuy](http://www.dailymotion.com/video/x2qlmuy)

<comment><USER12> Actually... How cool would it be to have an NN that could extend an image's background with plausible scenery? Not just photoshop 'smart' fill, but for example if it detected a building on the right side of an image, it could draw the rest of it? :)

<comment><USER13> This exists.

This approach takes other photos of the same scene to extend a cropped pic (see also MS PhotoSynth): [http://grail.cs.washington.edu/projects/sq_photo_uncrop/](http://grail.cs.washington.edu/projects/sq_photo_uncrop/)

This uses a GAN to fill in missing parts of a pic. Those parts could be on the edge of the picture (although the paper doesn't explore that): [http://hi.cs.waseda.ac.jp/~iizuka/projects/completion/data/c...](http://hi.cs.waseda.ac.jp/~iizuka/projects/completion/data/completion_sig2017.pdf)

<comment><USER14> Here's another earlier example of un-crop, that uses only the source image (doesn't need gps or an internet database of photos), and does both out- painting and in-painting.

[http://graphics.cs.cmu.edu/people/efros/research/EfrosLeung....](http://graphics.cs.cmu.edu/people/efros/research/EfrosLeung.html)

</comment></comment></comment></comment><comment><USER15> Wow, so just the weight sharing architecture does so much already? I am wondering if the same could be done with LSTMs on sequences or CNNs on voice...

<comment><USER2> I'm wondering the same thing too.

Note also that this finding strongly suggests that _neural net architecture_ actually is quite important, possibly even more important than having more data -- which contradicts the conventional wisdom!

<comment><USER16> There is some pretty strong evidence for this: all the toddlers in the world. You only need to show them something _once_ and they'll immediately be able to recognize more examples of the same thing from different angles and even when it is partially hidden. All they have to guide them is the structure of their brains, not the quantity of data they have been exposed.

<comment><USER17> "All they have to guide them is the structure of their brains, not the quantity of data they have been exposed."

A typical toddler (say 12 months' old) has spent 4000-5000 hours with open eyes. Even if you assume a low frame rate (10fps), resolution (1080p), and a 1000:1 compression ratio, that's still 1TB of training data.

<comment><USER8> Consider language acquisition:

[https://en.m.wikipedia.org/wiki/Poverty_of_the_stimulus](https://en.m.wikipedia.org/wiki/Poverty_of_the_stimulus)

</comment></comment></comment></comment></comment><comment><USER18> This shouldn't really be surprising. Machine learning is specifically not magic. The reason CNNs have seen so much success is precisely because they build in translation-invariance, which massively cuts down on parameters while forcing the final function to have the desired structure regardless of wherever gradient descent takes the weights.

Also why most papers in deep learning are network architecture innovation.

<comment><USER18> One more relevant note - (Olshausen and Field, 1997) showed that the filter employed by V1 simple cells could be learned using some simple assumptions about sparse coding and a single image. Translation invariance built in by way of the sampling scheme of the image, small patches.

The filters learned by the first layer of CNNs is usually of the same type, Gabor filters. Not a coincidence.

That was twenty years ago. What's old is new?

<comment><USER10> What do Gabor filters have to do with this?

<comment><USER7> Parent is saying Gabor filters are typically effectively recapitulated by the first layer of of the network anyway, as they are a natural representation.

<comment><USER10> But what does have to do with smoothness and translation invariance which this paper is a demonstration of? You even learn Gabor filters with local connectivity without spatial weight sharing.

</comment></comment></comment></comment></comment><comment><USER19> Can someone break this down for this layman?

<comment><USER20> Not an expert so take this with a grain of salt; I could be misinterpreting the paper.

It seems that the current accepted method is to train a network with distorted images as the input and the correct undistorted images as the targets. Then after training you can feed a new distorted image into the trained network and get the estimated "fixed" image.

However this team actually uses the distorted image as both the input and the target to the net. So if they were to let the training go on for too long the network will produce an exact copy of the distorted input image. But for some reason, the structure of the network means that the estimated output learns realistic features first, and then overfits to the noise afterwords. So if you stop the training early, you get an image that incorporates realistic features from the distorted image, but hasn't had time to "learn" the noisy features.

<comment><USER21> This is fascinating because I've been running into something similar with sequence to sequence models translating natural language into Python code. I got better results stopping "early" when the perplexity was still quite high, I thought it was a little crazy.

<comment><USER22> [https://en.wikipedia.org/wiki/Early_stopping](https://en.wikipedia.org/wiki/Early_stopping)

</comment></comment></comment></comment><comment><USER23> This is incredible! Can't help but wonder if the brain does something similar to fill up "gaps" in reality. e.g how we fill up our perception (not only vision, but general mental intuition) based on just context.

<comment><USER24> That's exactly what happens when the retina is damaged. The brain fills in the void imperceptibly so you aren't distracted by the deficit. But I think biology doesn't fill in the void synthetically. It's more likely that the brain "turns a blind eye" toward the missing 'pixels' and directs your attention elsewhere, perhaps to those remaining regions with high saliency (useful detail).

</comment></comment><comment><USER25> As I'm not an expert in the field; what exactly does the term

      min_x E(x; x0) + R(x)

mean?

I thought that E(x; x0) would denote the error/difference between original and corrupted images, and R(x) be the (searched-for) correction. But this doesn't seem to make sense with the next parts of their explanation.

<comment><USER26> x is actually the generated image they are testing against x0. A lower E(x; x0) means an image which fits well towards the objective based on the original image (depends on the task). The paper gives some examples. For example, for the task of image denoising, E(x; x0) is just the squared distance of the generated (denoised candidate) image x to the pixels to the original image x0. Obviously you would want this to be low in the generated version since it should still look close to x0.

R(x) is a regularization term to avoid overfitting. For example, in the denoising example, it could be a measure of the variation in color of x. Clearly, just taking x = x0, the squared error (E(x; x0)) is 0, but it will have high R(x) because of all the noise. That's why they try to minimize both quantities combined, so we get min_x E(x; x0) + R(x), to get close to the objective but also not overfit.

[https://en.wikipedia.org/wiki/Regularization_(mathematics)](https://en.wikipedia.org/wiki/Regularization_\(mathematics\))

</comment></comment><comment><USER27> Would a plausible explanation of this be that one part of an image tells you a hell of a lot about another part of it? And that ConvNet structure captures that really well?

<comment><USER28> Yes

</comment></comment><comment><USER29> How can it possibly know what was in the white areas of the library? Is there a residual image?

Seems impossible that it guesses correctly.

<comment><USER30> It doesn't guess "correctly" at all. Zoom in on the image, and focus on the filled-in areas, they look _really_ blurry. It just doesn't look very bad from a birds-eye view.

<comment><USER31> Somewhat similar to content aware fill in Photoshop [0]. The untrained network can latch onto frequent patterns and match them to holes in the data.

Why doesn't it paint everything white? Are these actually transparent images or are they somehow tagged?

[0][https://helpx.adobe.com/photoshop/using/content-aware- patch-...](https://helpx.adobe.com/photoshop/using/content-aware-patch- move.html)

<comment><USER32> Yes, for the inpainting, the parts to be painted (big white deleted areas) are supplied as masks, so it doesn't try to match them.

<comment><USER29> But its generating unique content in those areas...

<comment><USER32> Yes, every "run" of the network is generating pixels in those areas, but they're not being compared against the white (deleted) pixels. On the final run of the network, they're still not being compared against anything, except by us, visually, when we look at those pixels.

</comment></comment></comment></comment></comment></comment><comment><USER33> Maybe it's the presentation of the restoration process, but I'm particularly impressed with the inpainting sample.

The idea of not running the simulation `past` the realistic interpretation and using that result makes sense but the results are way beyond what I would have expected!

Great work on the write up.

</comment><comment><USER34> I feel like this is related to the information bottleneck idea that's been floating around for some time [1]. I only really understand both of these at a superficial level, but from what I think I understand one thing that they observed is that there are two phases when training a deep learning model: a phase which maximizes the mutual information (?) between the input and the output, and a compression phase which compresses the learned representation. In that light this work makes sense, since artifacts are essentially noise that the network would filter out in the fitting process.

Very cool work though.

[1]: [https://youtu.be/bLqJHjXihK8](https://youtu.be/bLqJHjXihK8)

</comment><comment><USER35> I don't see how any choice of a function g(theta) could have the property they desire, ie could eliminate R(g(theta)). Can anyone explain?

<comment><USER36> It's expressed somewhat awkwardly, but what's going on is that R(x) is zero if x is in range of g, and infinite otherwise. Choice of g is such that natural images are in range of g, and non-natural images aren't.

<comment><USER35> Sorry I still don't understand. They require g to be surjective. Edit: in their paper they call it f_theta and it's explicitly not surjective. Dunno why their writeup is so confused.

</comment></comment></comment><comment><USER37> Question to anyone that knows this area in depth.

I'm assuming that this impressive feat has a disadvantage (over the traditional example intensive technique of training a CNN with something like ImageNet) in the form of taking a long time to generate the corrected image.

With that assumption in mind, could this new technique be reversed? As in feeding in sharp images, getting back out corrupted ones, for the purposes of generating data sets where there isn't much data to begin with?

You could then take that data and use it to train a more traditional CNN to sort of amortize the results of the technique in the paper and have the process happening faster.

<comment><USER38> Do you mean generative adversarial networks?

[https://en.m.wikipedia.org/wiki/Generative_adversarial_netwo...](https://en.m.wikipedia.org/wiki/Generative_adversarial_network)

<comment><USER37> No not really, this isn't generation for the sake of classification. It's generation in order to generate a dataset to train a new network that exhibits the characteristics of the network in the paper that can be used quickly and in a more general fashion.

<comment><USER38> Meaning that your assumption is that there is an optimal generalized network structure for generate a neural network as described in the paper and that as such, what is missing from the research is how to create an optimal way to generate a malformed source/target seed.

Is that correct?

<comment><USER37> I'm not suggesting it's missing from the research as I don't have anywhere near the area of expertise to make that call.

But you're correct in your summary: I'm wondering if that's a possible follow up.

</comment></comment></comment></comment></comment><comment><USER39> Isn't it simply hardcore overfitting of network?

I haven't read the paper yet but it seems to me that it shows you can use overfitting for output interpolation with great results.

Same should be possibile for example for interpolation of video images instead od pixels.

</comment><comment><USER40> So what kind of generative stuff does it "learn" from really really sparse data, or from random noise? Trained generative models can produce some crazy things, I wonder if something similar could work here.

Recursive geometric patterns perhaps?

</comment><comment><USER41> An interesting question this raises is--what determines when the training has overfitted to the noise/error in the input image? If that's another NN that provides a perceptual model of quality, how was it trained? ;)

</comment><comment><USER42> This is as close to magic as humanly possible.

<comment><USER12> Safe to say that Arthur C Clarke would count this as 'sufficiently advanced technology' then I guess :)

</comment></comment><comment><USER43> Enhance. [http://knowyourmeme.com/memes/zoom-and- enhance](http://knowyourmeme.com/memes/zoom-and-enhance)

</comment><comment><USER44> Some of these image restoration techniques are extremely impressive, but its odd to think how much visual history will be faked in tiny ways if these "fix" techniques become widespread in commonly used photo software.

I suppose this is no different really than the extent to which images are edited anyway now, in that many images are never really a fastidiously accurate representation of the original scene, but interesting to consider none the less.

</comment><comment><USER45> I'm finding it hard to put into words what I find wrong with this paper, but ... here goes nothing.

So, the novel thing here is that an encoder-decoder network applied to an image can learn enough from a source image to be useful. In some ways that's obvious, but the effectiveness of it on reconstruction tasks is certainly surprising.

I have two problems, though. One is that I would take the reconstruction results with a grain of salt. The examples are clearly lab queens, where the occluded regions are not particularly interesting/challenging.

Two is the conclusion the authors reach. Somehow the authors go from the novel discovery I describe above, to saying that somehow the architecture of the network is a prior.

Well ... I mean, yeah a network's architecture _is_ a prior. But it's not actually significant.

See, in the dark ages of machine learning we had only fully connected networks. They sucked. They'd always overfit and underperform or were impossible to train. Then we finally got convolutional networks, and suddenly a whole slew of machine learning problems became easier and that hurdled us into the current renaissance.

But, you see, convolutional networks weren't the _only_ reason for the dawn of this new age. Rather it was three major things: 1) Convolutional layers, 2) more data, 3) more computing power.

Some time after the "discovery" of convolutional layers we found out that, hey, our old fully connected networks actually _do_ work. If you give them enough data and enough computational power, you can get them to perform as well as state of the art convolution networks. The great thing about fully connected networks is that they assume nothing. That means A) you can theoretically get better results and B) you don't have to spend time designing an architecture.

So we already know that architecture isn't ultimately important. You can have a giant, fully connected network, and it _will_ work, if you feed it enough data and have the computational power necessary to train such a beast.

Convolutional layers are just simplifications which make training easier. They are priors in the sense that we know a fully connected layer in image applications would just devolve into a convolutional layer anyway, so we might as well start with a convolution layer. That "design" is the prior. But it's not mandatory; the network would still function without that "prior".

So ... I'm not sure how the authors are taking their research and using it to come to the conclusion that their results are because of some magical property imbued into the network by the "priors" of the architecture.

They apparently tried other architectures and got poor results, and so they use that to claim that architecture is the only reason their technique works.

That's like if you started with ResNet for a classification problem, tried other architectures, saw that they performed worse, and then published a paper saying that Residual Networks somehow embody the fundamental forces of natural images in their architecture, and that's why they work. When the truth is that ResNets aren't special, they are just easier to train.

Another example from the annals of machine learning history: time and time again when there is a breakthrough in architectures, it's usually followed a few years later by a simplification of the architecture. For example we started with networks like VGG which are these big, hand crafted architectures. Slowly over time architectures have become _less_ exotic, instead opting to simply define a basic building block repeated N times.

The reason for this is because in the intervening years we gather more training data, better training techniques, and more computational power. So we can instead use a more homogeneous architecture which has _less_ assumptions (less priors) and at the end of the day we get _better_ results.

I'll repeat that. We put _less_ priors into our networks and we get better results.

So on the one hand we have _all_ of machine learning history telling us that priors in architectures are _bad_. On the other hand we have this paper which makes some really weird logical leap from "we tried a few architectures, they were worse, so architecture is _key_ to machine learning and it's important because we need good priors built into the architecture."

Anyone remember hand crafted feature vectors? I do. Those were priors. Guess what happened when we got rid of them and used generic networks feeding directly from the raw data? Oh right, all of modern machine learning...

<comment><USER46> > Convolutional layers are just simplifications which make training easier. > They are priors in the sense that we know a fully connected layer in image > applications would just devolve into a convolutional layer anyway, so we > might as well start with a convolution layer. That "design" is the prior. > But it's not mandatory; the network would still function without that > "prior".

As far as I know this is incorrect. Can you point to a paper that shows this? If by "easier to train" you mean that the models do not overfit training data, then that's the whole point of using correct priors / hypothesis classes.

I'm not sure what bugs you in this paper, but the point is that they decouple the prior architecture from the training/optimization mechanism, and that seems interesting.

</comment></comment><comment><USER47> This is unbelievable that it works so well.

</comment><comment><USER48> That Zebra one looks like a "before and after" image with really good MSAA. I realize the techniques are quite different, it's still interesting how the result seems so similar on a superficial level.

</comment><comment><USER49> This is interesting news to neuroscientists as well. Even without plasticity cortical networks might be performing useful functions. Plasticity may then perform some different function.

</comment><comment><USER50> I tried to run their ipynb but it wants something called skip. I don't think it's the "skip" on PyPi (that doesn't work anyway) - so what could they be using?

<comment><USER36> skip.py is in the repository under models directory.

</comment></comment><comment><USER51> ok.. this is revolutionary. Using the architecture as a way to capture an image prior hints at how network structure and captured invariance are related. By analogy it leads to thinking of brain areas as both hard coded prior knowledge through their natural arrangement and plastic learning structure. Turning the problem on its head shines a new light to how we could conceive network architectures.

</comment><comment><USER52> Is there a straightforward explanation of how to run this anywhere, for those not steeped in the arcane ways of Jupyter?

</comment><comment><USER53> Cool but I think a learned method is more interesting, especially how it can be used for image compression.

</comment><comment><USER54> Can someone put their results and explanation in laymen's terms please?

</comment><comment><USER55> i find it impressive how it placed a lamp over the library window. After that i was expecting a vase with flowers to be placed on the table in the next, palace shot :)

<comment><USER56> could you highlight which part this is ?. I think it can only fill in patterns that already exist in the image. I see a big artifact over the window but no lamp.

<comment><USER55> well, probably it is that deep (i hope) neural network inside my skull that interpreted the artifact as the lamp.

</comment></comment></comment><comment><USER57> how do I get the code running easily? I'm familiar with python but not with ipynb at all. Maybe somebody can give me some easy to follow instructions, thx.

</comment><comment><USER58> I wonder if you could do something similar with audio

<comment><USER59> And turn crappy music into good music.

</comment></comment><comment><USER58> This is amazing.

</comment><comment><USER60> Am I missing something here? How does this apply to DNNs that tried to recognize images? Does it mean we can train a Net with just image now?

</comment><comment><USER61> can someone please summarize this for us luddites? also wasnt there an HN summary project out there?

</comment><comment><USER62> Truly congratulations! Amazing.

</comment><comment> 0x53-61-6C-74 CSI:Miami...I'm sorry, I take it all back.

</comment><comment><USER63> The intel and espionage communities are going to be all over this. This makes the Soviet photo retouching look like child's play.

Let's say you want to start a war, and need some evidence of chemical weapons. Now, you can drop in some images of chemical weapons and claim a GAN found them. Sample press releases:

"We believe this photo was retouched to hide the chemical weapons. Using a GAN, we recovered clear photographic proof of the chemical weapons."

"We believe they are using this tin-roofed building to hide the chemical weapons. Using a GAN on our own satellite images, we recovered clear photographic proof of the chemical weapons."

<comment><USER37> Can't see "we used a GAN" as an explanation that would fly. To counter it you can simply say "well, the GAN could be wrong."

The technique outlined above is useful because we as humans can eyeball the results and say "the improvement on that blurry zebra photo is great!", but we know that it's an improvement based on a networks hueristics that's good enough for us, not an exact replica of the information lost through noise and compression.</comment></comment>


TechCrunch Design Suggestion (Photoshop mockup) - <AUTHOR> http://www.webdesigncompany.net/wp-content/uploads/2011/07/tc-tweaks.jpg

<comment><AUTHOR> I'm not sure if it's just new or if it's truly disgusting, but the new TechCrunch look just isn't sitting with me right... so instead of being a whiney prick, I figured I'd try to tweak their current layout to make it look & feel a little better. This 1 hr rough job looks a lot less distintive than their new design but it just feels more readable for me.</comment>


Show HN: Exobrain like IFTTT, but free, open source - <AUTHOR> https://github.com/pjf/exobrain <comment><USER2> The biggest concern I have, is I haven't found one of these that really meets all my needs. I want it self-hosted, it has to be expandable with plugins, and it needs to have a wide enough userbase and dev community for plugins to be actively developed and maintained.

Mayhem, as I mentioned, is really cool (for Windows), and it has plugins, but since it isn't maintained, it's limited. It supports doing INSTEON actions, but not listening for INSTEON events, and the types of modules it can handle are limited. I had to get a separate program (PopTrayU) to check my email, because Microsoft only built a plugin for Outlook, which isn't installed on the machine I run it on.

A lot of these, bip.io included, are more heavily dependent on cloud services. If I wanted to use cloud services, I'd use IFTTT. I want self-hosted alternatives. (Why would I need a Trello plugin or a Slack plugin if I like self-hosting my data?!?! I don't mind them existing, but there should be self- hosted options here.)

</comment><comment><AUTHOR> I kinda dig the whole automating thing.

The alternatives for above ( freshly found) that are worth mentioning are:

[https://bip.io/](https://bip.io/) and [https://github.com/cantino/huginn](https://github.com/cantino/huginn)

</comment><comment><USER2> Have you seen Microsoft's Mayhem project? They open sourced it, nobody seemed to notice it existed, and then they abandoned it.

I'm using it to run my home automation system right now.</comment>


Death To The Open Offices Floor Plan - <AUTHOR> http://www.fastcolabs.com/3024697/death-to-the-open-offices-floorplan <comment><USER2> Every time an article like this gets posted, a huge discussion ignites over the "better communication" of open office plans vs. the "less distraction, more focus" of offices. (A microcosm of this is "wear headphones if you need quiet" vs. "Why should I need headphones?")

Anecdotally, the open-office haters seem to outnumber the others.

Personally, (and this has been said before), I think an arrangement where you have private spaces, and a common central space, like a large/long table, would be best. The common space would be great for times when the team needs a lot of communication (i.e. bootstrapping a project) while the private spaces are good for more mundane tasks.

Allow teams to move between the two. Just my speculation, but the real culprit behind the productivity killing seems to be forcing people to work in an environment they don't like.

<comment><USER3> Exactly. I love working in a collaborative space sometimes, and it's really helpful for me sometimes. It's also basically necessary for me to spend at least one day a week entirely in a private room. Luckily that's an option where I work.

</comment></comment><comment><USER4> Some of these open offices are like factory chicken farms. You can just imagine those chickens going: cluck, this is better than cubicles, cluck, cluck, cluck...

To the people who prefer open offices I have two words. Cognitive. Dissonance.

:-) More seriously: It is somewhat a matter of personal preferrence but I think all/most people need some space and are not comfortable by others being constantly inside their personal space. You can do this experiment. Go to an empty meeting room and work there. Then ask your coworker to sit exactly behind you while you work. How does that feel?

I've had some of my best team and communication in an environment with offices. Need to talk, get into someone's office. On most days we had the entire team in someone's office talking for hours about architecture, code reviews, etc. We weren't disturbing anyone else and it was just the right space for our small team. Most of the time though we were able to focus on what we were trying to do without distractions. I contrast this with open offices where people never talk to each other and seem unable to do work that requires any concentration.

Not everyone can wear headphones all day. Those that do I recommend that you get your hearing checked. Visual distractions are also a prolem.

To conclude again on a slightly humorous note, is Oculus VR the next step in open space? Get an office while we cram you one on top of another? Your very own virtual personal space? I just need my man-cave, sorry.

Edit: It's also very much a question of how much space per person is available in whatever scheme is used and how many people per open area. It's not a problem for me to share one large office. I once visited one large known tech company where they built a new space and offered people the option of going two per office or one in the open. All the offices were full.

<comment><USER5> Business, as a whole, seems fixated on eliminating every facet of introversion:

"Individual analysis is bad! Collaboration is good!"

"Thinking hard takes too much time! Instead, keep shipping!"

It seems to reflect a culture that prefers ambient noise to quiet reflection, focus, and productivity. It's almost as if people would prefer to _see_ me typing over having the results of what I make. It is this notion that makes open offices a bit worrisome. I've been productive in both private and open office environments, but I'm very much an introvert. I need a certain amount of physical space that feels private; this manifests itself mentally as breathing space to explore whatever problem domain I'm working in.

Aside: it's amusing that SV prides itself on being progressive while not trusting their workers with a modicum of privacy. Guess it fits the whole 'privacy is not for plebes' notion that powers it, eh?

</comment></comment><comment><USER6> Open offices are great if you have a bunch of weird computer people who don't know how to talk to each other. It's easier to reach over and physically walk into someone's space to make them acknowledge you than it is to just walk into a closed office (or, worse, schedule a "meeting" to talk to someone).

Open offices also require trust on all levels. Management layers must recognize the reality of an 8 hour work day involves many breaks of non-work browsing. If you have an open office plan with management demanding you be on- task for 8 hours a day, that's a recipe for nervous breakdowns (or a recipe for people fighting over desks where monitors face windows instead of the open hallway areas).

Another endless plus for open offices: If you sit with your team, you have open-air information sharing. "Oh, X is walking over to talk to al of us about problem Y. We can collectively address their problem in 5 minutes instead of three days of emails back and forth."

You're probably not as important as you think you are. You only need an office if you're dealing with sensitive HR materials. (Or: if you have an annoying laugh. Or if you have flatulence problems. Or if you talk on the phone all day and have an annoying voice. Or if you bring your loud kids to work. Or if you video chat over speakerphone all day long.)

</comment><comment><USER7> Can we give each employee their own office cube like these? They're not very roomie, but they do provide noise isolation and also save space. If we can just arrange them in a line in a warehouse or maybe even stack them vertically.

[https://www.google.com/search?q=backyard+office+cube&client=...](https://www.google.com/search?q=backyard+office+cube&client=safari&rls=en&tbm=isch&tbo=u&source=univ&sa=X&ei=VCzQUpRZydmgBI- JgYgH&ved=0CHUQsAQ&biw=2560&bih=1320)

<comment><USER8> Probably not, because cubicles and open-floor plans are really about saving money, not about providing workers with the best environment.

I interviewed at Fog Creek once and probably the coolest thing about the company is every developer gets a private office. Notably, these offices (as best as I can remember) are not particularly big. They may not be much bigger than a cubicle. The main distinction is they have glass walls extending all the way to the ceiling together with a door that closes.

Do they take up extra space compared to normal cubicles? Probably, but not significantly more space--certainly not nearly as much space as traditional "this is 5x as much space as I actually need" offices, with unused couches sitting around just to fill the room.

But this sort of construction is expensive, so it's almost never done.

<comment><USER9> > _cubicles and open-floor plans are really about saving money, not about > providing workers with the best environment._

It's staggering how I had to wade half-way through piles of double-think comments until someone finally spoke the truth.

</comment></comment></comment><comment><USER10> A cursory glance at current office floor plans brings only a few options:

1) Cubicle Farm. In the article, this was voted as "even worse" than an open floor plan.

2) The Open Floor Plan, which is getting a pretty bad rap on HN as of late.

3) Everyone gets their own private offices yay! If you're not working at Sterling Cooper & Partners (where there are private offices for pretty much everyone but "creative" and the secretaries), this might be unfeasible.

4) Some sort of hybrid pseudo-open-but-still-private layout that engenders the best of both worlds. There are a few silicon valley companies that hit this one on the head, and that's awesome.

To get to the point, I think I'd rate each option I've outlined above (using their numeric value) as 3 > 4 > 2 > 1\. But it ultimately comes down to a comfort/feasibility ratio. Cubicles are the most feasible, but also the most hated. Private offices for all(TM) is the most desired, and the least feasible. Option 4 is somewhere in the middle, leaning toward less feasible (need a lot of space, etc...). It would seem that the open office is (currently) the most feasible option for companies with little creativity and a tight budget. This is the way I see it. Love them or hate them, in my opinion, open offices have the highest comfort/feasibility ratio.

Thoughts? I'd love other people's opinion on this.

 _Edit: formatting_

<comment><USER11> I'd personally rather have option 1 over option 2. The fabric walls absorb sound, block visuals, and give you at least some privacy. Of course, I'd prefer option 3 more, but I think the problem with cubicles is more of a perception problem than actual problems with the cubicles themselves.

</comment></comment><comment><USER12> I'm a teacher. We have open plan offices. When I need quiet (marking complex pieces of work or thinking up new ideas) I go to the quiet study area in the College library. Wifi (good speed) and staff to enforce the no talking rule.

So how about a library? Corporate space with study carrels and dead tree of a vaguely useful nature (books in mass on shelves absorb a lot of sound).

</comment><comment><USER13> Peopleware has some good information on quantitative value of private offices vs alternatives: [http://www.amazon.com/Peopleware-Productive-Projects- Teams-3...](http://www.amazon.com/Peopleware-Productive-Projects- Teams-3rd/dp/0321934113)

Their discovery was that people who had closed door offices were more productive. In general people with less interruptions got more done.

The book is somewhat dated, so this may have changed since, but one of the interesting things they found was that there weren't really any studies showing that open offices helped productivity for software teams. It was purely a cost saving measure by execs wrapped up in wrapping of "being collaborative".

<comment><USER14> The individuals may have been more productive; did they measure the output of the teams?

I don't think anyone disputes that people are more productive without distraction. That's not what open plans are for though: collaboration helps everyone orient around the _right_ thing, so all that individual productivity isn't wasted pursuing the wrong stuff.

<comment><USER13> That's a good point and I'd definitely like to see some quantitative studies on that.

</comment></comment></comment><comment><USER15> heh - wait until Zuck moves his 3,800 engineers into that one, giant, single room at the MPK-West campus building.

Only his circle of 5 get private offices.

That building will be amazing - but it is the pinnacle of the open office idea... that's OK for them though because everyone at FB drinks the kool-aid.

<comment><USER16> Not to mention young people are just smarter so just ignore all these old fogies telling you there's something wrong with open offices (except for yourself because, ya know, you're special).

[http://news.cnet.com/8301-10784_3-6171235-7.html](http://news.cnet.com/8301-10784_3-6171235-7.html)

</comment></comment><comment><USER17> If your WiFi actually works (unlike at my office where it sucks and often is home grown supported) there isn't any reason for people to be stuck in one place. We had open office where I had like 10 sq feet of space, then moved to a cube farm (7x8 ft) which was much better. But I like the idea of random areas with lots of good places to sit and a mix of quiet/collaborative as needed. But if you had both offices and open seating, how many people would like which one?

</comment><comment><USER18> I like my open office floor plan. I think it does a good job fostering communication, and I go crazy when I am isolated in my office ( yes I have both and I like the open part of the office _way_ more ).

Also in all these articles they say "its too noisy..." Has no one ever heard of headphones. They let each individual employee get the best level of noise they need. Want it quiet get a set of sound isolating headphones and put on quiet background noise. Do you like music to work to go ahead and put that on. Do you like the organic background of distant communication and the peaceful methodical hum of the ventilation system. Leave the headphones off and enjoy it. What I have found as the big turning point is you need to have an entire office that is respectful of others no matter what floor plan you have. If one worker insists on playing loud music or having conference calls all day no matter what set up you have chances are he will be annoying his direct neighbors.

As an aside as much as I like my open office I would trade it for a botanical garden. I love gardens and would gladly set up shop in a nice greenhouse maybe with some birds in it, and a nice central waterfall and river running down the middle.

<comment><USER19> > Also in all these articles they say "its too noisy..." Has no one ever heard > of headphones. They let each individual employee get the best level of noise > they need. Want it quiet get a set of sound isolating headphones and put on > quiet background noise. Do you like music to work to go ahead and put that > on. Do you like the organic background of distant communication and the > peaceful methodical hum of the ventilation system. Leave the headphones off > and enjoy it. What I have found as the big turning point is you need to have > an entire office that is respectful of others no matter what floor plan you > have. If one worker insists on playing loud music or having conference calls > all day no matter what set up you have chances are he will be annoying his > direct neighbors.

All of these items are fixed by giving me an office with a door. If I want interaction and noise, then I'll open the door, head out to the shared conference floor space and work from there. I'm not sure why more offices don't do this by default these days.

<comment><USER20> I prefer my brother in law's set up, it's an inverse of what you want, open office but with isolation caves assigned to no one that can be used when you need silence. I think making isolation an opt-in rather than opt-out makes for a much more inclusive environment.

<comment><USER5> Inclusive? I'm not at the office to socialize, I'm there to focus and do work.

<comment><USER20> I think that's a false dichotomy. Without socialization it's very difficult to understand the people you're working with and this, in my experience, is often the root of friction between teams and departments.

<comment><USER19> I've been working from home with my distributed team all over the USA (and world) for over a year now. I haven't met a few of them ever in person yet. Our company and team are doing fine just socializing over Skype and conference calls / Google Hangout (occasionally) when needed.

Someone doesn't need to be sitting right next to me able to interrupt me whenever they like to "promote synergy"

</comment></comment></comment></comment></comment></comment><comment><USER21> How about this? If you don't want to work in an [open office|cube farm|closed office|home office|etc], don't. You're a software developer. You have career options. Then companies can weigh the costs and benefits of these approaches (including financial costs like rent in busy cities like NY and SF, employee- retention issues, ability to call the layout a perk when recruiting, impacts of collaboration or lack thereof) and choose one that seems to fit their needs and the needs of their workforce (which presumably is something that will vary from employer to employer).

Oh sure, it's all well and good to whine about offices not being the way you like. While we're at it, I'd also like shorter work weeks and higher pay.

</comment><comment><USER22> Whether you program or not, sometimes you just need to sit back, maybe close your eyes and think. It's weird to do it in a open office plan where your desk is across someone else! Let's not mention the coughing person a couple desk away or the open-mouth-eater next to you! A cubicle may not be cool and in some offices, it may seem too dull, but in the right conditions, it not only gives privacy, but also a little protection from those little annoyances. The only way I know an open floor plan works is when it is help by a SINGLE and SMALL company, where all desks are against nice tall windows and everyone respects each other enough to not be loud across the room in "their own space".

</comment><comment><USER23> This is why I like the idea of shared private offices / team rooms. It strikes a good balance between the two. I can handle sharing offices, but once it goes beyond a certain amount of people it becomes distracting.

<comment><USER24> It really depends on who you're sharing an office with though. If you get trapped with one loudmouth, it's much worse than being in an open office because they're more likely to be right next to you.

</comment></comment><comment><USER25> Every time I read complaints from people dissatisfied with open office layouts, my first response is inevitably, "Have you ever had a roommate?" The ability to isolate is great when you need it, but I can't help but get the notion that at least some cases are people avoiding learning how to work in close proximity to each other. Thereby using cubicles or offices as a crutch for that. If I'm right, doesn't the company benefit from people having to hash those differences out?

</comment><comment><USER26> Traipsing around botanical gardens with my laptop sounds divine, but what if I need (yes, need) a multi-monitor PC and wired network?

If adjustable sitting/standing desks are all the rage nowadays, how about cubicle walls that raise/lower (all the way to the ceiling)? Scrum time? Lower the walls. Need a bit of quiet contemplation? Shields up!

I'll bet that somebody with better time/money/smarts/connections/looks/luck/location than me could get funding for this invention ;)

<comment><USER27> "Traipsing around botanical gardens with my laptop sounds divine, but what if I need (yes, need) a multi-monitor PC and wired network?"

That's what docking stations are for.

The vertically adjustable cubicle walls sound neat, though.

</comment></comment><comment><USER28> The crappy part of the article is that it provides shallow criticism with no real alternative or solution. There's a link at the bottom to "look" at the most "innovative" workspaces and they're OPEN FLOORPLAN!

So, open floorplan sucks, cubicles suck even more, can't give everyone offices (reduces collaborative work). WTF is one to do?

</comment><comment><USER14> I've heard great things about Valve's system, of desks on wheels and no set floor plan.

Idea: what about movable wall sections to complement this? Whenever you want to go heads-down, move a few partitions to your desk and block everything else out.

</comment><comment><USER29> This open/closed office argument has rapidly become an Android/iOS, emacs/vim sort of pointless opinion-stating circle jerk. Why are we upvoting this crap?

<comment><USER24> No, I think it's pretty much a CVS/SVN "Why are people still using this crap?" type of situation. I don't see much actual debate or argumentation happening around this.

</comment></comment><comment><USER30> No, no death to open plan. Cube farms are desolate places, offices isolate. Open plan fosters good communication. Need solo time? Put your headphones on.

<comment><USER31> Treat your employees like adults and provide options. The best employers provide a mix of all three: open space, offices, and working remotely.

<comment><USER32> This seems the only way to satisfy everyone. There are clearly those who strongly prefer some degree of open office space, and some who strongly prefer private space. However, few of the articles I've read give open+optional private offices as an option.

Currently, some at my company use unused meeting rooms as quiet space but generally prefer an open environment, with headphones on or off. A growing number prefer a casual, social environment like couches and tables rather than desks.

A strong influencing factor is the amount of collaborative work. Those who are working alone on projects tend to be solitary, while those who make many collaborative decisions want to be closer to others. This may reduce the need for formal meetings, which allocate blocks of time for a whole group that may not be necessary or efficient.

</comment></comment></comment><comment><USER33> I like open offices. I grew up with cubicles -- no thank you. I'll take the downsides to open offices to the downsides of being isolated. Just use headphones. And for companies, have rooms where people can go for focused, quiet work when they need that.

I was apprehensive moving from cubicles to open space, but I actually find that I appreciate the social aspect of it. I'd rather not be alone all day in my tiny enclosed corner of the world. There's a good reason movies like Office Space exist.

<comment><USER34> I don't personally care, but the headphone argument doesn't take into account that many people don't like working with music either.

<comment><USER35> Also, not everyone likes wearing headphones all day long either. It's like saying "oh, you don't like the smell of cigaretes? wear a gas mask!"

<comment><USER33> True. But cubicles don't completely eliminate noise either. If you're working in a space with people, some noise is inevitable. Unless you have your own complete closed office, which is just really unrealistic in most cases.

<comment><USER33> @ericd If you're working for a small company, and that company can actually find an office space setup like that, good for you. It will not work for larger companies because the real estate footprint would be unwieldy. I have yet to work at a company for which closed offices for everyone was even remotely possible, given the fiscal and office space availability constraints.

<comment><USER34> The companies I'm describing range from fortune 500s to research organizations to small startups. They didn't have it for 100% of people, but they did for every dev/researcher that I could see. They don't have to be huge offices, and two to an office was fine, it's really the full walls/lack of noise that mattered.</comment></comment></comment></comment></comment></comment>


Show HN: A Browser Extension to Show Network Speed in Bytes on Fast.com - <AUTHOR> https://github.com/satyarohith/fastinbytes <comment><AUTHOR> Chromium-based browsers - [https://chrome.google.com/webstore/detail/obgghgheidhjlkgolp...](https://chrome.google.com/webstore/detail/obgghgheidhjlkgolpabbncemnmpkpac) Firefox - [https://addons.mozilla.org/en- US/firefox/addon/fastinbytes/](https://addons.mozilla.org/en- US/firefox/addon/fastinbytes/)</comment>


Why movies look weird at 48fps but games are better at 60fps - <AUTHOR> http://accidentalscientist.com/2014/12/why-movies-look-weird-at-48fps-and-games-are-better-at-60fps-and-the-uncanny-valley.html

<comment><USER2> This really, really deserved more upvotes. I'm really glad I caught it. This effect bugs the hell out of me on newer TVs. I can't help but be completely distracted by how obvious it is that the set is a set and the actors are just people pretending. This explains the phenomenon better than anything else I've read.

<comment><USER3> > I can't help but be completely distracted by how obvious it is that the set > is a set and the actors are just people pretending.

This really bugs the hell out of me, especially on 120Hz TVs. Granted, sports and newscasts look like you're sitting right there which is neat, but for fictional shows it's unnerving. My sister has one of those TVs, and we occasionally go over there to watch The Walking Dead with them. It's as if the video is being slightly fast-forwarded, paused, then fast-forwarded again, but the dialogue still sounds normal throughout. It really takes us out of the moment and looks more like a documentary of the making of the show, rather than the show itself.

</comment></comment><comment><USER4> The level of detail in this post is insane. I wasn't expecting so much from a blog post.</comment>


Google internal political discussion and an example of related workplace rule - <AUTHOR> https://www.vox.com/2019/10/15/20915729/google-hong-kong-video-game-revolution-of-our-times <comment><AUTHOR> Related to this :

[https://www.theregister.co.uk/2019/10/16/gitlab_employees_ga...](https://www.theregister.co.uk/2019/10/16/gitlab_employees_gagged/)</comment>


Goji: a web microframework for Go - <AUTHOR> https://goji.io/ <comment><USER2> This is going to sound a little dismissive, but I don't mean it to be:

I'm not sure I understand the value that these frameworks offer beyond the HTTP server interface Golang supports out of the box, plus a URL router like "pat" (or whatever the cool kids are using now other than "pat").

I see the clean middleware abstraction, but I find the idiomatic closure-based implementation of middleware adds only a couple extra lines of code, and in return I get total flexibility.

What's this doing that I'm not seeing? I'm sure there's something; I'm writing this comment out of ignorance.

<comment><AUTHOR> Actually, Goji grew out of a single deficiency in "pat": the fact that it does not have a standard way of defining request context.

The big use case here is how you'd write a middleware that did authentication (using API keys, session cookies, ???) and emitted a username for other middleware to consume. With net/http, you end up with a lot of coupling: your end handler needs to know about every layer of middleware above it, and you start losing a lot of the benefit of having middleware in the first place. With an explicit middleware stack and a universal interface for middleware contexts, this is easy: everyone can code to the same single context object, and instead of standardizing on weird bound variables (or a global locked map a la gorilla), you just need to standardize on a single string key and a type.

I think my ideal world would involve Go providing a map[string]interface{} as part of the http.Request struct in order to implement this behavior, but until we get that, I think Goji's web.C ("the context object") is the next best thing.

There's one other thing pat hacks around: the issue of how to pass bound URL variables to the resulting handler. At first I was a little grossed out at how pat did it, but I've sort of come to terms with it. I still think Goji's way is better, but I don't think it's the reason I wrote (or a reason to use) Goji.

<comment><USER3> I agree that 'context' and passing this context between middleware is the biggest missing item from the standard http+"pat".

This library (Goji) solves it with a map[string]interface{}. This works, but has the downside that you need to type-assert your values from this map if you want to use them.

I wrote a library a while ago ([http://www.github.com/gocraft/web](http://www.github.com/gocraft/web)) that uses reflection so that your contexts can be strongly typed. Martini offers a similar approach.

<comment><AUTHOR> Yeah, I looked at gocraft/web for a long time before writing Goji. It's a good library, and I think it does a lot of things right. But at the end of the day, just like my disagreement with Martini, this comes down to a difference in principles. I think there are two theories of Go library design here. One of us has chosen one, one the other, and let me prefix this by saying that I'm not sure the way I chose is correct.

On one hand, a library can allow its users access to typed structs—this is the way gocraft/web does it. The upside here is huge: applications are now type- safe, and can lean on the compiler to enforce their own correctness. The library code that backs this behavior can be arbitrarily gnarly, but you only have to write it once, after which it'll Probably Be Correct (tm).

On the other hand, you can provide a standard "universal" context type like Goji does. Since the library makes no statements about the sorts of fields you have or their types, applications have to do a bunch of shenanigans with type casts in order to extract the typed values they expect. The upside here, however, is the standardization and what that allows. I've found that in most of the web applications I've built, only a small fraction of the middleware is authored by me: I end up pulling in the standard request logging middleware, or the middleware that assigns request IDs, or the middleware that does CSRF protection. There's a huge amount of value in being able to write (read: "let someone else write") those middlewares _once_ and being able to use them in any application. With arbitrary application-provided structs, this isn't possible [1], but if you can instead standardize on types and keys out-of- band, you can mix and match arbitrary middleware with impunity.

This alone is one of those awkward engineering tradeoffs where you're exchanging one nice thing (application-level type safety) for another (standard interface, and hopefully an ecosystem of drop-in middleware), and given just this I'd argue that the ecosystem is probably more important. But the frosting on this particular cake is that, with only marginally more awkwardness, you can actually get application-level type safety too: look at what I did for the request ID middleware that ships with Goji, for instance ([https://github.com/zenazn/goji/blob/master/web/middleware/re...](https://github.com/zenazn/goji/blob/master/web/middleware/request_id.go#L55-L69)). Again, it's not quite as nice as a user-defined struct, but in the grand scheme of tradeoffs and sacrifices, I don't think it's so terribly bad.

So TL;DR, Goji chose standardization and composability in the design of its context object over application type safety, but if you're willing to write a few helper functions I don't think you end up losing the type safety anyways.

[1]: actually, come to think of it, with a lot of reflection and some struct tags you could probably pull it off. (Something like "struct { RequestID string `context:"reqid"`}", and the request ID assigning middleware would hunt around for a string field tagged "reqid"). But it seems like it'd introduce a lot of coupling between middleware and the application which I'm not sure will turn out well—any time a middleware needed a bit of extra storage, every application would have to change. In any event, if someone builds this I'd love to play around with it.

</comment></comment></comment></comment><comment><USER4> It is good to have many web microframeworks in a language ecosystem. In Python probably there are a hundred of those. Many of those are not picked up by the community –a natural selection. Only really a few of those survived. It all depends on you if you are going to choose Revel, Martini, Goji or whatever you want. Today, thousands of apps run on web.py, yet most of the source code is untouched last 3-5 years ([https://github.com/webpy/webpy/tree/master/web](https://github.com/webpy/webpy/tree/master/web)) It's impressive it just works!

Personally, I am looking for frameworks that many people rely on, maintained frequently as needed and works just fine. There could be a +-10% difference on QPS those framework URL routers can handle and render a 'hello world' page.

So this is a nice attempt I would say, looks cleaner than Martini, still supports middlewares. On the other hand, Martini has support to serve static files, logging, panic recovery, which are also good and has a bigger fanboy community around it: [https://github.com/go- martini/martini](https://github.com/go-martini/martini)

<comment><AUTHOR> Static files: [http://golang.org/pkg/net/http/#FileServer](http://golang.org/pkg/net/http/#FileServer)

Logging: [http://golang.org/pkg/log/](http://golang.org/pkg/log/)

Panic recovery: [https://godoc.org/github.com/zenazn/goji/web/middleware#Reco...](https://godoc.org/github.com/zenazn/goji/web/middleware#Recoverer)

Not as many fanboys though (yet!)

</comment></comment><comment><USER5> Perhaps this is not the best place for this question, but as a frequent HN reader, I'm constantly told that Go is great to develop in and very performant. However, it's not clear to me how Go suits a web application with relational data.

From what I've gleaned, an ORM does not make sense in Go, so how would this type of application be approached? Writing a lot of ORM-type boiler-plate? A completely different way? Or is Golang a bad choice for such an application?

<comment><USER6> I suppose the Go philosophy would discourage developers from tightly coupling Go types and their relational representations through an ORM. The idiomatic way of mapping your objects to a database is by defining a thin interface around a sql.DB, with first-class operations for your concrete types.

        type User struct {
            ID        int
            Permalink string
        }

        type Storage sql.DB

Then you can either do

        func (s *Storage) WriteUser(user User) error {
            if _, err := s.Exec(
                "REPLACE INTO users VALUES (?, ?)",
                user.ID,
                user.Permalink,
            ); err != nil {
                return fmt.Errorf("write user failed: %s", err)
            }
            return nil
        }

or

        func (u User) Write(storage Storage) error {
            if _, err := storage.Exec(
                "REPLACE INTO users VALUES (?, ?)",
                u.ID,
                u.Permalink,
            ); err != nil {
                return fmt.Errorf("write user failed: %s", err)
            }
            return nil
        }

It's a bit more laborious in the sense of keystrokes, but it's also more explicit, which is, on balance and over the lifetime of a large software project, a good thing.

<comment><USER5> Thank you for the detailed answer.

 _It 's a bit more laborious in the sense of keystrokes, but it's also more explicit, which is, on balance and over the lifetime of a large software project, a good thing._

OTOH, it feels like a high development cost to pay, relative to the alternatives out there. You certainly wouldn't use it for an MVP. Also, it would increase the opportunities to introduce bugs into the application (unless one actually feels they can write something better than, say ActiveRecord, from scratch).

I have a side-project in RoR that would benefit greatly from the performance boost provided by Go. However, the idea of writing all of the ORM functionality that ActiveRecord handles for me (not just managing of objects as you've shown, but the relationships between them) is quite daunting.

<comment><USER6>
        > (unless one actually feels they can write something
        > better than, say ActiveRecord, from scratch).

The point is that ActiveRecord's method of modeling, especially when it comes to dynamically mapping language constructs to a storage layer via SQL, is too implicit. Too costly. Actively harmful! The point is to get developers to stop thinking in terms of ORM abstractions, and start thinking in terms of the actual transforms and manipulations that are occurring.

        > You certainly wouldn't use it for an MVP.

I think you overestimate the cost of pressing buttons on your keyboard.

<comment><USER5> _The point is to get developers to stop thinking in terms of ORM abstractions, and start thinking in terms of the actual transforms and manipulations that are occurring._

Isn't the point also to encourage code re-use and abstraction? I mean, golang has packages for a reason. I suppose my question is more along the lines of whether we'll ever see a package in go that would standardize the data-object divide or whether this will always be a "roll your own" domain?

 _I think you overestimate the cost of pressing buttons on your keyboard._

Then why don't we "roll your own" for everything?

<comment><USER6>
        > I suppose my question is more along the lines of whether
        > we'll ever see a package in go that would standardize
        > the data-object divide

There is no lucid standardization possible for the data-object divide, in Go or any other language. Too much depends on the semantics of the object and data system. Or, rather said: that standardization (that abstraction) is SQL itself.

</comment></comment></comment></comment></comment></comment><comment><USER7> Looks very nice. I do appreciate having more clean, well thought out web frameworks in the Go space. Type switches for your handlers is a good way to approach the net/http compatibility.

There are some people that find that Martini is a bit too magical for them, and that is completely okay. It's great to see another minimal framework that will suit their needs.

</comment><comment><USER8> About this part in README:

> I have very little interest in boosting Goji's router's benchmark scores. > There is an obvious solution here--radix trees--and maybe if I get bored > I'll implement one for Goji, but I think the API guarantees and conceptual > simplicity Goji provides are more important (all routes are attempted, one > after another, until a matching route is found). Even if I choose to > optimize Goji's router, Goji's routing semantics will not change.

Maybe you can just use HttpRouter without reimplementing it yourself?

[https://github.com/julienschmidt/httprouter](https://github.com/julienschmidt/httprouter)

> The router is optimized for best performance and a small memory footprint. > It scales well even with very long pathes and a large number of routes. A > compressing dynamic trie (radix tree) structure is used for efficient > matching.

        goji.Get("/hello/:name", hello)

        router := httprouter.New()
        router.GET("/hello/:name", Hello)

<comment><AUTHOR> Huh. I'm not sure I saw that particular project during my search. It looks neat!

Without having looked in detail at httprouter, I think the most obvious reason it might not be sufficient is that it doesn't support regular expressions. This might not be a dealbreaker, but I'm fond of the occasional regex route, and I'd have to think long and hard about whether it's worth giving up for a faster router. And plus, I'm still not sure router speed actually matters for most applications.

In any event, I _do_ have a long plane trip coming up, and I'm sure Goji will grow itself something at least slightly more efficient than a linear scan then. I'm think Goji's router and middleware stack are already zero- allocation, so it'll just be finding a way to binary search through routes.

<comment><USER8> You can find benchmarks in pull request for [https://github.com/cypriss/golang-mux- benchmark](https://github.com/cypriss/golang-mux-benchmark):

[https://github.com/cypriss/golang-mux- benchmark/pull/5](https://github.com/cypriss/golang-mux-benchmark/pull/5) (the last one)

</comment></comment></comment><comment><USER9> If you think you're ready, it might be fun to submit a techempower benchmark.

[http://www.techempower.com/benchmarks/](http://www.techempower.com/benchmarks/)

</comment><comment><USER10> I see why some would call Martini "magic" but I'm not entirely sure I prefer having to deal with a giant `map[string]inteface{}`. What you are really doing is moving the "magic" from the framework and onto the developer (I now have to do type checking and casting).

That said I'm a huge fan of Martini and I actually use codegangsta's Inject in my other projects to manage shared state/resources, so I am heavily partial to it.

<comment><AUTHOR> Yeah, this is a good question, and the unfortunate answer is that it was an engineering tradeoff. I wrote a pretty long reply to cypriss ([https://news.ycombinator.com/item?id=7632956](https://news.ycombinator.com/item?id=7632956)) which I think covers this.

</comment></comment><comment><USER11> Great job! It definitely feels like a microframework compared to others. I'm glad that there are so many starting points for Go web services available now of varying levels complexity. To me this feels like it fills the void between Gorilla and Revel/Martini/Beego. Also, the code is very well documented and easy to follow.

</comment><comment><USER12> Every time I see a web framework for Go, I just want to see an example or two of a website developed with it. Does anybody have any solid examples?

Hopefully, like me, some others enjoy exploring existing code as well as reading the examples / docs.

<comment><USER7> I can't speak for Goji. But one popular open source Martini app is Gogs [https://github.com/gogits/gogs](https://github.com/gogits/gogs)

</comment></comment><comment><USER13> Call it nitpicking, but I would rather not export a symbol 'C' from a library I write. Seriously, is 'Context' that hard to type? And the author seems to be far from lazy, the codebase is nicely and extensively commented (it is a nice read indeed). Apart from this issue, the library seems quite nice.

</comment><comment><USER14> Do we yet have the Django for Go? Goji is as said a microframework, what I want is an equivalent of Django.

<comment><USER15> I doubt this will be produced.

Microframeworks written in Go are more likely to replace Django Rest Framework [1] than Django itself

[1] [http://www.django-rest-framework.org/](http://www.django-rest- framework.org/)

</comment></comment><comment><USER16> A bit off-topic, but I love the site colors. I'd love that text theme for Atom if it's available?

<comment><USER17> Orange (keywords): #fd971f

Red (strings): #f24840

Green (variables): #96c22e

White (text): #ffffff

Grey (background): #222222

</comment></comment><comment><USER18> When I read things like "func Get(pattern interface{}, handler interface{})" I start questioning why this whole thing is ever written in Go at all? This kind of ditches half of Go's benefits by moving all type checks to run time.

<comment><AUTHOR> If Go supported method overloading, you could actually write the types of those functions out. The first one is either a string or a regexp.Regexp, and the second one is one of four variations on an http.Handler, giving a total of 8 varieties of each function.

I decided that the sin of exposing an interface{} as a parameter was less egregious than the sin of multiplying Goji's API surface area by a factor of 8, but you'll be happy to know that passing a value of the wrong type causes the invocation of Get (Post, etc.) to fatally exit immediately. If you're defining all your routes in a single goroutine before calling goji.Serve() (which is probably the most common way to define routes), your application will crash before it even binds to the socket.

So, not quite as good as a guarantee enforced at compile time, but it'll have to do.

<comment><USER18> Don't get me wrong, I perfectly understand what interface{} is and why someone is tempted to use it. It just kills the type checking and converts your golang code to a compiled python, without all those nice static analysis things. Yes, API would be larger, but it would be compile time type-checked and you wouldn't depend on things like "well, it will most probably crash very early enough".

</comment></comment></comment><comment><USER19> Just a quick look at the examples, and I can say that it reminds me very much of Clojure Ring. It provides really small and extensible core, and if the community pick it up and start writing useful middlewares, it can become very useful.

</comment><comment><USER20> Were you inspired by this ? [http://www.cherrypy.org/](http://www.cherrypy.org/) The juxtaposition of things and colors and code simplicity looks like so

</comment><comment><USER21> How does this compare with Martini?

[http://martini.codegangsta.io/](http://martini.codegangsta.io/)

</comment><comment><USER22> I ain't switching from Perl 5.8 to golang until a shared hosting provider becomes available.

<comment><USER23> Google App Engine

</comment></comment><comment><USER24> Sorry for my ignorance, how is this different, better than Martini? What is the main goal of creating a new framework (instead of getting the features you are missing implemented in the currently existing ones)?

<comment><AUTHOR> To be perfectly honest, I'm not sure it is better (I was hoping you would help me decide that!), and I wrote it mostly because every aspiring programmer writes a web framework at some point, and it was time I wrote mine (it was a lot of fun :) ).

But I think there's a good chance it is better.

First, I think one important difference is that Goji isn't full of magical reflection. If Go had support for method overloading, its entire interface is type-safe. In contrast, Martini does a lot of magical object injection, and it's not clear until runtime if your routes will even work, or what they'll even do, or where exactly the memory for them is coming from.

Second, I much prefer Goji's way of defining middleware. To me, middleware is like an onion (just like ogres!): each layer is a wrapper around the previous one. The way you write middleware in net/http is by wrapping the old http.Handler with a new one, and that's how I wanted Goji's middleware to work too. There's no magic "context.Next()", there's no magic dependency injection overrides, it's just http.Handlers all the way down.

Anyways, I'd like to know if you think I'm right: again, I'm really not sure this is actually better than Martini (or $YOUR_FAVORITE_FRAMEWORK), but I think it comes from a slightly different set of principles, and ones that I think are worth considering.

<comment><USER25> The fact that you're trying to stick with strong typing as much as possible is very appealing to me, and now I'm convinced to try to port my new project from Martini to Goji.

I think this is a good point to differentiate yourself on, and perhaps it should be included in your elevator pitch.</comment></comment></comment>


Android Diary I - <AUTHOR> http://www.tbray.org/ongoing/When/200x/2008/12/18/Android-Diary <comment><USER2> I'd be rather curious to see how advanced the Android hackers have gotten. Something I'd really like to see would be the ability to run apps outside the standard Android framework/jail, so that those with a dev phone straight from Google could, for instance, run an SSH server, or run X11 apps written in C/Python/etc.

I have a Freerunner, and I like being able to write my apps in Python/Gtk, and while the Android port for it looks really interesting (once Android finally gets a real on-screen keyboard that is), I don't want to give up my ability to write in a language that isn't Java.

Edit: and the Dev1 case looks a lot better than the standard G1, too. ;)

<comment><USER3> There's a pretty good write-up of installing a Debian userland on the G1, but the link seems dead today. Maybe this will work later: <http://www.saurik.com/id/10>

In the meantime, here's the Google Cache version: [http://126.96.36.199/search?q=cache:qTYLZQyrPnQJ:www.saurik....](http://22.214.171.124/search?q=cache:qTYLZQyrPnQJ:www.saurik.com/id/10&hl=en&client=firefox-a&gl=us&strip=0)

<comment><USER2> Unfortunately, the steps involved in that process seem an exercise in insanity. I'm thinking more along the lines of: the developer phone has an unlocked bootloader, therefore I would like to be able to flash an Android image that already has a userland set up and working. :P

</comment></comment></comment><comment><USER4> Java/Android certainly does not have the benefit of Xcode's slick Interface Builder, definitely a lot less inspiring when it comes to creating an app with a beautiful UI

<comment><USER5> You may be amused or appalled to hear that we mock up an Android UI (plus some initial functionality) in Visual Studio and VB.NET before running the result through a translator to get the equivalent Android Java code in an automatically initialised project. A great way to jump-start an application.</comment></comment>


Wikileaks Refused to Publish Manafort Family Texts, So Someone Else Did - <AUTHOR> https://www.techdirt.com/articles/20180723/23541840296/wikileaks-refused-to-publish-manafort-family-texts-so-someone-else-did.shtml <comment><USER2> Well it's pretty obvious that Wikileaks is compromised. Question is who will step into the breach. Because whilst the dry still might release valuable information it will now be tainted with "but what did they leave out". I guess that's a universal journalistic issue (I'm not here to debate if leaking raw documents is journalism). Who do you trust?

<comment><USER3> To me, Wikileaks is known to be compromised as soon as they have leaked the Hillary email. The timing is too nicely paired and the only actor benefitting is Russia.

But as with all propaganda, critical viewing and multiple sources are the tools to defeat them.

<comment><USER4> There's an explanation that makes a lot more sense:

Wikileaks leaked the emails of the campaign manager for a candidate that's official stance on Assange was "can't we just drone the guy?"

Or, perhaps, as you seem to suggest, everyone is a Russian agent

<comment><USER5> Either way (whether Wikileaks is compromised, or they have arguably justifiable preferences for/against specific candidates), I think they should no longer be trusted as they have obviously showed bias.

<comment><USER4> I don't think Wikileaks has ever claimed that they publish everything they receive.

Maybe they couldn't verify it's legitimacy.

Maybe they didn't find the texts of someone's daughter particularly interesting.

Maybe they haven't finished reviewing them yet.

Let's not throw the baby out with the bath water just yet.

</comment></comment></comment></comment></comment><comment><USER6> > [W]hen faced with an opportunity to post the equivalent of the Podesta > emails on the Trump side, it appears that Assange decided not to do it.

The Podesta emails were from the personal email account of John Podesta. Many of the emails were used for professional purposes. By contrast, the emails here are from Paul Manafort's daughter's account. In the first, Podesta is both the owner of the email account and the subject of interest. In the second, the subject of interest is not the owner of the email account. The two cases are not "equivalent."

</comment><comment><USER7> Just a hunch - I feel as though Wikileaks is hanging on to any dirt they have on the Republicans and Trump's clique so they can use it as leverage should Julian Assange be taken into custody in the UK. All it would take is a taste of what they have to make the administration apply pressure on the DoJ to halt deportation.

<comment><USER8> Based on what?

<comment><USER7> First, it's a hunch so there is nothing to support its veracity. But Assange has never been caught without a backup plan and he'd be stupid to put all of his cards on the table.

<comment><USER8> What would not being caught with a backup plan look like to you?

Would getting stuck in an embassy qualify?

<comment><USER7> Not sure whether you're saying being stuck in an embassy meant he lacked a backup plan or whether it was one, but I'd side with him being able to stay in the embassy as a backup plan considering WL suspected there was a secret grand jury indictment against Assange.</comment></comment></comment></comment></comment>


Xtranormal -- Text-to-Movie "If you can type, you can make movies" - <AUTHOR> http://www.xtranormal.com/

<comment><USER2> Let me tell you as an user of this software that this is fantastic! The work that has gone into this product is incredible! Look what I built for my app <http://www.creatorfinder.com/> (You can see the video towards the bottom).

You can control stuff like the camera location, gestures, facial expression all by clicking and typing. Really cool!

</comment><comment><USER3> Howard and Leslie is a great example of what can be done in this exciting new medium.

<http://howardandleslie.blogspot.com/> (slightly NSFW dialogue)

<comment><USER4> Wow. If that's your ideas of _slightly_ NSFW, I'd like to see the really dangerous stuff.

</comment></comment><comment><USER5> you know it was kind of nice back when artistic endeavors took major investments of time and money. now I don't have any excuse for not being creative >_>

</comment><comment><USER6> Why do they always have to have politics in these sorts of things? I don't care if you're an Obama supporter or a Palin supporter, but putting stuff like that in a site like this just ticks 50% of the people on your website, because you're making fun of "their man" or "their woman".

Unless that's what they're trying to do.

</comment><comment><USER7> Reminds me of that _old_ game from MS: <http://en.wikipedia.org/wiki/3D_Movie_Maker>

Couldn't do text-to-speech, but you could record voices.

</comment><comment><USER8> What a sweet web app! Awesome monetisation options to do with extra props / characters / extra long movies.

Sweet.

</comment><comment><USER9> I doubt you read this site Luc Tremblay, but congrats!

</comment><comment><USER10> Definitely look for the ones based on Dinosaur Comics.

</comment><comment><USER11> It's mainly dialog and speech based movie

<comment><USER12> I think that this covers majority of funny movies.

</comment></comment><comment><USER13> 'Looks like a neat idea. Mac? Linux?</comment>


Bruce Schneier: The Eternal Value of Privacy - <AUTHOR> http://www.schneier.com/essay-114.html <comment><USER2> Those who think they have nothing to hide now can't always see into the future.

<http://www.stockmaven.com/ibmstory.htm>

"This approach appears to have been used in the Netherlands when small-area tabulations of population data by religion from the 1930 Dutch census made up one of several data sources used in the development of the so-called 'dot maps' of Amsterdam," he says. "These maps, which showed areas of the city where the density of the Jewish population was the highest, were used in planning Nazi-inspired attacks on some of these neighborhoods in February 1941."</comment>


To Apply Machine Learning Responsibly, We Use It in Moderation – NYT Open - <AUTHOR> https://open.nytimes.com/to-apply-machine-learning-responsibly-we-use-it-in-moderation-d001f49e0644 <comment><AUTHOR> > For years, people on the Community Desk manually moderated all comments that > were submitted to Times articles.

Huh I had wondered for a while why NYT comments were only open on some articles.</comment>


Google barge revealed: Artistic structure with 'fish fins' - <AUTHOR> http://blog.sfgate.com/matierandross/2013/11/07/google-barge-revealed-artistic-structure-with-fish-fins/

<comment><USER2> The entire "what is that barge in the bay?" story line seems like a carefully constructed PR campaign, and I for one am opting out.

</comment><comment><USER3> Well that's somewhat underwhelming.

<comment><USER4> That seems to be a trend with google now. I know that whenever I see something big and secretive happening around google I keep thinking "Is this when they are going to reveal their AGI?!" and it just turns out to be a party barge or google hangouts.

</comment></comment><comment><USER5> I wonder if this is the realization of idea for creating more space in extra ordinarily expensive SF bay area. This barge can very easily serve as new office/commercial space in the city where every inch counts. A barge of this size can accommodate roughly 500 office workers as per my rough quick and dirty back of the envelop calculation. The cost of the "land" is simply the cost of the barge which is made using recycled shipping containers. I bet this would be magnitude cheaper than cost of actual land in same nearby area.

<comment><USER6> Good luck with that environmental impact study.

</comment></comment><comment><USER7> Any moment I expect to hear that Google has hired Henry Winkler to jump this thing in a leather coat.

</comment><comment><USER8> >Exactly who is By and Large? That's a little unclear, but it's reported to be firmly connected to Google. Some have noted that it looks like a play on the word "barge."

I thought it was a play on the company from Wall-E?

<comment><USER9> Since they've apparently got sails on this thing it's even more likely to be this:

[http://en.wiktionary.org/wiki/by_and_large#Etymology](http://en.wiktionary.org/wiki/by_and_large#Etymology)

<comment><USER8> The company in Wall-E is a play on this, and then the barge company was a play on the Wall-E company. Or so I thought.

</comment></comment></comment><comment><USER10> Do I have join Google+ to get on the barge?

</comment><comment><USER11> Does this read like a red herring to anyone else? Some of the quoted text sounds like a google-joke:

> "The artistic structure combines innovative architecture with a bit of > nautical whimsy," says the proposal, "creating a surprising environment that > inspires conversation, community and 'a-ha' moments."

</comment><comment><USER12> This is why I try and avoid speculation. A portable Google store? What a let down. Here I was imagining massive portable data centres.

<comment><USER13> A data center can be placed on commodity cheap land. A store has to be close to where people want to be, so on expensive real estate. The floating solution is very elegant, I hope they pull it off. Looking forward to floating boutiques all along the Seine.

</comment></comment><comment><USER14> Awesome! Wait for the ads there!</comment>