An introduction to NoFlo and flow-based programming - <AUTHOR> http://rawkes.com/articles/an-introduction-to-noflo-and-flow-based-programming

<comment><USER2> I think the _types_ of problems each programmer tends to confront make up a large part of the disconnect between the ones who "drink the koolaid" of a programming paradigm or process and the people who are unimpressed.

I typically do games. Games are useful to think, in part, in OOP terms; there is very little "data flow" and (typically) there are a lot of objects that need to interact in asynchronous ways. There are specific parts that could be reduced to flow based programming (FBP), but it seems like the wrong paradigm for writing an _entire_ game, as an example close to me. I could imagine FBP certain AI logic state machines being useful, for instance.

But other areas of development have flow-style tools and that's clearly the _right_ way to write code. Signal processing is an obvious example: When you want to send an audio signal through a reverb, split it to two different types of processors and re-merge it later with a fader tied to an external control, laying those components out with boxes is 1000x better than trying to write code in text to do the same thing.

So I think articles like this are useful to introduce everyone to various other paradigms, in case they might be applicable to a problem that you're working on. But it's never a "silver bullet" simply because each tool works best in certain problem domains. A friend tried to convince me that I should drink the functional programming koolaid, for instance, but on drilling deeper I found that he had never worked on a game, and didn't really grok what _kinds_ of code you would normally end up putting in a game.

What I'd _love_ to see is an article that compared a lot of different processes and paradigms and talked about _where_ each is most useful. Especially if that article surprised me. :)

Problem is that most people are familiar with their own areas and toolsets; we'd probably need a team of people from different specialties to collaborate on it. Maybe a wiki where people could add in their experiences trying each approach or tool on their own problem domains? Then when someone _does_ use functional programming to write a game, they could talk about the pros and cons and the kind of game they wrote. And _that_ would be useful data.

<comment><USER3> > I think the types of problems each programmer tends to confront make up a > large part of the disconnect between the ones who "drink the koolaid" of a > programming paradigm or process and the people who are unimpressed.

Sounds a lot like selection bias.

Even in games there is a growing movement to stop the OOP paradigm and focus on data-oriented or "data flow".

It's hard to know whether there are universally-applicable truths in a given paradigm... but it's rather nice to have the option. Flow-based tools are becoming popular in procedural content generation, have been in audio/music/dsp for some time, and are probably quite useful in other sort of "pipeline" analogies.

But certainly past experience will play a large role in determining which tools you think are applicable to a given problem.

<comment><USER2> Selection bias implies that people just like what they're familiar with; I'm sure that does keep people from switching to new tools, but that's not what I'm referring to.

I'm talking about people who actually try new tools and reject them as useful to their problem domains; I try a lot of new tools, and some fit with the problems I solve better than others.

I didn't quite remember what "data oriented meant," but a quick search brought up this article [1] on using data-driven design in games. It brings up some important points, but really the point of data flow design seems to be optimization and efficiency rather than making it easier to write the code.

And I'm all for that, and I thank you for making me look that up since I will likely use that idea, but I'll keep my text editor for most of the game development I do going forward. Except maybe the particle systems, which as the article below points out, are typically already data oriented, and have had good data-flow-based tools for years. ;)

[1] [http://gamesfromwithin.com/data-oriented- design](http://gamesfromwithin.com/data-oriented-design)

<comment><USER3> I think you're referring to _confirmation bias_ which is related but not quite the same. Never the less I understand what you're getting at. I took it to mean that the kinds of problems a programmer encounters is not sufficiently random to properly determine the usefulness of a given language or "paradigm." Which may keep people from switching to new, better tools (or from trying them out at all).

The data-oriented design thing is interesting and something I've heard come and go over the years. Check out:

[http://www.dataorienteddesign.com/dodmain/](http://www.dataorienteddesign.com/dodmain/)

and

[https://www.youtube.com/watch?v=ZHqFrNyLlpA](https://www.youtube.com/watch?v=ZHqFrNyLlpA)

A video demonstration of Jai; a new compiler from Jonathan Blow designed around this paradigm.

Cheers!

<comment><USER2> Ahh, I see. Yeah, I guess I meant that each paradigm _is inherently_ a different value for different domains, and so therefore has a different value for programmers who specialize in their domain(s). And when someone checks out a paradigm that doesn't match their domain, they may end up complaining about how the tool isn't good.

Love the links. Listening to Blow talk now. Will look at the longer book later.

</comment></comment></comment></comment></comment><comment><USER4> Visual programming has interested me for a long time because it is amazing how many times it has been tried and the result been unusable.

If I had the time and resources, I'd very much like to try developing a visual programming system that uses a user interface change that I think would greatly change things. Basically, using the middle mouse wheel to zoom in/out. If you've played the Supreme Commander game series, you will have the best idea of what I mean. As it is, you end up with absolutely everything trying to be jammed in to one gigantic flat space. That's terrible. Better would be zooming in on a 'box' (function, data structure, etc) and having its guts become the new 'context' you're working in. So far as I have been able to find, this has never been attempted.

<comment><USER5> A ZUI would be great for working with the FP paradigm, but good UIs are hard to make; I think it's one of the way that NoFlow is gaining the traction it is, the UI is pretty good.

<comment><AUTHOR> The GUI with NoFlo is one of the major reasons why this is different to the previous attempts at FBP (that and it's JavaScript). What I also love is that it's all released under MIT: [https://github.com/noflo](https://github.com/noflo)

</comment></comment></comment><comment><USER6> "Flow" programming is great for non programmers.I used it to develop VST plugins on pc and when I was working in video editing(compositing).

However,I wouldn't use that to develop web apps.It's often quicker to actually write the code,since app development is rarely just about piping stuff.

<comment><AUTHOR> You'd be surprised how quickly you get used to the idea of connecting components together with pipes, even for Web applications. The additional benefit being that once you've written a component once you don't need to rewrite it ever again, or even copy and paste the code.

The NoFlo site has some good examples: [http://noflojs.org/example/](http://noflojs.org/example/)

<comment><USER7> In particular, I think the growth of Docker/containerization and microservices will make this an area worth keeping an eye on.

I'm not convinced it makes sense at a micro level, but at the macro level of connecting larger services that have been coded in the traditional way.. there's something very appealing about the idea, especially in the way it could interact with ops' concerns too (like having a visual representation of scaling or moving services between machines).

<comment><USER8> [https://jujucharms.com/](https://jujucharms.com/) sounds pretty much like what you are describing

<comment><USER7> That screenshot does look somewhat like what I pictured in my head. It's a small world nowadays :)

</comment></comment></comment></comment></comment><comment><USER9> One of the main issues with Flow-Based Programming is often while it optimizes generally in the opposite direction of Functional-Programming (Currying vs. Topological Sorting I.E.: Execute when needed vs. Execute as soon as possible) is that you tend to lose control of your software.

While I'm not suggesting a purely imperative approach, to a certain degree gains can be had in an imperative context over either Functional/Flow Programming via in depth knowledge of OS/Hardware processes and are generally NOT developer fast/friendly.

From a personal stand point Flow-Based Programming is amazing for concurrency since it lets you see where you can use threads/processes. As which sections of code are less _attached_ to the rest.

<comment><USER5> > it optimizes generally in the opposite direction of Functional-Programming

Really? It strikes me as exactly the opposite - the output of each discrete component is solely dependent on its input. There is no shared state between components, only what is passed between them.

<comment><USER9> Your addressing the platform specific solution not the global problem I am.

If you build a compiler that uses Flow Based programming without dipping into imperative blocks (very possible, and they exist). You get the problems I describe.

<comment><USER5> Flow based programming actually has quite a bit in common with purely functional programming, in that the individual processes are generally stateless; their output depends solely on their input. The fact that the individual processes might be written in an imperative language simply doesn't matter at this point, so long as they always produce the same output for any given input.

This is not true for all FBP processes obviously - there exist processes which read to and write from file handles, but much like the popular FP languages, these "tainted" processes are contained and separated from the remaining portions of the program via their connections to other processes.

Most of the non-functional "programming" in Flow based programs actually occurs at the level where you are composing the connections between processes (be that the GUI, or in a hand-written method which makes the connections).

One additional benefit of FBP which it shares with traditional FP programs is that they are capable of exhibiting "lazy" execution properties through the application of back pressure - by limiting how many packets of data a given connection can contain, you create a program which consumes only enough content from any given generator to keep the entire system occupied.

Now then, can these abstractions break down if you start digging into the implementations of individual processes? Of course. It's not turtles all the way down, after all. Then again, all FP languages which run on Von Neumann based architectures suffer this problem. They all get compiled down to imperative machine code, and are all subject to the same state interruptions as every other program.

<comment><USER9> >Most of the non-functional "programming" in Flow based programs actually occurs at the level where you are composing the connections between processes (be that the GUI, or in a hand-written method which makes the connections).

This is false, there are entire languages that do line by line compilation via Flow Based Programming.

>One additional benefit of FBP which it shares with traditional FP programs is that they are capable of exhibiting "lazy" execution

This is blatantly wrong and the main thing I was trying to express.

Flow Based Programming is based off a topographical analysis of the code body. Which is a analysis of object dependencies (where a dependency is code that must be executed before the object (code) in question can be run).

Meaning objects are executed in the order in which they have the least number of dependencies. This means one off sections of code, that do not necessarily need input from external, or exist independent of otherwise coherent processes he processed first.

For example (Example 1)

            Path p = resolve_path();
            if(something_happened())
            {
                   FP fp = open(p, "rw);
                   write(fp, "logloglog");
                   close(fp);
            }

This above code is bad, because path P is always resolved, regardless of error state.

A Currying analysis (lazy evaluation) will change this flow via lazy evaluation (or should ideally):

(example 2)

            if(something_happened())
            {
                   Path p = resolve_path();
                   FP fp = open(p, "rw);
                   write(fp, "logloglog");
                   close(fp);
            }

A topological flow based compiler will prefer path P be constant. It will prefer example 1, to example 2 every time (if its given example 1 as base code). And likely ensure that path P is solved for absolutely first in a function AS resolve_path() depends on no code being executed before it.

This is the flaw in most flow based programs. Is they don't care errors like this, and when the errors do exit, they _optimize_ for them to be more apparent.

:.:.:

This is what I mean by a topographical analysis being the opposite of a Currying analysis. They group code differently, in almost an opposite fashion. Curry trys to group code by purpose. Topological by how much code has to be executed before it.

Both have their uses.

</comment></comment></comment></comment></comment><comment><USER10> I have to admit, haven't read all comments... but:

FBP, is nothing new ... In the enterprise world ( the world that I`m working in ) it is used quite frequently, and there are some products that live of this for example ( windows workflow foundation, workflowgen ) for the c# world.

FBP is a way of doing things, let's say it is a pattern, like MVC ( less than more, but bare with me ) they each serve a specific task. To be a tad more clear, let's say that MVC is more of a generic on size fits all approach while the FBP is more of a custom tailored way of doing things for (highly) specific things. So, MVC would be great for a wiki and FBP would be great for the whole wiki item management from treating the adding, verifing, drafts, plublish etc etc. ( I`m sure there are a lot of better examples out there ...)

The benefit of NoFlo is that they actually made something that is usually regarded as "enterprise-ish crap" and made it appealing. And again they did a great job with the visual editor, but from my own experience with visual flow design ... sooner that later you will be back to the code ( we you need to stray a bit from the Happy Path ).

Also, I think that frameworks like NoFlow are a very good step for Node in general showing that a lot more interesting things could be done apart for the tasteless and colorless REST apis.

</comment><comment><USER11> Things like this are great for data-processing pipelines with an enumerated set of common operations, most of which input and output the same data type. For instance, any data that can be redefined as a matrix, since a lot of matrix operations result in another matrix (or a vector, which is just a restricted matrix).

However, I can't help but feel that applying it any more generally is a hammer->nail issue. Or, at least, completely invalidates the idea of "massively reusable components". To make components more reusable, they have to make fewer assumptions about their input. To make fewer assumptions, they will typically need to do less. The reductio ad absurdum argument is that you end up with a bunch of components that each correspond to what would have been a line of code in a text editor, and all you've gained is a pretty picture.

</comment><comment><USER5> One other thing I'd like to bring up - Go is a surprisingly strong tool for building flow based programs. It does have a few weaknesses, though, such as the lack of a graphical front end, but this isn't a requirement to write a flow based program.

The other (more significant) weakness is that you will have to either deal with interface{} casting quite a bit, or write type specific versions of the basic utilities.

On the plus side, you get virtually free concurrency and nearly purpose-built communication channels!

<comment><USER12> Yep, there is [https://github.com/trustmaster/goflow](https://github.com/trustmaster/goflow) for that

</comment></comment><comment><USER8> Thanks for the write up, has inspired me to give NoFlo a go. Reminds me of Reaktor (an app used to design audio DSP effects/instruments)

Will be checking your blog for a more hands on series.

<comment><AUTHOR> Glad to hear it! I'm sure you'll enjoy trying out NoFlo, even if you don't end up using it for anything.

</comment></comment><comment><USER13> one advantage of this is that you can sandbox much smaller pieces of code.

<comment><USER14> Could you clarify how visual programming helps with building smaller isolated projects over traditional text-based means?

<comment><AUTHOR> There are a number of ways, the most obvious being that you can build that smaller, isolated project incredibly quickly using flow-based programming. The thing is though, if you find traditional approaches quicker and easier then they're probably the ones you should be using instead (I'd still recommend trying FBP).

</comment></comment></comment><comment><USER15> How is flow-based programming different to FRP?

<comment><AUTHOR> I don't know myself (not used FRP) but based on a quick look it seems to be based on asynchronous dataflow programming so I imagine there are a lot of similarities (seeing as FBP is based on dataflow too). I'd love to hear more from someone who has experience with both.</comment></comment>


Why do programmers prefer Python over Ruby? - <AUTHOR> I'm a Ruby programmer. Recently there was a poll about everyone's favorite programming language and Python came out ahead over Ruby. To me, Python seems to lack some of the niceties that Ruby has like blocks and it also seems to have some superfluous stuff like having to pass self into each method. I'm not trying to ignite a flamewar but rather I am genuinely interested in why a programmer might prefer Python. Thanks. <comment><USER2> I've done some programming in Python and am currently learning Ruby.

One of the differences I see is Python's philosophy of having one, correct way to do things, while Ruby supports having multiple ways. This difference in philosophy seems to be clear in the language design. Python usually has one or two accepted ways to do basic tasks, while Ruby has more.

It's a little frustrating because in Ruby, I have to remember different syntax and constructs for doing the same basic thing. I'd much rather just have to remember one way, and expect other people's code to use that one way (ie, Python is more readable).

Python also seems to have a larger community and more well-developed/useful libraries and tools. I don't think NumPy and SciPy have equivalents in Ruby.

Also, even though Ruby says it advocates the principle of least surprise, I'm often surprised by Ruby, and much less so by Python.

</comment><comment><USER3> I'm going through Ruby koans right now but am a Python programmer.

The #1 thing I like about Python is the community and has nothing to do with the language. I find the python community doesn't actively champion the language as the end all be all (for better or worse). Rather it is accepting of when Python sucks for a specific task.

I also find Ruby to be really web focused. Python tends to have lots of libraries that are not solely for web based needs. Not to say Python doesn't have web stuff...there is so much I can't even keep up.

<comment><USER4> I have that same feeling about the ruby comunity and its the main reason why I try to stay away from it, it gets on my nerves. About the python comunity, no complaints so far with one or two exceptions. I also feel the same about ruby being more web focused, it may be wrong but that's how I see it too, one "problem" of python is that there is too many modules and with some variations so its rather hard to know it. And maybe its just me but the python docs sometimes are rather confusing, with few examples (I like to read examples, its a lot easier IMHO)

</comment></comment><comment><USER5> As a scientific programmer, I like python because I'm used to it, because there are a wealth of useful libraries and packages that suit my needs, and that it can be "fast" (if you know how to take advantage of the underlying c code and use numpy whenever possible; and the obvious fast-prototyping advantage over compiled languages). I don't have much experience with Ruby so this is by no means a diss of the language. Its just why I have stuck with it for so long

</comment><comment><USER6> Disclaimer: This is very subjective and from a Python guy

1\. While Python and Ruby are roughly of the same age, Ruby was only popular in Asia before Rails. At that point Python already had a solid base of non-web stuff.

2\. Ruby feels more wild and crazy to me. Is monkey-patching still considered cool in the Ruby community? As a Pythonista I try to avoid such confusing stunts.

Due to both of those reasons I believe Python has more solid libraries. For example, Rails was extracted from a small productivity app, while Django was extracted from a serious newspaper website. While Ruby was fixing memory leaks, the Python interpreter was speeding up its hash map.

As a language enthusiast I envy Ruby for the blocks. As a Python programmer I never felt the need for them.

<comment><USER7> > As a language enthusiast I envy Ruby for the blocks. As a Python programmer > I never felt the need for them.

I think one of the reason why Python isn't hurt by the lack of blocks is the difference in namespacing rules. Since defining a top level function doesn't pollute the global namespace, there's much less of a need for blocks and mulit-line lambdas. If I feel that I've got too many functions sitting around in one of my project's files, I can simply open up a new file, utils.py, and all my utility functions will be in their own namespace.

So, I agree, blocks are cool, but when it comes to re-reading code, I like that I've been forced to spend the extra few seconds to give my functions names.

</comment></comment><comment><USER8> One word: Cython.

Having a fast way to interface C code, and speed up your bottlenecks is a godsend for anything that isn't a strict web app.

I'm sure Ruby has an FFI, and most likely even has a SWIG interface but Cython is far beyond that--its a typed version of Python that compiles directly down to C/C++, so you can port code between Python and Cython instanteously.

Doing just that usually makes it 2x faster. Annotate it with types to make it more than 10x faster. Change the algo to something you can only do efficiently in C, and it's 100-1000x faster. Seeing something go from taking 10 minutes to taking 600ms in less than a days work is fantastic.

\----

Also personally, I find I like white-space scoping. Python is like psuedocodethat actually runs.

<comment><USER9> Hmm... Could you recommend some resources for this? I need to wrap a couple libraries and make them accessible in both Python and Ruby.

<comment><USER10> The best thing to do is to take a look at the Cython docs (<http://cython.org/>) and check out some of the whitepapers they have posted. There is also a bunch of cython code on github. If you search "cdef extern" you should find examples where other people have wrapped external c libraries.

<comment><USER9> Thanks!

</comment></comment></comment></comment><comment><USER11> I like Perl, I like SmallTalk, and given that, I actually do like Ruby the language. My main peeve is the infrastructure. No, not even the community, you'll find nice people and bumwads on both sides. But I just can't stand the penchant for overly cutesy DSLs that seems to pervade the Ruby world. It's basically Lisp macro abuse all over again.

</comment><comment><USER12> I know that you are asking why Python might be better for some uses, so pardon a little tangent:

About 7 years ago I tried really hard to get into Python because someone I know at Google really liked the language, and I needed to pick up a better scripting language (I dislike Perl). I used Python a lot for about a year, reading a few books, using it for a lot of small projects. Python was nice!

Then I started looking at Ruby, and for me it was programming language love at first sight. I can not justify my strong preference for Ruby on technical grounds, rather I simply prefer it. Ruby is no longer just a scripting language for me (although I write a _lot_ of 20 line Ruby programs just to get stuff done).

So, I would argue that you should choose either Python or Ruby based on your personal gut feel, after spending time with both languages. The only exception to this advice is if you want to work for a company that prefers one over the other.

</comment><comment><USER13> Readability. If I'm working in a team, it's generally the _number_ _one_ concern for any language I'm using, but even if it's code that I know nobody else will ever touch, there's a great comfort in knowing that I won't have to spend too much effort figuring out what I was thinking when I'm reading year- old code.

<comment><USER14> This is moot you're not writing a low level math library nor are you using some archaic programming language, readability is relative and it's between the chair and the keyboard that defines that value. You can write unreadable code in python just the same as you can in ruby.

<comment><USER13> You're right in a sense, but I strongly disagree.

The trick is to write readable code, and Python encourages that at every step. The Python mantra is "don't be clever". Compared to Ruby, where developers routinely vie for the most clever way to do something that may or may not (and usually isn't) in the interest of readability.

Also, the significant whitespace enforces the readability of Python vs. something like Perl, which is a very information-dense and, in the opinion of myself and many others, a substantially less readable language.

Either way, for my money, having used both languages extensively, Python proves to be more readable over the long haul. YMMV.

</comment></comment></comment><comment><USER15> One point is that the python eco-system covers a lot more ground. Ruby and the ruby eco-system basically only focuses on web development, if ruby has other strength the community is pretty quite about it.

Python is also pretty good at web development, but is also excellent at scientific computing and visualization, data processing, statistical analysis (including nice bindings to R), network servers using twisted, natural language parsing, GIS analysis, computer vision and image processing and so on. Python also has Cython which makes it trivial do use C to speed up crucial functions.

Basically I prefer python because I can use the same language for everything I need and want to do.

<comment><USER16> I would second this, in my $DAYJOB which is focused on building search engines, python is one of the goto languages alongside R for doing a lot of exploratory stuff.

Sure ruby could, but it would have to play catchup adding things like sage, cython, numpy, scikits etc etc

</comment></comment><comment><USER17> I think it has nothing to do with the languages themselves, and more to do with a sort of tipping point around tools, libraries and adoption. Through random chance, I prefer Python simply because people I know use Python so I started to use Python.

At the end of the day, there really isn't much inherent that makes Python better than Ruby. Which is exactly why the world would benefit from the eventual dominance of one over the other. Even if it is a randomly driven tipping point process.

On the other hand: Ruby vs Java vs C++, each has clear advantages in certain situations. These languages seem less likely to compete with each other.

</comment><comment><USER18> I am not sure that this poll was representative of the whole programming community. We are a Berlin based startup and we're looking for a Python developers and they are a lot harder to find than Rails devs. Not to talk about PHP, C# and Java developers.

Also there are tons of existing libraries and you don't have to implement anything but your actual business logic.

</comment><comment><USER19> When I was starting learning Python, I considered Ruby. At the time, Ruby was Active Record only. Active Record does not let you use SQL procedures without defeating the whole purpose of Rails. For me, it meant that Rails was only good for toy CRUD apps.

</comment><comment><USER20> The opening of classes seems to cause people problems. A lot of Ruby's popularity is from it's killer app, Rails. But trying to program python after Ruby for me is difficult, python feels so constrained.

</comment><comment><USER21> Python is a lot easier to learn than Ruby, therefore there are more people know Python than know Ruby. The poll just shows the demographics.

</comment><comment><USER22> I came from a java background. I like Python because I feel like I think in python and it makes me write elegant (IMO) code.

</comment><comment><USER23> because FUCK RUBY (c) the black guy at security conferences

</comment><comment><USER24> Well, for me, it's readability and maintainability. I find Ruby unreadable, and if I want to hire a programmer, it's easier to take a Perl, PHP, or even C# developer and teach them Python than it is to teach them Ruby, at least from my experence.

I don't think there's anything wrong with Ruby as a language, and until it was added to most languages, the scaffolding held me in awe...

I just prefer Python's readability.

<comment><USER25> I agree. I am a PHP developer and just started learning Python a few days ago and must say, for now, it is indeed easy to learn and is very readable. Though I never tried Ruby so I can't tell you if I would find that easy also.</comment></comment>


Vegetarians 'less healthy, have lower quality of life than meat-eaters' (2014) - <AUTHOR> http://www.independent.co.uk/news/science/vegetarians-are-less-healthy-and-have-a-lower-quality-of-life-than-meat-eaters-scientists-say-9236340.html <comment> cat-dev-null This old study may only underscore people generally don't measure their diet or pay attention to nutrition in order to avoid preventable conditions of their own fault: balanced, limited caloric intake and essential vitamin sufficiency. Maybe a few readers desire to feel superior for lazy, carbon- and methane-intensive lifestyles, but that's likely only a small minority. There are healthy meat eaters and healthy vegans.

As an anecdotal sample size of 1, I've eliminated meat for many years now and my cholesterol numbers are outstanding... vitamins including B12 are all fine considering the variety of bioavailable sources in my diet. I only take high- quality fish oil, that's it. Works for me, YMMV.

<comment><USER2> Check out videos about fish oil supplements on [http://nutritionfacts.org](http://nutritionfacts.org). I took them for a while, but have since realized algae oil is a much safer source of EPA-DHA. Even krill oil would be an improvement.

<comment><USER3> That site you link to is a vegan propaganda site. It only has selective studies that back its not-so-hidden agenda!

<comment><USER2> Exactly who stands to gain from a vegan agenda, do you think? Big Broccoli? Dr. Greger Super Pills Inc.?

<comment><USER4> The fish that vegans don't want harvested for their oil.

<comment><USER3> Unfortunately, vegans kill a lot more animals by replacing forests and grasslands with monocultures and most importantly - they kill wild animals, where omnivores kill reproducible farm animals only... that wouldn't otherwise exist anyway!

<comment><USER2> So you mean to say the mass production of corn and soy as animal feed is not an example of crop monoculture? Also, how do vegans "kill wild animals" \- and are you so sure they do so in numbers of over 150 billion per year, as does modern animal agriculture? I think you are confused. Wildlife suffers mostly from the collateral damage of meat production - deforestation, drought, pollution, climate change, etc. Definitely check out documentaries such as Cowspiracy if you are not convinced, or just search around. Here's something I found about Amazon deforestation:

> A 2009 Greenpeace report found that the cattle sector in the Brazilian > Amazon, supported by the international beef and leather trades, was > responsible for about 80% of all deforestation in the region,[3] or about > 14% of the world's total annual deforestation, making it the largest single > driver of deforestation in the world.[4] According to a 2006 report by the > Food and Agriculture Organization of the United Nations, 70% of formerly > forested land in the Amazon, and 91% of land deforested since 1970, is used > for livestock pasture.[5][19]

[https://en.wikipedia.org/wiki/Deforestation_of_the_Amazon_ra...](https://en.wikipedia.org/wiki/Deforestation_of_the_Amazon_rainforest)

<comment><USER5> Exactly this. Animal agriculture necessarily kills far more animals than crops since you have to grow significantly more crops to feed livestock in the first place. Meat does not just magically appear without having been fed.

Not only that, but in addition to the unpaid negative externalities of meat production (e.g. climate change, deforestation, health problems), the price of meat is kept artificially low through subsidies for corn and soybean produced as animal feed.

<comment><USER3> My animals don't eat crops - they eat grass. Climate change? Get educated! It's not caused by sustainable animal farming! And what health problems? Vegans have worse health than people who eat sustainable animal products. Don't compare Mickey Dee's to quality meats! It's like saying all cars are Mickey Mouse cars because you only look at a Yugo!

<comment><USER2> This might come as a big shock, but grass-fed farming operations are in fact less sustainable and worse for the environment than even factory farming. I really recommend you check out the documentary Cowspiracy (it's available on NetFlix), or just look at the fact sheet on their web site - [http://cowspiracy.com/facts](http://cowspiracy.com/facts)

Your claim that vegans have worse health is entirely unsubstantiated. Do you have any evidence for this?

<comment><USER6> ...and it ends up with Ranchers holed up in a federal building with guns.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> a) This is from April 2014.

b) While I couldn't immediately find anything debunking this particular study, it's hard to take at face value considering all the research showing correlation between high-vegetable diets and lower mortality[1], lower risk of gastrointestinal cancers[2][3], etc.

[1] [https://www.ncbi.nlm.nih.gov/pubmed/23599238](https://www.ncbi.nlm.nih.gov/pubmed/23599238)

[2] [https://en.wikipedia.org/wiki/Diet_and_cancer#Processed_and_...](https://en.wikipedia.org/wiki/Diet_and_cancer#Processed_and_red_meat)

[3] [https://en.wikipedia.org/wiki/Diet_and_cancer#Fiber.2C_fruit...](https://en.wikipedia.org/wiki/Diet_and_cancer#Fiber.2C_fruits_and_vegetables)

<comment><USER7> Britain's NHS offered an exhaustive list of problems with this study when it came out [1].

[1]: [http://www.nhs.uk/news/2014/04April/Pages/Vegetarians- have-p...](http://www.nhs.uk/news/2014/04April/Pages/Vegetarians-have-poorer- quality-of-life-study-claims.aspx)

</comment></comment><comment><USER8> > in a poorer state of physical and mental health overall.

> mental health problems such as depression and anxiety.

Or: having depressions and anxiety and/or being in poor physical & mental health makes you eat crappy food

</comment><comment><USER2> Lots of quackery here.

[http://www.plosone.org/article/fetchObject.action?uri=info:d...](http://www.plosone.org/article/fetchObject.action?uri=info:doi/10.1371/journal.pone.0088278&representation=PDF)

> no statements can be made whether the poorer health in vegetarians in our > study is caused by their dietary habit or if they consume this form of diet > due to their poorer health status. We cannot state whether a causal > relationship exists, but describe ascertained associations. Moreover, we > cannot give any information regarding the long-term consequences of > consuming a special diet nor concerning mortality rates

So it's also entirely possible those with existing chronic conditions and poorer subjective health were more likely to try vegetarian eating (reverse causality).

But no, it gets worse ...

> While 0.2% (31) of the interviewees were pure vegetarians (57.7% female), > 0.8% (124) reported to be vegetarians consuming milk and eggs (77.3% > female), and 1.2% (186) to be vegetarians consuming fish and/or eggs and > milk (76.7% female). 23.6% (3,651) reported to combine a carnivorous diet > with lots of fruits and vegetables (67.2% female), 48.5% (7,505) to eat a > carnivorous diet less rich in meat (60.8% female), and 25.7% (3,977) a > carnivorous diet rich in meat (30.1% female)

> Since the three vegetarian diet groups included a rather small number of > persons (N = 343), they were analyzed as one dietary habit group. Moreover, > since the vegetarian group was the smallest, we decided to match each of the > vegetarians (1) with an individual of each other dietary habit group > (carnivorous diet rich in fruits and vegetables (2), carnivorous diet less > rich in meat (3) and a carnivorous rich in meat (4)).

31 vegans out of a group of 15,000 people, and the rest of the "vegetarians" weren't even clearly defined, nor any control on how long they've been veg*n. Overall, this is a very poorly designed study with several obvious flaws.

And anyway, common ovo-lacto vegetarians still include harmful dairy products in their diet, so even if all this were true, I wouldn't be surprised.

Finally, here is a newer study by the same author on the same data with an entirely contradictory conclusion:

[http://link.springer.com/article/10.1007%2Fs00508-013-0483-3](http://link.springer.com/article/10.1007%2Fs00508-013-0483-3)

> Subjects eating a carnivorous diet less rich in meat self-report poorer > health, a higher number of chronic conditions, an enhanced vascular risk, as > well as lower quality of life. In conclusion, our results have shown that > consuming a diet rich in fruits and vegetables is associated with better > health and health-related behavior. Therefore, public health programs are > needed for reducing the health risks associated with a carnivorous diet.</comment>


Do pro software devs memorize language syntax? - <AUTHOR> An article discusses cons of memorizing syntax and provides solutions to boost productivity after all <comment><USER2> I don't go out and actively memorize things, not do I know anyone who does. But the syntax you use often gets memorized nonetheless, simply through common use.

<comment><USER3> That with the IDE's autocomplete covers most of my needs.

</comment></comment><comment><USER4> Do writers memorize grammar?

Presenting memorization as "learning" is a crippling effect of the limited ways to measure results forced by the accounting needs of the school system.

Memorization is not understanding and is a very small fraction of the possibilities of learning.

So, no, of course we don't. We use the language, use the reference when needed, and little by little, use it less for the beaten paths of our particular use.

</comment><comment><USER5> Years ago, or perhaps decades ago, when there switching languages every few months wasn't common, yes - syntax was not only memorized, it was second nature. Not only that, but the language influenced how you thought and solved computing problems. I have always felt that it would take about a year to retrain your brain to efficiently program (or rather think) in a new language. Until then, you would structure your code as if it was written in the last language you use, because that's how you break down problems in a logical nature.

<comment><USER6> There was not a time in the past when switching languages was less common. The tools of the past assumed one would switch between any number of languages; each tool would have its own, often maximally powerful, language. This has become _less_ pervasive as communication and code sharing has become easier, allowing conventions and standards to unify.

</comment></comment><comment><USER7> I'll shoot off at a different angle from the other posts here. I'm a blue- collar dev doing web development. I feel the syntax of the major scripting languages is relatively small and the amount that you have to know even smaller. Many developers can just shoot from the hip and get by with some doc look-ups along the way and get things done. Maybe you don't know the prettiest syntax, but you can work around it something which works good enough. You could probably significantly strengthen that grasp with a little bit of scheduled time doing nothing but leveling up on syntax. Or maybe you never find the time to do that and you still manage to get things done. And keep in mind that editors help out a bit with this as well.

I find the difficulty in memorizing is in library usage and all the other stuff which is outside of the syntax. When I'm getting kicked in the head, it's maybe trying to remember how to do something in Git. Or maybe I have to remember the sequence of some development process. Or maybe I forgot a password I never bothered to enter in my manager. There's loads of things you get a cognitive beating from, but syntax doesn't strike me as the worst.

<comment><USER8> What does blue collar dev mean?

<comment><USER7> Nothing. But it's my way of saying, didn't study computer science, not an academic and fully self-taught.

</comment></comment></comment><comment><USER9> Programming since age eight here. I would say that before Google and Stack Overflow were prevalent, really early on, maybe like in the early 90s, I would remember things more. Because the alternative was actually looking through a book, and although I did that plenty of times, it was often more convenient just to remember.

But as the internet got better at instantly providing examples and references and more and more technologies came out and I started using many different programming languages, its at the point where I often Google very basic things if I am tired or just not 100% sure which way the particular language works or about all of the details.

Its essentially using Google/the internet as an extension of my brain. Like, I probably could remember this thing, but why bother? I believe it increases my productivity because it means I can save my "brain juice" or energy for actual problem solving.

<comment><USER10> Before I had a reference book of the language. Also I would program always in the same language before stack overflow.

Now I switch between so many languages. But still use references. But now it's easier in Dash.app I have references to most of the languages, API, libraries I use. This way if I want to know how to make if then else syntax work in Go or Javascript or Python I can look it up super fast. Same for Libraries and API's.

Now language idioms is something you need to keep on top off. These are better to remember so you can implement it.

</comment></comment><comment><USER11> In my many decades working as a professional programmer I have never made a conscious effort to try an memorize any information relating to programming.

I would also say that if you are finding yourself spending lots of time trying to memorize programming related information then your approach to programming is probably wrong.

</comment><comment><USER12> Stuff you use more often tends to get memorized, less common stuff googled when you need it. When it's memorized you do save a lot of time. I'd say if you're really experienced in a language you rarely google syntax because 95% of it has become memorized.

</comment><comment><USER13> I would say no, but actually it's a bit more nuanced than that. I use many different languages and if I am not actively working in one I might forget some syntax just after a couple of weeks. It only takes about an hour or so of working in that language (and RTFM) to get it all back. I don't think others have as much of an issue if they're in a single language all day but I jump around a lot and my hands-on-ness fluctuates.

</comment><comment><USER14> It really depends on the software professional.

I do, but I have a terrible memory. I use flashcards to remember names of technologies that are important but that I don't use daily. I create flashcards when I make a mistake in programming, so I'm less likely to make that mistake again. I create flashcards every time I look up information, so that I keep it in mind.

If you have a normal memory, you probably don't need it as much.

</comment><comment><USER15> One of the best parts of Lisp is the slobber-proof syntax (except in a few well-known oft-complained-about cases). Another is that the hyperspec is published online.

Between the two, there isn't a whole lot of memorization needed. You can still tell a pro's code over a hobbyist, but it doesn't take all that long for the distinction to blur.

I guess practice is a form of memorization afterall.

</comment><comment><AUTHOR> Missing link for this item: [https://medium.com/@tomaszs2/do-pro-software- developers-memo...](https://medium.com/@tomaszs2/do-pro-software-developers- memorize-programming-language-syntax-4e93b38bc2ec)

</comment><comment><USER16> This is contrary to most of the advice in this thread, most of the folks here probably skew towards the highly skilled, and thus they don't really know what they know since it has been engrained into their muscle memory.

The syntax of a language is difficult to lookup, I don't know what languages are you are using, but I would say yes, you should know 95% of the syntax of the languages are you are coding in.

Furthermore, I think you should be able to switch between idioms in the language fairly fluidly, between functional, oo, data driven, async, iterators, etc.

<comment><USER17> I can easily switch between C#, javascript and postgres SQL. But when it comes to powershell.. I google every little language construction. It happens to me every 2-3 weeks for a decade. I still cannot remember from the first time how to iterate over collection, how to define collection or hash table. I dont know why I struggle so much to remember it.

<comment><USER18> My coworker and I feel the same way. We're both .net devs for over 15 years each. We avoid power shell like the plague. The syntax is ugly and there is a bit too much magic in it.

As a point of comparison, we are both quite happy to use bash so we don't have an issue with command line tools or scripting languages.

That said, after writing a Windows forms app using powershell, I had a slightly better understanding of it.

</comment></comment></comment><comment><USER19> I do a lot of code puzzles and code golf which gives me a motivation to learn some obscurer language features. But for professional software development, it's more important to learn the _concepts_ that the language's syntax is encapsulating.

That said, there are a few syntactic snafus I regularly find myself in.

1\. I regularly switch between Python and Javascript, so I'm always typing `push` instead of `append` and vice versa.

2\. I also switch a lot between Redshift and RDS, so I always forget which functions are exclusive to Redshift and which to Postgres.

</comment><comment><USER20> I think that if you don't, you might end adopting superstitious "best practices" to compensate.

For example: Countless engineers I've worked with don't know operator precedence. So tons of "y = ((i * m) + b);" type stuff. Do not need any parentheses in this case, but if I remove them, then they have no idea what happens... So they litter all math with superstitious parens.

There are so many better examples, but you get the idea. I have suffered "coding standards" that existed solely because of the devs that won't RTFM.

<comment><USER21> I'd think most people would remember * before + from school math, but for other operators the precedence is kind of arbitrary (can vary between languages) and adding parentheses to make it less ambiguous and more readable doesn't seem like the worst idea.

<comment><USER20> Yes, it can vary, but it's kind of important. If you don't know the order of specific operators, or if your language violates math rules, a dev might want to know that instead of resigning to just not know things.

For readability, yes... sometimes, but for readability sake, not "I don't know what this does without them."

I'm not just talking about math operators, but logical ones too. I've seen people put parens around every "not" because they weren't sure. There are three essential logical operators. Surely those can be learned at least. And yes... in those cases where readability is an issue, of if they're necessary, please do add them, but please don't if you're unsure what "!a && b || c" does. Seeing "((!a) && b) || c" raises red flags.

</comment></comment></comment><comment><USER22> Not necessarily the syntax of a specific language, but more the ur-syntax of what sort of operations are available to work with and what things they do. A good programmer doesn't necessarily have the exact syntax memorized for mapping over an array, but "it's straightforward to build an array by applying a function to every element of an array" should be there when it's relevant to the problem you're solving.

</comment><comment><USER23> Language is just a tool. It's important to know which tool is best for which task. After identifying which language to use, picking up a language usually just takes a day or two max. Memorizing is definitely not a necessity, but is a side-effect when you continuously work using a language for more than a few days.

</comment><comment><USER24> I can type quickly so if i am not sure of the particular precedence rules or syntactic sugar I dont bother because it is quicker to correctly write something a little more verbose than it is to remember a cute language trick that the poor maintainer also has to look up.

</comment><comment><USER25> 20 years programming Often considered a 10x developer. Also been a .1x developer when sick. I forgot syntax constantly, basic things I use often.

Difference is I understand the underlying concepts extremely well. I can reference any needed syntax in seconds.

</comment><comment><USER26> to be productive, you have to minimize the need to rtfm, but you should also be cognizant of any new implementations -- a refresher may sometimes provide a better way.

I recently stumbled on the postgres method jsonb_insert(jsonb,path,value) which is much more elegant than jsonb_set(jsonb,path,jsonb||value) when you just want to add a thing to some json array inside an object.

I hadn't really looked at the docs much since 9.x, I deal with jsonb everyday, so I had what I needed memorized. But despite keeping my installs updated, I had been doing it the hard way for way too long.

</comment><comment><USER27> Yes. Not completely, but they will be proficient in the ones they use regularly to the level of going weeks/months without checking the documentation.

</comment><comment><USER28> Language and its syntax is the last thing I care about. You just have to find the right tool for each task.

</comment><comment><USER29> It's memorized in the same way that natural language syntax is memorized. It's just learned.

</comment><comment><USER30> I dont think this is required. Every language has some rules but more importantly control structures which are identical but have difference in syntax. ie, take Foreach loop, I almost every time confirm the foreach's language specific syntax. Try jumping b/w php, java, python, js in same day multiple time.

so i will say remembering it is not necessary, but definitely handy thing.

</comment><comment><USER31> Most interviewers believe this.

</comment><comment><USER32> Knowing the features of your language can expand your horizons. I had already been a programmer for years when JavaScript entered a new era, and I thought I knew a thing or two about computing.

With time, I understood that what I had was a lot of memorised knowledge about specific systems. The layout of a Master Boot Record. The numbers to BIOS interrupt calls. Memory addresses.

The transition from Pascal to C that I made at age 12 had only opened my eyes a little. I learned that there is no true distinction between a function and a procedure (subroutine), that statements and expressions are really two sides of the same coin, that a for() loop doesn't have to enumerate anything, and that pointers, direct memory access and direct access to system calls are a very powerful tool.

For the past 10 years, I found myself repeatedly involved in web development projects. I had minimal interest in JavaScript at first, viewing it as a toy language for simple automations, and I did most of the heavy lifting in PHP and Java Servlets. Gradually, as people began doing more and more "weird" stuff in it, I became more curious.

That's when I fell into the rabbit hole.

Functional programming, map-reduce, lambda notation, closures, asynchronous code, generators, observables, applying argument lists to functions at runtime, destructuring assignments, spread notation, treating functions as behaviours you can pass around...

Basically, stuff that modern JavaScript has borrowed from languages such as Lisp.

It made me aware that there's a distinction between computing as a Platonic ideal and computing as a practical discipline. Computing is a set of ideas about information and the logic of manipulating it.

Computer science is a branch of mathematics and information theory. Computer programs are living equations, written in a notation rigid enough that a machine can read them and perform computations with them.

I didn't have that kind of perspective 10 years ago. My skill as a programmer has taken a very solid step up. Unfortunately, this has also made me overqualified, and I'm often in a situation where I'm doing work that just about any developer with a couple of years' experience could pull off, and would learn much more from than me because they've never done it before.

I've come to the conclusion that the jobs that require you to actually appreciate the nuances of computing and programming on this level are virtually non-existent. Most customers don't care if the application software is a rotting carcass under the hood. They want it cheaply and quickly, and if it breaks, "That's what computer programs do, isn't it?" and then they pay someone to fix it.

In most cases, the practical craft of programming is very far removed from that Platonic ideal I mentioned.

I would argue that the craft of programming is in fact almost entirely unrelated to the field of computer science. Here is an analogy: Building elaborate cuckoo clocks involves a great deal of classical mechanics, but it doesn't teach you anything fundamental about physics. A watchmaker is not a theoretical physicist, even if his practical knowledge of the mechanical movement of small parts suspended in a mixture of oxygen and nitrogen in a narrow temperature range under the influence of the gravitational field of a particular planet may be quite excellent.

<comment><USER32> The field of information technology is still in its infancy. Had it been more mature, we would already have formalised the distinction between the researcher, the engineer and the craftsman, so commonly seen in other fields.

Our current use of titles is loose and very imprecise, and serve as marketing more than anything else. I would like to see a formal distinction between the different roles.

Douglas Engelbart, Dennis Ritchie, Richard Stallman and Linus Torvalds are not "developers" or "programmers".

Engelbart was a "computer scientist" in the true sense of the word. His systems saw little practical application, but they enhanced our understanding of what computing machines could do.

Ritchie, Stallman and Torvalds are engineers. If they were in the car industry, they would be renowned engine and drive train engineers.

Your typical programming job, on the other hand, is equivalent to the title of "mechanic", "construction worker" or "craftsman".

Many people who enter the field don't actually want to be craftsmen, but since we are so poor at making the distinction, we get misunderstandings. There are retired programmers who worked in the business for 30 years who never quite understood why the job turned out to be so different from what they had imagined.

One consequence of making this distinction is that a hierarchy of exclusivity inevitably emerges from this. There are far more car mechanics than drivetrain engineers. Only so many people have the privilege of doing that.

I don't know if the supply of computer engineers and computer scientists is smaller or greater than the demand, but what I do know is that these jobs aren't the kind you find on a job site. They are found through networking, which requires you to have the right kind of network, which isn't possible for everyone who would be a good fit for the position.

</comment></comment><comment><USER33> your hands learn to code.</comment>


Scaling the GitLab database - <AUTHOR> https://about.gitlab.com/2017/10/02/scaling-the-gitlab-database/ <comment><USER3> Funny thing is that they publicly own up to their performance problems in an unusual way: in their comparison with GitHub ([https://about.gitlab.com/comparison/](https://about.gitlab.com/comparison/)) they list "Fast page load" as a feature that GitLab lacks and GitHub has.

Nevertheless, the slowness is really annoying, especially because their product is so good on all other accounts. If scaling their database can help speed things up, I bet they will be glad to remove this embarrassing "missing feature".

In marketing terms, having fast page load would be called a "qualifier". For example: you expect a hotel to provide toilet paper. You won't pick any hotel because of it, but you will surely avoid one that doesn't.

<comment><USER4> The good news is that page loads are much better now. Our initial page load ping [http://stats.pingdom.com/81vpf8jyr1h9/1902794](http://stats.pingdom.com/81vpf8jyr1h9/1902794) is better than GitHub.com [http://stats.pingdom.com/81vpf8jyr1h9/1902795](http://stats.pingdom.com/81vpf8jyr1h9/1902795)

We got work to do in the 99% and merge request page load but the overall situation has improve dramatically.

We still got work to do in availability, so I changed the 'feature' to reflect this [https://gitlab.com/gitlab-com/www-gitlab- com/commit/3b3bddf5...](https://gitlab.com/gitlab-com/www-gitlab- com/commit/3b3bddf51e47f51e97a03fbb1a1ae8f40fdfd279)

<comment><USER5> Maybe I just can't find it, but what URLs are pingdom.com checking? Do both issues have the same content? Or does one have 50 comments and another has 1?

<comment><USER4> These are checking [https://github.com/gitlabhq/gitlabhq/issues/1](https://github.com/gitlabhq/gitlabhq/issues/1) and [https://gitlab.com/gitlab-org/gitlab- ce/issues/1](https://gitlab.com/gitlab-org/gitlab-ce/issues/1)

The one on GitLab is considerably longer.

</comment></comment></comment></comment><comment><USER6> My team is currently using the hosted cituscloud, which uses PG Bouncer. They note that the reason for sharding is because of high writes. We've actually seen big benefits for moving over to a sharded setup via Citus just as much for the read performance.

By sharding by customer we're able to more effectively leverage Postgres caching and elastically scale. Since switching over, our database has performed and scaled much better than when we were on a single node.

There is some up front migration work. But that's limited some minor patching of activeRecord ORM, and changes to migration scripts. We considered casandra, elastic search, as well as dynamoDB, but the amount of migration changes in the application logic would take an unacceptable amount of time.

<comment><USER7> Craig from Citus here. Thanks for the kind words Justin, as you mention there is some upfront work, but once it's in place it becomes fairly manageable.

We've been working to make that upfront work easier as well with libraries that allow things to be more drop-in (ActiveRecord-multi-tenant: [https://github.com/citusdata/activerecord-multi- tenant](https://github.com/citusdata/activerecord-multi-tenant) and Django- multitenant: [https://github.com/citusdata/django- multitenant](https://github.com/citusdata/django-multitenant))

<comment><USER8> Praying for a [http://docs.sequelizejs.com/](http://docs.sequelizejs.com/) plugin

</comment></comment></comment><comment><USER9> This article is also very useful in showing just how far you can push Postgres _without_ reaching for any of these optimizations. I've seen too many projects worry about these things very early on in their lifecycle, when in reality they are no where close to having enough traffic to cause a problem.

<comment><USER10> The whole point why you should worry from the beginning is so that you don't have to re-work everything and put a huge risk to the business when you have to do it.

<comment><USER11> Instead you appear to be advocating for over engineering a solution for a problem you don't yet have where the opportunity costs are probably features customers can use. That is also a huge risk to a business, but one that is _immediate_.

<comment><USER10> Thinking about how you would shard your data is hardly over engineering. On some storage systems you are forced to anyway: Cassandra, dynamodb.

And the cost of building the solution from the onset is hardly a killer. The pain usually comes from operational complexity.

</comment></comment></comment></comment><comment><USER12> for anyone new to gitlab, it is a bear. even their own cluster/HA documentation is mostly just a thin roadmap to a final goal. a statement of intent if you will. There are just far too many moving parts in this software and the glaring problem of "ruby doesnt scale well" is everpresent. packaged releases depend entirely on a bundled chef installation to get everything out the door and running.

The software tries to do too much. CI/wiki/code review/git/container repository/analytics package.

CI sharding per team and carving up gitlab into departments and groups has been my only solution so far, and its just distributed risk at that. its the difference between the unix ethos of do one thing and do it well, versus the F35 approach of do everything forever.

<comment><USER13>
        > and the glaring problem of "ruby doesnt scale well" is everpresent

GitLab has many performance problems (and many have been solved over the years), but Ruby has thus far not been one of them.

<comment><USER14> I remember installing GitLab on a small NAS-type box. It was a while ago, but on each start-up it ran a nodejs tool to pre-compile some assets, I suppose. On that machine, it took 10-15 minutes to start. Afterwards, the unicorn workers kept getting killed because the default memory limit (128 MB) wasn't enough to process more than literally a couple of requests. It did work, but pages took 2-5 seconds to load, which I couldn't stand.

For anyone needing a small Git Web UI, I suggest [https://github.com/go- gitea/gitea](https://github.com/go-gitea/gitea).

EDIT: Oh, you're that guy ^^;.

Anyway, my rant was probably a bit harsh. I know that GitLab isn't really written for hardware comparable to a Raspberry Pi and that it's used by small and large teams all over the world. But on my hardware Gitea renders pages in 20-100 ms, which is orders of magnitude faster than GitLab. I'm sure part of this is the fault of the language.

<comment><USER15> So just use cgit on your 128mb box and don't complain about gitlab? It's like installing Windows 10 on a machine with 1gb memory, starting Photoshop and complaining about swapping and that it's running slow.

Gitlab runs fine out of the box with 4gb memory and when screwing with some provided knobs it's fine with 2gb or less memory.

It's in the docs, it's in every FAQ, every Stackoverflow answer... so why the hate?

64mb memory? Use gitolite. 256mb memory? Use cgit + gitolite 1gb memory? use gitea or whatever else there is. 4gb memory? use gitlab if you want.

I'm not affiliated but I can't stand the "gitlab doesn't run on my home wifi router - it sucks" posts here... it works pretty well if you read & acknowledge the documentation and requirements.

<comment><USER16> I'm running a Gitlab instance for a smallish dev team of 10.

I haven't gotten it to run reliably with any less than 8Gb, even though we don't have a whole lot of activity there.

It takes surprisingly long to start up too, even on a relatively beefy host.

I do like the product though, the all-in-one solution with code hosting, issues, code review (needs work though...) and CI is great. (Not using the deployment, orchestration stuff).

My biggest issue is that they seem to be too focused on pumping out new features.

There are 9k open issues just for CE. That's probably 12k+ for all products (runner, EE,...).

That either is a sign of very bad project management or a way to big agenda. Probably both.

<comment><USER4> I'm surprised to hear GitLab isn't running reliably with 8GB. Are you running the Omnibus installations?

The open issues for CE [https://gitlab.com/gitlab-org/gitlab- ce/issues](https://gitlab.com/gitlab-org/gitlab-ce/issues) do include feature requests.

<comment><USER16> I'm using the official docker image.

It's been very pain-less in setup and maintainance. Upgrades were mostly painless as well until now.

With 8GB for the docker container it's solid and pretty stable, but any less and behaviour would be very unstable (timeouts, very long request load times, ...).

Of course those 8GB are for the whole stack (Postgres, Redis, Nginx, Unicorn instances, ...), so I'll admit that 8GB for the __Docker container __is quite important context here. ;)

<comment><USER4> Thanks for the answer. I would have thought that 4GB is enough. Thanks for providing a data point that it might not be.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> Running PG bouncer is a very basic optimization. You typically start out with PG bouncer in your stack if you have experience running Postgres.

If you're new to running your own postgres databases you should also check out Wall-e: [https://github.com/wal-e/wal-e](https://github.com/wal-e/wal-e) And the awesome pg stat statements [https://www.postgresql.org/docs/10/static/pgstatstatements.h...](https://www.postgresql.org/docs/10/static/pgstatstatements.html)

<comment><USER7> If you are already familiar with wal-e, or even if not, you might want to consider taking a look at wal-g[1]. Wal-g is a newer edition of wal-e written in go that we've seen can have up to 7x performance improvements[2].

[1] [https://github.com/wal-g/wal-g](https://github.com/wal-g/wal-g)

[2] [https://www.citusdata.com/blog/2017/08/18/introducing- wal-g-...](https://www.citusdata.com/blog/2017/08/18/introducing-wal-g-faster- restores-for-postgres/)

<comment><USER18> While wal-g looks great, having evaluated it a bit, I worry that nobody seems to be maintaining it based on issues/pr's I looked at.

<comment><USER18> Scratch that - I see fresh commits - will give it another spin!

</comment></comment></comment></comment><comment><USER19> Other options available:

\- CockroachDB 1.1

\- AWS RDS Aurora PostgreSQL-compatibility

[1] [https://news.ycombinator.com/item?id=15458900](https://news.ycombinator.com/item?id=15458900) [2] [https://news.ycombinator.com/item?id=13072861](https://news.ycombinator.com/item?id=13072861)

<comment><USER20> CockroachDB is a really cool product, but I think by all accounts it is still nowhere near performant enough to be used in production.

<comment> forgot-my-pw Alternative is Google Spanner. Also still new, but already in GA for 5 months now.

<comment><USER21> The spanner setup that can match PG on a high end x86 cluster will run way over 100K month

</comment></comment></comment></comment><comment><USER22> We're moving away from gitlab because the performance is too much to bear. We're hearing "aaaaaarg I really hate gitlab" a few times a day.

The pipeline pages takes 7 seconds to load. It's really bad. We stopped using issues because just listing them was a chore. Pushing 1 file takes at least 30 seconds to 1 minute.

Anyways, the work falls on me to replace it. We're going with Phabricator and Jenkins.

<comment><USER23> Is that self-hosted Gitlab or their hosted version? Would you mind sharing details on your setup? Just curious as we're currently building out our own setup.

<comment><USER22> Right now:

\- Self hosted Gitlab CE

\- Gitlab runners in pre-emptibles nodes with autoscaling

Future (almost done, still in testing phase):

\- Phabricator (for git, code review and issues)

\- Jenkins ("lightweight" CI, more details at the bottom)

\- Google Cloud Container Builder ("Heavyweight" CI)

Our main repository is in Phabricator, but it has mirroring to Google Cloud Repository (for backup and faster heavyweight builds)

On commits, we run the lightweight CI in Jenkins. What I mean by lightweight is that, there's only a few runners and what it does is check what has changed. We have a mono-repository with over 30 microservices, rebuilding all of them is a waste of time and resources, so we have scripts and easy to use manifests that defines what to do. We also have a lot of small CI checks that is just not worth it to run externally, like linting.

Here's what our homegrown manifests looks like

      build_graphql:
        when_changed:
          - <USER2>
          - node_libraries/player-graphdb
          - node_libraries/player-auth
          - node_libraries/player-models
        script_build: graphql/cloudbuild.yaml
        script_skip: graphql/cloudbuild-skip.yaml

The script skip takes the last successful build and retag the docker images to the current one. The build script sends the instructions to Google Cloud Container Builder to build the docker image, run the tests, and push it. We use Cloud Container Builder because managing those CI servers really sucks.

Let me know what else you want to know.

</comment></comment></comment><comment><USER24> Sounds like a design flaw... adding a centralized database to augment a decentralized version control system.

Comments, issues, pull requests, wiki, etc... should all be first class Git objects. I should be able to create pull requests while offline and push them up when I'm back online.

<comment><USER25> Issues can be in dvcs with something like BugsEveryehere.

Git _has_ native "pull requests" it's just not web based, it's email based.

A good wiki is markdown docs in a git/hg repo and something like Gollum to serve web viewers.

So - in a way those things can all be handled as regular content in git/hg.

</comment></comment><comment><USER26> > A side effect of using transaction pooling is that you cannot use prepared > statements, as the PREPARE and EXECUTE commands may end up running in > different connections.

There is a way to use prepared statements, it's called pre_prepare [0]. You still need to tweak your code though to not execute PREPARE directly but rather put statements in special table, but you do that only once, and after that you EXECUTE like before. Another minor (in my view) caveat is that you need to reset pgbouncer↔postgres connections each time you update that statements table.

[0] [https://github.com/dimitri/preprepare](https://github.com/dimitri/preprepare)

</comment><comment><USER27> Simple -- just clean the database from time to time [1].

BTW, I like how the banner on the bottom of [1] wants me to "Try GitLab Enterprise Edition risk-free for 30 days." currently.

[1] [https://about.gitlab.com/2017/02/10/postmortem-of- database-o...](https://about.gitlab.com/2017/02/10/postmortem-of-database- outage-of-january-31/)

</comment><comment><USER28> > Sharding would also affect the process of contributing changes to GitLab as > every contributor would now have to make sure a shard key is present in > their queries.

Why is this a problem? If you need to make a change to improve scalability, it doesn't seem unreasonable to ask contributors to follow code and performance guidelines.

</comment><comment><USER29> Looking through the public Grafana dashboards I noticed that the database backups page is not showing any data points ([http://monitor.gitlab.net/dashboard/db/backups](http://monitor.gitlab.net/dashboard/db/backups)). Is this meant to a public page?

<comment><USER4> This is meant to be a public page. It loads async. After 3 seconds I see the following [https://www.dropbox.com/s/5mt5f77e60yogk1/Screenshot%202017-...](https://www.dropbox.com/s/5mt5f77e60yogk1/Screenshot%202017-10-30%2015.50.05.png?dl=0)

</comment></comment><comment><USER30> Not seeing any mention of using something like Memcached (or anything similar) in front of PG for caching.

Is that because it's already been done and not in scope?

Asking because for (at least) web applications, using Memcached (or similar) can significantly reduce the load on the backend database. :)

<comment><USER4> We have a lot of caching in Redis, but that connects with the Rails app, not in front of PG. This way it can store the result (for example a page fragment) instead of just the db query that is part of it.

<comment><USER30> Cool, just making sure. :)

</comment></comment></comment><comment><USER31> It's interesting to see what parts of the product GitLab is struggling with compared to GitHub, e.g. [https://githubengineering.com/stretching- spokes/](https://githubengineering.com/stretching-spokes/)

<comment><USER4> I'm very impressed with the level GitHub database testing is at [https://githubengineering.com/mysql-testing-automation-at- gi...](https://githubengineering.com/mysql-testing-automation-at-github/) We still got a lot of work ahead of us. But Yorick and Greg their work has been outstanding.

Regarding Spokes we're not planning something like that at this time. From [https://gitlab.com/gitlab-org/gitaly/issues/650](https://gitlab.com/gitlab- org/gitaly/issues/650) "GitHub eventually moved to Spokes [https://githubengineering.com/building-resilience-in- spokes/](https://githubengineering.com/building-resilience-in-spokes/) that had multiple fileservers for the same repository. We will probably not need that. We run networked storage in the cloud where the public cloud provider is responsible for the redundancy of the files. To do cross availability zone failover we'll use GitLab Geo."

<comment><USER31> Thanks for chiming in, sytse and YorickPeterse!

Re: misuse, I remember this incident [https://news.ycombinator.com/item?id=11245652](https://news.ycombinator.com/item?id=11245652) made me chuckle and cry a little. If you build it, it will be misused.

</comment></comment></comment><comment><USER32> Did you consider trying NewSQL DBs that would let you scale horizontally with much less complexity? Curious to see if the performance is comparable yet.

<comment><USER13> No, because it probably would have required a complete rewrite of GitLab to make it work.

<comment><USER33> User mentioned "trying". With an app that gets this much use, it might be reasonable to take a small piece and fork all writes to both the current + an experimental DB and then A/B test the reads. If this type of abstraction requires "a complete rewrite of GitLab" for "trying NewSQL DBs" then something is amiss.

<comment><USER34> Given that Gitlab is open source, adding a NewSQL DB into the mix would be pushing that dependency to downstream users, who would then have to take on the maintenance burden. It may end up being the right call from a performance standpoint but just too complex to ask users to manage.

<comment><USER33> This burden already exists with dependencies (and we should probably say downstream "admins" instead of "users" to clarify). A self-hosted app should not be hamstrung by its initial dependency choices. If tested and implemented properly it can go from optional experimental flag, to multiple-storage- backends-supported abstraction, to [maybe] even we're-migrating-to-new- storage-backend, etc. The primary downside in these cases is the flexibility often provided to plugin and extension writers which marries them to early tech choices. The more admins that demand low change count on forward movement are indirectly the reason many companies choose to offer SaaS only where the visibility and flexibility are removed.

Note, I'm not saying this is the case with GitLab at all and I assume Postgres will remain the primary choice for most of their DB uses in perpetuity, but for some use cases there is better tech to use and the only reason to say no shouldn't just be a dependency addition (though it is often a good reason among many to say no).

</comment></comment></comment></comment></comment><comment><USER10> Apparently I should stop using gitlab.com By choosing to not shard your data you have basically put a cap on scalability... or at least one which will be reached faster than if you were to decide to shard.

<comment><USER35> Web service architectures are like shoes. You want what fits now, not what you hope will fit later.

Chances are very good that preemptive scaling will just result in unnecessary development overhead and an architecture/set of abstractions that _still_ fail to handle the expected load since they weren't battle tested against it.

</comment></comment><comment><USER36> I am a bit disappointed at this article and the hierarchy of how to optimise postgres. As a MySQL specialist that has worked on a few Postgres projects, I can tell you that Postgres has a VERY RICH toolkit to speed things up. Between optimising queries and load balancing, you have indexing, partitioning.. (Postgres even allows you to index EACH partition differently) and triggers to do summary/rollup tables for you.

Its just that Postgres may need a little more coding and is less automatic than MySQL that people shy away from it..

I do like the time and information spent about spreading load accross database servers and not nearly enough people are considering splitting reads and writes from a master/slave setup.

Its just that its frustrating to me that engineers view databases as some sort of mysterious black box that you need to solve by working around it or replacing it entirely.

</comment><comment><USER37> > 2\. Use a connection pooler ...

I had my jaw dropped on that point -- they didn't use connection pooling? WTF? I'm seeing connection pools on pretty much any project I've worked on, even under 1/req/day load, it's pretty much "must have" since 90s. And it's really just a library (like dbcp2) and a few lines of config, with almost no drawbacks.

I cannot believe engineers at GitLab level are so bad at, well, engineering. I'm losing faith in their produce, please show me that I'm wrong.

<comment><USER13> We've had connection pooling on the client side since forever, just like any other Rails project. This article specifically talks about server side pooling.

</comment></comment><comment><USER38> > A side effect of using transaction pooling is that you cannot use prepared > statements, as the PREPARE and EXECUTE commands may end up running in > different connections; producing errors as a result.

Uhhh... WHAT? I'll simply sit by and watch for SQL injection vulnerabilities resulting from this change. Either you have a _very_ good SQL query writer engine with rock solid escaping or you _will_ get pwned by this.

<comment><USER13> You can have perfectly valid input escaping without needing prepared statements. Many ORMs handle this without the need for PREPARE/EXECUTE.

<comment><USER38> > You can have perfectly valid input escaping without needing prepared > statements.

People have bypassed this far too often for my taste, and there is a reason why SQL (and other injections) are Top 1 of OWASP (1st place actually, both in RC1 and 2 of 2017, but also in the 2013 edition).

Never, ever trust any form of escaping or it _will_ bite you. Hard.

<comment><USER39> > Never, ever trust any form of escaping or it will bite you. Hard.

This is an absurd dogmatic statement. Everybody trusts a multitude of escapings everywhere every day all over the web without even knowing it.</comment></comment></comment></comment>


Kelly Sommers kellabyte Nobody believes me I 3x'd my salary by leaving .NET - <AUTHOR> https://twitter.com/kellabyte/status/1219035096348610561 <comment><USER2> Impossibly vague statement. What was her salary before? What is it now? How long did the shift take place? Where is she in her career?

You get the idea. Far too many variables to just say the 3x increase was just because she dropped dotnet...or even played any part.

There are plenty of people writing dotnet code with some making 3x more than others.</comment>


Econ 101 for Presidential Candidates - <AUTHOR> http://www.cato.org/publications/commentary/econ-101-presidential-candidates <comment><USER2> I find people often over simplify things like this. Sure, free trade is good. The issues is that we've never had a free trade deal. Our "free trade deals" come with thousands of caveats where we are the partner country agree to all kinds of rules, regulations, etc. I think true free trade would be the best, but failing that I think straight-forward protectionism is better than what we get now with "free trade".

<comment><USER3> You can't have trade without some kind of assumptions/rules of the game. Even a simple trade holds assumptions/rules your not going to come and take back your widget after we've traded. that you didn't lie about it. or switch it out at the last minute. As the agreements get more complicated the more rules you need.</comment></comment>


Evolution has given humans a huge advantage over most other animals: middle age - <AUTHOR> http://www.washingtonpost.com/national/health-science/evolution-has-given-humans-a-huge-advantage-over-most-other-animals-middle-age/2012/03/12/gIQAtVnccS_print.html <comment><USER2> Larry Niven's _Protector_ extrapolates this role of middle-aged grandparents. <http://en.wikipedia.org/wiki/Pak_Protector#Narrative_purpose>

Unfortunately, the accelerating returns of technology (particularly in computers) means there's limits to imparting specific technical knowledge to the next generation. But some things change more slowly that can be: human nature, emotions, communication, competitive advantage, mathematics, and general problem-solving techniques.

Wisdom, with long-term value, is worth acquiring.

<comment><USER3> Grandchildren share 1/4 of your DNA, great-grandchildren share 1/8, great- great-grandchildren share 1/16. From an evolutionary point of view, there are diminishing returns to helping your descendants, even if your knowledge remains relevant.

<comment><USER4> _From an evolutionary point of view, there are diminishing returns to helping your descendants, even if your knowledge remains relevant._

That is true in a sense, but it assumes that the only thing that matters are the physical characteristics and basic instincts propagated by DNA. Arguably the thing that makes us the dominant species on the planet today is our ability to propagate our way of thinking, our collective knowledge and wisdom, our goals and aspirations.

These influences also get diluted over time in each individual descendant, of course, but on the other hand to some extent these are more about the culture and the collective group rather than any individual. The number of individuals influenced by a particular ancestor tends to increase with each generation. As a species we actively select our mates based partly on compatibility in these respects. And as the article itself notes, there is a role for older generations in nurturing members of younger generations other than just their own direct offspring, which would tend to include influencing the offspring's mates and further generations.

So I'm not sure it's entirely fair to say that there are diminishing returns with each generation. It is more that the influence becomes more spread out and tends to affect culture rather than any one individual, and those whose ideas are recognised as benefitting the wider group more effectively will tend to retain more influence for longer.

</comment></comment></comment><comment><USER5> It is true, as the article correctly points out, people often misinterpret average life expectancy, because they don't fully take on board the impact of child mortality.

That being said, the "grandmother hypothesis", that our ancestors gained evolutionary advantage in taking care of grandchildren seems to be a matter of some contention still, with some experts taking the view that no one survived to be grandparents.

<comment><USER6> > some experts taking the view that no one survived to be grandparents.

Wouldn't this imply that most parents, then, died before they saw their children reach puberty? (I admit to knowing nothing about the social dynamics of ye olde anciente humanse; I'm just assuming that once you're of biological age, you become incredibly likely to get knocked up.)</comment></comment>


India Hiring Fair (Sanjose, CA : 12th & 13th Nov) - <AUTHOR> http://indiahiring.shine.com/californiafair/

<comment><USER2> I would upvote a link to something with more information</comment>


Ask HN: How can I prepare myself for a full-time remote? - <AUTHOR> I hold a CS degree and I have about 2 years of experience as an SE (backend). In the next two years, I would like to get a remote job while living/traveling around Africa. How can I best prepare myself to land a remote job? Thanks <comment><USER2> Being remote requires a lot of self discipline, you have to make sure you focus yourself everyday. I'd say the newer you are to the profession the harder it is to go remote, 4-5 years of experience is probably good since you'll be pretty solid skill wise and know your limits. When you do go remote, there are different routes to take. If you are going with essentially the nomad lifestyle (very worthwhile IMO), make sure you choose a job that you are 100% solid in the skill set needed. e.g. don't stretch yourself to take on a harder role as you will regret it and not get to enjoy the travel. If you are stable and living someplace you plan to be then the math changes and stretching or taking on a harder role is fine.

There are probably a ton of things to share about going remote, but that would be my initial feedback.

context: I have led, participated and been a team member on remote teams for over 10 years now. The teams have spanned the globe literally, right now I am on a team spread across 2 US States and India.</comment>


From productivity porn to mindful productivity - <AUTHOR> https://nesslabs.com/productivity-porn <comment><USER2> Use of "mindful" in the title is suggestive that this is softcore productivity porn. (No actual climax)

That word drives me nuts. I know a guy who can't go more than a few sentences without using the word "mindful" who is always crashing into things and walking into open pits.

Now maybe the neurotypical with "no inner life" has a desperate need to feel "full" but I've always been anxious and emptying the "s2t" out of my mind helps me feel better and be more productive.</comment>


Endangered Species: Young U.S. Entrepreneurs - <AUTHOR> http://www.wsj.com/articles/endangered-species-young-u-s-entrepreneurs-1420246116

<comment><USER2> So young people are deeply indebted at an age when we're also likely to lack skills, connections, and insights into open market niches... and it becomes strange or worrisome that we don't start businesses?</comment>


EC2 Micro Instance as a Remote Bittorrent Client - <AUTHOR> http://mwmanning.com/2010/11/29/ec2-micro-instance-as-a-remote-bittorrent-client/ <comment><USER2> The _only_ thing I would ever use this for would be legal files shared with friends - I'm not sure how often Amazon ripples their IP addresses, but I'm sure the RIAA/MPAA would have a field day with an IP address linked to a billing account in the US.

<comment><USER3> Use a prepaid credit card; use a payphone for the phone call confirmation.

<comment><USER4> sounds like a lot of work to just get around buying content :)

<comment><USER5> Most of the content that is worth pirating can't be brought at all. Many US tv shows are two or more years behind outside the US.

</comment></comment></comment></comment><comment><USER6> Ouch. What would that cost for the bandwidth?

I looked into using cloud instances for file sharing (legal files, of course) and decided it would cost too much for what it was worth to me.

<comment><USER7> Yeah, with the free usage tier you get 30GB data transfer (15GB in + 15GB out) included but that would soon run out.

<comment><USER8> From what I see Data Transfer is $0.10 per GB in $0.15 out <http://aws.amazon.com/ec2/pricing/>

Assume you seed 50% more than you use 1GB (10c in) + (15c * 1.5 seeding) + (15c download) = ~47.5 cents per GB ouch.

However, a blue ray rip is something like 8.5GB * 47.5 cents = 4.04$ which is not that bad. Assuming you get the actual content that you want, don't end up in jail etc, it's still cheaper than HD PPV let alone buying the Disk.

Edit: I chose a BR rip as the lowest value content per GB, for most other uses it seems fairly reasonable.

<comment><USER9> Agreed, it's hard to understand this making any financial sense. If you're looking to do substantial seeding there are low cost unlimited bandwidth vps' that specialize there. If you're simply looking for a delayed but fast torrent download there are services that essentially transfer torrents to files on their server for you to download. And if you simply want to download things fast & on demand, there is always nntp.

Imagine if someone failed to set a max seeding amount and forgot about their EC2 instance for a few days or weeks. It'd be easy to transfer hundreds or thousands of gigabytes (1TB out = $150)

AWS is one of the most expensive ways to buy bandwidth out there. To do worse you have to look at high end dedicated & managed colos and bad deals at big CDNs.

<comment><USER10> > If you're simply looking for a delayed but fast torrent download there are > services that essentially transfer torrents to files on their server for you > to download.

Do you have a links to any services like that? The fact that I couldn't find any is why I decided to try this at all.

<comment><USER9> <http://www.furk.net/>

<http://www.peerharbor.com/>

I haven't used these services and can't vouch etc. etc.

(It's notable that two others I was thinking of, httptorrents.com and imageshack.us have either gone out of business or have cancelled the service. So predictably it's a pretty spotty market).

And, at the risk of breaking rule #1, have you heard of usenet?

<comment><USER10> Thanks.

> And, at the risk of breaking rule #1, have you heard of usenet?

Of course :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> I'm surprised home routers don't have bittorrent clients built in. I expect most people leave them on 24/7 as it is.

<comment><USER12> There's a number that do... and apparently you can install transmission as an add-on to dd-wrt:

[http://www.dd- wrt.com/wiki/index.php/Optware#Installing_torr...](http://www.dd- wrt.com/wiki/index.php/Optware#Installing_torrent_client)

<comment><USER13> OpenWRT <http://openwrt.org/> also has a number of clients (and even a tracker) ready to be installed in the repository:

      # opkg list | grep -i torrent
      btpd - 0.15-1 - BTPD is a bittorrent client consisting of a daemon and client commands,
       The daemon is capable of running several torrents simultaneously and only uses one tcp port.
      cbtt - 20060727-1 - Bittorrent tracker
      cbtt-mysql - 20060727-1 - Bittorrent tracker (with mysql support)
      ctorrent - dnh3.3.2-5 - CTorrent is a BitTorrent client written in the C programming language,
      ctorrent-nossl - dnh3.3.2-5 - CTorrent is a BitTorrent client written in the C programming language,
      libtorrent - 0.12.6_r1130-3 - LibTorrent is a BitTorrent library written in C++ for *nix, with a focus
      rtorrent - 0.8.6_r1130-1 - rTorrent is a BitTorrent client for ncurses, using the libtorrent library.
      transmission-daemon - 1.92-1 - Transmission is a simple BitTorrent client.

</comment></comment></comment><comment><USER14> That's cool.

I did the same thing for Minecraft SMP (passing 512M instead of 1024M for memory usage).

</comment><comment><USER15> This is cool, you could also use S3 to help you seed if you don't want to keep your computer on all the time. S3 allows you to use torrents by adding ?torrent to the end of your urls. The data transfer is treated the same as regular S3 data access.

<comment><USER16> Using S3 torrents wouldn't help you seed, since you'd have to use the same tracker (original) as everyone else.

<comment><USER15> Couldn't you just add the tracker url from the s3 torrent to the torrent file you generate?

<comment><USER16> Sure, but you then need to get everyone else downloading the (original) torrent to download and use your new torrent. The new amazon torrent might split the file into differently sized chunks compared to the original. In short, you'd need to get all original downloads/uploaded to use your new torrent.

</comment></comment></comment></comment><comment><USER17> Your safest and cheapest bet is to get a seedbox in France. <http://dediseedbox.com>

<comment><USER18> Interesting. I wonder how they make it so cheap, especially considering the traffic.

<comment><USER19> But making all of their payments non-refundable and reserving the right to terminate your account for any reason (including posting "defamatory" comments online about them!).

See their Terms of Service at [http://dediseedbox.com/clients/knowledgebase.php?action=disp...](http://dediseedbox.com/clients/knowledgebase.php?action=displayarticle&id=8)

</comment></comment></comment><comment><USER20> This is a cool idea, but if I used it with the amount of data I use at home, I would be spending upwards of $500 a month.

</comment><comment><USER21> recently discovered aria2c, it's an excellent client for bittorrents files, would be run great on any EC2 instance.

</comment><comment><USER22> IMHO, Exactly this kind of abuse is something will lead to closing evaluation period provider by amazon.

<comment><USER10> How does this constitute abuse?</comment></comment>


Ask HN: Founders of startups, do you expect equity to be enticing? - <AUTHOR> I've worked at a plethora of startups by now, and even the one that went public, I've still made almost no money on given my equity. It's sort of what I expected, but even the companies that I really truly believed would do well, still didn't because of some random other issue (usually incompetent upper management/founders).<p>I ask, why do you expect a potential hire to appreciate X% of nothing when it's well known that most VC's that invest in your company know that you have a 99% chance of failing anyway, and thus diversify their portfolios with many startups to increase the risk of success. Why are employees expected to pretend anything otherwise? I'm so beyond the eyerolling phase of hearing that they'll pay you .75X of your current salary but give you equity to make up for that, it's just exhausting at this point, to be honest. <comment><USER2> From my experience, options for employees have NO VALUE at all. This is not because VCs and C-Suite don't think that the company will make it, but because they arrange things that way to maximize payouts for themselves.

Here is how this happens. Let's say for argument sake, there are 20 million shares or options floating about. Let's say you own 500k of them. Now Mega_Company and your company New_and_Cool announce a 20 million dollar purchase. You think your might be in for a $300k windfall (or more), but here is what is likely going to happen.

VCs will have a preferred stake that must be satisfied as part of any buyout. Usually this preferred stake will require satisfaction that is a significant multiple of the at risk position. The VC put up a million bucks but now requires the first ten million as part of any buyout.

Next, C-Suite will arrange direct compensation and retention packages outside of the context of common shares/options as part of the buyout. The CEO will get a five million dollar direct payout while the rest of the C-Suite and Board split an addition 5 million.

Then low and behold, your shares/options are worth... Nothing. Imagine that.

An equity position in New_and_Cool is snake oil and should be evaluated as such. Yes, I know there are counter examples and "so and so's secretary" is now living on a private island someplace thanks to their options. In my opinion, these gilded accounts only serve to make the gullible more susceptible.

</comment><comment><USER3> Simply put, startups don't have the budget to match higher salaries.

I did a startup setting aside a large portion of shares for employees, but nobody asked for it. It was easier to just pay more - large businesses were paying $1000/month for an mid level programmer. So it was just easier to offer $1000/month to juniors.

However early stage VCs offered a rate of about 1% for 2 months salary. So sometimes it's easier to offer that early stage employee the 1%. Some have enough money in the bank that they'd like to invest in something.

</comment><comment><USER4> It's an exhausting proposition to you, but not to others, probably folks new to our industry, who haven't carefully considered the tradeoffs.

<comment><AUTHOR> and i don't disagree that it's OK for new grads to work for startups to learn a lot. but a lot of startups hire for senior engineers/staff engineers</comment></comment>