Swing: A new open-source e-commerce platform written in Swift - <AUTHOR> https://medium.com/the-theam-journey/introducing-the-swing-project-e5ccc73a2c7f#.4568kvqft <comment><AUTHOR> At my company we're starting to build a next generation open-source e-commerce platform, and we believe it makes a lot of sense to use Swift as the main programming language given its popularity, performance, and functional spirit. We know that this won't be an easy road given that using Swift in the server is new, but that makes the quest even more interesting for us, and we think that strong initiatives like this one can benefit the overall Swift community. We'd love to get some feedback about this idea, and we're very welcoming to anyone who's interested in helping.

<comment><USER2> Generally it seems that you know what you are doing, so the idea looks good. However I've got few comments:

1\. Lack of schema is a horrible idea, especially at the beginning. It may be necessary in some selected places, but if that's your approach to data design you'll end up with unmaintainable mess that keeps tripping over differences in data interpretation and inserting garbage due to trivial mistakes. Then you'll be the one explaining to your clients that images are not being displayed because "imgaeid":"45" was found in place of expected "imageId": 45. It doesn't simplify anything.

2\. I am not sure what resources do you have, but introducing innovative (and largely untested) language and various aspects of architecture _all at once_ is very ambitious. Swift looks nice (though I prefer Rust), but both language and its libraries will be moving target for years, and that's pain even for languages that were well designed. Being event driven looks nice too... but I am not sure if anyone can figure out if its better to be 100% ED or maybe only 45 is just right. This might be to much to chew on.

<comment><AUTHOR> First of all, thanks for your feedback, I really appreciate it.

About your first point, I essentially agree with you. While I talk about "schemaless" data, I'm not really thinking on leaving it uncontrolled or letting each service write whatever they want to the DB. I'm thinking more on avoiding restrictions in the schema in order to extend the system for a specific retailer needs. We're planning to use a strongly typed programming language aiming for application-level data consistency, and we'll probably need to define a shared set of type definitions for the whole system and specially for the message passing protocol; but we also want to make it extremely easy to change so it can be adapted for the specific needs of each business.

About the second point, yes, we definitely know this is a very ambitious project, but it can be broken into smaller projects that are more approachable, so we can target a smaller objective and re-evaluate after the first iterations. We won't achieve everything we want in a single step, but I'm sure we can walk in that direction. Here's where I think the open source approach can make a difference if we catch the attention of enough people.

Choosing Swift is risky too, but we're really happy with that programming language and we believe that the only way to make it evolve to be a serious backend language is that some crazy people like us make a strong bet for it. And about the event-driven architecture, we believe that there are some specific problems in eCommerce that are specially suitable for this approach, but it's not definitely a silver bullet. For instance, the Magento core triggers all kind of events when different business objects are updated and we've found it to be really useful when you need to create an extension without touching the core. We see on events a way to decouple logic, so one module can trigger events without knowing who will be listening and the listeners can receive information without knowing who generated it.

So yes! This is crazy, but after many years on this field using the same tools, we're really excited with the idea of bringing it into life! :-)

</comment></comment></comment><comment><USER3> First of all, very interesting project! Had to do some small Magento jobs in the past and it was my worst experience as developer in my career so far. Do you plan to build it on top of one of the existing Swift frameworks like Vapor, Kitura, etc?

Apart from that, i don't think the name of the project is a good choice. I guess 9 out of 10 Google results for a problem with your project will return information about the Swing Java Framework.

<comment><AUTHOR> Yes, Magento tends to trigger some strong feelings :-D

We're still making some tests and prototypes, but Kitura is a very strong candidate to be a basic building block of our system.

We've received a lot of feedback about the name, we'll give it a thought.</comment></comment>


Comprehending Ringads (2016) [pdf] - <AUTHOR> http://www.cs.ox.ac.uk/jeremy.gibbons/publications/ringads.pdf <comment><USER2> The author is developing a theory of expressions (generalizations of list comprehensions) operating on 'ringads', which are monads with a few additional operations and laws.

The academic style probably blurs the fact that this has very deep practical applications, as it provides a framework that permits to write down database joins and OLAP multidimensional queries (including aggregation) in multiple ways that can be considered equivalent through application of monad and ringad laws. This in turn can enable us to interpret such expressions efficiently, by picking the one that corresponds to a desirable execution plan.

Another paper that I would consider related is the one that points out how comprehension expressions can express SQL features like 'order by' and 'group by', namely 'Comprehensive comprehensions' by Wadler and Peyton-Jones.

<comment><USER3> > The academic style probably blurs the fact that this has very deep practical > applications

Whilst this is certainly true, and worth pointing out, it's also rather a shame. The paper's abstract says exactly this, in quite a clear way, but I think many people wouldn't even make it as far as reading the abstract :(

> _List comprehensions_ are a widely used programming construct, in languages > such as Haskell and Python and in technologies such as Microsoft's Language > Integrated Query. They generalize from lists to arbitrary _monads_ , > yielding a lightweight idiom of imperative programming in a pure functional > language. When the monad has the additional structure of a so-called > _ringad_ , corresponding to 'empty' and 'union' operations, then it can be > seen as some kind of collection type, and the comprehension notation can > also be extended to incorporate _aggregations_. Ringad comprehensions > represent a convenient notation for expressing _database queries_. The > ringad structure alone does not provide a good explanation or and efficient > implementation of relational _joins_ ; but by allowing heterogeneous > comprehensions, involving both bag and indexed table ringads, we show how to > accommodate these too.

</comment></comment><comment><USER4> Not related to the content, but I always enjoy looking at beautifully typeset LaTeX documents.</comment>


Work on stuff that matters - Part One - Tim O'Reilly - <AUTHOR> http://www.start-up.co.nz/work-on-stuff-that-matters-part-one-tim-o-reilly

<comment><USER2> There doesn't seem to be anything here that wasn't covered in his blog a week ago:

<http://news.ycombinator.com/item?id=429231>

</comment><comment><USER3> With all due respect to the man, but thanks Captain obvious! Tell me something I don't know.</comment>


Stop Wasting Money on Unnecessary Monthly Subscriptions - <AUTHOR> https://www.wsj.com/articles/stop-wasting-money-on-unnecessary-monthly-subscriptions-11557331377 <comment><USER2> How ironic. [https://imgur.com/a/yeOOppW](https://imgur.com/a/yeOOppW)</comment>


Librarians prepare to take a harder line with publishers - <AUTHOR> https://www.insidehighered.com/news/2019/03/27/librarians-prepare-take-harder-line-publishers <comment><USER2> If you loudly have to announce your harder line /stance on something, you are showing how little internal stability your group/alliance/cartel/whatever has.

Because what you are trying to accomplish, is basically to force any strike- breakers in your alliance to be publicly shamed.

Those who really drive a hard line, are most of the time unheard of, they just come, negotiate you down into the ground and you have to smile and wave. Cause atleast the factory you have does not make minus and earn 1 cent a hour.</comment>


CSS Grid Generator - <AUTHOR> https://cssgrid-generator.netlify.com/ <comment><USER2> CSS grid, where were you 15+ years ago?

CSS was such a mess for so long because of the complete lack of a consistent easy-to-understand layout mechanism. This has spawned the creation of countless workarounds and javascript gizmos, JUST SO you could stick something on a webpage where you want to go and have it behave in a sane way under resize and different browsers.

I welcome CSS grid, it reminds me of the sane layout managers we're used to from writing desktop applications. It makes me wonder why we had to wait so long for this functionality to be part of the CSS standard.

As for the generator, I think it's nice. Most folks will get the hang of banging out arbitrary css grid layouts very quickly. This tool will accelerate that for many.

<comment> the-dude Because HTML was a document format, not an application container. Try positioning something exact in Latex.

<comment><USER3> I am not going to start using Latex, but sadly this:

...

still works just as well for layout than anything I've seen using CSS/HTML. Even worse, all I want to do is tab over from some text:

      (SUB X Y)            subtract

<comment><USER4> Serious question: Why can't I use tables and define cells sizes in terms of vh and vw?

<comment><USER5> Tables are totally fine if what you're visualising is data that belongs in a table. Tables for layouts are not semantically correct and breaks a lot of expected behaviour for people with assisting technology.

Html shouldn't consider layouting, that's for css to do.

</comment></comment></comment></comment></comment><comment><USER6> Nice. There are a few bugs:

\- Clicking multiple times in a box causes an unreadable mess

\- If the code gets too long the "Please may I have some code" model disappears off the page, including the only way to close it ("Done" button). Clicking outside the model doesn't close it nor does the escape key.

\- If you set rows or columns to -1 it generates a display artifact which remains indefinitely.

\- "What does this project do?" model cannot be closed or read fully (outside page limits).

I'm on Chrome retail latest (74.0.3729.169) with 125% Windows DPI.

<comment><USER7> 125% DPI on windows, and similarly, large font on Android systemwide, regularly cause problems.

Should software accomodate those settings or is that up to the user? What's the HN consensus oh that?

<comment><USER8> If 125% DPI is a standard for your users, of course you should accommodate for it when you can.

UI development has always been developing for the most common denominator and then the best you can make it for the most amount of users.

If the user has a manual zoom in the browser, there's not much you can do from a developer perspective.

<comment><USER9> Is there any data on that? I am interested how many people use different % DPI on desktop, what % of the people use scaled font sizes on iOS and Android.

<comment><USER8> There's plenty of resources out there for 'Browser Stats'. it would really depend on the projects target audience. I have not personally seen any numbers on DPI%, but I wouldn't be surprised if they exist.

You can achieve some flexibility in layout for device, resolution and pixel ratio constraints using CSS media queries, it just depends how granular you want to go and how much time it takes to implement all the use cases from a business perspective (from experience)

</comment></comment></comment></comment></comment><comment><USER10> All I needed for my grids was display: table with row wrapping. I got magazine layouts instead with worse vertical alignment of cell content. CSS Grid is cool and does previously impossible things, but I wish the industry fixed old techniques (where is 'gap' in flexbox?), instead of jumping on the next big thing.

<comment><USER11> As far as I can tell, flexbox grid-gap might be coming soon. It's apparently currently supported only in Firefox.

[https://developer.mozilla.org/en- US/docs/Web/CSS/gap#Browser...](https://developer.mozilla.org/en- US/docs/Web/CSS/gap#Browser_compatibility)

</comment></comment><comment><USER12> Absolutely love Sarah Drasner's projects! Attention to detail and great sense of humour, thank you!

</comment><comment><USER13> Is there a reason to use a CSS grid and not a
? Seems like web devs have been crying about a missing feature for a long time when it's already existed. But I only know the basics of web development and don't know best practices, so I don't know why devs would be insisting on using CSS to create a grid when an HTML table works just fine.

<comment><USER14> The main reason is mobile responsiveness. With
, you need to conform to all the table related elements like
, , etc, which is fine on desktop.

However when you want the table to collapse vertically instead of staying horizontal on mobile, trying to overwrite the browser defaults of these elements with CSS is a pain and not recommended. Using grid just makes it easier to change the display in CSS.

You could argue that using CSS display:table works too, but display:grid just brings a lot of other niceties for controlling how you want the page to look.

<comment><USER15> You are correct but many decried the use of tables for layout as far back as 1999.

</comment></comment></comment><comment><USER16> Neat!

One comment though: The generated code contains nested selectors, which are not valid CSS.

</comment><comment><USER17> It sadly doesn't solve the problem with literally all and every grid generator out there - no proper support for spans and areas. The biggest power of grid is the free-form layouting it provides - it just gets lost in these simplistic editors.

<comment><USER18> I consider your comment to be constructive feedback.

The real power of grid is the features you speak of plus the alignment features. With this editor you have a long way to go to create useful responsive layouts with niceties such as implicit grids.

I am coming to the conclusion that you don't need a tool to make a grid for you, that knowing how to write the CSS is where it is at.

The problem is that CSS Grid is a new mindset, if you have been brought up to think in terms of things like '960 pixel wide' grids or how print newspapers do grids then you are not going to fully get it.

I also do not like it how this example perpetuates the div element. According to the specification the 'div' element is not to be used except for in a last resort.

The CSS Grid way of working means there is no need for div wrappers. Centering and aligning content can be done without the div. Again this requires new thinking and people used to slapping divs everywhere with a multitude of class attributes are still thinking they are working with a framework.

If you look at the code for the page rather than the generated code there are a few off-spec 'crimes' that are not needed with CSS Grid. The 'main' element should be in the 'body' as a direct descendent, with this page it nested inside a section that is nested inside a div. This is then styled with flex layout rather than CSS Grid. The page uses complicated calc hacks rather than CSS Grid. The document structure is also quite wrong, the side panel is not an 'aside' outside of the 'main' content.

For this page I would go with '1fr auto' to define the columns. The sidepanel would then take the space it needs and the grid take up the rest. No 'calc' sums needed in the CSS.

Now if you type 'auto' in one of the columns (instead of 1fr) then you get some patronising message saying "Must use real CSS units, goofball". The thing is that for CSS Grid, 'auto' is what you want to be typing once or twice in every project, not being able to use 'auto' (because apparently only goofballs do it) indicates to me that this is quite a beginner project with a lot more work needed.

CSS Grid enables you to strip out all presentational markup from a page, it can be pure content using the HTML5 content sectioning elements. This is a fundamentally different way of working than the hacks of the past. If you wish to further bloat hacky HTML then this tool will help you do it, it won't challenge ingrained attitudes and get people to the promised land of concise, unbloated HTML that separates the concerns.

</comment></comment><comment><USER19> What do you all think of this in contrast to Firefox's Grid Inspector? More useful? Less useful? Serving different purposes?

[https://developer.mozilla.org/en- US/docs/Tools/Page_Inspecto...](https://developer.mozilla.org/en- US/docs/Tools/Page_Inspector/How_to/Examine_grid_layouts)

To put my question another way, I'm trying to figure out where people really need help while creating Grid layouts. Whether we need tools to deeply inspect existing layouts, or whether we just need tools to quickly generate layouts (like Sarah's tool).

Disclosure: I work on Chrome DevTools.

<comment><USER20> I am a dev so I would not use a generator. A tool which gives me instant feedback on the code I write is much more useful. So Firefox CSS Grid and Flexbox inspectors are extremely useful.

<comment><USER21> _I am a dev so I would not use a generator._

Every good dev I've ever worked with uses generators of some sort or another. There are entire languages dedicated to code generation and scaffolding apps (Cargo, Rails, Yeoman etc.. Yeoman even has generator generators for making new generators). If you're not using them then you're missing a _massively_ useful productivity hack.

<comment><USER20> You're right. Mine was not a blanket statement but sounded like one.

I do use rails generators; if possible I'd never want to touch webpack config.

However I would not use css grid generators because I'd prefer the understanding and precision of hand coding css grid specifically.

</comment></comment></comment></comment><comment><USER22> I'm surprised to see this big of a reaction. I created something with nearly identical features (only missing the ability to specify gaps) right when grid was released. There was some interest, but not a ton.

</comment><comment><USER23> As usual with these tools, they're very useful (probably) but almost unusable by anyone with bad contrast vision. Please up the contrast on all the various borders and controls like the arrows on the various number selectors. I can barely use them and I don't think my contrast perception is actually all that bad, this must just be a big black square to someone with properly bad contrast perception.

</comment><comment><USER24> CSS Grid and its named areas are fantastic!

In my experience teaching web dev to complete beginners, everyone has been able to create reasonable layouts with CSS Grid.

That's an improvement over flexbox which shines and is meant for 1D layouts but requires heavy pushing and shoving for more difficult 2D layouts.

</comment><comment><USER25> The linked github repo ([https://github.com/sdras/cssgridgenerator](https://github.com/sdras/cssgridgenerator)) isn't (publicly) available yet?

<comment><USER11> It is now!

</comment></comment><comment><USER26> I've had a lot of success using flexbox personally, and at least on my own projects I develop features on a page in 1D vs 2D...

</comment><comment><USER27> I was working on something using CSS grids and this has given me so many new ideas- this is great!

</comment><comment><USER28> this tool feels very clunky and non obvious, can't imagine using it for anything useful

</comment><comment><USER29> what the heck are these grid tags in the generated code?

I was thinking I was about to applaud a flex-box wizard that masterfully did ordering and more, but instead I see tags that I'm seriously not sure about. Are they compatible with the rest of my flex-box related site with Bootstrap all over the place? Are they compatible with my child elements and positioning?

Too many questions for this day and age. I don't think enough people are familiar with those particular tags that get generated, but if thats the "new way" lets revisit this in 2021

<comment><USER30> Um, what? It's CSS Grid [1]. It's separate from Flexbox.

[1]: [https://css-tricks.com/snippets/css/complete-guide-grid/](https://css- tricks.com/snippets/css/complete-guide-grid/)</comment></comment>


Facebook Is Blocking Ads From MySpace, Friendster, Hi5, Orkut and 3Jam? - <AUTHOR> http://www.techcrunch.com/2008/06/04/facebook-is-blocking-ads-from-myspace-friendster-hi5-orkut-and-3jam/

<comment><USER2> in other news NBC disallows ads of upcoming Fox, ABC, and CBS shows and vice versa

<comment><USER3> The headline got the emphasis wrong; the article focuses on the fact that Facebook blocks 3Jam, which apparently is "a multi-person sms startup which allows users to text message multiple contacts at once." 3Jam also has a Facebook App.

<comment><USER2> my bad - that is interesting</comment></comment></comment>


Webserver in bash (2007) - <AUTHOR> http://paulbuchheit.blogspot.ca/2007/04/webserver-in-bash.html

<comment><USER2> Hardly implemented in Bash, it uses nc. I have a friend who implemented using no external binaries in Bash. I'll ask him and see if I can get it

EDIT: I think this is it: [http://ur1.ca/f2ozi](http://ur1.ca/f2ozi)

<comment><USER3> It is a convention that when you say "written in bash" it means "using bash language and standard *nix binaries". Bash alone can't do anything, nearly every keyword in it is actually a binary, including "echo", "true" and "false" or "[" used in if statements, which you can usually find in /bin or /usr/bin. You couln't write simplest bash scripts without using any binaries.

Netcat (nc) is built in to nearly every system that has bash. Wget and Curl on the other hand, are not.

<comment><USER4> Wrong!

echo, true, false and [ are builtins in bash and are in the binary. You can even do TCP/UDP through bash by using the pretend redirect files /dev/(tcp|udp)/$host/$port.

curl or wget exists on most systems by default in a minimal installation now days, nc does not, but if you run a normal installation you will get it though.

<comment><USER5>
       josephredfern@ubuntu:~$ which [
       /usr/bin/[

<comment><USER6> Use 'type', not 'which'

</comment></comment></comment></comment></comment><comment><USER7> Fun times in bash, but very boring in the Inferno shell:

listen -A 'tcp!*!9000' { {req := "{sed '/^\r$/q'}; echo 'HTTP status and headers elided'; echo $req} & }

</comment><comment><USER8> I was just thinking about this the other day and wondering if anyone had tried it yet. Too lazy to look for it.

Throw a way to do some real routing in there and you might be on to something. It's an interesting idea, with the trends of blogs (like mine even) going to octopress, simplifying the serving of static text is greatly beneficial.

</comment><comment><USER9> If you're on OS X, change the netcat command to: nc -l localhost 9000</comment>


Ask HN: Recommendation on Demo Video for Startups - <AUTHOR>

Hello!<p>I am a co-founder of a startup and we are getting ready to raise our first round. We would like to include in the pitch a brief intro video (around 1 minute long) and already have a script created of what we want said. We want to have an animation video to go along with our script. I found companies that specialize in demo videos but they are all very expensive (cheapest I found was $1,000 per finished minute) and I was wondering if any startupers our there know of anyone that can create awesome videos on a boot-strap startup budget? Students that are looking to make some extra money and get experience that have done it before would be great. If anyone could recommend someone it would be very helpful! <comment><USER2> <http://news.ycombinator.com/item?id=2646753>

<comment><AUTHOR> awesome thank you!!!</comment></comment>


A Better Way to Manage the Rails Secret Token - <AUTHOR> http://daniel.fone.net.nz/blog/2013/05/20/a-better-way-to-manage-the-rails-secret-token/ <comment><USER2> We've had discussions about this several times, and haven't come up with something that's satisfactory as a generic replacement, other than "configuration could probably be improved."

For one example, see this from a year ago: [https://github.com/rails/rails/pull/3777#issuecomment-289375...](https://github.com/rails/rails/pull/3777#issuecomment-2893759)

> If we ignore them, this means a recently created, pushed and then cloned > project is not going to work at all.

Some people replace it with a new file upon deployment, some people use ENV vars, some (most) people never open-source their app, and don't mind employees seeing it...

I personally do [https://github.com/hotsh/rstat.us/blob/master/config/initial...](https://github.com/hotsh/rstat.us/blob/master/config/initializers/secret_token.rb)

Being generic is hard.

<comment><USER3> Have you considered generating a key at first startup and storing it in a database? Or would that introduce too much unnecessary overhead while introducing an attack vector through the database?

<comment><USER4> The extra overhead could be kept pretty small. After being retrieved once, it can be cached in the memory of a server process. So, there's one short SQL query at process startup (or perhaps first request, depending on how you do it), and negligible overhead after that.

<comment><USER5> This is what I do, works great.

Most of my application secrets/configuration/keys/tokens are stored in the database.

The only one that's not is the information about how to connect to the database. That's stored in the DATABASE_URL environment variable and it's stored on each machine. envdir is used to start the apps, reading that environment data.

</comment></comment></comment></comment><comment><USER6> I wish Rails supported two secrets the way Rack::Cookie does by always signing with the first, but accepting either. That way you can rotate the secret without signing everyone out.

<comment><USER7> I'm surprised that it doesn't? gorilla/sessions[1] does the same; and you can eventually remove your old keys provided you keep your expiry times sane.

[1]: <http://www.gorillatoolkit.org/pkg/sessions#NewCookieStore>

</comment></comment><comment><USER8> > _Knowing the secret token allows an attacker to trivially impersonate any > user in the application._

Worse. Knowing the secret token allows an attacker to trivially _execute code_ in your application.

Don't ever let your secret token become public knowledge, and if it does, you need to change it straight away.

<comment><USER3> Doesn't Rails use randomly generated session IDs? Also, how does it allow an attacker to trivially execute code?

<comment><USER9> > Session data is marshal-ed Ruby data, deserializing it has the same risks as > YAML.load.

[http://daniel.fone.net.nz/blog/2013/05/20/a-better-way-to- ma...](http://daniel.fone.net.nz/blog/2013/05/20/a-better-way-to-manage-the- rails-secret-token/#comment-902646816)

<comment><USER3> That's a bit surprising. I'd be interested to read the reasoning behind this design decision as most frameworks I have used store session data in a database rather than directly in the cookie. Well I guess there is a pretty damn good reason given Rails' reputation.

<comment><USER10> Rails' _default_ session is cookie store, _highly recommended_ is DB. It's a config issue that depends on your app's particular DB setup, so default isn't DB. But in session_store.rb you'll see this comment recommending against the default:

      # Use the database for sessions instead of the cookie-based default,
      # which shouldn't be used to store highly confidential information
      # (create the session table with "rails generate session_migration")
      # ShopMobile::Application.config.session_store :active_record_store

<comment><USER11> also, in config/initializers/secret_token.rb:

       # Your secret key for verifying the integrity of signed cookies.

Does this mean that if you use redis/db-backed sessions you can safely ignore this secret_token parameter completely or even delete this initializer?

UPDATE: I just tried to remove it from our environment, and everything seems fine. Unless I'm missing something out, I'd say that's a far better and easier solution overall. It gives you much better control over you session + you don't have to worry about this configuration variable.

<comment><USER10> Since it's not only used for session cookies, I'd keep it around, on the off chance I ever wanted a signed cookie, like:

      cookies.signed['some-id'] = model.id

Rails will use the secret here too.

<comment><USER11> just curious: when would you need a signed cookie though? i.e. What need does it fulfil that neither the session store nor your database do?

<comment><USER10> I dunno, good question. Maybe something like this: there's some cases where I'll prefer cookies to session data, esp if it's non-volatile data that I can take or leave, like convenience values for the user on the js side, "last selected item" or the like.

I don't _necessarily_ want that data in my session, or as a user attribute that I have to migrate the DB for. If it's a pkid or something that isn't necessarily sensitive, but I also don't want it to be too easy to get at, I might use a signed cookie.

Now, I've never actually _done_ this... ;p

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> Your environment is not secure. The solution for this type of problem (keeping secrets out of source control) is to create a deployment-specific configuration file that is not kept in source control. It can be created out of a generic version that is kept in source control. Then, OS-level permissions are applied so the file is only readable by the processes that need access.

<comment><USER13> How is the environment any less secure than memory? If someone can read or mutate your environment, you should assume your app is already compromised and OS-level permissions aren't going to do anything.

<comment><USER14> Certain weird-shit UNIX operating systems do not provide privacy for a process's environment. (eg. another user on the same box can see them with 'ps e'). More relevantly, POSIX does not require it. The same is not true of process memory.

</comment></comment></comment><comment><USER15> > if Rails.env.development? or Rails.env.test?

I hate code like this that is explicitly aware of the environment. The code says what it will do in an environment, rather than the environment saying what the code should do in it.

<comment><USER16> Something more like the following is probably a better solution:

if ENV['SECRET_TOKEN'].blank? raise 'SECRET_TOKEN environment variable is not set!' end

App::Application.config.secret_token = ENV['SECRET_TOKEN']

<comment><USER17> ryannielson's solution is the best IMO, as it requires the environment variable to be set, & most importantly, shows a nice error to the developer should they miss it.

Even better, raise 'SECRET_TOKEN not set! Please refer to the doc in xyz'

So, the specific method for setting is in an "xyz" doc that your team keeps in a SEPARATE location from the code repo.

And, we really need a standard way to do this, or Github pulls / forks will have more friction or bad security when setting up forks.

Also, I really would rather put it in a file, not system env, as the env might be setup different on different systems, & you'd hate to have that env potentially shared in multi-user systems. Files are more reliably locked down.

</comment></comment></comment><comment><USER18> For Python website projects, I enforce the creation of a _secrets.py file in the config folder that is ignored by git and contains all sensitive constants like database information and tokens. Then, in the _base.py settings file (what all other settings files inherit from), I make sure to `from _secrets import *`. I'm not a fan of setting tokens by environment because it gets a little too unwieldy to make sure bash/zsh/whatever sets the variable.

I haven't run into any problems using that method. Does anyone see a reason to prefer environment variables over it?

<comment><USER19> For me the main reason is consistency. I can write apps in Python, Ruby, node, etc. and configure them all the same way.

The envdir program (from daemontools) is useful for setting environment variables in a shell-agnostic way, e.g., run your app with:

      envdir ./env python app.py

See also

<http://12factor.net/config>

<comment><USER18> Another good point. Thanks for the link, too!

</comment></comment></comment><comment><USER16> I wrote a gem called envious that could be used as an alternative to dotenv: <https://github.com/RyanNielson/envious> from what I can see it does a few things dotenv doesn't. Good guide though, I wrote Envious when I found out this problem existed.

</comment><comment><USER20> A generic solution to this problem that I don't see mentioned is to have different configuration files for all of ones deployment environements (dev, test, integration....) and have an encrypted config file for production. All the config files go on source control so there are complete records of all changes. Then the key to decrypt the production configuration file is known to the build maintainer and also known by a dedicated build machine (maintained by the build manager). Doing things this way one can be as secure as one likes while at the same time, builds can be fully automated; builds can be machine independent (if you have a dynamic server environment or ever worry about losing a server), and builds have a complete change history in source control.

</comment><comment><USER21> Eh... Keeping that in the system environment isn't really any better than hardcoded in a file. There's a long history of "do not trust the system environment" when it comes to security so I can't say I'd recommend this. Last I checked it was also fairly trivial to dump this data out of a running program...

Unless you're grabbing that key out of "secure memory", a HSM or a TPM then its not really particularly secure.

<comment><USER22> If it doesn't show up in a repo file then it is a bit better.

<comment><USER21> Yes, but only marginally; You can get the environment of another process with the "ps" command trivially. If you're in a shared environment you just made it that much easier for other people to monkey with your stuff.

Here's a great example of the repo file checkin fail though: <http://bit.ly/10dLiDz>

<comment><USER3> > If you're in a shared environment

The common wisdom seems to be that you can forget about security in a shared environment regardless. The "secret key in environment variable" technique is mostly useful at protecting against malicious employees since it's easy to limit access to the production server but not so easy to limit access to configuration files which are in a Git repository.

<comment><USER21> While I generally agree, there's a lot of shared environment out there that people seem to think is secure. I also would posit that its probably easier to get the calling environment through some flaw in bad programming vs getting a file off the filesystem. I would also posit that not having strong controls on your source tree is probably not a good thing as well...

<comment><USER13> The only secure shared environments I would trust are jailed environments or virtualized OSes, and even then only if I could control the hardware. Even then there have been vulnerabilities which allow virtualized OSes to access the host system (and "sideways" into other OSes), meaning even something like EC2 is potentially vulnerable.

Don't trust your environment.

</comment></comment></comment></comment></comment></comment><comment><USER23> The way, we've solved this is to default to a hard-coded secret if the environment doesn't have it.

        App::Application.config.secret_token = ENV['COOKIE_SECRET'] || '<default secret>'

Secured environments like production get their own secret. Developer machines can use the default w/o additional overhead.

</comment><comment><USER24> I've found that the figaro gem is also a nice gem to use re: secret tokens and ENV variables in general - esp on Heroku. <https://github.com/laserlemon/figaro>

</comment><comment><USER25> I'm not sure I understand what dotenv does (or why you would need it to do it).

<comment><USER26> It loads environment variables from a .env file when starting your app so that you don't have to do

$ SECRET_TOKEN=abcdef SOME_OTHER_VAR=hello rails s

or pollute your .profile with a bunch of app-specific variables.

You don't need it to do it; it just makes it easier.

<comment><USER13> Couldn't you just read the file directly? e.g.

        secret_token = File.read('secret_token_dont_check_in')

I fail to see how loading the file into the environment and then into the variable is anything but worse.

<comment><USER27> The original issue is people checking their secret token files into their VCS repository and publishing that. Getting the secret token from the ENV means it probably won't be checked into the repo.

<comment><USER13> It's still a file (how do you think it gets into the environment?). Whether it's a ruby file or a .env file or a yaml file, it's still equally at risk of being checked in.

</comment></comment></comment></comment></comment><comment><USER28> LOL BROGRAMMERS

</comment><comment><USER29> I don't understand why a secret token is even necessary. This seems like bad design. As a matter of principle, the server should never trust the client. If authentication is necessary, it should be done on every request. If that is the case, what purpose does the token serve?

The entire principle behind HTTP - what enabled it to conquer and dominate the internet, is its statelessness. Storing and trusting things in cookies is a fundamental security design flaw.

<comment><USER30> I'm not sure you understand what's going on, which is probably why you've been downvoted. The secret token ensures that the server _does not need to_ trust the client--that's what signing or encrypting session cookies does.

Authentication on every request requires you to store the user's authentication details on the client, which is considerably worse for security purposes than a signed or encrypted session cookie.

<comment><USER29> I wrote this comment with the assumption that we all understand the security implications of what's going on, which is why I probably did a poor job of making my point.

However, I think that relying on a single easy-to-compromise security token as a single point of failure for your app's security is terrible practice.

A secure user authentication system is usually built by having a users database that contains [user_id, nonce, password_key = pbkdf2(password, nonce)]. When logging the user in, you should store this information in a cookie: [user_id, login_timestamp, authentication_token = hmac-sha256(nonce, login_timestamp, password_key)]. Then, on every subsequent request, you would verify that the authentication_token matches what you'd expect based on the login_timestamp and user_id that the client has provided. (Substitute this for any equivalent-security scheme and crypto primitives.)

Now, if you have such a scheme in place, a secret token would provide no additional security. You really shouldn't be storing session information in cookies, but if you must, you can use a key derived from the nonce to sign the cookie, removing the need for an app-global secret. But really, don't do that, you should use a server side datastore for session information (like shopping carts, etc.).

If you do not have such a scheme in place, you app is much more vulnerable to attack. Something based on one global secret token (which by default is checked in to your source code) opens up many more attack vectors that might compromise the security of your app.

<comment><USER30> That's a completely valid route, for sure, but if your app's owned, your database is owned and this is going to be blown as wide open as a secret token.</comment></comment></comment></comment>


Where is π today? The nature of the mathematical universe - got-any-grapes https://billwadge.wordpress.com/2019/04/30/where-is-π-today-the-nature-of-the-mathematical-universe/ <comment><AUTHOR> I suppose it works well because it's designed that way. We abstract away the non essential and do our math. The results measure out in the non-abstract world well enough to be useful. So we keep on with it. This process is recurring, so we do math on our math.

I think it's fun to ask if this abstract world is real. I think it is. It's a wonderful place in the mind to visit if you're inclined to wander around without trying too hard to get anywhere. That's my hobbyist point of view anyway.

</comment><comment><USER2> > Mathematicians discovered a new [whole] number. It's between six and seven > and is called "bleen".

I believe the actual term is τ (FWIW: Verdana's Greek glyphs are absolutely atrocious)

<comment><USER3> Good one. A "whole number" of "turns". Exactly one of them.

"Bleen" is as good a name for it is "tau".

<comment><USER4> When you can't fathom something between 6 and 7, it may be a smoot point.

[https://en.wikipedia.org/wiki/List_of_humorous_units_of_meas...](https://en.wikipedia.org/wiki/List_of_humorous_units_of_measurement#Smoot)

</comment></comment></comment><comment><USER5> So my understanding is he believes math is invented, and exists in a "neo- fictional" space where things can be both true and false. Because if it were pure fiction like Star Trek then Math's statements would be false.

To me this seems a bit off, but I'll assume that it's my understanding of what he is trying to say that is flawed.

Also, didn't we all use the unit circle in Calc?

</comment><comment><USER6> >What does it mean to exist independent of time and space? Nothing, as far as I can see. I was once at a UVIC philosophy seminar where this came up. I asked if, in the time of the dinosaurs, bounded linear operators already existed. Yes, I was told. I have no idea what this meant.

I think a more interesting question is if an intelligent species that lived a billion light years away from us would eventually come up with bounded linear operators (or something homomorphic to it).

<comment><USER7> This is of the same flavor as wondering if said species would invent money, a collective fiction of usefulness so obvious it has been created many times. Or perhaps it's closer to wondering if they would develop futures on a stock market.

<comment><USER8> Dinosaurs didn't have numbers as they hadn't been invented yet. Merely being surround by quantities of stuff doesn't make human abstractions spring into existence.

The natural world doesn't perform arithmetic beyond summation and subtraction or symbolic manipulation of anything. We may use invented symbols to model abstract concepts aspects of the natural world but that doesn't make such things exist outside of collective human knowledge.

</comment></comment></comment><comment><USER9> π is not an invention, it's the relation between a circle's lenght and its radius. Not hard to understand.

<comment><USER10> The obvious counter-argument is that circles are an invention.

<comment><USER11> By humans? Do circles not exist without humans to point them out?

<comment><USER10> Not according to the author:

> According to intuitionism, mathematical objects are products of our mind, > like characters in a novel. I agree, as far as it goes...

> Fictionalism holds that the mathematical universe is a collective fiction, > like Star Trek or Game of Thrones.

<comment><USER11> Well, different characters behave differently in different novels. How is this like an idea of a circle?

<comment><USER10> > the Cantor set or the Borel Hierarchy is like the USS Enterprise or the Iron > Throne. In fact Star Trek fans talk of the Star Trek universe and GOT fans > of Westeros and they talk like these places really exist.

You're right, it feels like quite a loose analogy, and only holds when discussing a "canonical" fictional universe. Still, I'm sympathetic to the idea that when we're talking about a circle we're talking about a shared abstract understanding rather than something which objectively exists.

<comment><USER9> The pupil in the eye. The sun through the clouds. The craters after an impact. The waves when you drop a rock in water. The section of a bamboo stick... And it gets more evident when you go to microscopic world. Some eggs are perfect spheres, as small bubbles in water. I think circles are a real natural shape. Not an invention. Nature brings us all kind of shapes, some of them are discovered studying living creatures, like the scutoid.

</comment></comment></comment></comment></comment></comment></comment><comment><USER3> Well, this is correct as far as it goes.

But it raises the question of what, in modern parlance, is "canon".

As far as I can tell, mathematics becomes canonical if enough mathematicians think it's beautiful. There is plenty of seriously presented mathematics that never achieves canon status. The practical effect of being canon is that other mathematicians care to do proofs with it, or about it.

Riemann's hypothesis is canon even though it hasn't been proven, because so many proofs use it as an axiom.

<comment><USER2> AFAIK, Riemann's hypothesis is not quite canon but many mathematicians posit that it is likely to be true due to a variety of "evidence" they feel leans in that direction.

<comment><USER3> I have seen a paper that says it's undecidable, and then seems to choose a continuation in which it is. I am not equipped to evaluate it.

[http://phys.lsu.edu/~fmoxley/bbm.pdf](http://phys.lsu.edu/~fmoxley/bbm.pdf)

<comment><USER12> I'm also not equipped to understand that paper, but my find tool is equipped to say the words "decidable", "undecidable", and "decidability" are not present.

<comment><USER3> Oops, wrong link.

[http://phys.lsu.edu/~fmoxley/r1.pdf](http://phys.lsu.edu/~fmoxley/r1.pdf)</comment></comment></comment></comment></comment>


A tool for measuring Java object size in memory - <AUTHOR> http://plumbr.eu/blog/how-much-memory-do-i-need-part-3-measure-dont-guess

<comment><USER2> also see <https://code.google.com/p/memory-measurer/></comment>


The Rules for Long S - <AUTHOR> http://babelstone.blogspot.de/2006/06/rules-for-long-s.html <comment><USER2> To jump to the final part "The Demise of the Long S": [http://babelstone.blogspot.de/2006/06/rules-for- long-s.html#...](http://babelstone.blogspot.de/2006/06/rules-for- long-s.html#Demise)</comment>


My Post-Startup School 2007 Video - Xobni, Justin.tv, Dropbox, Divvyshot in March 2007 - <AUTHOR> http://vimeo.com/3170520

<comment><AUTHOR> I just found tape from that day, and edited it over the past three. See Justin.TV one week after launch. Also, see the first 'office' of Dropbox and Xobni in California. It's very funny, and shows why I want to live in California.

<comment><USER2> Great video vlad. Brings back a lot of memories for me personally (startup school 08).

You should move to California if you get the chance. It was the best 9 months or so of my life. It's a culture you can't replicate and it's a ton of fun.

<comment><AUTHOR> Thanks Jason! I will!</comment></comment></comment>


GameCrush Lets Guys Pay Money To Play Online Games With Women (Seriously) - <AUTHOR> http://techcrunch.com/2010/09/28/gamecrush-lets-guys-pay-money-to-play-online-games-with-women-seriously/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Techcrunch+%28TechCrunch%29&utm_content=My+Yahoo

<comment><USER2> The vibe I am getting from this is that it is not about online dating or finding a partner. So what's the point? Well, if you are a man who can't talk to women and haven't figured out how to write a convincing ad about yourself on Craigslist this could be the service for you. Basically, a site for men who are lonely and women who will take advantage of that. What a positive and endearing business model.

<comment><USER3> That is the vibe it gives. It's kind of unfortunate, too, because a gamer dating site that enforced a no-douchebag rule would be an interesting experiment. Instead it looks like an even more pathetic version of LiveJasmin.

</comment></comment><comment><USER4> I wish them well but funny things happen to community based sites when the motivation is purely financial. Once you make it all about the money, then it's only about the money. As such other niceties tend to fall by the wayside as you end up with, well, Johns and providers. I can't see it getting a good reputation, but that's not necessarily an impediment to making a profit.</comment>


A quirky Facebook bug. - <AUTHOR> http://marcsdesign.com/blog/a-quirky-facebook-bug/

<comment><USER2> That's certainly weird, I've never seen that mystery bottom bar!

Not uncommon for Facebook though, it still annoys me that they have a footer on their main website which is impossible to access because of infinite scrolling..</comment>


GitHub Jobs - <AUTHOR> http://jobs.github.com/ <comment><USER2> The important piece to remember is that this costs companies $300 per posting on sites like Github ($350 for JoS, $400 on 37Signals), but $0 to post on their own site. So companies like Twitter (<http://twitter.com/jobs>) are still going to post on their site first, and are only going to post their hard-to-fill jobs on these boards.

So yay for GitHub and 37Signals for offering more alternatives to Dice and Craigslist, but remember that the best jobs still come directly from the employer's website.

(Speaking of Twitter, am I the only one who loves their new recruting video: <http://www.youtube.com/watch?v=wU6epAkC9wg>)

<comment><USER3> About that video.

I was going to write something scathing about it, but that'd be useless. All I'll say is imitating Wes Anderson is tired and boring. Looks like it was fun to make and some people will like it for sure, but I think it pales in comparison to the Etsy recruiting video:

<http://codeascraft.etsy.com/2010/07/09/etsy-is-hiring/>

I hate to bring this up this tired discussion, but is this a NYC vs SF thing? The Etsy video could almost be considered dark and pessimistic, while the Twitter video is full of whimsy and infantilization.

<comment><USER4> The twitter video looks like it's recruiting for a private high school (join our clubs! have friends! be awesome! we're so wacky!). The etsy video looks like a formalistic Hollywood trailer from the 80s.

I don't want to go back to high school _or_ be stuck in the 80s. :(

</comment></comment></comment><comment><USER5> The search thing says "filter by city, state, zip code or area", which seems a bit US centric. You already have a job listing in Germany:

[http://jobs.github.com/positions/1dd0bfda-a3c0-11df-9efb-0be...](http://jobs.github.com/positions/1dd0bfda-a3c0-11df-9efb-0beb9df2c0da)

But if you search for 'Germany', it doesn't turn up.

<comment><USER6> Or if you search for "Europe" for that matter.

<comment><USER7> interestingly, if you search for a random term,it shows listing "anywhere" randomly: 6 results for asdf [http://jobs.github.com/positions?description=&location=a...](http://jobs.github.com/positions?description=&location=asdf) 3 results for asd [http://jobs.github.com/positions?description=&location=a...](http://jobs.github.com/positions?description=&location=asd)

(number of results seems related to input size)

<comment><USER8> Sounds like someone should fork the job-board and fix this.

</comment></comment></comment></comment><comment><USER9> Looks good, just try to keep it free of recruitment agents. I have seen lots of job websites rendered useless once recruitment agents start posting – that is, you have no idea what company you are applying for.

Also, could you add flags, like mentioning if there's help for relocation. I am interested in jobs located in nearby countries, but don't want to waste their time if they only want local candidates. I know from friends, that some companies seem to provide lots of support including temporary accommodation, yet barely mention it.

</comment><comment><USER10> Every time someone makes one of these I'm just dumbfounded by how much money there is in it. I mean granted, they're sideline businesses for the three I'm most familiar with, but they are sideline businesses which would pay for almost a full engineering team...

<comment><USER11> They kind of have to be sideline businesses. Expensive niche job boards work well only if you already have a valuable community and reputation. They can't be created out of thin air, you need to have "main business" to piggyback upon.

Most of the value is created elsewhere, job board just helps with extracting it.

</comment></comment><comment><USER12> I'm happy this exists. I'm disappointed it is so US-centric again. There are other countries, and a 'filter by state, zip code' does not cut it. (Yes, yes, some European company could - and probably should - create a similar site.)

</comment><comment><USER13> My question is how much difference will there be between job posting here and on other niche boards. I searched in Chicago and everyone on the GitHub jobs is also on the Joel on Software's job board. Maybe the results are better for other places that have more good software jobs.

Are there too many job boards chasing the same quality listings?

<comment><USER14> It's early days; too early to compare Github to JoS. Note that we posted 2 reqs to Github but would never post to JoS; the overlap you noticed is imperfect.

(We're hiring in Chicago too, but we have no trouble reaching Chicago candidates and so didn't spend on a third posting).

<comment><USER13> Really? I didn't see Matasano in the results for "Chicago"

<comment><USER14> Sorry, I'm writing very badly today. We didn't post an ad for Chicago, even though we're actively hiring there, because we feel like we have pretty good candidate outreach in Chicago already; that, and we wanted to see how Github job ads did.

</comment></comment></comment></comment><comment><USER15> what would be great is a filter to show jobs that allow telecommuting, and what percentage of the job is onsite. Something like at <http://jobs.rubynow.com/> Makes it enormously more useful for us freelancers

<comment><USER16> Came here to say the same thing, this is a must have. They should allow to pick "remote" or "telecommute" for people posting jobs too. For now I see posts using different wording such as "anywhere", "telecommute" and "remote". This is very important for devs.

</comment></comment><comment><USER17> Any reason (I can't think of) why they might choose such long URL's for the job listings rather than something short like /positions/air-bnb/ruby- developer-needed/?

Just makes the URL's a bit more readable especially if we're sharing them about :)

</comment><comment><USER18> I was going to say why do we need another one but there has been a surprising number of jobs posted in the short time it has been up. Guess it cant hurt to have another board for smaller type companies.

</comment><comment><USER19> I actually IM'ed HR at this when I first heard about the launch. Seems the liked the idea ( <http://jobs.github.com/companies/Acquia> )

:)

</comment><comment><USER20> Neat! The title made me think it was job positions open at github, this is far more useful. Not that I'm 'available', but I know some people that are and I'll pass it on.

thanks for building this!

</comment><comment><USER14> Ugh. Don't forget to put "NO RECRUITERS" in your job ads.

</comment><comment><USER21> nice clean design, clearly inspired by: <http://careers.joelonsoftware.com/Jobs>

<comment><USER22> Our design was finalized months ago, imagine our surprise when we saw their redesign :) Good for them for making their job board not hideous, though.

</comment></comment><comment><USER23> If I search for jobs in California, only location-agnostic jobs shows up. But there are clearly openings in the Bay Area, etc.

</comment><comment><USER24> Got an ad that said "Remember The Milk is hiring: Software Engineer (Scala) _in Anywhere_ "... Awkward wording at best...

<comment><USER25> I saw that too but didn't find it awkward. It was pretty clear to me they meant their organization is distributed like 37Signals or the SO guys. Reading the job post confirms that.

</comment></comment><comment><USER26> I was expecting to do it's contextual-job-ad magic without me having to upload a resume, so was disappointed.

But a great product nonetheless.

<comment><USER27> It actually will. There's a lot of things we weigh the contextual job ads with. A lot of it is the (public) code you write and the location on your public profile.

We do ask that people explicitly opt-in for the ads (available for hire checkbox). One of the decisions we made while trying to weigh connecting jobs with job-hunters and not becoming an advertising company.

<comment><AUTHOR> I'm not "available for hire", yet I like to see what companies are offering jobs in my area. Is the checkbox going to be used for more than just showing ads?

<comment><USER27> Right now, no. In the future, yes. So if you're willing to keep an eye on our blog (we'll warn you before we make this info public) you can check it. Or you can just search on the jobs site itself — <http://jobs.github.com>

</comment></comment></comment></comment><comment><USER28> Seems like a good idea to advertise developer jobs where the hackers are. Keep up the good work githubbers.

</comment><comment><USER29> Needs better filtering options.</comment>


Why universe has three dimensions? - <AUTHOR> http://www.sciencealert.com/new-paper-explains-why-the-universe-is-in-three-dimensions <comment><USER2> [http://arxiv.org/abs/1502.01843](http://arxiv.org/abs/1502.01843) <\- arxiv link to paper</comment>


Ask HN: How much equity should we split? - <AUTHOR>

My friend came up with an idea for an app, and he would like 40-50%.
All he is doing for this idea is coming up with the original idea. I make the idea (an app), design, and updates/changes.
What do you think is a fair equity split? <comment><USER2> I tend to think not.

I think a better way to approach this is to think about the evolution of the equity split over time - i.e. how do the two of your "earn" the equity?

It might make sense now if he has the idea and you bring the skills that today the company is split 20% him, 10% you, 70% held aside for investment and staff.

If after six months you're still both working on the project then you've earned a lot of equity with your work, and he's earned some as well (assuming he's working in a sales / ops role). After 6 months it becomes 30% him, 30% you, 40% others.

After another year it's 40% each (or similar)... etc until you start taking funding and giving equity to staff.

</comment><comment><USER3> 0/100 or incorporate and make him responsible for selling and promoting the idea and app and doing finances. Then 50/50 with vesting for everyone. Track hours worked with harvest for both of you for vesting purposes and have a minimum each must meet monthly.</comment>


Ask HN: Should I be using Git instead of Bzr? - <AUTHOR>

I've been using Bazaar for a few months now and not had any trouble, but after reading lots of talk about Git on HN I've been wondering if I should switch.<p>Any big advantages of using Git instead of Bazaar? <comment><USER2> You should continue to use what is most comfortable for you. The fact that you are already using a DVCS (Distributed Version Control System) already puts you at an advantage to those not using it (e.g. Subversion users).

Just relax, and focus on writing your code now.

</comment><comment><USER3> I haven't used Bazaar before, but with these questions I tend to recommend that you just try git on one smaller project and see how it "fits" with your style. Even if it isn't a fit you will learn something and in the future know exactly why you are using what you use.

</comment><comment><USER4> I've used both, but prefer Bazaar because it always felt more intuitive to me. But again, try them both out, its not that difficult to import/export history from/to.

One big difference is git's speed over Bazaar. You'll typically only be annoyed with it only large projects but still, it is noticeable. Though, there has been huge progress from the earlier versions of Bazaar.

</comment><comment><USER5> If you're running an open product, Github by itself has huge advantages I'm beginning to realize. It's a known ecosystem that lowers barriers for everyone.</comment>


Ask HN: Is anyone a sysadmin in high frequency trading? - <AUTHOR>

Hey
I was wondering if anyone here is a system administrator for a HFT company. A lot of the time you hear about the code, low latency and high powered machines but I've never heard anything about the system administrator's who piece all of this together, monitor it and keep it running. I'm looking for people who work in this area to shed some light on firstly what it take's to be a sysadmin at that level and secondly to share some insight on the architecture of systems in that area.
I really hope there are some out there who can talk about this stuff. And for anyone else who is a sysadmin working on high performance systems please speak up. A bit of knowledge sharing and stories would be awesome. There is so much out there on programming but as a system administrator I want to hear about our side of things. <comment><USER2> I was a sysadmin at a company that was extremely connected to some very important finance people. I can't say anything remotely specific since these guys are _ridiculously_ litigious and I personally witnessed multiple lawsuits.

Honestly I didn't find it very interesting or particularly challenging from a technical perspective. I had experience running big datacenters previously and it wasn't any more or less difficult. All the normal stuff you need to do to run hundreds of servers.

You have a bunch of servers running a bunch of strategies, similar to what a human could do, but faster. The strategies are tuned in real-time by the guy that created them (usually). He watches them run and either pauses, modifies, or kills them based on performance.

The strategies are a mix of code and configs and run on a bunch of servers. You stick the servers as close to the exchange's (or other third-party's) servers as you can to get as much of a latency edge as possible. We had access to all kinds of advantages in this regard that normal ccompanies wouldn't get.

The programming challenges were somewhat more interesting, but I wasn't super involved in that. Things like garbage collection performance tuning (so they could use higher level languages) and hard core math. But a lot of boring stuff too, like dealing with hundreds of legacy data feeds (many of which cost tens of thousands per month).

The very core is just really crafty individuals sitting around all day thinking of ways of screwing^Wtaking a slice of other people's pie and turning that into a strategy which can be milked as quickly and for as long as possible.

<comment><USER3> sysadmin stuff is pretty much standard because the challenging work is actually done by the strategists / developers. Stuff like processor affinity magic, which at one time would be covered by sysadmins, are now being covered by the people who deploy the strategies. I guess there's something to be said about the extent to which roles have melded (the line between strategist and developer and sysadmin is not clear, and in some circumstances a single person fills all roles).

"We had access to all kinds of advantages in this regard that normal ccompanies wouldn't get." <\-- it costs about 15K a month, soup to nuts, to get the "advantages"

<comment><USER2> The star at a trading company is the traders, as it should be. Nothing has changed that or probably ever will. The programmers are just one layer closer to that than the sysadmins, which tends to make it more interesting.

> _it costs about 15k a month_

That's like saying "a piece of meat costs about $25".

I don't think there was one POP (of dozens) that was the same price as another. Not to mention the equipment and connections required made the deal unique every time.

There's more to it than that regardless. I'm not going to get into specifics, but don't presume to know something you would have no way of knowing.

<comment><USER3> Having bootstrapped and built a successful ultra low latency trading operation from start to finish, I can assure you that in this space the star is the technology. In other forms of HFT, such as statistical arbitrage, the traders are the stars. But my understanding is that the question specifically regarded low latency trading.

To give an example, a whole cohort of traders used a simple lead-lag signal between futures and equities. Up until mid 2009, it was possible to consistently make money merely by waiting for futures to move, sending the information down from chicago (where the CME sits) to new jersey (most of the equities exchanges are housed in new jersey) and trading the relevant stocks. At the time, a popular trade regarded ES (the on-the-run emini s&p 500 futures contract) and SPY (the s&p 500 depositary receipts, designed to replicate the daily returns of the S&P 500). Here, everyone and their mothers knew that this relationship existed, and this relationship was published in various papers, both academic and professional. In this context, the "trader" who brings the idea isn't really adding any value to the process. The success of the trade was wholly determined by the performance of the platform.

In a large HFT, the management cleverly structure the relationships so that the technology staff is lopped together with support and told that the traders bring value. This is intentional: there are numerous traders out there now who still believe they have value-add, when by now all of the tricks have been spilled.

The quoted estimate is averaged over a year of actual expenses, whereby relationships with clearing brokers and exchanges were set up (bypassing external vendors)

"We had access to all kinds of advantages in this regard that normal ccompanies wouldn't get." <\-- that makes it sound like the tools to actually do ultra low latency trading are out of the reach of most people, and in reality they aren't. A competent person can build the business from 100K of personal savings :)

<comment><USER4> Vernon is quite correct here, traders are largely redundant though they still hold the balance of power in almost cases.IT and quantitative strategies still add value while operational trading can be almost totally automated. There are definite opportunities for running trading firms as technology firms.

</comment></comment></comment></comment></comment><comment><USER3> What is your email?

Let me give a meaningful response: in low latency trading, the value is truly in the platform. The strategies are simple, and you can pick up a textbook (ie Aldridge [http://www.amazon.com/High-Frequency-Trading-Practical- Algor...](http://www.amazon.com/High-Frequency-Trading-Practical-Algorithmic- Strategies/dp/0470563761)) to learn the strategies employed at that level. That being said, the traditional "sysadmin" role is oftentimes filled by people traditionally described as "developers". The roles have melded over the years.

Hence, the question you are asking really delves into the "alpha" of the platform. This is something a practicing HFT sysadmin wouldn't want to divulge (or can't because of the highly litigious nature of the industry -- take Sergey Aleynikov or Samarth Agarwal as examples of what could happen if people talk too much)

<comment><USER5> Be careful of the Aldridge book he cited, most of the helpful reviews are very negative, and the gang at nuclearphynance has similarly nasty things to say.

<comment><USER3> I don't think the book is particularly good for what people buy the book for (learning some strategy that can immediately be applied to make money), but it was chosen as an example of a book that describes some of the trades. I would like to emphasize that I wouldn't recommend any of the high frequency trading books.

</comment></comment></comment><comment><USER6> Not a sysadmin at a HFT, but am in personal contact with many of them. The only thing that I can tell you is that getting any information out of anyone in the industry is extremely difficult. Every variable (hardware and software) is considered something that can be turned into a competitive advantage.

To get an idea of the sort of platforms that are dealt with, check out <http://www.stacresearch.com/>

</comment><comment><USER7> My bet is the code is so closely related to the hardware that there isn't a clear distinction between software developers and sysadmins, they are known as devops, something that is becoming more and more popular.

I bet any "software developer" at a HFT firm would be able to entertain any of your questions.</comment>


Yehuda was right: Steak is a scam - <AUTHOR> http://steak-is-a-scam.heroku.com

<comment><USER2> I really didn't find this useful. I don't know what steak is. I don't know what it's relation to RSpec is. I don't understand how it's a scam. It seems like these slides depend on a lot of context that I don't have.

I'm primarily a rails programmer. I've used rspec and cucumber before. (And I'm aware that rspec can be used outside rails.) I understand these slides may not have been for me but then who are they for?

Edit: I think I know what bugs me. These slides feel like they are in response to some events I haven't seen. Like an open letter[1], but with less context.

1\. Not a stab at the author, letters are generally more verbose and context rich than speeches.

</comment><comment><USER3> Sorry to be dense ... are you saying that Steak is really just Webrat and Rspec, but that you gave it a new name?

</comment><comment><AUTHOR> The slides from my lighting talk at Conferencia Rails 2011 fully annotated for your personal enjoyment

</comment><comment><USER4> Um, what's "Steak"?</comment>


NSA Prism logo is a copyright infringment - <AUTHOR> http://www.theregister.co.uk/2013/06/12/nsa_logo_scandal/

<comment><USER2> "Dear Mr. XXX, this is the NSA. We'd like to use the image we've found on your homepage as the logo for a secret massive surveillance and spying program called 'prism'. Could you please sign the attached 20 page license and send it back to us: NSA, Maryland. Thanks in advance, Mr Smith, Chief Officer for Domestic Affairs."

<comment><USER3> Strawman

</comment></comment><comment><USER4> Actually, though... If they've been monitoring everyone's internet traffic, it's feasible they have unwittingly downloaded illegal copies of media. Meaning they've committed copyright infringement on a massive scale?

<comment><USER5> NSA probably has specific exceptions written into law.

In the UK GCHQ has exemptions to cover things like anti-circumvention of copyright law; images of child sexual abuse; copyrights; etc etc.

<comment><USER6> It's good to be king.

</comment></comment></comment><comment><USER7> Office worker uses Google Images to find photos for a Powerpoint. News at 11.

</comment><comment><USER8> It says "Free to download and use" So how is that copyright infringement?

<comment><USER9> Right after that it says '© Author', which links to a Terms page, which states "If you make money from using our stuff, share it with us!" and "You acknowledge the author, including in derivative works, if possible with a caption" and "You make an effort to contribute similar material of your own creation to the Web under similar terms, if and when you can."

[http://gallery.hd.org/terms.html](http://gallery.hd.org/terms.html)

<comment><USER10> The license has enough holes to drive a tank through. They're clearly not using the image to make money. Acknowledgement is done "if possible". Contributing is done "if and when you can".

The "if possible" line is left up to the interpretation of the licensee (in this case, the NSA) and they can simply argue that it wasn't possible because the program was classified and including the name of a civilian who has no clearance might have impacted the classification status. Regarding contribution, that too is left to the discretion of the licensee... "Sorry, we couldn't".

As much as one might like to find fault with their use of the logo, I fail to see how this is infringement.

<comment><USER9> I agree, it's all very wishy-washy and full of contradictory statements. My point was simply that there was more text than "Free to download and use".

</comment></comment></comment></comment><comment><USER11> The Federal government is has sovereign immunity, so you can only file claims against them to the extent that they allow them to.

You can sue the feds for copyright infringement, but only for reasonable compensaton -- you cannot recover the costs of litigation.

</comment><comment><USER12> Terms of photo use: "Don't use our stuff to deceive or mislead others." [http://gallery.hd.org/terms.html](http://gallery.hd.org/terms.html)

</comment><comment><USER13> The photographer, Adam Hart-Davis is great. He does history/sciencey shows on the BBC.

Local Heroes was great, but seems to have avoided being uploaded to YouTube.

Here's an episode of What the Romans Did for Us:

[http://www.youtube.com/watch?v=VDU5zBYQprc](http://www.youtube.com/watch?v=VDU5zBYQprc)

[http://en.wikipedia.org/wiki/Adam_Hart- Davis](http://en.wikipedia.org/wiki/Adam_Hart-Davis)

</comment><comment><USER14> I guess this means the original photographer is guilty of leaking a secret document to the enemy. Because the secret FISA order classifying this NSA content, by extension classifies the original work, and all documents that include it.

</comment><comment><USER15> Can't believe they would do something like this, maybe the author is credited on the final slide of the presentation?

</comment><comment><USER16> NSA doesn't follow the law shocked? :))

</comment><comment><USER17> that just looks like the hosting company's business records to me

</comment><comment><USER18> Jumping the shark!

<comment><USER19> The NSA or Hacker News?

</comment></comment><comment><USER20> Reminds me of this great video: [https://www.destroyallsoftware.com/talks/wat](https://www.destroyallsoftware.com/talks/wat)

<comment><USER20> Major oops, actually somehow I commented on the wrong thread! (hope I'm not the first in HN history to do that), how utterly bad it may look, too many HN tabs open I guess.

As stupid as it may sound, it was intended for this thread: [https://news.ycombinator.com/item?id=5866837](https://news.ycombinator.com/item?id=5866837) anyone with mod permissions, please delete the my parent comment (and this one as well) oh, the shame :0</comment></comment>


JavaScript vs. Python in 2017 - <AUTHOR> https://hackernoon.com/javascript-vs-python-in-2017-d31efbb641b4#.e6fhkgl4i <comment><USER2> Something this article glosses over is that JavaScript achieved all the benefits of ES6, ES7, gradual typing, etc, without breaking backwards compatibility. I can still pull in a library written 10 years ago alongside my fancy new async/await code.

Perhaps this can be partly credited to JS's decision to go with a more minimal standard library, meaning it didn't end up with the Python 2/3 situation as standards evolved.

<comment><USER3> I can do the same with a Python program written 8 years ago given that Python 3.0 was released in December 2008.

I don't think that it's Python's large standard library that was the issue since that was immediately compatible with Python 3, rather it was the large ecosystem (in fact particularly a small number of popular packages) which held things back by not porting soon enough.

<comment><USER4> The biggest issue was Python standardizing on Unicode by default in Python 3 and breaking backward compatibility with a lot of (bad) string handling code the hard way by also forcing developers to unlearn a lot of old habits.

JS won't experience a directly similar thing because it started with default Unicode strings support (because it had to, and also because JS is slightly younger than Python and had that advantage).

</comment></comment></comment><comment><USER5> > To make matters worse, npm module names are doled out on a first-come, > first-serve basis. Much like domain names, this means that great names often > go to undeserving projects. (For example, judging from its download count, > the npm module named logging makes it one of the least popular logging > packages for JavaScript.) This makes the comparison of third-party modules > all the more time-consuming since the quality of the name is not a useful > signal for the quality of the library.

This is also my biggest complaint about global npm registry: lack of namespacing.

> write JavaScript code that uses async/await and type annotations, switch to > my terminal, run node example.js

How about:

        node --harmony

</comment><comment><USER6>
        To the best of my knowledge, the Python community does
        not have an equivalent, popular mechanism for
        experimenting with DSLs within Python.

I think this is because modern javascript practice already includes a compiler pipeline to handle modules, polyfills, etc so one more step doesn't really matter. In contrast, since Python already works out of the box, even one extra step incurs a large step. (Linters and such don't count because I can run my unlinted code as is; I can't run my custom DSL without preprocessing it).

</comment><comment><USER7> > I gave a presentation back in 2013 at mloc.js where I argued that static > typing is an essential feature when picking a language for a large software > project

OK, so you went shopping for a statically typed language and somehow ended up with Python (which only _just_ started having annotations in 3.6)? Why not pick any of the multitudes of languages that actually do have static type- checking?

<comment><USER8> _which only _just_ started having annotations in 3.6_

This is incorrect.

Python gained syntactic support for annotations of functions and methods in 3.0. It was anticipated that this feature would be used to annotate types of arguments and return values, but nothing about it _required_ using it for this purpose (and in fact, nothing requires using it for that purpose today).

Python 3.5 added the 'typing' module to the standard library (and it was also released standalone on the Python Package Index, allowing it to be installed on older versions of Python). This module provides a standardized way to describe types, and an ecosystem of tools has popped up which make use of the typing module's approach to perform static-style type checking of Python source code.

Python 3.6 added syntax for annotating variables, and for declaring variables without assigning a value to them. Prior to this, type-checking tools used type-declaring comments to read the types of variables.

Also, note that the annotations -- which are correctly termed "type hints" when used to describe types -- result in no enforcement whatsoever on the part of the Python runtime, which does not assign any special meaning to annotations, and aside from exposing them via an introspectable attribute will not do anything with them either prior to or during execution. To use them for type-checking purposes, you must install and run a third-party tool which reads the annotations and compares them to actual usage in the code.

<comment><USER9> ad the last point: Isn't this perfectly fine in principle (minus the fact that Mypy is third-party)? Compilers in static languages also work only with the source code, in a way, as there is no runtime interpreter. So there should be no reason for type checkers on dynamic languages to require the runtime. Granted, this forces you to essentially write type-statically in a dynamic language, which might limit your expressiveness, but that's probably your point anyway if you do type checking. Now I'm not saying that Mypy is anywhere near this hypothetical situation. My point is that type checkers over dynamic languages should not need the runtime in principle. Conversely, the runtime should not need to do the type checking, for performance.

<comment><USER8> Many people, when they hear "static types for Python", expect that the Python interpreter would perform some kind of pre-execution step of checking types and usage, and refuse to execute code with type errors.

Since this is not the case (and would be difficult/impossible to implement given that Python code can import new modules at any time, and also change the import paths), it's worth pointing out.

</comment></comment></comment></comment><comment><USER10> In terms of popularity Javascript clearly wins. According to stats:

[https://jobsquery.it/stats/language/group](https://jobsquery.it/stats/language/group)

JavaScript is mentioned in 17% of all tech job offers VS 10% for Python.

However in terms of average salary - Python is the winner. Average salary for Python openings is $117k annually VS $96k for Javascript.

(same stats page [https://jobsquery.it/stats/language/group](https://jobsquery.it/stats/language/group) )

<comment><USER11> Most job offers require more than one language - often a front end language and a back end language so its not surprising that JavaScript is higher as its the only real option on the front end. I use both in my job but I spend far more time in Python (and as far as preference goes Python wins hands down over JS).

</comment></comment><comment><USER12> > JavaScript is no longer an edit/refresh language.

That's weird, my edit/refresh experience didn't disappear, even after adding Babel/typescript.

In fact, maybe you could say it's more advanced and powerful after adding live reloading?</comment>


The Cockpit Project - <AUTHOR> http://cockpit-project.org/ <comment><USER2> While this seems nice, this still helps people create servers as "works of art" instead of repeatable constructions saved into configuration management, which Cockpit explicitly does not do.</comment>


Patent raid at German tech fair - <AUTHOR> http://www.iht.com/articles/ap/2008/03/06/technology/EU-TEC-Germany-Tech-Fair-Raid.php

<comment><USER2> Germany doesn't sound like a very fun place to be at the moment... first they pass a ridiculous law preventing security professionals from doing their jobs [http://arstechnica.com/news.ars/post/20070528-germany- adopts...](http://arstechnica.com/news.ars/post/20070528-germany-adopts-anti- hacker-law-critics-say-it-breeds-insecurity.html) and now this?</comment>


Where to find great ideas and arresting images (for free) - <AUTHOR> http://sethgodin.typepad.com/seths_blog/2008/03/where-to-find-g.html <comment><USER2> Hm, I just tried it and downloaded an image I found in that way. But then upon closer look the license requires attribution and disallows derivative works.

So the search function of flickr is still not good enough, should be more finegrained with respect to the license options.

And why does flickr not have an "add to favorites" button? Puzzles me every time I visit the site. It just goes to show that usability is apparently overrated as a factor for success...

<comment><USER3> Flickr _does_ have fine-grained license options on the advanced search page, and also has an "add to favorites" button. It's just above every image.

You do have to register and log in to store favorites, though.

<comment><USER2> OK now that I logged in I see it (very small and in grey). I guess since I have been frustrated so many times before (I have this problem EVER time I go on flickr), I didn't expect to find one anymore.

Usually I end up going to the FAQ and then somehow I figure out how to do it. I don't use flickr that often, so I keep forgetting.

I guess I would expect a big, fat button, since it seems rather important, but it might be different for different people.

</comment></comment></comment><comment><USER4> Another good one is sxc.hu.

<http://www.sxc.hu/advanced_search>

Just select "no" under the restricted ok button. You'll need to sign in- just use a bugmenot account - <http://www.bugmenot.com/view/sxc.hu>

</comment><comment><USER5> <http://openphotovr.org> uses many CC-licensed images from Flickr, honoring license terms and giving attribution with backlinks. For example, we built San Marco from Flickr, unlike Photosynth's San Marco which was shot in one day by one photographer.

The problem with architectural photos on Flickr: everybody chooses the "best" viewpoints. This is why I couldn't build a model of Taj Mahal, one of the most photographed places on Earth - all good images of it are taken from the same point.

</comment><comment><USER6> One site I really like is <http://ffffound.com>

It's an image bookmarking site where people post images they find around the web. I'm pretty sure all the images are copyrighted, but it links back to the original.

The site is currently invite only to register, but the people they've got tend to have pretty good taste in images.

</comment><comment><USER3> Quote from the original article: "[...]And the license is a generous gift from the photographer to you."

As an amateur photographer with almost 4000 CC-licensed images on Flickr, this pisses me off to no end. _My_ images are under the non-commercial attribution- required variant of the Creative Commons license, which means that simply downloading a copy of one of them to include in your business presentation or website is a flagrant violation of the license terms.

My "gift-giving" is directed towards the general betterment of society, not lazy web designers or bloggers who don't think other people's work is worth anything.

<comment><USER7> That's an out of context quote. He explicitly says to only search for commercially licensed photos.

</comment></comment><comment><USER8> yep, flickr is a great resource for amazing images, have been using it for my blogs for some time and always find something good. beats paying hundreds per month for lame ass stock photos that scream STOCK

<comment><USER9> you're scaring me....hundreds a month?

<comment><USER8> ya, photos.com is pretty expensive, and don't even think about getty images, it cost thousands just to use one image, no thanks, I'll take flickr for both quality and price (free) anyday.</comment></comment></comment>


How chaos drives the brain - <AUTHOR> http://www.newscientist.com/article/mg20227141.200-disorderly-genius-how-chaos-drives-the-brain.html?full=true <comment><USER2> Good heavens me! What an extraordinarily interesting article, a real feather ruffler if you ask me. I've always contended that there is no such thing as truly original thought, only complex amalgamations of stimuli - there are no "flashes in the pan," so to speak, only haphazardly arranged piles of nonsense which spontaneously combust, and produce such visionary ideas as the toaster oven or the baffling shade of green/yellow used to color tennis balls. I'm not entirely sure whether I'm right, as I only skimmed the article, but something tells me I know more than these crackpot/loose cannon scientists.</comment>