Launch HN: FloydHub (YC W17) – Heroku for Deep Learning - <AUTHOR> Hi HN! I'm Sai, one of the cofounders at FloydHub (https://www.floydhub.com). We're building FloydHub to be a "Heroku for deep learning". We are in the current batch (W17) at YC. But I still like to think of FloydHub as being an HN incubated startup.
10 months ago, I was working at Microsoft and doing a lot of deep learning (DL) there. While the DL community is terrific, I was often frustrated by how difficult it was to get started and build upon others' work. For example, running any popular Github project often started with an exercise in dependency hell. As I untangled these for myself, I wrote up some notes on setting up popular DL frameworks, which unexpectedly started trending on HN after someone posted it there (https://news.ycombinator.com/item?id=11697571). That's when I realized that engineering was a huge bottleneck in deep learning and a problem worth solving after all.
I've since quit my job and have been working fulltime for the last 9 months on building FloydHub to make deep learning easier. Our goal is to let the data scientists focus on the science, while we handle the engineering grunt work (provisioning and scaling infra, running reproducible experiments, enabling sharing and collaboration, supporting DL frameworks with zero setup, shipping trained models to production easily, etc.) Lots of interesting challenges - happy to talk about them!
We have a lot of work ahead, but we're excited to share with you what we have so far! Looking forward to your feedback. <comment><USER3> I was one of their earliest beta customers. Despite the initial quirks, the experience has been nothing but magical. It allowed me to go from code to training/model generation in one command, without any of the devops nonsense to do deal with. _While digging into deep learning, I honestly felt that the devops stuff was actually more complicated than the math /backprop/neural nets stuff._

It felt like a Heroku moment for me. They have the potential to do to Tensorflow what Heroku did to Rails. Super simple deploy!

Obviously their vision is much broader (with an entire eco-system/'hub', reproducibility, etc.), but to me atleast the first part is super useful and exciting!

One advantage they have is that GPUs are INSANELY expensive on the cloud - they can actually make it cheaper for everyone with clever binpacking and proper termination.

My advice is that in the initial stage, they should partner with all the Moocs to ensure that all deep learning students are using Floyd. It's cheaper, faster and the students can focus on the science. And they provide, 100 free hours!

Disclosure: I've known the guys for quite a while.

<comment><USER4> I think partnering with MOOCs is a great idea. I've been taking the Deep Learning Foundations Nanodegree from Udacity and thought it would be awesome if we got some sort of discount to try this out. Course developers have been trying to get us AWS credits for a while now. What would be the best way to contact the guys?

<comment><AUTHOR> Hey! Glad you brought this up. We've gotten quite a few requests from students enrolled in Udacity courses and have been helping get them set up & run their class projects on Floyd.

Here's our instructions for the Self Driving Car Engineer nanodegree program: [https://github.com/floydhub/CarND-Term1-Starter- Kit](https://github.com/floydhub/CarND-Term1-Starter-Kit). Happy to do the same for your class as well! How can I reach you? Feel free to to mail us directly: firstname.lastname@example.com.

We've also reached out to folks at Udacity to see if we can offer any official support for the courses.

<comment><USER4> Thanks for the reply! I just sent you guys an email. Will take a look at what you did for the Self Driving Car Nanodegree.

</comment></comment></comment></comment><comment><USER5> Sidenote: We always say the hardest things in software is cache invalidation and naming things, but I also found the problem to be dependency tracking and reproducibility. It's just so hard to get started with ANY non trivial software project and your os/tools/libs expire way before you finish the projects. Software should be cheap and repeatable, but for some reason it takes active maintenance and is therefore very expensive. If a superhuman intelligence looked at us from afar, we would probably look like how ants look to us: Millions of small workers with very inefficient probabilistic behavior. Sure, we get the job done, but very slowly with a lot of waste. That said, ants lived for ever, so maybe it is the right thing to do ;)

Either way, I often find myself choosing backward compatibility and stability over innovation and polish and choose to learn vim and bash instead of replacing the silver bullet every year.

Shameless plug, I am working on a platform similar to FloydHub, but for frontend engineers [0]. The problem is a real one.

[0] [https://pipez.io](https://pipez.io)

<comment><USER6> I can definitely resonate with this. Deep learning is in such an early stage, the frameworks and tooling are still maturing and evolving rapidly. This makes it really hard to reproduce other's work. Maybe there will be one winner in the frameworks war (Tensorflow?) and things will be better.

Pipez sounds really useful, good luck!

</comment></comment><comment><USER7> I'm a Data Scientist and I work in a large-ish corp. We have a dedicated engineering team who take care of all of our infrastructure needs - we mostly focus only on data science. I would expect most medium and large enterprises to have the same setup. Why would they use FloydHub?

<comment><USER8> Smaller companies and startups, or medium sized companies that just start with ml/dl will find it very useful.

Currently working in a small startup and our data scientists (just a team of 6) are frequently fighting over the gpus in the office for time and online services atm are overpriced for our small budget.

<comment><USER6> Agreed, that is the target audience we are going after with FloydHub. If you have some time to chat about your startup, please send me an email naren[AT]floydhub.com. I would love to get your feedback and see what would make Floyd useful for companies like yours.

</comment></comment></comment><comment><USER9> I have been using your docker container for 6 months or so now, thanks for putting it together :)

The jupyter jobs look neat, but I assume they are charged continuous time? Would be cool if somehow that only ended up charged for compute time, but I understand that would be difficult.

Are these instances guaranteed to be in a given region, for if I wanted to route more complex debug output / intermediate files to S3?

<comment><AUTHOR> Thanks - glad you found it useful! The attention and feedback that I got from building dl-docker has been terrific. Definitely one of the reasons we started working on this seriously :)

<comment><USER10> Little off topic, does DL-docker work in Ubuntu on Windows for the GPU?

<comment><AUTHOR> You mean bash on Windows? The CPU version of dl-docker will work on Windows! But unfortunately, the GPU version does not. You would need GPU passthrough, which is currently not supported. See [https://github.com/NVIDIA/nvidia- docker/issues/197](https://github.com/NVIDIA/nvidia-docker/issues/197)

</comment></comment></comment></comment><comment><USER11> Hi Naren, really glad to see this launch, your "Deploy your Trained Models" that feature is awesome... what about external dataset access? I would like to access the Google YouTube ML dataset [0], how could that be done within FloydHub without uploading? Also, perhaps related, have you thought of teaming up with Kaggle [1]?

[0] [https://research.googleblog.com/2016/09/announcing- youtube-8...](https://research.googleblog.com/2016/09/announcing- youtube-8m-large-and-diverse.html) [1] [https://www.kaggle.com/c/youtube8m](https://www.kaggle.com/c/youtube8m)

<comment><USER6> Hi, Thanks a lot! We are always adding to our public datasets (with appropriate licences). So we would be very happy to set it up for you on Floyd. Agreed that uploading large datasets is not quick. Instead we recommend you download them directly in to Floyd. See: [http://docs.floydhub.com/home/managing_output/](http://docs.floydhub.com/home/managing_output/) for an example.

Re: Kaggle, we haven't had a chance yet but that sounds like a great idea.

<comment><USER12> I was able to download a kaggle dataset using kaggle-cli.

</comment></comment></comment><comment><USER13> Looks really amazing! Reproducibility of models and experiments is huge. It should be almost a requirement if one is going to publish results claiming SotA, etc. Seems like you could become the GitHub for deep learning in addition to the Heroku for deep learning.

<comment><AUTHOR> Thanks, that's an awesome comment! Agreed. IMHO, end-to-end reproducibility is important from multiple angles - provenance for research, enabling collaboration, driving down costs by eliminating redundant runs of the same jobs, etc.

We were initially calling ourselves Heroku + Github for DL, but realized that was too confusing, haha.

</comment></comment><comment><USER14> This looks neat! We have been doing a lot of deep learning for NLP at our startup recently. Several "engineering bottlenecks" in the process (1) managing multiple jobs is definitely worth solving. git for deep learning would be neat (2) collaborating is a pain when the team is remote. I guess this ties to (1) too.

And oh, about the time I forgot to turn off our GPU instance for a couple of weeks… racked up a nice bill...

<comment><AUTHOR> We've definitely felt the bane of forgetting to turn off some really expensive GPU instances. Efficient scheduling and spinning instances up/down as required is one of the first things we built to cut down on the costs.

Git is an apt analogy. The search space of hyperparameters is usually fairly large for most DL algorithms, so a good amount of experimentation is required to tune them. Things can start to get haywire without end-to-end version control of code, data, parameters, results, environments, etc. Definitely one of the core problems we solve.

</comment></comment><comment><USER15> This sounds interesting and useful - I hope you guys make it!

A couple of years ago I worked for a local cloud server provider, as a backend developer. Some of the work I did moved the company into deploying VPS instances using OpenStack. Our backend code was mainly PHP-based; so we used OpenCloud for the purpose - extending it where needed (when we started it didn't support all we needed; I extended things in such a way so that when we did need to upgrade OpenCloud, it would gracefully work without breaking anything - it was a gamble that I didn't know if it would really work - <USER2> had a hunch - 9 months in we upgraded, and it all worked perfectly).

Anyhow - at that time, seeing what we had available for servers and such (we were competing somewhat with DO) - I suggested we add support for GPU instances and maybe pivot toward an ML offering of some sort. Not gut our bread-n-butter, but offer up some kind of ML package for those that needed or wanted it.

I was shot down by management as it being too "pie in the sky" \- not even demand or something like that. To be honest, I'm not even sure they understood what I was trying to convey, so maybe part of the problem was mine as well.

The company was eventually sold and I moved on, but seeing now how these kinds of services are in demand, I sometimes wonder on what "could've been". Ever since taking my first MOOC in ML (Ng's ML Class in 2011) - I've tried to interest employers in applying what (little) I know on the subject. I'm not an expert, but I'd love to apply my learning (on top of the 25+ years of software dev experience I already have). Today, I'm in the middle of the Udacity Nanodegree MOOC - I doubt much of that will transfer for my current employer, but maybe the general knowledge I'm getting of TensorFlow and Keras, among other bits, might help in the future.

I think, though, that FloydHub might be fun to play around with for future personal ML/DL projects as time goes on; I look forward to trying it out someday soon!

<comment><AUTHOR> That's awesome! ML to some extent, and DL to a large extent are still not considered mainstream enough by many. The big players (Google, Facebook, Baidu) seem to recognize this and invest appropriately, but the others not so much yet. That's hopefully a sign of a burgeoning market!

Good luck with your Udacity class! If you want to play around, take a look at our guide for Neural Style Transfer: [http://docs.floydhub.com/guides/style_transfer/](http://docs.floydhub.com/guides/style_transfer/)

</comment></comment><comment><USER6> Hi! I'm Naren, the other co-founder of FloydHub. I'll be happy to answer any questions and really appreciate any feedback you can provide. Thanks!

<comment><USER16> Features page says "Develop interactively on the cloud using Jupyter Notebook. Your code, results and outputs are always preserved."

But [http://docs.floydhub.com/guides/jupyter/](http://docs.floydhub.com/guides/jupyter/) says "IMPORTANT: Floyd does not save your Jupyter notebooks after you stop the floyd job. So you need to download any relevant notebooks by selecting File > Download As menu from the Jupyter notebook."

Which is it?

<comment><USER6> The warning in the documentation page is no longer valid. We _DO_ save the notebook files and keep them after the session is terminated. They will be part of the run output.

The docs have been updated to reflect this. Thanks for pointing this out.

</comment></comment></comment><comment><USER17> So I guess we're competitors in a sense, but congrats nonetheless! We hope to launch at least a private beta of NeuralObjects like "Real Soon Now"™.

It'll be interesting to see where we decide to go down different paths, or how we take different approaches to things.

<comment><AUTHOR> Thanks for the comment! There's lots of challenges to be solved in this space, and I'm sure there's room for all of us. Excited to see what you guys are up to. I will look forward to your beta release "real soon" :)

<comment><USER17> What was it Steve Blank said? "Startups don't die from competition with other startups, they die because they built a product nobody wants" (or something like that).

As you say, there's plenty of room out there. And we're all competing with Amazon, Google, Microsoft, etc. anyway. :-)

</comment></comment></comment><comment><USER18> Running DL and sharing results is a huge pain. I feel versioning is a big challenge, also experiments with multiple architectures are a pain since large parts of the same calculations are repeated. Do you also solve this problem? (Feature request if not)

<comment><USER6> Yes, our Enterprise offering allows for building more complex workflows. Everything (code, parameters, environment and data) is versioned for reproducibility. One of the big value-adds is efficient caching of parts of the pipeline to avoid repetitions and save resources. We have noticed that this usually results in 10x increase in the number of experiments run by teams.

</comment></comment><comment><USER19> You said you worked at MS, so you're presumably pretty familiar with Azure's offering. I'm no expert (at all) but I played with it briefly and it was all pretty slick and easy to get up and running. How does FloydHub compare to that?

<comment><AUTHOR> I assume you're talking about AzureML Studio. It's a pretty neat UI-centric tool for building machine learning workflows! It's great if you're starting out with ML, but offers little in terms of customizability. For example, it only supports R and Python, has no GPUs, no CLI, no container support for managing reproducible environments, etc. I think these are kind of deal breakers for doing deep learning :)

FWIW, I worked as a data scientist in Bing for 6 years and haven't seen/heard any other data scientist use it internally. We ended up building our own GPU clusters and going through the regular drill.

<comment><USER20> Curious how Microsoft's approach compares to what you have heard about Google's approach to ML as a service. My impression is that it looks kinda/sorta like it's internal approach/infrastructure...if you squint.

Which also makes me curious about FloydHub's infrastructure. Any gory details?

<comment><USER6> Floyd's Infrastructure runs entirely on Docker. That makes it backend agnostic (our cloud offering currently runs on AWS). Floydhub uses nvidia-docker for the deep learning jobs that require GPU. We also version the entire pipeline (code, data, params and environment) for exact reproducibility.

GPUs instances are really expensive. One of the biggest challenges at the moment is around reducing this cost. Eg. Spot Instances and Spot Blocks. Still some challenges to be solved there.

We also want Floyd to be an end-to-end solution for building, training and deploying deep learning models. In that vein, we are also investing in adding support for Tensorflow serving but it has been a rough ride so far. Getting a generic solution that can host any Tensorflow model has not been straightforward.

<comment><USER21> Try deepdetect, it s easier than serving, though probably still less flexible when it comes to the TF backend.

</comment></comment></comment></comment></comment><comment><USER22> It seems like everyone and their dog wants to solve this problem; why is it going to be you?

<comment><USER23> YC Magic?

<comment><USER6> YC definitely helps in giving us a lot of credibility when talking to customers especially enterprises. We have also been learning a ton about sales during the last couple of months - for a couple of engineer / data scientist it has been a humbling experience. Other than that there is not really any magic here!

</comment></comment></comment><comment><USER24> Skimming through the site this looks very polished for a startup offering.

<comment><AUTHOR> Thanks! We've been iterating on it for a while. My sister is a UX designer and helped a ton. I still think there's room for improvement, but it's great to hear a positive comment about it!

<comment><USER25> Who did the artwork for your landing page? Looks great.

<comment><AUTHOR> Nothing fancy - bought some cheap stock art and my sister, who's a UX designer, modified and brushed them up :)

</comment></comment></comment></comment><comment><USER26> Nice! There is a real need for this in the market right now--especially among students, who struggle to setup working environments. I really benefited from your early Github project aimed to make setting up DL machines easier. Curious how defensible your product will be in the event Heroku/AWS come in with a competitor?

<comment><AUTHOR> Defending against big players with almost infinite resources is always an interesting problem. Kind of have to hope they don't come at you head on :) That said, there might be other aspects that come into play. For example, the market is fragmented with big providers each having their own frameworks and infra that they prioritize (Google - Tensorflow/GCP, Facebook - Torch, Microsoft - CNTK/Azure, Amazon - MXNet/AWS, Baidu - PaddlePaddle, etc.) Since there is yet no clear winner, which will likely be the case for the near future, vendor lock-in is not really desirable. We intend to be backend / framework / language agnostic.

</comment></comment><comment><USER27> Thanks Sai, this looks great. I found myself dreading the clunky workflow with AWS GPUs, even using CLI, so this is nice to see a Heroku-like product that's a bit easier to work with.

Also, awesome 100 hours offering -- Looking forward to using FloydHub for deep learning!

<comment><AUTHOR> Sounds good! Give it a spin and let us know what you think. If you want a reference - here's a guide we wrote to run Neural Style Transfer ([http://docs.floydhub.com/guides/style_transfer/](http://docs.floydhub.com/guides/style_transfer/))!

</comment></comment><comment><USER28> You guys seem to make setting up Deep Learning infrastructure easy for data scientists which is awesome! What about situations where the end user/business isn't sure exactly how to use Deep Learning (which algorithm, how to partition the data into training and result sets, stability of results etc.) for the problem/data set at hand? Would it be possible to use FloydHub as a marketplace of sorts where I could hire a deep learning enthusiast to appropriately construct the experiment for me and then explain the results?

<comment><USER6> Yes, we are planning to build a FloydHub marketplace for talented data scientists to find gigs. We believe Floydhub can showcase their work and expertise easily (similar to StackOverflow) and help find a suitable partners to work with. A wide range of industries like Medical, Oil and Finance have been collecting huge amounts of data and now with Deep Learning, they can effectively make use of that. So I definitely see a rise in demand for deep learning practitioners and we want to support them on Floydhub.

</comment></comment><comment><USER29> Congrats on your launch! Seems like a really slick product.

Have you taken a look at [https://cloud.google.com/ml/](https://cloud.google.com/ml/)? I assumed you would have but didn't see anyone mention it in the comments.

Do you see your product as complementary to a service like that? e.g. could you use the Google service as the execution engine for yours? or do you see that as competition?

</comment><comment><USER30> Is this currently running on a one of the available GPU cloud services (IE AWS, Azure, Nimbix, etc...) or some self hosted hardware?

<comment><USER30> Nevermind, another comment mentions AWS. You seem to pretty much run at cost then (a little less), if you GPU hours are $0.432 @ 2 cores, 32G, and a k80. Then packing a p2 instance would run you $7.2 (or $14.4, etc..) and back of the napkin math says you can pack about 15 gpu jobs in a p2.8x. So you hour cost on a job is $0.48.

Just an observation :).

<comment><USER31> I guess the profit margin comes from the $99/user fee

<comment><USER30> ahhh, thats true.

<comment><AUTHOR> Our Individual plan has no monthly fees, and is purely pay-per-use @ <50% of AWS pricing.

</comment></comment></comment></comment></comment><comment><USER32> Have been using this for a couple of weeks and it looks really promising! It definitely feels like Heroku for deep learning!

</comment><comment><USER33> The github page for floydhub has a lot of open source projects. What parts of floyd is not open sourced?

<comment><USER6> Deep Learning is a very open source friendly community - most of the popular frameworks/algorithms are in fact open source. In fact, Floydhub is built on top of a lot of open source projects. We definitely considered open sourcing the core components of Floyd but the Rethinkdb fiasco made us "rethink" that. Instead, we are supporting the open source community by hosting Deep Learning Docker images, popular datasets, and projects. In the future, we are planning to open source more parts of Floyd but not all (similar to Github).

</comment></comment><comment><USER34> It feels like a good product. I will try it soon!

One small piece of feedback. If you open your landing page from mobile, e.g. My iPhone 7, the header height is changing all the time due to the text changing dynamically. It get hard to scroll past it and keep reading.

Cheers

<comment><AUTHOR> Thanks - fixed now! :)

</comment></comment><comment><USER35> Per second pricing and jupyter mode is pretty nice. Like heroku you're trying to resell AWS and stay competitive which is a challenge. Anyway, this is indeed a common problem and this looks like a solid approach.

<comment><AUTHOR> Fully agree! In the longer run, I believe there might be some great opportunities wrt infrastructure. GPUs instances are super expensive (now combine with long runtimes). Self hosting infra at scale can drastically drive down prices 10x+ (we're already 2x cheaper than AWS). Till we get there, there's a few lower hanging fruits to pick, like streamlining development processes and making things easier. Glad you like the Jupyter Notebooks!

</comment></comment><comment><USER36> This seems pretty awesome, but I can't get it working with my code. Do you have any plans to release extra documentation, or examples of how to adjust existing models to run on FloydHub?

<comment><USER6> Sorry to hear that, we are constantly improving the docs. You can find the latest version here: [http://docs.floydhub.com/](http://docs.floydhub.com/)

If something is not clear, contact us in the communication app on the website. We will be happy to assist you in any way possible.

<comment><USER36> I was able to get a different model working, and FloydHub is great- I'm really finding it useful. I'm looking forward to see where you take it!

</comment></comment></comment><comment><USER31> This looks really cool, I'm slightly confused why the type of CPU intensive workloads only has a single core though?

<comment><AUTHOR> We picked this as a starting point to assess demands, e.g. CPU vs. GPU, before scaling out. We'll be adding more tiers very soon. What kind of instance would suit your needs?

</comment></comment><comment><USER37> Congrats with the launch!

</comment><comment><USER38> Congrats on the launch!</comment>


Ask HN: Review my startup suplify.me - <AUTHOR>

Shared inboxes suck. But it gets even suckier when you also have to keep an eye on twitter, facebook, forums and all the other channels you might use to talk to people outside of your company. We want to help you with that mess.
Suplify launched into private beta about two weeks ago. It is very neat productivity tool. It combines support ticket-like systems with a lightweight CRM and tasks tool. We want to channel most of the conversations you have with customers, so that our users have one single place to check messages. We've had minimal info on our website for quite some time but felt this was not good. We needed some marketing info when we were talking to people. They wanted to tell others about this but we didn't have any marketing material they could pass on.
So over the weekend my cofounder and I wrote together some information on our product and put it on our homepage in hope that we can use this to point potential clients to. Especially as long as we don't have any information brochures yet.
Since I'm not a native english speaker, I also want to make sure that there are not any major grammatical or spelling flaws.
I'd love to hear your thoughts and/or suggestions.
You find us here: http://suplify.me <comment><USER2> I really like the color scheme and the layout of the site. I suggest separating the features section from the sneak peak section a bit more. They seem to run together and my eyes get distracted when I am trying to read them and it gets a little confusing.

Also the call to action (Signing up) seems to fade in the background. Make it stick it so people know exactly what you want them to do!

Keep up the good work!

<comment><AUTHOR> Thanks for your feedback!

I will experiment with moving the screenshots away from the sidebar and make it a normal row in the content. This should make it more easy to see + read the features.

We will think of what we can do to the CTA. I guess its that we have a lot of green in other parts on the homepage, and the green background does not make it stand out.

Thanks again for your feedback!

</comment></comment><comment><USER3> I love the design. It's clean, simple, and NOT all web 2.0 with clouds in the background. It's a bit text-heavy, but honestly I think this is fine. Most of the time when I see a site for a product, I want to read about what it does, not sift through a bunch of pages with huge images at the top.

So good job on the design. I also don't notice any glaring grammatical errors (native English speaker).

One tip I have: the screenshots on the right hand side of "Features" look a bit like they are part of the page, but then they are also cut off so it's a bit visually confusing. I'd either put them all in a row somewhere so it's more obvious they are just images OR fade them out a bit (lower opacity) so it's more obvious they're graphics and not actual page elements.

Overall I love it!

<comment><AUTHOR> I went for the quick fix with fewer opacity for now. Will look into restructuring the screenshots into a row later.

Thanks for your feedback! Glad you like the website :)</comment></comment>


Show HN: Generate perfect semantic HTML - <AUTHOR> http://begriffs.github.io/showpiece/

<comment><USER2> Just a heads up, I'm getting a require.js error and can't see any demos.

Mac OS X 10.8 - Chrome 27.0.1453.116

<comment><AUTHOR> Thanks for reporting. What does the error say?

Alternately you can try the "playground:" [http://begriffs.github.io/showpiece/playground/](http://begriffs.github.io/showpiece/playground/)

<comment><USER3> I have Ghostery installed and noticed the playground doesn't work unless I pause it temporarily. (Chrome 27)

<comment><USER4> Ghostery is definitely the issue.

</comment></comment></comment></comment><comment><USER5> Does writing a nested list in json cut down on the work much? I feel like a ul block is a small "repeater" (a bunch of li following the same pattern) and using this is a "6-of-1, half dozen of the other" scenario.

The concept sounds more useful for a table scenario, especially considering the table might be populated by json to begin with. It would make the HTML work easier and provide a easy connection between whatever feed I'm using and the parsed & rendered table.

If you (or anyone else) is building out new "blocks" that work with this project, I'd prefer to use it for parts of a site/page I'm likely to pull from a live/updated/for_each type section of code.

*ninja edit: How do you do inline code?

<comment><AUTHOR> The original goal of the project was to examine elements of web sites and find a single style of markup that would be flexible enough to be styled like any of them. I started with menus because they vary so much.

While adjusting the markup to fit examples, I found it was easier to translate it from an intermediate description. Small changes in the JSON domain specific language can cause bulky changes in the markup. If you add a "description" attribute to a menu item it transforms into a definition list. If you remove the "url" attribute it erases the anchor and replaces it with raw text.

While the translation has proved to be useful, I think it's not as important as simply identifying a markup standard for menus and other page elements. If developers can stick to such a standard their job gets easier because it will be easy to reuse CSS.

It's kind of like an anti-twitter-bootstrap because the meaning and flexibility of the markup is treated as most important, and styles can be applied à la carte.

You make a good point that pages are often built from repeated server-side partials, and that the looped block needs most attention. I'm curious about the table scenario. Can you send me a link to a table which is populated by JSON?

</comment></comment><comment><USER6> in the "Groups" example, there is an
 nested inside an
 with no list element in between. is this valid markup?

            File

              <fieldset>

Open

              </fieldset>

i would think that an
 must be the direct child of an ol, ul, or menu.

<comment><AUTHOR> Good catch. I'll update the template.

I should really run the outputs through an HTML validator as part of continuous integration...

</comment></comment><comment><USER7> The page seems very . . . static? Is it meant to be demoing something?

<comment><USER4> Pretty sure that's what: [http://begriffs.github.io/showpiece/playground/](http://begriffs.github.io/showpiece/playground/) is for :)

<comment><USER8> Now it makes sense. Where is this demo page linking to the playground?

<comment><USER4> Good question...

</comment></comment></comment></comment><comment><USER9> The use of details/summary as a hierarchy device doesn't strike me as very semantic. Anyone got an explanation which would fit it being so?

</comment><comment><USER10> Sorry I may be confused as to what this is? Is it just for creating semantic menu systems?

</comment><comment><USER11> "Suppose you want to add a main menu. It's not always an easy task."

Tying your shoes are not always an easy task, let's spend 2 years building a shoe tying machine! In a hundred lifetimes well saved in the time it took building the machine on tying shoelaces. I bet it's well worth it!

Edit: "You need simple semantic markup that gives you room to grow."

As far as I can tell your script only generates a list. If it's semantic or not depends entirely on the context...

<comment><USER12> That sarcasm feels unnecessary.</comment></comment>


Ask PG/HN people: How to manage personal growth? - <AUTHOR>

How would/have you managed personal/staff development in an organization?(in a start up or start up-like enviroment) Just realizing that, left alone, I would not care that much to develop the so called "soft skills"; and how much of an impact could that have in my whole experience as a person. I know, traditionally, this is a more bussines oriented matter, but I want the hackers indsigth. <comment><USER2> Good, a concept that doesn't get much attention!

I'm lucky in that I work for an organization that identifies areas that employees need to be competent in and offers a number of courses aimed at getting better in those areas. Also managers have a list of courses that are _required_. Stuff like basic Finance, people skills, interviewing techniques, management skills, etc. As a new first-level manager I'm still working my way through all this stuff and I am learning a lot.

If it were not for this I probably wouldn't even have realized the areas where I was lacking. Instead I've taken a ton of these classes and learned a lot about myself.

If I were designing something like this I'd start with a personality profile that would let the user understand where they needed improvement and how to also improve the areas where they were already strong.

<comment><AUTHOR> Thats exactly my case, I actually was taking a course when began thinking about this; im kinda taken care of, not only by HR people but also my boss; what do companies with just a few employees get to do this? do they even? is it relevant for people in start ups?</comment></comment>


Ask HN: What set your hair on fire in 2012? - <AUTHOR> I'm curious to know what serious technological / IT problems you encountered in 2012. What things made you work late at night, scrambling for a solution? EC2 outages did it for some people. What else? What failed unexpectedly? For what problems do you still not have a solution, and just hope for the best? <comment><USER2> Funny, the "new" technological problems I think OP is referring to are rarely things that are that difficult to deal with.

But these are the things that continue to keep me up at night year after year:

      10. How clever to get with javascript in the browser without going overboard.
      9. How much javascript to depend upon on the client.
      8. Optimistic or pessimistic database locking?
      7. Do it at compile time or run time?
      6. Do it on the client or the server?
      5. Keep data on the client or make the round trip?
      4. Mouse, keyboard, either one, or something new?
      3. Which browsers and/or platforms to support?
      2. Simplicity vs. capability in the user interface.
      1. How to deal with difficult/stupid people

<comment><USER3> The problems as you've listed here have no context, which will make them extremely difficult to answer, if not impossible.

For example:

      > How clever to get with javascript in the browser
      > without going overboard.

Depends completely on whether anyone other than you needs to read/maintain the code base, what kind of advantages the "cleverness" brings (performance, maintainability), etc.

<comment><USER4> edw519 has not listed problems to be solved but dilemmas/trade-offs to be managed.

</comment></comment></comment><comment><USER5> Wordpress E-commerce plugin. Stay. Away. From. It.

There's a reason why it has such terrible ratings and reviews and complaints. It has a lot of features that the others don't, and that tricks you into thinking it's right for your online store. Only to be 1 month in, and start discovering all the terrible programming, glitches, bugs, php errors, and short commings.

Stay away from WP E-Commerce. You've been warned.

<comment><USER6> Especially if your theme, or anything close to your wordpress install (other plugins, etc) use any javascript libraries that might share variables or have different version numbers. Agreed.

</comment></comment><comment><USER7> Sencha Touch + Phonegap, as it turns out it's not very pleasant to try to make a good looking responsive multiplatform app, specially with android browser weirdness and bugs.

Lesson learn, if you care about the user experience there is no other way than native.

<comment><USER8> Couldn't second this more.

</comment></comment><comment><USER9> Jython. It's an unfinished research project/toy/proof of concept that never should have been used in a production environment under any sort of load. Unfortunately my predecessors made the wrong choice and I inherited that technical debt.

<comment><USER10> Can't you easily rewrite the critical parts in Java? (That should be much easier than writing C extensions for CPython.)

<comment><USER9> The problems are in the core implementation of Jython, not in using Jython to solve problems. For example: <http://bugs.jython.org/issue1634167>

</comment></comment></comment><comment><USER11> Finding freelancer peers to sub-contract and refer work to. When you're overflowing with work, it's more difficult than ever to find people through events / traditional slow networking, and there's no good online alternative.

I wish there were a site that wasn't transaction-focused, that provided a way to find people with specific skill sets, and that provided search / profile data that was specific to freelancers vs job seekers (e.g. what's your hourly rate? what's your availability? are you looking for sub-contracting / referrals / collaborations?). And most importantly, that connected to the social graph and provided a good reputation system. If the people I've worked with before aren't available, the next best is people they've worked with before, and rated positively.

</comment><comment><USER12> well for me "hair on fire" means "what were you really inspired about/by" to the point you stay up late working on it. For me:

ML for the working programmer - started down this road as I wanted to make sure I really "get" what is being expounded upon by haskell/shen etc.

shen/klambda - every time I dive into this bottomless pool I come back up with a new insight to how programming could/will someday look.

amber smalltalk - close to my ideal collaborative programming environment - the new helios ide and compiler progress bring it even closer!

edn - seriously I don't think enough hype has been made about what edn can bring to the table, specifically w/ "tagged" data. Stayed up one night and wrote jsedn so I could start to use edn in projects.

datomic - There are no silver bullets in database programming, but datomic is damn close. Trying to wrap my head around the specifics culminated in prototyping datomicism, my smalltalk esque environment for working w/ datomic.

minecraft - which has led to working on a full blown 2d/3d schematic system (stay tuned for <http://schematic.io>) and a rediscovery of "primitive" computer geometry (namely "geometric principles and procedures for computer graphic applications" by Sylvan H. Chasen). This will hopefully serve as a nice open source example of how to meld coffeescript/backbone/datomic and a edn based DSL for helping kids "code by osmosis" as it follows the bret victor principle of showing the code change as the objects are manipulated (and vice versa).

<comment><USER13> Can you describe a bit more about schematic.io? I quick google search didn't turn up much. What is this going to be?

<comment><USER12> Basically a CAD tool for minecraft in which you can create schematics for structures in a 2d editor w/ a live 3d preview. The "use case" is for sharing a complex build plan with multiple users on a "layer by layer" basis so that many people can collaborate either in real or elapsed time. e.g. I may start building a castle and leave it w/ a sign in the center giving the schematic.io url so that another user can come along and make progress on it according to said schematic.

Right now this works for "block" structures e.g. there is not yet the ability to place redstone wires/circuits - but that is coming soon. The use case there being to increase the level of complexity and radness users can build w/o having to "frame by frame" a youtube video on building complex piston doors!

</comment></comment></comment><comment><USER14> By an large, candles. Candles and long hair are a bad combination.

On a serious note, a piece piece of software called Player/Stage which I am working with is driving me up the wall. Learning how three.js and the collada spec work to try and get the two playing nicely. Cross-browser CSS3/HTML5. Weirdness troubleshooting a flash app running with a PHP backend on a hundred android 1.6 tablets you don't have code access to.

</comment><comment><USER15> Chef Scripts. But now that I have Vagrant the fire is out. Seriously. Vagrant is amazing to the point of being nearly a silver bullet like Rails was. Funny thing was, when I first looked at it, I couldn't even figure out why I would want to use it, but now I can't live without it.

</comment><comment><USER16> What killed me:

Ms11-100 security patch. Utter disaster that broke our entire cache and UI layer. Literally our kit jumped from 10% utilisation to 90% overnight resulting in a last minute rollback. Didn't show up in load testing.

business analysts lack of grasp on reality "its only a few billion rows" (to download our audit data via HTTP in CSV format - not going to happen).

SQL Server 2012 pricing.

Mark shuttleworth being an ass over a amazon integration causing us to move all our Linux stuff to debian.

VMware 2tb lun limit on iscsi. This sucks badly.

Compilation speed of c# - this is knackering us. Java is much better as it doesn't have to create monolithic dependency units (assemblies).

Exploding kit.

</comment><comment><USER17> Not knowing how to research 'hair on fire' problems other people have, so that I could create a solution, add value, make money, one-man-startup style. I know it's kinda meta.

Reading Kalzumeus and other entrepreneurs opened the door a little. Lean startup pushed me along the way as well.

</comment><comment><USER18> I got to clean up after a developer went on vacation just before setting up a nightly Java/Spring/log4j/FTP/MySQL/SQLServer process which went crazy over the fourth of July when it couldn't download a file from a business partner and went into an infinite loop.

It managed to send out 250K emails before it was terminated. Everyone's inboxes were flooded. The ops team didn't notice since their mobile devices were out of power, presumably from trying to download the emails...

</comment><comment><USER19> Timezones in javascript. We ended up stitching together 3 libraries into something that sort of looks like 10% of joda-time. Definitely not a solved problem.

<comment><USER20> Which libraries did you end up using?

<comment><USER21> Yeah, which ones, I also have timezone hair pulling for the front end.

<comment><USER21> So far moment.js seems to be the best solution for me.

</comment></comment></comment></comment><comment><USER22> Trying to find the formula to help companies with no MySQL experience (yet running mission critical MySQL instances) get stable, redundant, & backed up _fast_.

There's nothing like being paged 10 times an hour at 11pm to get you motivated to get things fixed.

<comment><USER23> When people in tech say "paged" do they/you literally mean it? Or do you just mean text message but the terminology has stuck as a throwback to the days when pagers were how people were on emergency call.

I suspect there still is a small pager business out there somewhere, perhaps for doctors and adhering to some legacy hippa rules, but they must be sparse.

<comment><USER24> I carry around an honest-to-goodness old-school pager. I love it. It works even when I get next to no cell signal, it works when I screw up the newest ROM of my phone. It works when I'm out of battery. It runs weeks on a AA.

I have messages go to SMS as well, but when the servers are down, damn it, I need to know.

</comment></comment></comment><comment><USER25> Online payments systems in South America.

</comment><comment><USER26> After building online business for the last 16 years, I took serious venture capital for the first time in 2012. So managing that relationship and meeting someone else's expectations has shaped my 2012 (set my hair on fire). I have a very patient investor, and it's still not easy, I don't know how people deal with the real sharks of VC.

</comment><comment><USER27> Sending payments via Paypal. Yes, I understand it is very simple, but Paypal has restrictions in my country. And i only have debit card and no credit cards. And debit cards don't work online apparently. I can connect my back account but that requires my pan info (like SSN) and a bank confirmation. My PAN info is still not confirmed. It surprises me how difficult it can be to send money online when you are in a particular country. No only that but I was unable to use my debit card anywhere on the entire internet. I wanted to buy windows 8 and had to use my uncle's credit card who does not live in this country. Had to figure out a way to send him money too. It's ridiculous how you become a second class citizen if you don't have a credit card. This is 2013 (in my timezone it is) and there has to be a better way to transfer money.

<comment><USER28> Prepaid Visa/Mastercard cards work and they're pretty much the equivalent of a debit card. You put money on them and use them to buy stuff online.

<comment><USER27> AFAIK we don't have them here. Even if we had they won't work online due to government restrictions. I live in India and they had a few problems with how paypal can be used to launder money.

</comment></comment></comment><comment><USER29> Clients changing schemas or replacing knowledgeable employees at critical stages of projects. Microsoft security issues, licensing and lack of vision with new products. Network and SAS provider outages. Technical debt at current PoE.

</comment><comment><USER30> How to combat frauds with e-commerce transactions:

1\. Pro-actively block Tor nodes and all hide-my-ass type of open proxy. To e-commerce merchants, tor node users are not freedom fighters, but fraudsters, every-single-one-of-them.

2\. For credit card transaction, always check the return "credit card" country

3\. Shopping carts need to track IP address. Fraudsters always can bounce from Brazil to UK to Italy in a single cart session.

4\. Have to come up with a score system to determine fraud risk of transactions. Maxmind API seems to be helpful.

5\. When in doubt, ask user to take a picture of their credit card, and upload to your site.

<comment><USER31> You really cannot be serious about asking a user to take a photo of their credit card and upload it.

<comment><USER30> The only other alternative recommended by payment gateway, would be to ask you to fax your passport. The point is, to prove that you really do process the card, not getting it through some shady sites.

Try registering an hosting account on many "managed hosting" company, you are required to fax your passport to them. I will be offended if somebody asks me to do so.

Uploading credit card image is definitely less evil than faxing your passport.

</comment></comment></comment><comment><USER32> Getting require.js to play nicely with rails 3 asset pipeline on heroku. It would all work fine locally but compiling js always had some dependency issue. Finally got it working some week or two later.

</comment><comment><USER33> This year I switched to Ruby on Rails after years of doing mostly Java. I'm quite happy with it, but by far the biggest pain point has been Ruby's dependence on native code libraries. Dealing with CLASSPATHs and Maven was never fun, but it sure beats trying to maintain a working set of Ruby gems, many of which have cascading dependencies on C++ compilers, Python interpreters, SSL libraries, Javascript runtimes, etc.

</comment><comment><USER34> facebook APIs. Random breakages, incomprehensible changes, unanswered PRs etc. And AWS outages. And Heroku outages. And OAuth integrations.

Basically, everything not 100% under my control keeps going wrong. And so does everything under my control, but apparently causing smaller fires.

<comment><USER35> Yup.

Just about everybody else on this thread seems to be thinking at the wrong granularity.

I've been thinking about your problem all year. <http://akkartik.name/blog/libraries2>

</comment></comment><comment><USER36>
        1. Authorize.net's ARB (integrated with it before really good solutions came into view)
        2. A chaotic - but good - life (difficulty in regaining my daily routine due to startup + girlfriend + other commitments)

</comment><comment><USER37> Refactoring one of my old websites (200,000 lines of code). Code was horrible. I think in 5 years I will tell the same story about my recently refactored code =)

</comment><comment><USER38> My biggest pain in the ass this year was as simple as disk space constantly running out on our VPS servers. That, and automating postgres backups.

<comment><USER39> How did you automate your postgres backups? I'm going to be tackling this for a client soon and have a few ideas, but would love to hear how you did it.

<comment><USER38> Well, first of all I mistakenly assumed our VPS backups would be sufficient, but the first step is actually dumping the database every interval you'd like. If you don't dump it, you might end up with a corrupt database should you be relying on the VPS backups.

Then, we set up a process to zip the database dump and send it off to tarsnap. We delete the backup done 30 days ago each time we run.

It works fine for us, but I much prefer using Heroku and not having to worry about it :)

</comment></comment></comment><comment><USER40> Paypal payflow integration. There is a zoo of paypal products, and outdated paypal integration docs that obfuscate the process.

</comment><comment><USER8> The incredible price of large (i.e. > 128GB) SSD's for Ultrabooks (my development machine of choice).

</comment><comment><USER41> Scaling. Oh god. Scaling.

</comment><comment><USER42> trying to deploy node.js on a private VPS running on CentOS, to say I was in over my head is an understatement but I did find a new appreciation for sysadmins

<comment><USER43> I was shocked to find out that the node package for CentOS is not actively maintained anymore...

</comment></comment><comment><USER44> Realizing that I would have to use Javascript.</comment>


Brexit: Berlin eyes Britain's tech talent - <AUTHOR> http://www.bbc.co.uk/news/business-36642560 <comment><USER3> Eh.. Berlin is a freaking bad place for tech-talent to be in. I relocated there and worked for about 6 months before I actually came to the conclusion that it is nothing more than a rip off. I would happily move to the UK over Berlin in 9.5 out of 10 times.

The German tax rates for singles are ridiculous (40+%) and the salaries are way below what the UK pays. In addition, the German culture's work mentality is very similar to that of factories. On top of that, the benefits are non- existent compared to the UK (let alone the US).

[http://demonshalo.com/the-sad-state-of-the-european-tech- sce...](http://demonshalo.com/the-sad-state-of-the-european-tech-scene/)

<comment><USER4> > German culture's work mentality is very similar to that of factories.

Which is very positive for workers.

I rather have that than the exploitation of workers in IT without unions.

I can enjoy my evenings and weekends without spending a second thinking to get reached from work, and if I need to be on call I will get properly paid for it.

<comment><USER3> > Which is very positive for workers.

for YOU! not workers. Thinking that what you like is good for everyone else is rather narcissistic don't you think?

> I rather have that than the exploitation of workers in IT without unions.

What does unions have to do with anything? ... I was referring to the work mentality as a cultural phenomenon. It is nothing to do with unions -.-

> I can enjoy my evenings and weekends without spending a second thinking to > get reached from work, and if I need to be on call I will get properly paid > for it.

That's what I am talking about. Your work for you is just that, work. Which is perfectly fine. However, There is no way for Germany to ever get a remotely interesting tech scene with that mentality in place. That's not how you foster a startup culture.

<comment><USER5> Won't mention my overall opinion, though I have some knowledge. Just a warning about this particular point:

"I'm Zach Holman. I was number nine at GitHub ..."

"More than once I had been confusingly introduced as a founder or CEO of the company."

"I had been the top committer to the main github/github repository for the last two years, I had just led the team that shipped one of the last major changes to the site, and around that time I had had a mid-year performance review with my manager that was pretty glowing ..."

"I'm pretty broke after working part-time and living in San Francisco for so long. Even though I helped move a company's valuation almost two billion dollars, I haven't made a dime from the company outside of making a pretty below-to-average salary. That's after six years."

([https://zachholman.com/talk/firing- people](https://zachholman.com/talk/firing-people))

</comment></comment></comment></comment><comment><USER6> It can eye the tech talent all it wants.

I've lived in Frankfurt and in Berlin. I've lived in Paris. I've lived in Barcelona.

I've spoken with many stock brokers.

The one common denominator that I have found. Is that you can't just beat the feel of London.

I know if I had the choice, to live in Europe or live in London. It certainly would not be Europe.

<comment><USER7> The "feel" of London sounds so vague. What do you mean by it?

<comment><USER8> OP is talking about that sinking feeling one feels when over half your paycheque is spent on rent.

<comment><USER3> Yea I know that feeling. It is the same feeling I got when I first saw that my tax rate in Berlin was @ 42% and then realized that whatever remains has to go into paying for the rent anyway.

<comment><USER9> You do mean the _marginal_ tax rate, right? Not the average tax rate. To be taxed @ 42% (average) in Germany your income would have to be way north of 300k€ [1]. With that kind of income, your social security (health, unemployment, pension) would already be somewhere below 10% so nothing to complain about, really, given the benefits (yes, I know the benefits are no longer what they never used to be).

Add to that the absurd housing costs in, say, London, and things don't look too shabby for Berlin. Although you need to be tolerant towards the locals. I can understand some people having problems with that nonchalant "don't care if you care because, you know, I don't care" attitude that sometimes shows. But if you're capable of letting the occasional inflammatory/stupid remark slide, these guys and gals are quite nice. (Protip: This ability is great to have pretty much anywhere. Hanlon's Razor, anyone?)

To me, London is great to spend a few days of vacation. The atmosphere, the people, British humour, all great. Still, I never felt the urge to permanently move there to work. Too crowded, hard to get a decent flat... I guess it's not always just the money.

[1] [https://en.wikipedia.org/wiki/Taxation_in_Germany](https://en.wikipedia.org/wiki/Taxation_in_Germany)

<comment><USER3> > To be taxed @ 42% (average) in Germany your income would have to be way > north of 300k€

That is flat out wrong... If it wasn't for the sensitive information, I would have posted my own tax returns here for all to see.

<comment><USER9> Erm, no. It is not. Any site describing the German income tax will tell you that what I wrote is perfectly true. You can verify quite easily for yourself by following the conveniently provided Wikipedia link.

Maybe you misunderstood your tax return?

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> Generally speaking the largest portion of tech workers are young. Young usually means less tied to the ground they're standing and much more willing to relocate.

Once an exodus has begun the benefits and social compliance will pull a large portion of hold-fasts into wherever the tech sector would be.

The problem, however, is every country wants the tech sector to thrive, yet none are really attractive yet. We had a post on HN earlier about Romania, there was one yesterday about Spain and Netherlands has been on here twice.

It's a chicken and egg problem, where the companies move the people will follow, you can't expect people to go to Berlin if there aren't enough jobs and it's a hard sell to a company to move if there is no talent.

But companies absolutely have to make he first move.

<comment><USER11> > you can't expect people to go to Berlin if there aren't enough jobs

But people have been moving to Berlin for years now, regardless of what Britain was doing.

Affordable flats are already scarce and only getting more expensive - <USER2> anything, I'm afraid Brexit will only accelerate that development.

</comment></comment><comment><USER12> I noticed that as well. Part of me is saying 'easy come easy go' \- weightless nomads &c.

Part of me is thinking where do England's new companies come from.

We'll find out in the next three to five years. The news machine wants dramatic events _now_ but this one is more of a glacier.

<comment><USER13> I agree that the outcome will take time to truly be seen, especially in startup scene.

It turns out though that roughly 30% of VCs money in the UK comes from the Europe Investment Fund (as Limited Partners in their funds), which is funded by the European Central Bank, of which the UK is currently a shareholder, although it will only be as long as it is a member of the EU.

From an FT article on the subject: "To put some numbers on it, as of the end of 2015, there were 144 UK private equity funds supported by the EIF and almost 30,000 small businesses. The EIF estimated that €3bn in equity capital in 2015 had been enabled by its €655m direct investment that year."

I wonder if that will have an impact on the ability to attract talent?

</comment></comment><comment><USER14> You gotta love how incredibly anti-Brexit the media are. They have to construct all kinds of imaginary scenarios (that haven't happened and probably will never happen) to bash the democratic decision of the British people.

<comment><USER15> They democratically elected to basically jump off a cliff so you can't blame the media for having the time of it's life.

And referendums aren't always a good idea, prop 8 in CA was overturned because we eventually realised its a bad idea.

<comment><USER14> _They democratically elected to basically jump off a cliff_

Scare-mongering. If you _have_ to be in the EU to be able to trade, yes, to survive, then it is time to destroy the EU once and for all. No single entity should be able to terrorize the citizens of a nation state like that.

<comment><USER16> Straw man - no one is saying we won't survive but you would have to be monumentally dim to not perceive that what has just happened, and what will continue to happen in Britain as a result of this decision will be orders of magnitude worse than the alternative - at least in the short to medium term. I can't predict the long term future and it may well be that the UK somehow manages to come out of this with more than the south west still part of it but the path there will involve a lot of suffering.

<comment><USER14> _Straw man - no one is saying we won 't survive_

He said jumping off the cliff. That is literally just saying the Brits won't survive. Besides, England will be able to make a trade deal with China (something the EU doesn't manage to do). Medium- to longterm, the benefits for the British people will be vast - and I'm not even starting to argue about political independence.

<comment><USER16> Jumping off a cliff != certain death, but I guess thats splitting hairs. You seem very confident in the face of expert opinion and evidence in front of you. Does the fact that the people who have masterminded this spectacle seem to have no plan for what happens next worry you in the slightest?

I notice you use England, possibly a mistake but it seems this is the underlying attitude of the Leave vote - England is better off by itself. Which strikes me as frankly hilarious given how much we have to offer the world besides grumpiness, self deprecation and financial services...

<comment><USER14> Well, the plan for what happens next depends in a large part on negotiations with the EU - who doesn't seem to care about negotiating if there is no clear wakeup call and time limit (like 2 years after taking article 50). The vote was a first step and clear signal that the British people are unhappy with how the EU treats them.

I meant to say Britain so yes it was an error. The tensions between England and Scotland (and to a lesser degree Gibraltar) are unfortunate, especially that the leave vote increased them again. Same with northern Ireland tensions. I have no opinion on whether or not they are supposed to stay together since this seems to be a problem rooted deeply in culture that I might not be able to fully comprehend.

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> It will mostly depend on how the work permit situation plays out (and I suppose on the pound not tanking and making wages less attractive). A lot of London tech companies have a non-trivial number of non British employees. If there's any sort of work permit hickup that could be pretty troublesome.

Another advantage of London, the financial sector could get trimmed substantially if there's an exodus to Frankfurt. I'm not as bullish on London not facing major issues as a startup hub due to the Brexit as most people I've talked to.

<comment><USER18> The UK government has demonstrated a willingness to help the UK's tech sector get the talent it needs from outside the EU by streamlining the process for getting Tier 1 visas. Even if movement of labour _is_ restricted after Brexit, I expect there to be a huge difference in the restrictions applied to unskilled migrants versus skilled migrants. In short, I don't think UK companies need to worry about being able to bring in the sort of talent they need.

The fate of London's financial sector will depend on what sort of trade deal the UK and EU negotiate. The worst case scenario would be if the UK was denied access to the single market for financial services, including "passporting". However, the UK imports a lot more from the EU than it exports[1], so countries like Germany and the Netherlands are likely to accept a deal which means London continues as Europe's financial capital, in order to protect their own economies.

1: [https://twitter.com/JackGavigan/status/747338413754331136](https://twitter.com/JackGavigan/status/747338413754331136)

</comment></comment><comment><USER19> I live in Paris, politicians here say and hope for the same thing.

</comment><comment><USER20> The article seems to be mostly just comments from shovel salesmen.

</comment><comment><USER21> Berlin = 0% equity. That's all you need to know.

<comment><USER22> Well, some people prefer real money to vague promises.

<comment><USER21> "Arm aber sexy" \- Berlin's motto for you ("poor but sexy").

You will be paid super low comparing to other big cities (MUC/HBR/FFM), your rent will be already on a very similar level, your managers will be selling you the "vibe", you'll be expected to work like a machine, and the only way for you to escape by winning the startup lottery by having significant chunk of equity won't be available to you. Win!</comment></comment></comment>


Haskell in the Datacentre - <AUTHOR> https://simonmar.github.io/posts/2016-12-08-Haskell-in-the-datacentre.html <comment><USER2> > At Facebook we run Haskell on thousands of servers

Wow, I didn't know this. Anyone knows for which programs they use Haskell? The article doesn't say anything.

<comment><USER3> its for spam detection. Everything that gets posted on Facebook is processed by this code. There is a talk on it from a year or two ago at a functional programming conference. I'll see if I can find it and edit this post.

<comment><USER4> I remember this one [https://m.youtube.com/watch?v=UMbc6iyH- xQ](https://m.youtube.com/watch?v=UMbc6iyH-xQ) is that it?

<comment><USER3> No, but I apparently didn't bookmark it and I'm having trouble finding it again.

This might be it, don't have time to verify right now: [http://cufp.org/2015/fighting-spam-with-haskell-at- facebook....](http://cufp.org/2015/fighting-spam-with-haskell-at- facebook.html)

</comment></comment></comment></comment><comment><USER5> Really great to read more about real world large scale Haskell use.

I'd be curious to find out if the success FB has had with Haskell for spam filtering means they might consider Haskell for different parts of their stack too? Does anyone have any insight on this?

</comment><comment><USER6> Happy to see they pushed the changes to the upstream GHC!

</comment><comment><USER7> > GHC's runtime is based around an M:N threading model which is designed to > map a large number (M) of lightweight Haskell threads onto a small number > (N) of heavyweight OS threads. [...] To cut to the chase, we ended up > increasing N to be the same as M (or close to it), and this bought us an > extra 10-20% throughput per machine.

Ah, yes. As a Go developer, really wish Go moved to an 1:1 threading model.

<comment><USER8> 1:1 in Go would mean moving servers from blocking-style code to callbacks/futures.

(I believe the story is slightly different in Haskell.)

Nice sequential blocking-style code is my favorite thing about Go. I realize it's not C, and it makes runtime work much harder, but the payoff up in complex servers is completely worth it.

<comment><USER7> > 1:1 in Go would mean moving servers from blocking-style code to > callbacks/futures.

No, it would not mean such a thing at all. Where did you get this idea?

In fact, on some operating systems/linker combinations, gccgo uses a 1:1 threading model.

> Nice sequential blocking-style code is my favorite thing about Go.

Mine too.

<comment><USER8> It would in practice.

A typical server under load has more outstanding requests to answer than it can OS threads. It is not uncommon in a Go server at Google to find a million or so goroutines.

If you want to have an OS thread that can also run C code, it needs a large enough stack to run C code. A million such stacks is not practical.

On an OS that lets you avoid creating stacks with threads like Linux, and which has quite light-weight kernel-side accounting for threads, it would be possible to give each goroutine a thread. You would still need an N:M pool for cgo.

As an added problem, 1:1 is slower for certain kinds of common channel operations. Anywhere you have an unbuffered channel and block waiting for a value, a 1:1 model requires a context switch to pass the value, whereas M:N means the userland scheduler can switch goroutines on the OS thread.

It is precisely these benchmarks that led Ian to implement M:N in gccgo. If there are combinations where it is 1:1, that is either a new decision he has made in the last year, or (more likely) OS/ARCH combinations that he hasn't moved to M:N yet.

I have seen a similar attempt at 1:1 lightweight tasks in C++ that ran into this. Without the ability to preemptively move tasks between OS threads, it ran into performance problems. Programs that needed the speed in that model had to switch to a futures-style of programming.

<comment><USER7> > A typical server under load has more outstanding requests to answer than it > can OS threads.

First of all, I object to the use of "can" here. There was a time where this was true, but, as I said, that time has passed. There is no problem running millions of kernel threads today.

That being said, there are advantages of not doing that. Right now Go uses a network polling mechanism like epoll to implement network I/O without consuming threads. This is totally orthogonal to the 1:1 vs. N:M discussion. Go can use 1:1 scheduling while still performing asynchronous I/O behind the scenes, just as it does today.

The nature of the mapping between g's and m's does not influence the other {g, g, ...} -> {g0{m}} mapping used to implement network I/O.

> It is not uncommon in a Go server at Google to find a million or so > goroutines.

No problem with that, they can continue to do so.

> If you want to have an OS thread that can also run C code, it needs a large > enough stack to run C code. A million such stacks is not practical.

C code needs its own stack, but this is again orthogonal to scheduling. Right now there is a N:1 mapping {g, g, ...} -> {g0} which makes it easy to switch to g0 stack. This will have to change to a N:M mapping {g, g, ...} -> {g0, g0, ...} and Go code would have to acquire C stacks, just as it acquires any other resource that it needs to use.

This is not expensive, in fact, C calls are now expensive precisely because there is a deep interaction between calling C code and the scheduler (that needs thousands of cycles). All this cycles will Go away. In a 1:1 model _all_ you'd have to do is acquire a C stack, which is very fast and happens in userspace for the uncontented case (the common case), probably less than a hundred cycles.

> You would still need an N:M pool for cgo.

As explained above, you need a N:M pool, but this does not need to be integrated with the scheduler any more, making it much simpler to implement (and more performant).

> Anywhere you have an unbuffered channel and block waiting for a value, a 1:1 > model requires a context switch to pass the value, whereas M:N means the > userland scheduler can switch goroutines on the OS thread.

Only if you naively implement channels as mutex-based queues.

There is still a runtime underneath that can switch stacks on different threads. I want to move general scheduling decisions out of the Go runtime and into the kernel, but pointwise, stateless, swap-of-control type of things can still happen synchronously in the Go runtime.

<comment><USER8> I do not believe the networking issue is orthogonal. I believe you will find that any communication between two OS threads is significantly slower than switching one thread between an active and a parked goroutine. Futexes are slow and spinlocks burn CPU. (This is exactly the case the C++ model I mentioned ran into, and why gccgo got an N:M model.)

But as I said I'm happy to be convinced. The easiest way to demonstrate it would be to move gccgo linux/amd64 back to 1:1 without hurting the channel benchmarks. You could use that as an argument that fanning out an epoll loop among threads can be made fast.

</comment></comment></comment></comment></comment></comment><comment><USER9> I think the bigger question is why they're using a C++ Thrift server. Haskell is good at transforming data - they should be able to use an OS socket and do the message decoding/encoding in pure Haskell.

<comment><USER10> _the bigger question is why they 're using a C++ Thrift server_

Because it's already there and it already works just fine.

It's so frustrating to see this sort of comment on nearly every post about a language outside the mainstream. It's a religious argument and not a productive one.

<comment><USER9> > Because it's already there and it already works just fine.

Well clearly it didn't work just fine if its threading model was interacting poorly with that of their Haskell code. If there's a mismatch between those two components then that will continue to bite them in the future, and it's legitimate to ask whether moving the boundary would be a better way to solve the problem.

<comment><USER11> your right, makes more sense to have both in c++

<comment><USER12> That was the last tried solution, didn't work out well. And what is holding is nothing haskell specific. Is just wasting effort re implementing good broad used code at Facebook.</comment></comment></comment></comment></comment>


Sublimetext is not dead, new version! - <AUTHOR> https://www.sublimetext.com/3 <comment><USER2> Trying to convince myself to buy the darn thing. I use it sparingly but I have found nothing else that even remotely compares. Been trying out others lately (Atom, Komodo Edit, etc) and Sublime seems to just kill them from a performance standpoint.

Might just be a personal preference, but something about its performance and way of doing things perfectly fits my usage style.</comment>


Experian acquires London startup ClearScore for £275M - <AUTHOR> http://uk.businessinsider.com/experian-buying-clearscore-for-275-million-2018-3?r=US&IR=T <comment><AUTHOR> The article says it is a 2 year old startup but its more like 3-4. Incorporated in Sep 2014</comment>


Yahoo's Marissa Mayer tops the best paid female CEOs list with $36M - <AUTHOR> http://www.dailymail.co.uk/news/article-3610711/Female-business-leaders-earn-average-male-competitors-second-year-row-Yahoo-s-Marissa-Meyer-topping-list-36-million.html <comment><USER2> How the FUCK do CEOs get paid so much to run companies further into the ground?

<comment><USER3> golden handcuffs, golden parachute

</comment></comment><comment><USER4> Well that's definitely different from the original title.</comment>


Washington Redskins trademark canceled by U.S. Patent Office - <AUTHOR> http://www.chicagotribune.com/business/breaking/chi-washington-redskins-trademark-cancelled-20140618,0,1927895.story

<comment><USER2> I support this, but it makes me wonder: how can political correctness be part of the USPTO's duties? It seems reasonable to me that someone should be allowed to have a trademark on a distasteful, disparaging, but original and representative symbol. I imagine that's how much of the pornography industry's trademarks work.

<comment><AUTHOR> This is what struck me about the decision. The article claims that "[t]rademarks that belittle other groups are not permitted under federal law."

Yet it seems problematic that the office gets to decide then thi is the case.

If something is legally protected speech, however offensive it may be, it should be eligible for trademark protection.</comment></comment>


Ask HN: Career Advice in software development field - <AUTHOR> I am fresh graduate with engineering in non CS field. But, I have been programming for almost 2 years. Now, I'm not sure of which way should I go. I am really interested in Systems programmnig/engineering but as it seems the entry point in that field requires experience. Other popular field is Application development (majorly web development) which I fear if I follow will remain constrained to that field only. The last popular one is related Data science of I'm unsure too about how much experience is needed, what are the prerequisites etc.<p>Would really appreciate any advice. <comment> jerome-jh The first few years will be critical for your career, and after 10 years, it is very hard to move to another field.

If you have a bit of time, try finding a job in your area of interest. You should find junior positions in any of them. If not, take one which is close enough and hop from there towards your goal. Not being graduated in CS should not be a problem as long as it is an engineering degree.</comment>


The Amazon Tax - <AUTHOR> https://stratechery.com/2016/the-amazon-tax/ <comment><USER2> AWS isn't a monopoly unless customers make it one. Tying a company's infrastructure to a single vendor is irresponsible and shortsighted.</comment>