The Early History of Smalltalk - b-man http://gagne.homedns.org/~tgagne/contrib/EarlyHistoryST.html <comment><AUTHOR> A classic piece which contains one of the best things ever written about design:

 _Take the hardest and most profound thing you need to do, make it great, and then build every easier thing out of it._

It's interesting to me that the idea that inspired Kay to this insight was FEXPRs: letting functions control the evaluation of their arguments. FEXPRs have long been banished from the Lisp world, which makes a sharp distinction between functions and macros (only the latter get to control evaluation). I still don't quite understand the reasons for this, and it seems like Kay doesn't either -- or at least didn't when he was inventing Smalltalk.

<comment><USER2> There's no real difference between a FEXPR, and a macro that calls eval or apply during expansion. The real meaning of "banishing" FEXPRs is that we actually get something _extra_ : guaranteed-non-macro functions that behave according to a predictable pattern (evaluate arguments, pass them in, return a value.) Functions are basically a codified Lisp design-pattern, while macros (or FEXPRs, as they may be) are the real core of Lisp's abstraction mechanism, and really what make Lisp, Lisp.

It's just the same with Smalltalk: functions take lambdas as arguments, not primitive or composite values. "Receives values" is simply a design pattern.

<comment><AUTHOR> That's a pretty good explanation; if it's accurate, I wonder why no one else explains it that way.

In Smalltalk, I thought arguments are always evaluated unless they're blocks. Is that wrong? Or is it that, at a lower level, everything's a block, and functions are implemented on top of that as something that always evaluates args?

<comment><USER2> No, you're right—arguments _are_ always evaluated unless they're blocks in the full Smalltalk design we have today. However, like I said, this is a _codified_ design pattern—the language could get away without doing it while still retaining the "core Smalltalk conceit" of everything being a message sent to an object. This is basically what the io language does—it's basically a Smalltalk at its core, but with less cruft.

</comment></comment></comment></comment><comment><USER3> This is really odd, I'm having a bug on chrome windows where the html gets printed out as plaintext. MIME type issues on the server side maybe? Other than that, looks like an interesting read.

<comment><USER4> Yeah...

       Content-Type: text/plain; charset=ISO-8859-1

Interesting that Safari is "smart" and ignores the content type in this case.

<comment><USER5> My Safari printed the raw HTML source at first, but rendered the HTML upon reload.

<comment><USER6> Yeah, mine too. I'm almost motivated to go look why... but not quite.

I looked for a 'View as' menu item prior to reloading, but very interesting that a reload fixed it.

</comment></comment></comment></comment><comment><USER7> Really good read, abeit a bit long. It has a lot more details about the Xerox Parc days then the movies like "Triumph of the nerds had". It was interesting to hear a deeper understanding of the actor model in Smalltalk and simula</comment>


Top Hat Raises $22M to Go After Pearson, McGraw-Hill - <AUTHOR> https://www.bloomberg.com/news/articles/2017-02-15/top-hat-raises-22-5-million-to-go-after-pearson-mcgraw-hill <comment><USER2> I teach biology at a community college and most of the services listed in that article wouldn't improve my courses. I've already switched three of my four courses over to open textbooks, which students can download for free or purchase for the cost of printing. I use Openstax for these.

[https://openstax.org/subjects](https://openstax.org/subjects)

For the online learning software, I've also dumped the publisher products and switched to the free spaced-repetition software Memrise.

I think most of my colleagues will be moved over to open educational resources within fifteen years, and I'm not sure there's long-term profit to be made in this market.

<comment><USER3> Check out [https://cogenteducation.com/products](https://cogenteducation.com/products) when you have a chance. We make interactive case studies for biology that go much further than any textbook, digital or otherwise.

We have a free trial: [https://cogenteducation.com/trial](https://cogenteducation.com/trial)

 _full disclosure: I 'm a developer there_

<comment><USER4> > I think most of my colleagues will be moved over to open educational > resources within fifteen years, and I'm not sure there's long-term profit to > be made in this market.

Sounds like he's looking for a free solution, not trialware.

<comment><USER5> "Interactive case studies" is a value-add that might be worth paying for.

<comment><USER6> What will you be paying with after your 5 to 30k/year tuition? You can't bleed a stone.

<comment><USER3> I agree. It's rare for a teacher to pay for our cases out of their own pocket; it's generally the school system who purchases a package out of their yearly budget.

</comment></comment></comment></comment></comment></comment><comment><USER7> I absolutely hated Top Hat during my undergrad. It was just another thing I had to buy that teachers used as an excuse to be lazy because it was ~interactive~. You were so tied to a device for all your classes that if multiple teachers used it it wound up becoming a pain.

I noticed it was mostly instructors who taught one-off elective/strictly credit classes that used it in almost a punitive way to make sure you paid attention to them.

edit - another ironic thing regarding the title of the article, is that the instructors would upload the Pearson, McGraw, etc provided slides with the textbook to tophat.

<comment><USER8> Do you really expect an underpaid adjunct _not_ to outsource the grading to the department's textbook company of choice? Adjuncts don't get to pick textbooks.

But what's with the griping about textbook publishers? The overreliance on adjuncts allows the university to charge lower tuition and the publishers make the feat possible!

<comment><USER7> So the cost should be pushed onto me instead? I understand the struggles of adjuncts and grad-students as they were most of my friends in Undergrad because I came into school older. But not one of them used Top Hat or a clone equivalent.

<comment><USER8> The cost will continue to be passed on to you until there is a robust discussion about the financing model and the working conditions in academia. If any change is going to happen it's going to come through student effort; despite their protestations the position of the unions and the professors is to preserve the _status quo_. Back then we'd call this "revolutionary consciousness".

The goals of the students will also need to be taken into account. A colleague gave up giving detailed comments on the lab protocols he graded, the only thing the students were interested in was the grade. My experience matches. It's understandable, at our institution students mostly plump for medical school or associated profession, and it's the grade that counts, not the subject knowledge. The GPA really haunts you.

<comment><USER9> I'm sorry you're so jaded, but how is textbook/online pricing not relevant to the financing model? There's no need to dismiss that avenue of conversation.

</comment></comment></comment></comment></comment><comment><USER10> As a developer in the Toronto area, I suspect I would be working for Top Hat if their recruiters were better. I've had two or three interactions with them and each time they completely dropped the ball.

Extra funding is great, but if they can't hire, they can't use it.

<comment><USER11> what are some of the other more interesting companies/startups in Toronto?

<comment><USER12> Shopify seems interesting.

There really are not that many tech companies in Toronto (and that are hiring unless you have 5+ yrs of experience).

Either 1. you have very very early stage startups (that expect you to work for free till they get funded),

2\. funded startups that have a short runaway and expect you to work like crazy till they get to the next round of funding (during an interview with a fintech startup - it was mentioned that they expected me to work for ~12+ hrs / day but I'd still get paid for only 8 hrs. But when they scaled up, I'd be "rewarded" for my hustle, grit and commitment".

3\. Big companies (Google, Mozilla, etc.) who are looking for quite a lot of devs but seem to be insanely picky about hiring them.

To add to this, there seem to be a lot of devs than jobs (or that companies can go for long without hiring for those open roles).

<comment><USER13> As someone who's worked as a Software Developer in Toronto for the past 13+ years, I totally disagree with this assessment.

First off, very few very early stage startups expect you to work for free until you get funded. Exceptions being that you're a co-founder or an unpaid intern that never touches code.

Secondly, while the startups in Toronto are probably less well funded than the ones in the US, not all require you to work for 12+ hours a day while paying you for 8 hours. It looks really shady to set these expectations especially since we have clear laws around overtime pay in Canada (describing such a work situation to friends will raise eyebrows -- definitely not the standard practice, whatever industry you work at).

Lastly, I'd say that any place worth working at (big or small) will be insanely picky about who they hire. Current employer included.

All of the above are from personal first hand experience. Of course I haven't worked for every single tech company in Toronto but I have worked for several (mainly early stage startups).

<comment><USER12> > First off, very few very early stage startups expect you to work for free > until you get funded. Exceptions being that you're a co-founder or an unpaid > intern that never touches code.

Obviously this is not something I have a lot of statistically accurate data for but, me and my friends are recent graduates and collectively have applied to quite a lot of very early stage startups in Toronto and out of those, we all have experienced the bait and switch of unpaid work till funding at least 50% of the time.

> Secondly, while the startups in Toronto are probably less well funded than > the ones in the US, not all require you to work for 12+ hours a day while > paying you for 8 hours. It looks really shady to set these expectations > especially since we have clear laws around overtime pay in Canada > (describing such a work situation to friends will raise eyebrows -- > definitely not the standard practice, whatever industry you work at).

I didn't say all startups are like what I've mentioned. The one I'm currently working at is awesome! That said, it's more often the case that startups use their short runaway as an excuse to make you work long hours (and just to be clear - the 12hr / day was not an estimation - I was literally asked my thoughts about it during an interview - that company is still recruiting on HN Who's hiring for more employees and a quick google search shows they have increased headcount to 35 people this year and is profitable).

> Lastly, I'd say that any place worth working at (big or small) will be > insanely picky about who they hire. Current employer included.

I have 0 problems with employers being picky. I worked closely with the CEO of a previous startup and I totally understand that. What I don't understand is the point of claiming you need to hire a developer (with a start date of immediately) and then leaving the position open for months altogether, or having crappy working conditions.

BTW I've interviewed at Nulogy (if that's where you are currently working) - Since this is a throwaway I can't give too much details, While I was still disappointed for the reason I was rejected it was one of the good interviews I've had (interviewer was knowledgeable, I learnt about the company and product, the tech stack, the problems they were facing, what was expected of me, and just some personal chit-chat during the coffee walk).

</comment></comment></comment></comment></comment><comment><USER14> I've worked at Top Hat for the last couple years (on the engineering floor) and it's been nothing but positive. Out of all the Toronto-area startups I've worked at in the last 6 or so years Top Hat seemed the most serious about succeeding. And I love the product-space – one I can potentially actually make a difference in. The hiring process was tough, but I didn't think it was unfair.

I worked previously at accounting and marketing experience startups in the area, too. Their products always seemed awkwardly positioned — not silly enough to be fun like Snapchat or solving a serious enough need to be Shopify.

Anyway take that for what you will. Just my 2 cents and thoughts (admittedly biased) on working here the last couple years. They've been some of the most productive ones of my career so far.

</comment><comment><USER15> I am the VP Engineering at Top Hat. Our biggest goal is to create an environment where talented people love coming in to work every day and enjoy being a part of something great! I think disrupting one of the last old media holdouts is more than a worthy goal.

In the past couple of years we've grown our company from under 100 employees to well over 200. All this growth meant rapid, challenging role changes and adaptation and we've had some rough patches. Over the past 18 months, we've had a very low attrition rate in Engineering and I think it reflects the positive experience of the vast majority of our team.

I am sorry some people have had a bad experience interacting with us. I would love to hear any complaints or suggestions, my email is george at tophat dot com

</comment><comment><USER16> Disclosure: I work for a large University (combined residential+online presence of ~100k) and we have contracts with Pearson, McGraw, Cengage, and Top Hat.

Top Hat has been surprisingly enjoyable to work with from a contractual side. They've been accommodating to our needs and worked with us to improve their product -- I'm actually working on a SOW with them right now where they'll be adding new features for us at 0 cost.

Even though we have strategic partnerships with these large publishers, I wouldn't mind seeing them go by the wayside in favor of integrating more of Top Hat's digital content.

<comment><USER17> Does it bother you that the majority of student see mandatory subscriptions to all of these services as a giant ripoff?

I mean, you could save every student maybe 600 a year if the university purposely used books that are one edition out of date. That's the same as lowering tuition around 5% at most places. Would this really affect the quality of education in any meaningful way?

"Strategic partnership" sounds like another way of saying the university gets some money out of the deal at the expense of their students pocket books.

How is this justified from the position of someone who maintains relationships with these companies?

<comment><USER16> I'm not involved in the curriculum development side of things, so I don't have much input into that. And prior to the implementation of Top Hat, our students had all been using physical "clickers" to respond to in-class questions.

However, Top Hat implemented a program where the students could trade in their old clicker and get a 5 year Top Hat subscription. Plus a "Lifetime" Top Hat subscription is cheaper than buying a clicker, so students have seen it as a net win.

We do not receive financial compensation from Cengage, Pearson, or McGraw. The strategic part is that they offer more services to us, let us test new stuff they're toying with, and help our profs develop custom content.

I still take classes, so I definitely feel the hit of these textbook prices. It's also why I've advocated for pushing more towards a "direct integration" method of provide course content in our LMS. Students pay ~$60 for access to the content, instead of $100+ for a textbook. It still adds up, but it's a way to start to lower costs.

(and the prices keep going down. I just found out yesterday that one of our vendors is dropping the price of a range of course content because of high adoption rate)

<comment><USER18> The problem with these lifetime subscriptions is that you never know if you're going to actually use them. Textbook prices are definitely a problem, but it's usually much easier to rent or resell textbooks than the equivalent of software.

You also don't buy textbooks until you actually need them. I know many students who have felt pressured into "saving money" on "all 4 years!" plans only to find they don't need a product most terms or they stay and extra semester/term/year and need to spend even more than they were planning.

</comment></comment></comment></comment><comment><USER19> Well if they can get rid of the $200+ textbooks that you have to buy (and never use) to get web access that would be great. Almost all material in school textbooks can be found online for free, it really makes no sense to require students to spend anything of $20 on coursebooks.

<comment><USER20> I agree that access to information online has lowered the price that people are willing to pay for textbooks (digital or physical). But I still think there's a market for a textbook that essentially organizes all the relevant information for a course and vets it for correctness. This is worth a little more than $20 to me, but definitely not $200.

</comment></comment><comment><USER21> It's ambitious and I wish them the best of luck. However, as someone who has worked in the space, they're going to run into a very real problem that the textbook industrial complex maintains its stranglehold because of the very real financial incentives it gives professors etc. Add that in with the fact that content isn't all that expensive to create in the first place, getting students to adopt is expensive, and it's a very noisy marketplace, they're going to have an uphill battle.

</comment><comment><USER22> Having used TopHat in a couple classes circa 2014-2015, the product is pretty bad. It's main use case is, honestly, attendance. Two anecdotes stick out.

The university subsidizes the cost down to $5. Through some technical error, TopHat applied a full discount. One month later, they noticed their mistake and retroactively charged each student $5. I think I'm the only one who complained to their support enough to get it for free; a lawyer's time isn't worth it and I'm sure they spent more than $5 of support personnel time on it. Charging software engineering students like this is a great way to poison the well for recruitment efforts down the road.

The second anecdote is a guest lecturer, who apparently had questions already set up in the system, was completely and utterly unable to figure out the UI. He abandoned TopHat entirely in favour of a show of hands.

<comment><AUTHOR> You're right that there are still too many profs who use Top Hat for really basic stuff like polls in class and taking attendance - thankfully that's in the minority these days.

Our goal is to get profs to at least take these baby steps to get started and then to swap out their $200 textbook with content on our platform, which would save students a ton of money and create a much better experience in the process.

Here are some representative examples of content that's on the platform:

[https://tophat.com/marketplace/openstax/concepts- biology/](https://tophat.com/marketplace/openstax/concepts-biology/)

[https://tophat.com/marketplace/english- composition-i/](https://tophat.com/marketplace/english-composition-i/)

[https://tophat.com/marketplace/publicspeaking/](https://tophat.com/marketplace/publicspeaking/)

[https://tophat.com/marketplace/generalchemistry/](https://tophat.com/marketplace/generalchemistry/)

<comment><USER23> > > [https://tophat.com/marketplace/generalchemistry/](https://tophat.com/marketplace/generalchemistry/)

$60 plus a "subscription," whatever that is.

I can't speak for the quality of the books, but searching "general chemistry" on half.com gives me a bunch of entries in the $5-30 range. They're a few years old, but I can't imagine general chemistry has had many radical changes in the past decade.

These textbooks work with Linux or any other operating system. They can be re- sold, probably for the same price at which they were purchased. There is no learning curve to using a textbook. The textbook's servers will never be down. The university has to pay no licensing fee to the textbook. Textbooks do not have technical glitches. I can read a textbook on the bus, on a plane, in a car, or anywhere, with no worries about running out of batteries or losing my network connection.

Why on Earth would I ever pay more than twice the price for a far worse user experience?

</comment></comment></comment><comment><USER24> I didn't like tophat because it brought computers into the classroom where they don't belong.

But anything that attacks person is honestly probably a good thing, they seem to be actively hostile to students.

</comment><comment><USER25> A marketplace where teachers can sell educational content directly to students, so.... The internet?

But with... an entrance fee? The gorilla which eats all schools will have an entrance fee?

</comment><comment><USER26> <rant> I hate Pearons' My Lab & Mastering products. It was a nightmare using the mastering chemistry and having a similar experience with Physics currently. I felt that $80 for just the software is over priced because they want you pay another $50 for the e-coursebook.

It's not that horrible of a software on itself but for that price, i think it's bad. It takes a lot of effort to put in the symbols and stuff, its very picky about answer format ( 1/2 != 2/4 for example and if its expecting .5 then .5 != 1/2). The professor can add their own content and sometime when my physics professor does it, it's really hard to figure out what the software wants. My calculus class is using "MyOpenMath" which is so much better as a software itself. On top of that it's free. And has a free text book associated with it and my professor decided to use that whole combo, so its great.

I wasn't happy with a software called "TestOut" used for my computer class either but at least it was cheap ($40) compared to Pearson. Maybe I am just poor but price is a big factor for me in these mandatory software the college makes me buy for every other class. I want something that enhances my learning experience, not one that hinders it. Especially when I am wasting good money on education. ^_^ </rant>

<comment><AUTHOR> Our goal is to put an end to that kind of crap, where publishers charge $100-300 for a text and then gouge students another $50 on top of that for a homework system.

Most content on Top Hat is free or around the $20-40 price range, with most of the money going back to the author (vs. publisher model of paying a 5-10% royalty)

<comment><USER17> Do the professors get paid to use tophat? If so I will avoid them at any cost. What a perverse incentive to take money from students and put it directly into the professors wallet

<comment><USER27> nope...

</comment></comment></comment></comment><comment><USER28> I used Top Hat last semester at Iowa State University and the mobile app was excellent. Unfortunately, many of our projectors' are still 1280x1024 which resulted in some of the UI and text of the professor's web-app being cut off.

Still worlds better than anything else I've used and very moderately priced.

</comment><comment><USER29> I work at Top Hat in the Engineering Department but only joined in the past year. I have heard second hand that things were bad in the past (2+ years ago now) in terms of culture and work/life balance.

Things have been great since I've joined and the exec team is VERY interested in what they can do to make things even better.

Here are a few of my personal observations, both good and bad: \- people most often work 10-6 with flexibility and everyone actually uses their vacation and "personal days" \- most devs actually break from work at lunch (supplied) to socialize and play board games \- you get great visibility into how the company is doing and what other departments are up to \- considering we are a company of 200+ people the execs are very accessible and are happy to spend time with you for any questions/concerns you have \- all work is extremely team oriented. They care about where people want to go in their careers and several people have been progressively given more senior responsibilities. \- the majority (not everyone) is very engaged and excited to be here. \- the handful of people who have left in the past couple years all left on good terms and regularly stay in touch and even come out to Engineering events still. There's definitely a sense of community (although I believe this didn't always exist and will be a challenge to keep around as we grow aggressively). \- great location (for me) right on the subway downtown Toronto

Bad things: \- space is cramped. You get a decent sized standing desk but there's not a lot of breathing room other than that. They've outgrown the space though there are plans in the works to fix this. \- there is time allocated for "Engineering" projects but mostly we are very date driven and have an aggressive product delivery schedule. \- diversity. It's a top management priority for the next year but historically they've clearly dropped the ball here. There are several people in Engineering very interested in improving diversity.

I am clearly a biased party, actively working here, but I genuinely feel it's a great place build your career. There are very few places this size that are growing ~70-80% YoY continuously, actually making profit, and have great ambitions for the future.

<comment><USER16> The accessibility to executives is something I've noticed even just working on contracts with Top Hat. I am surprised at the level of engagement that your VPs and C-suite execs have with customers. It certainly makes customers feel cared for.

</comment></comment><comment><USER30> I'm not sure I understand why coursera and edX are not discussed in this space; as someone who works with higher ed institutions, I see these platforms often adopted to replace traditional publishers.

<comment><USER31> OpenStax as well.. All great initiatives.

As for the article, I'd assume it is because it is Bloomberg, and so, these things being more nonprofit/altruistically driven, will never be as 'sexy' as a 'market disrupting' company set up to shake things up and create more 'value'..

</comment></comment><comment><USER32> PDFs are free and all students already know this. The only people who care are the people with a vested interest in selling pointless and useless garbage(textbooks). I didn't pay for my textbooks in school. Nobody should. If it's useful you should copy it and share it with your friends. Software, textbooks, tools, designs, ideas, whatever. Information is free whether you believe it or not. The medium is what you should pay for....but it is not the message.

</comment><comment><USER33> quote from email:

I know you recently saw a demo of Top Hat in action, and I want to personally make sure you are aware of a brand new promotion we're running!

To further enhance your lecture experience, we are now providing iPad Airs to professors using Top Hat with a total of 75 students or more this upcoming Spring or Summer term.

If this sounds like it's up your alley, simply click the button below!

Happy teaching,

</comment><comment><USER34> I previously worked with their VP Engineering, at Janna Systems. A very, very sharp guy who's been through a couple of startups and exits.

I've heard from a different friend that they're very results oriented w.r.t estimates, etc. It means that they get a lot of stuff done, but there's no time for research or non-approved exploration.

<comment><USER35> That makes it sound like a ticket-shop; you're treated like a bricklayer rather than a software engineer?

<comment><USER15> Thanks for the vote of confidence! Like a broadway show, the start of classes is fixed in stone so meeting dates is pretty important to the business.

Good development organizations tailor their methodology to the needs of the business and here at Top Hat the whole company rallies behind the start of classes.

Having a process to meet a date doesn't preclude good engineering. No software engineer wants to be told what to do and how to do it. As a business we are very transparent as to what we want to accomplish and my engineering teams have a lot of leeway on how they want to accomplish it.

</comment></comment></comment><comment><USER36> They need to publish traditional textbooks as well if they are going to compete. Maybe with images and QR codes for the parts where video adds something to the text.

If they don't then they will be Just Another Multimedia Learning Company.

<comment><USER19> It would be neat if they offered some free textbooks and course materials for certain subjects.

<comment><AUTHOR> We totally do offer that! :)

</comment></comment></comment><comment><USER12> Something that annoys me the most about top hat is not the product but the company:

1\. almost every month they enter their info for the HN Who's Hiring thread.

2\. BUT they don't seem to ever respond apart from something automated about receiving your resume (this is not just from personal experience, I've seen people generally comment about the no real response)

3\. They advertise the same position over and over again (mobile developer, backend/frontend/fullstack dev)

and finally to top it all of, I don't think they really care about being called out on this at all.

<comment><USER37> What's not to get. They're just casting a wide net looking for that special unicorn coder that only appears when the planets align. All other mere mortals are not worthy but they will gladly waste your time until then. I guess they forgot the old adage about interviews are 2 way streets.

These guys don't have a very good rep in Ottawa either. It's not really their fault, it's more of a Canadian business attitude thing.

<comment><USER38> > These guys don't have a very good rep in Ottawa either. It's not really > their fault, it's more of a Canadian business attitude thing.

What does " _Canadian business attitude_ " mean?

Are you referring to attitude of Canadians toward businesses or their attitude as a Canadian corporation?

<comment><USER37> The latter. There's a belief that the "labour" should be grateful to the "owner" for a job instead of a looking at it as a straight business exchange of labour for dollars. It runs pretty deep in medium to large businesses and sadly many people tolerate that sort of treatment.

Probably a by-product of our colonial days that we've yet to cast off. Although immigration will solve it soon enough.

<comment><USER12> A lot of Canadians have this attitude too.

A very close friend of mine works for OPG (via a contractor).

OPG and his contractor both have agreed to give him 2 weeks paid time off, and ~ 1 week of sick time off.

However, taking time off ensures that you WILL get laid off (reasons would be made up ofc) during the next quarter.

This basically forces everyone to not take time off (and work during vacations - my friend worked full time during statutory holidays like Christmas and New Year).

When I tried to tell him this was very illegal, he basically said that he was lucky to have his job or he'd be unemployed so he does not mind this.

Now this might be an exception but I've heard and seen numerious stories such as these all over Ontario from various companies (from crappy ones like TCS to startups to small established business like Axiom and even some big multi- national companies - especially in the fintech world and banks).

That said, there are also a lot of companies (like the one I currently work for and the previous company I worked at) that try their best to make sure employees have a great time working there and do their best to achieve that.

<comment><USER38> > OPG and his contractor both have agreed to give him 2 weeks paid time off, > and ~ 1 week of sick time off.

> However, taking time off ensures that you WILL get laid off (reasons would > be made up ofc) during the next quarter.

That's pretty bad though I wonder if it's actually illegal. In the USA most employment is "at will" so companies can fire you for anything or nothing. I'm not sure where taking vacation/sick days counts from a labor law perspective but I wouldn't be surprised if it was within the rights of the company to fire employees who use them.

> This basically forces everyone to not take time off (and work during > vacations - my friend worked full time during statutory holidays like > Christmas and New Year).

You see this a lot in the finance world as well. Nobody takes vacation during the year and piles it on at the end of the year when things are "change freezese". It's pretty silly.

> Now this might be an exception but I've heard and seen numerious stories > such as these all over Ontario from various companies (from crappy ones like > TCS to startups to small established business like Axiom and even some big > multi-national companies - especially in the fintech world and banks).

> That said, there are also a lot of companies (like the one I currently work > for and the previous company I worked at) that try their best to make sure > employees have a great time working there and do their best to achieve that.

So how rampant is this though? If it's prevalent throughout Canada then it sounds like a symptom of a jobs gap. If most employers can be picky about picking employees who will skip vacations (again ignoring how stupid this sounds), then I'd argue there aren't many choices for employees to begin with.

<comment><USER12> > That's pretty bad though I wonder if it's actually illegal. In the USA most > employment is "at will" so companies can fire you for anything or nothing. > I'm not sure where taking vacation/sick days counts from a labor law > perspective but I wouldn't be surprised if it was within the rights of the > company to fire employees who use them.

It might be illegal in Canada (not 100% sure) but a group of ex-employees that have been fired are suing the company.

> You see this a lot in the finance world as well. Nobody takes vacation > during the year and piles it on at the end of the year when things are > "change freezese". It's pretty silly.

This would still be understandable (as you'd expect people to take vacations at year end) - However he was indirectly told to forfeit his vacation completely (which in my opinion is scummy if not illegal).

> So how rampant is this though? If it's prevalent throughout Canada then it > sounds like a symptom of a jobs gap. If most employers can be picky about > picking employees who will skip vacations (again ignoring how stupid this > sounds), then I'd argue there aren't many choices for employees to begin > with.

I've personally seen it happen in Ontario (don't know if it happens in other provinces) but I don't think it's any different in other provinces either. In my opinion, it's a combination of:

1\. fewer jobs

2\. employers willing to sit tight till they get their "right" and submissive employee (because you can see thousands of job openings every single day and take it from me - I've applied to 100s of them but I have a feeling they are just fake - as in, companies post them to show growth/scale etc. but don't really hire - for example, Index Exchange and IBM both have a few positions open for which a friend got interviewed and passed on as he was not a culture fit - 10 months later....still unfilled)

3\. most employees don't mind it in the slightest - or those who do, have already done what's necessary and move into better positions (or I'm hanging out with some some weird folks).

I've seen very few people who are really passionate about their job / stuff they build / improving etc. - for most, it's all about making enough $, and having enough to sustain your daily life PERIOD (and the cycle continues). You are considered crazy if you try to break this or don't like this routine.

</comment></comment></comment></comment></comment></comment></comment><comment><USER39> Thanks to the addition of Top Hat by my statistics professor, I'm now registered on _at least_ six functionally similar/equivalent services this semester:

\- Top Hat

\- GradeScope

\- Canvas (UMD rebranded it to "ELMS")

\- WebAssign

\- Piazza

\- The CS department's internal grading/recordkeeping system

I understand the need for competition, and I think that shaking Pearson and McGraw-Hill from their dominant positions is good.

However, I am measurably _less_ productive in the class that requires me to use three of these than the ones that have little-to-no online interaction - devices in lecture are a distraction, automatically graded homework is a frustratingly fuzzy experience, and paying $50+ to multiple third parties to subsidize the workload heaped on my professor by the school feels exploitative all-around.

</comment><comment><USER40> I think it's great that they are trying to disrupt the textbook industry. However, the industry is so large and the majority of US universities are so entrenched, it's going to be an uphill battle.

There have been many startups that attempted this and all have failed (or the VCs just wanted a payout and were bought for millions when the big publishing companies felt threatened).

It's like trying to disrupt Ticketmaster. You might be able to get a few venues over to your side, but if the artists aren't switching over, you won't get very much traction.

Universities also have no incentive to save money on software. They know that not only is there more of a demand to go to college than the supply of colleges, but that they are guaranteed tuition through the federal loans program.

If we had no federal student loan program, they would be forced to compete on the free market and all of these ridiculous prices for textbooks and software would free fall.

<comment><USER41> I do think the textbook industry is ready for disruption, but it won't happen by someone VC funded that essentially just wants the money going to the incumbents to themselves.

The disruption I'm expecting to happen is quality CC licensed content. Well, depending on the topic that already exists, but what needs to happen I guess is mainstream mindshare.

A _lot_ of the $$$ in the textbook industry is just air ("Hey, basic math hasn't changed in 100 years and we haven't figured out a better way to tell the story, but anyway lets add a few fluff paragraphs and reshuffle all the exercises so students are forced to buy the new edition").

But TBH, at my university the teaching staff largely disliked the textbook racket too, and came up with their own exercises, thus allowing students to use any edition of the chosen textbook.

That being said, I'm just appalled at the poor quality of many of the textbooks from the major publishers. It's like they're selling by the pound, and thus end up with phonebook sized monsters that spend an amazing amount of pages explaining very little.

<comment><USER42> _> The disruption I'm expecting to happen is quality CC licensed content_

Bingo! But there is still money to be made here.

Bundle a high quality open source textbook with tons of extra examples, auto- graded quizzes/assignments when possible, cheater detection, etc.

Provide regular updates so that Google doesn't know the answers to all the homework assignments/quizzes. Scrape the answers that are provided by google and incorporate them into the cheater detection.

Also provide a bit of analytics on top of the quiz/homework infrastructure. Or at least provide robust excel export so that instructors can figure out if there are clusters of students struggling with this or that concept.

And then sell the bundle and give some kick-back to the author(s) of the open source text (because it's the right thing to do, but also because the authors are thought leaders and will plug your product if they feel you're providing a valuable product at a reasonable price without screwing anyone).

Trying to write better textbooks than seasoned and altruistic professors is a losing battle.

<comment><USER43> One important service that the existing textbook companies provide is editing. That's the gap that seems to be unaddressed by having professors self publish.

<comment><USER42> I don't agree.

Editing for content or style?

Style editing is usually detrimental IME. Especially when the style editor doesn't understand the content and is just applying typographical rules.

And content editing is usually done by fellow professors or students. Which if the book is open source will happen anyways as new professors use the book in their courses and notice errors.

<comment><USER18> Publishers provide style, content/grammatical editing, domain experience and pedagogical review.

Most professors don't have strong backgrounds in pedagogy and even those that do benefit from having others edit their work.

</comment></comment></comment></comment></comment></comment><comment><USER44> This story has a bunch of internal hyperlinks, but none pointing at the actual product. To save a Google: [https://tophat.com](https://tophat.com)

An aside: why is news on the web still so bad at this? Presumably because sites are incentivised to not let people leave through an external link? Is there a way to incentivise better linking?

<comment><USER45> Opening links in a new tab with the target="_blank" attribute in your <a> html tag? Keeps the user on the main site in a different tab, and when they close out the new tabs page, they are kicked back into the original site.

<comment><USER46> Warning: Always add rel="noopener noreferrer" if you use target="_blank". Otherwise the target site could redirect the main site to a phishing page or worse.

</comment></comment></comment><comment><USER47> _> In November, they launched an online content marketplace, where professors can create course materials and sell it around the world._

Professors already post a lot of study materials for free on their websites. However, that hasn't replaced traditional textbooks. Why this should be any different? Also given all that free course materials why would someone pay for additional materials?

<comment><USER48> I don't know anything about TopHat.

But I'm a professor and I've offered material for free for twenty years. I can think of some very good reasons to prefer this marketplace-thingy from an author's perspective.

1) Exposure. If you want to sell you want to be on the shelves at the local mart.

(And some authors don't want to support the download traffic.)

2) Platform. If what you are offering is guaranteed to work because you produced it according to this vendor's standards then you don't have to field queries about platforms you don't understand. (I offer a straight PDF of a math text. The only twist is that if you click on the question it bounces you to the answer and if you click on the answer it bounces you to the question. I get queries about that.)

In particular, I'm concerned about standards for interactivity. I don't want to code that stuff, I want to write a text. If a vendor provides the widgets and takes a cut of sales, for instance, I'd read their pitch further.

3) Integration. If it all works with some online gradebook that'd be great.

4) Version control. When version 3 comes out, you'd like version 2 to go away.

<comment><USER47> Thank you for the response. I understand why authors might find this beneficial but do you think it can outweigh the advantage of being free?

<comment><USER48> I offer a text, _Linear Algebra_ , both for Free download and for sale (at Amazon $22). It generates a respectable number of sales. People will pay, or at least some people will pay, for quality. They do get mad, though, at paying hundreds.

I am not a business person but I can imagine a scenario where schools purchase a block of the materials for classes. Say, a prof wants to use a Graph Theory document. Their bookstore pays $13 each for the materials, and a bill for that appears on the student's college charge. Seems possible to me. And beyond that individuals could buy one at a time, of course.

</comment></comment></comment></comment><comment><USER49> Does an open source version of Top Hat exists? I think a lot of educational institutions might be interested in a cheaper and more flexible open source alternatives.</comment>


We'll build *any* webapp in 7 days - <AUTHOR> http://appdone.com/

<comment><USER2> Sounds great. What I'm missing are some references. If those are not available, a nice "here's our team" page would be great, providing more information about the "not-cowboy-coders with 10 years exp". Like "this is steve, steve has worked for the last 3 years at microsoft doing this and that, he's responsible for ...".

<comment><AUTHOR> thanks for the feedback, I'll get on that straight away

</comment></comment><comment><AUTHOR> Also, thanks to whoever sent this in:

        From: firstname.lastname@example.com
        Social Web App, like FB, but better.

<comment><USER3> I find this funny. Prepare to be sued.

</comment></comment><comment><USER4> we tried this some time ago for RoR prototypes(also price was lower about 5k as far as i remember).

it didnt end up well. we had to make few refunds as clients had to much expectations and no clue about software development. we tried to manage their expectations, but thats very hard when you deal with someone who has 0 exp in software development and operates phrases like "i need twitter script so that i can copy it to my server"

most wanted was twitter/digg/facebook clone(or as many call it 'script') + some unique features build for peanuts within 2 days

<comment><AUTHOR> This is a problem with being freelancers/contractors generally, especially when dealing with non technical people. We're hoping we can weed those kind of customers out early at the contact stage.

<comment><USER4> i agree. but this problem especially matters when you start offering complex things such as sw development as a simple package that anyone with plain idea can get.

i seriously wish you good luck with this project though. let us know what was your experience with this promo.

</comment></comment></comment><comment><USER5> similar service (- the visual design part), different technologies: <http://cleancode.it/we-will-build-your-prototype/>

</comment><comment><USER6> Out of curiosity, was this inspired by my service?

<http://iwbyp.chris-granger.com/>

<comment><AUTHOR> No, I've been mulling this over for a few months, yesterday we just decided to build a site and launch. I did see your post though, how are you getting on?

</comment></comment><comment><USER7> In FF4, the "launch fast" part of your title is almost completely hidden. What happened, only had 6 days to make it? ;-)

<comment><AUTHOR> really? that's strange, it looks fine in FF4 here. The site was about 12hrs including copy writing, so there may be a few mistakes.

</comment></comment><comment><USER8> we've been offering "prototype in a week" for years at <http://astrails.com/>

Didn't get much demand for it. Its rare a customer will agree to the minimal enough feature-set for it to be viable.

</comment><comment><USER9> I have some _unlimited_ hosting to go with that for 5$/m.

</comment><comment><USER10> Good luck with the code base on that such apps... :)

<comment><AUTHOR> if you're implying that it'll be a mess, you're wrong :)</comment></comment>


Ask HN: Free Neighborhood Boundary Data? - <AUTHOR>

I'm working on a directory project and would like to allow users to search within various city neighborhoods. I have the lat, lng for the various businesses, but was unable to find any freely available neighborhood data sets to correlate the geo data with neighborhoods. <comment><AUTHOR> Update: Just find this page from Zillow (<http://www.zillow.com/howto/api/neighborhood-boundaries.htm>). It's in the shape file format.

</comment><comment><USER2> These are difficult to find. In the past, I have found these in individual city GIS department websites, but haven't found a single source that aggregates lots of cities. Quality and formats tend to vary as well.

<comment><AUTHOR> That's what I'm finding as well. I searched through the Census site and found PUMA references and was able to find some neighborhood data and various local university library pages, but no single source with everything -- besides expensive commercial data sets.

</comment></comment><comment><USER3> Have you checked out SimpleGeo? I believe their places have Neighborhood info available but not sure if you can filter on it.

<comment><AUTHOR> Looks interesting. Their context database seems to have the neighborhood data. Thanks.

</comment></comment><comment><USER4> I've never used it, but I know Yelp has one ([http://www.yelp.com/developers/documentation/neighborhood_ap...](http://www.yelp.com/developers/documentation/neighborhood_api)).

</comment><comment><USER5> Ordnance Survey OpenData is rather comprehensive. The licence (AFAIK, IANAL) allows free commercial and non-commercial use, as long as you attribute the data to them.

<http://data.ordnancesurvey.co.uk/.html>

It has boundaries for Counties, Cities, Electoral Constituencies and Wards, Civic Parishes, and probably other divisions, too.</comment>


ColorSpace – Color Palettes Generator and Color Gradient Tool - <AUTHOR> https://mycolor.space/ <comment><USER2> I use uigradients. I will try this too.</comment>


Time travelling RTS (Achron) accepting Pre-orders - <AUTHOR> http://www.achrongame.com/index.html

<comment><USER2> Interesting... a nice stab but I think there is a critical aspect missing.

t = 0 No one is attacking me

t = 1 I attack someone

t = 2 They wish they attacked me

t = 3 They jump back to t = 0 to attack me

t = 4 (0') They attack me, but there is no "me" to defend. The troops will just do their default actions.

So you have a lot of play against AI.

<comment><AUTHOR> When they move to T0 and attack you, the changes caused by this attack move forward to the present in a "time wave" - you see these changes and decide to intervene by moving to T=1 and ordering your troops to intercept their troops.

The effects of this also move forward in a time wave, which they can see approaching on a little summary at the bottom of the screen.

As you modify the past you use up "chrono-energy" limiting the amount of influence you can have.

See [http://chronofrag.com/viewtopic.php?f=2&t=36](http://chronofrag.com/viewtopic.php?f=2&t=36) for some more time travel strategies.

See [http://video.google.com/videosearch?hl=en&q=achron+youtu...](http://video.google.com/videosearch?hl=en&q=achron+youtube&um=1&ie=UTF-8&ei=v-xES8-mHs- LkAXBhMmxDw&sa=X&oi=video_result_group&ct=title&resnum=6&ved=0CB0QqwQwBQ#) for some videos of achron.

<comment><USER2> Yeah, I understand. But the problem is that if t = 0 and t = 1 are far apart, then by the time I jump back to t = 1' and defend, I could have lost a bunch of troops (for example).

I'm not saying it doesn't work, but I'm trying to think of this from a more "realistic" (within the scope of a time travel video game) point of view. In theory, at t = 0' (before I've jumped back) you'd want my troops controlled by someone with my skill, but not with my future knowledge.

Lets say I'm a great strategist, so good in fact that even when I'm attacked at t = 0' I would be good enough to fed off the attack. But, if I was asleep till t = 1' (as is essentially the case in what you are saying) I will get decimated. Jumping back to t = 1' doesn't help me then.

Obviously I have to accept limitations based on the fact that the players are in fact temporal beings, so I'm not saying the game mechanic is bad. I'm just stating a limitation.

<comment><AUTHOR> You need to watch a couple of their time travel videos. The time travel is flexible and the time steps are tiny. Because you can move forwards, backwards, pause and fast forward time you can try several strategies until you find the best one for a given fight.

Your objection about spending a lot of time fighting AI is a valid one, it was my main objection too. It seems that it is so easy for the actual player to go back and tweak the battle if they didn't like the results that the AI complaint is moot.

Also, you can provide pretty rich standing orders and hierarchies for your troops. If anything, you will fight AI less than in a typical RTS - since a normal (not superhuman) opponent cannot manage all their troops at once, whereas in this game any opponent can slow stuff down until they can manage everything.</comment></comment></comment></comment>


Google Adds "People Related To" Google+ Results - <AUTHOR> http://www.seroundtable.com/google-people-related-to-15491.html <comment><AUTHOR> ... and if you do a search for Ruby, _why shows up at the top of the list.</comment>


Professional Transitions - <AUTHOR> http://www.ianbicking.org/blog/2014/09/professional-transitions.html <comment><USER2> Why isn't the title On Shutting down Mozilla Labs.

<comment><USER3> Yeah, I thought the article was really interesting, and maybe it would have been frontpaged if it had a good title.

<comment><USER4> This is why editors give titles to articles instead of letting the writers do it ;)</comment></comment></comment>


Amelia Earhart Plane Fragment Identified? - <AUTHOR> http://news.discovery.com/history/us-history/aluminum-fragment-appears-to-belong-to-earharts-plane-141028.htm

<comment><USER2> A lot of people are skeptical of TIGHAR's Nikumaroro hypothesis and of TIGHAR itself. I used to be in the skeptic camp myself, until one day I decided I might ought to do some investigation rather than just dismissing the idea out- of-hand. It turns out that TIGHAR has a lot of information on their website, including but not limited to artifacts they've recovered on expeditions to Nikumaroro.

It's the artifacts that get the most press, but in my layperson's opinion it's not the artifacts recovered that constitute the best evidence for the theory but rather archival evidence that's been discovered over the last 20 or so years.

Here are a couple of links that I personally found most compelling: [http://tighar.org/Projects/Earhart/Archives/Research/Bulleti...](http://tighar.org/Projects/Earhart/Archives/Research/Bulletins/52_NumbersGame/52_NumbersGame.html). I posted this link to both Hacker News and r/history a few months ago under the title "What happened to Amelia Earhart: The sextant box with two serial numbers".

The second link is: [http://tighar.org/Projects/Earhart/Archives/Documents/Bones_...](http://tighar.org/Projects/Earhart/Archives/Documents/Bones_Chronology.html). This page contains radio telegram transcripts from the British colonial authorities responsible for administering Nikumaroro in the early 1940s. The part about the sextant and two serial numbers is about half way down the first page.

<comment><USER3> So, I had a quick look at your links, and found nothing new there. The artifacts point to human presence (hardly a surprise), but there is little more than conjecture to link them to Earhart or Noonan. In fact, the telegraph writers clearly thought an Earhart link was unlikely.

<comment><USER2> The artifact is suggestive rather than conclusive. However, I'd argue that the "human presence" part actually was a surprise. If that really was a Brandis sextant box that had been surplused from the U.S. Navy, it indicates a human presence on the island during a relatively narrow time period, roughly 1919-1937. During that time period the only documented human presence on the island was the survivors of the wreck of the SS Norwich City and their rescuers in 1929, and accounts of that event indicate that they were never on the part of the island where the sextant box was found. The sextant box was also found near a number of large bones which were clearly identifiable as human. Where did that person come from? Were they shipwrecked on the island sometime between 1919 and 1937, but in a different event than the Norwich City wreck? What ship went missing in that area of the Pacific during that time period?

So anyway, this is just some circumstantial evidence. But it's not the only evidence that TIGHAR has. If you start combing through their website, you'll find they have a bunch of independent circumstantial evidence, none of which is conclusive by itself, but taken together starts to look suspicious.

</comment></comment></comment><comment><USER3> I am highly skeptical of any analysis performed by TIGHAR. They are way too biased in their interpretation of their "finds." I would like to see a detailed report on how they determined that piece of metal is the one they say it is. If and only if that report draws reasonable conclusions, I would then be interested in what a third party analysis had to say about it. That is my minimum bar for even considering TIGHAR's claims.

<comment><USER4> Indeed, we seem to see announcements of TIGHAR "finds" about once a year, with each one being pretty vague and not leading specifically to the discovery of Earhart's actual aircraft.

If one were cynical, one could conclude that the annual "finds" are just exercises to keep TIGHAR's name in the news.

<comment><USER5> Actually to be cynical you would have to see them as funding teasers to elicit funding for yet another trip out to the atoll.

That said, if they dive down and find the plane they are going to be pretty pleased with themselves. As far as I can tell there aren't any facts that would rule out their theory that Earhart ended up here.

</comment></comment></comment><comment><USER6> Here's TIGHAR's map of possible Earhart evidence on Nikumaroro: [http://tighar.org/Projects/Earhart/Archives/MapsandPhotos/ma...](http://tighar.org/Projects/Earhart/Archives/MapsandPhotos/maps/Nikumap.html)

Note: their copyright disclaimer in the footer technically disallows linking to or visiting their site, so click at your own risk! ;)

</comment><comment><USER7> It sounds fishy. See:

[http://en.wikipedia.org/wiki/Phoenix_Islands_Settlement_Sche...](http://en.wikipedia.org/wiki/Phoenix_Islands_Settlement_Scheme)

Earhart disappeared in 1937, since 1938 there were people on Nikumaroro and it seems they didn't report anything that would certainly be easily visible one year later on a 5 x 1.6 miles big island if Earhart and Noonan managed to land and live there.

See also:

[http://en.wikipedia.org/wiki/TIGHAR](http://en.wikipedia.org/wiki/TIGHAR)

<comment><USER2> > Earhart disappeared in 1937, since 1938 there were people on Nikumaroro and > it seems they didn't report anything that would certainly be easily visible > one year later on a 5 x 1.6 miles big island if Earhart and Noonan managed > to land and live there.

So here's the thing -- they did find evidence and they did report it. See my other comment (the one starting with "A lot of people are skeptical of TIGHAR's Nikumaroro hypothesis...")

<comment><USER7> No, I'm not interested in TIGHAR's "evidence" that they discovered 20 years after people for 20 years attempted to live there. Of course there will be some traces of that people. I'm interested why anybody who was on the island one year (1938) after supposed landing/crash (1937) haven't reported anything. The island is so small (see Wikipedia) that everything would be discovered in just a single day.

</comment></comment></comment><comment><USER8> It's kind of funny, looking back at that age, how unreliable machinery was, how famous aviators disappeared all the time. Antoine de Saint-Exupery, who wrote _Terre des Hommes_ and _The Little Prince_ , was another who went away.

<comment><USER9> If we can lose an entire 777 today, losing a small aircraft seems really reasonable back then, I don't think things were inherently more unreliable back then. They had some pretty advanced technology at their disposal as well, they used radio signals to determine their headings, probably the biggest disadvantage they had was poor weather predictions and limited search/rescue capabilities.

<comment><USER10> "If we can lose an entire 777 today..."

This starts to look more and more embarrassing in retrospect.

<comment><USER11> Not really. Sonar scans on vast, deep seabeds are hard. Also consider how far a crashed plane can drift on descent, how thoroughly it could be disintegrated with sea impact, quickly it can be covered up by sand, et cetera... :)

<comment><USER10> I'm not talking about after it has reached the sea bed.

I'm aware of some of the issues regarding continuous RF transmission of basic telemetry while inflight. The bandwidth for all relevant flight info is higher than you'd think, and the receivers over open ocean are sparse. Et cetera.

But still: in hindsight, relative to the high standards we expect from flight systems, the inability to localize a 777 is looking more and more, for lack of a nicer term: stupid.

I'm reminded of the crash of an early Mars lander (Mars Polar Lander, 1999), which didn't have telemetry during EDL, so the cause of the crash could not be fully determined. They pretty much determined after that, that every Mars landing would have telemetry, so we'd at least know what went wrong.</comment></comment></comment></comment></comment>


Facebook's spring animation library for Android UIs - <AUTHOR> http://facebook.github.io/rebound/

<comment><USER2> Very disappointing that this isn't available via Maven. Is Facebook expecting wide adoption of their libraries even not supporting the standard tooling?

I suppose someone could fork the project and put it together in a more useful way.

<comment><USER3> FYI: I just merged a pull request that adds gradle support. Would love for somebody to send a pull request to add maven as well.

<comment><USER2> Very cool! Though I wasn't very clear I actually meant I'd like to see this made available via Maven Central as an aar, so that it could more easily be incorporated into Gradle based applications - though having a build.gradle makes it fairly trivial to pull the project and manage that oneself, especially now that Guava is no longer a dependency as well :)

Really awesome to see Facebook producing stuff like this.</comment></comment></comment>


How fast can a BufferedReader read lines in Java? - <AUTHOR> https://lemire.me/blog/2019/07/26/how-fast-can-a-bufferedreader-read-lines-in-java/ <comment><USER2> The first issue I can see with that code is it's not doing what he expects. He does this to read the file into a StringBuffer:

      bf.lines().forEach(s -> sb.append(s));

However, this ends up reading all the lines into one giant line, since the String's that lines() produces have the newline character stripped. This leads to the second lines() call to read a 23MB line (the file produced by gen.py). This is less than optimal.

The fastest version I managed to write was:

        public void readString5(String data) throws IOException {
             int lastIdx = 0;
             for (int idx = data.indexOf('\n'); idx > -1; idx = data.indexOf('\n', lastIdx))
             {
                parseLine(data.substring(lastIdx, idx));
                lastIdx = idx+1;
             }
             parseLine(data.substring(lastIdx));
        }

Not the prettiest thing, but it went from 0.594 GB/s to 1.047 GB/s. Also, it doesn't quite do the same as the lines() method, but that's easily changed.

<comment><USER3> That's a pretty big error if you're correct. What does it say about the language when a CS professor falls for this on a 40 line file?

<comment><USER4> Getting benchmarks right is difficult, even for a CS professor. The language doesn't even enter there.

What does it say about you that you're asking a leading question in this way?

<comment><USER3> I'm always interested in thoughts about language design and programming in general. We have fifty+ years of commercial software development and it still sucks. This caught my eye as the kind of error that shouldn't even be allowed to happen.

<comment><USER4> There are several simple data transformation steps. There's nothing inherently wrong with these steps. The compiler can't read your mind and guess what you wanted to do.

How would this be possible to be prevented with any programming language, existing or hypothetical?

Even if the meta information that the appended string doesn't contain any newlines would be passed on, the second call to lines() would rather use is as an optimization hint instead of raising an error.

<comment><USER3> To give an example: if the lines() method returned instances of a Line class, the string buffer and it's append() method would be aware and still create a collection of lines instead of one long string. Not necessarily nice and would probably create issues in many other interfaces, but problems like this _can_ be solved with careful language and API design. Note that his end goal was to 'read lines' and not use any particular string abstractions. Nothing to get worked up about here.

<comment><USER5> That would be a very clunky approach to put it mildly - the explicit design intent of BufferedReader is to let you read lines without caring what the line separator is - it's a lossy transformation. You also can't subclass String. You'd be adding something very unwieldy for an edge case that is directly contrary to the purpose of the API. The 'mistake' (such as it is) of the author here is simply using the wrong API - there are shorter, more direct ways to read a file into a string without any of these contortions.

        data = new String(Files.readAllBytes(Paths.get(filename)));

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> I don't know what Java's BufferedReader is doing, but it's probably not the optimal thing in terms of IO throughput. I would blame the algorithm long before blaming anything inherent about the JVM.

Erlang is another language where "naive" IO is kind of slow. [https://github.com/bbense/beatwc/](https://github.com/bbense/beatwc/) is a project someone did to test various methods of doing IO in Erlang/Elixir, and their performance for a line-counting task, relative to the Unix wc(1) command.

It's interesting to see which approaches are faster. Yes, parallelism gains you a bit, but a much larger win comes from avoiding the stutter-stop effect of cutting the read buffer off whenever you hit a newline. Instead, the read buffer should be the same size as your IO source's optimal read-chunk size (a disk block; a TCP huge packet), and you should grab a whole buffer-ful of lines at a time, do a _pattern-matching binary scan_ to collect all the indices of the newlines, and then use those indices to part the buffer out as _slice references_.

This achieves quite a dramatic speedup, since most of the time you don't need movable copies of the lines, and can copy the line (or more likely just part of it) yourself when you need to hold onto it.

This approach is _probably_ also already built in to Java's "better" IO libraries, like NIO.

<comment><USER7> Java NIO does exactly what you describe. IO is chunked optimally, the entire buffer is pulled, the API exposes a select() mechanism just like any libc, and the user is expected to frame its own received data.

edit: Lemire is showing a lack of what Fowler describes as "mechanical sympathy".

[https://martinfowler.com/articles/lmax.html#QueuesAndTheirLa...](https://martinfowler.com/articles/lmax.html#QueuesAndTheirLackOfMechanicalSympathy)

<comment><USER8> A bit of nitpicking on that: Even Javas nio API isn't very efficient. select() introduces a ton of garbage due to dynamically allocating lists for selected keys, and keys have a lot of synchronization overhead. That's why frameworks like Netty gained a lot of performance by building their own selectors using JNI.

</comment></comment></comment><comment><USER9> Honestly, it seems that nearly everyone here is missing his point.

Some of the blame for that probably lies with his headline choice, but he clearly states at the end of this post:

""" This is not the best that Java can do: Java can ingest data much faster. However, my results suggest that on modern systems, Java file parsing might be frequently processor-bound, as opposed to system bound. That is, you can buy much better disks and network cards, and your system won't go any faster. Unless, of course, you have really good Java engineers.

Many firms probably just throw more hardware at the problem. """

It's not _about_ this piece of code. It's not even about Java In the previous post he mentions at the start of this one, he pointed out:

""" These results suggest that reading a text file in C++ could be CPU bound in that sense that buying an even faster disk would not speed up your single- threaded throughput. """

So, I take his point to be that one shouldn't make assumptions about performance. Rough performance scales -- such as have been posted here many times (e.g. [1]) -- make great rules of thumb for implementation choices or as a guide for where to look first for bottlenecks. To optimize in the __real world __, though, you 're best served using __real measurements __.

[1] [https://www.prowesscorp.com/computer-latency-at-a-human- scal...](https://www.prowesscorp.com/computer-latency-at-a-human-scale/)

</comment><comment><USER10> The post does nothing to explain how and why, it just throws a couple of outputs from a non specified machine and does no comparison.

It has no baseline and no specs. For all I know, he could have got his 0.5 GB/sec on ab old Pentium II processor.

There is no analysis.

I am perplexed.

<comment><USER11> Lemire is one of the leading experts on string matching and the author of several core libraries you probably use every day.

 _edit_ fine, so instead maybe click on the links in the post to see that this article is just one of a series. He's probably tired of copy-pasting the specs of his reference hardware (Skylake [https://arxiv.org/pdf/1902.08318.pdf](https://arxiv.org/pdf/1902.08318.pdf)) since all he's concerned about is the relative performance of different software.

There is a difference between "I'm being dumb because I don't know what I'm doing" and "I'm being lazy because I've done it 1,000 times and the target audience knows what I mean".

<comment><USER12> Then they should know enough to give at least some theories that can explain the difference and tell us something more about the test setup.

</comment></comment></comment><comment><USER13> I'm amazed at how upset some commenters are about a blog post that did a toy experiment and didn't actually make any strong claims.

I'm actually a stickler about good benchmarks - it riles me when people draw sweeping conclusions from poorly-designed experiments. Lemire is actually one of the good ones. If you want something more fully developed than a blog post, read one of his papers.

I personally really enjoy his blog because of this - he's good at picking interesting exploratory experiments that provide some insight, without trying to over-generalize from the results. If you read his conclusion, the point is that there is a good probability that even relatively simple programs are CPU- bound. His experiment supports that point. My experience also matches that - I've seen a lot of data processing code that could be I/O bound in theory (i.e. a perfect implementation could max out CPU or network) but is CPU bound in practice. Usually because of string manipulation, regexes, or any number of other things.

> This is not the best that Java can do: Java can ingest data much faster. > However, my results suggest that on modern systems, Java file parsing might > be frequently processor-bound, as opposed to system bound. That is, you can > buy much better disks and network cards, and your system won't go any > faster. Unless, of course, you have really good Java engineers.

<comment><USER14> I can make a lot of things cpu bound with crappy implementation. I'm not going to write a blog post about any of them.

</comment></comment><comment><USER7> This is absurd, the original platform libraries do not account for the fastest use-cases in any specialized IO case.

Java NIO channel should have been used for this. It was demonstrated back in the early 2000s with the "Grand Canyon" demo achieving very good throughput for its time, and it's still the gold standard.

</comment><comment><USER12> So what's the reason for this? Is it maybe because of some unicode shenanigans? Java characters are 16bit iirc, and strings have some forty bytes of constant overhead.

<comment><USER15> I'm no Java ninja, but a few things jump out of [https://github.com/AdoptOpenJDK/openjdk- jdk11/blob/19fb8f93c...](https://github.com/AdoptOpenJDK/openjdk- jdk11/blob/19fb8f93c59dfd791f62d41f332db9e306bc1422/src/java.base/share/classes/java/io/BufferedReader.java#L314) :

\- at least one heap allocation for every line. After it finds the EOL it first uses 'new String' followed by '.toString()

\- the C++ version will almost certainly be backing on to memchr() behind the scenes, which will be using SIMD instructions where it makes sense (e.g. large enough scan size, probably true in this case). the Java version is a manual bytewise-coded loop.

\- the C++ version is reusing its output buffer, no reallocations assuming the same string length or less

No idea about encodings in Java, maybe that is playing a role too

<comment><USER16> I haven't looked recently but several years ago I was shocked to discover the GNU libstdc++ didn't use strchr or memchr. It used a hand-coded for loop because it was a template for various kinds of character. There was no specialization for 8-bit char, either.

As a result std::string was disgustingly slow compared to C code.

</comment></comment></comment><comment><USER17> Java has many inefficient parts. For example there's no immutable array concept (or owning concept, like in Rust), so there's a lot of unnecessary array copies happens in JDK. String is not well designed. There was an attempt to abstract String concept into CharSequence, but a lot of code still uses Strings.

I made a similar benchmark. The idea is as follows: we have 2 GB byte array (because arrays in Java have 32 bit limit, LoL) filled with 32..126 values, imitating ASCII text and 13 values imitating newlines.

The first test is simply does XOR the whole array. It's the ideal result which should correspond to memory bandwidth.

The second test wraps this array into ByteArrayInputSteram, converts it into Reader using InputStreamReader with UTF-8 encoding, reads lines using BufferedReader and in the end also XORs every char value.

For 2 GB I have 516 ms as an ideal time (3,8 GB/s which is still almost order of magnitude less than theoretical 19.2 GB/s DDR4 speed) and 3566 ms as a BufferedReader, so you can have almost 7x speed improvement with better implementation.

Benchmark: [https://pastebin.com/xMD4W8mn](https://pastebin.com/xMD4W8mn)

</comment><comment><USER18> Eh, he didn't use NIO. BufferedReader is an ancient Java relic. Like reading from STDIN in c, it's not made to be fast, it's there for convenience and backwards compatibility.

Read a file using something like Vert.X, which is optimized for speed. I'm 100% confident it will be faster than the naive c approach

<comment><USER19> Do you have example of alternatives for the BufferedReader with the NIO APIs?

I do a lot of work with large GZIP that are read line by line using the standard IO (i.e. GzipInputStrem(FileInputStream)) etc) but your comment has really made me second guess my choice of doing that...

<comment><USER18> I would check out the compression handlers in Netty, which underlies Vert.X and many other projects that need high IO performance.

You should be able to hack something together that feeds zero copy buffers into Netty compression handler. Maybe using Netty or Vert.X file API, or maybe just raw NIO2.

I'm not sure how fast this would be, but my gut says "very". Netty can easily saturate 40 Gigabit Ethernet lines, and file IO should have less overhead. That's ~5 gigabytes a second.

It's going to be a good bit of coding for sure. Vert.X/Netty/NIO2 are all async and pretty low level. They're generally 1 thread per core, and along with SSD read patterns you're probably best off reading files in parallel, one per core. Might not be worth the effort.

You may want to look into ZStandard as well. It's Superior to Gzip in most ways when you need something fast but decent.

</comment></comment></comment><comment><USER20> One thing that jumps out at me is the test code writes to "volume" variable in the read loop, I assume for counting the number of bytes in the file, but never reads it back. A clever compiler will optimize away those writes, the string length check, the loop over the lines, and actually reading the file.

I'm not saying that's happening here, but it's a basic fact when writing benchmarks that you have to actually test something real and not a transient property of the program _after_ the compiler has had its chance to be really smart.

</comment><comment><USER21> I am a bit confused. The scanFile reads from a file. But the readLines inside the for loop is reading from a StringReader - and not from a file?

</comment><comment><USER22> At least two problems with the java code. Concatenation of strings using the plus operator creates a new string and copies the content of the old, that pushes the complexity of the code from o(n) to o(n2) where n is the number of lines. Secondly order is not guaranteed with the for each operation on streams.

The correct way to do it is using collect(Collectors.joining("\n")) or straight forward imperative style (without streams).

I don't think the general statement holds (that java or buffered reader is cpu bound in particular).

<comment><USER5> Where do you see string concatenation with the plus operator in the posted code?

<comment><USER22> Haha - nowhere ... I completely misread his code.

</comment></comment></comment><comment><USER23> But all the stream API isso terribile for performance..you write a one line code and you are already at O(n^5)

</comment><comment><USER24> Completely wrong.

It is like asserting something about C based on GCC specific behaviour.

Java is not a single language implementation.

<comment><USER25> Agreed. I think you are being downvoted because you forgot to paste in your benchmark results.

<comment><USER24> Yeah, I forgot that we aren't allowed to have opinions without doing the hard work.

So now I have to go out and do benchmarks across all these implementations just to prove they aren't the same?!?

[https://en.m.wikipedia.org/wiki/List_of_Java_virtual_machine...](https://en.m.wikipedia.org/wiki/List_of_Java_virtual_machines)

</comment></comment></comment><comment><USER26> Java is... java.

I was once working on an Android app on a cheap custom board with 128 M ram (don't ask why Android on a single function custom board, wasn't my decision).

Among other things, I had to parse a 80000 line csv file. Splitting and the rest of the processing created so many temporary strings the system ran out of ram. We eventually gave up.

<comment><USER27> Myself I've worked with gigabyte sized CSV files without issues, so it was likely your implementation rather than the fault of the Java language.

<comment><USER12> Did you do it in 128 megs of RAM?

<comment><USER28> Depending on what needs to be done with the CSV files, it's very possible to do it in 128MB of RAM. For example, if we need to read the rows, transform them a bit, and then write to another file, we can read up to N rows, transform them, and then write them. That should result in bounded memory consumption because only up to N rows need to be kept. Similar strategies are possible if the rows are used as input to an ETL job, calling a Web service with the results of parsing the file, etc.

Editing a file gets trickier, though it's not impossible. Maybe using a [piece table]([https://en.wikipedia.org/wiki/Piece_table](https://en.wikipedia.org/wiki/Piece_table)) plus some smart buffering the file can keep memory consumption below some constant, letting it function for large files, but with the downside of lower performance for files larger than whatever the constant is?</comment></comment></comment></comment>


MySQL caching methods and tips - <AUTHOR> http://itechtalks.blogspot.com/2011/09/mysql-caching-methods-and-tips.html

<comment><USER2> Blogspam, this is just copy-and-paste from another source.</comment>


Ask HN: Do you use default top color for HN? - <AUTHOR> Interested to see if people go with default top color or choose to use a custom one (from profile page).<p>I use #ebebeb. <comment><USER2> You can see the most popular HN bar colors here: [https://news.ycombinator.com/topcolors](https://news.ycombinator.com/topcolors)

I personally like #f6f6ef.

</comment><comment><USER3> I didn't realize that feature exists until I read your post. Neat!

<comment><USER4> I'm with you two! I like #FF80DD

</comment></comment><comment><USER5> Apparently you need to have over 251 karma to use the feature

<comment><USER6> Was about to try and see what #2789c2 would look like, only to not see the setting and get frustrated. Thanks for clarifying.

Time to good looking for karma.

</comment></comment><comment><USER7> I changed my color as soon as I got enough points to do so. Just for the sake of the game. I'm now at 438 and looking forward to the next threshold which I believe is 500 (ability to downvote).

It's nice to have a game element. It makes it exciting when I open up HN. I always glance to the top bar to see what's happening with my score.

</comment><comment><USER8> I use F6F6EF to make it "invisible".

Though currently with: [https://userstyles.org/styles/71445/better-hacker- news-desig...](https://userstyles.org/styles/71445/better-hacker-news-design) makeing it visible again :) but I like the colors now

</comment><comment><USER9> Yes, I do.

I don't know what other colors that match with my eyes. But, after reading this thread (thank you for asking this to HN users), I am going to try different colors on HN top bar. See... what will happen...

It's nice that people brought their color preferences here.

</comment><comment><USER10> Nop, I have it white. Looks minimalistic.

</comment><comment><USER11> Hotpink: ff6990

</comment><comment><USER12> Default color

</comment><comment><USER13> #ff6699</comment>


Topology Illustrated (2015) - <AUTHOR> https://calculus123.com/wiki/Topology_Illustrated <comment><USER2> What are the prerequisites for learning the basics, or developing a rudimentary understanding of this subject? I find the idea of it (as far as I can understand what that even might be) fascinating, but also exotic and forbidding. I had a couple of friends who reconsidered their major after taking topology :(

<comment><USER3> There aren't really any mathematical prerequisites, except for a basic understanding of writing proofs. Most of topology is about defining a set of axioms (specifically, a topology is a set X together with a collection T of subsets such that...) and studying the consequences of those axioms. Studying topology is a great way to build up mathematical maturity.

However, I'd say it's very difficult to see the "point" of topology without taking a proof-based real analysis class (where you use epsilons and deltas to prove things about sequences, limits, derivatives, integrals, etc.). The concepts in topology generalize everything that is typically covered in a real analysis class, and many of the standard examples rely on some familiarity.

(in a similar way, it's very hard to understand the "point" of category theory until you've noticed that many proofs across group theory, linear algebra, real analysis, etc. are all suspiciously similar)

As for how to study: Topology is great because you really can build it all from the ground up. Find a book with good exercises, and _do all the exercises_! That way, you'll be forced to learn for yourself how all the pieces fit together. Unfortunately I don't have a book to recommend since I learned from a set of unpublished lecture notes.

EDIT: I dug up the old lecture notes, here [1] you go! Authored by Harrison Bray (no relation to me afaik) for University of Michigan, MATH 490, Fall 2016.

[1] [https://github.com/benrbray/benrbray.github.io- source/blob/m...](https://github.com/benrbray/benrbray.github.io- source/blob/master/content/static/files/umich_math490_f16_sbray.pdf) [2] [http://www-personal.umich.edu/~hbray/](http://www-personal.umich.edu/~hbray/)

<comment><USER4> > in a similar way, it's very hard to understand the "point" of category > theory until you've noticed that many proofs across group theory, linear > algebra, real analysis, etc. are all suspiciously similar)

Would you be able to give an example of such a proof cluster?

<comment><USER5> One example I'm aware of is Lawvere's fixed point theorem, of which Gödel's (first) incompleteness theorem, the undecidability of the Halting problem, and Cantor's theorem are all special cases.

Not really related to "group theory, linear algebra, real analysis, etc.", but interesting nevertheless.

It's quite a wide generalisation which really just captures the nature of diagonalisaion arguments, but it does formally tie together various proofs/theorems which "smell the same".

</comment></comment></comment></comment><comment><USER6> I've been looking for a good calculus book. I was curious if this worked well, and I found:

1) It costs $300 for a paper copy 2) There are no previous. I have no idea if it's actually illustrated

I'm looking for something which is intuitive, and illustrated. A lot of pictures, good example, and perhaps less math and text (in this case, assuming the educator will be talking through most of this, and the learner doing things together).

<comment><USER7> Calculus Made Easy is exactly what you are looking for. [http://calculusmadeeasy.org/](http://calculusmadeeasy.org/)

<comment><USER6> Thank you!</comment></comment></comment>


Trucking Deregulation (1993) - <AUTHOR> http://www.econlib.org/library/Enc1/TruckingDeregulation.html <comment><USER2> What a wildly anti-union article.

"Deregulation has worked well. ... Before deregulation ICC-regulated truckers paid unionized workers about 50 percent more than comparable workers in other industries."

I understand that cheaper trucking is helpful for some industries, but cutting workers' income is definitely a negative, not a positive.

<comment><USER3> > I understand that cheaper trucking is helpful for some industries, but > cutting workers' income is definitely a negative, not a positive.

It's negative for them, and positive for everyone who buys something shipped by truck — which is to say, everyone.

The study also notes that truckers are _still_ paid more than comparable workers in other industries.

<comment><USER4> First, that assumes the savings mostly go to customers rather than into the pockets of executives and shareholders. Trends over the last few decades suggest otherwise.

Second, it's a negative for truckers and their families but also for everyone who works in any industry that supplies anything truckers buy, directly or indirectly - which is to say, nearly everyone.

<comment><USER3> > First, that assumes the savings mostly go to customers rather than into the > pockets of executives and shareholders. Trends over the last few decades > suggest otherwise.

The price of shipping over the last several decades has fallen, as indicated in that article. Those savings _have_ gone to shippers, not to trucking executives. Meanwhile, given competitive markets, customers of shippers have realised those same savings.

> Second, it's a negative for truckers and their families but also for > everyone who works in any industry that supplies anything truckers buy, > directly or indirectly

Dubious. Reduced friction tends to make everyone better off.

<comment><USER4> And yet, in developed countries, decade after decade GDP on paper keeps going up and at the same time in real terms nearly everyone keeps getting poorer, so something has gone wrong with the assumptions on which this plausible-sounding economic model is based.

</comment></comment></comment></comment></comment><comment><USER5> How old is this? It's certainly not current, as the ICC has been abolished and mostly replaced with the Surface Transportation Board in the Department of Transportation.

<comment><USER6> Pretty old.

Thomas Gale Moore, "Trucking Deregulation." The Concise Encyclopedia of Economics. 1993. Library of Economics and Liberty. 15 December 2015. <[http://www.econlib.org/library/Enc1/TruckingDeregulation.htm...](http://www.econlib.org/library/Enc1/TruckingDeregulation.html>).

<comment><USER7> Ok, we added 1993 as the year above. Thanks!

</comment></comment></comment><comment><USER8> Nixon is frequently given credit for several "left wing" initiatives like the EPA; Carter doesn't get nearly enough for his aggressive deregulation.</comment>