How we made our home page load faster - <AUTHOR> http://blog.deskaway.com/how-we-made-our-home-page-load-faster

<comment><USER2> Few of my own tips, most of which you covered.

\- Load everything asyncronously. RequireJS etc... (if you don't use AMD loading, then seriously start reading up about it).

\- The overhead of including Facebook Like button and Twitter, is like 150kb. It's ridiculous. We load them after the page has loaded. In their place, we load placeholder Facebook buttons. That mimic the like button. (Similar to what TechCrunch does).

\- Bundle everything. We use shrinksafe for this. (Sometimes Closure... it really depends on which one is playing nicer at the time of doing the build). This allows us to create _enormous_ JS files of all of our application. What we do is create one "kernel.js" file. This is always loaded first. And once the page has loaded we fire off using AMD loading asyncronously to the actual application.js. I've played around testing on this as to whether it's best to only include the javascript that is relevant to the current page, but in the end we settled on loading absolutely everything. This makes the JS about 280kb gzipped. The user doesn't really ever notice the hit, since we load the "kernel.js" first, which contains the bare bones minimum.

\- Use a CSS preprocessor. We use LessJS. We managed to reduce our CSS overhead by an overhead of 20% in file size while using the same sheet. When you use a preprocessor (although we supply the compiled versions) you are making the file gzip heaven. The ungzipped version is larger, but the compression is great. Additionally, maintainability improved by an order of magnitutde that I can't even count. Incidentally, we also provide the entire CSS for the entire site in a single .css file. Again, tests were done with splitting the CSS, but it just makes more sense to provide everything and improve the page loading on second requests (We don't have a high bounce rate so it might make more sense on our types of sites).

\- SpriteSheets. You can't really over emphasise the importance of these. HTTP requests are expensive when you are firing off multiple requests to the same domain. Depending on the browser you could hit blocking/waiting pretty quickly. It really pays off to merge your images into a sheet. Don't use a premade sprite sheet with 100 icons on it, where you only use 8. Make your own.

\- Cookieless Domains. Serve all of your static assets from cookieles domains. This one isn't neccessarily a major win, since the gain you get is reduced based as you optimize more. However, before I started optimizer, a single request would fire 80 HTTP requests just to static assets. The average user on the site would have about 500bytes worth of cookie data. I set myself the target of getting the request down to 400kb for _everything_. So like 2% (I can't do maths) of that was being taken up _just_ on the users /GET Request. It's not neccessary! Move everything to a CDN. We use CloudFront.

\- Browser Blocking. We don't have this issue any more, but we used to load _a lot_ of assets. As a result we would hit "blocking" really quickly. (This is where the browser stops loading assets from the same domain). This number changes depending on the browser you are using, but the normal number is 6. (Some even as low as 2!). The iPhone is the only phone which is also around 6. Most other phone devices are still at 2. This means that requests to the same domain (where you have lots) really sucks. So as a possibility you might want to consider using an asset serving library. The idea of this is that you provide multiple links to the same assets in your application, and round robin through the sources to make sure that you are rotating between hosts where the asset can be found. If you do implement somethign like this you need to make sure that the user is served the same host otherwise they wouldn't benefit from caching. The "hack" way that we implemented it was to use their HEADERS as a "seed key" which created a cache, which we would then perform some type of modulus caclulation on to serve the correct server. It isn't perfect, but it works.

\- Headers. Use correct headers.. expires... cache... if you are serving up cacheable content. Say so! Some study I read recently showed that over a period of a month, rechecking the same 1000 sites daily and checking each asset. Of those that never changed, over 50% didn't make correct use of expiry headers! If you do this you are telling your users that you don't care about performance!

\- Speed isn't always about waiting for that DOMContentLoaded to fire. It's also about perceived performance and loading. This means that you should always give your images a height and width attribute too.

\- Use a reverse proxy cache! The stack that we use is nginx(port 80) -> varnish -> nginx. This is so that we can benefit from server side includes which are cached in varnished, and rebuilt by nginx.

\- Eager Loading. I've only just started experimenting with this as a way of improving my knowledge on machine learning. As I mentioned above we have a really load bounce rate, so it might make more sense for us than it does for your site. Basically, I've started using Markov Models to build a probalistic model of the liklihood of a user navigating to another internal page and figuring out based on all other users which page that might be. I then drip feed load the assets from the other page.

<comment><USER3> Can you elaborate a little more on the reverse proxy tip? Why the double nginx, varnish in-the-middle approach? I'm not familiar with this technique. The rest of your comment is really good.

<comment><USER4> My guess would be SSL. Varnish doesn't handle SSL directly.

</comment></comment></comment><comment><USER5> Here are a few that were not mentioned:

\- If you have a lot of images like in a product catalog, try lazy loading the images - <http://www.appelsiini.net/projects/lazyload>

\- For sprite-ing your images, use some tool which will automatically generate the css for you: <http://spritepad.wearekiss.com>

\- Have a git commit hook for compressing images. (It could be part of your build, but then you are doing the work every time and the build will be slow if you have a lot of images.)

\- If you are using LessCSS or something similar (hint: you should), have your compiler compress the css for you. - <http://wearekiss.com/simpless>

\- Have multiple hosts referring to your CDN to facilitate parallel downloads in the browser.

\- Compress your HTML too before deploying - <http://code.google.com/p/htmlcompressor/>

\- Have proper versioning in the name of your JS/CSS files and then cache them for ever. Whenever you change your files, the version should change and only that file will be updated in the client. We normally take the hash of the latest git commit on that file:

       cookie-wrapper.js -> cookie-wrapper.bb9303b809210674.js

\- Offload whatever you can from your cookies to localStorage or things like that to reduce request size.

<comment><USER6> Can you expand on simpleless a bit, and why it is better than using lessc from the lesscss project? I'm not seeing any good resources on the project page and a googling is turning out to be basically useless to find more info.

<comment><USER7> I believe the lessc compiler requires a node environment and simpleless is self contained.

</comment></comment></comment><comment><USER8> Lots of good tips in here already. Yahoo has a great speed guide (<http://developer.yahoo.com/performance/rules.html>) which I'd suggest reading through, they cover pretty much everything in this blog (and chrisacky's comment above), and go into some more detail.

For testing your site, I recommend WebPageTest (<http://webpagetest.org/>). Run the test a couple times to get an averaged score, then start tweaking. Watch the HTTP requests. Watch the total download size.

Google also provides a Page Speed tool (<https://developers.google.com/pagespeed/>) (also built into Chrome) that will point out any mistakes you may be making.

The biggest things to focus on are caching, limiting HTTP requests (combining images into sprite sheets, bundling scripts), and optimizing images. Learn about these.

My favorite tools for optimizing images are PngOut (<http://advsys.net/ken/utils.htm>) and JpegTran (<http://jpegclub.org/jpegtran/>). Both are lossless, I generally just run the tools batch through my websites image folder. If you want lossy Jpeg compression (you probably do, just keep the original file safe) then try Jpeg- Optimizer (<http://jpeg-optimizer.com/>). If working with a large photo on the homepage or something like that, try multiple levels of compression and find the best quality/filesize tradeoff. You can often cut the filesize in half with no noticeable degradation at 100% zoom.

</comment><comment><USER9> > Make sure you define a width and a height for all images on your webpage. > This helps save time since the browser doesn't have to do the work of > calculating the dimensions.

This isn't correct in my opinion (it might be technically but not to a level where anyone would notice it), what it actually does it reserves the space in advance for the image as the rest of the page loads. If you don't do this then what happens is that the content keeps being pushed down which makes it seem like it's loading slower. So this impacts perceived loading time over actual loading time.

<comment><USER10> > what happens is the content keeps being pushed down

What happens is that the browser is re-flowing and re-painting part or all of the page, rather than updating just the area the image is known to occupy. Reflows are expensive, particularly on mobile.

[https://developers.google.com/speed/docs/best- practices/rend...](https://developers.google.com/speed/docs/best- practices/rendering#SpecifyImageDimensions)

Also of note is that in some circumstances (as mentioned in the link above,) the browser may have to delay rendering some parts of the page until it knows the dimensions of either the image or one of its containing elements, which means rendering the container might have to wait until it fetches enough of the image to calculate the height and width.

</comment></comment><comment><USER11> About the twitter/facebook buttons. You can just delay until after the window.load event has occurred. It will have no impact on the page load, since the page is fully rendered at that point.

      $(window).load(function() { /* load twitter and Facebook buttons */ });

<comment><USER5> We do something like this:

      $(function(){
          setTimeout(function(){
              /* load twitter, facebook */
          }, 2000);
      });

This tries to make sure that all the essential work to be done on the page is completed before the less important Twitter, FB buttons are loaded.

<comment><USER10> You are making part of your page always load at least 2 seconds slower than is probably necessary.

$(function(){}); is shortcut for $(document).ready(); which waits until all other external content has been loaded. [1]

If for some reason you are loading something else you need after .ready() fires, and you can't load that inside of a single .ready() call for some reason, you can potentially add multiple callbacks to .ready() (they will be executed in order [2]), or attach a .load() to that particular item instead (maybe even inside your .ready().)

Timeout hacks are not a performance improvement.

[1] <http://api.jquery.com/ready/> [2] [http://stackoverflow.com/questions/5263385/jquery- multiple-d...](http://stackoverflow.com/questions/5263385/jquery-multiple- document-ready)

<comment><USER12> $(document).ready does "not wait until all other external content has been loaded". It fires when the DOM is fully parsed, irrespective of whether there are still images or iframes or other assets still loading.

[http://stackoverflow.com/questions/4395780/difference-bw- onl...](http://stackoverflow.com/questions/4395780/difference-bw-onload-and- document-readyfunction)

</comment></comment></comment></comment><comment><USER13> Regarding gzipping, you don't want your webserver doing this on the fly. At least in nginx, you can tell the webserver to load pre-gzipped files, I'm sure apache has something similar.

Put this into nginx.conf: gzip_static on;

Then run this command in /var/www to gzip your content: find . -name "*' + ext + '" -print | xargs -I filename echo "gzip -c filename > filename.gz" | sh

This is a small win (reduced cpu/disk IO), but at two lines it's an even smaller cost.

Also, if you use django mediagenerator, it will handle a lot of this stuff for you.

[http://www.allbuttonspressed.com/projects/django- mediagenera...](http://www.allbuttonspressed.com/projects/django- mediagenerator)

<comment><USER14> Keep in mind that serving already properly minimized png, jpg and gif images with gzip can actually increase their size.

</comment></comment><comment><USER15> Nice overview, for even more tips on performance the best resource is Steve Souders' blog:

<http://stevesouders.com/>

</comment><comment><USER16> The most important thing for page loading times is reducing the number of network roundtrips required to view a page. A visitor across the globe will have a latency of around 300ms per request. If you include a javascript that adds a link to a stylesheet, this means you have at least 3 serial requests, and the page loading time is now at least 1 second, no matter how large the bandwidth or how much you compress files.

This is also important on mobile. Mobile broadband is optimized for high throughput, but you often have very high latency. Therefore it's especially important to reduce the number of round trips.

</comment><comment><USER17> I'm surprised they left out concatenating your javascript files into as few files as you can without breaking caching. The concept is the same as using sprite sheets, with similar gains if you are using many different scripts.

</comment><comment><USER18> Kind of looks like you're loading jQuery and not using? That's a good chunk of bandwidth and execution time there!

</comment><comment><USER9> With regards to Twitter/FB, frustratingly in my Webmaster tools it shows our homepage loads slowly, and a huge percentage of this loading time is actually the FB/Twitter buttons.

For users this doesn't impact their perceived loading time much at all so when Google reports it as a slower load time it's an unfair bias in my opinion.

It's important to get it fast as it's a well known fact they use it as a metric in their ranking algorithms. I've tried various ways of delaying it but nothing seems to stop it being counted in page load time tools:

[http://stackoverflow.com/questions/9502972/stop-pingdom- coun...](http://stackoverflow.com/questions/9502972/stop-pingdom-counting- social-media-in-website-speed-report)

<comment><USER19> Wait for a scroll or mousemove event after window.load

Google's crawler doesn't trigger events, but it does run the js past window.load. If page speed just waited for window.load, it would be useless.

</comment></comment><comment><USER20> How we trimmed 2 seconds from page load times: remove AdSense.

</comment><comment><USER21> We did something similar with KeepAlive on <http://settleapp.com> \- we figured that a new user with an empty cache (it's just a holding site, so most users are first-time visitors) would need to load the index and 23 other assets.

We set keepalive to 24 requests over a single http connection - the whole site loads noticeably quicker as a result.

<comment><USER10> Your stylesheet is referencing "../img/.png", which clearly doesn't exist. 404s hurt a lot.

You are loading google analytics [1] and jquery synchronously at the top of the page. Moving jquery to right before </body> is advisable at the very least if you can't load it async for some reason, so that it won't block the rest of the page. [2]

Most of your static assets have no caching headers (max-age or expires) at all.

Your map image could be 128 color png (neuquant + pngout) and nearly 50% smaller with no discernable to the human eye loss [3], and similar can be said for the phone template [4].

Your background tile image is unnecessary, or at the very least could be a 1x1 instead of a 100x100.

All of these things will improve performance for everyone, and would have been better targets than messing with keepalive. There is also more left to do...

Hope this helps. :)

[1] [https://developers.google.com/analytics/devguides/collection...](https://developers.google.com/analytics/devguides/collection/gajs/#quickstart)

[2] <http://developer.yahoo.com/performance/rules.html#js_bottom>

[2] <http://dl.dropbox.com/u/571990/settle/map.png>

[3] <http://dl.dropbox.com/u/571990/settle/phone_template_bg.png>

<comment><USER21> Wow this is great stuff. Some of this I knew, but some is new to me. Thanks!

On [2] the map pins are noticeably poorer - but this could probably be addressed.

</comment></comment></comment><comment><USER22> A somewhat silly tip: don't include the Google PlusOne button. Even when it's set to load after every other asset, and load asynchronously, it still takes ages to load. I've sometimes browsed off the page before it finishes loading.

</comment><comment><USER23> I'm not a web designer, but to me 400kB sounds like a huge amount of data.

</comment><comment><USER24> ah, I so much wish that my app reaches "optimization" stage.

<comment><USER25> 80% of client side optimization can be done in 2 hours or less. Research shows time and time again that faster websites convert and retain vistors better. It's worth the 2 hours no matter what stage your project is.

<comment><USER17> I have to disagree with you there. Some optimization steps significantly hamper development and should be saved until near the end of the project. For example, JS minification is very fast and easy, but it makes debugging much more difficult.

And you shouldn't compile things into CSS sprites until you are fairly settled on what images you'll be using. If the look and feel of your site is still in flux, you're wasting time by making sprites.

<comment><USER25> Use automated tools to minify files on production but not on your development server or localhost.

I'd put CSS sprites into the 20% you do later. Some images are very simple to merge and don't change (such as icons) but the rest, yeah, wait until you sort it out.

<comment><USER17> >Use automated tools to minify files on production but not on your development server or localhost.

Yeah you can do that, but there are still points in the project where it makes little difference to do that, and it's not hard to get stuck in those points.

Of course, once there's wide support for source maps, you'll be able use minification even in development.</comment></comment></comment></comment></comment>


A tool to embed PowerShell inside PNG files - <AUTHOR> https://github.com/peewpw/Invoke-PSImage <comment><AUTHOR> (not mine, just thought it was cool)</comment>


Coq: A program specification and mathematical higher-level language - <AUTHOR> https://coq.inria.fr/ <comment><USER2> Dozens of previous threads: [https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...](https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=Coq%20comments%3E2&sort=byDate&type=story)

</comment><comment><USER3> Coq 8.11.0 was recently released: [https://github.com/coq/coq/releases/tag/V8.11.0](https://github.com/coq/coq/releases/tag/V8.11.0)

This version introduces a new tactic language, Ltac2.</comment>


Show HN: ScienceTone – The latest scientific discoveries' aggregation network - <AUTHOR> http://www.sciencetone.com <comment><AUTHOR> Hi! I just launched a project, which I have made. You can find it here [https://sciencetone.com/](https://sciencetone.com/)

A brief explanation of what this project is about.

In this website people can share the latest studies, which they find interesting. There's a voting system, so the most interesting submissions go to the top. On each submission people can comment and ask questions. Comments must be objective and based on other studies. Same with the replies to the comments and questions. So each submission can turn into an objective discussion on the subject.

Why it is useful?

I'm a sports science graduate, so I constantly follow the latest publications, related to wellness and training methodologies. Also I'm interested in computer science too. So I follow a lot of people on social media, who are experts in those two fields. They often share interesting findings. I hope that platform like this would allow those people to share studies & conclusions, what they find interesting, with some additional and more diverse audiences. Also those shared studies might generate more ideas and questions by attracting other people, who are interested in the same subject, and expand its value by participating in the discussion section, below the submission.

I still have a lot of features in my mind, which could improve the website. For now I published the core features to see if anyone is interested in the idea itself. I think a platform like that might be useful and make people a little bit more aware about the latest discoveries. I think right now a lot of exciting things are happening in the majority of science fields.

P.S. Sorry for the duplication. In the previous post I forgot to add "Show HN:". It wasn't my intention to post two times. It's not possible to delete previous post.</comment>


Ask HN: Need a career advice - <AUTHOR>

Hi,
I'm passing out from medical school at the end of this year. However I'm not interested in working the rest of my life in a hospital as a regular doctor. I love programming and computers since childhood and even while at medical school I learn new stuff and make mobile and web apps and stuff for learning and for fun. Also I learn different languages and frameworks whenever I have free time. And also does freelance stuff for my friends doing IT and CS at college. And I have basic knowledge in CS. Although I don't have any degree in computer science I can try to get one in the future.
Is there a way for me to work in the computer industry with the experience I got from medical school and using the computer knowledge I have?
I checked bio-informatics field and from the things I found on the internet the salary in bio-informatics profession is half in that of a physician in US.
Are there any other fields other than bio-informatics where I can follow an alternative career other than becoming a doctor and also because I'd love to work in the computer field.
Thanks <comment><USER2> I'm a software engineer who has spent most of my career working for medical device companies. My advice may not be a good short-term fix and it's just one of many paths, but it's based on my experience.

Be a doctor. A good one. The more specialization, the better. Gravitate towards a specialization where technology either plays or will play a big role (surgery, radiology, etc.). Get experience with these technologies (new or old), such that you know what works and what needs improving/replacement. If you can, write cases and do research that influences your field. Work at a hospital with a strong bio-tech research element and find ways to collaborate (hint: collaborating clinicians are _always_ in demand). Through collaboration, you'll inevitably be exposed to spin-off and startup opportunities. You'll work with all sorts of programmers and engineers, who will drop everything to spend time with you when you have a free moment.

Once you've got some good experience under your belt and perhaps have built a small name for yourself, congrats, you are now a relatively hot commodity for a multitude of medical device companies. We often use the term KOL (key opinion leader) for certain clinicians - they command big bucks and get huge influence on product development. If you network well, you'll get early access to new technologies, perhaps the ability to work with the engineers during the design phase, your opinions will be treated as gold when it comes to product development.

This all hinges on you actually practicing medicine in a fairly active fashion for at least a few years if not a decade. I know that's a big commitment, but you should know that it comes with a lot of non-monetary rewards. It opens doors.

Even as a relatively unknown MD, a few years of active practice are invaluable in your back-pocket. If you ever decide to do a medical-based startup, investors will surely value your credentials.

To be perfectly honest, I'm an engineer, I love software and technology, but if were to do it all over again, I'd go to med-school first!

<comment><AUTHOR> That's the hardest part, the stress and the depression of the job will surely drive me nuts in couple more years, that's the main reason I want a change as the stress, night shifts, working with patients listening to their problems all day all night is depressing and frustrating.

Thanks I've never thought of that path of working at a medical device company. Good that you mentioned it :)

</comment></comment><comment><USER3> If you are comparing salary of a physician to a tech related field it is going to be hard to find something on pair with out a lot of schooling.

<comment><USER4> Yes, indeed, and because of this, I'd suggest you not rule out bioinformatics before talking to someone who does it.

I'm a bioinformatician, I love it, and having both an M.D. and skill in programming would be a huge competitive advantage. Take a look at the work of Atul Butte, who is a M.D. that has got into bioinformatics at Stanford, and is very successful.

It is true that until you reach high levels of the research hierarchy, (where salaries can be $250K+), the pay is not good. However, if you get into bioinformatics for industry, or do clinically-oriented startups using bioinformatics methods, you can do quite well. It is a career that combines the ability to use computer science to solve real-world biomedical problems with an environment of high intellectual freedom. To me, those things are worth the opportunity cost in pay. Also, I would not pay that much attention to "average" salaries as you can do better if you are exceptionally skilled, and your M.D. probably would also command some kind of premium.

But I can easily see how I might think differently if I had a lot of student debt from pursuing an M.D.

</comment></comment><comment><USER5> I have two people in my close circle who made jump between medical profession and information technology and vice-versa.

[1] My cousin became a medical doctor and practiced for a year. He just couldn't disconnect himself from his patients sickness and that was causing him depression. Long term, he just couldn't see himself around sick people without becoming mental nut-case himself. He was very interested in Technology but couldn't see himself starting from entry level. He decided to go for MBA at a top 5 school. It seems not many medical doctors apply for MBA so it was easy for him to get admission. After MBA, he initially worked in healthcare technology consulting. He found his medical degree, Dr before his name and MD and MBA after his name very valuable since then in his career. Today, he leads healthcare technology practice for a major technology service provider and responsible for over billion dollar in revenue and P&L for his division.

[2] My eye doctor has graduate Electrical Engineering degree and worked for about ten years in development with SoCal defense contractors and NorCal technology companies before quitting to go to medical school. He is now a successful Eye Surgeon. While working in technology, he didn't like the politics of the workplace and overall work environment and expectations of technology companies. He has been very happy since becoming Eye Doctor and enjoys working with his patients. He still gets his technology fix by getting involved in medical technology innovations. He says his EE + Medicine qualifications makes him quite a desirable person in medical device industry.

It might take you a decade in technology to even get to the level of starting salaries of medical doctors. If you have high medical school debt, you may want to consider this in your decision to switch career.

You may benefit from your medicine background in a "relevant" technology company if you slide into the company at mid-level where you medical background is valued. Entry-level developer or IT person may not offer you any extra credit for being a medical doctor. Also in a non-medical technology company, you may be at a disadvantage compared to a person with CS degree with promotions and being taken seriously.

Unless you are interested in going to Business School, personally I will not recommend switching field from medicine to technology. You can always continue as medical physician while being involved with technology on the side.

<comment><AUTHOR> Thanks, I think I'm more like your cousin. I just can't take the stress of working at a hospital and with patients, heavy night sifts sleep deprivation just drives me nuts.

Well since I get the education at medical school for free I don't have any debt to pay. I think I've made the wrong decision in entering medical school in the first place, here getting in to a med school for free is like a once in a life time opportunity at they young age I couldn't resist it but now older I feel I made the wrong decision.

None of my friends who are outside of med school believes in the stress and depression as they only see the high salaries and good life doctors are living but doesn't see the stress until you enter the field,

<comment><USER5> "None of my friends who are outside of med school believes in the stress and depression as they only see the high salaries and good life doctors are living but doesn't see the stress until you enter the field,"

Grass is always greener on the other side. I will encourage you to explore more your options between medicine and IT and make sure you are not afflicted with same syndrome as your friends.

Medical profession has a good reputation and doctors are held in high regards in non-medical professions. I will suggest leveraging this reputation and respect to get yourself into a good position in non-medical profession if you don't want to practice as a medical doctor. Don't squander it away by becoming an entry level drone in technology industry.

Another alternative may be to consider medical profession as means to achieving the end you want. Tough it out for a few years, achieve financial independence as quickly as possible and then do what you want to do in technology whether medical related or not.

Have you heard of White Coat Investor [http://whitecoatinvestor.com](http://whitecoatinvestor.com)? He is an ER doctor who started blogging about personal finance from doctor's viewpoint. He seems to have become quite popular in finance area with books and speaking engagements specially targeted at medical professionals. It seems he is drawing almost $200K from his finance-related side activities on top of whatever his ER compensation is. You could follow on his footsteps. Suck up and be a medical doctor for a few years, keep technology as side activity until your side activities become as big as being medical doctor then do whatever you like.

<comment><AUTHOR> Thanks, no i'm not giving up on my current degree, as I'm going to graduate by the end of this year I'll be a doctor anyhow, it's after getting the degree I'm thinking of following an alternative field with the use of my medical degree.

I was reading this article on the Atlantic, that talks about following MBA together with the MD degree [http://www.theatlantic.com/education/archive/2014/09/the- ris...](http://www.theatlantic.com/education/archive/2014/09/the-rise-of-the- mdmba-degree/380683/)

Although I love computers and technology it doesn't mean I want to work in a software company, what I need is an alternative field so I can have a good family life and I don't have to work in a depressing environment and committing suicide few years later.

Thanks for the suggestions they are really helpful

</comment></comment></comment></comment><comment><USER6> Reach out to some of the participants at Medicine X, they may have some helpful perspective > [http://medicinex.stanford.edu/2014-schedule/](http://medicinex.stanford.edu/2014-schedule/)

</comment><comment><USER7> Why limit to something that relates to the medical field? Just focus on programming as a career.

<comment><AUTHOR> Well I was not sure whether if I can get into a company without any degree in computer field, there are times I wonder if it would have been better me doing CS than medicine from the beginning

<comment><USER7> Focusing on making it medical related won't increase your chances by much if at all, only to narrow your opportunities. If you can code, it shouldn't matter. Speaking from experience and from people I know personally, your lack of a degree isn't the main deterrent from getting a job programming in most cases</comment></comment></comment>


Amazon Simple Notification Service - <AUTHOR> http://aws.amazon.com/sns/?ref_=pe_8050_15058410

<comment><USER2> A few weeks ago I was looking at signing up for an smtp service rather than using gmail to send emails, due to the limitations in sending mail via gmail.

However, from what I understand, it seems setting up your own server is pretty much a guarantee that your emails will get labelled as spam.

Any idea on how spam filters treat email from Amazon?</comment>


Great things and people that I discovered, learned, read, met, etc in 2017 - <AUTHOR> http://blog.fogus.me/2018/01/02/the-best-things-and-stuff-of-2017/ <comment><USER2> They apparently read 100 books in a single year. How does one accomplish that? I think the most I've ever managed to read was two books in a single week. That was light reading though and I was traveling with lots of downtime.

<comment><USER3> I read 100 books last year as well. Commuting by bus an hour each way is a lot of time to fill. I highly recommend the Libby app if your local library supports it. Free books on your phone or Kindle!

<comment><USER4> Haven't heard of Libby, but there's also 'Axis 360', 'hoopla', and 'OverDrive'. My library uses all three. You choose a book and it'll only be available on one of the apps.

</comment></comment></comment><comment><USER5> I find it cool that he sees Steve Yegge's as a great blog post even though it's hyping Kotlin and critical of Clojure.

</comment><comment><USER6> I'll second Finite and Infinite Games. It's kinda weird, but carries many interesting thoughts.

<comment><USER7> I just wrote about this on fogus.me: oddly enough (the book is not really long) is one of the rare books I quit maybe 3/4 into it: the ideas are interesting, but To me it seemed like the author had a couple of concepts and he just kept repeating and reformulating these over and over. Maybe I should have another look at it.

</comment></comment><comment><USER8> What does "fruit-fly code" mean? Short programs?

<comment><USER9> See things like

[https://github.com/aosabook/500lines](https://github.com/aosabook/500lines)

[http://plzoo.andrej.com/](http://plzoo.andrej.com/)

Or any somewhat readable, compact codebases. Redis, Lua, etc. Often they are pedagogical in purpose, but sometimes on accident due to ruthless factoring.

No one can "learn compilers" from reading LLVM source code. But one could get the idea in a couple hours of reading

[http://www.iro.umontreal.ca/~felipe/IFT2030-Automne2002/Comp...](http://www.iro.umontreal.ca/~felipe/IFT2030-Automne2002/Complements/tinyc.c)

And in a couple more by porting it to a new language.

Think of building an exact scaled down replica of a plane or a ship before embarking on the larger version. Now build an even lighter weight model, just for learning how to build the model. This is a "fruit-fly" code.

</comment></comment><comment><USER10> A good list, however quite voluminous. If I went through that much material in one year I would never have got any work done.

</comment><comment><USER11> Michael's yearly list is always good. I am curious why he is interested in Prolog implementations.

<comment><USER12> Could be because he uses Datomic, which uses Datalog, a subset of Prolog.</comment></comment>


Ask HN: How do you make a big life change? - <AUTHOR> Throwaway account. I'm 31, gay male, single, live in SF. I've worked in tech in various non-technical roles for a few years and managed to save $1m getting lucky being in the right place at the right time.
I'm not really a career person and career has gone sideways for some time. I don't see myself being a VC-backed founder or VC or tech exec anytime soon. I don't really enjoy the SF tech scene. I'm also terrible at networking.
I don't have any particular need to stay in SF but feel adrift and unsure what to do with myself.
I'm aware I'm in a privileged situation.
Anyone face a similar situation? How did you decide what to do next? <comment><USER2> Leave all your stuff with a trusted friend. Buy a harness, a tent, sleeping bag, and TC Pro shoes. Put it in a backpack. Don't bring anything else. Pull out $500 cash from the bank, get on a bus and go to Yosemite. Live in Camp 4. Meet the dirtbags, learn to climb. When the rangers kick you out follow the migration and go to Index, or Squamish.

</comment><comment><USER3> When I felt a bit in a similar situation I jumped ship. I moved country (Easier for me, since I live in Europe) and I changed everything. New friends, new hobbies. The amount of things you learn when putting yourself in "uncomfortable" situations is incredible. I am aware this is not an easy change, but for me it was easier to do it jumping in with both feets. I did it 3 times, and I always jumped in the unknown. I always landed better off than I left.

</comment><comment><USER4> After my divorce I quit my job, moved to Thailand and have been unemployed for 18 months. I dont have anything like that sort of money in the bank and I dumped most of my possessions before I left, but I was overpaid before I left and had money from the house sale and company shares. Ive got enough cash to last a few years.

Im working on a business that may some day earn some serious cash, but for now, money is not important. Being free of possessions (aka shit) is the best feeling. Very liberating.

Ive met a beautiful Thai lady and we are very happy. I spend time in Thailand and New Zealand, have travelled to europe and all over asia.

Couldnt be happier.

</comment><comment><USER5> Two things have helped when I've felt adrift in the past:

* Reflection: Think about what activities are really useful for introspection (this is person-specific) and do a lot of it. For me, it's journaling -- I started writing a lot. I came across an interesting site recently (750words.com) where people write 3 pages a day and it trains you to just get your thoughts out. Seems interesting.

* Make a big change to your environment: Take a sabbatical or a big trip. Go to a place where you can immerse yourself in an environment that feels completely different. Actively try to get inspired and to find yourself.

Good luck, friend!

</comment><comment><USER6> Move.

The location you are staying in has become toxic to you. For whatever reason. Go to a gay-friendly location so that you don't hate yourself. You might like a city/place and "life" because of a coffee shop.

Obviously, life is more complicated than that and you'll need infrastructure from time to time and maybe daily. If I were you, I'll look up gay-friendly locations and give them a short trip. Minimize costs as much as possible.

Don't stay in tourist friendly locations. Try to go to places you'd go if you were living there. See what it feels like. Some country/community/city will be a right match for you and then you'll fix a significant psychological problem.

</comment><comment><USER7> I've been thinking about this a lot lately, and have started making the effort to save money so I can take a few months off when I'm done with this job.

I've gotten halfway there without changing much. Reading Lead Yourself First and Digital Minimalism have taught me the importance of being intentful with anything you do and to leave yourself time alone with your own thoughts. The latter has proven especially useful for me recently.

</comment><comment><USER8> Only you can answer that question. What do you love doing? Do more of that. With 1m and no one to support you might not need to work again, but that doesn't mean you can't!

</comment><comment><USER9> $1m affords you a lot of options.

You can quit your job and take some time off to figure out what you want to do next.

</comment><comment><USER10> Go travel the world and stay in hostels</comment>


Ask HN: What are some communities focused on web design and dev? - <AUTHOR> <comment><USER2> MDN (Mozilla Developer Network) came immediately to mind: [https://developer.mozilla.org/](https://developer.mozilla.org/)

</comment><comment><USER3> Dev.to is a decent place</comment>


My wife used Dressrush's pitchdeck project to build her landing page - <AUTHOR> http://www.borrowblue.com

<comment><AUTHOR> Also used Wufoo for the contact form. They were really great to work with.</comment>


Miami Police Used Clearview AI Facial Recognition in Protester's Arrest - <AUTHOR> https://www.nbcmiami.com/investigations/miami-police-used-facial-recognition-technology-in-protesters-arrest/2278848/ <comment><AUTHOR> NYPD too [https://www.theverge.com/2020/8/18/21373316/nypd-facial- reco...](https://www.theverge.com/2020/8/18/21373316/nypd-facial-recognition- black-lives-matter-activist-derrick-ingram)</comment>


Sci-Hub's cache of pirated papers is so big, subscription journals are doomed - happy-go-lucky http://www.sciencemag.org/news/2017/07/sci-hub-s-cache-pirated-papers-so-big-subscription-journals-are-doomed-data-analyst <comment><USER2> I'm very happy to see SciHub going strong - for all the obvious reasons. Now let's just hope they back up to IPFS (if they do, I'll happily pin some of it).

I want to go off a tangent here, though. Now that open access (whether arXiv or SciHub style) is becoming the norm, I wonder what can be done to improve the format of scientific papers? Like e.g. making them more like this:

[http://worrydream.com/ScientificCommunicationAsSequentialArt...](http://worrydream.com/ScientificCommunicationAsSequentialArt/)

instead of regular PDFs?

<comment><USER3> I don't know what that link is, but after 10 seconds of loading over LTE I gave up in favor of my data bundle. I hope it doesn't go towards whatever that is.

Personally I'm a big fan of blog posts: they're written for a broader audience in understandable language. I don't have _any_ trouble reading English, but papers are almost universally written in a way that is more complicated than necessary. And in blog posts, if someone made a "novel new algorithm", at least you can be sure there's code, raw test data, screenshots, or whatever else is needed to DIY. And someone on the web is talking about it, perhaps even with comments below the post itself.

<comment><USER2> The link is to the Bret Victor's rewrite of a scientific paper, in which he introduces illustrations and interactivity for the important (and difficult) points of the paper.

I edited the link to the more "lightweight" version (i.e. just the page, without loading his entire homepage as well, which is kind of heavy), so please consider trying again.

I agree with you RE blog posts, and the point of me linking to Bret Victor's work is to give an example of a form that's better-suited to the media we operate with today. Papers still live in the paper age (pun not intended).

<comment><USER4> I love Bret's work, but I consider this rewrite pretty much entirely unreadable. I'm sure it's great for skimming - what with the bolding, and the flood of pictures - but that's not the point of scientific papers.

<comment><USER5> It would be better with a layout that flowed only from top to bottom. Reading text mixed with diagrams left-to-right, and then top-to-bottom, is hard, as you point out.

Since Bret has liberated the scientific article from paper, we may as well take advantage of the more flexible layout possible with digital media. Interactive diagrams are so nice for learning that I think it's worth pursuing Victor's idea further. Fred Akalin's blog has a couple great examples! [https://www.akalin.com/quintic-unsolvability](https://www.akalin.com/quintic- unsolvability)

</comment></comment></comment></comment></comment><comment><USER6> Good riddance, limiting access to scientific articles is a detriment to the advancement of humanity.

<comment><USER7> The sooner all these assholes go out of business, the better

I hope scihub has a well thought out exit plan, in case of seizure.

~ 40 M files, at 1 MB each (my own estimate). I hope they have backups / insurance files with all the papers collected this far. Given the data volume it's not trivial but surely not impossible.

<comment><USER8> They backup to Library Genesis as far as I understand it, Library Genesis is available via Torrents & Usenet - [http://gen.lib.rus.ec/repository_torrent/](http://gen.lib.rus.ec/repository_torrent/)

<comment><USER9> The IPFS network is ideal for this kind of situation.

I should say I'm bias as I run ipfsstore.it but to host all 26gb there would not be expensive.

<comment><USER10> So for your service 26TB * 0.044 USD/Gigabyte = $1144/month to host all the Scihub data

Quite a big fee :p

<comment><USER11> That seems rediculously cheap and easy to come up with funds for. Storing the entire progress of science for $1k/mo? That's a good deal.

<comment><USER10> True, a low grade ICO for SciHub, backing an multi-country IPFS store with a decent web interface on top could raise that money easily. You could create a mini forum/social network around it for commenting on papers. Buying a coin to make an account or freemium buying a 'VIP' accounts ala Dribbble...maybe even a DocumentCloud style annotation system, so it has some self-sustaining business model other than raising initial capital.

Although I don't know much about creating 'DApps' or how Ethereum or IPFS works or what SciHub contains beyond a primitive grasp [1], just sayin :p

[1] I watched a video of a talk "Distributed Apps with IPFS (Juan Benet) - Full Stack Fest 2016" [https://www.youtube.com/watch?v=jONZtXMu03w](https://www.youtube.com/watch?v=jONZtXMu03w)

<comment><USER11> Why does everything have to be an ICO? What's the point of a token here? Just give the appropriate organization some charitable funds.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> Basically, the publishers asked for this. Denying open access to old papers from humanitys point of view is wastefull. The planet is full of hungry minds. Who knows where the next Ramanujan comes from and which discipline he or she chooses but given the non existent transaction cost of reading an old paper it would be beyond silly if they could not do it for free.

<comment><USER13> Publishers _can_ provide value, and should not be forced to do so at a loss. Peer review, curating for quality, editing, typesetting, and creating physical copies could be expensive.

True, current publishers don't compensate peer reviewers, limit curation to the set of people who will pay their submission fee, force editing and typesetting back on the authors, and charge egregious fees disproportionate to the cost of distribution. But the concept of the business isn't invalid.

<comment><USER14> Nobody cares if publishers could _theoretically_ add value in some alternate universe. In practice, none of them _do_.

This isn't throwing the baby out with the bath water, this is people trying to stop this nasty ass bath water from being thrown out because there _used to_ be a baby in it.

<comment><USER15> Working at one, I have to say I don't think this is true for _all_ publishers.

Prices are a different thing, but I'm definitely spending my day here adding value.

<comment><USER12> Some think anything can be open sourced but that does not work in the current world where we live. For an obvious example SpaceX could not have been an open source project.

I suppose the general notion is that the only problem publishers solve is the distribution which is kinda non-value-adding nowadays, but don't think about the editorial process, which needs actual work.

Sure, I think you guys add value. My critique was towards the cost of old papers and the ownership which journals take for published work. You guys need to get paid but the world also needs it's open access to the scientific corpus.

The papers which have very little value adding editorial input do have a harder time legitimizing their slice of the pie, though.

<comment><USER16> Maybe not open source, but an open hardware project. The particular problem for SpaceX is that a lot of the IP is covered by ITAR: [https://en.wikipedia.org/wiki/International_Traffic_in_Arms_...](https://en.wikipedia.org/wiki/International_Traffic_in_Arms_Regulations) See this FAQ entry: [https://www.reddit.com/r/spacex/wiki/faq/company#wiki_what_p...](https://www.reddit.com/r/spacex/wiki/faq/company#wiki_what_patents_do_spacex_hold.3F_would_they_consider_releasing_them_like_tesla_motors_did.3F)

But in theory it's possible, e.g. Facebook's Open Compute Project, it just needs the right kind of funding model.

</comment></comment></comment></comment></comment></comment><comment><USER17> I had quite a bit of exposure to pirate journal archives before sci-hub arrived. A couple of easy improvements that I saw with past pirate libraries, that it'd be nice to have on sci-hub:

\- Strip download watermarks ("Downloaded by Wisconsin State University xxx.xxx.xxx.xxx on January 12, 2017 13:45:12"). Many times, journals published by the same publisher do the watermarking similarly so you need write just one pdftk (or other PDF manipulation software) script for every journal under their banner. At worst, it's a one-script-per journal effort.

\- PDF optimization. A lot of publishers produce un-optimized PDFs that could be 25% (or more) smaller with a completely lossless space optimization pass. This should save storage/network costs for access to individual papers and, more importantly, reduce the burden for bulk mirrors.

(I'd contribute the scripted passes myself if I had contacts within sci-hub.)

<comment><USER18> I deal with making PDFs a lot, and some of my processing steps (using pdfjam) make the file size explode. Currently I have no efficient way to make them smaller again. I would love to see how you do it.

<comment><USER17> The Multivalent PDF tool is still my go-to. It hasn't been updated in ages, but the jar still works fine:

[http://multivalent.sourceforge.net/download.html](http://multivalent.sourceforge.net/download.html)

If you run

java tool.pdf.Compress file.pdf

it will generate file-o.pdf and often trim quite a bit of weight.

That's probably what I would suggest sci-hub use too because it's easy to use and automatic.

EDIT: this looks quite promising too: [https://github.com/pts/pdfsizeopt](https://github.com/pts/pdfsizeopt)

But it has bugs with handling some images... converts them to black boxes. I'd only use this one if you manually verify the result first (so not a fully scripted process). (Or this could be due to me not installing all the bleeding edge dependencies from source as instructed.)

<comment><USER18> Thanks. I didn't try Multivalent as I didn't have a Java VM. Maybe I should take another look. I tried pdfsizeopt, and IIRC it didn't actually reduce my particular PDFs.

</comment></comment></comment></comment><comment><USER19> The change won't be immediate though. I don't think universities, which are journals' bread and butter, are going to stop their subscriptions anytime soon. Stopping a journal subscription because everyone is using sci-hub anyway (even if they researchers really are on an individual basis) might open the door for copyright suits against the universities, which would undoubtably be more expensive than just keeping the subs going, especially since its just a line item in an accountant's book. I'm sure it will happen eventually, but journals might have enough time for some to pivot to some more nuanced business model before they go bust.

<comment><USER20> > I don't think universities, which are journals' bread and butter, are going > to stop their subscriptions anytime soon.

Around 60 German universities have cancelled their contracts with Elsevier: [http://www.the- scientist.com/?articles.view/articleNo/49906/...](http://www.the- scientist.com/?articles.view/articleNo/49906/title/Major-German-Universities- Cancel-Elsevier-Contracts/)

Obviously, the stated reason for cancellation would be that costs or too high, not that everyone downloads papers from sci-hub.

<comment><USER21> The European Union requires publicly funded papers to be publicly available. Elsevier did not provide open access, and was therefore deemed illegal for publicly funded research.

They have since introduced open access options, and universities have renewed their subscriptions.

</comment></comment></comment><comment><USER22> Alexandra Elbakyan's work is one of the most positive and important things to happen in the last 3 decades in the field of science, which has been gradually losing its luster due to the bastardization and devaluation of the field by politicians and salespeople using it like hucksters.

Elbakyan's work has inspired me to only publish my work in jornals that embrace open access and open data. I'll be damned if I am a slave to impact factor and other haughty metrics.

</comment><comment><USER23> The origin of the web was to disseminate scientific knowledge. The guardians of that knowledge- the journal publishers- have absolutely failed to make a viable business model out of this, while many companies who adopted the web made billions.

While I do not use Sci-Hub, I think that users who use it are doing so morally and ethically (in the sense of conscientious objection). i hope they are also willing to pay penalties if they are found to be violating copyright (this is generally considered a requirement for intential protest).

<comment><USER2> SciHub shows the way papers should work on the Internet.

The other day in a HN discussion, someone cited a paper in response to my comment. I was able, in 30 seconds, get to the _full text_ of that paper, which allowed me to reevaluate my opinion in context of what I read.

This is how science can, and _should_ be useful for individuals. And beyond arXiv and SciHub, it generally isn't.

<comment><USER24> I'm an academic and fully making research open access. But when I read discussions like this I always wonder about the people who work at journals. The process of peer-review, copy-editing, and online publication is not something that can be done for free.

Granted, the current journal publishers spend too much money on overhead, and I certainly don't support the for-profit ones like Elsevier that rake in huge profits. But I also don't see much allowance made for the fact that publishing research in a peer-reviewed format involves labor which should continue to be compensated in some way (incidentally, it's not entirely true that researchers themselves aren't compensated - publishing offers an indirect benefit but a real one in the sense that publications are directly linked to salary increases down the line).

<comment><USER25> > But when I read discussions like this I always wonder about the people who > work at journals. The process of peer-review, copy-editing, and online > publication is not something that can be done for free.

Peer review is done by others in the field (peers), typically for free. Online publication is "where do I stick this PDF", and does not require per-paper work (or if it does that can be done by the author). That leaves us with copyediting, and in many cases, copyediting issues are caught by peer review rather than any paid editor.

<comment><USER24> I do peer review myself and am familiar with the model. IMO, when I'm contacted to review someone's work, the journal employee contacting me is performing a legitimate service. They did research to find my name and email and determine that I'm competent to assess the paper. They or another editor have also reviewed the paper in the first place to determine if it's worthy of being peer-reviewed at all, which requires some domain-specific knowledge. And if I pass on peer reviewing, they have to find another person to ask, and so on. Multiplied by dozens of papers, that can be a substantial amount of work.

I also think that you underestimate the importance of copy-editors. It's not the job of a peer reviewer to make sure that the author uses an apostrophe properly, etc. There needs to be a specialist who is dedicated to catching those errors, particularly since, as you note, peer-reviewers aren't directly compensated so it's unfair to burden them with a whole other set of responsibilities.

<comment><USER26> > There needs to be a specialist who is dedicated to catching those errors,

I think "needs" might be too strong here. We (the field of CS) get by without it just fine -- the standard practice is to include copy-editing "nits" at the end of one's review. A shepherd, assigned during the final phase, does a final pass on the paper before approving it.

Yup. Typos slip through. There are papers with poor English. Oh well. For the most part, it works out pretty well.

(And as a reviewer, I have little objection to also noting writing fixes while I'm reading your paper. I'm going to spend anywhere from 30 minutes to 5 hours reading the thing -- the writing fixes are a small additional cost. _If_ you've done a decent job on the writing in the first place. If it's totally botched, I'll reject your paper and tell you to fix it before submitting it again. :-)

Now, would I prefer that the authors of submitted papers had to pay $50 for someone to do a copy-editing pass before I reviewed it? Heck yes. But perhaps we'll get DNNs to fix this for us one of these days. :)

<comment><USER24> That's a fair point. One thing I'm getting from this discussion is that it's difficult to generalize when it comes to academic publishing, because norms vary substantially between fields. I've seen the copy editing "nits" you mention at the end of reviewers' letters in my field (history) but I suspect that historians would rebel if the copy editor's job was entirely foisted off on us.

Likewise, it's normal for editors in my field to make meta-level suggestions about writing style. I'd wager that history journals place a greater emphasis on prose style than CS journals do, since making an historical argument often depends on telling a compelling narrative. Hence a publishing model that works for a CS journal might not work for humanities journals and vice versa.

<comment><USER26> Absolutely. And our papers are a fair bit shorter than yours, for the most part. :). 14 pages, 2 column, 10 or 11pt type is the norm for us, with figures and references included.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER27> So, fundamental question here - if scientific articles (or anything that can be copy protected, etc.) can be released online in this manner to "free the knowledge", and yet, given such free access, there are still people that will pay for a subscription to access the same scientific articles, wouldn't that be the best solution?

I see people commenting that just because of this release, universities won't cancel their subscriptions to the journals. Well, that would be great - <AUTHOR> them keep paying, while the content also gets out for free.

This is like the trend where you can pay what you want for stuff, or nothing. I wonder if that model would apply to scientific research - pay what you want for the paper, or nothing - but if you want to support that research.. hopefully people would still pay.

Just thinking out loud... probably already been thought of or wouldn't work (or I'm just self-defeatist). :)

<comment><USER28> Let me give you an unpopular answer -

\- outside top 500 scientists across all the fields combined no one produces quality papers.

\- the manuscripts received by EICs are garbage. EICs themselves rarely touch anything, or edit anything.

\- "peer review" process in all non-the-most-upper-tier publication is a joke.

\- copy-editing and proofreading is non-existent.

About 15 years ago a lot of journals that are now owned by the major publishers were society publications. This means that societies owned the title, owned the copyright and owned the process. Societies could not find people capable and willing to run a journal for $5k or so a paper including distribution and printing. So the societies went to the likes of Springer, Informa, T&F, Elsevier etc. Those said "Ok, sure. $2k per paper and we produce and distribute it or $5k per paper and we do everything". That was too much for societies. Instead the societies said said "Hey, what if we sold you the rights to the journal? Would you in that case do all of it for free, and let our EIC suck his or her thumb while still being the man/woman on the title page?" "Of course" said the publishers.

And so now we are here. Production costs did not go down. Distribution costs did not go down. In fact, distribution costs increased because before that the journals were printed by a little printers in the US as the volume of most of journals is microscopic ( later printing went to China in the same microscopic quantities ). Now, however, journals also need to be distributed in all kinds of wacky XML formats on all stages of production, which needs to be coded, in most cases by hand.

So unless every Joe, Jack and Jill, the scientist, wants to go back into the publishing world _nothing_ is going to change.

[Source: pillow talk]

<comment><USER29> > outside top 500 scientists across all the fields combined no one produces > quality papers

Just to make sure I read this correctly: Are you saying that, when judging by paper quality, there are only 500 good scientists on Earth?

<comment><USER28> Only about 500 produce readable manuscripts to start. Publishers have their _names_ because it changes schedules _that_ much.

<comment><USER18> That sounds absurd. What are your sources?

<comment><USER28> Pillow talk with someone who ran production for hundreds of STEM journals for two of the publishers

</comment></comment></comment></comment></comment></comment><comment><USER30> I'm all for Sci-Hub disrupting the dominance of RELX Group (a.k.a. Elsevier) and other for-profit publishers that make such a big profit off the backs of researchers (who write and edit for free) and grant-making organizations (who fund those researchers).

But it's unfortunate that Sci-Hub is also disrupting non-profit scholarly associations that cover their own budgets through journal subscriptions. In these cases, the fact that libraries and readers have to pay for access to an article is somewhat balanced out by the fact that those fees are going to pay for staff, conferences, and the other worthwhile activities of the non-profit associations.

</comment><comment><USER31> So is Sci-Hub like Oink[1] for scientific papers?

EDIT: For those not familiar, Oink was a torrenting site but what distinguished it from the tons of other sites was how highly curated it was. High quality audio, proper grouping and genres, and best of all you could request anything that was missing and the community would magically add it.

[1]: [https://en.wikipedia.org/wiki/Oink%27s_Pink_Palace](https://en.wikipedia.org/wiki/Oink%27s_Pink_Palace)

<comment><USER32> Kind of the opposite, sci-hub isn't really curated at all. Its goal is to have a PDF for everything that has a DOI, more or less. And the primary way of retrieving papers is to already know the exact paper you're looking for, found via some other search method like Google Scholar. It's not at all focused on categorizing or browsing papers, just slurping all of them into a big repository.

<comment><USER33> > Kind of the opposite, sci-hub isn't really curated at all. Its goal is to > have a PDF for everything that has a DOI, more or less.

Side-stepping the politics and legality for a moment, I find it interesting how this is a very natural example of the fault-tolerant design of networks.

A 'DOI' is a resource locator. There's a 'fault' in the network in that some people cannot locate the resource when providing the DOI. Sci-hub has sprung up to route around this fault, and provide people with the resource.

Practically, there are real people making decisions who are involved in this process, but on an abstract level it feels like this is somehow inevitable? It feels like an emergent behavior from any sufficiently advanced network, and any attempt to stop it seems futile.

<comment><USER34> I guess another important part is that it's the universities' connection that scihub 'uploaders' tunnel through that are key - they have the special network access to DOI resources that make it free for on-site university users to get anything for free, and then the 'uploaders' relay them to outsiders. That's a difference between papers and something like 'scihub for any pdf ebook via ISBN' \- there's no special network that can get all ebooks for free just via ISBN - except maybe the Library of Congress ;)

<comment><USER35> > I guess another important part is that it's the universities' connection > that scihub 'uploaders' tunnel through that are key

Does this happen in realtime? I read that sci-hub sometimes uses credentials from legitimate users, but in that case they would be easily identified.

So, perhaps someone with access submits papers on request? And if so, what about watermarking?

<comment><USER36> > And if so, what about watermarking?

years ago i recommended to scihub that they should use [https://github.com/kanzure/pdfparanoia](https://github.com/kanzure/pdfparanoia) but they weren't too concerned.

</comment></comment></comment></comment></comment></comment><comment><USER37> The death of for-profit scientific journal companies will be a beautiful thing for the world. It's really rare to see something that is so purely valueless. This industry is sort of unicornlike - they've managed to extract rents in an area where they add _literally_ zero value. It's truly an amazing thing, and it will be even more amazing to watch it die.

</comment><comment><USER38> How does this doom subscription journals? I mean it would be nice but realistically it just means they move to exploit the university subscriptions since the professors can't admit use of illegally obtained copies. They can further exploit the authors since many journals require payment from the author for submission and some journals charge in the hundreds. One might say "just publish to a different journal" but it's not that easy. Because of the heavy reliance on Impact Factor in scientific publishing, it is the journals with those high impact factors that the authors will try to publish to. Regardless of whether or not they are being pirated.

This is sad to say, but in reality I think this isn't going to massively impact things for the publishers. Academia at its core is where the problem lies. Sure paid subscriptions are a big part of things, but it's the stuff most don't realize (the authorship fees and institution sub fees) that give the publishers power.

<comment><USER39> It dooms subscription journals because it breaks their leverage.

Right now, publishers offer take it or leave it bundles of hundreds of journals at prices the publisher set. If a library doesn't want to pay, the library/university system loses access to everything. And many people at the university can't do their work without such access.

SciHub existing allows negotiators for the universities and libraries to shrug and say whatever when Elsevier threatens to cut access to every Elsevier journal.

<comment><USER38> Even if this were the case, then it's still very easy to control the market because of the authorship fees coupled with the drive toward publishing in subscription-model journals (thanks again to Impact Factor). Maybe your advisor took care of some of those costs, but most journals do have a significant fee for submission alone, not to mention the re-submission after a review requires a significant change. Sci-Hub can't change that so it still doesn't doom them.

Not to mention Elsevier, till exempel, WILL be cracking down on piracy especially after their grant from the US courts regarding Sci-Hub. Sci-hub hasn't doomed anything and the publishers still control academia. You should be mad about this, and you should blame academia itself. Push for open journals like PLoS One when possible and try to convince people to move off that god-awful Impact Factor metric and you might inflict damage on the big publishers.

</comment></comment></comment><comment><USER40> I've met the author of the study, Daniel Himmelstein, who is quite passionate about making information free. Projects in his github account ([https://github.com/dhimmel](https://github.com/dhimmel)) tend to use a CC0 license. Some of his work involves aggregation of data (e.g., [https://github.com/dhimmel/hetionet](https://github.com/dhimmel/hetionet)) that is encumbered and he has put a lot of effort into making it as free as possible. His project carefully documents the license for each data point and he took the time to ask copyright holders that do not provide an explicit license to do so.

</comment><comment><USER41> The Noah's Pirate Ark that will save all of humanity's knowledge from unreliable publishers.

</comment><comment><USER42> Looks like Aaron Swartz's vision for the free, collective ownership of mankind's scientific knowledge is well on its way. I wish he were still alive to see Sci-Hub in action.

</comment><comment><USER17> I don't think that sci-hub is going to kill off institutional journal subscriptions in the developed world. It's similar to how developed-country universities didn't stop buying licensed software and start passing around cracked versions to their faculty and students. Journal revenue isn't going to plummet like CD sales after Napster, because it's not individuals doing most of the purchasing in the first place.

Individuals and institutions in poor countries may well turn to sci-hub. I certainly have. But I would venture that not much of the journals' revenue came from individuals or poor institutions in the first place. I didn't pay to read paywalled papers before sci-hub either; I got them via authors' sites or personal contacts, or just didn't get to read them at all.

<comment><USER43> Tend to agree. There might even be an argument that SciHub provides something of a pressure release valve that will allow paywalls to continue. Similar to how pirated Windows copies can be said to have helped solidify Windows' market share. Or even, in my own mind, how pirated Adobe applications helped people learn the tools and go on to work at companies with legitimate licenses.

Elsevier, et al, can now make a legitimate claim that they are not restricting access to those who cannot afford a sub. But their tax on academia will continue.

</comment></comment><comment><USER41> The academic world has missed some decades of advancement of communication. In a world where all published science is open for meta-processing, the burden of validating science would shift to search engines. There would be search engines competing with scientific-SEO of course, but in the medium-run this would improve scientific writing, and possibly speed up science in general. In the end there will always be some private actors doing the work of "ranking" scientists. Academics are hanging on to the current peer-review journals precisely because they don't want to give that power to other actors.

</comment><comment><USER44> Does Sci-Hub actually have all the papers or are they just retrieving them on- demand?

Publishers are tracking mass downloads (see the Aaron Swartz case) so given some of the very obscure papers I've retrieved from Sci-Hub I assume it's unlikely they downloaded them beforehand. My go-to assumption for how it works is that a bunch of people have donated access to their university network access and Sci-Hub is just a load-balancing / cache layer.

<comment><USER17> Yes, it lazy-loads papers using credentials from institutional subscribers and then stores them for fast retrieval. When you're the first person to request a paper you'll see an animation of a biphenyl molecule rotating while the site fetches the paper from an institution with access. Sometimes, there is no corresponding institution with access currently hooked up to sci-hub and the retrieval process fails. But I've only seen that happen a few times.

</comment></comment><comment><USER45> I'll be that guy who will gladly eat some downvotes for this apparently unpopular opinion:

"Science" and "subscription" (or any monetary incentive) _don 't compute in a single sentence_. Aren't scientists funded by governments and/or corporations? Why should anyone pay them a royalty above that?

It's a legit question and not trolling, don't mistake my slightly angry tone for degrading please.

</comment><comment><USER46> Can someone who's familiar with this research paper subscription model that is threatened a la Elsevier explain to me how we got here?

I am curious if at one time did Universities publish these independently and were they more accessible to the public? When did this practice of restricting access to papers via subscriptions begin?

<comment><USER47> This long article about Robert Maxwell does a great job answering your question: [https://www.theguardian.com/science/2017/jun/27/profitable-b...](https://www.theguardian.com/science/2017/jun/27/profitable- business-scientific-publishing-bad-for-science)

<comment><USER46> This is a really fascinating read. Thanks! Cheers.

</comment></comment></comment><comment><USER48> I work for a scholarly publisher and I'd be very interested in hearing about what—aside from cost—would cause you to go to Sci-Hub for a paper?

Is it reading experience? Site performance? Difficulty in navigating publishers' sites?

Are there any good experiences you can point to? I'm really interested in making this better.

</comment><comment><USER49> They should expand into engineering: I don't see any IEEE or ISO standards in there, for instance.

</comment><comment><USER50> Now if we could get the government version of this...

</comment><comment><USER51> Thanks! That reminds me I should donate to Sci-Hub!

</comment><comment><USER52> This will be a catalyst for open-access

<comment><USER53> Yeah, but open access is just 'stuff the cost of publishing into the grant application' and because journals make money for each work published it encourages rubber stamp reviewing and publishing noise.

Open access is probably a net good, but it just shifts the costs around.

<comment><USER54> There's stuff like the fair open access manifesto: [https://fairoa.org/](https://fairoa.org/) .

I suppose that's partly a backlash against predatory open access journals that publish any rubbish as long as (significant) fees are paid..

</comment></comment></comment><comment><USER55> If only. I'm convinced they will find a way to shut her down.

</comment><comment><USER56> I was surprised that it's still considered rude to link to sci-hub: [https://news.ycombinator.com/item?id=14714577#14715252](https://news.ycombinator.com/item?id=14714577#14715252)

Anyone know if this is a typical sentiment? I'm just curious if it's true that many researchers are offended by this movement, and what the reasons are.

I firmly believe that there are always two sides to any topic, so we should explore the flipside. What are some arguments against blatantly opening up access to paywalled articles?

<comment><USER57> further down, the author in question says he supports sci-hub and is happy to see it linked so that more people can read his work. he was just surprised that someone would post a link so blatantly to a pirate site. on the other hand, he says that he knows researchers who would be upset by this but doesn't give their rationale.

<comment><USER56> Yeah, I should've clarified that I was curious about the people he mentioned. Researchers contribute some of the most valuable comments to HN, so mainly I wanted to avoid making them feel unwelcome. If linking to sci-hub ends up shooing them off HN, it's not worth the cost.

It seems like the sentiment is in the other direction, though. Maybe most researchers don't care.

</comment></comment></comment><comment><USER58> It's not just the publishing industry that is the problem. _It is merely a symptom of the greater malaise in higher education as a whole_.

The focus is on degrees, not on true learning. So much of what occurs is in universities is total waste. But people put up with it to get the paper. As long as people keep blindly giving absurd sums of money to get the paper, these expensive publications will last. The answer is for people to wake up and value learning over a diploma. When that happens, then finally issues like this will go away. Heck, as a bunch of people have pointed out, many of these papers aren't even for real learning. They are worded in such a way as to make them sound smart to their peers, but unintelligible to the public.

<comment><USER59> Yes and it's no good if who you referring to is smart enough to read between the lines

</comment></comment><comment><USER60> and it's better than all of their websites!

</comment><comment><USER61> Anybody mirrored (or attempted to do so) libgen ?</comment>


Steve Ballmer AMA - <AUTHOR> http://www.reddit.com/r/IAmA/comments/2m7cmt/were_steve_ballmer_harvard_college_class_of_77/?utm_source=twitter&utm_medium=social&utm_campaign=hu-twitter-general

<comment><USER2> It's pretty much the worst AMA I've seen. A lot of circular talking and non- responses, just shows that he's a company man till the end.</comment>


Ask HN: How did delivery of drugs on Silk Road work? - <AUTHOR>

Were the people selling/buying drugs on Silk Road top-level drug merchants?  Or were they suburban kids?<p>I understand that the service kept everyone anonymous, but there's still the problem of delivery.  How did the actual delivery of drugs work?  And doesn't the government have ways to detect drugs in the mail? <comment><USER2> _> Were the people selling/buying drugs on Silk Road top-level drug merchants?_

This is unknown, but my assumption after looking at transaction records is yes.

 _> Or were they suburban kids?_

I'm not even sure what "suburban kids" means. I think there's a wider demographic of low end product buyers, than what you describe, but yes.

 _> How did the actual delivery of drugs work?_

From public sources, it appears the dealer would typically use prepaid shipping and drop it into a public postal box for delivery.

 _> And doesn't the government have ways to detect drugs in the mail?_

Yes, and it seems like criminals are always one step ahead. I know that this summer, just prior to DPR's arrest that dealers were using some scheme that involved multiple layers of sealed mylar to counter the gov't's detection techniques. Sometimes these bags were stashed inside a figurine or some other object, and sometimes they were slipped inside the pages of a magazine.

</comment><comment><USER3> Throwaway time!

Though I myself have never purchased anything from Silk Road, I knew a group of (rich, suburban) kids who purchased drugs (and other stuff, like fake IDs) off of Silk Road and Black Market Reloaded, another illegal Tor marketplace. From what I understand shipments were hidden, and very hard to detect. For example, if a number of pills we're being shipped, they'd be sowed into the lining of a purse, or hidden in the battery compartment of a cheap calculator. That item would then be shipped like a normal item, usually through USPS.

Though the government has certain ways to detect certain drugs in the mail (cocaine and pot are easy to detect, where as LSD or Class Two pills are almost impossible), most of its efforts are focused on international mail and keeping drugs from being smuggled into the country. As domestic detection was almost unheard of (unless the police were specifically tipped off) most people would only purchase from sellers in the US.

<comment><USER3> People selling were mostly top level merchants, people buying were mostly younger people with a lot of money but not enough connections/street cred to buy in person

</comment></comment><comment><USER4> It is my understanding (never smoked myself nor owned any ficus that lived for more than a week) that growing quality plants is actually a very complicated process which takes up a lot of space, requires adequate lighting which in itself is hard to conceil, spacy, as well consuming a lot of power. I doubt suburban kids could be that deep into it.

I mean, just look at what some people are doing in terms of selfbrewed beers and stuff. That is hightech meticulously planned do it yourself machinery and botany.

<comment><USER3> 1) (from what I've heard) Yes, you're correct, growing pot is very difficult

2) you'd be supposed by the dedication of some suburban stoner kids who don't have a care in the world for anything else, but marticulously tend to their pot plants.

</comment></comment><comment><USER5> Actually, everyone was responsable for their own shipment. It was just through mail and they even caught one guy who had written his own address on the reply-package ;-)

It was the anonimity on the web that made it a big deal, not the shipment. That was easy :)</comment>