Ask HN: How do you prevent slouching at your desk?  - <AUTHOR>

I have a bit of a problem with slouching. I often catch myself leaning closer and closer towards the computer screen, especially when I'm on my laptop, and this does quite a number on my back and shoulders.<p>How can I maintain good posture when I'm working on my laptop or desktop, without setting up a ridiculous treadmill desk? Thanks guys. <comment><USER2> I've had a couple herniated discs in my lower back for over eight years now, so I've spent significant time focusing on improving my posture while sitting.

Here are a few tips:

1\. Use a lumbar roll -- you place this in your lower back and it forces you to maintain a lordosis (curve of the spine) by keeping it in position. This will be tiring for the first couple of weeks, but eventually it will feel natural.

Here is a link to a lumbar roll similar to what I've used: [http://www.amazon.com/Original-McKenzie-D- Section-700-Lumbar...](http://www.amazon.com/Original-McKenzie-D- Section-700-Lumbar/dp/B000GPM520/)

2\. Monitor height -- you want your monitor to be positioned so that you are looking straight ahead, rather than down, towards your monitor. If you are forced to look down there is a better chance of leaning forward instead of maintaing good posture.

If you're using a laptop, consider buying a laptop stand to increase the height of the laptop. [http://www.amazon.com/AliMed-Smart-Stand- Laptop/dp/B003T1LWW...](http://www.amazon.com/AliMed-Smart-Stand- Laptop/dp/B003T1LWW2/)

If you can't get your monitor high enough relative to your position, you'll need a taller desk. \</comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>-

Those are my two biggest tips, but there are others:

Chairs are not made equal... different chairs have definite discrepancies in the amount of pain they cause me on a day to day basis. Even expensive ones will send shooting pains down my leg within a few hours of sitting in them. If you can test some new chairs out for a day or two at a time, I'd recommend doing so.

Get up, walk, and/or stretch regularly for a couple of minutes. At work I use a small cup for water, which requires me to refill every 45 minutes or so. A quick 2 minute walk to the water tank for a fill-up is a refreshing break for our backs.

<comment><USER3> The drink trick is perfect, but beware you will be going to the bathroom often. Also be aware that you may develop a habit of sipping on your cup very often and that can lead to bad habits while drinking alcohol.

<comment><USER4> I've used the glass of water trick myself, and it is very helpful. It's especially helpful because when you get stuck while solving a problem, you have a great reason to get up and walk around to think about said problem.

Just another data point: This hasn't affected my drinking habits in other areas.

</comment></comment></comment><comment><USER5> You probably have too-small a screen or too small letters on your screen, though for what it's worth there really i no ergonomic way to use a laptop that doesn't involve at least one of an external keyboard or screen. Increase your font sizes, and you may not feel the need to lean in so much. (And/or see an eye doc.)

And working at home (or in a startup you run, I guess) means you can do things to help your health and productivity even if they might seem ridiculous, like take naps and walk while you type. Yet another reason I don't see myself ever having a 'normal' job again. (And most people seem to think my treadmill desk is pretty cool, fwiw.)

<comment><USER6> This is fine advice, but you're burying the lede:

 _There really i no ergonomic way to use a laptop that doesn't involve at least one of an external keyboard or screen._

Amen. Laptops have lousy ergonomics. Get an external keyboard, and/or a portable external keyboard, and/or find some way to prop up the screen, and/or get an external monitor.

<comment><USER7> I completely agree. Make sure that the top of the screen is at about the same height has your eyes when you're sitting upright. That's the best ergonomic height for a screen and will prevent you from slouching.

</comment></comment></comment><comment><USER8> (At the risk of sounding like a broken record...)

(At the further risk of alienating everyone younger than me; I'm only barely old enough to remember records, and never actually owned any...)

Get a standing desk. No slouch. Good stuff. Treadmill gizmo is completely unnecessary.

<comment><AUTHOR> I feel like I would absolutely hate standing for any extended period of time. Doesn't this tire you out?

<comment><USER9> I'm really not trying to be mean, but imagine your ancestors for a second. You know, the guys who ran down and killed animals that weighed several tons. Imagine how befuddled they'd be, sweating and bloodied from defeating the mastodon that just tried to kill them, when you told them that you can't be bothered to stand up for a little while while you type. It says something pretty embarrassing about society when we've collectively decided that myriad orthopedic problems are better than standing up.

That rant over with (sorry, again), I just went to the local hardware store and got some cinder blocks. The price difference (~$7 for the cinder blocks versus ~$1000 for a height-adjustable desk frame) was worth the slight lack of convenience. That, and without the adjustability, I won't be tempted to just leave the desk in the 'down' position.

<comment><USER10> There's a big difference between walking (or jogging) for hours and standing in one spot for hours. I can walk for significantly longer than I can stand in the same spot before I experience discomfort.

<comment><USER9> The reason that I don't experience that might have something to do with my latent hyperactive tendencies. With my standing desk, I'm able to pace around the office, walk to the whiteboard, and then return to the keyboard.

I think that helps me focus more, and it's probably one reason that I don't get sore from standing in one spot. Others may experience different results.

</comment></comment></comment></comment></comment><comment><USER11> Holding an upright posture requires exercise. It _IS_ exercise, actually, which is why it is hard to maintain for a while. So for that aspect of the problem, one part of the solution == training, strengthening back muscles, yoga, meditation, etc. By this, I mean as a extra-curricular activity.

As for posture itself, there is no simple answer. The way you sit, the placement of your peripherals, how much time you spend sitting, these are all factors. In order, then, and in brief:

Rolling your hips forward pushes the spine up. This is fundamental, literally. Try a sitting position where your legs decline from your butt at a downward angle. Sitting "indian style" on a decline is a good place to start (or the "beaver" stretch position, with flat feet together. Obviously, your average chair is not conducive to this. ALternatively, you could sit on a pillow on the floor. You can sit with the lower half of your ass off a flat chair, with legs tucked under, but it is not the best for circulation.

Your workspace is all important. If you're a pro programmer, it's worth every effort to make your workshop (or command center, if you prefer) as close to some ideal as possible. For instance, putting your monitor above the parallel plane of vision is probably a big deal. Parallel plane = straight ahead. Ergo(nomically): you should look upward to your monitor. This will encourage your posture.

Again, since maintaining the position is actual work itself, it follows that your should take regular breaks, stand up, walk away (or pace), return after a few minutes, and get back to work. This also helps circulate blood. A programmer I work with espouses the Pomodoro Technique, which is a time- on/time-off regimen for deep cycle work flow. The effect is you set a timer, and follow basic rules without fail. It's also good for your eyes to to turn away from the screen every 45 minutes.

Bonus: breathing is life. Breathe deep for a deeper life. Regulate breath, keep yourself extra-oxygenated. This is good for awareness and staves off fatigue, which are critical for not recessing down into a slouch.

<comment><USER12> > putting your monitor above the parallel plane of vision is probably a big > deal. Parallel plane = straight ahead. Ergo(nomically): you should look > upward to your monitor. This will encourage your posture

All the ergonomics guidelines I've seen recommend looking slightly downward at your monitor. See the main image here:

<http://en.wikipedia.org/wiki/Ergonomics>

> Breathe deep for a deeper life.

What the hell does that mean?

<comment><USER13> I think there are conflicting goals regarding monitor height. Higher helps your posture, but lower keeps you eyes from drying out and straining because they are not as wide-open.

</comment></comment></comment><comment><USER14> The only thing that has improved my posture (other than concentration and awareness, which are usually engaged with other things) is exercise. For me, running helps a little, but the best by far is lifting weights. After I lift weights, my posture takes care of itself for the next 12-24 hours. Lifting weights also helps me learn what a strong, healthy posture feels like, so when I consciously try to stand up straight, it's easier to achieve a relaxed, natural-feeling position instead of straining and feeling unsure.

I expect this advice only applies to exercises done on your feet with free weights, such as squatting and overhead pressing, not to bench pressing or machine work.

</comment><comment><USER15> I've found the absolute best thing you can do is to switch positions frequently. I've got terrible posture - I'm frequently curled up & twisted in my chair - and yet have never had RSI or back problems, probably because I don't hold that posture for long. At work, I usually migrate between my desk, the futon, a beanbag, and the massage chair. At home, I switch between couch, floor, bed, and balcony, with frequent breaks to get up and pace.

When I don't get up and switch, I frequently _do_ have problems, like I threw out my shoulder leaning on the couch while reading the NaNoWriMo forums last night. Vegging out can be hazardous to your health...

</comment><comment><USER16> I found that using a Kinesis freestyle keyboard improved my sitting posture a lot. The keyboard is split in half and connected by a wire and I have them set up about shoulder width apart. With a normal keyboard I pull my shoulders inward which causes me to slouch. Now I can lean back while typing.

<comment><USER17> Consider one of these. They make a huge difference for me.

[http://www.kinesis-ergo.com/images/solo- ascent-90_512x390.jp...](http://www.kinesis-ergo.com/images/solo- ascent-90_512x390.jpg)

(My cardboard/duct tape clone cost $4, about 50x less than the official version.)

<comment><USER18> I am confused as to what you are suggesting to get or how it works based on the picture.

<comment><USER16> I have the keyboard in the picture (the Freestyle) but I do not have the "tenting" accessory that is holding the keyboard in a vertical position. He was suggesting I get/build that.

I have the VIP accessory which lifts each half 15 degrees, but that's still pretty far from 90.

</comment></comment></comment></comment><comment><USER19> I'm a pretty tense (high-strung) guy when it comes to being in one place for a long time. To help with the slouching, and also with the Restless Leg Syndrome (<http://www.rls.org> ...yes, we have our own damn foundation!), I have to use a chair which feels optimally comfortable when sitting back and IN the chair (with each part of your back and posterior being supported by the chair). There's the initial investment of testing several chairs to find the one which fits your shape and comfort, but pressing myself into a chair which is comfortable to sit in and supports how I sit was 90% of the battle.

</comment><comment><USER20> I keep my contrast up -- my screen has a _horrible_ problem with the contrast and subpixel organization. It just makes it harder and harder to read over time.

I've also change my interface fonts... My monospace font has gone to a 12 point Monofur, which makes a very readable font imo, and for my normal font I've moved to a 14 point Verdana. I have also found putting my chair higher made a HUGE difference.

high contrast white-on-black and green-on-black works better for me than other color schemes, so that's one big thing.

</comment><comment><USER21> I boosted my desk and now stand up. When I feel like sitting down, I have a stool. Having no back to your seating will improve your posture if you consciously think about sitting up. I was on the crew team in college and working out my core muscles every day did more to improve my posture than anything I've ever done.

</comment><comment><USER22> This week I have been house/cat sitting for a friend. While reading on their computer my posture would begin to hunch, followed rapidly by a cat leaping onto the area behind my neck.

Sitting up straight solved the cat issue. Maybe a cat will solve your slouch issue.

</comment><comment><USER23> Rethink the treadmill desk. Bodies are not designed for sitting all day. Building a treadmill desk was probably the best single thing I've done for both my health and productivity. Either that or take half-hour breaks to do vigorous exercise throughout the day.

</comment><comment><USER24> I lean back, put my feet up on my desk or the computer underneath, put my keyboard on my lap, and increase my font size till I don't have to strain to read it. :)

</comment><comment><USER25> You could try this:

<http://www.nadachair.com/>

It feels good for my back, but everybody is different.

<comment><USER26> This looks like a good support ("an extra muscle"), but it's important to train your own muscles.

Rowing is great for strengthening the core and for stretching the back. I "row with Xeno" <http://www.row2go.us/>

"Belly button to the spine, chest up, shoulders down."

</comment></comment><comment><USER27> "Treat Your Own Back" by Robin McKenzie was recommended to me by a physiotherapist and my grandfather. General advice to consult your own GP or physiotherapist applies.

[http://www.amazon.com/Treat-Your-Back-Robin- McKenzie/dp/0958...](http://www.amazon.com/Treat-Your-Back-Robin- McKenzie/dp/0958269238/ref=sr_1_1?ie=UTF8&qid=1286825545&sr=8-1)

</comment><comment><USER28> I agree with rjett about not having anything to lean on. It really does improve your posture. I sit on a balance ball at my office, and have consistently for about 10 months. It's really hard to slouch on a balance ball. It's improved my overall core strength considerably, and I don't think I ever slouch.

</comment><comment><USER29> While we're on the topic, anyone have tips for having a good posture, but packing your setup in a bag? I want to work remotely but not kill my back in the process.

So far the best suggestions seem to be to use a nada-chair and a laptop stand + light external keyboard.

</comment><comment><USER30> What I just discovered recently that has made all the difference is: Keep your feet flat on the floor. It makes it much easier to sit up straight; or, perhaps, sticking your feet out in front of you or tucking them under your chair makes it much easier to slouch.

</comment><comment><USER31> Perhaps you need glasses? I've read recently that slouching is actually easier on the back than sitting straight, however, I think that was about slouching away from the screen. I tend to do that - of course a lot of employers are not very thrilled about it.

<comment><USER32> "I've read recently that slouching is actually easier on the back than sitting straight"

Any chance you might have some reference to where you read that? I'd be interested in seeing how they did the study and what sort of results they came up with.

<comment><USER33> <http://news.bbc.co.uk/1/hi/6187080.stm>

Not a link to the source but a decent article surmising the findings.

<comment><USER34> I do sit like this on an aeron chair, leaned back as far as it can go. I almost never have back issues, but in this position (exactly like the far right picture in the article) my neck eventually starts to ache.

Good for back, bad for neck...

<comment><USER35> You should check out the freedom chair with headrest. I have 3 and love them for exactly this reason, though now that my daughter has taken over my home office, I never get to actually use them...

[http://www.amazon.com/HumanScale-Freedom-Chair-Headrest- Fabr...](http://www.amazon.com/HumanScale-Freedom-Chair-Headrest- Fabric/dp/B00062X3II)

</comment></comment></comment></comment></comment><comment><USER36> Put something moderately sharp (but rather blunt still) on the edge of your desk? So leaning on it is uncomfortable but not damaging to your clothes?

Also, tell your optometrist about this. It really sounds like something is wrong with your vision.

<comment><USER37> > Put something moderately sharp (but rather blunt still) on the edge of your > desk

Like a Macbook maybe?

I've never understood how people work around those sharp edges for extended periods of time. Do your palms build some kind of protective layer after a while?

</comment></comment><comment><USER38> Years ago I tried a kneeling chair (<http://en.wikipedia.org/wiki/Kneeling_chair>). It was quite pleasant, although I've never spent an extended period of time with one.

<comment><USER39> We have many customers at www.backcarebasics.com that really swear by their kneeling chairs. And, we have a whitepaper that you can access from our Home Page that gives you a great deal of information and other written references. Check it out!

</comment></comment><comment><USER29> I use a swopper chair at home, and at work I stick a bunch of reams of paper under my monitors to make sure they stay up.

Next on my home list is to buy either a laptop stand or get a good monitor+stand.

</comment><comment><USER40> I would recommend a standing desk. The Ikea utby (105 cm) is great. <http://www.ikea.com/us/en/catalog/products/S49843462>

</comment><comment><USER41> I strongly recommend leaning back, with the legs propped up. I don't even know what back pain is anymore, and I only realize how comfortable it is when I sit at someone else's desk.

</comment><comment><USER42> Anyone had any good experiences with those saddle and/or kneeling chairs? I find it odd that I get less backpain riding a motorcycle all day than sitting at a desk ...

</comment><comment><USER43> Use a Swiss ball for a desk chair.

Squat, deadlift.

<comment><USER44> "It is concluded that the advantages with respect to physical loading of sitting on an exercise ball may not outweigh the disadvantages," the researchers wrote." [http://lifehacker.com/5645943/exercise-balls-might-not- be-th...](http://lifehacker.com/5645943/exercise-balls-might-not-be-the-best- posture-fixers)

<comment><USER45> Eeeewww, "spinal shrinkage"!

</comment></comment></comment><comment><USER46> Lean back in your chair.</comment>


Show HN: StatusNotify monitors status of 250+ cloud services - <AUTHOR> https://statusnotify.com/ <comment><USER2> Does this just scrape the AWS status page, or is it getting something that's indicative of failure sooner?</comment>


APT, the app store for geeks (why the iPad is not our only salvation) - <AUTHOR> http://blog.blinker.net/2010/02/03/apt-the-app-store-for-geeks/

<comment><USER2> Since when is APT on the same level at all with the App Store? You're definitely not going to be finding the same quality in the utilities and applications you find in your distributions APT repository as you will on the App Store, simply because developers are unable to charge for them. I hope this article is a joke or something.

<comment><USER3> So you argument is that because say... the git devs can't charge for git through apt that it's not as good as say... BitKeeper?

<comment><USER2> Sorry, I realized after I posted that that I was hugely inaccurate in my wording. I meant to say that in my experience, "paid" software was much higher quality in the design and usability area than their open source counterparts. There are certainly exceptions, though.</comment></comment></comment>


Gay community not gay enough to win new gTLD - <AUTHOR> http://domainincite.com/17484-gay-community-not-gay-enough-to-win-new-gtld

<comment><USER2> Link to the ruling [https://www.icann.org/sites/default/files/tlds/gay/gay- cpe-1...](https://www.icann.org/sites/default/files/tlds/gay/gay- cpe-1-1713-23699-en.pdf)</comment>


Show HN: Mongodb diagnostics - <AUTHOR> Dear nice and great people on HN,
I'm writing to you to get some feedback on my work. I'm a retired software professional trying to take advantage of my new-found freedom to build things as I wish, with no business pressure.
To start with, I built a mongodb diagnostics service - worktheme.com - to provide all the visibility that can possibly be extracted from the logs.
In future, I would like turn this into a generic component in a metric-processing system with emphasis on simple buildable views.
Please have a look and provide your valuable feedback. How would you have done it if you were to build this? What should I do next?
Thanks! <comment><USER2> Clickable link [https://worktheme.com/#/welcome](https://worktheme.com/#/welcome)</comment>


New to Cambridge, MA. Any meet ups? - <AUTHOR>

Hi all, I just moved to Cambridge area 2 months ago for a new job. I enjoy making my own apps in the free time. I did couple side projects in web and mobile area before and I want to start my hobby again! Wondering if there are any good places and events to meet people with similar interest and make some friends! <comment><USER2> Check out the Boston User Group Calendar at <http://bugc.org> and also ye olde meetup.com (which has groups that don't publish to BUGC).

<comment><AUTHOR> Thanks! Looks like there are things going on everyday. I will def attend some of those events.</comment></comment>


Motorola Xoom will ship without Flash support - <AUTHOR> http://www.engadget.com/2011/02/21/motorola-xoom-will-ship-without-flash-support-on-february-24th/

<comment><USER2> Flash is a technology that had it's best days in PC era.

Probably noone can imagine YouTube or banner ads without Flash technology nowdays, but new devices are coming to market where Flash isn't best choice anymore. Just as floppy disks served world in it's days but are now replaced with USB keys or broadband connections and clouds storages so it looks like Flash had it's days and will be replaced by other solutons (like H.264 codec).

Putting Flash on Android tablets seems like they want to compete "has flash" VS. iPad's "does not have Flash", but that doesn't really serve to users.

It goes the way nature has invented long time ago and it's called evolution - where old dies (even very best of it's time) away and new takes over.

<comment><USER3> Usually you replace an encumbent technology with a better version rather than taking ten steps back. Evolution is about progress, not regression.

The truth is, Adobe's engineering incompetence aside, Flash is a lot, lot better to make programs with than HTML5. Silverlight is a lot, lot better to make programs with than HTML5. Java, C#, Python, Ruby, C++ are all, as a whole, a lot, lot better programming languages than Javascript.

Imagine replacing DVDs with 8 tracks. That's what's happening. It's a perverse scenario, but I can't understand how you can say that it's evolution, evolution is about progress, not regression.

It's Adobe's own fault for neglecting the Mac and Linux for so long, crashing browsers, etc., added with the general menace that PDFs are too and Silverlight was just too late, but let's not pretend the technology's better. It's pretty pathetic and the snail's pace progress of html5 is embarrassing.

And where's ECMAScript 5? What the hell has happened to that, it seems to have dropped off the planet!

<comment><USER4> > The truth is, Adobe's engineering incompetence aside, Flash is a lot, lot > better to make programs with than HTML5. Silverlight is a lot, lot better to > make programs with than HTML5. Java, C#, Python, Ruby, C++ are all, as a > whole, a lot, lot better programming languages than Javascript.

Your comment isn't very coherent.

\-- Other than the fact that "better" is entirely subjective, how many modern web applications in widespread use are based on Flash rather than JS+HTML5? Gmail, facebook, twitter, google docs: no flash. Youtube: flash only for the video codec, not functionality.

\-- "ActionScript is a dialect of ECMAScript": <http://en.wikipedia.org/wiki/ActionScript> So what language features were you thinking of that make ActionScript/Flash better than Javascript?

\-- Security holes in Flash vs. HTML5? hmm...

\-- HTML5 on 64bit Linux? Check. Flash? still buggy.

</comment></comment></comment><comment><USER5> The Motorola Milestone advertised it was "Flash ready" on the packaging. That was in mid 2009. I bought the phone and have been waiting on Motorola to deliver the necessary upgrades ever since. Their release date for Android 2.2 Froyo for the Milestone changed, what, ten times - from mid 2010 to June to August to September to Q4 to end of Q4 to early 2011 to sometime 2011… in October a 2.2.1 test ROM from Motorola leaked (the "G.O.T." ROM), and they still haven't managed to put out a release.

I don't trust Motorola on anything anymore. Their hardware was pretty good, but their service and communication with their customers is abysmal.

</comment><comment><USER6> So what is the competitive advantage that the Motorla Xoom now has?

And what does it have to justify it's $100 price premium over the iPad?

<comment><USER7> It has a brand-new UI that is tablet-optimized. I think the principle advantage of any Android tablet over iPad is Honeycomb; with widgets and real multitasking, it's much much more usable as a real computer than the iPad is.

I think it also has two cameras, a significantly faster CPU, other features like that. Really though, the main thing it has going for it is Honeycomb. Most people will be very impressed with it after playing with a demo model in a store. It's much more futuristic and cooler than the now-old iOS icon-grid paradigm.

<comment><USER8> You lost me at "real computer." In my own experience, one of the biggest reasons that normal customers like the iPad is _because_ it's not like their desktop computer.

The obsession with emulating a desktop/laptop experience is mainly the domain of those of us who are already experts in that experience; for average users who may generally dislike the "real computer", it's not a selling point.

<comment><USER9> I think he meant that the Xoom is capable of _replacing_ a desktop or laptop; not that it _emulates_ them.

</comment></comment></comment></comment><comment><USER10> I go out of my way to avoid using Flash on my Android phone.

Steve Jobs and others are right: Flash apps are not meant for touch screens, or mobile devices in general. I miss it only a little when I'm using iOS.

</comment><comment><USER5> Well, it seems like Spring 2011 is not actually Spring 2011. Here's Adobe's blog post on the matter: [http://blogs.adobe.com/flashplatform/2011/02/update- for-fp-1...](http://blogs.adobe.com/flashplatform/2011/02/update-for- fp-10-2-on-tabs.html)

Money quote: _Adobe will offer Flash Player 10.2 pre-installed on some tablets and as an OTA download on others within a few weeks of Android 3 (Honeycomb) devices becoming available, the first of which is expected to be the Motorola Xoom._

</comment><comment><USER11> Will the browser have full HTML5 support? Can I go to YouTube in it and watch videos without flash? I admittedly don't know much about the Android 3.0 browser.

<comment><USER7> Android's browser has supported much of HTML5 for years now. Nobody can claim "full" support since HTML5 is a very vague term with a currently changing set of technologies included. I'm not even sure if it's still called HTML5 actually.

Edit: About Youtube: Yes. You can go there and watch the videos just fine. I think you're (transparently) redirected to the YouTube app.

<comment><USER11> Yeah, but I don't want to _have_ to be moved to the YouTube app. I want to know if I can use the built in browser to watch HTML5 videos. I know I can do it on YouTube with my PC browser if I enable it on my YT account...

</comment></comment></comment><comment><USER12> Why don't they adopt click-to-play trick to get the best from both worlds?

<comment><USER13> Most likely because the browser on the device will be positioned as _the real "full web"_ when it's upgraded to the 10.2 release. A useful feature that runs counter to a marketing strategy will always get cut.

This is part of the reason Flash was blacklisted from iOS: features have to wait on Adobe's schedule to be enabled, instead of being available out-of-box.

<comment><USER14> Or they could have just done what Android did. Allow browser plugins and make it Adobe's problem to put the plugin in the market.

Google isn't holding back on major OS updates because Adobe isn't ready.

<comment><USER15> From what I've read, and experienced with Flash on the OS X desktop, the issue was caused by Flash beating up hard on the CPU (and thereby causing a battery drain). The last thing Apple wants is to list a battery life in the specs, and then watch some Flash based game decimate it.

</comment></comment></comment></comment><comment><USER16> Adobe has been in a serious funk since they took over Macromedia. We shouldn't be surprised competition breeds better products.

<comment><USER17> Adobe should never have absorbed Macromedia. Adobe's mainstays (Photoshop, Illustrator) have taken a terrible turn for the worse (slower, buggier, non- native UIs), while getting more expensive at the same time.

Adobe's software used to be indespensable to me, but now there's Pixelmator, which carries the true (old) Adobe legacy. Does anyone know if Pixelmator is going to make a vector tool? If they do, lots of designers will be completely Adobe-free!

<comment><USER16> Yep. They've had no competition forcing them to do better. At least with Flash they now feel the heat from native H.264 playback and JavaScript/CSS animations.

<comment><USER17> I voted you up, but I'm not quite sure that's the whole story.

I think Adobe absorbed a bad corporate culture when they acquired Macromedia, and that better accounts for why they're making such bad products now. On top of this, Adobe will never be a successful platform vendor because they suck at making platforms, and the market has too many platforms already.

Will competition somehow make Adobe good again? It might, but it's by no means certain. The long-term alternative for them is failure in the marketplace.

Signs of things turning around for Adobe will be when they: > Make HTML5 the primary target for all Adobe software (make authoring for Flash not the default). > Dump Air. > Make the new Acrobat a slim wrapper around HTML5, facilitating a transition away from Acrobat toward standards. > Stop trying to be cross-platform. Use native UI in all software. Dump the installer on platforms where installers aren't standard (Mac OS X). > make completely new creative applications for iOS. An iOS-based desktop computer is probably coming. If Adobe was smart, it would try to make great native tools for creative professionals again. > Fire the executives they picked up from Macromedia and lay off legacy staff.

</comment></comment></comment></comment><comment><USER18> Like.</comment>


Ask HN: Best practices for ethical web scraping? - <AUTHOR> Hello HN!
As part of my learning in data science, I need/want to gather data. One relatively easy way to do that is web scraping.
However I'd like to do that in a respectful way. Here are three things I can think of:
1. Identify my bot with a user agent/info URL, and provide a way to contact me 2. Don't DoS websites with tons of request. 3. Respect the robots.txt
What else would be considered good practice when it comes to web scraping? <comment><USER2> When scraping just behave as to not piss off the site owner - whatever that means. Eg. not cause excessive load or making sure you don't leak out sensitive data.

Next put yourself in their shoes and realize they don't usually monitor their traffic that much or simply don't care as long as you don't slow down their site. It's usually only certain big sites with heavy bot traffic such as linkedin or sneaker shoe sites which implement bot protections. Most others don't care.

Some websites are created almost as if they want to be scraped. The json api used by frontend is ridiculously clean and accessible. Perhaps they benefit when people see their results and invest in their stock. You never fully know if the site wants to be scraped or not.

The reality of scraping industry related to your question is this

1\. scraping companies generally don't use real user agent such as 'my friendly data science bot' but they hide behind a set of fake ones and/or route the traffic through a proxy network. You don't want to get banned so stupidly easily by revealing user agent when you know your competitors don't reveal theirs.

2\. This one is obvious. The general rule is to scrape over long time period continuously and add large delays between requests of at least 1 second. If you go below 1 second be careful.

3\. robots.txt is controversial and doesn't serve its original purpose. It should be renamed to google_instructions.txt because site owners use it to guide googlebot to navigate their site. It is generally ignored by the industry again because you know your competitors ignore it.

Just remember the rule of 'not to piss off the site owner' and then just go ahead and scrape. Also keep in mind that you are in a free country and we don't discriminate here whether it is of racial or gender reasons or whether you are a biological or mechanical website visitor.

I simply described the reality of data science industry around scraping after several years of being in it. Note that this will probably not be liked by HN audience as they are mostly website devs and site owners.

<comment><USER3> 1\. is the only one I don't like. I think you should use your real user agent first on any given site, as a courtesy; whether you give up or change to a more "normal" user agent if you get banned is up to you.

Oh, and for 3.: if you can, apply some heuristics to your reading of the robots.txt. If it's just "deny everything", then ignore it, but you really don't want to be responsible for crawling all of the GET /delete/:id pages of a badly-designed site… (those should definitely be POST, and authenticated, by the way).

<comment><USER4> I disagree. The risks are similar to those of disclosing a security vulnerability to a company without a bug bounty. You cannot know how litigious or technically illiterate the company will be. What if they decide you're "hacking" them and call the FBI with the helpful information you included in your user agent? Crazier things have happened.

Anonymity is part of the right to privacy; IMO, such a right should extend to bots as well. There should be no shame in anonymously accessing a website, whether via automated means or otherwise.

<comment><USER5> > such a right should extend to bots as well

No, it very much shouldn't, but (as you probably meant) it _should_ extend to the _person_ (not, eg, company) _using_ a bot, which amounts to the same thing in this case.

</comment></comment></comment></comment><comment><USER6> It won't help you learn to write a scraper, but using the common crawl dataset will get you access to a crazy amount of data without paying to acquire it yourself.

[https://commoncrawl.org/the-data/](https://commoncrawl.org/the-data/)

<comment><AUTHOR> Cool, didn't know about this. Thanks!

<comment><USER7> > As part of my learning in data science, I need/want to gather data.

Also not web scraping, but a few other public data set sources to check.

[https://registry.opendata.aws](https://registry.opendata.aws)

[https://github.com/awesomedata/awesome-public- datasets](https://github.com/awesomedata/awesome-public-datasets)

<comment><AUTHOR> Thanks!

<comment><USER8> also [https://www.reddit.com/r/datasets](https://www.reddit.com/r/datasets)

<comment><AUTHOR> The comments that keep on giving :D

<comment><USER9> I appreciate you starting this thread, these are great resources people are posting.

Common Crawl is _the_ data set to master if someone wants to use the fruits of web scraping without actually doing the web scraping.

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> Nice you to ask this question and think about how to be as considerate as you can.

Some other thoughts:

\- Find the most minimal, least expensive (for you and them both) way to get the data you're looking for. Sometimes you can iterate through search results pages and get all you need from there in bulk, rather than iterating through detail pages one at at a time.

\- Even if they don't have an official/documented API, they may very likely have internal JSON routes, or RSS feeds that you can consume directly, which may be easier for them to accommodate.

\- Pay attention to response times. If you get your results back in 50ms, it probably was trivially easy for them and you can request a bunch without troubling them too much. On the other hand, if responses are taking 5s to come back, then be gentle. If you are using internal undocumented APIs you may find that you get faster/cheaper cached results if you stick to the same sets of parameters as the site is using on its own (e.g., when the site's front end makes AJAX calls)

<comment><AUTHOR> That's great advice! Especially the one about response times. I didn't think of that, and will integrate it in my sleep timer :)

</comment></comment><comment><USER11> I always add an "Accept-Encoding" header to my request to indicate I will accept a gzip response (or deflate if available). Your http library (in whatever language your bot is in) probably supports this with a near trivial amount of additional code, if any. Meanwhile you are saving the target site some bandwidth.

Look into If-Modified-Since and If-None-Match/Etag headers as well if you are querying resources that support those headers (RSS feeds, for example, commonly support these, and static resources). They prevent the target site from having to send anything other than a 304, saving bandwidth and possibly compute.

<comment><USER12> > Meanwhile you are saving the target site some bandwidth.

And costing them some CPU :) It's probably a good idea in most cases, agreed, but there are exceptions such as if you are requesting resources in already- compressed formats, like most image/video codecs.

<comment><USER13> Frankly, it would be difficult to find a part of your post that is correct.

1\. You're never causing their server to do anything they didn't configure their server to do. Accept headers are merely information for the server telling them what you can accept: what they return to you is their choice, and they can weigh the tradeoffs themselves.

2\. The tradeoff you think is happening isn't even happening in a lot of cases. In a lot of cases they'll be serving that up from a cache of some sort so the CPU work has already been done when someone else requested the page. CPU versus bandwidth isn't an inherent tradeoff.

</comment></comment></comment><comment><USER14> In addition to the steps you're already taking, and the ethical suggestions from other commenters, I suggest that you aquaint yourself thoroughly with intellectual property (IP) law. If you eventually decide to publish anything based on what you learn, copyright and possibly trademark law will come into play.

Knowing what rights you have to use material you're scraping early on could guide you towards seeking out alternative sources in some cases, sparing you trouble down the line.

<comment> yjftsjthsd-h I'm curious how this would be an issue; factual information isn't copyrightable, and most of the obvious things that I can think to do with a scraper amount to pulling factual information in bulk. Even if it's information like, "this is the average price for this item across 13 different stores". (Although I'm not a lawyer and only pay attention to American law, so take all of this with the appropriate amount of salt)

<comment><USER14> How much can you quote from a crawled document? Can you republish the entire crawl? What can you do under "fair use" of copyrighted material and what can't you do? Can you articulate a solid defense of your publication that it truly contains only pure factual information? Will BigCo dislike having its name associated with the study but can you protect yourself by limiting your publication to "nominative use" of its trademarks? What is the practical risk of someone raising a stink if the legality of your usage is ambiguous? Who actually holds copyright on the crawled documents?

You have a lot of rights and you can do a lot. Understanding those rights and where they end lets you do _more_ , and with confidence.

<comment> yjftsjthsd-h So I think I just was being unimaginative on "scraping"; I wouldn't have thought to save quotes/prose, just things like word counts, processed results (sentiment analysis), pricing, etc. In which case most of that shouldn't come up, but yes I can see where other options are less simple.

</comment></comment></comment></comment><comment><USER15> Simple,

respect robots.txt

find your data from sitemaps, ensure you query at a slow rate. robots.txt has a cool off period. See [https://en.wikipedia.org/wiki/Robots_exclusion_standard#Craw...](https://en.wikipedia.org/wiki/Robots_exclusion_standard#Crawl- delay_directive)

example: [https://www.google.com/robots.txt](https://www.google.com/robots.txt)

<comment><AUTHOR> Yeah that's a must do, but I think most websites don't even bother making a robots.txt beyond "please index us, google". However that wouldn't necessarily mean they're happy about someone vacuuming their whole website in a few days.

</comment></comment><comment><USER16> I think your main obligation is not to the entity from which you're scraping the data, but the people whom the data is about.

For example, the recent case between LinkedIn and hiQ centered on the latter not respecting the former's terms of service. But even if they had followed that to the T, what hiQ is doing — scraping people's profiles and snitching to their employer when it looked like they were job hunting — is incredibly unethical.

Invert power structures. Think about how the information you scrape could be misused. Allow people to opt out.

<comment><USER17> I tried to find a source to back up what you're saying about hiQ "snitching" to employers about employees searching for jobs, but all I can find is vague documentation about the legal suit hiQ v. LinkedIn.

Do you have a link to an article or something?

<comment><USER16> Sure, it's mentioned in the EFF article about the lawsuit: [https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq- v-l...](https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-linkedin- protects-scraping-public-data)

 _> HiQ Labs' business model involves scraping publicly available LinkedIn data to create corporate analytics tools that could determine when employees might leave for another company, or what trainings companies should invest in for their employees._

</comment></comment></comment><comment><USER18> Indirectly related, if you have some time to spare follow Harvard's course in ethics! [1]

Here is why: while it didn't teach me anything new (in a sense), it did give me a vocabulary to better articulate myself. Having new words to describe certain ideas means you have more analytical tools at your disposal. So you'll be able to examine your own ethical stance better.

It takes some time, but instead of watching Netflix (if that's a thing you do), watch this instead! Although, The Good Place is a pretty good Netflix show sprinkling some basic ethics in there.

[1] [https://www.youtube.com/watch?v=kBdfcR-8hEY](https://www.youtube.com/watch?v=kBdfcR-8hEY)

<comment><AUTHOR> Great recommendations, thanks!

<comment><AUTHOR> I must insist. This course is great! Thanks :)

<comment><USER18> Glad you like it! When I studied CS, I was really happy I found this course as well. Getting some ideas about what ethics is made me a bit better in reflecting on what the implications are with whatever I'm creating.

</comment></comment></comment></comment><comment><USER19> My general attitude towards web scraping is that if I, as a user, have access to a piece of data through a web browser, the site owners have no grounds to object to me using a different program to access the data, as long as I'm not putting more load on their servers than a user clicking all the links would.

Obviously, there may be legal repercussions for scraping, and you should follow such laws, but those laws seem absurd to me.

</comment><comment><USER20> Common CMS are fairly good at caching and can handle a high load, but quite often someone deems a badly programmed extension "mission critical". In that case one of your requests might trigger dozens of database calls. If multiple sites share a database backend, an accidental DOS might bring down a whole organization.

If the bot has a distinct IP (or distinct user agent), then a good setup can handle this situation automatically. If the crawler switches IPs to circumvent a rate limit or for other reasons, then it often causes trouble in the form of tickets and phone calls to the webmasters. Few care about some gigabytes of traffic, but they do care about overtime.

Some react by blocking whole IP ranges. I have seen sites that blocked every request from the network of Deutsche Telekom (Tier 1 / former state monopoly in Germany) for weeks. So you might affect many on your network.

So:

* Most of the time it does not matter if you scrape all information you need in minutes or overnight. For crawl jobs I try to avoid the time of day I assume high traffic to the site. So I would not crawl restaurant sites at lunch time, but 2 a.m. local time should be fine. If the response time goes up suddenly at this time, this can be due to a backup job. Simply wait a bit.

* The software you choose has an impact: If you use Selenium or headless Chrome, you load images and scripts. If you do not need those, analyzing the source (with for example beautiful soup) draws less of the server's resources and might be much faster.

* Keep track of your requests. A specific file might be linked from a dozen pages of the site you crawl. Download it just once. This can be tricky if a site uses A/B testing for headlines and changes the URL.

* If you provide contact information read your emails. This sounds silly, but at my previous work we had problems with a friendly crawler with known owners. It tried to crawl our sites once a quarter and was blocked each time, because they did not react to our friendly requests to change their crawling rate.

Side note: I happen to work on a python library for a polite crawler. It is about a week away from stable (one important bug fix and a database schema change for a new feature). In case it is helpful: [https://github.com/RuedigerVoigt/exoskeleton](https://github.com/RuedigerVoigt/exoskeleton)

<comment><USER21> If you use Selenium & Chrome WebDriver you can disable loading images by : AddUserProfilePreference("profile.default_content_setting_values.images", 2)

</comment></comment><comment><USER22> Some time ago I wrote an answer on stackoverflow: [https://stackoverflow.com/questions/38947884/changing- proxy-...](https://stackoverflow.com/questions/38947884/changing-proxy-while- data-scraping/38985146#38985146)

Maybe that can help.

<comment><USER23> You should probably just paste your answer here if it's that good.

</comment></comment><comment><USER24> as sort of a poor man's rate limiting, I have written spiders that will sleep after every request, for the length of the previous request (sometimes length of the request times a sleep factor that defaults to 1). My thinking is that if the site is under load, it will respond slower, and my spider will slow down as well.

</comment><comment><USER25> Another option is to not scrape at all, and use an existing data set. Common crawl is one good example, and http archive is another.

If you just want meta data from the homepage of all domains we scrape that every month at [https://host.io](https://host.io) and make the data available over our API: [https://host.io/docs](https://host.io/docs)

</comment><comment><USER26> This might be overboard for most projects but here is what I recently did. There is a website I use heavily that provides sales data for a specific type of products. I actually e-mailed to make sure this was allowed because they took down their public API a few years ago. They said yes everything that is on the website is fair game and you can even do it on your main account. It was actually a surprisingly nice response.

</comment><comment><USER27> I work with a scientific institution and it's still amazing to me that people don't check or ask if there are downloadable full datasets that anyone can have for free. They just jump right in to scraping websites.

I don't know what kind of data you're looking for, but please verify that there isn't a quicker/easier way of getting the data than scraping first.

</comment><comment><USER28> I've gone through this process twice- one about six months ago, and once just this week.

In the first event the content wasn't clearly licensed and the site as somewhat small, so I didn't want to break them. I emailed them and they gave us permission but only if we only crawled one page per ten seconds. Took us a weekend, but we got all the data and did so in a way that respected their site.

The second one was this last week and was part of a personal project. All of the content was over an open license (creative commons), and the site was hosted on a platform that can take a ton of traffic. For this one I made sure we weren't hitting it too hard (scrapy has some great autothrottle options), but otherwise didn't worry about it too much.

Since the second project is personal I open sourced the crawler if you're curious- [https://github.com/tedivm/scp_crawler](https://github.com/tedivm/scp_crawler)

</comment><comment><USER29> My policy on scraping is to never use asynchronous methods. I've seen a lot of small e-commerce sites that can't really handle the load, even if it's a few hundred requests per second, and the server crashes. So even if it takes me longer to scrape a site I prefer to not cause any real harm on them as long as I can avoid it.

</comment><comment><USER30> The suggestions in the comments are excellent. One thing I would add is this: contact the site owner in advance and ask for their permission. If they are okay with it or if you don't hear back, credit the site in your work. Then send the owner a message with where they can see the information being used.

Some sites will have rules or guidelines for attribution already in place. For example, the DMOZ had a Required Attribution page to explain how to credit them: [https://dmoz-odp.org/docs/en/license.html](https://dmoz- odp.org/docs/en/license.html). Discogs mentions that use of their data also falls under CC0: [https://data.discogs.com/](https://data.discogs.com/). Other sites may have these details in their Terms of Service, About page, or similar.

</comment><comment><USER31> The rules you named are some I personally followed. One other extremely important thing is privacy when you want to crawl personal data like social networks. I personally avoid crawling data that inexperienced users might accidentally expose, like email adresses, phone numbers or their friends list. A good rule of thumb for social networks for me always was, that I only scrape the data that is visible when my bot is not logged in (also helps to not break the providers ToS).

The most elegant way would be to ask the site provider if they allow scraping their website and which rules you should obey. I was surprised how open some providers were, but some don't even bother replying. If they don't reply, apply the rules you set and follow the obvious ones like not overloading their service etc.

<comment><AUTHOR> I tried the elegant way before, after creating a mobile application to find fuel pumps around the country for a specific brand. My request was greeted with a "don't publish; we're busy making one; we'll sue you anyway". I guess where I'm from, people don't share their data yet...

Totally agree with the point on accidental personal data, thanks for pointing that out!

PS: they never released their app...

</comment></comment><comment><USER32> If all scrapers did what you did, I'd curse a lot less at $work. Kudos for that.

Re 2 and 3: do you parse/respect the "Crawl-delay" robots.txt directive, and do you ensure that works properly across your fleet of crawlers?

<comment><USER33> In addition to crawl-delay there's also HTTP 429 and the retry-after header.

[https://tools.ietf.org/html/rfc6585#page-3](https://tools.ietf.org/html/rfc6585#page-3)

<comment><USER34> Sites also use 403 and 503 to send rate-limit signals, despite what the RFCs say.

</comment></comment></comment><comment><USER35> Be careful about making the data you've scraped visible to Google's search engine scrapers.

That's often how site owners get riled up. They search for some unique phrase on Google, and your site shows up in the search results.

<comment><USER36> This isn't really an "ethical" practice, more like how to hide that you are scraping data practice. If you have to hide the fact that you are scraping their data, maybe you shouldn't be doing it in the first place.

<comment><USER35> Depends. Maybe, for example, you're doing some competitive price analysis and never plan on exposing scraped things like product descriptions...you only plan to use those internally to confirm you're comparing like products. But you expose it accidentally. Avoid that.

</comment></comment></comment><comment><USER37> It's helpful to filter out links to large content and downloadable assets from being traversed. For example, I assume you wouldn't care about downloading videos, images, and other assets that would otherwise use a large amount of data transfer and increase costs.

If the file type isn't clear, the response headers would still include the Content-Length for non-chunked downloads, and the Content-Disposition header may contain the file name with extension for assets meant to be downloaded rather than displayed on a page. Response headers can be parsed prior to downloading the entire body.

</comment><comment><USER38> In some cases, especially during development, local caching of responses can help reduce load. You can write a little wrapper that tries to return url contents from a local cache and then falls back to a live request.

</comment><comment><USER39> As many pages are at least half-way SPAs, make sure to really understand the website's communication with their backend. Identify API calls and try to make API calls directly instead of downloading the full pages and extracting the required information from HTML afterwards. If you have certain data sets from specific API calls that almost never change, try to crawl them less regularly and instead cache the results.

</comment><comment><USER40> You may need to get more specific about your definition of "ethical".

For example, do you just mean "legal"? Or perhaps, consistent with current industry norms (which probably includes things you'd consider sleazy)? Or not doing anything that would cause offense to site owners (regardless of how unreasonable they may seem)?

I do think it's laudible that you want to do good. Just pointing out that it's not a simple thing.

</comment><comment><USER41> Haven't seen anyone mention this, but asking permission first is about the most ethical approach. If you think sites are unlikely to give you permission, that might be an indication that what you're doing has limited value. Offering to share your results with them could be a good plan.

I work for a company that does a lot of web scraping, but we have a business contract with every company we scrape from.

</comment><comment><USER42> Schema.org is a nice resource. If you can find that meta-data on a site, you can be just a little more sure they don't mind getting that data scraped. It's the instruction book for teaching google and other crawlers extra information and context. Your scraper would be wise to parse this extra meta information.

</comment><comment><USER43> The only sound advice one can give is: there are two elements to consider: 1) ethics is different from law 1.1) the ethical way: respect robots.txt protocol 2) consult a lawyer 2.1) prior written consent, they will say, prevents you from being sued, and not much else.

</comment><comment><USER44> Those 3 are the main, sharing the data in the end could be also a way to avoid future scrapings.

<comment><USER45> That's an interesting proposition. For example there is Google Dataset Search where you can "locate online data that is freely available for use".

<comment><AUTHOR> Didn't know about that search engine. Thanks a lot! Actually found a few fun datasets, made my day :)

</comment></comment></comment><comment><USER46> [https://scrapinghub.com/guides/web-scraping-best- practices/](https://scrapinghub.com/guides/web-scraping-best-practices/) may be of interest to you.

</comment><comment><USER47> IMO, the best practice is "don't". If you think the data you're trying to scrape is freely available, contact the site owner, and ask them whether dumps are available.

Certainly, if your goal is "learning in data science", and thus not tied to a specific subject, there are enough open datasets to work with, for example from [https://data.europa.eu/euodp/en/home](https://data.europa.eu/euodp/en/home) or [https://www.data.gov/](https://www.data.gov/)

<comment><USER48> Where this _' best practice is "don't"'_ idea comes from? I saw it couple of times when scraping topic surfaces. I think that it is kind of hypocrisy and actually acting against own good and even good of the internet as whole because it artificially limits who can do what.

Why are there entities which are allowed to scrape web however they want (who got into their position because of scraping the web) and when it comes to regular Joe then he is discouraged from doing so?

<comment><USER47> In my book, "Not best practice" doesn't imply "never do", but web scraping should be your option of last resort. Doing it well takes ages, and time spent doing it will often detract you from your goal.

As I said, in this case "learning data science" likely doesn't require web scraping; it just requires some suitable data set.

The OP claimed in another comment that that doesn't exist, but (s)he doesn't say what dats (s)he's looking for, so that impossible to check.

</comment></comment></comment><comment><USER49> I like this approach. Personally I wait an hour if I get an invalid response and use timeouts of a few seconds between other requests.

</comment><comment><USER50> Don't fake identity. If the site requires a login, don't fake that login. This has legal implications.

</comment><comment><USER51> Contact site owner, tell them who you are and what you're doing, ask about data dump or api.

</comment><comment><USER52> Ask for permissions and have nice timeout/retries.

</comment><comment><USER53> Ethical web scraping? Is that even a thing?

<comment><USER54> No, it's not and discussing it like it is a thing is irrational. Ethics are based on morals and morals are based on determining a "right" course of action for a given act.

Just because something is legal, by absence of law, doesn't mean it's right or fair for all cases. Just because something is illegal (copyright) doesn't mean it's not right or fair for all cases. What if the information saved a million lives? Would it still be ethical to claim "ownership" of that information?

What if the information caused a target audience to visualize that thing over and over again? Is it right to allow that information out into the public at all?

g'disable javascript in your browser'

<comment><USER55> And moral is subjective. There is no one "right" course of action.</comment></comment></comment>


Cloudinary adds video to its roster of services - <AUTHOR> http://thenextweb.com/apps/2015/05/05/cloudinary-adds-video-to-its-roster-of-services/

<comment><AUTHOR> Cloudinary's new video management solution covered by The Next Web: "Cloudinary's video service provides a technical solution for all aspects of handling videos online, allowing developers to focus on their product rather than having to use resources to build and support in-house video operations."</comment>


Ask HN: Anyone else noticed the 3px margin-top on embedded Google map iframes? - <AUTHOR>

A client complained that I added a white line above his map the other week. Sure enough, there was a 3 pixel line above the map. I investigated further and it's actually in the iframe, inline, applied to the body tag.
I figured it was an anomaly at first, but then I started to look at other projects I'd used embedded iframes on. Now I'm noticing it all over the place. I found a bug report, but it's been a while. How long does it take them to get someone to tweak a stylesheet?
Example: http://curtziegler.com/sitedemo/studio960/wordpress/contact-us/
Bug report: http://codecanyon.net/item/ajax-contact-form/50846 <comment><USER2> Not sure if you found this or not, but some of these solutions may help:

[http://code.google.com/p/gmaps-api- issues/issues/detail?id=2...](http://code.google.com/p/gmaps-api- issues/issues/detail?id=2762)</comment>


Reveal.js - HTML Presentation Framework - <AUTHOR> http://lab.hakim.se/reveal-js/#/ <comment><USER2> it's a good thing to allow PDF export, most of the other frameworks I looked at do not support it at all. I'll give it a try!

<comment><AUTHOR> Good point - easy to forget with all of the fun visual effects.</comment></comment>


Hackathons don't win you customers - <AUTHOR> http://paul.kinlan.me/hackathons-dont-win-you-customers/

<comment><USER2> It sounds like the author is trying to tell companies what to do based upon his personal preference on why companies should run hackathons rather than based on any kind of evidence.

I know plenty of developers and founders who picked up an API (Twilio, Stripe, Paypal and Azure come to mind) at a hackday and then ended up using it commercially elsewhere. These companies aren't stupid, they're often investing 6-7 figures in marketing spend in running hackathons. They look at ROI.

It's fine if you prefer non-marketing focused hackdays, but that's totally different from saying they don't work.

<comment><USER3> It is based on a lot of experience running Hackathons. I chose not to elaborate much further on the reasoning in the article to keep it to the point. Ultimately the cost of running hackathons for the limited scale of developers that you reach at anyone time could be best spent in many many more ways.

The Hackathon _rarely_ wins the customer, and it is not sustainable as a practice often due to cost and ROI is near impossible to determine.

<comment><USER4> Personally, I'd be interested in some data and reasoning to further understand your point of view. Without elaborating, the points you make come across a bit terse and declarative, whereas this comment makes me think you actually have a deep understanding of the subject and good reasons for stating the points you made.

As a standalone developer, I can think of several companies that I feel have built great "developer brand" and goodwill through hackathons, whether running their own, or sponsoring others (e.g. TC Disrupt, Startup Weekend). Twilio is the first one that comes to mind. But whether that translates into actual revenues is where my anecdotal evidence clearly falls short.

</comment></comment></comment><comment><USER5> Honestly, Hackathons are more about what the developers get out of it.

I went to a couple to try and build a wider network among my local devs and to try and win cool stuff.

Some went to hone existing projects using the APIs presented (to know if they actually wanted to use it)

Some went for the prizes

Some went to try and score jobs

To say any of those options are wrong really isn't right. But to tell a company why they should be there they should ask what the developers are hoping to achieve and decide if there place belongs there or not.

I will agree completely on the comment about company reps _being present_ however. A lot of people were turned off from various companies because they were difficult to work with through different online channels and elected to not use their API simply because the cost to get it working with flawed documentation or sparse contact was too risky if they were gunning for a prize. Often the companies didn't return for the next hackathon for what I assume was because of bad reception the first time around...

</comment><comment><AUTHOR> I think it's a great point to reward developers who submit bugs and give feedback.

Though I do think hackathons can serve different purposes. The most important thing is to be clear on the objective and don't try to win on every front. e.g. If it's to publicise your API among developers, great, focus publicity, talks, challenges around that. If it's to get feedback, same thing.

And this mirrors how developers should approach hackathons, which is again to have one clear objective. It could be to build something amazing, it could be to learn an API, but you'll rarely achieve both.

</comment><comment><USER6> The only worthwhile hackathons I've ever seen or participated in were related to the release of a new API, one that none of the participants had any access to before release. This arrangement benefits both the hackathon attendees and the sponsor: attendees know that they're starting from a level playing field, that if there is a competitive angle to the hackathon, then everyone will be judged on how well they use the new API in their software and not on what they wrote before they showed up. For the sponsors, it is a trial by fire, testing their API architecture, backend, documentation, and usefulness.

I completely agree that the optimal result for an engineering-driven hackathon is a long list of bugs and a couple of surprisingly awesome demos. But I would add a room full of excited-about-pioneering developers, and a battle-tested developer support team.

</comment><comment><USER7> Hackathons do a great job of attracting developers to your company though.

<comment><USER8> Isn't this kind of the point of hackathons? It has nothing to do with gaining customers. Who even asked thought that it did? Hackathons, from my understanding, have always been about developers getting together to improve their skills. Also I'm not sure how much value a PM would get out or give to a hackathon.

</comment></comment><comment><USER9> It seems like hackathons and other hacker conferences and events have successfully capitalized on the popularization of start up culture. It makes this* seem like one big fad. The deeper purposes of sharing, freedom, coding and solving problems have been forgotten.</comment>


Herbicide-resistant superweeds overpowering crops - <AUTHOR> http://www.cbc.ca/news/technology/story/2011/10/07/technology-superweeds-roundup-ready.html <comment><USER2> You mean the company that patents crops _(1)_ , locks them away from farmers _(2)_ , and then files drive-by lawsuits wherever their seed drifts _(3)_ is having trouble with mutant weeds? Say it ain't so.

Looks like mother nature had prior art, and boy is she pissed.

(1) <http://en.wikipedia.org/wiki/Monsanto>

(2) [http://www.monsanto.com/newsviews/Pages/saved-seed-farmer- la...](http://www.monsanto.com/newsviews/Pages/saved-seed-farmer- lawsuits.aspx)

(3) [http://current.com/technology/89264825_california- legislatur...](http://current.com/technology/89264825_california-legislature- passes-bill-protecting-farmers-against-monsanto-lawsuits.htm)

<comment><USER3> Very snippy and all quite possibly true. Nevertheless, Monsanto has had a huge _positive_ impact on agriculture all across the world. Nobody puts a gun to the head of the farmers. They _choose_ to buy Monsanto because of higher yields. Higher yields bring in more revenue to the farmers and reduce prices worldwide, at the same time. Lower prices = fewer people starving.

<comment><USER4> I spent twenty years as an agronomist. Herbicide resistant weeds are not new, we had them before Roundup was as widely used. Most university scientists predicted Roundup resistant weeds before Roundup resistant crops were grown outside the lab.

Monsanto isn't the only company to sell GM seed resistant to herbicides. But they were the first and are still by far the largest. There is a hatred of them similar to how the software industry feels about say Microsoft.

Genetically modified seed is the single biggest advance in agriculture in the past fifty years. Do you have any idea of how many millions of pounds of insecticides aren't sprayed anymore? Roundup is also both cheaper and safer than a lot of the previous alternatives. Yet there is still strong opposition to GM seed and Monsanto, especially in the press.

Farmers and the fertilizer people who do a lot of the spraying live in the community. Having been a part of that fraternity I can guarantee you they aren't going to do anything to pollute their world or ship a product they wouldn't let their own family consume. But I guess you're going to have to take my word on that one.

There is seed ready to be marketed that is resistant not only to Roundup but two other herbicides with different modes of action. That is a fact the article does not mention. It won't solve the problem entirely, but will make Mother Nature work a lot harder to defeat it.

<comment><USER5> >Farmers and the fertilizer people who do a lot of the spraying live in the community. Having been a part of that fraternity I can guarantee you they aren't going to do anything to pollute their world or ship a product they wouldn't let their own family consume.

how idyllic. I guess it is some other, bad-bad-bad, farmers and fertilizer people who did all this agricultural damage to the nature.

<comment><USER6> Following on to that, farm runoff is a widely known source of contaminates to rivers and lakes. It's a non-point-source pollution, and the EPA says "agriculture, including crop production, animal operations, pastures, and rangeland, impacts 18% of the total river and stream miles assessed, or 48% of the river and streams identified as impaired". Agribusiness has generally been against increased restrictions on non-point source pollution.

There are solutions, but most are expensive and with no immediate local gain. Farmers, like everyone, balance risk, time, and money. In this case the risk is literally downstream and several years or decades in the future. To follow the original metaphor, it's someone else's problem in a different world.

It's easy to see how a distant risk might not be considered worth minimizing.

The common solutions to that are laws and financial incentives, both of which bring either the risk or reward closer to hand.

Therefore, to say that "they aren't going to do anything to pollute their world" reflects a simplification of the issue. If there's no longer EQIP, CRP, CREP, or any of a large number of other programs to encourage good land management practices, then do you think the farmers will continue to do so?

If so, then let's cut that funding since obviously it's not needed.

</comment></comment></comment></comment></comment><comment><USER7> Well, that's what happens when you don't rotate your herbicides every season like all the agriculture experts always told you to. Good luck with the next business model, Monsanto.

</comment><comment><USER8> There was an article about this a month ago, i think here. Saying that ALL of monsanto's competitors are swooping in and seeing this as a golden opportunity to shill their own seeds and herbicides as well. monsanto needs to adapt quick. They've had quite some time to figure this out.</comment>


Apache Kafka on Kubernetes - <AUTHOR> <comment><USER2> Confluent (the company housing the core Kafka team) has created a Helm chart to get their platform (either the open source version or the entire enterprise stack) deployed on Kubernetes.

Blog post here: [https://www.confluent.io/blog/getting-started-apache- kafka-k...](https://www.confluent.io/blog/getting-started-apache-kafka- kubernetes/)

</comment><comment><AUTHOR> I am about to embark in this journey, maybe. Have been working with Kubernetes for quite sometime to the point where I'm fairly comfortable managing it, tweaking it and fine tuning it. Business demands something like Kafka but I'm not sure if I'm inflicting into on-call nightmare if I go with Kafka on Kubernetes. Thoughts?

</comment><comment><USER3> I would also encourage you to take a look at NATS which, like Kubernetes, is hosted by CNCF.

[https://landscape.cncf.io/category=streaming- messaging&forma...](https://landscape.cncf.io/category=streaming- messaging&format=card-mode&grouping=category)

</comment><comment><USER4> You might want to check out strimzi[1]. They've got a couple quickstarts for minikube or okd to help get folks up and running.

[1] - [https://strimzi.io/](https://strimzi.io/)

<comment><AUTHOR> This looks interesting. Seems to be a bit more of a generic Kafka installation compared to the Confluence option. The fact that it makes it easy to run with Minikube is definately a plus.

</comment></comment><comment><USER5> Please look at [http://pulsar.apache.org/](http://pulsar.apache.org/) which is multiple fold better than kafka on scaling as well as easy to operate</comment>


Christiansen Patreon Call Transcript - barry-cotter https://docs.google.com/document/d/1U0mQjUA0T5INc_GDkwPJ2mfhO7tbaIogisSqqxHw0hc/mobilebasic <comment> barry-cotter [https://twitter.com/nickmon1112/status/1076886857445711872?s...](https://twitter.com/nickmon1112/status/1076886857445711872?s=21)

Long twitter thread on the relation between MasterCard, Visa and Patreon and how the credit card cartel is attempting to minimise political risk.

[https://skytideseo.com/news/subscribestar-website-gets- shutd...](https://skytideseo.com/news/subscribestar-website-gets-shutdown-by- paypal-stripe/)

The Patreon alternative SubscibeStar was banned by payment processors Stripe and PayPal, as well as their other six payment processors in one day.

It's all Visa and MasterCard.

<comment><AUTHOR> Not OK. Political risks are not all the same.

For one, those seeking to chalkenge the current economic status quo, to improve the general welfare have every right to do so and will get lumped in with nefarious actors.

<comment><USER2> The only way to challenge the credit card companies and big banking is cash in the mail.

Clubs dues used to be collected this way. A company that can take cash in hand (or through the mail) and handle disabling access for non-payment could be mostly immune to this sort of manipulation.

<comment><AUTHOR> Good to know.

</comment></comment></comment></comment><comment><USER3> Please don't break the site guidelines by editorializing in submission titles. On HN, being the submitter of an article conveys no special rights over the content, such as how to frame it for readers. If you want to express your opinion, the place for that is in a comment; then your view is on a level playing field with other users'. Titles are by far the biggest influence on threads, so this rule is kind of a big deal.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

(Submitted title was "Patreon are banning people at MasterVisa's direction".)

<comment> barry-cotter Ok

</comment></comment><comment><USER4> Fascinating. But didn't you admit in the transcript that you broke the law in your state and California by recording the call without consent? Or was that a lie?

<comment><USER5> A transcription doesn't have to be made from a recording. Check out how court transcriptionists work.

While I fully believe he recorded the call, there are other plausible answers and without a copy of the recording or him admitting it, we're just guessing.

<comment><USER6> A semantic point: a stenographic transcript is a recording, just not an audio recording. Furthermore, court stenographers prepare the final transcript from an audio recording, and setting up and controlling the audio equipment to record court proceedings is one of the court stenographer's duties. Their in- court duties do, of course, include reading back statements made moments ago, which is much more practical to do from a page than from a recording, hence why voice-to-text software today and steno machines previously are so closely associated with court stenography.

<comment><USER5> Thanks for the background and information.

In this particular situation, the two-party consent laws revolve specifically around capturing audio in audio form, not writing.

</comment></comment></comment></comment><comment><USER7> The previous deplatforming has been legal, but the blocking out of SubscribeStar is likely to be ruled anti-competitive behavior. There is an anti-trust case in the works for the Federal Trade Commission: [https://www.youtube.com/watch?v=-2ySC7edHO0](https://www.youtube.com/watch?v=-2ySC7edHO0) If they take it up, they will look much more deeply into what's actually going on behind the scenes.

Googles very existence stems from the fact that Microsoft was under FTC investigation and had to step carefully. If not, Microsoft would have squashed Google when it was little. The FTC changes behavior by just the act of looking into the thing.

</comment><comment><USER8> Archive:

[http://archive.is/6xSnA](http://archive.is/6xSnA)</comment>


IBM's legendarily clicky Model F keyboard is coming back - <AUTHOR> http://www.popularmechanics.com/technology/gadgets/a27123/model-f-project-buckling-spring-keyboard/ <comment><USER2> I'm in on this group buy. I really appreciate the hardcore QC that Joe's doing. It's taking longer than expected, but it wouldn't be worth it if it were half baked.</comment>


WebM: Missing The Assurances Open Source Needs? - <AUTHOR> http://www.computerworlduk.com/community/blogs/index.cfm?entryid=2973&blogid=41

<comment><USER2> including field of use restrictions in the patent license was a pretty terrible move from google, I'm sure they had their reasons, but it isn't really a good way to progress the open source codec scene.

<comment><USER3> There was some discussion about different licences for the codebase and the format. I'm not sure of the details but this is something the ffmpeg guys raised I think.

Given the (somewhat surprising, considering who it's coming from) lack of rigor in his other complaints, it's entirely possible this is a non issue. I guess we'll see.

edit:

The bitstream licence: <http://www.webmproject.org/license/bitstream/>

The software licence: <http://www.webmproject.org/license/software/>

Only the latter includes the phrase "this implementation of VP8" that is objected to, the former has the more general "implementations of this specification" in the same place.

What that actually means in legal theory or practice I have no idea.</comment></comment>


PlayStation 4 revealed - <AUTHOR> http://www.edge-online.com/news/playstation-4-revealed/ <comment><USER2> "Anonymous sources disclose unverifiable information!"

</comment><comment><USER3> I fail to see how this "revealed" the PS4</comment>


Tracking the Google Streetview Car in Berlin - <AUTHOR> http://fffff.at/google-streetcar-berlin/

<comment><USER2> I couldn't distract myself from watching the guy humping the Google logo long enough to get the value from this. Classy stuff.</comment>


What Keeps NFC From Explosive Growth? A Gateway Product - <AUTHOR> http://www.fastcodesign.com/1671623/what-keeps-nfc-from-explosive-growth-a-gateway-product#1

<comment><USER2> The products I want are BT 4.0LE or NFC challenge-response electronic leash and authentication device for my phone, tablet, laptop, desktop, etc. i.e. if someone grabs my running phone out of my hands, and for some reason I can't run him down, the system locks when it gets out of range.</comment>