Upwork Freelancers Who Want Guaranteed Pay Have to Opt in to Tracking Software - <AUTHOR> https://www.buzzfeednews.com/article/carolineodonovan/upwork-freelancers-work-diary-keystrokes-screenshot <comment><USER3> I'm a consultant and have been for a long time, and I know this business. I don't think counting keystrokes and in many cases screenshots make any sense, so I think the implementation of this is totally bogus, but the concept makes complete sense.

"I've never had a client expect to be able to look over my shoulder"

Right well UpWork isn't your client. I don't think your clients should, and to another person's point, yes your clients should trust you.

But this is in the case of when you do the work and the client _doesn't_ pay. This is UpWork covering your ass, when they really have no obligation to. They want a way to prove that you are least did some work, so likely they can go after the client later and recover costs.

This seems absolutely reasonable and honestly a heck of a bargain. Again, I think the implementation of counting keystrokes seems like a stupid way to do it, but coming up with a tracking method to _optionally_ guarantee payment even in the case of client non-payment is the deal of the century in the freelance world. Don't believe me? Try freelancing for a while and see how many clients stiff you after you've completed the job. There are literally advertisements in the NYC subway about it.

(Yes, retainers, payment up front, et ceteta -- no the world does not always work that way.)

<comment><USER4> Isn't there some deliverable that proves the work was done?

<comment><USER5> Yes but Upwork wants a metric that doesn't need a software engineer to adjudicate. If you say "this XHMTL validator works as specified because RFC 1913 was never formally accepted" it takes someone with a clue to figure out what you mean. Upwork would rather say "you types all day, you get paid."

<comment><USER6> Still very easy for people to bypass. Just pushes the adjudication into the "interpret screenshot" layer. The screenshot can be a completely valid Photoshop mockup that looks exactly like the client's specifications for the Rails app they hired you for, showing that you pretended to do the work.

<comment><USER7> I'm guessing that that ploy would work once or twice before you got pushed off the platform anyway. Same goes for a client that regularly claims the work is incomplete.

</comment></comment></comment></comment></comment><comment><USER8> I've been consulting off and on for the past five years and I've steered way clear of these marketplaces. It's a very obvious race to the bottom sort of situation they're plotting for which I get. I'd be optimizing for that outcome too if I were one of them.

Here's my advice to developers who want to freelance: network and network some more. Write blogs, create content marketing material for yourself. Go to meetups, tap your alum networks, go to hackathons, and work your networks. There's a lot of noise on LinkedIn but if used correctly, it can yield a lot for you.

Every single client I've had is someone I've met in person. I get to know them, they get to know me, we establish trust. Any sort of opening meeting or two is free. Formally, we use it to talk requirements. Ostensibly, we use it to vet each other.

<comment><USER9> _> Here's my advice to developers who want to freelance: network and network some more. Write blogs, create content marketing material for yourself._

There are many developers who can't do it because they live in Eastern Europe or Asia or Latin America and speak only conversational English.

Upwork has a lot problems but it's providing $25+/h rates for thousands of professionals who otherwise would get only $10-15/h or less locally. (I'm one of them.)

<comment><USER10> _Eastern Europe or Asia or Latin America and speak only conversational English._

It's the opposite - they have access to markets that English-only speakers don't. A Brazilian developer networking with Brazilian companies is going to be much happier than one scrabbling for work in a race-to-the-bottom marketplace like Upwork

<comment><USER9> These markets rarely pay as much as US or even UK. I've seen dozens of developers with 2-5 years of experience earning more on Upwork than senior developers in their countries.

Also, it isn't a "race-to-the-bottom" if you have a good portfolio and know how to present your skills.

<comment><USER11> If they want to be paid well, they are probably not going on Upworks anyway... These websites are the bottom of what you can get as a consultant.

<comment><USER9> But where should they go with no presence in English-speaking Internet and only conversational language level?

</comment></comment></comment></comment></comment></comment><comment><USER12> That doesn't even scratch the surface of Upwork issues though. Has anyone recently seen how much clients are willing to pay for work? I'm seeing avg pays of about 9-10$ an hour for mid-complexity jobs. Even see some clients willing to pay 600$ for fully functional IOS, Android and Web App iterations of an App.

<comment><USER13> There definitely are clients who are happy to pay $40-$75 an hour for senior full stack eng talent. We are such a client, and two VPE friends also commonly use upwork for 1-3 month gigs where they don't want to pay $175/hour for an sfbay contractor. The reason you're not seeing them is good clients eventually find a good contractor and either just run fulltime ongoing work through upwork (for eg payment convenience) or move off upwork.

<comment><USER14> I was earning $95/hr as a ux UI designer. Now I run a consultancy and charge per project which equates to 260-300$ an hour. Upwork has many issues and limitations but if you do well by good clients you can make it work pretty well.

</comment></comment></comment><comment><USER15> Not to shamelessly plug, but this Upwork monolith (and the general sub- optimized gig economy right now) is harmful not just for the freelancers themselves, but also the clients (from a price optimization perspective).

We're trying to solve it with a blind marketplace model at ContentFly ([http://contentfly.co/](http://contentfly.co/))

<comment><USER16> Would you care to elaborate how you see the Upwork Monolith is trending to become harmful to freelancers? It would be nice to see your thought on this.

I'd like to see a more balanced Upwork type of freelance work, where perhaps a lot more people are given the chance to participate through quality, rather than quantity.

<comment><USER17> I think that's why it is harmful: it's basically a race to the bottom, and requires unsustainable prices to get any work.

</comment></comment></comment><comment><USER18> This work should not be hourly. I think if I don't know someone well enough to trust them logging hours, I don't pay hourly.

</comment><comment><USER19> I just allow manual hours. If I dont trust them to do the right thing why would I even get a freelancer the job?

</comment><comment><USER20> As far as I know this has always been the case. oDesk had a screenshot tracking program too for hourly jobs prior to the merger and rebranding.

<comment><USER21> It did/does. I did not have an issue with it, besides occasionally having to go delete a ton of screenshots when I accidentally left the tracker running while I wasnt working (not attempting to be fraudulent, just careless)

</comment></comment><comment><USER22> I wonder if anyone in developed countries can actually make a living from something like Upwork. It seemed to me whenever I have looked at it, that it's mostly people and companies from the developing world who work for less than minimum wage or close to it in developed countries... and the expected cost matches these sorts of rates. So it does not make sense for anyone other than the most desperate people in the first world to try to get work on these sorts of sites.

As such that sort of labor is easy to exploit. They have no bargaining power and are already working for peanuts.

<comment><USER23> "Not in San Francisco" and "easy to exploit" are very different. I had great, professional work done in Russia on Upwork. It didn't cost $200 per hour, but it didn't cost $2 per hour either. The engineer who did the work was the vendor, and I was the client, so I was definitely the boss and treated as such, but he was clearly not hurting for work and if I started behaving inappropriately he could have easily dumped me and found another client. I don't know how far his salary went where he was living, but there are many places in this world where 20 US dollars per hour can sustain a very nice lifestyle.

<comment><USER24> 20 US dollars per hour is already more than I make as a full-time SaaS salesman in New York City, and I'm not even hurting for cash either!

<comment><USER25> Do not forget about Upwork taxes:

Specifically, Upwork charges the freelancer a fee of: 20% for the first $500 billed with the client. 10% for lifetime billings with the client between $500.01 and $10,000. 5% for lifetime billings with the client that exceed $10,000.

<comment><USER24> God that's horrid. Not to mention the slew of taxes one pays by being a 1099 contractor. And it's not like the clients one attracts on Upwork are particularly good.

</comment></comment></comment></comment></comment><comment><USER26> I do some freelance work through Upwork. I like the tracking software, it shows that I'm doing work, all my hours are verified and they are guaranteed to be paid. It's completely worth it, people who have been stiffed on contracts before, its a no brainer.

<comment><USER27> Me too. I did 40k of work for an employer who ended up stiffing a ton of employees but I was protected because of upwork's setup.

I used to hate it and still I exit the app unless I'm actively working. But it assures payment which is important.

</comment></comment><comment><USER28> Recently a friend of mine pointed me towards TopTracker. It's an interesting concept, having kind of a papertrail for what you worked on can be really valuable in any setting, even the most friendly ones, because even yourself can loose track of all the yaks you've had to shave to fix a certain bug or make a gizmo spin a certain way.

I've seen so many freelancers waste company time and endangering otherwise great projects in the past and have even done so myself that I really think a proper, fair and well balanced tracking suite would have benefited all parties as it would have created a much more formal and clear reminder of what screentime in this setting is for.

Obviously the flip side is that through the "invasive tracking", work stays work and everything not-work needs to happen on another machine. This means less mixing of private stuff into your work life which many consider something positive, but is counter to what the larger trends in IT are. (Corporate Campuses that fulfill every need...)

At the end of the day i'm really not sure where I stand on this, as it opens so many positive possibilities (f.e. proactive encountering of procrastination) but also so many negative ones (abuse of power). Surveillance is never an easy topic.

<comment><USER29> Yeah this is really dumb. If you're charging hourly, but don't trust someone is actually working, don't charge hourly.

If you want full control of an employee don't get a freelancer. If you want to constant surveillance of any artists' computer; fuck off. If you don't trust the artist to work, get a new artist.

<comment><USER28> This, while not really being nuanced, strikes the most important point. Trust.

Building trust for a freelancer is hard, especially one working remote with little on-site. Transparency usually helps in these settings.

I've had to work on projects as a freelancer where there simply wasn't much progress for weeks because they were such monstrosities, that trust and a prose-documentation of my work was the only thing keeping me afloat. (think 5 different languages talking to each other via SOAP, consuming 3 different databases, all with encoding problems between every possible step)

Just being able to point at all the screenshots from the bazillion different debugging windows, log diffs, etc. would have made things much easier back then. Swapping me out for someone else two weeks into this project would have been a terrible mistake, but besides my assurances and some prose the manager had to trust me. Bad for him, bad for me.

<comment><USER29> If you're a company that commonly outsources, you should have good freelancing game. By that I mean you build a network of freelancers you've worked with before or have heard good feedback from trusted sources.

My girlfriend is a manager at a writing company. She has a group of about 30 freelancers she keeps in touch with on top of the 10 people who work directly under her at her company. When too much work comes in she leans on the freelancer that best fits what she needs. These are people she has worked with before, has had success with, and most of all trusts. She, of course, has to try new freelancers occasionally. But the bad ones get spit back out.

It sounds like the company you were working for doesn't have very good freelancing game.

<comment><USER28> The company actually had, that's why things worked out and I returned to more projects afterwards, but it was stressful and not a job they would have given someone else, because of the lack of trust inherent in any new freelancer.

If you want freelancing to stay mostly something for people great at selling themselves, managing customer relations etc. that model works well, but that's IMO not optimal for the company nor the majority of people who would enjoy being freelancers.

<comment><USER29> Hmmm, interesting. I wonder how important self advertising is in this case, though. I know that she usually posts content she can't get done with her normal freelancers on this writer's freelance site. Freelancers make bids, negotiate terms, etc, through the website. After she's worked with someone she likes through it she tends to keep their email and ask them directly next time.

I think you're right that this type of model benefits freelancers that manage their own group of clients rather than through an agency, however.

</comment></comment></comment></comment></comment></comment><comment><USER30> Anybody who dealt with upwork knows perfectly well that "guaranteed" ... there is nothing guaranteed with these systems.

And in my experience, the people wanting more control want it because they know that they themselves would, given the chance, betray the trust. If someone demands this happen, you can bet they're looking to exploit you. You don't want to work for these people, you don't want to touch them with a 10-foot pole.

<comment><USER31> I worked at a company with multiple teams doing basically the same job at multiple sites. The guys out in CA were always talking on the conference calls about the metrics being gamed by some other sites. It was all blame and innuendo with these jerks. Their numbers weren't as good as other sites (despite being paid substantially more than anyone else).

Finally the guys in CA got their wish and a stupid amount of energy was put into finding who might be gaming the system..... it turned out to be the people who worked for the guys in CA... their own direct reports.

</comment></comment><comment><USER32> I see many comments saying that this is fine because it's optional. Based on the previous posts discussing Upwork ([https://hn.algolia.com/?query=upwork](https://hn.algolia.com/?query=upwork)), it seems to me like today this is optional and tomorrow this will be mandatory and the next day it will add machine learning webcam face detection technology.

</comment><comment><USER33> Had a dev use macro keystrokes to cheat it. worthless.

<comment><USER34> Hahaha. What? That's awesome. What did the macros do?

I've always wanted to make a plausible code generator based on markov chains. It still seems like an effective way to trick people into thinking you're working on some hard problem.

<comment><USER33> No need for pseudo-code, random clicks/strokes will do (he used the arrow keys to move around code and switch open Atom tab).

</comment></comment></comment><comment><USER35> I'm not a fan of it, but it is a necessary evil sometimes. I had a client dispute my work after he asked me to spend hours on tangential crap, then claim I wasted his time (he asked me to do the work and bill the time, approved of it at the time too, but later reversed his decision)

Upwork's mediation process was hokey at best but in the end they reviewed the time logs and gave me all but a couple 10 minute chunks of time worth of payment.

I have just gotten used to being monitored while the clock is on, and don't do sensitive things while it's happening... I can always turn off the clock quick if needed.

I do feel like if I was buying services I would want some way to guarantee someone isn't billing me time for doing nothing, this encourages some sort of transparency and attempts to curb timecard "fraud"

<comment><USER36> > approved of it at the time too, but later reversed his decision

This is a very serious problem with upwork. I once had a client accept a finished project, give me an amazing review and then, close the project and complete the payment. A month later, he reversed the rating and review and started an issue with the Upwork system. The problem was that the third-party software service I used (as per the clients instructions) to perform a set of tasks, changed their policy and stopped allowing remote clients to their service.

I explained to the client during the development phase itself that this could happen. He explicitly told me that it was ok, and to go ahead.

Inspite of showing Upwork all the communications, my warnings, etc, they still sided with the client, blocked my account and transferred some balance amount in my account to the client.

The last time I used Upwork.

</comment></comment><comment><USER37> I've been using Upwork for side gigs since it was Odesk. The tracking system (which has scarcely changed in 5+ years) has always seemed fair and flexible to me. the tracker works well cross platform.. I can move from OSX to Win10 to Ubuntu (previously they supported Debian, and in the past I've managed to get it tracking successfully on Slackware) .. minimal downtime, good communication from their engineers. Being able to selectively trim the fat from my diary in 10 min increments on Sunday night is helpful in preparing clean weekly reports and reviewing my own workweek.

</comment><comment><USER38> I am absolutely not surprised about this BS tracking software stuff. I was interviewing with Upwork a couple of month back. I live in "western Europe" and what they offered me was actually less pay than being employed full time here. For the moment I accepted but more out of interest to see what the rest of their interview looks like.

The questions they had were quite specific about corner cases of the language used. I liked that a lot. However, it's also just stuff that you can read in a couple of blog posts about python corner cases.

What struck me the most was that they seriously asked me to write up a program to prove my coding skills - despite having send samples - at which point I dropped out.

Several things just turned me completely off about Upwork:

1) They claim that they have the "1%" of the best developers in the world. BS. With the kind of money they offer: forget about that. Maybe you have a couple of imposters who don't know that they can charge easily double of what they offer but the majority will be mid level for sure. So this claim is a lie.

2) They obviously cannot guarantee that you will be assigned a project anytime soon once you complete the interviews. But asked as well for a lot of engagement from your end to go through the process.

3) It was something like $6.000 per month - market rate for a contractor in Germany with comparable skills and experience is more in the range of $12.000

However, and that's my take from it: if I ever have to hire mid level developers for remote positions I definitely will look at Upwork's portfolio and pirate them away. Put the 20% that UW charges on top of their salary and the deal should be golden.

</comment><comment><USER39> Run the tracker in a virtual machine running locally. Run your "work" apps in that vm.

This is so stupid.

<comment><USER40> This is what I was planning to do when I landed my client but they trust me and enabled the option to log "manual hours".

I don't trust UpWork anymore than I trust PayPal so as far "guaranteed" hours are concerned it doesn't matter - I withdraw my funds every week.

</comment></comment><comment><USER41> Even with this screenshot tracking, I've been swindled on Upwork by fake devs. They take the time to fake it, why not just do the work?

<comment><USER42> It probably has more to do with not wanting be constantly spied on. That's how I think I would feel, anyway.

</comment></comment><comment><USER43> Shamefully hijacking, I'm actually building something similar but it _does_not_track_keystrokes_or_take_screenshots_ because we agree that that is too intrusive. It's made for lawyers to match their file activity with time entries.

[https://griffn.io/](https://griffn.io/)

We're also YC Winter 18 hopefuls..!

<comment><USER44> I believe 'his' should be 'he' in this context - "His has built widely used software ranging from award-winning individual apps to corporate workforce software."

</comment></comment><comment><USER45> The old Elance was great. Miss it a lot.

</comment><comment> King-Aaron We use 'TimeDoctor' as our tracking app at work, and it does an alright job. It also takes screenshots and checks for user activity.

A lot of people bitch and moan about it. But if you go work in a factory - <USER2> instance - you'll be clocking in and clocking out just the same.

<comment><USER46> _A lot of people bitch and moan about it._

... and your best people will leave. Over time you'll be left with just the ones who don't have any options to go anywhere else.

It's kind of a self-inflicted Dead-Sea Effect that you've set up.

</comment></comment><comment><USER47> They've done this for a decade since odesk. I've caught many freelancers watching porn or one was trying to sell data off the backend and freely chatting about it.

But I like the feature as an employer. I like seeing the process of the work being done.

</comment><comment><USER48> Knowing multiple business owners who have complained about remote employees faking hours, installing mouse simulators to trick time tracking software, etc, I'm not particularly sympathetic.

</comment><comment><USER49> Semirelated but I run timesnapper, which is a private tracking software running only on my PC so I can check if I've forgotten to save a doc and wanted to see what it looked like 2 hours ago.

<comment><USER50> Interesting!

Link: [http://timesnapper.com/](http://timesnapper.com/)

What would be the *nix alternative?

<comment><USER49> Great question, I needed a macOS equivalent for my macbook. Technically this is *nix since its unix flavored

[https://alternativeto.net/software/timesnapper/](https://alternativeto.net/software/timesnapper/)

I haven't tried any personally, but digging through the alternatives I actually didn't see any resembling it. All the other alternatives were designed to manage employees & their time. None of them seemed to offer video playbacks like timesnapper, only screenshots. And they worked online which is a giant red flag for me.

The closest one I could find though for linux and macOS is this one called productivity-peach [https://www.etopian.com/software/automatic-screenshots- windo...](https://www.etopian.com/software/automatic-screenshots-windows-mac- linux/) . I strictly wanted to avoid "remote monitoring" or "employee- tracking" type software

I did pay $30 for timesnapper (one license works on more than 1 PC). Timesnapper is nice, you can playback your entire day and see everything that was done. Super useful if you do sysadmin stuff as well, sometimes you might change a setting on AWS/cpanel/etc and it screws up everything. Now you can go back and see what you changed. You can add a password too which I do as well, your video playbacks are encrypted

<comment><USER50> Funny, I did the same thing.

The answer below by mmel is more appealing to me after looking through those alternatives...

I agree, that the closest is Productivity Peach, but being a closed source Java app, and my circumstances, that one liner could be going more my way, thanks Kagerjay and mmel!

</comment></comment></comment></comment><comment><USER51> ...meanwhile, books on managing anxiety are soaring.

</comment><comment><USER52> I don't think this is a bad idea at all. Micro-work should be tracked for both the worker and client's sake. Paying hourly is highly inefficient as we all know it's very difficult to work every second of an hour. So if something could more precisely track how often you work during an hour it may encourage you to work less not more as you'll be able to see when you start to taper off; the longer you work the less efficient you may become.

</comment><comment><USER53> I'm confused, why is this news? This is how its been for ever. I get my most interesting projects form upWork since I can check before applying what the job is about, which is something carder to do when you talk with a client of the street. Not to mention that you get guarantee pay every week. You don't have to spend time and energy to force the client to pay you. Which is the best part of upWork. Sure they get a 10% cut, but perennially is well worth it for the ease of mind.

And the tracking app also allows me to organize my work by living notes to the clients to let them know what I worked on. Which is always appreciated.

To sum it, up – not sure where the problem is. This is what it is, it works and if you are good you'll make plenty of money for yourself and upWork ;)</comment>


Incenting The User To Put Up A Profile Picture - <AUTHOR> http://avc.blogs.com/a_vc/2008/03/incenting-the-u.html

<comment><USER2> I get incensed when I see _incenting_ used this way.

<comment><USER3> Given that the more established alternative is _incentivize_ , which is at least 50% more awful, I welcome this newly abbreviated word.

<comment><USER4> You mean it could be as much as 60% more awful?

</comment></comment></comment><comment><USER5> Republicans use social networks too. The challenge would be to come up with some universally disliked person to use. I nominate Mr. Bean and Peewee Herman.

<comment><USER6> I like Mr. Bean...and Herman was okay in the Cheech and Chong movie, at least. I think they should use Emperor Palpatine...oh, wait, they're using Cheney? Nevermind.

</comment></comment><comment><USER7> We used to use a picture of a monkey.

Worked great as well.

<comment><USER8> But I would like the monkey!

<comment><USER9> Perhaps there was many creationists on his website, they hate the monkeys.

</comment></comment></comment><comment><USER10> Inciting the user to check for words that do not exist.

</comment><comment><USER11> I have a better rule.

People with pictures can see other profiles pictures otherwise they get a message explaining the rule.

</comment><comment><USER12> i understand the problem, but shouldn't the incentive be a service users _want_ to use and share with their friends?

</comment><comment><USER13> This is not a great idea. There are many people out there googling your name, and they will think that you choose that avatar.

<comment><USER14> Thus the incentive to change it. I like it!

</comment></comment><comment><USER15> Or if your typical user is a MySpace retard, your site will become known as that weird place where everyone has a boner for Dick Cheney.</comment>


No Code Opportunities - <AUTHOR> No Code is a collection of tools and methods that let people
easily create applications – landing pages, job boards, forums, apps, newsletters, etc. automate repetitive tasks – welcome emails, reports, notifications, connect multiple applications, etc manipulate data visually – calculate metrics, move data from one source to another, dashboards, etc
The ideal target for a No-Code solution is someone
would like to show value quickly can explain their problem in enough detail cannot access coding talent for budget or priority reasons
No-Code is having its moment because of the following
Growth in internet first businesses has led to increased demand for people who can build quick solutions to problems. Browsers have significantly expanded the list of things they can do. API first development ( led by AWS ) means with event-driven processing and juggling the right set of API's one can build almost anything. Tools like IFTTT and Zapier have lowered the learning curve for event-driven programming.
Create – APIs and integrations are the base layer for no-code. So any improvement here is a massive value add to the system. There are lots of opportunities here, take a business process that has a SaaS tool attached to it and build an integration. (Connect it to integration platform like Zapier)
Curate – one drawback of no-code is the lack of templatized solutions. The next best thing is a curated list of how to do X, (for any X that is a combination of tools). Find and build lists of people who have experience building in this paradigm. e.g No-Code developer as a role will be huge.
Sell – This is a small but growing movement, take advantage of the inherent "user-facing-ness" of no-code buy finding people/business processes that can take advantage and connect them to builders. You will see a lot of supporting infrastructures like development shops, startup studios, incubators, and funds. <comment><USER2> I agree, but how to monetize?

<comment><USER3> B2B Enterprise, see OutSystems, Mendix, and the Cloud provider options emerging.

Open Source: [https://github.com/hofstadter- io/hof](https://github.com/hofstadter-io/hof)</comment></comment>


Ask HN: What do you use for front end? - <AUTHOR> Hello HN,
A while ago after doing some research, I decided to go with VueJS for the frontend of a project I was working on but it feels very clunky and slow. The page size with all the libraries installed is nearly 8MBs.
I'm considering ditching our SPA all together building a flask app to interact with the backend with 0 javascript to reduce the page size and make it faster. I was wondering if there are any lightweight alternatives to Vue or know any good practices for speeding up the SPAs? <comment><USER2> 8mb is nuts. Is this the minified & gzipped size? Either way, code splitting and dead code elimination are your friends. I wonder if you're including multiple versions of the same libraries, or pulling in heavy libraries for only a handful of features?

Is your code available somewhere for us to look at?

<comment><AUTHOR> 8mb is the local version of the code without gzip and minification. I just tested at the minified version and it is 2.5 megabytes

<comment><USER3> 2.5 is still large.

Some ideas:

* split your application into several smaller ones, one for each "domain", with one bundle for your library code and one bundle for each "domain"

* use CDNs instead of bundling third party libraries

* make sure you import only what you use

</comment></comment></comment><comment><USER4> 8 MB?

What on earth are you including in your pages?

<comment><AUTHOR> It is a gigantic SPA using Vue, VueMaterial, and D3. It has around ~300 components. I don't know if it would be possible to lazy load some of the components</comment></comment>


There could be 36 communicating intelligent civilizations in our galaxy - <AUTHOR> https://edition.cnn.com/2020/06/15/world/intelligent-civilizations-galaxy-scn-trnd/index.html <comment><USER2> I hope there are more intelligent civilizations out there. Wouldn't want to be the smartest guys in the room :D.</comment>


Very Funny, Google Analytics :) - <AUTHOR> http://launchsky.com/ga-april-fools.png

<comment><AUTHOR> When I clicked through (excitedly) to see more details about who was visiting us from the International Space Station, I noticed that the top traffic source was a search for "April Fools." Google, you got me on this one.</comment>


Think piracy is killing the music industry? This chart suggests otherwise - <AUTHOR> http://www.washingtonpost.com/blogs/the-switch/wp/2013/10/07/think-piracy-is-killing-the-music-industry-this-chart-suggests-otherwise/

<comment><USER2> I don't see any indication, either here or in the original report, that these revenue numbers have been corrected for inflation. This matters because $50 billion in 1998 dollars would be worth about $69 billion in 2011 dollars. So if the revenues on the chart are nominal, this does indeed show a drop in total real revenue.</comment>


Physicists finally explain why your earphones are always tangled - <AUTHOR> http://blogs.discovermagazine.com/seriouslyscience/2014/06/18/scientific-explanation-earphones-always-tangled/#.VdqUZFNViko

<comment><USER2> Looks like they used string theory.</comment>


Most Americans back FBI over Apple in iPhone unlocking fight - <AUTHOR> http://www.dailydot.com/politics/apple-iphone-doj-fbi-unlocking-pew-survey/ <comment><USER2> That's because about 0.1% of Americans have anything remotely resembling the necessary context and expertise to even have a vaguely useful opinion or position on this issue.

When I'm facing a particularly vexing convergence problem for distributed data conflicts there are many, many very good reasons why I don't poll people in a random bar or cafe for what I should do. But the most important reason is because... they would absolutely, definitely give me terrible advice.

The good news for the government in this case though is that apparently in the absence useful context and expertise, and even when related to a company with an extremely favorable brand, the default position for the American people is to trend toward authoritarian oversight. That's a big social-psychological win for the government.</comment>


Apache vs. Nginx comparison - <AUTHOR> http://compargram.com/technology/apache-vs-nginx/ <comment><USER2> Visually nice but at least in the given example comparison lots of wrong data.

... apart from comments by someone who hasn't used either Apache or nginx for a serious project.</comment>


Want to move fast?  Just do this - <AUTHOR> http://codefastdieyoung.com/2011/03/want-to-move-fast-just-do-this-part-1-design/

<comment><USER2> This is excellent, very practical advice.

The one caveat I would add which the author glosses over is _Test in IE_ as a high priority unless you have a very tech-savvy audience. Compass/Blueprint abstract away most of the uglier CSS box-model hacks and I agree that IE users _can_ live without gradients and rounded corners. But if the site looks awkward without the CSS3 tricks that don't work on the browser that >60% of your audience uses, you're going to need to tweak that aspect as well .

<comment><USER3> >60%??? I haven't worked on a site in a while that had more than 45% IE users... most of mine are now below 1/3.

<comment><USER4> I do maintain a website that still has 65% of IE users (this month statistics).

<comment><USER3> MSDN?

<comment><USER4> MSDN is probably a lot lower (rough guess) :)

No, it's a corporate intranet site with multiple companies.

</comment></comment></comment></comment></comment><comment><USER5> Relating to Haml/Sass, you may have heard of the ruby gem StaticMatic. It's a great tool that lets you use Haml/Sass to building quick, static prototypes.

In fact, I loved the concept so much that I began improving upon it myself[1], adding support for CoffeeScript and Amazon S3. For anyone who might find it useful, any testing or feedback would be greatly appreciated.

[1] <https://github.com/mindeavor/staticmatic2>

<comment><USER6> I used Staticmatic, but I never liked having to move all my HAML/SASS over to my app. I recently replaced it with the showoff plugin [1], which is basically Staticmatic inside a Rails app.

[1] <https://github.com/adamlogic/showoff>

<comment><USER5> Interesting, thanks, I'll consider using that next time I'm building a prototype that will eventually become a Rails app. But who knows, I might write a rails-app-exporter for StaticMatic just for fun :D

</comment></comment></comment><comment><USER7> I liked it. Lot of folks may already know some or most of this. But it is distilled all in once place, and as the post mentions, it will surely help in quickly moving forward with a polished looking app.

<comment><USER8> I agree, there wasn't anything groundbreaking, but I definitely bookmarked the page halfway through.

</comment></comment><comment><USER9> _using pure black as the text color is a mistake_

Why is this?

<comment><USER10> Using #111 or #222 softens the page a tad without making it harder to read.

Hypocritically, the text on this article's page is black.

<comment><AUTHOR> Great catch. Although, in my defense I'm always looking to save time so I just used out of the box settings on the wordpress "Platform" theme. If you're starting from scratch on your own site, dont make this mistake :)

</comment></comment></comment><comment><USER11> There's a difference between moving fast and doing sloppy work. IMO, this post is a recipe for mediocre design (at best). Don't confuse movement for work - when you approach a design with the a rushed attitude it slows you down in the long run, because you have to clean up your mistakes and possibly start over again when you realize that your first attempt just didn't work at all.

I'm not so much arguing with the specific suggestions in the post as I am the general approach and worldview. There are some useful ideas in there, but they're weakened by being presented in "recepie" format. Sometimes they make sense, sometimes they don't. It depends on the context. Better to learn principles than methods. It's quicker in the long run.

<comment><USER12> The article is aimed at programmers without design skills. If you can teach them to create great designs instead of just half-way decent ones, write something up!

<comment><USER11> Good challenge! I think I'll take you up on it. That's a good Sunday project.

</comment></comment></comment><comment><USER13> This is certainly one not-terrible way of coming up with an uncontroversial design quickly.

</comment><comment><USER14> One of the most practical articles I've seen in a while, thanks. What are your thoughts on using UI frameworks like jqueryUI or YUI grid?

<comment><USER15> My advice wasn't the one solicited but: UI Frameworks are great at helping you get things done. If that is your primary goal, then you should use them. Plain and simple.

The only drawback is that you're incurring some technical debt because you may not be able to easily add needed features to the framework. That's okay for a MVP but needs to be considered before buying into the YUI Control #12 wholesale.

</comment></comment><comment><USER16> Does green convert better than red? Performable seems to like red over green. <http://blog.performable.com/631526233/>

<comment><USER17> I wonder whether orange would perform better than red. Orange is a striking color, and doesn't have the same connotation of "stop".

<comment><USER18> Orange has 'warning' and 'slow down' associated with it here in the US though (think of every traffic warning/construction sign and highway equipment you've seen).

It couldn't hurt to A/B test though!

</comment></comment></comment><comment><USER3> This is a good article, if for nothing else besides that button gem which I somehow haven't heard of before.

The downside is, there will be a bunch of sites that look the same, so folks will want to do some real work and find their own components.

<comment><USER19> I'd contest that it would only matter if you're the same as everyone else if it's contrary to your goal(s). What is your goal? For most it would be to get users to purchase something (aka, make money).

There's the other upside that if your website isn't radically different from others, a lot of your users will already know how to use it. There could be a lot to gain with a lot of your visitors knowing how to extract information from it.

I guess the only real way to know is by testing!

<comment><USER3> Yeah, I'd agree. I've seen far more UI design crime done in the name of "doing it a new way for the sake of it" than is caused by being similar to other implementations.

</comment></comment></comment><comment><USER20> I was super surprised at how well SASS let me move quickly with coding 3pics.me! The lighten, darken and mix functions make getting appropriate hex codes pain free.

Wonderful!

</comment><comment><USER21> anyone have a mirror to this?

<comment><AUTHOR> Sorry the server is on fire right now... got a tweet from smashing magazine. Working to bring it back up.

<comment><USER21> no problem, thanks for the update!

<comment><AUTHOR> back now.</comment></comment></comment></comment>


Ask HN: How do I learn how to become a good sysadmin? - <AUTHOR> Earlier today I read an article on the decline of proper sysadmins in the age of containers. I thought I was doing a decent job of being a sysadmin+developer at the place I work but looking at the problems containers can cause I couldn't help wonder how one should go about becoming a decent sysadmin.
Main scenario is the one where you as an individual might on your own (or with a couple of friends) start some tiny sass product, or build an open source product that you want to keep secure.
At the most basic level, when firing up a new server I follow the how to harden your server guides and install fail2ban, disable root login, enable ssh only login etc.
Things I don't fully understand, user permissions or rather, how to use permissions to ensure good security. I'm also unsure about logging. I'm essentially clueless about where to look to understand what goes into being a good sysadmin.
Any advice would be awesome. Thanks. <comment><USER3> Start with two mantras:

1\. I will know exactly what every command or script I run on a system I control is supposed to do - no exceptions. If I don't and are just following instructions, I really need to learn what it means and why. If you need to setup a test system and snapshot before and after to see how things work.

2\. I will document a lot. Imagine some poor person showing up after you have won the lottery (think happy thoughts, but watch out for buses just the same). Don't just blindly put down step, put the why down. If I cannot write why I am doing something then I need to think about it more.

The rest just flow from those. Learn to program, be a tool builder, find the best way to learn and dive in, solve problems, and insist on consistent, repeatable, backedup, secure systems.

Do remember though: all your successes will be hidden in the darkness and all your failures will be shown in the full light of day. Its not a fun gig at times.

<comment><USER4> This is good, and I'd add on to the part about the successes. Learn to document your successes and be able to verbally communicate why anything you do is important or useful to less technical users.

If you don't have a good boss who can see that you are good at what you do, you will have to be able to speak up if you want to be paid what you are worth.

You want to be a step up from a computer janitor who needs to be told what to do, to being someone who delivers value to the business and helps people get their jobs done, and can anticipate problems before they occur.

<comment><USER5> > Learn to document your successes and be able to verbally communicate why > anything you do is important or useful to less technical users.

And on the software engineer/developer side of things the same applies. This is why whenever I am given a self-assessment or asked to help with a review of myself, I go back through my git log, email, etc. looking for what I've done instead of just attempting to summarize based on memory. Then I keep a personal copy of my self-assessment. That way, I have a record of what I did, and so does my company. Wikis, file servers, and other document repositories change, and when you switch jobs, you have that available to look at to update your resume. If your company doesn't make you do at least annual and hopefully quarterly self-assessments, you should do it on your own.

</comment></comment></comment><comment><USER6> Sorry to disagree with a lot of posts here.

>At the most basic level, when firing up a new server I follow the how to harden your server guides and install fail2ban, disable root login, enable ssh only login etc.

This is simply wrong. There are two things wrong with this approach:

1\. If I understood correctly, you are essentially following random guides on the internet about setting up your security rig. Not too far from downloading untrusted binaries from the Internet.

2\. It shows that you do not understand the core issue and what the main purpose of such tools is, how they function , etc.

This is a bad approach because it does not scale well. Security should be built from ground up, not as icing on top. You have to interact with developers during the build cycle and be the paranoid one. So, for example, when somebody mentions FTP transfers you yell at them, instead of finding a workaround (eg, setting up a box with FTP but filtering IP addresses with iptables or whatever).

Security is an architectural issue, not the issue of which tools to install.

That being said about security, being a good sysadmin for me also means striving for simplicity, documenting and standardizing everything and being meticulous.

How do you learn it? From experience. Listen to your senior colleagues and learn by doing. Also, RTFM (but seriously though, those tech notes are important).

Disclaimer: Used to be a team leader of a large UNIX/Linux team at the big blue. Now I do DevOps.

<comment><AUTHOR> You are right with the following blog posts blindly is a bad idea. And I feel guilty that I didn't invest time to learn why I commented those lines in IP conf (it's bad enough that I don't remember which files I changed). So I've definitely got a place to start working on already.

The mentorship part is a lot more difficult. Good sysadmins are really difficult to find in Sri Lanka. I've worked with a few companies so far. Some examples.

1) One company I worked for didn't even have a policy of hashed passwords and protection against SQL injection. They were developing major enterprise software.

2) I worked as an internal systems developer for a non IT team within another company. This was the one place I could have learnt the most at but the IT team was this very opaque "don't tell people what exactly we do" kind of team.

3) One last example. This other company that I worked at, the sysadmin was pretty good in keeping stuff up and running, but a lot of it was copy paste scripts. I got what I could out of the person but I couldn't pull out much.

Where I'm from, the main cyber security body of the country gave blank looks when asked about heartbleed at a conference held recently after the whole thing exploded.

All that to just sum up why I turned to HN to seek out advice as to what resources I should look at. A lot of threads on the net seem to veer more towards "be a good communicator" and "know your system". While necessary, it's a little too abstract for someone trying to find out what gaps exist and which ones need filling ASAP.

Thanks a lot for the advice. I'll probably start reading up on all those files I had to edit when hardening the server. That should provide a good starting point.

<comment><USER7> You should reach out to @arunoda ([https://twitter.com/arunoda](https://twitter.com/arunoda)), founder of Meteor Hacks. He's one of the leading voices in deployment architecture for the open source meteor.js project, and based in Sri Lanka.

<comment><AUTHOR> I have actually. He's one of my heroes in SL. He works close to where I'll be moving to soon (it's all in one IT park). Guy is very very humble. Very nice to talk to him. Also one of the few all in believers of TDD :D

</comment></comment></comment></comment><comment><USER8> At the end of the day it's important to find a mentor or join a bigger company where you can study under a more senior sysadmin.

These days things are getting more complicated, to be a good sysadmin you also have to be a good developer. Usually as good and sometimes more than the people that write the apps that you will run, maintain and love long after they have decided they have something more shiny they could be working on.

You need to be able to dissassemble and fix programs built by other developers, diagnose issues in many runtimes, understand kernel subsystems and the various issues you can run into in kernel land.

You need indepth knowledge of C, system calls and the behaviour of hypervisors and hardware. Don't believe that running on EC2 and not programming in C gives you the luxury of glossing over these fundamentals.

You will also need networking knowledge, even if you don't intend to run your own networking equipment you will still need to understand things like the TCP handshake, what and skb is and why that's important, understand the differences between select() and epoll().

Learning to be a good sysadmin is relatively easy, learning how to be a great sysadmin takes a good 10,000 hours.

<comment><USER9> I'm a sysadmin for 16 years, I have worked in big and small companies and I don't agree with a great part of this post.

I'm not a developer, I have never needed to disassemble or fix programs built by others, never needed to understand kernel subsystems or anything else kernel related (except may be how to replace a broken driver/module). I know nothing of C, I know just the basics of system calls and I've never heard of skb or select() and epoll().

I don't even like developing software, which is the reason I once joined the sysadmin team at the company I worked for and fell in love with this job.

I see how these skills are a nice extra, but I understand that knowing too much also creates the problem of having to work harder and stay more often after hours. And saying no isn't an option because nobody else knows how to handle the problems that may arise.

To me, a good sysadmin knows when to say no and spends the least effort in fixing errors in code. The developers should provide good programs or it doesn't get installed in production. Keep strict boundaries in what you do and don't to prevent being abused by your employer. If you are often the last one to leave because you're doing work that someone else should be doing, or asked to be stand by after hours while being the only sysadmin, step on the brakes before this will eventually burn you out or makes you hate your job (or even hobby).

Also, knowing all of what parent describes + all the 'normal' sysadmin knowledge is (eventually) impossible to keep up and will also take too much of your free time. You will regret this later in life.

<comment><USER10> You are literally the problem with system administration.

The attitude that you don't need to know how to program or understand the architecture of the systems that you run on is a highly privileged attitude that you are practitioner who can remain ignorant of their tools.

The good news is that kids these days are being raised on SREs and there's a billion people in India who will take your job and won't have an attitude about learning how to program. Ultimately, you are going to be a curious dinosaur. You are a unique product of the late 90s Internet bubble where the need for system administrators expanded so much faster than the available supply and anyone halfway technically minded got hired for really good salaries and put in charge of servers.

That is going to change and evolve.

And I spend 5-10 years learning all that knowledge back in the 90s and its 15 years later and I can state with confidence that there's no regrets. If anything my only regret is that I didn't learn how to really practice "software development" as opposed to "programming" even earlier. Going forwards, I expect that more an more the lines between systems and software is going to blur and _MY_ advice to kids these days would be that they will only be able to be ignorant of software development practices at their own peril.

And I stated doing PC Tech work in the late 80s, became a Unix admin in the mid-90s, managed the configuration management system and base configuration of a site that grew to 30,000 linux servers in the 200Xs, and then after 15 years switched to being a Software Developer. I'm fully confident based on experience that you are offering absolutely terrible advice to someone who is just learning, and you are out of touch with the direction of your own field.

<comment><USER11> > You are literally the problem with system administration.

No personal attacks on HN, please.

</comment></comment></comment></comment><comment><USER12> To use permissions for security, follow the principle of least access. If a user or program doesn't need access to something, don't give it to them. User permissions are the first tier of this, Apparmor and Selinux are the next (and correspondingly complex) next tier.

For example, for a web stack which communicates exclusively over the network stack, run your entire stack as individual non-root user. Nginx can run as the nginx user, django/rails/node as a separate user which has no access to the nginx configs. MySQL/PostgreSQL/Mongo as yet another user, which can't access the previous two.

By default, config files should not be owned by the process reading them, they should be part of the group (if they owned them, they could be re-written by that process).

Logging, you have a few options. Syslog is simple, and re-directable, which can help increase security. Writing to normal log files is perfectly acceptable as well, just be sure to write to a program specific directory and set up a logrotate config.

Also, set up monitoring and notification on everything you possibly can. First, know when something has gone wrong before you're notified by your customers. Then figure out how to know something will go wrong before it actually goes wrong (like running out of disk space - this happens more often than you might imagine).

There are plenty of articles on hardening linux, and sysadmin best practices on the web - I've only outlined a few here. When acting in the sysadmin role, be skeptical, paranoid, and value service uptime above everything. With this attitude, you will be more prone to automate server setup, limit user access, and less inclined to just throwing things out there because they're shiny.

The real fun is when you design and develop a way to safely and securely let your fellow developers just throw code over the fence, because then they can do their job in a way that suits them, without compromising your servers.

<comment><USER13> > Also, set up monitoring and notification on everything you possibly can.

Notifying about everything is a very, very bad idea. You'll drown under notifications that do not matter and you will certainly miss those that do matter. There should be as little notifications as possible.

<comment><USER12> I used to believe this. I was aggravated by the pager, and would happily acknowledge problems in Nagios just to make the damned thing shut up. More and more pages would come in, the volume increasing each day as "non-issues" piled up.

The third time this behavior caused downtime for my clients, I wised up and took my own advice of "uptime is king". I took the "every alert is actionable" to heart, and took a few moments to realize that the action can also be against the monitoring.

Alert for disk at 80% on a 2TB volume? Action: Verify growth in Graphite isn't out of the ordinary, and increase the warning threshold to give you about 3-4 weeks of notice that you might need to get bigger disks.

Alert for an excessive number of 404 responses? Browse the nginx logs and identify someone trying to hack your corporate-mandated WordPress install. Verify they aren't making any traction, and add exceptions to your 404's so you don't alert on known (and non-whitelisted) endpoints they're hitting.

Alert for memory at 80%? What's consuming it; do I have a memory leak and need to restart something? If all is well, and MySQL is just being greedy, up the alert to 90%.

Disk capacity warning at 3am? Put some hours around the warning notification, and add a separate "things are growing out of control" alert which doesn't have hours.

API endpoint is not responding again, but it's not the system at fault? Add the API developer to the notifications for that alert and remove yourself for a week or two.

After an admittedly harrying week of this, you gain two things. One: an operational understanding of what is going on in your system. Two: an alerting system which is tuned to your use case, and which lets you know when you have real problems. Remember - uptime is king.

</comment></comment></comment><comment><USER14> I am not a sysadmin, so probably there are better specific advice.

But, from the point of view of a developer, the thing that I appreciate the most on a fellow sysadmins is to be calm and methodic at all times. Organised. Having a plan and know what to do. Being ready for disasters.

Do we need to update a security patch on every single server? Ok, list of servers, start with server one, finish with the last one, don't let any one fall behind.

A server suddenly catches fire? No problem, remove it from the load balancer, get a fire extinguisher, remove it, order another one, recover from backup.

Is there a problem on production? What could be wrong? Check logs, think a little, then try to fix it. Do postmortem and come with improvements. Try not to be bitten again for the same thing.

In mi opinion, the core of good sysadmin is to minimise risks and errors in the stability of the system. Mistakes can (and will) happen, but the aim is to make them only once. I think it has a big component of learn from battle stories.

Processes and servers can fail, but the whole system should hold up.

<comment><USER15> Learning to be calm and methodic as a system administrator is something that comes with experience. It's a great way to tell experienced and non- experienced sysadmins apart.

As you said, if a server catches fire, the newbie sysadmin will freak out and start running around in circles screaming. The experienced sysadmin has seen it 10x before and knows its not a big issue - removes, extinguishes and replaces it.

</comment></comment><comment><USER16> I was once asked at an interview to give my three most important rules of sysadmin. This is what I said:

1\. Never enter a command if you don't understand what it does.

2\. Never make a change on a production system if you don't know how to undo it.

3\. Get everything else wrong if you have to, but get the backups right.

I realize this paints with a broad brush but it's a good baseline.

</comment><comment><USER17> Automation, documentation, and uninstallation.

A good sysadmin automates everything he can. It saves time, makes everything uniform, and reduces errors (it can multiply errors but at least they're all the same error).

A good sysadmin documents everything he does. Code, configurations, everything. You want it to be easy for not just yourself to figure out what you did, but your colleagues, customers, or your replacement.

A good sysadmin uninstalls programs that are no longer needed. He doesn't leave 50 old or unused versions of scripts laying around. Not just to save disk space or reduce system resources, but for security and to avoid confusion.

</comment><comment><USER18> [http://www.opsschool.org/en/latest/](http://www.opsschool.org/en/latest/)

<comment><USER19> Wow, I didn't know about this. Great resource, thanks for providing.

</comment></comment><comment><USER20> Not on Hacker News! Most of the people here (including myself) are developer hacks that _think_ they can do sysadmin, which is probably worse than not knowing it at all.

<comment><AUTHOR> I dunno. The replies I've got have so far pointed me in a fairly consistent direction that can't possibly leave me worse than where I am right now. What I do from there will be up to me (and anyone else who benefits from this thread) I guess.

</comment></comment><comment><USER9> F*cking up a lot is a good start ;)

When you don't understand things like user permissions, best is to make up some scenario's and write them down on paper. It makes these things easier to grasp, when you can strike through impossible options or for instance visualize routes or outcomes.

Visualizing on paper also works very good for debugging networking. For instance when a packet coming from the internet arrives at your gateway, how does it travel from there to it's final destination and how does it travel back? Write down at every hop the port/subnet/netmask/gateway for both the incoming and (if applicable) the outgoing interface. Works great for finding connection issues.

Document your infrastructure. Documenting in detail can show design flaws that weren't visible before. It also helps a lot to gain more insight in how everything is connected and how it works together.

</comment><comment><USER21> One resource I always recommend is the FreeBSD handbook, can be found here: [https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/](https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/)

<comment><AUTHOR> Thanks. Saving it to the reading list.

</comment></comment><comment><USER22> Just because Hadoop's install process sucks that doesn't make you a bad sysadmin. Back in the day when we were building our kernels most of us weren't verifying download package sigs either. Containers aren't making us dumber either.

A good sys admin just knows all of the working parts of a server and knows about the latest tools. The only way to get better is hands on practice. Try setting up your own Hadoop cluster from source and run a job through it, then put Hive on it, then attempt to containerize a node ... which not even work but learning why that won't work becomes useful knowledge. Just stumble through it and learn.

</comment><comment><USER23> Sysadmin is (unfortunately) a role that is on the decline, so from a pure employability perspective I'd suggest you focus more on the dev side.

As for the skills, I'd suggest running a Linux distro as your personal, everyday machine, not just a server you log into on AWS or DO every so often to configure (which, also -- don't do that. You don't want snowflakes in your environment). It'll force you to learn a lot about how the system actually works.

Try out a distro that doesn't hold your hand so much -- for me it was Gentoo in the early 2000s and Slackware before that. Always read the man pages. Learn all the tools for performance profiling and get used to reading your logs. Spend a lot of time learning how networking works -- maybe start with really understanding iptables which will lead you into lots of other parts of the networking stack. Read "The Design and Implementation of the FreeBSD OS" if you're interested in Unix beyond Linux.

Ultimately it's a skill that you need to learn by doing. Just don't stop the dev side of your life because as things continue to be automated and abstracted away there are going to be less and less positions as a generic sysadmin.

<comment><USER24> Why would you say that the need for Sysadmins is on the decline? If anything, I should think it would be greater than ever. I'm very curious here.

<comment><USER23> It's not that sysadmins are going to disappear, it's just that with IaaS and the automation tooling that's been developed in the past decade teams don't need to be nearly as large. The role has also changed.

A few people can manage a deployment of a thousand server instances now fairly easily (I've been on teams like that). A decade ago you'd be renting colo space, racking/stacking yourself, managing your networks, swapping dead hardware, and managing all the software that goes on top (I've also been on a team like that). You'd need a large team dedicated to just ops and sysadmining.

Hiring today is different. A sysadmin didn't necessarily need to know how to code beyond some scripting with bash or perl. These days in order to manage the complexity of large cloud systems you probably should be a solid developer in addition to having a deep knowledge of systems. Or if you're a small startup you'll probably have your devs work additionally on your infrastructure or use a PaaS.

<comment><USER25> > It's not that sysadmins are going to disappear, it's just that with IaaS and > the automation tooling that's been developed in the past decade teams don't > need to be nearly as large. The role has also changed.

The sysadmin's role is to automate themselves out of a job: you should not need to do anything twice.

For some reason, the job never disappears and new stuff keeps coming along.

(I do know one guy who successfully automated most of this job. He got bored and got a new job.)

</comment></comment></comment></comment><comment><USER26> I've been doing Linux for 16 years, no big deal.

Most valuable advice I can offer is: learn an editor and use it for everything.

OpenSSH is boss for all networking issues.

Netfilter (iptables) once understood is used daily.

And something I call "The 2s Complement": know two of everything, two distros (RPM and .deb), two shells (bash and zsh), two MTAs, two... you get the picture.

In every environment you will be able to perform then.

Use zsh with GRML completion, saves so much time.

For hacking, read all Phrack issues.

Know shell scripting and at least one other interpreted language like Python, Perl, PHP, Ruby.

Don't waste time on desktops, changing looks etc. I wasted so much time early on customising everything, great fun but no ROI whatsoever.

But the most important thing is... how to think! That that my friend takes a long time and only you can construct your own effective thought processes and algorithms.

Build terse mnemonics to aid in command options.

Oh yes, keep everything you code, script etc. for future reference and learning and improving.

Learn to code and then learn to think in code when sysadmining systems. Infrastructure as code.

Another thing, build scripts, configurations, solutions etc. as objects and reuse those objects, eg. a rsyslog configuration stanza for a UDP input that is portable and can be reused.

Sorry for waffling on I can think of so much more...

Start a blog and copy/paste your ideas and interesting work configs, code, scripts etc. there.

</comment><comment><USER27> If you want to learn to "think like a sysadmin", the book _Time Management for System Administrators_ by Limoncelli is short, funny, and excellent---and covers far more than just time management. That will give you some sense of what to care about and why. It's very useful even as a dev or devops person.

If you are really serious about being a professional sysadmin, Limoncelli's other book is considered outstanding---but it's huge and I haven't read it yet. Again, it's a mix of non-technical goals and technical solutions. The Nemeth book is how I started (though I'm not a sysadmin) and it's also quite good, but more purely technical.

I wouldn't say to a beginner, "Know what every command does," but try to learn a new thing every day. Keep notes. I like to write private man pages ([https://github.com/pjungwir/manpj](https://github.com/pjungwir/manpj)), but do what works for you.

I think learning sysadmin skills mostly just takes time, so try to make each task double as a learning opportunity. Sometimes this requires a lot of digging, or reading, or "debugging". Here I probably agree with @protomyth: the holy grail is _understanding_. If you strive to achieve that (and don't just go with the first thing that appears to work) you will become better and better.

While you're learning the nitty-gritty tech stuff, also try to keep in mind the different priorities of a sysadmin over a developer. Both care about reducing their own effort and annoyances. A sysadmin wants stability (no 3 a.m. downtime), automation (easy to deploy/update/scale), transparency (monitoring, logging), auditability (logging), recoverability (failover, backups), controllability (runbooks), security. Maybe a real sysadmin could chime in with what I'm leaving out. :-)

<comment><USER25> > Time Management for System Administrators by Limoncelli

YES. Best comment here.

</comment></comment><comment><USER28> All of the mentioned points are very good (and applicable) to performing the job properly. But for me it is a matter of curiosity. If I need a function to work, I want to know how it works. If I need an application installed, I want to know its underlying pieces, how it communicates, what dependencies it has, etc. At some point you have to draw the line of course, as it is impossible to know everything, but that line is generally easy to find. For example, I could put you to sleep talking on how TLS handshakes work, but I have no interest or desire in learning the math behind any used algorithms (although I do keep abreast of security issues with ciphers, etc.). Learning new options (say the proliferation of systemd in linux land) is always good, even if you ultimately decide not to implement it at this point in time. So just keep learning, always find new ways to do things, audit your own systems, etc.

</comment><comment><USER13> How to become good sysadmin? Understand how your system works and do everything to not interfere with it. This simple principle is the most important for a sysadmin, all the rest is an implication of it.

How to use logging (shipping, processing), how to install software (packaging systems), how to one can separate services in different ways, how to communicate with systems, how to work with resources, how to tell what's going on in OS (what's running, what uses what, what subsystems are under how much load) are all derivatives of "know your system and work with it instead of around/against it".

</comment><comment><USER29> The opposite approach might also teach you a lot: try to hack your own server (even better with the help of someone who knows a decent amount of hacking). This won't ensure you'll fully secure your server, since you can't easily reach the level of "real" hackers, but will give you a decent understanding of a hacker might approach attacking your system.

I myself are also a "pretend to be" sysadmin. Not hardcore in any way. However I use Docker for encapsulating almost all the software components, basic security (no-password ssh logins etc.), and most importantly: logging.

</comment><comment><USER30> There's a lot of good advice already in this thread, and I'd rather not just repeat it all in new phrasing. So, read it and weigh it and take what you like.

And then, when you're done, fire up a console window and unplug your mouse and put it somewhere that's really annoying to get to. Live with this for a week.

This will force you to live your life in a terminal, which means that all those little tasks become commands and scripts and configuration files. You will not have a GUI and you will have to understand how to make stuff work anyway.

Trust me, you will learn fast once you don't have a choice about it :)

</comment><comment><USER31> You have to be organized and thorough. You are going to do a lot of stuff no one appreciates or cares about unless you don't do it. (such as backups)

Organized and thorough and don't forget backups. You can never have too many backups.

cd /bin and cd /sbin and cd /usr/sbin. Do you know what every command in those directories do? No? Learn them. On some systems /sbin and /usr/sbin have different files in them. Why? Try to figure that out on your own.

cd $MANPATH, do you see more man pages that you didn't know existed? Read them.

Signs of a poor administrator: System clocks are all different; not sure if the backups work; don't know who every one who has an account on the system is; don't know what the system does; system way out of date on patches; large garbage files floating around; inconsistent and incomplete monitoring; don't know where the server is; etc...

You don't have to be a top notched system ENGINEER to be a good system ADMINISTRATOR. It helps (a lot), but I think attention to detail, thoroughness, and organization are core skills. Also, you are the interface between the machine and the human world. You. Be prepared to deal with people. Developers can get away with living with their heads buried in code. A System Administrator cannot. 90% of the problems a System Administrator faces in their job is not the technology, but the people trying to use it and manage it and mess with it and own it.

</comment><comment><USER25> A lot of the comments here are about technical detail. But knowing the technical details is just a prerequisite.

Half the job is having a sense for technologies.

* You will have projects dropped on you that you literally never heard of 24 hours before, and you will be _expected_ to be able to support them. ... and, with some experience, you will.

* People will come to you with _great new technologies!_ ... that you'll get a sense of disaster about. You will need to articulate your concerns in a manner that doesn't piss off everyone invested in the bad idea.

* Be humble, don't turn into an expert beginner. (Look that up.)

* Find other sysadmins to drink and bitch about work with. Your geekosphere will keep you balanced, help your career and be a _vital_ source of info.

Politics is the other half of the job, even though you're expected to be a consummate techie. Treat every email as a press communication, it'll be picked over like one.

* This is not a technology festival, it's an organisation that does something. Always look from that angle.

* The BOFH stories are _stories_. Nobody, anywhere, actually wants to deal with the grumpy BOFH, in any circumstance.

I have been a sysadmin for 15 years. I fully expect to be gainfully employed until age 100 if I want to, because _even in the future, nothing works_.

</comment><comment><USER32> Experienced sysadmins knows that it's very likely for a successful application to be in place for well over 5 years. They will know this from having been the person who had had to take over such systems from others, several times.

They will know that any proprietary code will have to be supportable even after the developers are gone. The systems themselves will have to be supportable by new sysadmins. The system will have to be able to receive security patches and fixes. They will know all this will be needed for multiple systems in the same company. They will know that many promising technologies will come and go. They will know how much it sucks taking over systems where these things weren't thought through. They will remember systems they have left for others which would have not have these things considered. They will remember abandoning systems themselves that became unmanageable because these things weren't considered.

Inexperienced developers and system administrators treat servers like new desktops, with desktop focussed operating systems, new software being added to it all the time, changes being made in an increasingly unrepeatable manner, and the whole thing being replaced every 2 years.

</comment><comment><USER33> I'm a Windows Server Administrator, have been for a very long time. And, if you'll permit it, I'll say I'm a damn good one. I think there are two things that have gotten me to where I am today: 1) Incessantly asking why, and 2) voracious reading and research in my lab.

I acquired some servers a while back, then got some switches, then a router, all enterprise-grade gear, and I set up some virtual servers, vswitches, etc... Basically, I made a ridiculously complicated home lab. To do so, I followed tutorials online, but at every step of the way, I asked "why is this necessary, what does this do, why do I do it this way". That gave me a great foundational understanding.

Then, I realized that certifications aren't restricted to a certain group of people, _anyone_ can take them, so I started studying for certifications. The idea was, these large governing bodies think these are the kinds of things that experts should know, so if I learn these things, I should be an expert. Right? Not quite, but reading up on each of the features, etc, that are covered in the certification exams really expanded my horizons. Then of course, I started asking Why would I use that and things just snowballed. Hear about something, ask why it is what it is and why people need/want it, then read up on it.

My home lab has been one of my greatest learning tools. Everything I've read about I have set up, installed, configured in my home lab. Then, I usually ask my brother (who also does IT) to come break something in the lab. He of course doesn't tell me what it is, and I have to go figure out what my "junior admin" did wrong and repair it.

So, I guess the TL;DR version is this: You need three things: 1) A home lab. 2) The question Why. 3) Tons and tons of reading.

</comment><comment><USER34> There's a lot of good information in this thread, but I can add this: join a SA organization and/or special interest group:

       Usenix, https://www.usenix.org/

       LOPSA, https://www.lopsa.org/

Then read the journals, even the articles that don't interest you now, and follow the email lists.

<comment><USER35> I'd like to add that LOPSA also has a mentorship program, free for all members.

</comment></comment><comment><USER36> Maybe my advises are more abstract than what you are hoping for, but this is what I learnt after 15 years being a sysadmin.

\- The good sysadmin is not the one who makes difficult things, is the one who makes the things easy. If one you find yourself wanting to recompile a Linux Kernel, probably you are taking a really bad approach for the problem.

\- Document, document, document. Think about the bus factor. If a junior sysadmin is able to rebuild and manage your infrastructure using your documentation, you are doing the things in the right way.

\- Never make an step forward without a backup plan. If you are going to make changes in the production servers, always have a plan B in case something goes wrong. Doing an `apt-get update` and hope that everything will be ok, is not a good policy.

\- Always remember that your job is to serve the other departments, so they can do awesome things. If too many people in the company knows your name, you are doing something wrong.

</comment><comment><USER37> Aside from the technical side, a good sysadmin knows that the business has priority over how cool this or that technologies are.

So every time you fix something, you're not fixing a computer, you are fixing a piece or tool of a business process. Changing your mindset will help you prioritizing the really important things.

</comment><comment><USER38> > ... being a good sysadmin ... means striving for simplicity, documenting and > standardizing everything and being meticulous.

This. And you learn by _not_ doing this and still having to maintain an infrastructure. After a while you will either start, hang yourself or get fired.

</comment><comment><USER39> A lot of comments here already address how to get into the learning side, and the theoretical side.

If you also want to get your hand dirty and learn some more that way (eg: experience), get some VPS with BSD (my preference is OpenBSD) or some really bare linux distro and set up your own email server (eg: OpenSMTPd), IM server, etc there. You'll learn plenty on the way: about servers themselves, how emailing works, management, etc, etc.

I learnt huge amounts doing this years ago, and have an extremely detailed understanding of how email, anti-spam, validation, etc work.

Don't do this "instead of" grabbing books though, do it "on top of", especially if you want to make a career out of it in future.

</comment><comment><USER40> Here's my brain dump on what goes into being a well rounded Sysadmin:

[https://sysadmincasts.com/episodes/25-bits-sysadmins- should-...](https://sysadmincasts.com/episodes/25-bits-sysadmins-should-know)

</comment><comment><USER41> Let's all take a moment to lament the sorry state of man pages in Linux.

</comment><comment><USER42> Lots of good advice about concrete skills mentioned here already - <USER2> scripting, monitoring, documenting, low level OS knowledge etc.

I reckon a good sysadmin also has a hint of nagging paranoia or slight sense of impending doom and is never complacent about how well things are currently running or how secure they seem.

On top of this they need a good instinct for anticipating problems and evaluating risks so they can proactively fix problems before they arise.

Oh yeah - learn how DNS really works.

</comment><comment><USER43> There's a lot of good advice already, but here's one tip:

Learn how to read documentation. Consult man pages and official documentation before resorting to random people on websites. This is a skill that requires practice, because a lot of the material is mediocre. Some writers give overviews, some give examples, some list every feature. You may be more comfortable with one kind, but learn how to digest each one and extract the knowledge you need.

</comment><comment><USER44> The article mentions certain things, but _every_ piece of outside software run is a potential security problem. Every piece of inside software is a potential security problem for that matter. The authors don't even have to be malicious, just careless. How many remote holes did Drupal, Joomla and Wordpress have? (A lot)

Yes, the article is right that you should not just grab some random compiled binary and throw it on your production server. It mentions Debian. I suspect Red Hat and Suse have better solutions, as their customers demand it. Of course they may not have an officially blessed package of something that was released last month.

How to be a good sysadmin? For big installations there are production servers, staging servers, development servers, and then often some unofficial development servers. You control access to the production server, the procedure to do releases is formalized. You update server firmware, OS updates and package security updates. Do it regularly on staging, QA it, then do it on production.

Most security breakins I have seen are because a non sysadmin, non security person is doing something they're not supposed to. They're running an unauthorized server on their desktop not set up by the sysadmins, with a glaring security hole. Or an outside consultant is careless about how they connect to your systems, and someone breaks in through their account.

Maybe you're a sysadmin at a web site and you notice scripts trying to hack web usernames and passwords. Your workload is high, and you bring this to the attention of the head developers and management. No one cares, the business logic management wants implemented in the short term is very high, there is no time or budget for security. So you can either end your normal work at 6:15 PM and stay another hour at work each day fixing the problem, or ignore it and go home like everyone else.

I knew some people who were on the early tiger teams for the big accounting firms. They told me their success rate was 100% - they managed to get in to the company systems every time. They also mentioned they were at a disadvantage, as they had to remain within the law (beyond the blessing of management to probe security), while others doing so would not.

Insofar as logging - syslog calls from programs go to syslogd. This can be sent to various places, including to /var/log. You can tune facilities and logging levels in the syslog configuration file. Under systemd it might be different. Do you understand what I said in this paragraph? Good, you now know more than 95% of the Unix sysadmins I've interviewed over the past 20 years. I wish I was kidding.

</comment><comment><USER45> I think you are doing a decent job. However when you want to do the same for a big(ger) company, or a bank or multinational like Shell or IBM, the expectations go up. Then your setup will not be enough. I work like you, try to learn, every day. I'm not a professional sysadmin and I don't think I would get a job when I applied for one, but by lack of the real thing, I'm doing the sysadmin job.

</comment><comment><USER46> I would recommend following Hackr's System Administration section. It's been my starting point for most sysadmin learning I have had so far (even though I am a software developer) - [http://hackr.io/tutorials/system- administration](http://hackr.io/tutorials/system-administration)

</comment><comment><USER47> The content at opsschool.org has the potential to be a great resource - [http://www.opsschool.org/en/latest/security_101.html](http://www.opsschool.org/en/latest/security_101.html) \- but there's not a lot of "there" there. Maybe you'll like it more than I do.

</comment><comment><USER48> The best sysadmins I have seen understand the Operating system from top to bottom. The easiest way to understand an OS without being a OS developer is using tracing tools to solve problems such as Dtrace under FreeBSD or Solaris (Linux also has many tracing tools). I would also recommend watching CS 162 series from Berkeley on Youtube.

</comment><comment><USER49> It's a different discipline and you don't really need to be an expert in that _and_ development; you should pick one. That's not to say you should be totally ignorant of administration details, but, really, you could devote all of your time to it if you wanted.

</comment><comment><USER50> My greybeard ISP unix boss in the late 90's taught me that if it's worth getting sued over, it's worth sending your /var/log/auth.log to a printer that can print line-by-line (a line printer).

</comment><comment><USER51> FWIW, I'm part of an 'online hackerspace' 'hashbang.sh' where we try and teach people to be better sysadmins.

[https://hashbang.sh/#](https://hashbang.sh/#)!

</comment><comment><USER52> Learn how to make basic autotools project (configure.ac, makefile.am, etc)

Learn how FreeBSD's ports system (building packages from sources) work and why it is so.

Then learn what Redhat Package Manager (RPM) is and how to make a source rpm.

Get enlightened.)

</comment><comment><USER53> As a long-time sysadmin, my best advice for learning is to give yourself the opportunity to break things. Constantly ask yourself questions about how things work and then see if you can answer them. I wouldn't even begin to start worrying about containers until you're more comfortable with the basics.

Some questions and projects to get you started:

Do you know how to start and stop services, how to tell what's running, how to see what network activity is happening?

Try installing Apache or Nginx on an EC2 instance, then see if you can change how it works. Change where it logs to, or change what port it listens on, or change what directory it pulls files from. Can you change where it reads its configuration from? What happens if you give it a bad configuration? Where does it log errors? Can you break it and then fix it?

Spin up a new EC2 instance and try to see if you can lock yourself out of it or otherwise disable it. What happens if you kill the SSH daemon? Or change the port? Or delete your user's entry in /etc/shadow? What happens if you run "rm -rf /"? What do all the fields in /etc/passwd do? What if you delete your home directory? Or "export PATH="? What happens when you delete files that a running program is using? What happens when you fill up the disk? When you run too many processes? Can you make the thing crash? When something gets screwed up, do you understand why?

Run "ls /bin" and see how many of those commands you know what they do. Pick one you don't know how to use but which you've heard of and try to figure out how to use it. Look at the man page, run the command with "-?" or "\--help". Play around with it till you feel comfortable. Then pick another one tomorrow and do the same.

Run "ls /etc" and pick a file and try to find out what it's for. See if you can do something interesting by changing the contents of the file. You might need to reboot or restart a service. Tomorrow pick something else from /etc.

Do the same with /usr/bin, /usr/lib, /var/lib. Figure out where the files in /var/log come from and how you can write to them and how you can change their names and how many there are.

Set up two instances and see if you can get them to talk to each other. On EC2 each host has two network interfaces. Do you know how to find both IPs? Can you set up MySQL on one, and connect to it from the other? Once you get that to work, can you block it? Spin up a third instance and see if you can figure out how to make MySQL accessible to one and not the other. Once you figure out one way to do it, figure out another way. Can you get a service to listen on one IP and not the other? Both?

What's the difference between UDP and TCP? Set up NFS. Set up a RAID. Break a RAID, rebuild it. Do you understand crontabs? syslog? Can you send mail to yourself?

Once you're feeling more comfortable with the environment, and you start actually fulfilling a sysadmin role, your basic philosophy should be:

* Expect anything and everything to fail.

* Trust nothing, even your own software and machines.

* Grant the least possible access to make things work. Developers and vendors will always ask for more, and if you are not pushing back against their requests, you are probably giving them too much.

* Always have a plan for rolling back a change.

* When something breaks and you fix it, don't stop working until you understand what went wrong and you take some steps to avoid it in the future.

* When you have a task that takes more than one step to complete that you do more than once, write a script to do it for you.

That should get you started...

<comment><AUTHOR> That... Was brilliant. My weekend plans seem to be pretty sorted. Thanks a lot for this. I wish I could reply with something more than just thanks but honestly, there's so much here. Just.. Thank you :)

<comment><USER53> Glad you got some good ideas. When you're just learning, if you get stuck on something and get frustrated, just drop it and move on to something else. Sometimes you just need to come back later. Sometimes you'll figure out the solution while trying to solve or learn something else. Don't get hung up on any particular problem at first.

Ultimately, good luck and have fun!

</comment></comment></comment><comment><USER54> [https://serversforhackers.com/](https://serversforhackers.com/) launched finally a while back, and I thought I'd share it here.

</comment><comment><USER55> The first step is: Install Gentoo

<comment><USER55> Some of you may want to down vote this comment, however only those who ever installed Gentoo know the deep meaning behind that sentence.

<comment><USER56> Then perhaps you could elaborate on the sentence, seeing as the thread is about helping the learning process of a sysadmin. Posting something that only someone experienced in it would "understand" is entirely unhelpful and the likely reason for any down votes.

</comment></comment></comment><comment><USER57> making a lot of hours. Especially when it comes down to debugging problems created by endusers. Making edits on servers what are in production. Every sysadmin was fooling around in the beginning.

</comment><comment><USER58> I'd suggest to crash servers while trying. :)

</comment><comment><USER59> Don't take advice from Ubuntu forums.

</comment><comment><USER60> Learn to automate. Script.

</comment><comment><USER61> You'll learn the technical concepts and rubrics eventually on your own; here is what I'd be thinking about to start:

1) It's great that you ask and seek to learn to do it right; that's a first step many don't get past.

2) There are big differences between hacking on something at home and professional system administration. Most important is cost: Your time is expensive but system downtime can be incredibly costly. [1] You need to anticipate and prevent the problem in the first place (you are the expert!), have resources prepared in case of failure (including expert knowledge of the system), and resolve it quickly. Also, you are paid as an expert to get results that boost the bottom line. You can't spend a day fiddling around with something and gratifying your curiosity.

3) There is a very wide range of knowledge and skill among sysadmins. You don't need a license to do it; anyone can print "System Administrator" on their business card. There are many, many ignorant hacks; lots of decent ones who don't think beyond what they are told; and few true experts. Who you surround yourself with will determine, to a great extent, where you fall in that range. You probably will adopt their standards and you will learn their way of doing things; you can spend your time learning either the knowledge and techniques of the hacks or those of the experts.

4) Invest the time and effort to learn core technologies [2] exhaustively and to learn best practices. Never shy away from difficult technical material; push yourself to find the best sources and develop skill to understand complex material. Most of what's on the Internet is bullsh-t from and for amatuer hackers, good enough for your home server. You can spend your career without understanding much of what you are doing -- there's enough work out there for the hacks too. Find the very best sources and people (see #3), and take the time to learn from them. Learn something once and it pays off forever.

5) Learn to solve problems. Worry less about learning techniques (e.g., arcane details of command line switches); anyone can look those up. The real challenge is staring at a screen, seeing something that you have no idea about (and which is not in books or on the Internet), and finding a way to solve it (with all the time and other pressures mentioned above). To do that, you need a model in your mind, a deep understanding, of the technologies and systems involved (see #4).

6) Put yourself in situations where you will be in command of the situation and with time in reserve for making major enhancements, learning more, and for unexpected crises; not where you are struggling to keep up with a flood problems, frantically treading water (or slowly drowning). Also choose situations where you are prepared to handle the worst-case scenarios: When the sh-t hits the fan -- when all your plans and normal operations go to hell -- people will look to you to save them. Be their hero. (See #5.)

It can be a very intellectually stimulating and gratifying job. In every field, good people and true experts are hard to find. Make yourself into one and you will be in good shape.

\</comment><comment>---

[1] Consider the cost of 1,000 people not working (avg hourly rate * 1,000 * hours), hours of orders missed, facilities shut down because your system is the bottleneck, deadlines missed, angry customers, embarassment to the business and its executives, etc.

[2] What the core technologies are for you will vary. You can't know everything. Beware of investing time learning about technologies that change quickly. TCP/IP probably will stick around for awhile; the app-of-the-week, maybe not.</comment>


Ask HN: Working with GPS Cord to find distance and clusters. - <AUTHOR>

I have been tasked with a rather interesting project, in an area where I don't really have much experience.
I was given the address of 10,000 customers and reverse geocoded (GPS Cordinates) them but now i am interested in finding clusters of customers. I really have no clue how to start other than to visually plot them on google maps api and search visually.
They are all in 1 city so they will be pretty condensed anyway but there will be clusters. How would i go about program-matically finding these clusters. I have done some google searches and nothing has really turned up.
Thanks for any help. <comment><USER2> What you want to look at is k-means clustering. It's a machine learning technique that will abstract to any number of vectors, but it will handle what you're looking for.

<http://en.wikipedia.org/wiki/K-means_clustering>

<comment><AUTHOR> Wow there is a lot of information about that algorithm on the web. Im sure ill be able to cobble something together from that. Cheers</comment></comment>


A curated list of amazingly awesome open source sysadmin resources - <AUTHOR> https://github.com/kahun/awesome-sysadmin <comment><USER2> No commits since November, 162 open PRs. :-/</comment>


Plea deal assailed as "judicial extortion" in scathing 9th Circuit dissent - <AUTHOR> http://sentencing.typepad.com/sentencing_law_and_policy/2010/06/ninth-circuit-en-banc-habeas-dissent-assails-state-judge-plea-involvement-as-judicial-extortion.html

<comment><USER2> Well yeah, isn't this basically the entire point of that part of the system? "We think we can have you locked up for XX years and know you can't afford a decent lawyer to fight it, but if you make our jobs easier we'll only have you locked up for X years instead."</comment>


Some Thoughts on Facebook Groups - <AUTHOR> http://www.25hoursaday.com/weblog/2010/10/08/SomeThoughtsOnFacebookGroups.aspx <comment><USER2> Great points from Dare Obasanjo, noting how Facebook groups may not respond particularly well to a weakness Google's been trying to exploit. His conclusion:

"Facebook Groups cranks the awkwardness of dealing with this up to 11. Let's say I create a group for "People who work on social at Microsoft who regularly have lunch" and after a few months to years some of these people leave the company, get promoted or switch roles. As the owner of the group what do I do? Do I kick them out? Do I keep blathering on in private discussions that I know are no longer relevant for half of the recipients and in some cases actually violates work ethics since some of these people have left the company? What happens when I stop working on social at Microsoft?

Facebook Groups may solve some problems users have with Facebook but I suspect it is not the silver bullet that addresses the problem of people having friend groups that they'd like to keep separate on Facebook especially since it introduces a new set of problems for users. Time will tell if I'm right or wrong on this suspicion."</comment>