Back in 2012, Facebook experimented on you. For three months prior to Election Day, the social media company set out to manipulate the emotions of its U.S. users, showing 1.9 million randomly selected Facebookers more news stories in their newsfeeds. The result of this so-called “voter megaphone” tool, according to the social media network, was “measurably increased civic engagement and voter turnout”—a similar result to a previous experiment in which Facebook “nudged” up turnout by around 340,000 people when it created an “I voted” badge for users to reveal to their friends.

What made this “voter megaphone” effect different than, say, running a televised or radio ad campaign was its invisibility. Simply put, users didn’t know they were Facebook’s guinea pigs—and they were shocked. They were shocked again this past June, when the social media network revealed that it had manipulated over half a million newsfeeds to manipulate its users’ emotions. Facebook later apologized for “any anxiety it caused.”

Why were we getting upset? some commentators asked—and it was a good question. Not only had we agreed to this kind of thing under Facebook’s Terms and Conditions, but the company’s experimenting was no different from the sort of user-testing carried out every day by other big tech giants. Google, for instance, once user-tested 41 different shades of blue in an effort to find an interface element people were most likely to click on. The search leviathan similarly tweaks its search algorithms on a regular basis, promoting or demoting content in the process according to what it thinks you will be interested in. Google’s homepage rarely changes to clue users in that these alterations have been made.

Milennials grow up soaked in this sort of catered-to-you technology. We think it empowers us—and we’re right about that, to some degree. But in our thirst to share photos, to trade stocks and make restaurant reservations online, we cannot overlook the darker side to the tech boom: It has made us slaves to the algorithm—the underlying codes that govern the computer age. Is it possible that we live in a world in which Facebook keeps us connected, Google flatters us with tailored search results and Amazon knows what books and music we’re likely to enjoy without realizing the extent to which algorithms rule our lives?

As it happens, companies like Google and Facebook are just the tip of the iceberg. Today algorithms carry out 70 percent of trades on the U.S. stock market. Tomorrow, who’s to say they won’t be helping us pick political leaders in the same way that Amazon’s formulas suggest which products we are likely to buy: “You liked Mitt Romney, you might also like such-and-such.” Before that happens, we need to start asking the tough questions about what algorithms are and what purpose they are designed to achieve—for just because they are “scientific” certainly doesn’t mean they’re neutral. If Facebook’s algorithms can convince a certain percentage of users to vote, couldn’t they just as easily convince a certain subset of users not to?

***

The first thing we must do it shake off the widespread belief that algorithms are neutral, there to give us unbiased, objective results.

Trust in the impartiality of perceived science is far from a recent development. In his 1886 essay “The Ethic of Freethought,” the English statistical mathematician Karl Pearson argued that science’s “single-eyed devotion to truth [admits] no interested motive, no working to support a party, an individual, or a theory; such action but leads to the distortion of knowledge, and those who do not seek truth from an unbiased standpoint are, in the theology of freethought, ministers in the devil’s synagogue.” More than a century on, in 1989, the artificial intelligence philosopher Peter Slezak announced the dawning of a (not unrelated) age of “scientific discovery by computer … totally isolated from all social and cultural factors whatever.”

Today’s billion-dollar tech giants have picked up this objective ball and run with it. In a manifesto entitled “Ten Things We Know to Be True,” Google claims, “Our users trust our objectivity.” And Google has a point, too. Very few of us stop to consider the fact that the tools that deliver search results in an average of 0.48 seconds have agendas of their own. But the reality is that of course they do—and these agendas can reflect everything from what Google’s own engineers consider important to the preferences expressed by individual users.

Take something as simple as search results. Many Politico readers will be familiar with the concept of the “filter bubble,” the idea that due to personalized search services, Google will frequently produce two very different sets of search results for two different users searching for the same thing. In one famous example, the liberal who types “BP” into his or her Web browser is likely to receive information about the April 2010 oil spill in the Gulf of Mexico, while a conservative typing the same two letters is more likely to get investment information about the oil company.

Similarly, algorithms are more likely to respond to a female search for “wagner” by directing the user towards sites about the composer “Richard Wagner,” while the same search by a male is taken as meaning “Wagner USA” paint supplies. As such, what is presented by search algorithms is not a formula designed to give ideologically untampered answers but precisely the opposite: search results that flatter our personal mythologies by reinforcing what we already “know” about particular issues, while also downgrading in importance the concerns that do not marry up to our existing worldview.

Although this type of targeting is probably more about enhancing UX (the User Experience) than overtly political, the end result is often quite partisan. Take, for instance, the story of African-American Harvard University PhD, Latanya Sweeney. Searching on Google one day, Sweeney was shocked to notice that her search results were accompanied by adverts asking, “Have you ever been arrested?” These ads did not appear for her white colleagues. Sweeney began a study that ultimately demonstrated that the machine-learning tools behind Google’s search were being inadvertently racist, by linking names more commonly given to black people to ads relating to arrest records. A similar revelation is the fact that Google’s Android app store recommender system suggests users who download Grindr, a location-based social-networking tool for gay men, also download a sex-offender locating app. In both of these cases, are we to assume that the algorithm has made an error, or that they are revealing in-built prejudice on the part of their makers? Or, as is more likely, are they revealing distasteful large-scale cultural associations between—in the former case—black people and criminal behavior and—in the latter—homosexuality and predatory behavior?

This type of profiling doesn’t simply begin and end on your Google results page: As they become more powerful, more far-reaching—primary decision makers instead of support tools, algorithms are beginning to control more than just what we see on our screens—from our economy (as was the case earlier this year when a Hong Kong-based venture capital firm tasked an algorithm with making crucial decisions about which companies to invest in) to our laws.

Take predictive policing, for instance. Envisioned as a way to make law enforcement proactive rather than reactive, predictive policing is currently being rolled out around the world. It makes use of recent advances in real-time crime data recording and utilizes this for accurate forecasting. As it turns out, crimes don’t occur randomly within a particular area, but rather exist in small geographical clusters, known as “hotspots.”

In Seattle, for example, crime data gathered over a 14-year period shows that half of all crime can be isolated to just 4.5 percent of the city’s streets. A similar observation holds true for Minneapolis, Minnesota, where 3.3 percent of all street addresses generate more than 50 percent of calls to police. Over a 28-year stretch, a marginal 8 percent of streets in Boston accounted for a massive 66 percent of street robberies. Being able to use computer science to search for patterns in this data makes it possible to anticipate, prevent and respond more effectively to crimes that will take place in the future. If that all sounds a bit too Minority Report—referring to the 2002 Steven Spielberg sci-fi movie in which crime is all but wiped out thanks to a specialized psychic “PreCrime” police department—it is still very effective. During one of the first deployments of predictive policing, in Los Angeles, the trial period resulted in a 36 percent drop in crime rate. The algorithm’s predictions about which crimes were likely to take place and where were twice as accurate as those made by a human analyst.

With an area like law, we are convinced—often correctly—that algorithms can do a better job than we can. No matter how loudly we scream and shout about robots taking over the workplace, we humans regularly have more faith in computers than in our own reasoning. “We are all so scared of human bias and inconsistency,” explains Danielle Citron, professor of Law at University of Maryland.

As a result, the creation of laws themselves will increasingly reflect algorithmic reasoning. “Ambient Law” refers to the idea that laws can be both embedded within and enforced by our smart devices and environment. Like the human body, where temperature, breathing and heart rate are regulated without our having to be consciously aware of what is happening, so too is the dream for algorithms to self-manage, self-configure and self-optimize—without the need for physical or mental input on the part of users. As technologies like on-board computers and Google’s self-driving cars (which have now completed 700,000 autonomous miles on open roads) gain prominence, questions related to speeding, road safety and similar legal conundrums become enshrined in the form of algorithms. For example, an in-car system developed by Japanese carmaker Nissan monitors users from the moment they get behind the wheel, looking for signs of drunk-driving. To do this it checks the driver’s breath with an alcohol odor sensor, tests for alcohol in the sweat of the driver’s palm as he or she touches the gear stick, and looks for signs of drowsiness through a miniature dashboard camera. Based on an averaging of these biometrics, the car’s algorithms then decide whether the person in the driver’s seat is safe to drive. If not, the transmission locks, a “drunk-driving” alert sounds, and the seatbelt tightens to shock the person out of his or her stupor. Whether a person is fit to drive moves from a self-imposed standard to a computerized rule.

Similar algorithmic decision makers now play a part in deciding which prisoners are allowed out on parole, who should be given permission to board an airplane, whether assembled crowds are benign or dangerous and whether a person is eligible for welfare or other benefits. In all of these cases, politics are built directly into the codes that govern our lives.

But while allowing previously subjective ideas to be coded (or “codified”) into massive system-wide architectures is an extremely useful development, it carries with it a host of potential challenges. As slashed budgets have led to staff cuts across a variety of different sectors, automated systems have moved from simple administrative tools to primary decision-makers. In some instances, the result is a lack of transparency that obscures decision-making and reveals only the (incontestable) final decision. Shades of this problem were evident several years back in the Colorado Benefits Management System disaster, in which hundreds of incorrect rules and illegal simplifications were accidentally written into the system’s codes. When Amazon messes up its algorithms, you might order a book you end up hating; but when it comes to public policy, simplified or erroneous code can have enormous consequences. One mistake can expunge legitimate voters from electoral rolls without notice, incorrectly label small businesses as ineligible for government contracts or mistakenly identify individuals as “deadbeat” parents. There are precedents for all of these dystopian situations. And while humans are certainly capable of perpetuating these same errors, what is different with algorithms is the scale: A human might make dozens of errors, while a formula can make hundred of thousands, millions, even billions.

Worse, if we assume results churned out by an algorithm to be objective, we might not even realize there are errors. Back in 2012, the Washington Post developed a so-called “TruthTeller” algorithm to coincide with that summer’s presidential-election season. The algorithm scanned through transcripts of political speeches in near real-time and pattern-matched the “facts” to a database of previously recorded, proven facts. The purpose behind the algorithm, to catch politicians in their lies, was noble enough. But the idea we would stop doing our own thinking in favor of the polygraph version of a spellcheck is disconcerting.

So what can we do about the algorithmization of everything? It seems clear that we are at a signal point in history. Algorithms can carry out complex jobs like medical diagnosis and legal work often better than humans can—and almost certainly for less money. But they also have the ability to nudge us in directions we might not choose for ourselves, whether by recommending new products for consumption, opening up and shutting down certain avenues of exploration or reminding us that it’s Election Day.

As more of our lives are governed by hidden formulae, algorithms and neural networks, we need to seek understanding of these tools and, where sensible, to push for transparency. And as jobs continue to be lost to automation, its time we ask ourselves what the value of human work actually is—what it is that we can do better than expert systems and machine learning tools and what it is that, for all the reasons discusses above, we don’t want to relinquish to formulas and codes.

Algorithms allow us to navigate through the 2.5 quintillion bytes of data that are generated each day (a million times more information than the human brain is capable of holding) and draw actionable conclusions from it. They are some of the most powerful tools shaping our lives in the 21st century. It’s of vital importance that we understand them—and that we realize that whatever else they might be, they’re certainly not free of politics.

As technology historian Melvin Kranzberg says: Technology is neither good nor bad, but it’s certainly not neutral.