# Project Story: Proactive, AI-Driven Cooling and Power Management

## About the Project

### Inspiration and Market Validation

Our project was inspired by insights gained from the NSF ICorps course in May, where we received strong market validation for the problem we aimed to solve. To ensure we were addressing a real need, we conducted over 20 customer interviews with representatives from big tech firms, industry experts, and data center professionals who specialize in cooling and power distribution services. These interviews confirmed that the problem of inefficient cooling and power management in data centers is prevalent and impactful.

### Learning and Development

Throughout this project, we learned extensively about the transition from CPUs to GPUs in data centers and the associated challenges in cooling and power distribution. Traditional air-cooling systems are becoming inadequate as data centers shift towards liquid cooling to manage the increased heat output from GPUs. Despite this transition, even large tech companies are still experimenting with and refining liquid cooling systems. This gap in the market presents an opportunity for innovative solutions like ours.

### Building the Project

Our hackathon project aimed to create an MVP that leverages AI to make cooling and power management predictive rather than reactive. We used SLURM, a workload manager and job scheduling system, to gather data about GPU availability and job scheduling. Our system predicts when and where jobs will run and proactively triggers the cooling systems before the GPUs heat up, thereby optimizing cooling efficiency.

### Challenges Faced

We faced several challenges during the development of this project:

1. **Data Collection:** Gathering accurate and comprehensive historical job data, node availability, and job scheduling logs from SLURM was time-consuming and required meticulous attention to detail.
2. **Model Accuracy:** Building a predictive model that could accurately forecast job run times and node allocations was complex. We tested various machine learning models, including Random Forest, Gradient Boosting Machines, LSTM, and GRU, to improve prediction accuracy.
3. **Integration with Existing Systems:** Integrating our predictive system with existing data center infrastructure, which traditionally relies on reactive cooling mechanisms, required careful planning and implementation.

## Implementation Details

### Steps to Implement the Project

1. **Data Collection:**

	* **Historical Job Data:** Collect data on job submissions, including job ID, submission time, requested resources (CPU, memory, GPUs), priority, and actual start and end times.
	* **Node Data:** Gather information on node availability, current workload, and resource usage.
	* **Job Scheduling Logs:** Extract SLURM scheduling logs that detail job allocation and execution.
2. **Feature Engineering:**

	* **Create Relevant Features:** Include features such as time of submission, day of the week, job priority, resource requirements, and node state (idle, allocated).
	* **Time Series Features:** Use lag features (e.g., previous job allocations) and rolling statistics (e.g., average load in the past hour).
3. **Model Selection:**

	* **Classification Models:** Random Forest, Gradient Boosting Machines, and Logistic Regression for predicting server allocation.
	* **Time Series Models:** LSTM and GRU for predicting the time of allocation.
	* **Regression Models:** Linear Regression and Decision Trees for predicting the time until allocation.
4. **Predictive Model Approach:**

	* **Data Collection:** Gather historical scheduling data from SLURM logs, including job submissions and their attributes, node allocations, and resource usage.
	* **Feature Engineering:** Develop features related to job priority, requested resources, expected runtime, and node state.
	* **Modeling:** Use a classification approach to predict the node allocation for a job or a regression approach to predict resource allocation probabilities and select the node with the highest probability.

### Results and Benefits

By implementing this predictive cooling and power management system, we anticipate the following benefits:

1. **Increased Cooling Efficiency:** Proactively triggering cooling systems based on job predictions reduces the power required for cooling by at least 10%, resulting in significant cost savings.
2. **Extended Equipment Life:** Optimized cooling management increases the lifespan of data center equipment by reducing thermal stress.
3. **Environmental Impact:** Reducing the power required for cooling contributes to lower overall energy consumption, aligning with global sustainability goals.

### Future Plans

Post-hackathon, we plan to further refine our MVP and seek early adopters to implement this solution. The transition to GPU-based data centers is an ongoing trend, and our proactive cooling and power management system is well-positioned to address the associated challenges. By continuing to improve our predictive models and integrating more advanced AI techniques, we aim to revolutionize data center operations and significantly reduce their environmental footprint.
## Inspiration

**Machine learning** is a powerful tool for automating tasks that are not scalable at the human level. However, when deciding on things that can critically affect people's lives, it is important that our models do not learn biases. [Check out this article about Amazon's automated recruiting tool which learned bias against women.](https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G?fbclid=IwAR2OXqoIGr4chOrU-P33z1uwdhAY2kBYUEyaiLPNQhDBVfE7O-GEE5FFnJM) However, to completely reject the usefulness of machine learning algorithms to help us automate tasks is extreme. **Fairness** is becoming one of the most popular research topics in machine learning in recent years, and we decided to apply these recent results to build an automated recruiting tool which enforces fairness.

## Problem

Suppose we want to learn a machine learning algorithm that automatically determines whether job candidates should advance to the interview stage using factors such as GPA, school, and work experience, and that we have data from which past candidates received interviews. However, what if in the past, women were less likely to receive an interview than men, all other factors being equal, and certain predictors are correlated with the candidate's gender? Despite having biased data, we do not want our machine learning algorithm to learn these biases. This is where the concept of **fairness** comes in.

Promoting fairness has been studied in other contexts such as predicting which individuals get credit loans, crime recidivism, and healthcare management. Here, we focus on gender diversity in recruiting.

## What is fairness?

There are numerous possible metrics for fairness in the machine learning literature. In this setting, we consider fairness to be measured by the average difference in false positive rate and true positive rate (**average odds difference**) for unprivileged and privileged groups (in this case, women and men, respectively). High values for this metric indicates that the model is statistically more likely to wrongly reject promising candidates from the underprivileged group.

## What our app does

**jobFAIR** is a web application that helps human resources personnel keep track of and visualize job candidate information and provide interview recommendations by training a machine learning algorithm on past interview data. There is a side-by-side comparison between training the model before and after applying a *reweighing algorithm* as a preprocessing step to enforce fairness.

### Reweighing Algorithm

If the data is unbiased, we would think that the probability of being accepted and the probability of being a woman would be independent (so the product of the two probabilities). By carefully choosing weights for each example, we can de-bias the data without having to change any of the labels. We determine the actual probability of being a woman and being accepted, then set the weight (for the woman + accepted category) as expected/actual probability. In other words, if the actual data has a much smaller probability than expected, examples from this category are given a higher weight (>1). Otherwise, they are given a lower weight. This formula is applied for the other 3 out of 4 combinations of gender x acceptance. Then the reweighed sample is used for training.

## How we built it

We trained two classifiers on the same bank of resumes, one with fairness constraints and the other without. We used IBM's [AIF360](https://github.com/IBM/AIF360) library to train the fair classifier. Both classifiers use the **sklearn** Python library for machine learning models. We run a Python **Django** server on an AWS EC2 instance. The machine learning model is loaded into the server from the filesystem on prediction time, classified, and then the results are sent via a callback to the frontend, which displays the metrics for an unfair and a fair classifier.

## Challenges we ran into

Training and choosing models with appropriate fairness constraints. After reading relevant literature and experimenting, we chose the reweighing algorithm ([Kamiran and Calders 2012](https://core.ac.uk/download/pdf/81728147.pdf?fbclid=IwAR3P1SFgtml7w0VNQWRf_MK3BVk8WyjOqiZBdgmScO8FjXkRkP9w1RFArfw)) for fairness, logistic regression for the classifier, and average odds difference for the fairness metric.

## Accomplishments that we're proud of

We are proud that we saw tangible differences in the fairness metrics of the unmodified classifier and the fair one, while retaining the same level of prediction accuracy. We also found a specific example of when the unmodified classifier would reject a highly qualified female candidate, whereas the fair classifier accepts her.

## What we learned

Machine learning can be made socially aware; applying fairness constraints helps mitigate discrimination and promote diversity in important contexts.

## What's next for jobFAIR

Hopefully we can make the machine learning more transparent to those without a technical background, such as showing which features are the most important for prediction. There is also room to incorporate more fairness algorithms and metrics.
## 🌟 Inspiration

We were inspired by a friend's story about his family’s energy company in France, where they sometimes faced negative pricing—meaning they had to pay to offload surplus electricity during periods of low demand. It seemed unfair that people generating clean energy could be penalized for overproduction.

As of 2023, over 60 countries generate more than 10% of their electricity from wind energy. However, when supply exceeds demand, prices can drop below zero, forcing providers to pay others to take excess electricity. A striking example occurred in Germany in 2016, where prices fell to -130 euros per megawatt-hour.

Meanwhile, data centers consumed 416 terawatt-hours of electricity in 2023, about 3% of global power. This figure is rising, especially with the growing AI sector. We thought—why not harness that excess renewable energy for cloud computing? This idea became EcoCompute.

## ☁️ What it does

EcoCompute is a cloud computing platform that allows energy providers to monetize surplus renewable energy during periods of negative pricing. By offering affordable compute resources, it converts wasted energy into valuable computing power. Providers can deploy nodes, which dynamically allocate workloads based on real-time energy production. This gives users access to low-cost computing while helping providers avoid paying to offload excess energy.

## 🛠️ How we built it

We developed a React-based front end that operates like a cloud notebook, providing an interactive experience for users. This front end connects via WebSockets to a FastAPI server, which serves as the backbone of our application.

The server manages connections to energy provider nodes, which are orchestrated using Docker. These nodes not only handle compute tasks but also utilize real-time data to adjust workloads according to energy availability. We focused primarily on German data for wind power, which made it easier to access open-source information. This decision allowed us to build a more robust and reliable platform since Germany has a wealth of data available and is a leader in renewable energy production.

Additionally, we incorporated statistical models that predict energy production and pricing trends, allowing us to optimize the allocation of compute tasks during surplus periods. This proactive approach ensures that we maximize the use of renewable energy while providing users with efficient computing resources.

## 🚧 Challenges we ran into

Finding open-source data on renewable energy and managing WebSockets effectively were significant challenges. However, by focusing on German data, we found it easier to obtain relevant statistics and trends. We also learned that Europe, being a leader in wind energy, experiences the most negative pricing situations.

## 🏆 Accomplishments that we're proud of

We’re proud to have created a platform that turns wasted energy into useful compute power. Not only does this reduce energy waste, but it also provides an eco-friendly, cost-effective solution for compute tasks, benefiting both energy providers and users.

## 📚 What we learned

Through this project, we gained a deeper understanding of renewable energy markets, particularly the complexities of balancing energy production with computing needs. We learned that while battery storage is crucial for managing surplus energy, it can be prohibitively expensive. By focusing on regions with existing infrastructure for energy storage, such as dams in Québec, we can better manage excess energy.

## 🚀 What's next for EcoCompute

Our next step is to enhance the computing experience by allowing users to upload their own containers for execution on the platform. We plan to expand EcoCompute by partnering with more energy providers worldwide and integrating additional renewable sources like solar and hydroelectric power. Our goal is to make EcoCompute the go-to platform for turning surplus energy into sustainable cloud computing.