title: "Error: /run/airflow doesn't exist. Can't create pidfile." when using systemd for Airflow webserver
body: <p>I have configured my Airflow setup to run with systemd according to <a href="https://github.com/apache/airflow/tree/master/scripts/systemd" rel="noreferrer">this</a>. It was great for a couple of days but it has thrown some errors that I can't figure out how to fix. Running <code>sudo systemctl start airflow-webserver.service</code> doesn't really do anything but running <code>airflow webserver</code> works (however, using systemd is needed for our purposes). </p>
  
  <p>To understand what's the error, I run <code>sudo systemctl status airflow-webserver.service</code>, and it gives the following status and error:</p>
  
  <pre><code>Feb 20 18:54:43 ip-172-31-25-17.ec2.internal airflow[19660]: [2019-02-20 18:54:43,774] {models.py:258} INFO - Filling up the DagBag from /home/ec2-user/airflow/dags
  Feb 20 18:54:43 ip-172-31-25-17.ec2.internal airflow[19660]: /home/ec2-user/airflow/dags/statcan_1410009501.py:33: SyntaxWarning: name 'pg_hook' is assigned to before global declaration
  Feb 20 18:54:43 ip-172-31-25-17.ec2.internal airflow[19660]: global pg_hook
  Feb 20 18:54:43 ip-172-31-25-17.ec2.internal airflow[19660]: /usr/lib/python2.7/site-packages/airflow/utils/helpers.py:346: DeprecationWarning: Importing 'PythonOperator' directly from 'airflow.operators' has been deprecated. Please import from 'airflow.operators.[operat...irely in Airflow 2.0.
  Feb 20 18:54:43 ip-172-31-25-17.ec2.internal airflow[19660]: DeprecationWarning)
  Feb 20 18:54:43 ip-172-31-25-17.ec2.internal airflow[19660]: /usr/lib/python2.7/site-packages/airflow/utils/helpers.py:346: DeprecationWarning: Importing 'BashOperator' directly from 'airflow.operators' has been deprecated. Please import from 'airflow.operators.[operator...irely in Airflow 2.0.
  Feb 20 18:54:43 ip-172-31-25-17.ec2.internal airflow[19660]: DeprecationWarning)
  Feb 20 18:54:44 ip-172-31-25-17.ec2.internal airflow[19660]: [2019-02-20 18:54:44,528] {settings.py:174} INFO - setting.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800
  Feb 20 18:54:45 ip-172-31-25-17.ec2.internal airflow[19660]: [2019-02-20 18:54:45 +0000] [19733] [INFO] Starting gunicorn 19.9.0
  Feb 20 18:54:45 ip-172-31-25-17.ec2.internal airflow[19660]: Error: /run/airflow doesn't exist. Can't create pidfile.
  </code></pre>
  
  <p>The scheduler seems to be working fine, as verified after running both <code>systemctl status airflow-scheduler.service</code> and <code>journalctl -f</code>.</p>
  
  <p>Here's the setup of the following systemd files:</p>
  
  <p><strong>/usr/lib/systemd/system/airflow-webserver.service</strong></p>
  
  <pre><code>[Unit]
  Description=Airflow scheduler daemon
  After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
  Wants=postgresql.service mysql.service redis.service rabbitmq-server.service
  
  [Service]
  EnvironmentFile=/etc/sysconfig/airflow
  User=ec2-user
  Type=simple
  ExecStart=/bin/airflow scheduler
  Restart=always
  RestartSec=5s
  
  [Install]
  WantedBy=multi-user.target
  </code></pre>
  
  <p><strong>/etc/tmpfiles.d/airflow.conf</strong></p>
  
  <pre><code>D /run/airflow 0755 airflow airflow
  </code></pre>
  
  <p><strong>/etc/sysconfig/airflow</strong></p>
  
  <pre><code>AIRFLOW_CONFIG= $AIRFLOW_HOME/airflow.cfg
  AIRFLOW_HOME= /home/ec2-user/airflow
  </code></pre>
  
  <p>Prior to this error, I moved my airflow installation from root to home directory. Not sure if it would have affected my setup but putting it here in case it is relevant.</p>
  
  <p>Can anyone provide any explanation for the error and how to fix it? I tried my best to configure systemd as closely as possible to what is instructed but maybe I'm missing something? </p>
  
  <p><strong>Edit 2:</strong></p>
  
  <p>Sorry, I pasted the wrong code. So this is my code for airflow-webserver.service</p>
  
  <pre><code>[Unit]
  Description=Airflow webserver daemon
  After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
  Wants=postgresql.service mysql.service redis.service rabbitmq-server.service
  
  [Service]
  EnvironmentFile=/etc/sysconfig/airflow
  User=ec2-user
  Type=simple
  ExecStart=/bin/airflow webserver --pid /run/airflow/webserver.pid
  Restart=on-failure
  RestartSec=5s
  PrivateTmp=true
  
  [Install]
  WantedBy=multi-user.target
  </code></pre>