title: Notify for alerts : context deadline exceeded (possible proxy issue)
body: <p>I'm trying to connect an Alermanager server on ubuntu in EC2 AWS to Slack, but I reicieve this error:</p>
  
  <pre><code>Apr 14 18:22:26 prometheus-db-v01-01a.myserver.com alertmanager[5854]: level=error ts=2019-04-14T18:22:26.658601495Z caller=dispatch.go:177 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="Post &lt;redacted&gt;: context deadline exceeded"
  </code></pre>
  
  <p>my alertmanager.yum looks like:</p>
  
  <pre><code>templates:
  - '/etc/alertmanager/template/slack.tmpl'
  
  route:
    receiver: slack_general
    repeat_interval: 5m
    group_by: [alertname]
    routes:
  # severity=info alerts will not try to match to any other rule
      - match:
          severity: info
        receiver: slack_general
  
  receivers:
  - name: slack_general
    slack_configs:
    - api_url: https://hooks.slack.com/services/ID
      send_resolved: true
      username: 'Prometheus-bot'
      channel: '#errors'
      title: '{{ template "slack.my.title" . }}'
      text: '{{ template "slack.my.text" . }}'
  </code></pre>
  
  <p>I can see the alert firing my test alert with:</p>
  
  <pre><code>root@prometheus-db-v01-01a:~# amtool alert --alertmanager.url=http://localhost:9093 -v
  Alertname         Starts At                Summary
  Cassandra_yellow  2019-04-14 18:11:56 UTC  The cassandra  cluster is in yellow state
  </code></pre>
  
  <p>I started prometheus with systemd:</p>
  
  <pre><code>root@prometheus-db-v01-01a:~# cat /etc/systemd/system/alertmanager.service
  [Unit]
  Description=Prometheus Alertmanager Service
  Wants=network-online.target
  After=network.target
  
  [Service]
  Environment=https_proxy=http://proxy:80/
  Environment=http_proxy=http://proxy:80/
  User=alertmanager
  Group=alertmanager
  Type=simple
  ExecStart=/usr/local/bin/alertmanager \
      --config.file /etc/alertmanager/alertmanager.yml \
      --storage.path /var/lib/alertmanager/data
  Restart=always
  
  [Install]
  WantedBy=multi-user.target
  root@prometheus-db-v01-01a:~#
  </code></pre>
  
  <p>I tryed to set the proxy with the 2 <code>Environment</code> variable, but same result</p>
  
  <p>The server uses a proxy to communicate with Slack, and I can test it with:</p>
  
  <pre><code>root@prometheus-db-v01-01a:~# curl -X POST --data-urlencode "payload={\"channel\": \"#errors\", \"username\": \"webhookbot\", \"text\": \"This is posted to #errors and comes from a bot named webhookbot.\", \"icon_emoji\": \":ghost:\"}" https://hooks.slack.com/services/ID
  </code></pre>
  
  <p>that works, while </p>
  
  <pre><code>root@prometheus-db-v01-01a:~# curl -X POST --noproxy "*" --data-urlencode "payload={\"channel\": \"#errors\", \"username\": \"webhookbot\", \"text\": \"This is posted to #errors and comes from a bot named webhookbot.\", \"icon_emoji\": \":ghost:\"}" https://hooks.slack.com/services/ID
  curl: (7) Failed to connect to hooks.slack.com port 443: Connection timed out
  </code></pre>
  
  <p>All the above commands are from the alertmanger server.</p>
  
  <p>Am I right to think that is a proxy issue? How can I set it? </p>
  
  <p>Thanks in advance</p>