如何使用 Datadog 监控 NGINX（第三篇）
NGINX 监控
![](/data/attachment/album/201508/25/002724nfcf6b76f2bfbh7r.png)


如果你已经阅读了前面的[如何监控 NGINX](/article-5970-1.html)，你应该知道从你网络环境的几个指标中可以获取多少信息。而且你也看到了从 NGINX 特定的基础中收集指标是多么容易的。但要实现全面，持续的监控 NGINX，你需要一个强大的监控系统来存储并将指标可视化，当异常发生时能提醒你。在这篇文章中，我们将向你展示如何使用 Datadog 安装 NGINX 监控，以便你可以在定制的仪表盘中查看这些指标：


![NGINX dashboard](/data/attachment/album/201508/25/002733xwwxnu5i09ivwiew.png)


Datadog 允许你以单个主机、服务、流程和度量来构建图形和警告，或者使用它们的几乎任何组合构建。例如，你可以监控你的所有主机，或者某个特定可用区域的所有NGINX主机，或者您可以监视具有特定标签的所有主机的一个关键指标。本文将告诉您如何：


* 在 Datadog 仪表盘上监控 NGINX 指标，就像监控其他系统一样
* 当一个关键指标急剧变化时设置自动警报来通知你


### 配置 NGINX


为了收集 NGINX 指标，首先需要确保 NGINX 已启用 status 模块和一个 报告 status 指标的 URL。一步步的[配置开源 NGINX](/article-5985-1.html#open-source) 和 [NGINX Plus](/article-5985-1.html#plus) 请参见之前的相关文章。


### 整合 Datadog 和 NGINX


#### 安装 Datadog 代理


Datadog 代理是[一个开源软件](https://github.com/DataDog/dd-agent)，它能收集和报告你主机的指标，这样就可以使用 Datadog 查看和监控他们。安装这个代理通常[仅需要一个命令](https://app.datadoghq.com/account/settings#agent)


只要你的代理启动并运行着，你会看到你主机的指标报告[在你 Datadog 账号下](https://app.datadoghq.com/infrastructure)。


![Datadog infrastructure list](/data/attachment/album/201508/25/002735dem7mpr1mo7ba7g6.png)


#### 配置 Agent


接下来，你需要为代理创建一个简单的 NGINX 配置文件。在你系统中代理的配置目录应该[在这儿](http://docs.datadoghq.com/guides/basic_agent_usage/)找到。


在目录里面的 conf.d/nginx.yaml.example 中，你会发现[一个简单的配置文件](https://github.com/DataDog/dd-agent/blob/master/conf.d/nginx.yaml.example)，你可以编辑并提供 status URL 和可选的标签为每个NGINX 实例：



```
init_config:

instances:

    -   nginx_status_url: http://localhost/nginx_status/
        tags:
            -   instance:foo

```

当你提供了 status URL 和任意 tag，将配置文件保存为 conf.d/nginx.yaml。


#### 重启代理


你必须重新启动代理程序来加载新的配置文件。重新启动命令[在这里](http://docs.datadoghq.com/guides/basic_agent_usage/)，根据平台的不同而不同。


#### 检查配置文件


要检查 Datadog 和 NGINX 是否正确整合，运行 Datadog 的 info 命令。每个平台使用的命令[看这儿](http://docs.datadoghq.com/guides/basic_agent_usage/)。


如果配置是正确的，你会看到这样的输出：



```
Checks
======

  [...]

  nginx
  -----
      - instance #0 [OK]
      - Collected 8 metrics & 0 events 

```

#### 安装整合


最后，在你的 Datadog 帐户打开“Nginx 整合”。这非常简单，你只要在 [NGINX 整合设置](https://app.datadoghq.com/account/settings#integrations/nginx)中点击“Install Integration”按钮。


![Install integration](/data/attachment/album/201508/25/002737yphg9qdxs69pttvq.png)


### 指标!


一旦代理开始报告 NGINX 指标，你会看到[一个 NGINX 仪表盘](https://app.datadoghq.com/dash/integration/nginx)出现在在你 Datadog 可用仪表盘的列表中。


基本的 NGINX 仪表盘显示有用的图表，囊括了几个[我们的 NGINX 监控介绍](/article-5970-1.html)中的关键指标。 （一些指标，特别是请求处理时间要求进行日志分析，Datadog 不支持。）


你可以通过增加 NGINX 之外的重要指标的图表来轻松创建一个全面的仪表盘，以监控你的整个网站设施。例如，你可能想监视你 NGINX 的主机级的指标，如系统负载。要构建一个自定义的仪表盘，只需点击靠近仪表盘的右上角的选项并选择“Clone Dash”来克隆一个默认的 NGINX 仪表盘。


![Clone dash](/data/attachment/album/201508/25/002743b9ca3h1p1iz39nn0.png)


你也可以使用 Datadog 的[主机地图](https://www.datadoghq.com/blog/introducing-host-maps-know-thy-infrastructure/)在更高层面监控你的 NGINX 实例，举个例子，用颜色标示你所有的 NGINX 主机的 CPU 使用率来辨别潜在热点。


![](/data/attachment/album/201508/25/002754jux99994ae64v984.png)


### NGINX 指标警告


一旦 Datadog 捕获并可视化你的指标，你可能会希望建立一些监控自动地密切关注你的指标，并当有问题提醒你。下面将介绍一个典型的例子：一个提醒你 NGINX 吞吐量突然下降时的指标监控器。


#### 监控 NGINX 吞吐量


Datadog 指标警报可以是“基于吞吐量的”（当指标超过设定值会警报）或“基于变化幅度的”（当指标的变化超过一定范围会警报）。在这个例子里，我们会采取后一种方式，当每秒传入的请求急剧下降时会提醒我们。下降往往意味着有问题。


1. **创建一个新的指标监控**。从 Datadog 的“Monitors”下拉列表中选择“New Monitor”。选择“Metric”作为监视器类型。


![NGINX metric monitor](/data/attachment/album/201508/25/002757dn4t65nfxg88ubt8.png)
2. **定义你的指标监视器**。我们想知道 NGINX 每秒总的请求量下降的数量，所以我们在基础设施中定义我们感兴趣的 nginx.net.request*per*s 之和。


![NGINX metric](/data/attachment/album/201508/25/002759go4bp4o448jbuh44.png)
3. **设置指标警报条件**。我们想要在变化时警报，而不是一个固定的值，所以我们选择“Change Alert”。我们设置监控为无论何时请求量下降了30％以上时警报。在这里，我们使用一个一分钟的数据窗口来表示 “now” 指标的值，对横跨该间隔内的平均变化和之前 10 分钟的指标值作比较。


![NGINX metric change alert](/data/attachment/album/201508/25/002801kv63gfr5uj0virrj.png)
4. **自定义通知**。如果 NGINX 的请求量下降，我们想要通知我们的团队。在这个例子中，我们将给 ops 团队的聊天室发送通知，并给值班工程师发送短信。在“Say what’s happening”中，我们会为监控器命名，并添加一个伴随该通知的短消息，建议首先开始调查的内容。我们会 @ ops 团队使用的 Slack，并 @pagerduty [将警告发给短信](https://www.datadoghq.com/blog/pagerduty/)。


![NGINX metric notification](/data/attachment/album/201508/25/002803c262n2rllfbhshvb.png)
5. **保存集成监控**。点击页面底部的“Save”按钮。你现在在监控一个关键的 NGINX [工作指标](https://www.datadoghq.com/blog/monitoring-101-collecting-data/#metrics)，而当它快速下跌时会给值班工程师发短信。


### 结论


在这篇文章中，我们谈到了通过整合 NGINX 与 Datadog 来可视化你的关键指标，并当你的网络基础架构有问题时会通知你的团队。


如果你一直使用你自己的 Datadog 账号，你现在应该可以极大的提升你的 web 环境的可视化，也有能力对你的环境、你所使用的模式、和对你的组织最有价值的指标创建自动监控。


如果你还没有 Datadog 帐户，你可以注册[免费试用](https://www.datadoghq.com/blog/how-to-monitor-nginx-with-datadog/#sign-up)，并开始监视你的基础架构，应用程序和现在的服务。




---


via: <https://www.datadoghq.com/blog/how-to-monitor-nginx-with-datadog/>


作者：K Young 译者：[strugglingyouth](https://github.com/strugglingyouth) 校对：[wxy](https://github.com/wxy)


本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译，[Linux中国](https://linux.cn/) 荣誉推出


OK
If you’ve already read [our post on monitoring NGINX](https://www.datadoghq.com/blog/how-to-monitor-nginx/), you know how much information you can gain about your web environment from just a handful of metrics. And you’ve also seen just how easy it is to start collecting metrics from NGINX on ad hoc basis. But to implement comprehensive, ongoing NGINX monitoring, you will need a robust monitoring system to store and visualize your metrics and logs, and to alert you when anomalies happen. In this post, we’ll show you how to set up NGINX performance monitoring and log collection in Datadog so that you can use [customizable dashboards](https://www.datadoghq.com/dashboards/nginx-dashboard/) like this:

Datadog allows you to build graphs and alerts around individual hosts, services, processes, metrics—or virtually any combination thereof. For instance, you can monitor all of your NGINX hosts, or all hosts in a certain availability zone, or you can monitor a single key metric being reported by all hosts with a certain tag. This post will show you how to:

- Monitor NGINX performance metrics on
[Datadog dashboard](https://www.datadoghq.com/dashboards/nginx-dashboard/)s, alongside all your other systems - Send NGINX logs to Datadog for monitoring and analysis
- Set up alerts to automatically notify you of unexpected changes

[Configuring NGINX](#configuring-nginx)

To collect metrics from NGINX, you first need to ensure that NGINX has an enabled status module and a URL for reporting its status metrics. Step-by-step instructions [for configuring open source NGINX](https://www.datadoghq.com/blog/how-to-collect-nginx-metrics/#metrics-collection-nginx-open-source) and [NGINX Plus](https://www.datadoghq.com/blog/how-to-collect-nginx-metrics/#metrics-collection-nginx-plus) are available in our companion post on metric collection.

[Integrating Datadog and NGINX](#integrating-datadog-and-nginx)

[Install the Datadog Agent](#install-the-datadog-agent)

The Datadog Agent is [the open source software](https://github.com/datadog/datadog-agent) that collects and reports metrics from your hosts so that you can view and monitor them in Datadog. Installing the Agent usually takes [just a single command](https://app.datadoghq.com/account/settings/agent/latest?platform=overview).

As soon as your Agent is up and running, you should see your host reporting metrics [in your Datadog account](https://app.datadoghq.com/infrastructure).

[Configure the Agent](#configure-the-agent)

Next you’ll need to create a simple NGINX configuration file for the Agent. You can find the location of the Agent’s configuration directory for integrations (**conf.d**) for your OS [here](https://docs.datadoghq.com/agent/guide/agent-configuration-files/?tab=agentv6). In that directory you will find an **nginx.d** subdirectory containing an NGINX configuration template named **conf.yaml.example**. Copy this file to **conf.yaml**, and edit it to provide the status URL and optional tags for each of your NGINX instances:

conf.yaml

```
init_config:
instances:
- nginx_status_url: http://localhost/nginx_status/
tags:
- instance:foo
```


If you’re using NGINX Plus, you’ll also need to set the `use_plus_api`

parameter to `true`

for each instance. See the documentation for further information about [configuring the Agent for NGINX Plus](https://docs.datadoghq.com/integrations/nginx/?tab=host#configuration).

[Restart the Agent](#restart-the-agent)

You must restart the Agent to load your new configuration file. Execute the restart command for your platform, as shown in the [Agent documentation](https://docs.datadoghq.com/agent/guide/agent-commands/#restart-the-agent).

[Verify the configuration settings](#verify-the-configuration-settings)

To check that Datadog and NGINX are properly integrated, run the Agent’s [ status](https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information) command and look for

`nginx`

under the `Running Checks`

section.[Install the integration](#install-the-integration)

Finally, switch on the NGINX integration inside your Datadog account. It’s as simple as clicking the “Install Integration” button under the Configuration tab in the [NGINX integration settings](https://app.datadoghq.com/account/settings#integrations/nginx).

[Metrics!](#metrics)

Once the Agent begins reporting NGINX performance metrics, you will see [an NGINX dashboard](https://app.datadoghq.com/screen/integration/21/nginx---overview) among your list of available dashboards in Datadog.

The basic NGINX dashboard displays a handful of graphs encapsulating most of the key metrics highlighted [in our introduction to NGINX monitoring](https://www.datadoghq.com/blog/how-to-monitor-nginx/).

**Collect, analyze, and alert on NGINX metrics and logs with Datadog.**

You can easily create a comprehensive dashboard for monitoring your entire web stack by adding additional graphs with important metrics from outside NGINX. For example, you might want to monitor host-level metrics on your NGINX hosts, such as system load. To start building a custom dashboard, simply clone the default NGINX dashboard by clicking on the gear near the upper right of the dashboard and selecting “Clone Dashboard”.

You can also monitor your NGINX instances at a higher level using Datadog’s [host maps](https://www.datadoghq.com/blog/introducing-host-maps-know-thy-infrastructure/)—for instance, color-coding all your NGINX hosts by CPU usage to identify potential hotspots.

[Capturing NGINX logs](#capturing-nginx-logs)

Now that you’re collecting NGINX metrics, the next step is to start collecting logs as well. This will extend your visibility beyond metrics, so you can search and filter based on facets like `Status Code`

and `URL Path`

. To show how Datadog log management works, we’ll walk through how you can configure an NGINX server to submit logs to Datadog, and we’ll see how those will look in your Datadog account.

[Enabling logging](#enabling-logging)

Log collection is disabled in the Agent by default. To enable it, edit the Agent configuration file, **datadog.yaml**. (The path to this file varies by platform. See the [Datadog Agent documentation](https://docs.datadoghq.com/agent/) for more information.) Uncomment the relevant line and set `logs_enabled`

to `true`

:

datadog.yaml

```
# Logs agent is disabled by default
logs_enabled: true
```


Next, modify the configuration of the NGINX integration. Add the following content to the `logs`

item within the Agent’s [config file for the integration](https://docs.datadoghq.com/agent/), **conf.d/nginx.d/conf.yaml**. This tells the Agent where to find the NGINX logs, and specifies some metadata to be applied:

conf.yaml

```
logs:
- type: file
path: /var/log/nginx/access.log
service: my.nginx.service
source: nginx
sourcecategory: http_web_access
```


This applies a service name of **my.nginx.service** to all the logs generated by this NGINX instance. This service tag is also applied to request traces in [Datadog APM](https://www.datadoghq.com/blog/announcing-apm/), enabling you to pivot smoothly between sources of monitoring data for your applications and infrastructure. The configuration above also applies `source`

and `sourcecategory`

tags, which you can use to [aggregate data](https://docs.datadoghq.com/getting_started/tagging/) from multiple sources and hosts.

Finally, execute the Agent’s restart command for your platform, as shown in the [Agent documentation](https://docs.datadoghq.com/agent/guide/agent-commands/#restart-the-agent).

[Use JSON logs for automatic parsing](#use-json-logs-for-automatic-parsing)

Datadog will process any log formatted as JSON [automatically](https://docs.datadoghq.com/logs/processing/), turning key-value pairs into attributes that you can use to group and filter. You can configure your NGINX access logs to [escape JSON characters](http://nginx.org/en/docs/http/ngx_http_log_module.html#log_format), making it straightforward to get your logs into Datadog. The configuration snippet below shows how you can log embedded variables from the NGINX [core](http://nginx.org/en/docs/http/ngx_http_core_module.html#variables) and [proxy](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#variables) modules. Some of these values are collected as [standard attributes](https://docs.datadoghq.com/logs/processing/attributes_naming_convention/), which help you correlate logs from disparate sources by providing consistent names for data that can appear in logs throughout your stack.

```
log_format json_custom escape=json
'{'
'"http.url":"$request_uri",'
'"http.version":"$server_protocol",'
'"http.status_code":$status,'
'"http.method":"$request_method",'
'"http.referer":"$http_referer",'
'"http.useragent":"$http_user_agent",'
'"time_local":"$time_local",'
'"remote_addr":"$remote_addr",'
'"remote_user":"$remote_user",'
'"body_bytes_sent":"$body_bytes_sent",'
'"request_time":$request_time,'
'"response_content_type":"$sent_http_content_type",'
'"X-Forwarded-For":"$proxy_add_x_forwarded_for",'
'"custom_key":"custom_value"'
'}';
```


(Note that if you want to be able to [create a measure](https://docs.datadoghq.com/logs/explorer/) from a log attribute, you should leave its value unquoted, as we’ve shown with the `request_time`

and `http.status_code`

attributes in the example above.)

Along with NGINX configuration variables, Datadog will also process the pair `"custom_key":"custom_value"`

(or any pair you choose) into an attribute, helping you organize your log data using any dimension that’s relevant to your use case. Take care not to use custom variable names that would collide with any [reserved attributes](https://docs.datadoghq.com/logs/processing/#reserved-attributes).

[Viewing log info in Datadog](#viewing-log-info-in-datadog)

You can now see information from your NGINX logs begin to appear on the [Log Explorer](https://app.datadoghq.com/logs) page in your Datadog account. To isolate this service’s logs, click **my.nginx.service** under the Service list in the sidebar.

In the example image below, the log entry shows that NGINX processed a GET request for the file **info.php** and returned a status code of 200.

In the Log Explorer, you can filter logs to see information that’s not easily discernible in the log stream. To visualize your logs, you can click the Analytics button in the top-left corner and use the facet controls in the left column to filter and aggregate your log data. In the example below, we’ve used the `URL Path`

and `Status Code`

facets to see which endpoints return the most 5xx errors.

For more information, see [our docs on exploring, searching, and graphing your logs](https://docs.datadoghq.com/logs/explore/).

[Extending NGINX logs](#extending-nginx-logs)

By default, NGINX writes logs in the [Combined Log Format](https://httpd.apache.org/docs/current/logs.html#combined). You can customize the log format, as described in the [NGINX documentation](http://nginx.org/en/docs/http/ngx_http_log_module.html#log_format). (For an example of this, see [part 2 of this series](https://www.datadoghq.com/blog/how-to-collect-nginx-metrics/#metrics-collection-nginx-logs), which shows how to add request processing time to your NGINX logs.) When you install Datadog’s NGINX integration, a [processing pipeline](https://docs.datadoghq.com/logs/processing/) is enabled to automatically parse NGINX logs. This is how the raw data in your logs is automatically extracted and processed into structured data for analysis and graphing. If you modify your NGINX log format (for example, to include request processing time), you’ll need to modify the pipeline to extract data from the new format. See the Datadog [pipeline documentation](https://docs.datadoghq.com/logs/processing/#integration-pipelines) to learn how to clone and modify the NGINX pipeline.

[Alerting on NGINX metrics and logs](#alerting-on-nginx-metrics-and-logs)

Once Datadog is capturing and visualizing your metrics and logs, you will likely want to set up some alerts to automatically notify you when there are problems.

[Alerts based on your NGINX metrics](#alerts-based-on-your-nginx-metrics)

You can create alerts based on the metric values you define as normal for your infrastructure or application. A threshold-based alert triggers when a metric exceeds a set value (such as when your dropped requests count is above zero). A change-based alert can notify you, for example, when requests per second drops by more than 30 percent.

You can also create advanced alerts based on the automatic detection of [anomalies](https://www.datadoghq.com/blog/introducing-anomaly-detection-datadog/) (metrics that fall outside historical patterns) and [outliers](https://www.datadoghq.com/blog/introducing-outlier-detection-in-datadog/) (hosts performing differently from similar hosts in your environment).

See our [documentation on metric monitors](https://docs.datadoghq.com/monitors/monitor_types/metric/) for guidance on creating these types of alerts.

[Alerts based on your NGINX logs](#alerts-based-on-your-nginx-logs)

You can build alerts around key data collected in your logs—HTTP response codes and request processing time—that aren’t available on the standard NGINX status page.

[Alerting on HTTP response code](#alerting-on-http-response-code)

NGINX logs requests that are successful (e.g., 2xx) and unsuccessful (e.g., 4xx and 5xx). Because of this, in Datadog you can [build an alert](https://app.datadoghq.com/monitors#create/log) that notifies you when your server-side error rate exceeds an acceptable limit.

[Alerting on request processing time](#alerting-on-request-processing-time)

As another example, if you extend the NGINX log format to include request processing time and modify your log pipeline to extract the additional data, you can create an alert to notify you of an unexpected uptick in slow requests.

See [Datadog’s log monitor documentation](https://docs.datadoghq.com/monitors/monitor_types/log/) to get started creating alerts based on log data.

[Improve your NGINX visibility](#improve-your-nginx-visibility)

In this post we’ve walked you through integrating NGINX with Datadog to visualize your key metrics and logs, and to notify your team when your web infrastructure shows signs of trouble.

If you’ve followed along using your own Datadog account, you should now have greatly improved visibility into what’s happening in your web environment, as well as the ability to create automated alerts tailored to your environment, your usage patterns, and the indicators that are most valuable to your organization.

If you don’t yet have a Datadog account, you can sign up for [a free trial](#) and start monitoring your infrastructure, your applications, and your services today.

*Source Markdown for this post is available on GitHub. Questions, corrections, additions, etc.? Please let us know.*