integration_test/third_party_apps_test/applications/nginx/metadata.yaml (273 lines of code) (raw):
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
public_url: "https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent/third-party/nginx"
app_url: "https://nginx.org/"
short_name: nginx
long_name: nginx
logo_path: /images/partners/nginx.png # supplied by google technical writer
description: |-
The nginx integration collects connection metrics and access logs. Connection
metrics capture the current state of the connection: active, reading, or
waiting. Access logs are parsed for their connection details, which include
fields mapped to request, client, server, and message.
configure_integration: |-
You must enable the `stub_status` module in the nginx configuration file to set
up a locally reachable URL, for example, `http://www.example.com/nginx_status`
for the status page. To enable the `stub_status` module, complete the following
steps:
1. Edit the `status.conf` file, or create the file if it doesn't exist. You can
find this file in the nginx configuration directory, typically found at
`/etc/nginx/conf.d`.
1. Add the following lines to the `server` section:
```none
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
deny all;
}
```
Your configuration file might look like the following example:
<pre>
server {
listen 80;
server_name 127.0.0.1;
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
deny all;
}
location / {
root /dev/null;
}
}
</pre>
1. Reload the nginx configuration.
```
sudo service nginx reload
```
You can run the following command to automate the previous steps. It creates
a `status.conf` file if it doesn't exist or overwrites the existing one if it
does exist. The command turns on `stub_status`, reloads nginx, and verifies
that the expected information is exposed through the endpoint.
```bash
sudo tee /etc/nginx/conf.d/status.conf > /dev/null << EOF
server {
listen 80;
server_name 127.0.0.1;
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
deny all;
}
location / {
root /dev/null;
}
}
EOF
sudo service nginx reload
curl http://127.0.0.1:80/nginx_status
```
The sample output is:
<pre>
Active connections: 1
server accepts handled requests
23 23 74
Reading: 0 Writing: 1 Waiting: 0
</pre>
Note: `127.0.0.1` can be replaced with the real server name, for example,
`server_name mynginx.domain.com`. The curl command to verify would be
`curl http://mynginx.domain.com:80/nginx_status`.
Alternately, instead of using a separate `status.conf` file, you can also
directly embed the lines to the main `nginx.conf` file, which
is typically located in one of the following directories: `/etc/nginx`,
`/usr/local/nginx/conf`, or `/usr/local/etc/nginx`.
minimum_supported_agent_version:
metrics: 2.1.0
logging: 2.1.0
supported_operating_systems: linux
# TODO: Fix errors and enable tests on all platforms.
platforms_to_skip:
- suse-cloud:sles-12 # QueryLog() failed: nginx_access not found, exhausted retries; QueryLog() failed: nginx_error not found, exhausted retries
- suse-cloud:sles-15 # QueryLog() failed: nginx_access not found, exhausted retries; QueryLog() failed: nginx_error not found, exhausted retries
- suse-cloud:sles-15-arm64 # QueryLog() failed: nginx_access not found, exhausted retries; QueryLog() failed: nginx_error not found, exhausted retries
- ubuntu-os-cloud:ubuntu-2004-lts # QueryLog() failed: nginx_access not found, exhausted retries; QueryLog() failed: nginx_error not found, exhausted retries
supported_app_version: ["1.18", "1.20"]
expected_metrics:
- type: workload.googleapis.com/nginx.connections_accepted
value_type: INT64
kind: CUMULATIVE
monitored_resources: [gce_instance]
- type: workload.googleapis.com/nginx.connections_current
value_type: INT64
kind: GAUGE
monitored_resources: [gce_instance]
labels:
- name: state
value_regex: .*
- type: workload.googleapis.com/nginx.connections_handled
value_type: INT64
kind: CUMULATIVE
monitored_resources: [gce_instance]
- type: workload.googleapis.com/nginx.requests
value_type: INT64
kind: CUMULATIVE
monitored_resources: [gce_instance]
representative: true
expected_logs:
- log_name: nginx_access
fields:
- name: httpRequest.remoteIp
value_regex: (::1|127\.0\.0\.1)
type: string
description: "The IP address (IPv4 or IPv6) of the client that issued the HTTP request. This field can include port information. Examples: 192.168.1.1, 10.0.0.1:80, FE80::0202:B3FF:FE1E:8329."
optional: true
- name: httpRequest.requestUrl
value_regex: .*/forbidden.html
type: string
optional: true
description: "The scheme (http, https), the host name, the path and the query portion of the URL that was requested. Example: http://example.com/some/info?color=red."
- name: httpRequest.responseSize
type: string
description: ''
optional: true
- name: httpRequest.userAgent
type: string
description: ''
optional: true
- name: httpRequest.referer
type: string
description: ''
optional: true
- name: httpRequest.requestMethod
value_regex: GET
type: string
description: ''
optional: true
- name: httpRequest.status
type: string
description: ''
optional: true
- name: jsonPayload.host
type: string
description: Contents of the Host header (usually not reported by nginx)
optional: true
- name: jsonPayload.user
type: string
description: Authenticated username for the request
optional: true
- name: jsonPayload.level
type: string
description: Log entry level
optional: true
- name: severity
type: string
description: ''
- log_name: nginx_error
fields:
- name: jsonPayload.client
value_regex: (::1|127\.0\.0\.1)
type: string
description: Client IP address (optional)
- name: jsonPayload.level
value_regex: error
type: string
description: Log entry level
optional: true
- name: jsonPayload.message
value_regex: .*Permission denied.*
type: string
description: Log message
- name: jsonPayload.request
value_regex: GET /forbidden.html.*
type: string
description: Original HTTP request (optional)
- name: jsonPayload.pid
type: number
description: The process ID issuing the log
- name: jsonPayload.tid
type: number
description: Thread ID where the log originated
- name: jsonPayload.connection
type: number
description: Connection ID
- name: jsonPayload.server
type: string
description: Nginx server name (optional)
optional: true
- name: jsonPayload.subrequest
type: string
description: Nginx subrequest (optional)
optional: true
- name: jsonPayload.upstream
type: string
description: Upstream request URI (optional)
optional: true
- name: jsonPayload.host
type: string
description: Host header (optional)
optional: true
- name: jsonPayload.referer
type: string
description: Referer header (optional)
optional: true
- name: severity
type: string
description: ''
configuration_options:
logs:
- type: nginx_access
fields:
- name: type
default: null
description: This value must be `nginx_access`.
- name: include_paths
default: '[/var/log/nginx/access.log]'
description: A list of filesystem paths to read by tailing each file. A wild card (`*`) can be used in the paths.
- name: exclude_paths
default: null
description: A list of filesystem path patterns to exclude from the set matched by `include_paths`.
- name: record_log_file_path
default: false
description: If set to `true`, then the path to the specific file from which the log record was obtained appears in the output log entry as the value of the `agent.googleapis.com/log_file_path` label. When using a wildcard, only the path of the file from which the record was obtained is recorded.
- name: wildcard_refresh_interval
default: 60s
description: The interval at which wildcard file paths in `include_paths` are refreshed. Given as a [time duration](https://pkg.go.dev/time#ParseDuration), for example `30s` or `2m`. This property might be useful under high logging throughputs where log files are rotated faster than the default interval.
- type: "nginx_error"
fields:
- name: type
default: null
description: This value must be `nginx_error`.
- name: include_paths
default: '[/var/log/nginx/error.log]'
description: A list of filesystem paths to read by tailing each file. A wild card (`*`) can be used in the paths.
- name: exclude_paths
default: null
description: A list of filesystem path patterns to exclude from the set matched by `include_paths`.
- name: record_log_file_path
default: false
description: If set to `true`, then the path to the specific file from which the log record was obtained appears in the output log entry as the value of the `agent.googleapis.com/log_file_path` label. When using a wildcard, only the path of the file from which the record was obtained is recorded.
- name: wildcard_refresh_interval
default: 60s
description: The interval at which wildcard file paths in `include_paths` are refreshed. Given as a [time duration](https://pkg.go.dev/time#ParseDuration), for example `30s` or `2m`. This property might be useful under high logging throughputs where log files are rotated faster than the default interval.
metrics:
- type: nginx
fields:
- name: type
default: null
description: This value must be `nginx`.
- name: server_status_url
default: http://localhost/status
description: The URL exposed by the nginx stub status module.
- name: collection_interval
default: 60s
description: A [time duration](https://pkg.go.dev/time#ParseDuration) value, such as `30s` or `5m`.
troubleshoot: |-
On most distributions, nginx comes with `ngx_http_stub_status_module` enabled.
You can check if the module is enabled by running the following command:
```
sudo nginx -V 2>&1 | grep -o with-http_stub_status_module
```
The expected output is `with-http_stub_status_module`, which means the module
is enabled. In rare cases, if the command returns no output, you must compile
nginx from source with the `-with-http_stub_status_module` following the nginx
public documentation.