xdocs/usermanual/realtime-results.xml (181 lines of code) (raw):
<?xml version="1.0"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to you under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<!DOCTYPE document[
<!ENTITY sect-num '15'>
<!ENTITY hellip "…" >
]>
<document prev="generating-dashboard.html" next="best-practices.html" id="$Id: realtime-results.xml 1457581 2013-03-17 22:58:08Z sebb $">
<properties>
<author email="dev@jmeter.apache.org">Apache JMeter developers</author>
<title>User's Manual: Live Statistics</title>
</properties>
<body>
<section name="§-num;. Real-time results" anchor="realtime-results">
<p>Since JMeter 2.13 you can get real-time results sent to a backend through the
<a href="component_reference.html#Backend_Listener">Backend Listener</a> using potentially any backend (JDBC, JMS, Webservice, …)
by providing a class which implements <apilink href="org/apache/jmeter/visualizers/backend/AbstractBackendListenerClient.html">AbstractBackendListenerClient</apilink>.<br/>
JMeter ships with:
<ul>
<li>a GraphiteBackendListenerClient which allows you to send metrics to a Graphite Backend.<br/>
This feature provides:
<ul>
<li>Live results</li>
<li>Nice graphs for metrics</li>
<li>Ability to compare 2 or more load tests</li>
<li>Storing monitoring data as long as JMeter results in the same backend</li>
<li>…</li>
</ul>
</li>
<li>an InfluxDBBackendListenerClient introduced in JMeter 3.2 which allows you to send metrics to an InfluxDB Backend using UDP or HTTP protocols
This feature provides:
<ul>
<li>Live results</li>
<li>Nice graphs for metrics</li>
<li>Ability to compare 2 or more load tests</li>
<li>Ability to add annotations to graphs</li>
<li>Storing monitoring data as long as JMeter results in the same backend</li>
<li>…</li>
</ul>
</li>
</ul>
In this document we will present the configuration setup to graph and historize the data in different backends:
<ul>
<li>InfluxDB setup for InfluxDBBackendListenerClient</li>
<li>InfluxDB setup for GraphiteBackendListenerClient</li>
<li>Grafana</li>
<li>Graphite</li>
</ul>
</p>
<subsection name="§-num;.1 Metrics exposed" anchor="metrics">
<subsection name="§-num;.1.1 Thread/Virtual Users metrics" anchor="metrics-threads">
<p>
Thread metrics are the following:
</p>
<dl>
<dt><code><rootMetricsPrefix>test.minAT</code></dt><dd>Min active threads</dd>
<dt><code><rootMetricsPrefix>test.maxAT</code></dt><dd>Max active threads</dd>
<dt><code><rootMetricsPrefix>test.meanAT</code></dt><dd>Mean active threads</dd>
<dt><code><rootMetricsPrefix>test.startedT</code></dt><dd>Started threads</dd>
<dt><code><rootMetricsPrefix>test.endedT</code></dt><dd>Finished threads</dd>
</dl>
</subsection>
<subsection name="§-num;.1.2 Response times metrics" anchor="metrics-response-times">
<p>Response related metrics are the following:</p>
<dl>
<dt><code><rootMetricsPrefix><samplerName>.ok.count</code></dt>
<dd>Number of successful responses for sampler name</dd>
<dt><code><rootMetricsPrefix><samplerName>.h.count</code></dt>
<dd>Server hits per seconds, this metric cumulates Sample Result and Sub results (if using Transaction Controller, "Generate parent sampler" should be unchecked)</dd>
<dt><code><rootMetricsPrefix><samplerName>.ok.min</code></dt>
<dd>Min response time for successful responses of sampler name</dd>
<dt><code><rootMetricsPrefix><samplerName>.ok.max</code></dt>
<dd>Max response time for successful responses of sampler name</dd>
<dt><code><rootMetricsPrefix><samplerName>.ok.avg</code></dt>
<dd>Average response time for successful responses of sampler name.</dd>
<dt><code><rootMetricsPrefix><samplerName>.ok.pct<percentileValue></code></dt>
<dd>Percentile computed for successful responses of sampler name. There will be one metric for each calculated value.</dd>
<dt><code><rootMetricsPrefix><samplerName>.ko.count</code></dt>
<dd>Number of failed responses for sampler name</dd>
<dt><code><rootMetricsPrefix><samplerName>.ko.min</code></dt>
<dd>Min response time for failed responses of sampler name</dd>
<dt><code><rootMetricsPrefix><samplerName>.ko.max</code></dt>
<dd>Max response time for failed responses of sampler name</dd>
<dt><code><rootMetricsPrefix><samplerName>.ko.avg</code></dt>
<dd>Average response time for failed responses of sampler name.</dd>
<dt><code><rootMetricsPrefix><samplerName>.ko.pct<percentileValue></code></dt>
<dd>Percentile computed for failed responses of sampler name. There will be one metric for each calculated value.</dd>
<dt><code><rootMetricsPrefix><samplerName>.a.count</code></dt>
<dd>Number of responses for sampler name (sum of ok.count and ko.count)</dd>
<dt><code><rootMetricsPrefix><samplerName>.sb.bytes</code></dt>
<dd>Sent Bytes</dd>
<dt><code><rootMetricsPrefix><samplerName>.rb.bytes</code></dt>
<dd>Received Bytes</dd>
<dt><code><rootMetricsPrefix><samplerName>.a.min</code></dt>
<dd>Min response time for responses of sampler name (min of ok.count and ko.count)</dd>
<dt><code><rootMetricsPrefix><samplerName>.a.max</code></dt>
<dd>Max response time for responses of sampler name (max of ok.count and ko.count)</dd>
<dt><code><rootMetricsPrefix><samplerName>.a.avg</code></dt>
<dd>Average response time for responses of sampler name (avg of ok.count and ko.count)</dd>
<dt><code><rootMetricsPrefix><samplerName>.a.pct<percentileValue></code></dt>
<dd>Percentile computed for responses of sampler name. There will be one metric for each calculated value. (calculated on the totals for OK and failed samples)</dd>
</dl>
<p>
The default <code>percentiles</code> setting on the <complink name="Backend Listener"/> is "90;95;99",
i.e. the 3 percentiles 90%, 95% and 99%.
</p>
<p>
The <a href="https://graphite.readthedocs.io/en/latest/feeding-carbon.html#step-1-plan-a-naming-hierarchy">Graphite naming hierarchy</a>
uses dot (".") to separate elements. This could be confused with decimal percentile values.
JMeter converts any such values, replacing dot (".") with underscore ("-").
For example, "<code>99.9</code>" becomes "<code>99_9</code>"
</p>
<p>
By default JMeter sends metrics for all samplers accumulated under the samplerName "<code>all</code>".
If the Backend Listener <code>samplersList</code> is configured, then JMeter also sends the metrics
for the matching sample names unless <code>summaryOnly=true</code>
</p>
</subsection>
</subsection>
<subsection name="§-num;.2 JMeter configuration" anchor="jmeter-configuration">
<p>
To make JMeter send metrics to backend add a <a href="./component_reference.html#Backend_Listener" >BackendListener</a> using the InfluxDBBackendListenerClient.
</p>
<figure width="902" height="341" image="backend_listener.png">InfluxDB configuration</figure>
</subsection>
<subsection name="§-num;.3 InfluxDB configuration" anchor="influxdb_db_configuration">
<p>Do one of the following to store data sent by the Backend Listener:
<ul>
<li>For InfluxDB 2 setup, create a <code>jmeter</code> <a href="https://v2.docs.influxdata.com/v2.0/organizations/buckets/create-bucket/" target="_blank">bucket</a></li>
<li>For InfluxDB 1.x setup, create a <code>jmeter</code> database using the <a href="https://docs.influxdata.com/influxdb/v1.8/introduction/get-started/" target="_blank">Influx CLI</a></li>
</ul>
You can also use the HTTP API i.e.
<code>curl -i -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE jmeter"</code>
</p>
<subsection name="§-num;.3.1 InfluxDB setup for InfluxDBBackendListenerClient" anchor="influxdb">
<p>InfluxDB is an open-source, distributed, time-series database that allows to
easily store metrics.
Installation and configuration is very easy, read this for more details <a href="https://docs.influxdata.com/influxdb/latest/introduction/installation/" target="_blank" >InfluxDB documentation</a>.<br/>
InfluxDB data can be easily viewed in a browser through <a href="http://grafana.org/" target="_blank">Grafana</a>.
</p>
</subsection>
<subsection name="§-num;.3.2 InfluxDB 2 setup for InfluxDBBackendListenerClient" anchor="influxdb_v2">
<p>
The configuration should specify the <code>influxdbToken</code> parameter and also specify <code>bucket</code> and <code>org</code> as query parameters in the <code>influxdbUrl</code>. See the <a href="https://v2.docs.influxdata.com/v2.0/api/#operation/PostWrite" target="_blank">InfluxDB v2 API</a> for more details.
</p>
<p>
How to retrieve the required information in the InfluxDB UI:
<ul>
<li><a target="_blank" href="https://v2.docs.influxdata.com/v2.0/security/tokens/view-tokens/">influxdbToken</a></li>
<li><a target="_blank" href="https://v2.docs.influxdata.com/v2.0/organizations/buckets/view-buckets/">bucket</a></li>
<li><a target="_blank" href="https://v2.docs.influxdata.com/v2.0/organizations/view-orgs/">org</a></li>
</ul>
</p>
<figure width="882" height="428" image="backend_listener_influxdb_v2.png">InfluxDB 2 configuration</figure>
</subsection>
</subsection>
<subsection name="§-num;.4 Grafana configuration" anchor="grafana_configuration">
<p>
Installing grafana<br/>
Read <a href="https://docs.grafana.org/" target="_blank">documentation</a> for more details.
Add the <a href="https://docs.grafana.org/features/datasources/influxdb/" target="_blank">datasource</a><br/>
</p>
Here is the kind of dashboard that you could obtain:
<figure width="1265" height="581" image="grafana_dashboard.png">Grafana dashboard</figure>
</subsection>
<subsection name="§-num;.5 Graphite Configuration" anchor="graphite_configuration">
<p>To make JMeter send metrics to backend, add a BackendListener using the GraphiteBackendListenerClient.</p>
<p><a href="./component_reference.html#Backend_Listener">GraphiteBackendListenerClient</a> section will help you do the configuration.</p>
<figure width="902" height="341" image="backend_listener_graphite.png">Graphite configuration</figure>
</subsection>
<subsection name="§-num;.5.1 Graphite Sender" anchor="graphite_sendor">
<p>Two types of Senders are available. TextGraphiteMetricsSender, PickleGraphiteMetricsSender</p>
<ul>
<li>For plaintext protocol, set graphiteMetricsSender parameter to <code>org.apache.jmeter.visualizers.backend.graphite.TextGraphiteMetricsSender</code></li>
<li>For pickle protocol, set graphiteMetricsSender parameter to <code>org.apache.jmeter.visualizers.backend.graphite.PickleGraphiteMetricsSender</code></li>
</ul>
<p>To send large amounts of data, use the Pickle sender. It is a more efficient transmission method compared to textplain.
Read <a href="https://graphite.readthedocs.io/en/latest/feeding-carbon.html" target="_blank">the Graphite documentation</a> for more details.</p>
<figure width="902" height="341" image="backend_listener_graphite_pickle.png">Graphite pickle sender</figure>
</subsection>
</section>
</body>
</document>