Ignore or include Prometheus metrics

Avoid sending Prometheus OpenMetrics integration data that is not relevant to your monitoring needs. Instead, use filters to ignore or include specific metrics. This will help you control the amount and types of data you send to New Relic. This will also help you avoid additional billing charges, as explained in this document.

Prevent billing increases

We use the Prometheus discovery and scrape annotations. If you set the Prometheus OpenMetrics integration to scrape all the available targets and to send all the data that's exposed from those targets, you may exceed New Relic's platform limits and increase your billing charges. To help prevent this from happening, use the integration's filtering capabilities.

For more information, see the Prometheus OpenMetrics integration requirements for Docker and Kubernetes. Also see the troubleshooting procedures for NrIntegrationError events.

Identify metrics to ignore or include

To decide what data to include or exclude, use New Relic's Metric API to explore your metric data. Then, refine your filters to scrape only relevant targets and send useful metrics.

• To filter out unwanted metrics from a target, use the ignore_metrics configuration option.
• To filter out targets instead of the metrics, use the scrape_enabled_label configuration option.

The nri-prometheus-latest.yaml manifest file includes the nri-prometheus-cfg config map showing an example configuration. The integration will ignore or include metrics before executing the other functions to add, rename, or copy attributes.

Filter out unwanted metrics (ignore)

To ignore unwanted metrics, use the following transformation.

Example: Configuration

To drop all metrics that start with go_ or process_:

transformations:
  - description: "General processing rules"
    ignore_metrics:
      - prefixes:
        - "go_"
        - "process_"

Example: Input

go_goroutines 13
process_virtual_memory_bytes 2.062336e+07
mysql_global_status_commands_total{command="ha_close"} 0
mysql_global_status_commands_total{command="ha_open"} 0

This is taken from the MySQL exporter. Besides the MySQL metrics, it also exposes metrics about the exporter that may not be of interest to you.

Example: Output

mysql_global_status_commands_total{command="ha_close"} 0 
mysql_global_status_commands_total{command="ha_open"} 0 

Include only specific metrics (except)

If you only want to include specific metrics, you can use the except list under the ignore_metrics section. As the name implies, this will ignore all the metrics except the ones that contain the with the given prefixes.

Example: Configuration

To drop all metrics except kube_hpa_:

transformations:
  - description: "General processing rules"
    ignore_metrics:
      - except:
        - kube_hpa_

For more help