Forwarding your logs to New Relic makes all of your logging data available in one location, providing deeper visibility into both your application and your platform performance data. With your logs in one spot, you can collect, process, explore, query, and alert on errors or anomalies found in your log data.
From your host's UI, your logs are placed in context of events for the selected time period. You can drill down into detailed data for any of the highlighted attributes.
How you forward your logs depends on how you installed the infrastructure agent, since the infrastructure agent enables the log forwarding feature. You can install the infrastructure agent via:
- Guided install (recommended for most users)
- Manual installation
- Linux tarball
The Linux version of infrastructure agent, specifically version 1.42.0, has transitioned from using the td-agent-bit package to the fluent-bit package. This change was necessitated by the fact that fluent-bit is no longer distributed in the td-agent-bit flavour after the major version 2.x update.
To ensure smooth operations and provide the option to revert to td-agent-bit in the event of any issues with the fluent-bit package, the infrastructure agent now installs both packages (td-agent-bit and fluent-bit). By default, the agent is configured to use fluent-bit.
See Rollback to Fluent Bit 1.9 for more information about how to rollback.
Got lots of logs? Check out our tutorial on how to optimize and manage them.
- Infrastructure agent version 1.11.4 or higher
- Fluent Bit. The infrastructure agent already installs the latest version for you. To update or downgrade it to a specfic version, refer to the Fluent Bit installation procedures.
- OpenSSL library 1.1.0 or higher
- Built-in support for ARM64 architecture on Linux systems (for example, AWS Graviton architecture) added in infrastructure agent 1.20.6.
- Amazon Linux 2 and 2023 (ARM64 not supported for 2023)
- CentOS version 7, 8 and 9 Stream (Rocky Linux and AlmaLinux are also supported)
- RedHat version 7, 8 and 9
- Debian version 9 (Strech), 10 (Buster) and 11 (Bullseye).
- SUSE Linux Enterprise Server (SLES) version 12 and 15 (ARM64 not supported).
- Ubuntu versions 16.04.x, 18.04.x, 20.04.x, 22.04.x (LTS versions).
- Windows Server 2012, 2016, 2019, and 2022, and their service packs.
- Windows 10, Windows 11.
When you use our guided install to install the infrastructure agent, the log forwarding feature is automatically configured during the install process. If you haven't already, create your free New Relic account below to start monitoring your data today.
To initiate your install, choose your deployment method:
Configuration files direct your system to forward the log sources you want to appear in New Relic. You can add as many config files as you want. Our infrastructure agent uses
.yml files to configure logging. If you install the infrastructure agent via Add data in the UI, the file
logging.yml is created automatically.
To add a new configuration file for the log forwarding feature:
Navigate to the log forwarder configuration folder:
C:\Program Files\New Relic\newrelic-infra\logging.d\
logging.ymlconfiguration file, and add the parameters you need. The
logging.ddirectory has various
.yml.examplefiles you can use as a reference or starting point. For Windows examples, see our Github repo.# Log forwarder configuration file example# Source: file# Available customization parameters: attributes, max_line_kb, patternlogs:# Basic tailing of a single file- name: basic-filefile: /var/log/logFile.log# File with spaces in its path. No need to use quotes.- name: file-with-spaces-in-pathfile: /var/log/folder with spaces/logFile.log# Specify a list of custom attributes, as key-value pairs, to be included# in each log record- name: file-with-attributesfile: /var/log/logFile.logattributes:application: tomcatdepartment: salesmaintainer: email@example.com# Use wildcards to refer to multiple files having a common extension or# prefix. Newly generated files will be automatically detected every 60# seconds.## WARNING: avoid using wildcards that include the file extension, since# it'll cause logs to be forwarded repeatedly if log rotation is enabled.- name: log-files-in-folderfile: /var/log/logF*.log# Lines longer than 128 KB will be automatically skipped. Use 'max_line_kb'# to increase this limit.- name: log-file-with-long-linesfile: /var/log/logFile.logmax_line_kb: 256# Use 'pattern' to filter records using a regular expression- name: only-records-with-warn-and-errorfile: /var/log/logFile.logpattern: WARN|ERROR
The agent automatically processes new configuration files without having to restart the infrastructure monitoring service. The only exception to this is when configuring a custom Fluent Bit configuration.
You're required to set up a
name and log source parameter in the log forwarding
.yml config file. To start, define a
name of the log or logs you want to forward to New Relic.
What you use for the log source depends on the location your logs are sourced from. Available options for the log source include:
Although these configuration parameters aren't required, we still recommend you apply these configurations to your
logging.yml file so you get the most out of log forwarding.
Here is an example of a
logging.d/ configuration file in YAML format. For more configuration examples, see the infrastructure agent repository.
If everything is configured correctly and your data is being collected, you should see logs and related telemetry data in these places:
- Our logs UI
- In the infrastructure UI, from the host table, click the icon for a specific host, and then click View logs.
- Our tools for running NRQL queries. For example, you can execute a query like this:
SELECT * FROM Log
With the infrastructure agent installed, you can enable automatic log parsing and forwarding for our most popular on-host integrations with one step. To enable this feature, rename the
on-host-log.yml.example file to
on-host-log.yml. Once done, your integration's logs are automatically parsed and sent to New Relic.
This option is available for our supported Linux platforms.
To enable the on-host integration log forwarding feature:
Our custom Linux installation process for infrastructure monitoring allows you to tailor all aspects of the installation process, and to place files and folders on your machine. If you choose the assisted or manual tarball installation process, follow these steps to implement the log forwarder feature:
- Create the following directories:
Download and install New Relic's fluent-bit-package (RPM) by running a command similar to:bash$yum localinstall fluent-bit-<some-version>.rpm`
Download New Relic's fluentbit plugin and save it as
Download or copy the
parsers.conffile from this Github repository, and save it as
Did this doc help with your installation?
If you encounter problems with configuring your log forwarder, try these troubleshooting tips.
Explore logging data across your platform with our logs UI.
- Get deeper visibility into both your application and your platform performance data by forwarding your logs with our logs in context capabilities.
- Set up alerts.
- Query your data and create dashboards.
To disable log forwarding capabilities, go to your
logging.d directory, and remove files with the
.yml extension that were originally added during the configuration process.
C:\Program Files\New Relic\newrelic-infra\logging.d\