With our OpenAI integration, you have the ability to seamlessly monitor all OpenAI queries. By adding just two lines of code, your team can gain access to key performance metrics such as cost, response time, rate limits, error rates, chat history, and sample inputs and outputs. Our OpenAI integration provides real-time metrics to help businesses optimize usage, reduce costs, and achieve better results.
This guide will show you how to integrate your OpenAI applications so you can start monitoring your data.
Install for Python applications Install OpenAI with pip After you've completed your OpenAI import, run:
$ pip install nr-openai-observability
Set your environment variable Get your license key and set it as environment variable: NEW_RELIC_LICENSE_KEY
.
For more details and instructions, check out our REST API documentation .
$ export NEW_RELIC_LICENSE_KEY = < license key >
os . environ [ "NEW_RELIC_LICENSE_KEY" ] = "<license key>"
Your NEW_RELIC_LICENSE_KEY
can also be sent as a parameter at the monitor.initialization()
call.
Modify your code Add the following two lines to your code:
from nr_openai_observability import monitor
Add your dashboard To add the dashboard to your New Relic account:
from nr_openai_observability import monitor
openai . api_key = os . getenv ( "OPENAI_API_KEY" )
openai . Completion . create (
model = "text-davinci-003" ,
prompt = "What is Observability?" ,
Install with Node.js Install OpenAI Use npm and run:
$ npm install @newrelic/openai-observability openai
Modify your code Call monitorOpenAI
with your configured OpenAI
client.
Here is some example code:
import { Configuration, OpenAIApi } from 'openai';
import { monitorOpenAI } from '@newrelic/openai-observability';
const configuration = new Configuration({
apiKey: 'OPENAI_API_KEY',
const openAIApi = new OpenAIApi(configuration);
monitorOpenAI(openAIApi, {
applicationName: 'MyApp',
newRelicApiKey: 'NEW_RELIC_LICENSE_KEY',
const response = await openAIApi.createChatCompletion({
messages: [{ role: 'user', content: 'What is Observability?' }],
console.log(response.data.choices[0].message);
Make sure that you add module type to package.json:
Initialization options export interface MonitorOpenAIOptions {
* Your application name in New Relic, must provided.
* API key with insert access used to authenticate the request.
* For more information on creating keys, please see:
* https://docs.newrelic.com/docs/insights/insights-data-sources/custom-data/introduction-event-api#register
* Optional host override for event endpoint.
* Optional port override for trace endpoint.
EU account users If you're using an EU account, you should set your EVENT_CLIENT_HOST
as follows:
$ export EVENT_CLIENT_HOST = "insights-collector.eu01.nr-data.net"
os . environ [ "EVENT_CLIENT_HOST" ] = "insights-collector.eu01.nr-data.net"
Find your data After you've integrated OpenAI, follow these steps to find observability data:
From one.newrelic.com . Click on Model Performance . Search under Applications . See how it works If you're struggling with these instructions, we encourage you to watch this demonstration video.