openlayer.llm_monitors.AzureOpenAIMonitor#

class openlayer.llm_monitors.AzureOpenAIMonitor(client=None)#

Monitor inferences from Azure OpenAI LLMs and upload traces to Openlayer.

Parameters:
clientopenai.AzureOpenAI

The AzureOpenAI client.

Examples

Let’s say that you have a GPT model you want to monitor. You can turn on monitoring with Openlayer by simply doing:

  1. Set the environment variables:

export AZURE_OPENAI_ENDPOINT=<your-azure-openai-endpoint>
export AZURE_OPENAI_API_KEY=<your-azure-openai-api-key>
export AZURE_OPENAI_DEPLOYMENT_NAME=<your-azure-openai-deployment-name>

export OPENLAYER_API_KEY=<your-openlayer-api-key>
export OPENLAYER_PROJECT_NAME=<your-project-name>

  1. Instantiate the monitor:

>>> from opemlayer import llm_monitors
>>> from openai import AzureOpenAI
>>>
>>> azure_client = AzureOpenAI(
>>>     api_key=os.environ.get("AZURE_OPENAI_API_KEY"),
>>>     api_version="2024-02-01",
>>>     azure_endpoint=os.environ.get("AZURE_OPENAI_ENDPOINT"),
>>> )
>>> monitor = llm_monitors.OpenAIMonitor(client=azure_client)

  1. Use the Azure OpenAI model as you normally would:

From this point onwards, you can continue making requests to your model normally:

>>> completion = azure_client.chat.completions.create(
>>>     model=os.environ.get("AZURE_OPENAI_DEPLOYMENT_NAME"),
>>>     messages=[
>>>         {"role": "system", "content": "You are a helpful assistant."},
>>>         {"role": "user", "content": "How are you doing today?"},
>>>     ]
>>> )

The trace of this inference request is automatically uploaded to your Openlayer project.

Methods

get_cost_estimate(num_input_tokens, ...)

Returns the cost estimate for a given model and number of tokens.

monitor_thread_run(run)

Monitor a run from an OpenAI assistant.

start_monitoring()

(Deprecated) Start monitoring the OpenAI assistant.

stop_monitoring()

(Deprecated) Stop monitoring the OpenAI assistant.