LLM Observability in OpsRamp

[lsantiagos01]

Hello,

Is there any roadmap or recommended approach within OpsRamp LLM Observability?

I’m interested in understanding how the platform can be used or integrated to observe Large Language Model (LLM) workloads, including inference metrics, GPU utilization, response latency, endpoint availability, and vector/embedding behavior.

Does OpsRamp currently provide any native integration, API Poller, or customizable template that could be adapted to monitor AI or LLM inference pipelines, whether deployed locally or via providers like OpenAI, Hugging Face, or Azure AI?

Thank you,
lsantiagos01

[RaghuIyengar]

We already monitor GPU and other infra components.

Regarding LLM - I'm not aware as of now.

[RaghuIyengar]

We also provide ability to do custom Monitoring using RSE - Remote script executor - https://docs.opsramp.com/solutions/monitors/agentless-monitors/remote-script-executor/

For many public cloud workloads, we already have integration for AWS (including AWS Lex) Azure -  (Azure Machine learning etc.)

We also provide ability to integrate using Webhooks.