Enter fullscreen mode

Introduction

Currently, we are working on many products infused with AI across the company. Instead of relying solely on a harness to provide AI agents with tools, context, memory, and other capabilities to make them smarter and more aware of the environments in which they operate, we also need the ability to evaluate these agents and provide feedback so they can continuously improve.

We have researched and experimented with several AI evaluation platforms, including Arize Phoenix, Microsoft Foundry, and Langfuse. After evaluating them against our requirements and roadmap, we shortlisted Microsoft Foundry and Langfuse because they fit well with our company's direction.

Let's start with Langfuse in this post, and we will cover Microsoft Foundry in a future post.

With Langfuse, there is currently no official SDK specifically designed to instrument OpenTelemetry data and ingest it into Langfuse. Fortunately, the OpenTelemetry community is working actively to extend the existing specification with Generative AI semantic conventions.

This is significant for us because it means we can use the official OpenTelemetry SDKs for different languages — such as Node.js, .NET, Java, Rust, and Go — to instrument our applications and send telemetry data to our chosen destination, in this case, Langfuse.

However, we still need a few tips and tricks to make the integration work properly. These are the details we will cover in this post.

High-level architecture of example app

Look at the Dashboard (Aspire) and Langfuse components at the bottom of the above picture; we notice that the code in Weather Station Agent and Weather MCP Server needs to push telemetry info to both Dashboard and Langfuse. And to make it work, we need to modify the ingest code:

var langfuseOtlpEndpoint = Environment.GetEnvironmentVariable("LANGFUSE_OTLP_ENDPOINT");
var langfuseOtlpHeaders  = Environment.GetEnvironmentVariable("LANGFUSE_OTLP_HEADERS") ?? "";

builder.Services.AddOpenTelemetry()
    .ConfigureResource(r => r.AddService("weather-station-agent"))
    .WithTracing(t =>
    {
        t.AddSource("WeatherStationAgent")
         .AddSource("Experimental.ModelContextProtocol")
         .AddSource("Experimental.Microsoft.Extensions.AI")
         .AddHttpClientInstrumentation()   // ← keep outgoing HTTP spans
         .AddOtlpExporter();  // → Aspire dashboard (OTEL_EXPORTER_OTLP_* env vars)
        if (!string.IsNullOrEmpty(langfuseOtlpEndpoint))
            t.AddOtlpExporter(o =>
            {
                o.Endpoint = new Uri(langfuseOtlpEndpoint.TrimEnd('/') + "/v1/traces");
                o.Headers  = langfuseOtlpHeaders;
                o.Protocol = OtlpExportProtocol.HttpProtobuf;
            });
    });

Then, in the apphost.cs, we need to set up some environment variables:

var weatherStationAgent = builder.AddProject("weather-station-agent", "MafLangfuseClient/MafLangfuseClient.csproj")
    .WithHttpEndpoint(port: 5002)
    .WithEnvironment("ASPNETCORE_ENVIRONMENT", "Development");

var langfuseHost = GetEnv(dotEnv, "LANGFUSE_HOST", "http://localhost:3000");
var otlpEndpoint = $"{langfuseHost.TrimEnd('/')}/api/public/otel";
var langfuseAuth = Convert.ToBase64String(
    System.Text.Encoding.UTF8.GetBytes(
        $"{GetEnv(dotEnv, "LANGFUSE_PUBLIC_KEY", "")}:{GetEnv(dotEnv, "LANGFUSE_SECRET_KEY", "")}"));
var otlpHeaders = $"Authorization=Basic {langfuseAuth},x-langfuse-ingestion-version=4";

// ...

weatherStationAgent
    .WithReference(weatherMcp)
    .WithEnvironment("MCP_ENDPOINT", weatherMcp.GetEndpoint("http"))
    .WithEnvironment("LANGFUSE_OTLP_ENDPOINT", otlpEndpoint)
    .WithEnvironment("LANGFUSE_OTLP_HEADERS", otlpHeaders)
    .WithEnvironment("OTEL_SERVICE_NAME", "weather-station-agent")
    .WithEnvironment("OTEL_RESOURCE_ATTRIBUTES", "service.name=weather-station-agent")
    .WithEnvironment("OPENAI_BASE_URL", GetEnv(dotEnv, "OPENAI_BASE_URL", "http://localhost:4000/v1"))
    .WithEnvironment("OPENAI_API_KEY", GetEnv(dotEnv, "OPENAI_API_KEY", "placeholder-key"))
    .WithEnvironment("OPENAI_MODEL", GetEnv(dotEnv, "OPENAI_MODEL", "gpt-4o-mini"));

builder.Build().Run();

With some of the official OpenTelemetry NuGet packages:

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Agents.AI" />
    <PackageReference Include="Microsoft.Agents.AI.OpenAI" />
    <PackageReference Include="Microsoft.Extensions.AI.OpenAI" />
    <PackageReference Include="ModelContextProtocol" />
    <PackageReference Include="OpenTelemetry" />
    <PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" />
    <PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" />
    <PackageReference Include="OpenTelemetry.Instrumentation.Http" />
    <PackageReference Include="OpenTelemetry.Extensions.Hosting" />
  </ItemGroup>

</Project>

Nothing is really special or customised here; we just use very basic and standard OpenTelemetry NuGet packages.

Test it all

> git clone git@github.com:langfuse/langfuse.git
> cd langfuse
# Make sure you have Docker Desktop or OrbStack running on your box
> docker compose up -d

Open the browser, make sure that you can access http://localhost:3000, create a test project and test user, and then you are all set.

Now, open another terminal, go to the root of the .NET app, then type:

> aspire run
# Wait for the app to start

Go to http://localhost:5002, you should see:

Click the Run Text Scenario button and wait a bit:

Look at the link in the red box above: http://localhost:3000/trace/385617c79a20fc8b6355321a738c0ce3. Click this link; it will bring you to the langfuse dashboard:

Click the 385617c79a20fc8b6355321a738c0ce3 trace; you will be taken to:

Check the screen; you can see that the agent prompt, cost, number of tokens used, LLM model, and all traces are there. Now you are ready to do the work on Langfuse. Happy coding.