By Greg Kalapos
Elastic APM supports OpenTelemetry at multiple levels. One well-understood scenario that we 've blogged about before is straight Open Telemetry Protocol (OTLP) support in an APM server. This means you can connect any OpenTelemetry agent to the Elastic APM server, the APM server will happily fetch that data, pull it into Elasticsearch®, and you can view that OpenTelemetry data in your APM application in Kibana®.
This blog post will show a different use case: In Elastic APM, we have our own APM agent . Some of them have reached tens of millions of downloads, and some of them predate OpenTelemetry. Of course, we realize that OpenTelemetry is important and here to stay, so we want to make these agents OpenTelemetry compatible.
Today, most of our Elastic APM agents are capable of sending OpenTelemetry spans as part of a trace. This means that if you have any component in your application that emits OpenTelemetry spans, it will be part of the trace captured by the Elastic APM agent. This can be a library you use that is already instrumented by the OpenTelemetry API, or any other OpenTelemetry span that the application developer added to the application code for manual instrumentation.
This feature of the Elastic APM agent not only reports on these spans, but also correctly maintains the parent-child relationship between all spans, making OpenTelemetry a first-class citizen of these agents. For example, if an Elastic APM agent initiates a span for a specific operation via auto-detection, and then the OpenTelemetry API initiates another span within that span, the OpenTelemetry span will be a child of the outer span created by the agent. This is reflected in the span's parent.id field. The reverse is also true: if the OpenTelemetry API creates a span, and within that span, the Elastic APM agent captures another span, the span created by the Elastic APM agent will be a child of the span created by the OpenTelemetry API.
This functionality exists in the following agents:
Capturing OpenTelemetry spans in Elastic .NET APM agent
As a first example, let's take an ASP.NET Core application. We put the .NET Elastic APM agent into this application, and then turned on the feature, which automatically bridges the OpenTelemetry spans, so the Elastic APM agent will make those spans part of the trace it reports.
The following code snippet shows a controller:
namespace SampleAspNetCoreApp.Controllers
{
public class HomeController : Controller
{
private readonly SampleDataContext _sampleDataContext;
private ActivitySource _activitySource = new ActivitySource("HomeController");
public HomeController(SampleDataContext sampleDataContext) => _sampleDataContext = sampleDataContext;
public async Task<IActionResult> Index()
{
await ReadGitHubStars();
return View();
}
public async Task ReadGitHubStars()
{
using var activity = _activitySource.StartActivity();
var httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Add("User-Agent", "APM-Sample-App");
var responseMsg = await httpClient.GetAsync("https://api.github.com/repos/elastic/apm-agent-dotnet");
var responseStr = await responseMsg.Content.ReadAsStringAsync();
// …use responseStr
}
}
}The Index method calls the ReadGitHubStars method, after which we simply return the appropriate view from that method.
The Elastic APM proxy automatically catches incoming and outgoing HTTP calls from HttpClient—this is part of the auto-instrumentation we've been using for a long time.
ReadGitHubStars is where we use the OpenTelemetry API. OpenTelemetry in .NET uses ActivitySource and Activity API. The _activitySource.StartActivity() call simply creates an OpenTelemetry span that automatically gets the name of the method by using the CallerMemberNameAttribute C# language feature, and the span ends when the method finishes running.
Also, within this span, we use the HttpClient type to call the GitHub API. For this type, the .NET Elastic APM proxy again provides auto-detection, so HTTP calls will also be automatically caught as spans by the proxy.
The following is the flow diagram of the transaction in Kibana:
As you can see, the agent is able to capture the OpenTelemetry range as part of the trace.
Bridging OpenTelemetry spans in Python using the Python Elastic APM agent
Let's see how this works in Python. The idea is the same, so all the concepts introduced earlier apply to this example as well.
Let's take a very simple Django example:
from django.http import HttpResponse
from elasticapm.contrib.opentelemetry import Tracer
import requests
def index(request):
tracer = Tracer(__name__)
with tracer.start_as_current_span("ReadGitHubStars"):
url = "https://api.github.com/repos/elastic/apm-agent-python"
response = requests.get(url)
return HttpResponse(response)The first step in enabling capture of OpenTelemetry spans in Python is to import the Tracer implementation from elasticapm.contrib.opentelemetry.
Then on this Tracer, you can start a new span -- in this case, we manually named the span ReadGitHubStars.
Similar to the previous example, the call to http://127.0.0.1:8000/otelsample/ is caught by the Elastic APM Python agent, then the next span is created by the OpenTelemetry API, as you can see, it is automatically caught by the agent, and finally HTTP calls to the GitHub API are caught again by the proxy's auto-detection.
Here's what it looks like in the flow diagram:
As mentioned earlier, the broker maintains a parent-child relationship for all OTel scopes. Let's look at the parent.id of the GET api.github.com call:
As you can see, the span's id is c98401c94d40b87a.
If we look at the span.id of the ReadGitHubStars OpenTelemetry Span, then we can see that the Span's id is exactly c98401c94d40b87a — so the APM Agent internally maintains a parent-child relationship across OpenTelemetry and non-OpenTelemetry Spans, which makes OpenTelemetry Spans part of the Elastic APM Agents Waiting for citizens.
other languages
At this point, I'll stop duplicating the exact same example code in other languages - I think you get the point: In each of the languages listed above, our Elastic APM agents are able to bridge OpenTelemetry traces and display them in Kibana in as a native span. We also blogged about using the same API in Java , and you can see examples for the rest of the languages in the corresponding agent docs (linked above).
When to use this feature and when to use pure OpenTelemetry SDK
It's really up to you. If you just want to use pure OpenTelemetry in your application, and really want to avoid any vendor-dependent software, then feel free to use the OpenTelemetry SDK directly - this is a use case we explicitly support. If you go that route, then this feature is less relevant to you.
However, our Elastic APM agents already have a very large user base, and they provide functionality that does not exist in OpenTelemetry. Some of these features include span compression , centralized configuration , inferred spans , distributed tail sampling using multiple APM servers, and more.
If you are one of the many existing Elastic APM Agent users, or plan to use Elastic APM Agent because of the features described above, bridging the OpenTelemetry span enables you to still use the OpenTelemetry API without relying on any vendor-specific API usage. This way, your development team can instrument your application with OpenTelemetry, and you can use any third-party libraries that OpenTelemetry has integrated with, and the Elastic APM agents will happily report these spans as part of their reported traces. This way, you can combine the vendor independence of OpenTelemetry and still use the feature-rich Elastic APM agent.
The OpenTelemetry bridge feature is also a great tool if you wish to change your telemetry library from Elastic APM Agent to OpenTelemetry (or vice versa), as it allows you to use both libraries at the same time and switch them with an atomic change.
Next step
In this blog post, we discussed how to bridge OpenTelemetry spans with Elastic APM agents. Of course, OpenTelemetry is more than just tracing. We know this and plan to cover more ground: we are currently bridging OpenTelemetry metrics in our Elastic APM agent in a very similar way. You can check the progress here .
Learn more about adding Elastic APM as part of your Elastic Observability deployment .
原文：How to combine OpenTelemetry instrumentation with Elastic APM Agent features | Elastic Blog