Skip to content

Traces do not appear in Datadog Web UI when AOT Cache is enabled, #10479

New issue
New issue
Open
Open
Traces do not appear in Datadog Web UI when AOT Cache is enabled,#10479
Labels
type: bugBug report and fix
@footaku

Description

@footaku
footaku
opened on Jan 30, 2026

Tracer Version(s)

1.58.2(, 1.57.0, 1.58.0 too)

Java Version(s)

25

JVM Vendor

Eclipse Adoptium / Temurin

Bug Report

Behavior

When the application is started with AOT Cache enabled, traces do not appear in Datadog Web UI.
Using the same container image and the same JAR, traces do appear if AOT Cache is disabled.
Also, if AOT Cache is enabled but AOT Class Linking is disabled, traces do appear.

With DD_TRACE_DEBUG=true, the following log appears much more often in the NG case than in the OK cases, but it is unclear if this is the root cause:
DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=xxx.xxx.Xxx instrumentation.target.context=yyy.yyy.Yyy->zzz.zzz.Zzz

From the logs, sending to the Datadog Agent seems successful:

[dd-trace-processor] EXCLUDE_TELEMETRY datadog.trace.agent.common.writer.ddagent.DDAgentApi - Success while sending 8 (size=6KB) traces. Total: 18, Received: 18, Sent: 18, Failed: 0.
[dd-trace-processor] DEBUG datadog.trace.agent.common.sampling.RateByServiceTraceSampler - Update service sampler rates: http://dd-agent:8126/v0.4/traces -> {rate_by_service={service:,env:=1.0}}
[dd-trace-processor] DEBUG datadog.trace.agent.common.writer.PayloadDispatcherImpl - Successfully sent 8 traces to the API

I would appreciate any guidance on potential causes or possible workarounds.

Expected Behavior

Traces appear in Datadog Web UI

Reproduction Code

Case 1 (NG)

The application starts, but traces do not appear.

java -javaagent:/dd-java-agent.jar \
  -Xlog:aot \
  -XX:AOTCache=/workspace/application.aot \
  -jar application.jar
`Failed to add context-store field` output 
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.zaxxer.hikari.pool.ProxyStatement instrumentation.target.context=java.sql.Statement->datadog.trace.bootstrap.instrumentation.jdbc.DBQueryInfo
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.zaxxer.hikari.pool.ProxyLeakTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=software.amazon.jdbc.util.monitoring.AbstractMonitor instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.zaxxer.hikari.pool.HikariPool$KeepaliveTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.zaxxer.hikari.pool.HikariPool$MaxLifetimeTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.zaxxer.hikari.pool.HikariPool$HouseKeeper instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.apache.tomcat.util.net.SocketWrapperBase$OperationState instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.apache.tomcat.util.net.Acceptor instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessorMonitor instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=io.sentry.backpressure.BackpressureMonitor instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=io.sentry.PreviousSessionFinalizer instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=io.sentry.MovePreviousSession instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=tools.jackson.databind.util.internal.PrivateMaxEntriesMap$AddTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=ch.qos.logback.classic.spi.LoggingEvent instrumentation.target.context=ch.qos.logback.classic.spi.ILoggingEvent->datadog.trace.bootstrap.instrumentation.api.AgentSpanContext
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.mysql.cj.jdbc.StatementImpl instrumentation.target.context=java.sql.Statement->datadog.trace.bootstrap.instrumentation.jdbc.DBQueryInfo
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.springframework.beans.factory.support.DisposableBeanAdapter instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.mysql.cj.jdbc.ConnectionImpl instrumentation.target.context=java.sql.Connection->datadog.trace.bootstrap.instrumentation.jdbc.DBInfo
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.mysql.cj.jdbc.AbandonedConnectionCleanupThread instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.zaxxer.hikari.pool.ProxyConnection instrumentation.target.context=java.sql.Connection->datadog.trace.bootstrap.instrumentation.jdbc.DBInfo
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=jdk.internal.net.http.common.SequentialScheduler$SchedulableTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=jdk.internal.net.http.Http1Exchange$Http1Publisher$WriteTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=sun.rmi.transport.tcp.TCPTransport$AcceptLoop instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.util.concurrent.CompletableFuture$UniCompletion instrumentation.target.context=java.util.concurrent.CompletableFuture$UniCompletion->datadog.trace.bootstrap.instrumentation.java.concurrent.ConcurrentState
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.util.concurrent.FutureTask instrumentation.target.context=java.util.concurrent.RunnableFuture->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.util.concurrent.ThreadPoolExecutor$Worker instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.util.concurrent.ThreadPoolExecutor instrumentation.target.context=java.util.concurrent.ThreadPoolExecutor->java.lang.Boolean
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.net.HttpURLConnection instrumentation.target.context=java.net.HttpURLConnection->datadog.trace.bootstrap.instrumentation.httpurlconnection.HttpUrlState
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.util.concurrent.ForkJoinTask instrumentation.target.context=java.util.concurrent.ForkJoinTask->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.lang.VirtualThread instrumentation.target.context=java.lang.VirtualThread->datadog.trace.bootstrap.instrumentation.api.AgentScope
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.zaxxer.hikari.pool.ProxyStatement instrumentation.target.context=java.sql.Statement->datadog.trace.bootstrap.instrumentation.jdbc.DBQueryInfo
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.zaxxer.hikari.pool.ProxyLeakTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=software.amazon.jdbc.util.monitoring.AbstractMonitor instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.zaxxer.hikari.pool.HikariPool$KeepaliveTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.zaxxer.hikari.pool.HikariPool$MaxLifetimeTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.zaxxer.hikari.pool.HikariPool$HouseKeeper instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.apache.tomcat.util.net.SocketWrapperBase$OperationState instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.apache.tomcat.util.net.Acceptor instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessorMonitor instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=io.sentry.backpressure.BackpressureMonitor instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=io.sentry.PreviousSessionFinalizer instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=io.sentry.MovePreviousSession instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=tools.jackson.databind.util.internal.PrivateMaxEntriesMap$AddTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=ch.qos.logback.classic.spi.LoggingEvent instrumentation.target.context=ch.qos.logback.classic.spi.ILoggingEvent->datadog.trace.bootstrap.instrumentation.api.AgentSpanContext
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.mysql.cj.jdbc.StatementImpl instrumentation.target.context=java.sql.Statement->datadog.trace.bootstrap.instrumentation.jdbc.DBQueryInfo
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=org.springframework.beans.factory.support.DisposableBeanAdapter instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.mysql.cj.jdbc.ConnectionImpl instrumentation.target.context=java.sql.Connection->datadog.trace.bootstrap.instrumentation.jdbc.DBInfo
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=com.mysql.cj.jdbc.AbandonedConnectionCleanupThread instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=jdk.internal.net.http.common.SequentialScheduler$SchedulableTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=jdk.internal.net.http.Http1Exchange$Http1Publisher$WriteTask instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=sun.rmi.transport.tcp.TCPTransport$AcceptLoop instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.util.concurrent.CompletableFuture$UniCompletion instrumentation.target.context=java.util.concurrent.CompletableFuture$UniCompletion->datadog.trace.bootstrap.instrumentation.java.concurrent.ConcurrentState
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.util.concurrent.FutureTask instrumentation.target.context=java.util.concurrent.RunnableFuture->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.util.concurrent.ThreadPoolExecutor$Worker instrumentation.target.context=java.lang.Runnable->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.util.concurrent.ThreadPoolExecutor instrumentation.target.context=java.util.concurrent.ThreadPoolExecutor->java.lang.Boolean
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.net.HttpURLConnection instrumentation.target.context=java.net.HttpURLConnection->datadog.trace.bootstrap.instrumentation.httpurlconnection.HttpUrlState
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.util.concurrent.ForkJoinTask instrumentation.target.context=java.util.concurrent.ForkJoinTask->datadog.trace.bootstrap.instrumentation.java.concurrent.State
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.lang.VirtualThread instrumentation.target.context=java.lang.VirtualThread->datadog.trace.bootstrap.instrumentation.api.AgentScope

Case 2 (OK)

Traces appear when AOT Cache is not specified.

java -javaagent:/dd-java-agent.jar \
  -Xlog:aot \
  -jar application.jar
`Failed to add context-store field` output 
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.lang.VirtualThread instrumentation.target.context=java.lang.VirtualThread->datadog.trace.bootstrap.instrumentation.api.AgentScope
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.lang.VirtualThread instrumentation.target.context=java.lang.VirtualThread->datadog.trace.bootstrap.instrumentation.api.AgentScope

Case 3 (OK)

Traces appear when AOT Class Linking is disabled.

java -javaagent:/dd-java-agent.jar \
  -Xlog:aot \
  -XX:-AOTClassLinking \
  -XX:AOTCache=/workspace/application.aot \
  -jar application.jar
`Failed to add context-store field` output 
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.lang.VirtualThread instrumentation.target.context=java.lang.VirtualThread->datadog.trace.bootstrap.instrumentation.api.AgentScope
[main] DEBUG datadog.trace.agent.tooling.context.FieldBackedContextMatcher - Failed to add context-store field - instrumentation.target.class=java.lang.VirtualThread instrumentation.target.context=java.lang.VirtualThread->datadog.trace.bootstrap.instrumentation.api.AgentScope

Added

Environment

  • Datadog
    • dd-trace-java 1.58.2 / Agent 7.73.3
  • Kubernetes (Amazon EKS)
  • Eclipse Temurin 25 
    java --version
openjdk 25 2025-09-16 LTS
OpenJDK Runtime Environment Temurin-25+36 (build 25+36-LTS)
OpenJDK 64-Bit Server VM Temurin-25+36 (build 25+36-LTS, mixed mode, sharing)
  • Spring Boot 4.0.0
    • hikariCP
    • MySQL (Amazon Aurora)

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: bugBug report and fix

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions