Compatibility¶

Spark, Scala and Java¶

The Spark 4.0 artifact has also been run on Spark 4.1.2. Runs covered Java 8, 11, 17 and 21.

OpenTelemetry agent¶

Flare is an agent extension, not a library. The agent loads it into its own classloader, where it shares the agent's SDK and ByteBuddy, so the agent version matters.

• Same agent version: recommended. The extension API is published with an -alpha suffix and may change between minor releases.
• Newer agent: may work if the extension API has not changed. Test before deploying.
• Older agent: likely to fail. Flare uses extension API types that have changed across agent releases.

A mismatch usually shows up at Spark start-up as a NoSuchMethodError or ClassNotFoundException in Flare's instrumentation.

Check an agent's version:

unzip -p opentelemetry-javaagent.jar META-INF/MANIFEST.MF | grep Implementation-Version

Why the agent's dependencies are not bundled¶

Flare shares classes with the agent at runtime. Bundling its own copies of the SDK or ByteBuddy would put two versions of the same classes side by side and fail with LinkageError or ClassCastException. The one exception is the OpenTelemetry API, which the Flare JAR does bundle, unshaded: spark.plugins loads Flare into Spark's classloader, where neither Spark nor the agent provides an API, and the agent's bridge recognises the API by its real package names.