Google Dataproc¶

Verified on Dataproc on Compute Engine (now "Managed Service for Apache Spark"), one master and two workers:

Image 2.3 runs Java 11: Flare releases before 1.3.0 export nothing there. See Upgrading.

Stage the JARs in GCS¶

Clusters are internal-IP only by default. Nodes reach Google services through Private Google Access but not the public internet, so an initialization action cannot download from GitHub or Maven Central. Put both JARs in a bucket and copy them from there:

AGENT=https://repo1.maven.org/maven2/io/opentelemetry/javaagent/opentelemetry-javaagent/2.31.1/opentelemetry-javaagent-2.31.1.jar
FLARE=https://repo1.maven.org/maven2/io/github/neutrinic/flare-spark-3-5_2.12/1.4.0/flare-spark-3-5_2.12-1.4.0.jar


curl -fsSL -o opentelemetry-javaagent.jar "$AGENT"
curl -fsSL -o flare-spark.jar "$FLARE"
gcloud storage cp opentelemetry-javaagent.jar flare-spark.jar gs://<bucket>/

Replace 3-5_2.12 with the coordinate for your image; see Downloading the JARs.

Initialization action¶

The jobs API runs the driver on the master in client mode, so --files never reaches it. An initialization action puts both JARs at a fixed path on every node:

#!/bin/bash
set -euo pipefail
mkdir -p /opt/flare
gcloud storage cp gs://<bucket>/opentelemetry-javaagent.jar /opt/flare/
gcloud storage cp gs://<bucket>/flare-spark.jar /opt/flare/
chmod 644 /opt/flare/*.jar

Submit¶

There is no classpath to append to: image 2.3 sets no extraClassPath or extraJavaOptions, and 3.0 sets only spark.executor.defaultJavaOptions, which Spark prepends on its own.

--properties-file does not parse values that contain spaces and =. Use --properties with a custom delimiter instead; ^#^ makes # the separator:

gcloud dataproc jobs submit spark --cluster <cluster> --region <region> \
  --class com.example.Main --jars gs://<bucket>/app.jar \
  --properties "^#^spark.plugins=io.flare.spark.plugin.FlareSparkPlugin#spark.driver.extraClassPath=/opt/flare/flare-spark.jar#spark.executor.extraClassPath=/opt/flare/flare-spark.jar#spark.driver.extraJavaOptions=-javaagent:/opt/flare/opentelemetry-javaagent.jar -Dotel.javaagent.extensions=/opt/flare/flare-spark.jar -Dotel.service.name=my-app-driver#spark.executor.extraJavaOptions=-javaagent:/opt/flare/opentelemetry-javaagent.jar -Dotel.javaagent.extensions=/opt/flare/flare-spark.jar -Dotel.service.name=my-app-executor"

Notes¶

• Service account. Organisations that enforce iam.automaticIamGrantsForDefaultServiceAccounts, the default for newer ones, leave the default compute service account with no roles. It needs roles/dataproc.worker.
• Dataproc's own HTTP calls. The agent traced the driver's calls to the metadata server and the GCS connector's requests, 63 to 85 separate traces per job. Flare drops these by default; see Noise.
• Reaching an external collector needs Cloud NAT on the subnet, or a collector inside the VPC.