JAR task for jobs

Use the JAR task to deploy Scala or Java code compiled into a JAR (Java ARchive).

Important

Scala has support limitations in Unity Catalog shared access mode. See Language support for Unity Catalog shared access mode and Spark API limitations and requirements for Unity Catalog shared access mode.

Shared access mode requires an admin to add Maven coordinates and paths for JAR libraries to an allowlist. See Allowlist libraries and init scripts on shared compute.

You must store JAR files in a location supported by your compute configurations. See Java and Scala library support.

Requirements

  • You must choose a compute configuration that supports Scala workloads.

  • You must upload your JAR file to a location or Maven repository compatible with your compute configuration.

  • The code in your JAR file must use SparkContext.getOrCreate to obtain a Spark context.

Configure a JAR task

Add a JAR task from the Tasks tab in the Jobs UI by doing the following:

  1. In the Type drop-down menu, select JAR.

  2. Specify the Main class.

    • This is the full name of the class containing the main method to be executed. This class must be included in a JAR configured as a Dependent library.

  3. Use Compute to select or configure a cluster that supports the logic in your JAR.

  4. Click + Add under Dependent libraries. The Add dependent library dialogue appears.

    • You can select an existing JAR file or upload a new JAR file.

    • Not all locations support JAR files.

    • Not all compute configurations support JAR files in all supported locations.

    • Each Library Source has a different flow for selecting or uploading a JAR file. See Libraries.

  5. (Optional) Configure Parameters as a list of strings passed as arguments to the main class. See Configure task parameters.

  6. Click Save task.