Databricks Runtime 10.2 (EoS)

Note

Support for this Databricks Runtime version has ended. For the end-of-support date, see End-of-support history. For all supported Databricks Runtime versions, see Databricks Runtime release notes versions and compatibility.

The following release notes provide information about Databricks Runtime 10.2 and Databricks Runtime 10.2 Photon, powered by Apache Spark 3.2.0. Databricks released this version in December 2021. Photon is in Public Preview.

Note

These release notes may include references to features that are not available on Google Cloud as of this release.

New features and improvements

Databricks Utilities adds an update mount command

The new dbutils.fs.updateMount command is similar to dbutils.fs.mount, but this new command updates an existing mount point instead of creating a new one. See updateMount command (dbutils.fs.updateMount).

Delta Lake supports renaming of columns (Public Preview)

Delta Lake now supports column renaming. To rename the column of a Delta table, you must upgrade your Delta table’s version and then set the column mapping mode to name mapping. See Rename and drop columns with Delta Lake column mapping.

Delta Lake table history adds Databricks Runtime version information

When a write is now made to a Delta table, information about the Databricks Runtime version that made the write is also recorded. To get the version information, run the DESCRIBE HISTORY command on the table and then see the engineInfo column in the output. See DESCRIBE HISTORY and Work with Delta Lake table history.

Databricks Utilities adds latest modification time information when directories are listed

The dbutils.fs.ls command for listing the contents of a directory now outputs a modificationTime field. This field contains the latest modification time for each listed file or directory. See ls command (dbutils.fs.ls).

Spark SQL adds an approx_top_k function to return top items and their counts

The new approx_top_k function returns the most frequently occurring item values, along with their approximate counts, for the specified expression. Some examples include:

df = spark.sql('SELECT approx_top_k(col, 10, 100) FROM VALUES (0), (1), (1), (2), (2), (2) AS tab(col)')

display(df)

import pyspark.sql.functions as F

df = spark.createDataFrame([
       (0,), (1,), (1,), (2,), (2,), (2,)
     ]).select(F.expr('approx_top_k(_1, 10)'))

display(df)

The result of running either example is:

[{"item": 2, "count": 3}, {"item": 1, "count": 2}, {"item": 0, "count": 1}]

See approx_top_k aggregate function.

Trigger.AvailableNow now available in Python as well as Scala

This change supports the Delta source streaming queries Trigger.AvailableNow in Python as well as Scala. Databricks Runtime 10.1 introduced support for this trigger in Scala. Databricks Runtime 10.2 now supports this trigger in Python as well. See coverage of Trigger.AvailableNow in What is Auto Loader?.

Library upgrades

  • Upgraded Python libraries:

    • filelock from 3.3.1 to 3.4.0

    • plotly from 5.3.0 to 5.3.1

  • Upgraded R libraries:

    • base from 4.1.1 to 4.1.2

    • compiler from 4.1.1 to 4.1.2

    • datasets from 4.1.1 to 4.1.2

    • graphics from 4.1.1 to 4.1.2

    • grDevices from 4.1.1 to 4.1.2

    • grid from 4.1.1 to 4.1.2

    • methods from 4.1.1 to 4.1.2

    • parallel from 4.1.1 to 4.1.2

    • Rserve from 1.8-8 to 1.8-10

    • splines from 4.1.1 to 4.1.2

    • stats from 4.1.1 to 4.1.2

    • stats4 from 4.1.1 to 4.1.2

    • tcltk from 4.1.1 to 4.1.2

    • tools from 4.1.1 to 4.1.2

    • utils from 4.1.1 to 4.1.2

    • backports from 1.2.1 to 1.4.0

    • broom from 0.7.9 to 0.7.10

    • cli from 3.0.1 to 3.1.0

    • cpp11 from 0.4.0 to 0.4.1

    • crayon from 1.4.1 to 1.4.2

    • credentials from 1.3.1 to 1.3.2

    • future from 1.22.1 to 1.23.0

    • gert from 1.4.1 to 1.4.3

    • glmnet from 4.1-2 to 4.1-3

    • glue from 1.4.2 to 1.5.0

    • memoise from 2.0.0 to 2.0.1

    • parallelly from 1.28.1 to 1.29.0

    • readr from 2.0.2 to 2.1.0

    • sessioninfo from 1.1.1 to 1.2.1

    • stringi from 1.7.5 to 1.7.6

    • tibble from 3.1.5 to 3.1.6

    • tinytex from 0.34 to 0.35

    • tzdb from 0.1.2 to 0.2.0

    • usethis from 2.1.2 to 2.1.3

    • uuid from 0.1-4 to 1.0-3

    • vroom from 1.5.5 to 1.5.6

    • xfun from 0.27 to 0.28

  • Upgraded Java libraries:

    • com.fasterxml.jackson from 2.12.2 to 2.12.3

    • io.airlift.aircompressor from 0.19 to 0.21

    • io.netty.netty-all from 4.1.63.Final to 4.1.68.Final

    • joda-time.joda-time from 2.10.5 to 2.10.10

    • org.apache.commons.commons-compress from 1.20 to 1.21

    • org.apache.orc from 1.6.10 to 1.6.12

    • org.eclipse.jetty from 9.4.40.v20210413 to 9.4.43.v20210629

    • org.scalanlp.breeze-macros_2.12 from 1.0 to 1.2

    • org.scalanlp.breeze_2.12 from 1.0 to 1.2

    • org.org.typelevel.algebra_2.12 from 2.0.0 to 2.0.1

    • org.typelevel.cats-kernel_2.12 from 2.0.0 to 2.0.1

Apache Spark

Databricks Runtime 10.2 includes Apache Spark 3.2.0. This release includes all Spark fixes and improvements included in Databricks Runtime 10.1 (EoS), as well as the following additional bug fixes and improvements made to Spark:

  • [SPARK-37447] [SQL] Cache LogicalPlan.isStreaming() result in a lazy val

  • [SPARK-37468] [SQL] Support ANSI intervals and TimestampNTZ for UnionEstimation

  • [SPARK-37464] [SQL] SCHEMA and DATABASE should simply be aliases of NAMESPACE

  • [SPARK-37452] [SQL] Char and Varchar break backward compatibility between v3.1 and v2

  • [SPARK-36861] [SQL] Use yyyy-MM-dd as the date pattern in partition discovery

  • [SPARK-37456] [SQL] CREATE NAMESPACE should qualify location for v2 command

  • [SPARK-37357] [SQL] Add small partition factor for rebalance partitions

  • [SPARK-37446] [SQL] Use reflection for getWithoutRegisterFns to allow different Hive versions for building

  • [SPARK-37444] [SQL] ALTER NAMESPACE … SET LOCATION should handle empty location consistently across v1 and v2 command

  • [SPARK-36357] [SQL] Support pushdown Timestamp without time zone for orc

  • [SPARK-36346] [SQL] Support TimestampNTZ type in Orc file source

  • [SPARK-36646] [SQL] Push down group by partition column for aggregate

  • [SPARK-34960] [SQL] Aggregate push down for ORC

  • [SPARK-37386] [SQL] Simplify OptimizeSkewedJoin to not run the cost evaluator

  • [SPARK-36073] [SQL] EquivalentExpressions fixes and improvements

  • [SPARK-35352] [SQL] Add code-gen for full outer sort merge join

  • [SPARK-37292] [SQL] Removes outer join if it only has DISTINCT on streamed side with alias

  • [SPARK-37371] [SQL] UnionExec should support columnar if all children support columnar

  • [SPARK-36645] [SQL] Aggregate (Min/Max/Count) push down for Parquet

  • [SPARK-37379] [SQL] Add tree pattern pruning to CTESubstitution rule

  • [SPARK-37332] [SQL] Allow ANSI intervals in ALTER TABLE .. ADD COLUMNS

  • [SPARK-37224] [SS] Optimize write path on RocksDB state store provider

  • [SPARK-37304] [SQL] Allow ANSI intervals in v2 ALTER TABLE .. REPLACE COLUMNS

  • [SPARK-36924] [SQL] CAST between ANSI intervals and IntegralType

  • [SPARK-37289] [SQL] Remove partitionSchemaOption function

  • [SPARK-37270] [SQL] Fix push foldable into CaseWhen branches if elseValue is empty

  • [SPARK-35345] [SQL] Add Parquet tests to BloomFilterBenchmark

  • [SPARK-37341] [SQL] Avoid unnecessary buffer and copy in full outer sort merge join

  • [SPARK-37266] [SQL] View text can only be SELECT queries

  • [SPARK-37288] [PYTHON][3.2] Backport since annotation update

  • [SPARK-37262] [SQL] Don’t log empty aggregate and group by in JDBCScan

  • [SPARK-37283] [SQL] Don’t try to store a V1 table which contains ANSI intervals in Hive compatible format

  • [SPARK-37221] [SQL] The collect-like API in SparkPlan should support columnar output

  • [SPARK-37202] [SQL] Use AnalysisContext to track referred temp functions

  • [SPARK-37023] [CORE] Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry

  • [SPARK-37307] [CORE] Don’t obtain JDBC connection for empty partitions

  • [SPARK-37267] [SQL] OptimizeSkewInRebalancePartitions support optimize non-root node

  • [SPARK-36799] [SQL] Pass queryExecution name in CLI

  • [SPARK-37261] [SQL] Allow adding partitions with ANSI intervals in DSv2

  • [SPARK-37203] [SQL] Fix NotSerializableException when observe with TypedImperativeAggregate

  • [SPARK-37199] [SQL] Add deterministic field to QueryPlan

  • [SPARK-37253] [PYTHON] try_simplify_traceback should not fail when tb_frame.f_lineno is None

  • [SPARK-37214] [SQL] Fail query analysis earlier with invalid identifiers

  • [SPARK-37240] [SQL] Handle ANSI intervals by ColumnVectorUtils.populate()

  • [SPARK-35437] [SQL] Use expressions to filter Hive partitions at client side

  • [SPARK-32940] [SQL] Collect, first and last should be deterministic aggregate functions

  • [SPARK-37231] [SQL] Dynamic writes/reads of ANSI interval partitions

  • [SPARK-37038] [SQL] DSV2 Sample Push Down

  • [SPARK-36665] [SQL] Add more Not operator simplifications

  • [SPARK-37150] [SQL] Migrate DESCRIBE NAMESPACE to use V2 command by default

  • [SPARK-37196] [SQL] HiveDecimal enforcePrecisionScale failed return null

  • [SPARK-37212] [SQL] Improve the implement of aggregate pushdown.

  • [SPARK-37190] [SQL] Improve error messages for Cast under ANSI mode

  • [SPARK-37161] [SQL] RowToColumnConverter support AnsiIntervalType

  • [SPARK-37160] [SQL] Add a config to optionally disable padding for char type

  • [SPARK-36554] [SQL][PYTHON] Expose make_date expression in functions.scala

  • [SPARK-37200] [SQL] Support drop index for Data Source V2

  • [SPARK-37179] [SQL] ANSI mode: Add a config to allow casting between Datetime and Numeric

  • [SPARK-37168] [SQL] Improve error messages for SQL functions and operators under ANSI mode

  • [SPARK-36935] [SQL] Extend ParquetSchemaConverter to compute Parquet repetition & definition level

  • [SPARK-37066] [SQL] Improve error message to show file path when failed to read next file

  • [SPARK-32567] [SQL] Add code-gen for full outer shuffled hash join

  • [SPARK-37176] [SQL] Sync JsonInferSchema#infer method’s exception handle logic with JacksonParser#parse method

  • [SPARK-37149] [SQL] Improve error messages for arithmetic overflow exceptions under ANSI mode

  • [SPARK-37136] [SQL] Remove code about hive buildin function but not implement in spark

  • [SPARK-16280] [SPARK-37082] [SQL] Implements histogram_numeric aggregation function which supports partial aggregation.

  • [SPARK-36975] [SQL] Correct the hive client calls‘s metrics in HiveClientImpl

System environment

  • Operating System: Ubuntu 20.04.3 LTS

  • Java: Zulu 8.58.0.13-CA-linux64

  • Scala: 2.12.14

  • Python: 3.8.10

  • R: 4.1.2

  • Delta Lake: 1.1.0

Installed Python libraries

Library

Version

Library

Version

Library

Version

Antergos Linux

2015.10 (ISO-Rolling)

appdirs

1.4.4

argon2-cffi

20.1.0

async-generator

1.10

attrs

20.3.0

backcall

0.2.0

bidict

0.21.4

bleach

3.3.0

boto3

1.16.7

botocore

1.19.7

certifi

2020.12.5

cffi

1.14.5

chardet

4.0.0

cycler

0.10.0

Cython

0.29.23

dbus-python

1.2.16

decorator

5.0.6

defusedxml

0.7.1

distlib

0.3.3

distro-info

0.23ubuntu1

entrypoints

0.3

facets-overview

1.0.0

filelock

3.4.0

idna

2.10

ipykernel

5.3.4

ipython

7.22.0

ipython-genutils

0.2.0

ipywidgets

7.6.3

jedi

0.17.2

Jinja2

2.11.3

jmespath

0.10.0

joblib

1.0.1

jsonschema

3.2.0

jupyter-client

6.1.12

jupyter-core

4.7.1

jupyterlab-pygments

0.1.2

jupyterlab-widgets

1.0.0

kiwisolver

1.3.1

koalas

1.8.2

MarkupSafe

2.0.1

matplotlib

3.4.2

mistune

0.8.4

nbclient

0.5.3

nbconvert

6.0.7

nbformat

5.1.3

nest-asyncio

1.5.1

notebook

6.3.0

numpy

1.19.2

packaging

20.9

pandas

1.2.4

pandocfilters

1.4.3

parso

0.7.0

patsy

0.5.1

pexpect

4.8.0

pickleshare

0.7.5

Pillow

8.2.0

pip

21.0.1

plotly

5.3.1

prometheus-client

0.10.1

prompt-toolkit

3.0.17

protobuf

3.17.2

psycopg2

2.8.5

ptyprocess

0.7.0

pyarrow

4.0.0

pycparser

2.20

Pygments

2.8.1

PyGObject

3.36.0

pyparsing

2.4.7

pyrsistent

0.17.3

python-apt

2.0.0+ubuntu0.20.4.6

python-dateutil

2.8.1

python-engineio

4.3.0

python-socketio

5.4.1

pytz

2020.5

pyzmq

20.0.0

requests

2.25.1

requests-unixsocket

0.2.0

s3transfer

0.3.7

scikit-learn

0.24.1

scipy

1.6.2

seaborn

0.11.1

Send2Trash

1.5.0

setuptools

52.0.0

six

1.15.0

ssh-import-id

5.10

statsmodels

0.12.2

tenacity

8.0.1

terminado

0.9.4

testpath

0.4.4

threadpoolctl

2.1.0

tornado

6.1

traitlets

5.0.5

unattended-upgrades

0.1

urllib3

1.25.11

virtualenv

20.4.1

wcwidth

0.2.5

webencodings

0.5.1

wheel

0.36.2

widgetsnbextension

3.5.1

Installed R libraries

R libraries are installed from the Microsoft CRAN snapshot on 2021-11-30.

Library

Version

Library

Version

Library

Version

askpass

1.1

assertthat

0.2.1

backports

1.4.0

base

4.1.2

base64enc

0.1-3

bit

4.0.4

bit64

4.0.5

blob

1.2.2

boot

1.3-28

brew

1.0-6

brio

1.1.2

broom

0.7.10

bslib

0.3.1

cachem

1.0.6

callr

3.7.0

caret

6.0-90

cellranger

1.1.0

chron

2.3-56

class

7.3-19

cli

3.1.0

clipr

0.7.1

cluster

2.1.2

codetools

0.2-18

colorspace

2.0-2

commonmark

1.7

compiler

4.1.2

config

0.3.1

cpp11

0.4.1

crayon

1.4.2

credentials

1.3.2

curl

4.3.2

data.table

1.14.2

datasets

4.1.2

DBI

1.1.1

dbplyr

2.1.1

desc

1.4.0

devtools

2.4.2

diffobj

0.3.5

digest

0.6.28

dplyr

1.0.7

dtplyr

1.1.0

e1071

1.7-9

ellipsis

0.3.2

evaluate

0.14

fansi

0.5.0

farver

2.1.0

fastmap

1.1.0

fontawesome

0.2.2

forcats

0.5.1

foreach

1.5.1

foreign

0.8-81

forge

0.2.0

fs

1.5.0

future

1.23.0

future.apply

1.8.1

gargle

1.2.0

generics

0.1.1

gert

1.4.3

ggplot2

3.3.5

gh

1.3.0

gitcreds

0.1.1

glmnet

4.1-3

globals

0.14.0

glue

1.5.0

googledrive

2.0.0

googlesheets4

1.0.0

gower

0.2.2

graphics

4.1.2

grDevices

4.1.2

grid

4.1.2

gridExtra

2.3

gsubfn

0.7

gtable

0.3.0

haven

2.4.3

highr

0.9

hms

1.1.1

htmltools

0.5.2

htmlwidgets

1.5.4

httpuv

1.6.3

httr

1.4.2

hwriter

1.3.2

hwriterPlus

1.0-3

ids

1.0.1

ini

0.3.1

ipred

0.9-12

isoband

0.2.5

iterators

1.0.13

jquerylib

0.1.4

jsonlite

1.7.2

KernSmooth

2.23-20

knitr

1.36

labeling

0.4.2

later

1.3.0

lattice

0.20-45

lava

1.6.10

lifecycle

1.0.1

listenv

0.8.0

lubridate

1.8.0

magrittr

2.0.1

markdown

1.1

MASS

7.3-54

Matrix

1.3-4

memoise

2.0.1

methods

4.1.2

mgcv

1.8-38

mime

0.12

ModelMetrics

1.2.2.2

modelr

0.1.8

munsell

0.5.0

nlme

3.1-152

nnet

7.3-16

numDeriv

2016.8-1.1

openssl

1.4.5

parallel

4.1.2

parallelly

1.29.0

pillar

1.6.4

pkgbuild

1.2.0

pkgconfig

2.0.3

pkgload

1.2.3

plogr

0.2.0

plyr

1.8.6

praise

1.0.0

prettyunits

1.1.1

pROC

1.18.0

processx

3.5.2

prodlim

2019.11.13

progress

1.2.2

progressr

0.9.0

promises

1.2.0.1

proto

1.0.0

proxy

0.4-26

ps

1.6.0

purrr

0.3.4

r2d3

0.2.5

R6

2.5.1

randomForest

4.6-14

rappdirs

0.3.3

rcmdcheck

1.4.0

RColorBrewer

1.1-2

Rcpp

1.0.7

RcppEigen

0.3.3.9.1

readr

2.1.0

readxl

1.3.1

recipes

0.1.17

rematch

1.0.1

rematch2

2.1.2

remotes

2.4.1

reprex

2.0.1

reshape2

1.4.4

rlang

0.4.12

rmarkdown

2.11

RODBC

1.3-19

roxygen2

7.1.2

rpart

4.1-15

rprojroot

2.0.2

Rserve

1.8-10

RSQLite

2.2.8

rstudioapi

0.13

rversions

2.1.1

rvest

1.0.2

sass

0.4.0

scales

1.1.1

selectr

0.4-2

sessioninfo

1.2.1

shape

1.4.6

shiny

1.7.1

sourcetools

0.1.7

sparklyr

1.7.2

SparkR

3.2.0

spatial

7.3-11

splines

4.1.2

sqldf

0.4-11

SQUAREM

2021.1

stats

4.1.2

stats4

4.1.2

stringi

1.7.6

stringr

1.4.0

survival

3.2-13

sys

3.4

tcltk

4.1.2

TeachingDemos

2.10

testthat

3.1.0

tibble

3.1.6

tidyr

1.1.4

tidyselect

1.1.1

tidyverse

1.3.1

timeDate

3043.102

tinytex

0.35

tools

4.1.2

tzdb

0.2.0

usethis

2.1.3

utf8

1.2.2

utils

4.1.2

uuid

1.0-3

vctrs

0.3.8

viridisLite

0.4.0

vroom

1.5.6

waldo

0.3.1

whisker

0.4

withr

2.4.2

xfun

0.28

xml2

1.3.2

xopen

1.0.0

xtable

1.8-4

yaml

2.2.1

zip

2.2.0

Installed Java and Scala libraries (Scala 2.12 cluster version)

Group ID

Artifact ID

Version

antlr

antlr

2.7.7

com.amazonaws

amazon-kinesis-client

1.12.0

com.amazonaws

aws-java-sdk-autoscaling

1.11.655

com.amazonaws

aws-java-sdk-cloudformation

1.11.655

com.amazonaws

aws-java-sdk-cloudfront

1.11.655

com.amazonaws

aws-java-sdk-cloudhsm

1.11.655

com.amazonaws

aws-java-sdk-cloudsearch

1.11.655

com.amazonaws

aws-java-sdk-cloudtrail

1.11.655

com.amazonaws

aws-java-sdk-cloudwatch

1.11.655

com.amazonaws

aws-java-sdk-cloudwatchmetrics

1.11.655

com.amazonaws

aws-java-sdk-codedeploy

1.11.655

com.amazonaws

aws-java-sdk-cognitoidentity

1.11.655

com.amazonaws

aws-java-sdk-cognitosync

1.11.655

com.amazonaws

aws-java-sdk-config

1.11.655

com.amazonaws

aws-java-sdk-core

1.11.655

com.amazonaws

aws-java-sdk-datapipeline

1.11.655

com.amazonaws

aws-java-sdk-directconnect

1.11.655

com.amazonaws

aws-java-sdk-directory

1.11.655

com.amazonaws

aws-java-sdk-dynamodb

1.11.655

com.amazonaws

aws-java-sdk-ec2

1.11.655

com.amazonaws

aws-java-sdk-ecs

1.11.655

com.amazonaws

aws-java-sdk-efs

1.11.655

com.amazonaws

aws-java-sdk-elasticache

1.11.655

com.amazonaws

aws-java-sdk-elasticbeanstalk

1.11.655

com.amazonaws

aws-java-sdk-elasticloadbalancing

1.11.655

com.amazonaws

aws-java-sdk-elastictranscoder

1.11.655

com.amazonaws

aws-java-sdk-emr

1.11.655

com.amazonaws

aws-java-sdk-glacier

1.11.655

com.amazonaws

aws-java-sdk-glue

1.11.655

com.amazonaws

aws-java-sdk-iam

1.11.655

com.amazonaws

aws-java-sdk-importexport

1.11.655

com.amazonaws

aws-java-sdk-kinesis

1.11.655

com.amazonaws

aws-java-sdk-kms

1.11.655

com.amazonaws

aws-java-sdk-lambda

1.11.655

com.amazonaws

aws-java-sdk-logs

1.11.655

com.amazonaws

aws-java-sdk-machinelearning

1.11.655

com.amazonaws

aws-java-sdk-opsworks

1.11.655

com.amazonaws

aws-java-sdk-rds

1.11.655

com.amazonaws

aws-java-sdk-redshift

1.11.655

com.amazonaws

aws-java-sdk-route53

1.11.655

com.amazonaws

aws-java-sdk-s3

1.11.655

com.amazonaws

aws-java-sdk-ses

1.11.655

com.amazonaws

aws-java-sdk-simpledb

1.11.655

com.amazonaws

aws-java-sdk-simpleworkflow

1.11.655

com.amazonaws

aws-java-sdk-sns

1.11.655

com.amazonaws

aws-java-sdk-sqs

1.11.655

com.amazonaws

aws-java-sdk-ssm

1.11.655

com.amazonaws

aws-java-sdk-storagegateway

1.11.655

com.amazonaws

aws-java-sdk-sts

1.11.655

com.amazonaws

aws-java-sdk-support

1.11.655

com.amazonaws

aws-java-sdk-swf-libraries

1.11.22

com.amazonaws

aws-java-sdk-workspaces

1.11.655

com.amazonaws

jmespath-java

1.11.655

com.chuusai

shapeless_2.12

2.3.3

com.clearspring.analytics

stream

2.9.6

com.databricks

Rserve

1.8-3

com.databricks

jets3t

0.7.1-0

com.databricks.scalapb

compilerplugin_2.12

0.4.15-10

com.databricks.scalapb

scalapb-runtime_2.12

0.4.15-10

com.esotericsoftware

kryo-shaded

4.0.2

com.esotericsoftware

minlog

1.3.0

com.fasterxml

classmate

1.3.4

com.fasterxml.jackson.core

jackson-annotations

2.12.3

com.fasterxml.jackson.core

jackson-core

2.12.3

com.fasterxml.jackson.core

jackson-databind

2.12.3

com.fasterxml.jackson.dataformat

jackson-dataformat-cbor

2.12.3

com.fasterxml.jackson.datatype

jackson-datatype-joda

2.12.3

com.fasterxml.jackson.module

jackson-module-paranamer

2.12.3

com.fasterxml.jackson.module

jackson-module-scala_2.12

2.12.3

com.github.ben-manes.caffeine

caffeine

2.3.4

com.github.fommil

jniloader

1.1

com.github.fommil.netlib

core

1.1.2

com.github.fommil.netlib

native_ref-java

1.1

com.github.fommil.netlib

native_ref-java-natives

1.1

com.github.fommil.netlib

native_system-java

1.1

com.github.fommil.netlib

native_system-java-natives

1.1

com.github.fommil.netlib

netlib-native_ref-linux-x86_64-natives

1.1

com.github.fommil.netlib

netlib-native_system-linux-x86_64-natives

1.1

com.github.luben

zstd-jni

1.5.0-4

com.github.wendykierp

JTransforms

3.1

com.google.code.findbugs

jsr305

3.0.0

com.google.code.gson

gson

2.8.6

com.google.crypto.tink

tink

1.6.0

com.google.flatbuffers

flatbuffers-java

1.9.0

com.google.guava

guava

15.0

com.google.protobuf

protobuf-java

2.6.1

com.h2database

h2

1.4.195

com.helger

profiler

1.1.1

com.jcraft

jsch

0.1.50

com.jolbox

bonecp

0.8.0.RELEASE

com.lihaoyi

sourcecode_2.12

0.1.9

com.microsoft.azure

azure-data-lake-store-sdk

2.3.9

com.microsoft.sqlserver

mssql-jdbc

9.2.1.jre8

com.ning

compress-lzf

1.0.3

com.sun.istack

istack-commons-runtime

3.0.8

com.sun.mail

javax.mail

1.5.2

com.tdunning

json

1.8

com.thoughtworks.paranamer

paranamer

2.8

com.trueaccord.lenses

lenses_2.12

0.4.12

com.twitter

chill-java

0.10.0

com.twitter

chill_2.12

0.10.0

com.twitter

util-app_2.12

7.1.0

com.twitter

util-core_2.12

7.1.0

com.twitter

util-function_2.12

7.1.0

com.twitter

util-jvm_2.12

7.1.0

com.twitter

util-lint_2.12

7.1.0

com.twitter

util-registry_2.12

7.1.0

com.twitter

util-stats_2.12

7.1.0

com.typesafe

config

1.2.1

com.typesafe.scala-logging

scala-logging_2.12

3.7.2

com.univocity

univocity-parsers

2.9.1

com.zaxxer

HikariCP

4.0.3

commons-cli

commons-cli

1.2

commons-codec

commons-codec

1.15

commons-collections

commons-collections

3.2.2

commons-dbcp

commons-dbcp

1.4

commons-fileupload

commons-fileupload

1.3.3

commons-httpclient

commons-httpclient

3.1

commons-io

commons-io

2.8.0

commons-lang

commons-lang

2.6

commons-logging

commons-logging

1.1.3

commons-net

commons-net

3.1

commons-pool

commons-pool

1.5.4

dev.ludovic.netlib

arpack

1.3.2

dev.ludovic.netlib

blas

1.3.2

dev.ludovic.netlib

lapack

1.3.2

hive-2.3__hadoop-3.2

jets3t-0.7

liball_deps_2.12

info.ganglia.gmetric4j

gmetric4j

1.0.10

io.airlift

aircompressor

0.21

io.delta

delta-sharing-spark_2.12

0.2.0

io.dropwizard.metrics

metrics-core

4.1.1

io.dropwizard.metrics

metrics-graphite

4.1.1

io.dropwizard.metrics

metrics-healthchecks

4.1.1

io.dropwizard.metrics

metrics-jetty9

4.1.1

io.dropwizard.metrics

metrics-jmx

4.1.1

io.dropwizard.metrics

metrics-json

4.1.1

io.dropwizard.metrics

metrics-jvm

4.1.1

io.dropwizard.metrics

metrics-servlets

4.1.1

io.netty

netty-all

4.1.68.Final

io.prometheus

simpleclient

0.7.0

io.prometheus

simpleclient_common

0.7.0

io.prometheus

simpleclient_dropwizard

0.7.0

io.prometheus

simpleclient_pushgateway

0.7.0

io.prometheus

simpleclient_servlet

0.7.0

io.prometheus.jmx

collector

0.12.0

jakarta.annotation

jakarta.annotation-api

1.3.5

jakarta.servlet

jakarta.servlet-api

4.0.3

jakarta.validation

jakarta.validation-api

2.0.2

jakarta.ws.rs

jakarta.ws.rs-api

2.1.6

javax.activation

activation

1.1.1

javax.annotation

javax.annotation-api

1.3.2

javax.el

javax.el-api

2.2.4

javax.jdo

jdo-api

3.0.1

javax.transaction

jta

1.1

javax.transaction

transaction-api

1.1

javax.xml.bind

jaxb-api

2.2.2

javax.xml.stream

stax-api

1.0-2

javolution

javolution

5.5.1

jline

jline

2.14.6

joda-time

joda-time

2.10.10

log4j

apache-log4j-extras

1.2.17

log4j

log4j

1.2.17

maven-trees

hive-2.3__hadoop-3.2

liball_deps_2.12

net.java.dev.jna

jna

5.8.0

net.razorvine

pyrolite

4.30

net.sf.jpam

jpam

1.1

net.sf.opencsv

opencsv

2.3

net.sf.supercsv

super-csv

2.2.0

net.snowflake

snowflake-ingest-sdk

0.9.6

net.snowflake

snowflake-jdbc

3.13.3

net.snowflake

spark-snowflake_2.12

2.9.0-spark_3.1

net.sourceforge.f2j

arpack_combined_all

0.1

org.acplt.remotetea

remotetea-oncrpc

1.1.2

org.antlr

ST4

4.0.4

org.antlr

antlr-runtime

3.5.2

org.antlr

antlr4-runtime

4.8

org.antlr

stringtemplate

3.2.1

org.apache.ant

ant

1.9.2

org.apache.ant

ant-jsch

1.9.2

org.apache.ant

ant-launcher

1.9.2

org.apache.arrow

arrow-format

2.0.0

org.apache.arrow

arrow-memory-core

2.0.0

org.apache.arrow

arrow-memory-netty

2.0.0

org.apache.arrow

arrow-vector

2.0.0

org.apache.avro

avro

1.10.2

org.apache.avro

avro-ipc

1.10.2

org.apache.avro

avro-mapred

1.10.2

org.apache.commons

commons-compress

1.21

org.apache.commons

commons-crypto

1.1.0

org.apache.commons

commons-lang3

3.12.0

org.apache.commons

commons-math3

3.4.1

org.apache.commons

commons-text

1.6

org.apache.curator

curator-client

2.13.0

org.apache.curator

curator-framework

2.13.0

org.apache.curator

curator-recipes

2.13.0

org.apache.derby

derby

10.14.2.0

org.apache.hadoop

hadoop-client-api

3.3.1-databricks

org.apache.hadoop

hadoop-client-runtime

3.3.1

org.apache.hive

hive-beeline

2.3.9

org.apache.hive

hive-cli

2.3.9

org.apache.hive

hive-jdbc

2.3.9

org.apache.hive

hive-llap-client

2.3.9

org.apache.hive

hive-llap-common

2.3.9

org.apache.hive

hive-serde

2.3.9

org.apache.hive

hive-shims

2.3.9

org.apache.hive

hive-storage-api

2.7.2

org.apache.hive.shims

hive-shims-0.23

2.3.9

org.apache.hive.shims

hive-shims-common

2.3.9

org.apache.hive.shims

hive-shims-scheduler

2.3.9

org.apache.htrace

htrace-core4

4.1.0-incubating

org.apache.httpcomponents

httpclient

4.5.13

org.apache.httpcomponents

httpcore

4.4.12

org.apache.ivy

ivy

2.5.0

org.apache.mesos

mesos-shaded-protobuf

1.4.0

org.apache.orc

orc-core

1.6.12

org.apache.orc

orc-mapreduce

1.6.12

org.apache.orc

orc-shims

1.6.12

org.apache.parquet

parquet-column

1.12.0-databricks-0003

org.apache.parquet

parquet-common

1.12.0-databricks-0003

org.apache.parquet

parquet-encoding

1.12.0-databricks-0003

org.apache.parquet

parquet-format-structures

1.12.0-databricks-0003

org.apache.parquet

parquet-hadoop

1.12.0-databricks-0003

org.apache.parquet

parquet-jackson

1.12.0-databricks-0003

org.apache.thrift

libfb303

0.9.3

org.apache.thrift

libthrift

0.12.0

org.apache.xbean

xbean-asm9-shaded

4.20

org.apache.yetus

audience-annotations

0.5.0

org.apache.zookeeper

zookeeper

3.6.2

org.apache.zookeeper

zookeeper-jute

3.6.2

org.checkerframework

checker-qual

3.5.0

org.codehaus.jackson

jackson-core-asl

1.9.13

org.codehaus.jackson

jackson-mapper-asl

1.9.13

org.codehaus.janino

commons-compiler

3.0.16

org.codehaus.janino

janino

3.0.16

org.datanucleus

datanucleus-api-jdo

4.2.4

org.datanucleus

datanucleus-core

4.1.17

org.datanucleus

datanucleus-rdbms

4.1.19

org.datanucleus

javax.jdo

3.2.0-m3

org.eclipse.jetty

jetty-client

9.4.43.v20210629

org.eclipse.jetty

jetty-continuation

9.4.43.v20210629

org.eclipse.jetty

jetty-http

9.4.43.v20210629

org.eclipse.jetty

jetty-io

9.4.43.v20210629

org.eclipse.jetty

jetty-jndi

9.4.43.v20210629

org.eclipse.jetty

jetty-plus

9.4.43.v20210629

org.eclipse.jetty

jetty-proxy

9.4.43.v20210629

org.eclipse.jetty

jetty-security

9.4.43.v20210629

org.eclipse.jetty

jetty-server

9.4.43.v20210629

org.eclipse.jetty

jetty-servlet

9.4.43.v20210629

org.eclipse.jetty

jetty-servlets

9.4.43.v20210629

org.eclipse.jetty

jetty-util

9.4.43.v20210629

org.eclipse.jetty

jetty-util-ajax

9.4.43.v20210629

org.eclipse.jetty

jetty-webapp

9.4.43.v20210629

org.eclipse.jetty

jetty-xml

9.4.43.v20210629

org.eclipse.jetty.websocket

websocket-api

9.4.43.v20210629

org.eclipse.jetty.websocket

websocket-client

9.4.43.v20210629

org.eclipse.jetty.websocket

websocket-common

9.4.43.v20210629

org.eclipse.jetty.websocket

websocket-server

9.4.43.v20210629

org.eclipse.jetty.websocket

websocket-servlet

9.4.43.v20210629

org.fusesource.leveldbjni

leveldbjni-all

1.8

org.glassfish.hk2

hk2-api

2.6.1

org.glassfish.hk2

hk2-locator

2.6.1

org.glassfish.hk2

hk2-utils

2.6.1

org.glassfish.hk2

osgi-resource-locator

1.0.3

org.glassfish.hk2.external

aopalliance-repackaged

2.6.1

org.glassfish.hk2.external

jakarta.inject

2.6.1

org.glassfish.jaxb

jaxb-runtime

2.3.2

org.glassfish.jersey.containers

jersey-container-servlet

2.34

org.glassfish.jersey.containers

jersey-container-servlet-core

2.34

org.glassfish.jersey.core

jersey-client

2.34

org.glassfish.jersey.core

jersey-common

2.34

org.glassfish.jersey.core

jersey-server

2.34

org.glassfish.jersey.inject

jersey-hk2

2.34

org.hibernate.validator

hibernate-validator

6.1.0.Final

org.javassist

javassist

3.25.0-GA

org.jboss.logging

jboss-logging

3.3.2.Final

org.jdbi

jdbi

2.63.1

org.jetbrains

annotations

17.0.0

org.joda

joda-convert

1.7

org.jodd

jodd-core

3.5.2

org.json4s

json4s-ast_2.12

3.7.0-M11

org.json4s

json4s-core_2.12

3.7.0-M11

org.json4s

json4s-jackson_2.12

3.7.0-M11

org.json4s

json4s-scalap_2.12

3.7.0-M11

org.lz4

lz4-java

1.7.1

org.mariadb.jdbc

mariadb-java-client

2.2.5

org.objenesis

objenesis

2.5.1

org.postgresql

postgresql

42.2.19

org.roaringbitmap

RoaringBitmap

0.9.14

org.roaringbitmap

shims

0.9.14

org.rocksdb

rocksdbjni

6.20.3

org.rosuda.REngine

REngine

2.1.0

org.scala-lang

scala-compiler_2.12

2.12.14

org.scala-lang

scala-library_2.12

2.12.14

org.scala-lang

scala-reflect_2.12

2.12.14

org.scala-lang.modules

scala-collection-compat_2.12

2.4.3

org.scala-lang.modules

scala-parser-combinators_2.12

1.1.2

org.scala-lang.modules

scala-xml_2.12

1.2.0

org.scala-sbt

test-interface

1.0

org.scalacheck

scalacheck_2.12

1.14.2

org.scalactic

scalactic_2.12

3.0.8

org.scalanlp

breeze-macros_2.12

1.2

org.scalanlp

breeze_2.12

1.2

org.scalatest

scalatest_2.12

3.0.8

org.slf4j

jcl-over-slf4j

1.7.30

org.slf4j

jul-to-slf4j

1.7.30

org.slf4j

slf4j-api

1.7.30

org.slf4j

slf4j-log4j12

1.7.30

org.spark-project.spark

unused

1.0.0

org.springframework

spring-core

4.1.4.RELEASE

org.springframework

spring-test

4.1.4.RELEASE

org.threeten

threeten-extra

1.5.0

org.tukaani

xz

1.8

org.typelevel

algebra_2.12

2.0.1

org.typelevel

cats-kernel_2.12

2.1.1

org.typelevel

macro-compat_2.12

1.1.1

org.typelevel

spire-macros_2.12

0.17.0

org.typelevel

spire-platform_2.12

0.17.0

org.typelevel

spire-util_2.12

0.17.0

org.typelevel

spire_2.12

0.17.0

org.wildfly.openssl

wildfly-openssl

1.0.7.Final

org.xerial

sqlite-jdbc

3.8.11.2

org.xerial.snappy

snappy-java

1.1.8.4

org.yaml

snakeyaml

1.24

oro

oro

2.0.8

pl.edu.icm

JLargeArrays

1.5

software.amazon.ion

ion-java

1.0.2

stax

stax-api

1.0.1