Google
Cloud Platform
Amazon Web Services
Microsoft AzureDatabricks Runtime 16.1 (Beta)
Beta
Databricks Runtime 16.1 is in Beta. The contents of the supported environments may change during the Beta. Changes can include the list of packages or versions of installed packages.
The following release notes provide information about Databricks Runtime 16.1, powered by Apache Spark 3.5.0.
Note
These release notes may include references to features that are not available on Google Cloud as of this release.
Tip
To see release notes for Databricks Runtime versions that have reached end-of-support (EoS), see End-of-support Databricks Runtime release notes. The EoS Databricks Runtime versions have been retired and might not be updated.
Behavioral changes
The VARIANT data type can no longer be used with operations that require comparisons
In DBR 16.1 and above, you cannot use the following clauses or operators in queries that include a VARIANT data type:
DISTINCTINTERSECTEXCEPTUNIONDISTRIBUTE BY
Additionally, you cannot use these DataFrame functions:
df.dropDuplicates()df.repartition()
These operations perform comparisons, and comparisons that use the VARIANT data type produce undefined results and are not supported in Databricks. If you use the VARIANT type in your Databricks workloads or tables, Databricks recommends the following changes:
Update queries or expressions to explicitly cast
VARIANTvalues to non-VARIANTdata types.If you have fields that must be used with any of the above operations, extract those fields from the
VARIANTdata type and store them using non-VARIANTdata types.
To learn more, see Query variant data.
New features and improvements
Enhanced data integrity for compressed Apache Arrow batches
In Databricks Runtime 16.1 and above, to further protect against data corruption, every LZ4 compressed Arrow batch now includes the LZ4 content and block checksums. See LZ4 Frame Format Description.
Library upgrades
Upgraded Python libraries:
ipyflow-core from 0.0.198 to 0.0.201
pyccolo from 0.0.52 to 0.0.65
Upgraded R libraries:
Upgraded Java libraries:
io.delta.delta-sharing-client_2.12 from 1.2.0 to 1.2.2
org.lz4.lz4-java from 1.8.0 to 1.8.0-databricks-1
software.amazon.cryptools.AmazonCorrettoCryptoProvider from 1.6.2-linux-x86_64 to 2.4.1-linux-x86_64
Apache Spark
Databricks Runtime 16.1 includes Apache Spark 3.5.0. This release includes all Spark fixes and improvements included in Databricks Runtime 16.0, as well as the following additional bug fixes and improvements made to Spark:
[SPARK-50482] [SC-182879][CORE] Deprecated no-op
spark.shuffle.spillconfig[SPARK-50032] [SC-182706][SQL][16.x] Allow use of fully qualified collation name
[SPARK-50467] [SC-182823][PYTHON] Add
__all__for builtin functions[SPARK-48898] [SC-182828][SQL] Fix Variant shredding bug
[SPARK-50441] [SC-182668][SQL] Fix parametrized identifiers not working when referencing CTEs
[SPARK-50446] [SC-182639][PYTHON] Concurrent level in Arrow-optimized Python UDF
[SPARK-50430] [SC-182536][CORE] Use the standard Properties.clone instead of manual clone
[SPARK-50471] [SC-182790][PYTHON] Support Arrow-based Python Data Source Writer
[SPARK-50466] [SC-182791][PYTHON] Refine the docstring for string functions - part 1
[SPARK-50194] [DBR16.x][SC-182593][SS][PYTHON] Integration of New Timer API and Initial State API with Timer
[SPARK-50437] [SC-182586][SS] Reduce overhead of creating deserializers in TransformWithStateExec
[SPARK-49676] [DBR16.x][SC-182538][SS][PYTHON] Add Support for Chaining o…
[SPARK-49294] [SC-182730][UI] Add width attribute for shuffle-write-time checkbox.
[SPARK-50426] [SC-182540][PYTHON] Avoid static Python data source lookup when using builtin or Java data sources
[SPARK-48356] [SC-182603][SQL] Support for FOR statement
[SPARK-50333] [SC-182136][SQL] Codegen Support for
CsvToStructs(by Invoke & RuntimeReplaceable)[SPARK-50285] [SC-182575] Metrics for commits to StagedTable instances
[SPARK-50081] [SC-182344][SQL] Codegen Support for
XPath*(by Invoke & RuntimeReplaceable)[SPARK-50440] [SC-182592] [SQL] Refactor AttributeSeq.resolveCandidates
[SPARK-50067] [SC-179648][SQL] Codegen Support for SchemaOfCsv(by Invoke & RuntimeReplaceable)
[SPARK-49873] [SC-178577][SQL] fix post merge failure on error testing
[SPARK-50118] [SC-181259][CONNET] Reset isolated state cache when tasks are running
[SPARK-49873] [SC-178577][SQL] Assign proper error class for LEGACYERROR_TEMP_1325
[SPARK-50381] [SC-182197][CORE] Support
spark.master.rest.maxThreads[SPARK-46725] [SC-182448][SQL] Add DAYNAME function
[SPARK-50270] [SC-181179][SS][PYTHON] Added custom state metrics for TransformWithStateInPandas
[SPARK-50118] Revert “[SC-181259][CONNET] Reset isolated state cache when tasks are running”
[SPARK-50075] [SC-181820][SQL][PYTHON][CONNECT] Add DataFrame APIs for table-valued functions
[SPARK-49470] [SC-175736][UI] Update dataTables stylesheets and javascripts from 1.13.5 to 1.13.11
[SPARK-50235] Revert “[SC-180786][SQL] Clean up ColumnVector resource after processing all rows in ColumnarToRowExec”
[SPARK-50324] [SC-182278][PYTHON][CONNECT] Make
createDataFrametriggerConfigRPC at most once[SPARK-50387] [SC-182441][SS] Update condition for timer expiry and relevant test
[SPARK-50287] [SC-182400][SQL] Merge options of table and relation when creating WriteBuilder in FileTable
[SPARK-50066] [SC-181484][SQL] Codegen Support for
SchemaOfXml(by Invoke & RuntimeReplaceable)[SPARK-50092] [SC-181568][SQL] Fix PostgreSQL connector behaviour for multidimensional arrays
[SPARK-50318] [SC-181641][SQL] Add IntervalUtils.makeYearMonthInterval to deduplicate code between interpreted and codegen
[SPARK-50312] [SC-181646][SQL] SparkThriftServer createServer parameter passing error when kerberos is true
[SPARK-50246] [SC-181468][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_2167:INVALID_JSON_RECORD_TYPE[SPARK-50214] [SC-180692][SQL] From json/xml should not change collations in the given schema
[SPARK-50250] [SC-181466][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_2075:UNSUPPORTED_FEATURE.WRITE_FOR_BINARY_SOURCE[SPARK-50248] [SC-181467][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_2058:INVALID_PARTITION_VALUE[SPARK-50118] [SC-181259][CONNET] Reset isolated state cache when tasks are running
[SPARK-50235] [SC-180786][SQL] Clean up ColumnVector resource after processing all rows in ColumnarToRowExec
[SPARK-50156] [SC-180781][SQL] Integrate
_LEGACY_ERROR_TEMP_2113intoUNRECOGNIZED_STATISTIC[SPARK-50069] [SC-180163][SQL] Integrate
_LEGACY_ERROR_TEMP_0028intoUNSUPPORTED_FROM_TO_EXPRESSION[SPARK-50154] [SC-180663][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_0043:INVALID_RESET_COMMAND_FORMAT[SPARK-49967] [SC-179534][SQL] Codegen Support for
StructsToJson(to_json)[SPARK-50055] [SC-180978][SQL] Add TryMakeInterval alternative
[SPARK-50397] [SC-182367][CORE] Remove deprecated
--ipand-iarguments fromMaster/Worker[SPARK-50238] [SC-181434][PYTHON] Add Variant Support in PySpark UDFs/UDTFs/UDAFs and Python UC UDFs
[SPARK-50079] [SC-179830][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_2013:NEGATIVE_VALUES_IN_FREQUENCY_EXPRESSION[SPARK-50182] [SC-180346][EXAMPLE] Add
submit-sql.shREST API example[SPARK-49966] [SC-179501][SQL] Use
Invoketo implementJsonToStructs(from_json)[SPARK-50302] [SC-182518][SS] Ensure secondary index sizes equal primary index sizes for TransformWithState stateful variables with TTL
[SPARK-50301] [SC-182241][SS][16.x] Make TransformWithState metrics reflect their intuitive meanings
[SPARK-50175] [SC-182140][SQL] Change collation precedence calculation
[SPARK-50148] [SC-180292][SQL] Make
StaticInvokecompatible with the method that declare throw exception[SPARK-50280] [SC-181214][PYTHON] Refactor result sorting and empty bin filling in
compute_hist[SPARK-50190] [SC-182458][PYTHON] Remove direct dependency of Numpy from Histogram
[SPARK-50382] [SC-182368][CONNECT] Add documentation for general information on application development with/extending Spark Connect
[SPARK-50296] [SC-181464][PYTHON][CONNECT] Avoid using a classproperty in threadpool for Python Connect client
[SPARK-49566] [SC-182239][SQL] Add SQL pipe syntax for the EXTEND operator
[SPARK-50036] [SC-179533][CORE][PYTHON] Include SPARK_LOG_SCHEMA in the context of REPL shell
[SPARK-49859] [SC-178259][CONNECT] Replace multiprocessing.ThreadPool with ThreadPoolExecutor
[SPARK-50141] [SC-182378][PYTHON] Make
lpadandrpadaccept Column type arguments[SPARK-50379] [SC-182142][SQL] Fix DayTimeIntevalType handling in WindowExecBase
[SPARK-49954] [SC-179110][SQL] Codegen Support for SchemaOfJson (by Invoke & RuntimeReplaceable)
[SPARK-50398] [SC-182341][CORE] Use ExitCode
0for--helpusage in Spark scripts[SPARK-50377] [SC-182238][SQL] Allow to evaluate foldable RuntimeReplaceable
[SPARK-50241] [SC-181444][SQL] Replace NullIntolerant Mixin with Expression.nullIntolerant method
[SPARK-50084] [SC-179672][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_3168:MISSING_TIMEOUT_CONFIGURATION[SPARK-50078] [SC-179649][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_0038:DUPLICATED_CTE_NAMES[SPARK-50057] [SC-179573][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_1049:INVALID_ATTRIBUTE_NAME_SYNTAX[SPARK-50070] [SC-179579][SQL] Integrate
_LEGACY_ERROR_TEMP_0039intoUNSUPPORTED_SQL_STATEMENT[SPARK-50378] [SC-182235][SS] Add custom metric for tracking spent for proc initial state in transformWithState
[SPARK-50029] [SC-179531][SQL] Make
StaticInvokecompatible with the method that returnAny[SPARK-49990] [SC-179497][SQL] Improve performance of
randStr[SPARK-50048] [SC-179528][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_2114:UNRECOGNIZED_STATISTIC[SPARK-50053] [SC-179532][SQL] Turn
_LEGACY_ERROR_TEMP_2104intoINTERNAL_ERROR[SPARK-49665] [SC-180054][SQL] Trim collation support for string functions
[SPARK-48549] [SC-176472][SQL][PYTHON] Improve SQL function
sentences[SPARK-50022] [SC-179503][CORE][UI] Fix
MasterPageto hide App UI links when UI is disabled[SPARK-50087] [SC-182152] Robust handling of boolean expressions in CASE WHEN for MsSqlServer and future connectors
[SPARK-49991] [SC-179481][SQL] Make HadoopMapReduceCommitProtocol respect ‘mapreduce.output.basename’ to generate file names
[SPARK-50038] [SC-179521][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_0008:MERGE_WITHOUT_WHEN[SPARK-50236] [SC-181671][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_1156:COLUMN_NOT_DEFINED_IN_TABLE[SPARK-50021] [SC-179500][CORE][UI] Fix
ApplicationPageto hide App UI links when UI is disabled[SPARK-49911] [SC-179111][SQL] Fix semantic of support binary equality
[SPARK-50025] [SC-179496][SQL] Integrate
_LEGACY_ERROR_TEMP_1253intoEXPECT_VIEW_NOT_TABLE[SPARK-49829] [SC-179480][SS] Fix the bug on the optimization on adding input to state store in stream-stream join
[SPARK-50004] [SC-179499][SQL] Integrate
_LEGACY_ERROR_TEMP_3327intoFIELD_NOT_FOUND[SPARK-50380] [SC-182210][SQL] ReorderAssociativeOperator should respect the contract in ConstantFolding
[SPARK-50340] [SC-181859][SQL] Unwrap UDT in INSERT input query
[SPARK-50237] [SC-181660][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_2138-9:CIRCULAR_CLASS_REFERENCE[SPARK-50258] [SC-181993][SQL] Fix output column order changed issue after AQE optimization
[SPARK-49773] [SC-178369][SQL] Uncaught Java exception from
make_timestamp()with bad timezone[SPARK-49977] [SC-179265][SQL] Use stack-based iterative computation to avoid creating many Scala List objects for deep expression trees
[SPARK-50153] [SC-181591][SQL] Add
nametoRuleExecutorto make printingQueryExecutionMetrics’s logs clearer[SPARK-50320] [SC-181668][CORE] Make
--remotean official option by removingexperimentalwarning[SPARK-49909] [SC-179492]Revert “[SQL] Fix the pretty name of some expressions”
[SPARK-50330] [SC-180720][SC-181764][SQL] Add hints to Sort and Window nodes
[SPARK-50364] [SC-182003][SQL] Implement serialization for LocalDateTime type in Row.jsonValue
[SPARK-50016] [SC-182139][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_2067:UNSUPPORTED_PARTITION_TRANSFORM[SPARK-49899] [SC-181175][PYTHON][SS] Support deleteIfExists for TransformWithStateInPandas
[SPARK-49757] [SC-177824][SQL] Support IDENTIFIER expression in SET CATALOG statement
[SPARK-50315] [SC-181886][SQL] Support custom metrics for V1Fallback writes
[SPARK-42838] [SC-181509][SQL] Assign a name to the error class LEGACYERROR_TEMP_2000
[SPARK-50353] [SC-181985][SQL] Refactor ResolveSQLOnFile
[SPARK-48344] [SC-181967][SQL] Prepare SQL Scripting for addition of Execution Framework
[SPARK-49345] [SC-174786][CONNECT] Make sure using the current running Spark Session
[SPARK-49925] [SC-178882][SQL] Add tests for order by with collated strings
[SPARK-50167] [SC-181199][PYTHON][CONNECT] Improve PySpark plotting error messages and imports
[SPARK-49368] [SC-174999][CONNECT] Avoid accessing protobuf lite classes directly
[SPARK-50056] [SC-181378][SQL] Codegen Support for ParseUrl (by Invoke & RuntimeReplaceable)
[SPARK-49601] [SC-180770][SS][PYTHON] Support Initial State Handling for TransformWithStateInPandas
[SPARK-49908] [SC-178768][SQL] Assign proper error condition for LEGACYERROR_TEMP_0044
[SPARK-50144] [SC-180307][SS] Address the limitation of metrics calculation with DSv1 streaming sources
[SPARK-49874] [SC-178303][SQL] Remove trim and ltrim collation specifiers.
[SPARK-49513] [SC-180339][SS] Add Support for timer in transformWithStateInPandas API
[SPARK-49119] [SC-175607][SQL] Fix the inconsistency of syntax
show columnsbetween v1 and v2[SPARK-49206] [SC-173704][CORE][UI] Add
Environment Variablestable to MasterEnvironmentPage[SPARK-49934] [SC-179581][SQL] Add implicit cast for accessing collated map with literal
[SPARK-50049] [SC-181659][SQL] Support custom driver metrics in writing to v2 table
[SPARK-50171] [SC-180295][PYTHON] Make numpy optional for KDE plot
[SPARK-49962] [SC-179320][SQL] Simplify AbstractStringTypes class hierarchy
[SPARK-50226] [SC-181483][SQL] Correct MakeDTInterval and MakeYMInterval to catch Java exceptions
[SPARK-48775] [SC-170801][SQL][STS] Replace SQLContext with SparkSession in STS
[SPARK-49015] [SC-175688][CORE] Connect Server should respect
spark.log.structuredLogging.enabled[SPARK-50327] [SC-181667][SQL][16.x] Factor out function resolution to be reused in the single-pass Analyzer
[SPARK-49995] [SC-180762][SQL] Add named argument support to more TVFs
[SPARK-49268] [SC-174903][CORE] Log IO exceptions in SHS history provider
[SPARK-48123] [SC-164989][Core] Provide a constant table schema for querying structured logs
[SPARK-49217] [SC-174904][CORE] Support separate buffer size configuration in UnsafeShuffleWriter
[SPARK-50325] [SC-181664][SQL][16.x] Factor out alias resolution to be reused in the single-pass Analyzer
[SPARK-50322] [SC-181665][SQL] Fix parameterized identifier in a sub-query
[SPARK-48400] [SC-175283][CORE] Promote
PrometheusServlettoDeveloperApi[SPARK-50118] Revert “[SC-181259][CONNET] Reset isolated state cache when tasks are running”
[SPARK-50306] [SC-181564][PYTHON][CONNECT] Support Python 3.13 in Spark Connect
[SPARK-50152] [SC-181264][SS] Support handleInitialState with state data source reader
[SPARK-50260] [SC-181271][CONNECT] Refactor and optimize Spark C…
[SPARK-47591] [SC-163090][SQL] Hive-thriftserver: Migrate logInfo with variables to structured logging framework
[SPARK-49312] [SC-174672][PYTHON] Improve error message for
assertSchemaEqual[SPARK-49439] [SC-175236][SQL] Fix the pretty name of the
FromProtobuf&ToProtobufexpression[SPARK-50092] [ES-1258521] Fix PostgreSQL connector behaviour for multidimensional arrays
[SPARK-49913] [SC-181565][SQL] Add check for unique label names in nested labeled scopes
[SPARK-49563] [SC-181465][SQL] Add SQL pipe syntax for the WINDOW operator
[SPARK-49661] [SC-179021][SQL] Implement trim collation hashing and comparison.
[SPARK-38912] [SC-181543][PYTHON] Remove the comment related to classmethod and property
[SPARK-49770] [16.x][SC-179802][SC-179270][SS][RocksDB Hardening] Improve RocksDB SST file mapping management, and fix issue with reloading same version with existing snapshot
[SPARK-49002] Revert “[SC-172846][SQL] Consistently handle invalid locations in WAREHOUSE/SCHEMA/TABLE/PARTITION/DIRECTORY
[SPARK-50290] [SC-181453][SQL] Add a flag to disable DataFrameQueryContext creation
[SPARK-49668] [SC-178268][SQL] Implement collation key support for trim collation
[SPARK-50262] [SC-181193][SQL] Forbid specification complex types during altering collation
[SPARK-48898] [SC-181435][SQL] Add Variant shredding functions
[SPARK-48273] [SC-181381]Revert “[SQL] Fix late rewrite of PlanWithUnresolvedIdentifier
[SPARK-50222] [SC-180706][CORE] Support
spark.submit.appName[SPARK-50208] [SC-180662][CORE] Support
spark.master.useDriverIdAsAppName.enabled[SPARK-50224] [SC-180689][SQL] The replacements of IsValidUTF8|ValidateUTF8|TryValidateUTF8|MakeValidUTF8 shall be NullIntolerant
[SPARK-50247] [SC-180962][CORE] Define
BLOCK_MANAGER_REREGISTRATION_FAILEDasExecutorExitCode[SPARK-50282] [SC-181221][ML] Simplify
TargetEncoderModel.transform[SPARK-50112] [SC-180763][SQL] Allowing the TransformWithState operator to use Avro encoding
[SPARK-50267] [SC-181180][ML] Improve
TargetEncoder.fitwith DataFrame APIs[SPARK-37178] [SC-180939][ML] Add Target Encoding to ml.feature
[SPARK-50118] [SC-181259][CONNET] Reset isolated state cache when tasks are running
[SPARK-50085] [BEHAVE-176][SC-179809][PYTHON] Make
lit(ndarray)with np.int8 respect the numpy datatype[SPARK-50256] [SC-181048][SQL] Add lightweight validation to check if a logical plan becomes unresolved after every optimizer rule
[SPARK-50196] [SC-180932][CONNECT] Fix Python error context to use a proper context
[SPARK-50274] [SC-181181][CORE] Guard against use-after-close in DirectByteBufferOutputStream
[SPARK-49999] [SC-180063][PYTHON][CONNECT] Support optional “column” parameter in box, kde and hist plots
[SPARK-50273] [SC-181178][SS] Improve logging for RocksDB lock acquire/release cases
[SPARK-50033] [SC-180720][SC-180659][SQL] Add a hint to logical.Aggregate() node
[SPARK-50163] [16.x][SC-180201][SC-180664][SS] Fix the RocksDB extra acquireLock release due to the completion listener
[SPARK-50253] [SC-180969][SS] Stream-Stream Join should not fetch checkpoint ID if not supported.
[SPARK-50255] [SC-180964][PYTHON] Avoid unnecessary casting in
compute_hist[SPARK-50228] [SC-180780][SQL] Move the
RewriteCollationJoinrule toFinishAnalysis[SPARK-50001] [SC-179524][PYTHON][PS][CONNECT] Adjust “precision” to be part of kwargs for box plots
[SPARK-49637] [SC-180160][SQL] Changed error message for INVALID_FRACTION_OF_SECOND
[SPARK-49530] [SC-180658][PYTHON] Get active session from dataframes
[SPARK-50195] [SC-180654][CORE] Fix
StandaloneRestServerto propagatespark.app.nametoSparkSubmitproperly[SPARK-50229] [SC-180773] Reduce memory usage on driver for wide schemas by reducing the lifetime of AttributeReference objects created during logical planning
[SPARK-50231] [SC-180815][PYTHON] Make function
instraccept Columnsubstring[SPARK-49854] [SC-179812][16.x][SQL] Clone artifact manager during session clone
[SPARK-50028] [SC-180699][CONNECT] Replace global locks in Spark Connect server listener with fine-grained locks
[SPARK-49615] [SC-180672] [ML] Make all ML feature transformers dataset schema validation conforming “spark.sql.caseSensitive” config.
[SPARK-50219] [SC-180694][SQL] Refactor
ApplyCharTypePaddingso that helper methods can be used in single-pass resolver[SPARK-50077] [SC-179827][SQL] Introduce a new pattern object for LogicalRelation to help avoiding default full params pattern
[SPARK-50128] [Backport][16x][SC-180677][SS] Add stateful processor handle APIs using implicit encoders in Scala
[SPARK-50061] [SC-179961][SQL] Enable analyze table for collated columns
[SPARK-49993] [SC-180084][SQL] Improve error messages for Sum and Average
[SPARK-49638] [SC-179665][SQL] Remove the ANSI config suggestion in INVALID_URL
[SPARK-50204] [SC-180660][SQL] Factor out
HiveTableRelationread path resolution[SPARK-50193] [SC-180651][SS] Fix exception handling for validating time modes
[SPARK-50179] [SC-180342][CORE] Make
spark.app.nameproperty optional in REST API[SPARK-50068] [SC-180300][SQL] Refactor
TypeCoercionandAnsiTypeCoercionto separate single node transformations[SPARK-49411] [SC-179483][SS] Communicate State Store Checkpoint ID between driver and stateful operators
[SPARK-50124] [SC-180294][SQL] LIMIT/OFFSET should preserve data ordering
[SPARK-49506] [SC-180165][SQL] Optimize ArrayBinarySearch for foldable array
[SPARK-50097] [SC-179908][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_1248:ALTER_TABLE_SERDE_FOR_DATASOURCE_TABLE[SPARK-50071] [SC-180159][SQL][PYTHON] Add try_make_timestamp(_ltz and _ntz) and related tests
[SPARK-50024] Revert “[SC-179515][PYTHON][CONNECT] Switch to use logger instead of warnings module in client”
[SPARK-50054] [SC-180228][PYTHON][CONNECT] Support histogram plots
[SPARK-50015] [SC-179964][SQL] Assign appropriate error condition for
_LEGACY_ERROR_TEMP_1125:MISSING_DATABASE_FOR_V1_SESSION_CATALOG[SPARK-50155] [SC-180240][3.5] Move scala and java files to their default folders
[SPARK-49980] [SC-180353][CORE][SQL] Fix potential file stream leaks caused by interruption in canceled tasks
[SPARK-49010] [SC-172304][SQL][XML] Add unit tests for XML schema inference case sensitivity
[SPARK-50024] [SC-179515][PYTHON][CONNECT] Switch to use logger instead of warnings module in client
[SPARK-49562] [SC-180211][SQL] Add SQL pipe syntax for aggregation
[SPARK-49663] [SC-180239][SQL] Enable RTRIM suggestions in collation expressions
[SPARK-48965] [SC-175926][SQL] Use the correct schema in
Dataset#toJSON[SPARK-48493] [SC-175893][PYTHON] Enhance Python Datasource Reader with direct Arrow Batch support for improved performance
[SPARK-49734] [SC-180226][PYTHON] Add
seedargument for functionshuffle[SPARK-50174] [16.x][SC-180253][SQL] Factor out UnresolvedCatalogRelation resolution
[SPARK-49989] [SC-179512][PYTHON][CONNECT] Support kde/density plots
[SPARK-49805] [SC-180218][SQL][ML] Remove private[xxx] functions from
function.scala[SPARK-49808] [SC-179490][SQL] Fix a deadlock in subquery execution due to lazy vals
[SPARK-49929] [SC-180144][PYTHON][CONNECT] Support box plots
[SPARK-50008] [SC-179290][PS][CONNECT] Avoid unnecessary operations in
attach_distributed_sequence_column[SPARK-49767] [SC-180161][PS][CONNECT] Refactor the internal function invocation
[SPARK-49683] [SC-178341][SQL] Block trim collation
[SPARK-49939] [SC-178941][SQL] Codegen Support for json_object_keys (by Invoke & RuntimeReplaceable)
[SPARK-50031] [SC-179582][SQL] Add the
TryParseUrlexpression[SPARK-49766] [SC-178933][SQL] Codegen Support for
json_array_length(byInvoke&RuntimeReplaceable)[SPARK-50046] [SC-180026][SS] Use stable order of EventTimeWatermark node to calculate watermark
[SPARK-49540] [SC-180145][PS] Unify the usage of
distributed_sequence_id[SPARK-50060] [SC-179965][SQL] Disabled conversion between different collated types in TypeCoercion and AnsiTypeCoercion
[SPARK-49004] [SC-173244][CONNECT] Use separate registry for Column API internal functions
[SPARK-49811] [SC-177888][SQL]Rename StringTypeAnyCollation
[SPARK-49202] [SC-180059][PS] Apply
ArrayBinarySearchfor histogram[SPARK-49203] [SC-175734][SQL] Add expression for
java.util.Arrays.binarySearch[SPARK-50034] [SC-179816][CORE] Fix Misreporting of Fatal Errors as Uncaught Exceptions in
SparkUncaughtExceptionHandler[SPARK-50093] [SC-179836][SQL] Collations that use ICU should have the version of the used ICU library
[SPARK-49985] [SC-179644][SQL] Remove support for interval types in Variant
[SPARK-49103] [SC-173066][CORE] Support
spark.master.rest.filters[SPARK-50090] [SC-179819] Refactor ResolveBinaryArithmetic to separate single-node transformation
[SPARK-49902] [SC-179650][SQL] Catch underlying runtime errors in
RegExpReplace[SPARK-49126] [SC-173342][CORE] Move
spark.history.ui.maxApplicationsconfig definition toHistory.scala[SPARK-50094] [SC-179820][PYTHON][CONNECT] Better error message when using memory profiler on editors with no line numbers
[SPARK-50062] [SC-179835][SQL] Support collations by
InSet[SPARK-50035] [Backport][16x][SC-179530][SS] Add support for explicit handleExpiredTimer function part of the stateful processor
[SPARK-49982] [SC-179814][SQL] Fix negative caching in InMemoryRelation
[SPARK-49082] [SC-173350][SQL] Widening type promotions in
AvroDeserializer[SPARK-50088] [SC-179680][SQL] Refactor
UnresolvedStarBase.expand[SPARK-49802] [SC-179482][SS] Add support for read change feed for map and list types used in stateful processors
[SPARK-49846] [SC-179506][SS] Add numUpdatedStateRows and numRemovedStateRows metrics for use with transformWithState operator
[SPARK-50050] [SC-179643][PYTHON][CONNECT][16.X] Make lit accept str and bool type numpy ndarray
[SPARK-49821] [SC-179527][SS][PYTHON] Implement MapState and TTL support for TransformWithStateInPandas
[SPARK-49558] [SC-179485][SQL] Add SQL pipe syntax for LIMIT/OFFSET and ORDER/SORT/CLUSTER/DISTRIBUTE BY
[SPARK-48144] [SC-165725][LC-4080][SQL] Fix
canPlanAsBroadcastHashJointo respect shuffle join hints[SPARK-50051] [SC-179571][PYTHON][CONNECT] Make
litworks with empty numpy ndarray[SPARK-50018] [SC-179488][SQL] Make AbstractStringType serializable
[SPARK-50016] [SC-179491][SQL] Improve explicit collation mismatch error
[SPARK-50010] [SC-179487][SQL] Expand implicit collation mismatch error
[SPARK-48749] [SC-170116][SQL] Simplify UnaryPositive and eliminate its Catalyst Rules with RuntimeReplaceable
[SPARK-49857] [SC-178576][SQL] Add storageLevel to Dataset localCheckpoint API
[SPARK-50058] [SC-179538][SQL] Factor out plan normalization functions to later use those in single-pass Analyzer testing
[SPARK-50052] [SC-179535][PYTHON][16.X] Make NumpyArrayConverter support empty str ndarray
[SPARK-47261] [SC-173665][SQL] Assign better name for errors LEGACYERROR_TEMP_1172, LEGACYERROR_TEMP_1173, and LEGACYERROR_TEMP_1174
[SPARK-49147] [SC-173471][CORE] Mark
KryoRegistratorwith DeveloperApi interface[SPARK-48949] [SC-173070][SQL] SPJ: Runtime partition filtering
[SPARK-50044] [SC-179523][PYTHON] Refine the docstring of multiple math functions
[SPARK-48757] [SC-170113][CORE] Make
IndexShuffleBlockResolverhave explicit constructors[SPARK-50037] [SQL] Refactor AttributeSeq.resolve(…)
[SPARK-48782] [SC-177056][SQL] Add support for executing procedures in catalogs
[SPARK-49057] [SC-173081][SQL] Do not block the AQE loop when submitting query stages
[SPARK-48824] [SC-176772][BEHAVE-167][SQL] Add Identity Column SQL syntax
[SPARK-48773] [SC-170773] Document config “spark.default.parallelism” by config builder framework
[SPARK-48735] [SC-169810][SQL] Performance Improvement for BIN function
[SPARK-48900] [SC-172433] Add
reasonfield for all internal calls for job/stage cancellation[SPARK-48488] [SC-167605][CORE] Fix methods
log[info|warning|error]inSparkSubmit[SPARK-48708] [SC-169809][CORE] Remove three unnecessary type registrations from
KryoSerializer[SPARK-49958] [SC-179312][PYTHON] Python API for string validation functions
[SPARK-49979] [SC-179264][SQL] Fix AQE hanging issue when collecting twice on a failed plan
[SPARK-48729] [SC-169795][SQL] Add a UserDefinedFunction interface to represent a SQL function
[SPARK-49997] [SC-179279][SQL] Integrate
_LEGACY_ERROR_TEMP_2165intoMALFORMED_RECORD_IN_PARSING[SPARK-49259] [SC-179271][SS]Size based partition creation during kafka read
[SPARK-48129] [SC-165006][PYTHON] Provide a constant table schema in PySpark for querying structured logs
[SPARK-49951] [SC-179259][SQL] Assign proper error condition for LEGACYERROR_TEMP_(1099|3085)
[SPARK-49971] [SC-179278][SQL] Assign proper error condition for LEGACYERROR_TEMP_1097
[SPARK-49998] [SC-179277][SQL] Integrate
_LEGACY_ERROR_TEMP_1252intoEXPECT_TABLE_NOT_VIEW[SPARK-49876] [SC-179262][CONNECT] Get rid of global locks from Spark Connect Service
[SPARK-49957] [SC-179202][SQL] Scala API for string validation functions
[SPARK-48480] [SC-173055][SS][CONNECT] StreamingQueryListener should not be affected by spark.interrupt()
[SPARK-49643] [SC-179239][SQL] Merge LEGACYERROR_TEMP_2042 into ARITHMETIC_OVERFLOW
[SPARK-49959] [SC-179109][SQL] Fix ColumnarArray.copy() to read nulls from the correct offset
[SPARK-49956] Revert “[SC-179070] Disabled collations with collect_set expression”
[SPARK-49987] [SC-179180][SQL] Fix the error prompt when
seedExpressionis non-foldable inrandstr[SPARK-49948] [SC-179158][PS][CONNECT] Add parameter “precision” to pandas on Spark box plot
[SPARK-49970] [SC-179167][SQL] Assign proper error condition for LEGACYERROR_TEMP_2069
[SPARK-49916] [SC-179108][SQL] Throw appropriate Exception for type mismatch between ColumnType and data type in some rows
[SPARK-49956] [SC-179070] Disabled collations with collect_set expression
[SPARK-49974] [16.x][SC-179071][SQL] Move resolveRelations(…) out of the Analyzer.scala
[SPARK-47259] [SC-176437][SQL] Assign names to error conditions for interval errors
[SPARK-47430] [SC-173679][SQL] Rework group by map type to fix bind reference exception
[SPARK-49067] [SC-172616][SQL] Move utf-8 literal into internal methods of UrlCodec class
[SPARK-49955] [SC-178989][SQL] null value does not mean corrupted file when parsing JSON string RDD
[SPARK-49393] [SC-175212][SQL] Fail by default in deprecated catalog plugin APIs
[SPARK-49952] [SC-178985][SQL] Assign proper error condition for LEGACYERROR_TEMP_1142
[SPARK-49405] [SC-175224][SQL] Restrict charsets in JsonOptions
[SPARK-49892] [SC-178975][SQL] Assign proper error class for LEGACYERROR_TEMP_1136
[SPARK-49904] [SC-178973][SQL] Assign proper error condition for LEGACYERROR_TEMP_2140
[SPARK-47257] [SC-174244][SQL] Assign names to error classes LEGACYERROR_TEMP_105[3-4] and LEGACYERROR_TEMP_1331
[SPARK-49915] [SC-178869][SQL] Handle zeros and ones in ReorderAssociativeOperator
[SPARK-49891] [SC-178970][SQL] Assign proper error condition for LEGACYERROR_TEMP_2271
[SPARK-49829] [ES-1254513] Revise the optimization on adding input to state store in stream-stream join (correctness fix)
[SPARK-49918] [SC-178748][CORE] Use read-only access to conf in
SparkContextwhere appropriate[SPARK-49666] [SC-177891][SQL] Add feature flag for trim collation feature
[SPARK-48885] [SC-171050][SQL] Make some subclasses of RuntimeReplaceable override replacement to lazy val
[SPARK-49932] [SC-178931][CORE] Use
tryWithResourcereleaseJsonUtils#toJsonStringresources to avoid memory leaks[SPARK-49949] [SC-178978][PS] Avoid unnecessary analyze task in
attach_sequence_column[SPARK-49924] [SC-178935][SQL] Keep
containsNullafterArrayCompactreplacement[SPARK-49905] [ES-1265435] Use dedicated ShuffleOrigin for stateful operator to prevent the shuffle to be modified from AQE
[SPARK-49615] [SC-178877] Bugfix: Make ML column schema validation conforms with spark config
spark.sql.caseSensitive.[SPARK-49121] [SC-174787][SQL] Support
from_protobufandto_protobuffor SQL functions[SPARK-47496] [SC-160529][SQL] Java SPI Support for dynamic JDBC dialect registering
[SPARK-49359] [SC-174895][SQL] Allow StagedTableCatalog implementations to fall back to non-atomic write
[SPARK-49895] [SC-178543][SQL][ES-1270338] Improve error when encountering trailing comma in SELECT clause
[SPARK-47945] [SC-163457][SQL] MsSQLServer: Document Mapping Spark SQL Data Types from Microsoft SQL Server and add tests
[SPARK-46037] [SC-175495][SQL] Correctness fix for Shuffled Hash Join build left without codegen
[SPARK-47813] [SC-162615][SQL] Replace getArrayDimension with updateExtraColumnMeta
[SPARK-49542] [SC-178765][SQL] Partition transform exception evaluate error
[SPARK-47172] [SC-169537][CORE] Add support for AES-GCM for RPC encryption
[SPARK-47666] [SC-161476][SQL] Fix NPE when reading mysql bit array as LongType
[SPARK-48947] [SC-174239][SQL] Use lowercased charset name to decrease cache missing in Charset.forName
[SPARK-49909] [SC-178552][SQL] Fix the pretty name of some expressions
[SPARK-47647] [SC-161377][SQL] Make MySQL data source able to read bit(n>1) as BinaryType like Postgres
[SPARK-46622] [SC-153004][CORE] Override
toStringmethod foro.a.s.network.shuffledb.StoreVersion[SPARK-48961] [SC-171910][PYTHON] Make the parameter naming of PySparkException consistent with JVM
[SPARK-49889] [SC-178474][PYTHON] Add argument
trimfor functions`trim/ltrim/rtrim`[SPARK-49863] [ES-1184693][SC-178546][SQL] Fix NormalizeFloatingNumbers to preserve nullability of nested structs
[SPARK-47537] [SC-160747][SQL] Fix error data type mapping on MySQL Connector/J
[SPARK-47628] [SC-161257][SQL] Fix Postgres bit array issue ‘Cannot cast to boolean’
[SPARK-49843] [SC-178084][ES-1268164][SQL] Fix change comment on char/varchar columns
[SPARK-49549] [SC-178764][SQL] Assign a name to the error conditions LEGACYERROR_TEMP_3055, 3146
[SPARK-49782] [SC-178756][SQL] ResolveDataFrameDropColumns rule resolves UnresolvedAttribute with child output
[SPARK-49791] [SC-177730][SQL] Make DelegatingCatalogExtension more extendable
[SPARK-49559] [SC-178551][SQL] Add SQL pipe syntax for set operations
[SPARK-49906] [SC-178536][SQL] Introduce and use CONFLICTING_DIRECTORY_STRUCTURES error for PartitioningUtils
[SPARK-49877] [SC-178392][SQL] Change classifyException function signature: add isRuntime argument
[SPARK-47501] [SC-160532][SQL] Add convertDateToDate like the existing convertTimestampToTimestamp for JdbcDialect
[SPARK-49044] [SC-175746][SQL] ValidateExternalType should return child in error
[SPARK-47462] [SC-160460][SQL] Align mappings of other unsigned numeric types with TINYINT in MySQLDialect
[SPARK-47435] [SC-160129][SQL] Fix overflow issue of MySQL UNSIGNED TINYINT caused by SPARK-45561
[SPARK-49398] [SC-176377][SQL] Improve the error for parameters in the query of CACHE TABLE and CREATE VIEW
[SPARK-47263] [SC-177012][SQL] Assign names to the legacy conditions LEGACYERROR_TEMP_13[44-46]
[SPARK-49605] [SC-176605][SQL] Fix the prompt when
ascendingOrderisDataTypeMismatchinSortArray[SPARK-49806] [SC-178304][SQL][CONNECT] Remove redundant
blank spaceaftershowinScalaandConnectclients[SPARK-47258] [SC-175032][SQL] Assign names to error classes LEGACYERROR_TEMP_127[0-5]
[SPARK-49564] [SC-178461][SQL] Add SQL pipe syntax for the JOIN operator
[SPARK-49836] [SC-178339][SQL][SS] Fix possibly broken query when window is provided to window/session_window fn
[SPARK-47390] [SC-163306][SC-159312][SPARK-47396][SC-159376][SPARK-47406] Handle SQL timestamps mapping for Postgres and MySQL
[SPARK-49776] [SC-177818][PYTHON][CONNECT] Support pie plots
[SPARK-49824] [SC-178251][SS][CONNECT] Improve logging in SparkConnectStreamingQueryCache
[SPARK-49894] [SC-178460][PYTHON][CONNECT] Refine the string representation of column field operations
[SPARK-49764] [SC-177491][PYTHON][CONNECT] Support area plots
[SPARK-49694] [SC-177376][PYTHON][CONNECT] Support scatter plots
[SPARK-49744] [SC-178363][SS][PYTHON] Implement TTL support for ListState in TransformWithStateInPandas
[SPARK-49879] [SC-178334][CORE] Move
TransportCipherUtilto a separate file to eliminate Java compilation warnings[SPARK-49866] [SC-178353][SQL] Improve the error message for describe table with partition columns
[SPARK-49867] [SC-178302][ES-1260570][SQL] Improve the error message when index is out of bounds when calling GetColumnByOrdinal
[SPARK-49607] [SC-177373][PYTHON] Update the sampling approach for sampled based plots
[SPARK-49626] [SC-177276][PYTHON][CONNECT] Support horizontal and vertical bar plots
[SPARK-49531] [SC-177770][PYTHON][CONNECT] Support line plot with plotly backend
[SPARK-49444] [SC-177692][ES-1203248][SQL] Modified UnivocityParser to throw runtime exceptions caused by ArrayIndexOutOfBounds with more user-oriented messages
[SPARK-49870] [SC-178262][PYTHON] Add Python 3.13 support in Spark Classic
[SPARK-49560] [SC-178121][SQL] Add SQL pipe syntax for the TABLESAMPLE operator
[SPARK-49864] [SC-178305][SQL] Improve message of BINARY_ARITHMETIC_OVERFLOW
[SPARK-48780] [SC-170274][SQL] Make errors in NamedParametersSupport generic to handle functions and procedures
[SPARK-49358] [SC-178158][SQL] Mode expression for map types with collated strings
[SPARK-47341] [SC-178157][SQL] Fix inaccurate documentation of RuntimeConfig#get
[SPARK-48357] [SC-178153][SQL] Support for LOOP statement
[SPARK-49845] [SC-178059][CORE] Make
appArgsandenvironmentVariablesoptional in REST API[SPARK-49246] [SC-174679][SQL] TableCatalog#loadTable should indicate if it’s for writing
[SPARK-48048] [SC-177525][SC-164846][CONNECT][SS] Added client side listener support for Scala
[SPARK-48700] [SC-177978][SQL] Mode expression for complex types (all collations)
[SPARK-48196] [SC-177899][SQL] Turn QueryExecution lazy val plans into LazyTry
[SPARK-49749] [16.x][SC-177877][CORE] Change log level to debug in BlockManagerInfo
[SPARK-49561] [SC-177897][SQL] Add SQL pipe syntax for the PIVOT and UNPIVOT operators
[SPARK-49823] [SC-177885][SS] Avoid flush during shutdown in rocksdb close path
[SPARK-49820] [SC-177855] [PYTHON] Change
raise IOErrortoraise OSError[SPARK-49653] [SC-177266][SQL] Single join for correlated scalar subqueries
[SPARK-49552] [SC-177477][PYTHON] Add DataFrame API support for new ‘randstr’ and ‘uniform’ SQL functions
[SPARK-48303] [16.x][SC-166251][CORE] Reorganize LogKeys
[SPARK-49656] [16x][Backport][SS] Add support for state variables with value state collection types and read change feed options
[SPARK-48112] [SC-165129][CONNECT] Expose session in SparkConnectPlanner to plugins
[SPARK-48126] [16.x][SC-165309][Core] Make spark.log.structuredLogging.enabled effective
[SPARK-49505] [SC-176873][SQL] Create new SQL functions “randstr” and “uniform” to generate random strings or numbers within ranges
[SPARK-49463] [SC-177474] Support ListState for TransformWithStateInPandas
[SPARK-48131] [SC-165007][Core] Unify MDC key
mdc.taskNameandtask_name[SPARK-49557] [SC-177227][SQL] Add SQL pipe syntax for the WHERE operator
[SPARK-49323] [16.x][SC-174689][CONNECT] Move MockObserver from Spark Connect Server’s test folder to the Server’s main folder
[SPARK-49745] [SC-177501][SS] Add change to read registered timers through state data source reader
[SPARK-49772] [16.x][SC-177478][SC-177214][SS] Remove ColumnFamilyOptions and add configs directly to dbOptions in RocksDB
Databricks ODBC/JDBC driver support
Databricks supports ODBC/JDBC drivers released in the past 2 years. Please download the recently released drivers and upgrade (download ODBC, download JDBC).
System environment
Operating System: Ubuntu 24.04.1 LTS
Java: Zulu17.54+21-CA
Scala: 2.12.18
Python: 3.12.3
R: 4.4.0
Delta Lake: 3.2.1
Installed Python libraries
Library |
Version |
Library |
Version |
Library |
Version |
|---|---|---|---|---|---|
annotated-types |
0.7.0 |
asttokens |
2.0.5 |
astunparse |
1.6.3 |
autocommand |
2.2.2 |
azure-core |
1.31.0 |
azure-storage-blob |
12.23.0 |
azure-storage-file-datalake |
12.17.0 |
backports.tarfile |
1.2.0 |
black |
24.4.2 |
blinker |
1.7.0 |
boto3 |
1.34.69 |
botocore |
1.34.69 |
cachetools |
5.3.3 |
certifi |
2024.6.2 |
cffi |
1.16.0 |
chardet |
4.0.0 |
charset-normalizer |
2.0.4 |
click |
8.1.7 |
cloudpickle |
2.2.1 |
comm |
0.2.1 |
contourpy |
1.2.0 |
cryptography |
42.0.5 |
cycler |
0.11.0 |
Cython |
3.0.11 |
databricks-sdk |
0.30.0 |
dbus-python |
1.3.2 |
debugpy |
1.6.7 |
decorator |
5.1.1 |
Deprecated |
1.2.14 |
distlib |
0.3.8 |
docstring-to-markdown |
0.11 |
entrypoints |
0.4 |
executing |
0.8.3 |
facets-overview |
1.1.1 |
filelock |
3.15.4 |
fonttools |
4.51.0 |
gitdb |
4.0.11 |
GitPython |
3.1.37 |
google-api-core |
2.20.0 |
google-auth |
2.35.0 |
google-cloud-core |
2.4.1 |
google-cloud-storage |
2.18.2 |
google-crc32c |
1.6.0 |
google-resumable-media |
2.7.2 |
googleapis-common-protos |
1.65.0 |
grpcio |
1.60.0 |
grpcio-status |
1.60.0 |
httplib2 |
0.20.4 |
idna |
3.7 |
importlib-metadata |
6.0.0 |
importlib_resources |
6.4.0 |
inflect |
7.3.1 |
ipyflow-core |
0.0.201 |
ipykernel |
6.28.0 |
ipython |
8.25.0 |
ipython-genutils |
0.2.0 |
ipywidgets |
7.7.2 |
isodate |
0.6.1 |
jaraco.context |
5.3.0 |
jaraco.functools |
4.0.1 |
jaraco.text |
3.12.1 |
jedi |
0.19.1 |
jmespath |
1.0.1 |
joblib |
1.4.2 |
jupyter_client |
8.6.0 |
jupyter_core |
5.7.2 |
kiwisolver |
1.4.4 |
launchpadlib |
1.11.0 |
lazr.restfulclient |
0.14.6 |
lazr.uri |
1.0.6 |
matplotlib |
3.8.4 |
matplotlib-inline |
0.1.6 |
mccabe |
0.7.0 |
mlflow-skinny |
2.15.1 |
more-itertools |
10.3.0 |
mypy |
1.10.0 |
mypy-extensions |
1.0.0 |
nest-asyncio |
1.6.0 |
nodeenv |
1.9.1 |
numpy |
1.26.4 |
oauthlib |
3.2.2 |
opentelemetry-api |
1.27.0 |
opentelemetry-sdk |
1.27.0 |
opentelemetry-semantic-conventions |
0.48b0 |
packaging |
24.1 |
pandas |
1.5.3 |
parso |
0.8.3 |
pathspec |
0.10.3 |
patsy |
0.5.6 |
pexpect |
4.8.0 |
pillow |
10.3.0 |
pip |
24.2 |
platformdirs |
3.10.0 |
plotly |
5.22.0 |
pluggy |
1.0.0 |
prompt-toolkit |
3.0.43 |
proto-plus |
1.24.0 |
protobuf |
4.24.1 |
psutil |
5.9.0 |
psycopg2 |
2.9.3 |
ptyprocess |
0.7.0 |
pure-eval |
0.2.2 |
pyarrow |
15.0.2 |
pyasn1 |
0.4.8 |
pyasn1-modules |
0.2.8 |
pyccolo |
0.0.65 |
pycparser |
2.21 |
pydantic |
2.8.2 |
pydantic_core |
2.20.1 |
pyflakes |
3.2.0 |
Pygments |
2.15.1 |
PyGObject |
3.48.2 |
PyJWT |
2.7.0 |
pyodbc |
5.0.1 |
pyparsing |
3.0.9 |
pyright |
1.1.294 |
python-dateutil |
2.9.0.post0 |
python-lsp-jsonrpc |
1.1.2 |
python-lsp-server |
1.10.0 |
pytoolconfig |
1.2.6 |
pytz |
2024.1 |
PyYAML |
6.0.1 |
pyzmq |
25.1.2 |
requests |
2.32.2 |
rope |
1.12.0 |
rsa |
4.9 |
s3transfer |
0.10.2 |
scikit-learn |
1.4.2 |
scipy |
1.13.1 |
seaborn |
0.13.2 |
setuptools |
74.0.0 |
six |
1.16.0 |
smmap |
5.0.0 |
sqlparse |
0.5.1 |
ssh-import-id |
5.11 |
stack-data |
0.2.0 |
statsmodels |
0.14.2 |
tenacity |
8.2.2 |
threadpoolctl |
2.2.0 |
tokenize-rt |
4.2.1 |
tomli |
2.0.1 |
tornado |
6.4.1 |
traitlets |
5.14.3 |
typeguard |
4.3.0 |
types-protobuf |
3.20.3 |
types-psutil |
5.9.0 |
types-pytz |
2023.3.1.1 |
types-PyYAML |
6.0.0 |
types-requests |
2.31.0.0 |
types-setuptools |
68.0.0.0 |
types-six |
1.16.0 |
types-urllib3 |
1.26.25.14 |
typing_extensions |
4.11.0 |
ujson |
5.10.0 |
unattended-upgrades |
0.1 |
urllib3 |
1.26.16 |
virtualenv |
20.26.2 |
wadllib |
1.3.6 |
wcwidth |
0.2.5 |
whatthepatch |
1.0.2 |
wheel |
0.43.0 |
wrapt |
1.14.1 |
yapf |
0.33.0 |
zipp |
3.17.0 |
Installed R libraries
R libraries are installed from the Posit Package Manager CRAN snapshot on 2024-08-04.
Library |
Version |
Library |
Version |
Library |
Version |
|---|---|---|---|---|---|
arrow |
16.1.0 |
askpass |
1.2.0 |
assertthat |
0.2.1 |
backports |
1.5.0 |
base |
4.4.0 |
base64enc |
0.1-3 |
bigD |
0.2.0 |
bit |
4.0.5 |
bit64 |
4.0.5 |
bitops |
1.0-8 |
blob |
1.2.4 |
boot |
1.3-30 |
brew |
1.0-10 |
brio |
1.1.5 |
broom |
1.0.6 |
bslib |
0.8.0 |
cachem |
1.1.0 |
callr |
3.7.6 |
caret |
6.0-94 |
cellranger |
1.1.0 |
chron |
2.3-61 |
class |
7.3-22 |
cli |
3.6.3 |
clipr |
0.8.0 |
clock |
0.7.1 |
cluster |
2.1.6 |
codetools |
0.2-20 |
colorspace |
2.1-1 |
commonmark |
1.9.1 |
compiler |
4.4.0 |
config |
0.3.2 |
conflicted |
1.2.0 |
cpp11 |
0.4.7 |
crayon |
1.5.3 |
credentials |
2.0.1 |
curl |
5.2.1 |
data.table |
1.15.4 |
datasets |
4.4.0 |
DBI |
1.2.3 |
dbplyr |
2.5.0 |
desc |
1.4.3 |
devtools |
2.4.5 |
diagram |
1.6.5 |
diffobj |
0.3.5 |
digest |
0.6.36 |
downlit |
0.4.4 |
dplyr |
1.1.4 |
dtplyr |
1.3.1 |
e1071 |
1.7-14 |
ellipsis |
0.3.2 |
evaluate |
0.24.0 |
fansi |
1.0.6 |
farver |
2.1.2 |
fastmap |
1.2.0 |
fontawesome |
0.5.2 |
forcats |
1.0.0 |
foreach |
1.5.2 |
foreign |
0.8-86 |
forge |
0.2.0 |
fs |
1.6.4 |
future |
1.34.0 |
future.apply |
1.11.2 |
gargle |
1.5.2 |
generics |
0.1.3 |
gert |
2.1.0 |
ggplot2 |
3.5.1 |
gh |
1.4.1 |
git2r |
0.33.0 |
gitcreds |
0.1.2 |
glmnet |
4.1-8 |
globals |
0.16.3 |
glue |
1.7.0 |
googledrive |
2.1.1 |
googlesheets4 |
1.1.1 |
gower |
1.0.1 |
graphics |
4.4.0 |
grDevices |
4.4.0 |
grid |
4.4.0 |
gridExtra |
2.3 |
gsubfn |
0.7 |
gt |
0.11.0 |
gtable |
0.3.5 |
hardhat |
1.4.0 |
haven |
2.5.4 |
highr |
0.11 |
hms |
1.1.3 |
htmltools |
0.5.8.1 |
htmlwidgets |
1.6.4 |
httpuv |
1.6.15 |
httr |
1.4.7 |
httr2 |
1.0.2 |
ids |
1.0.1 |
ini |
0.3.1 |
ipred |
0.9-15 |
isoband |
0.2.7 |
iterators |
1.0.14 |
jquerylib |
0.1.4 |
jsonlite |
1.8.8 |
juicyjuice |
0.1.0 |
KernSmooth |
2.23-22 |
knitr |
1.48 |
labeling |
0.4.3 |
later |
1.3.2 |
lattice |
0.22-5 |
lava |
1.8.0 |
lifecycle |
1.0.4 |
listenv |
0.9.1 |
lubridate |
1.9.3 |
magrittr |
2.0.3 |
markdown |
1.13 |
MASS |
7.3-60.0.1 |
Matrix |
1.6-5 |
memoise |
2.0.1 |
methods |
4.4.0 |
mgcv |
1.9-1 |
mime |
0.12 |
miniUI |
0.1.1.1 |
mlflow |
2.14.1 |
ModelMetrics |
1.2.2.2 |
modelr |
0.1.11 |
munsell |
0.5.1 |
nlme |
3.1-165 |
nnet |
7.3-19 |
numDeriv |
2016.8-1.1 |
openssl |
2.2.0 |
parallel |
4.4.0 |
parallelly |
1.38.0 |
pillar |
1.9.0 |
pkgbuild |
1.4.4 |
pkgconfig |
2.0.3 |
pkgdown |
2.1.0 |
pkgload |
1.4.0 |
plogr |
0.2.0 |
plyr |
1.8.9 |
praise |
1.0.0 |
prettyunits |
1.2.0 |
pROC |
1.18.5 |
processx |
3.8.4 |
prodlim |
2024.06.25 |
profvis |
0.3.8 |
progress |
1.2.3 |
progressr |
0.14.0 |
promises |
1.3.0 |
proto |
1.0.0 |
proxy |
0.4-27 |
ps |
1.7.7 |
purrr |
1.0.2 |
R6 |
2.5.1 |
ragg |
1.3.2 |
randomForest |
4.7-1.1 |
rappdirs |
0.3.3 |
rcmdcheck |
1.4.0 |
RColorBrewer |
1.1-3 |
Rcpp |
1.0.13 |
RcppEigen |
0.3.4.0.0 |
reactable |
0.4.4 |
reactR |
0.6.0 |
readr |
2.1.5 |
readxl |
1.4.3 |
recipes |
1.1.0 |
rematch |
2.0.0 |
rematch2 |
2.1.2 |
remotes |
2.5.0 |
reprex |
2.1.1 |
reshape2 |
1.4.4 |
rlang |
1.1.4 |
rmarkdown |
2.27 |
RODBC |
1.3-23 |
roxygen2 |
7.3.2 |
rpart |
4.1.23 |
rprojroot |
2.0.4 |
Rserve |
1.8-13 |
RSQLite |
2.3.7 |
rstudioapi |
0.16.0 |
rversions |
2.1.2 |
rvest |
1.0.4 |
sass |
0.4.9 |
scales |
1.3.0 |
selectr |
0.4-2 |
sessioninfo |
1.2.2 |
shape |
1.4.6.1 |
shiny |
1.9.1 |
sourcetools |
0.1.7-1 |
sparklyr |
1.8.6 |
spatial |
7.3-17 |
splines |
4.4.0 |
sqldf |
0.4-11 |
SQUAREM |
2021.1 |
stats |
4.4.0 |
stats4 |
4.4.0 |
stringi |
1.8.4 |
stringr |
1.5.1 |
survival |
3.6-4 |
swagger |
5.17.14.1 |
sys |
3.4.2 |
systemfonts |
1.1.0 |
tcltk |
4.4.0 |
testthat |
3.2.1.1 |
textshaping |
0.4.0 |
tibble |
3.2.1 |
tidyr |
1.3.1 |
tidyselect |
1.2.1 |
tidyverse |
2.0.0 |
timechange |
0.3.0 |
timeDate |
4032.109 |
tinytex |
0.52 |
tools |
4.4.0 |
tzdb |
0.4.0 |
urlchecker |
1.0.1 |
usethis |
3.0.0 |
utf8 |
1.2.4 |
utils |
4.4.0 |
uuid |
1.2-1 |
V8 |
4.4.2 |
vctrs |
0.6.5 |
viridisLite |
0.4.2 |
vroom |
1.6.5 |
waldo |
0.5.2 |
whisker |
0.4.1 |
withr |
3.0.1 |
xfun |
0.46 |
xml2 |
1.3.6 |
xopen |
1.0.1 |
xtable |
1.8-4 |
yaml |
2.3.10 |
zeallot |
0.1.0 |
zip |
2.3.1 |
Installed Java and Scala libraries (Scala 2.12 cluster version)
Group ID |
Artifact ID |
Version |
|---|---|---|
antlr |
antlr |
2.7.7 |
com.amazonaws |
amazon-kinesis-client |
1.12.0 |
com.amazonaws |
aws-java-sdk-autoscaling |
1.12.638 |
com.amazonaws |
aws-java-sdk-cloudformation |
1.12.638 |
com.amazonaws |
aws-java-sdk-cloudfront |
1.12.638 |
com.amazonaws |
aws-java-sdk-cloudhsm |
1.12.638 |
com.amazonaws |
aws-java-sdk-cloudsearch |
1.12.638 |
com.amazonaws |
aws-java-sdk-cloudtrail |
1.12.638 |
com.amazonaws |
aws-java-sdk-cloudwatch |
1.12.638 |
com.amazonaws |
aws-java-sdk-cloudwatchmetrics |
1.12.638 |
com.amazonaws |
aws-java-sdk-codedeploy |
1.12.638 |
com.amazonaws |
aws-java-sdk-cognitoidentity |
1.12.638 |
com.amazonaws |
aws-java-sdk-cognitosync |
1.12.638 |
com.amazonaws |
aws-java-sdk-config |
1.12.638 |
com.amazonaws |
aws-java-sdk-core |
1.12.638 |
com.amazonaws |
aws-java-sdk-datapipeline |
1.12.638 |
com.amazonaws |
aws-java-sdk-directconnect |
1.12.638 |
com.amazonaws |
aws-java-sdk-directory |
1.12.638 |
com.amazonaws |
aws-java-sdk-dynamodb |
1.12.638 |
com.amazonaws |
aws-java-sdk-ec2 |
1.12.638 |
com.amazonaws |
aws-java-sdk-ecs |
1.12.638 |
com.amazonaws |
aws-java-sdk-efs |
1.12.638 |
com.amazonaws |
aws-java-sdk-elasticache |
1.12.638 |
com.amazonaws |
aws-java-sdk-elasticbeanstalk |
1.12.638 |
com.amazonaws |
aws-java-sdk-elasticloadbalancing |
1.12.638 |
com.amazonaws |
aws-java-sdk-elastictranscoder |
1.12.638 |
com.amazonaws |
aws-java-sdk-emr |
1.12.638 |
com.amazonaws |
aws-java-sdk-glacier |
1.12.638 |
com.amazonaws |
aws-java-sdk-glue |
1.12.638 |
com.amazonaws |
aws-java-sdk-iam |
1.12.638 |
com.amazonaws |
aws-java-sdk-importexport |
1.12.638 |
com.amazonaws |
aws-java-sdk-kinesis |
1.12.638 |
com.amazonaws |
aws-java-sdk-kms |
1.12.638 |
com.amazonaws |
aws-java-sdk-lambda |
1.12.638 |
com.amazonaws |
aws-java-sdk-logs |
1.12.638 |
com.amazonaws |
aws-java-sdk-machinelearning |
1.12.638 |
com.amazonaws |
aws-java-sdk-opsworks |
1.12.638 |
com.amazonaws |
aws-java-sdk-rds |
1.12.638 |
com.amazonaws |
aws-java-sdk-redshift |
1.12.638 |
com.amazonaws |
aws-java-sdk-route53 |
1.12.638 |
com.amazonaws |
aws-java-sdk-s3 |
1.12.638 |
com.amazonaws |
aws-java-sdk-ses |
1.12.638 |
com.amazonaws |
aws-java-sdk-simpledb |
1.12.638 |
com.amazonaws |
aws-java-sdk-simpleworkflow |
1.12.638 |
com.amazonaws |
aws-java-sdk-sns |
1.12.638 |
com.amazonaws |
aws-java-sdk-sqs |
1.12.638 |
com.amazonaws |
aws-java-sdk-ssm |
1.12.638 |
com.amazonaws |
aws-java-sdk-storagegateway |
1.12.638 |
com.amazonaws |
aws-java-sdk-sts |
1.12.638 |
com.amazonaws |
aws-java-sdk-support |
1.12.638 |
com.amazonaws |
aws-java-sdk-swf-libraries |
1.11.22 |
com.amazonaws |
aws-java-sdk-workspaces |
1.12.638 |
com.amazonaws |
jmespath-java |
1.12.638 |
com.clearspring.analytics |
stream |
2.9.6 |
com.databricks |
Rserve |
1.8-3 |
com.databricks |
databricks-sdk-java |
0.27.0 |
com.databricks |
jets3t |
0.7.1-0 |
com.databricks.scalapb |
scalapb-runtime_2.12 |
0.4.15-10 |
com.esotericsoftware |
kryo-shaded |
4.0.2 |
com.esotericsoftware |
minlog |
1.3.0 |
com.fasterxml |
classmate |
1.3.4 |
com.fasterxml.jackson.core |
jackson-annotations |
2.15.2 |
com.fasterxml.jackson.core |
jackson-core |
2.15.2 |
com.fasterxml.jackson.core |
jackson-databind |
2.15.2 |
com.fasterxml.jackson.dataformat |
jackson-dataformat-cbor |
2.15.2 |
com.fasterxml.jackson.dataformat |
jackson-dataformat-yaml |
2.15.2 |
com.fasterxml.jackson.datatype |
jackson-datatype-joda |
2.15.2 |
com.fasterxml.jackson.datatype |
jackson-datatype-jsr310 |
2.16.0 |
com.fasterxml.jackson.module |
jackson-module-paranamer |
2.15.2 |
com.fasterxml.jackson.module |
jackson-module-scala_2.12 |
2.15.2 |
com.github.ben-manes.caffeine |
caffeine |
2.9.3 |
com.github.fommil |
jniloader |
1.1 |
com.github.fommil.netlib |
native_ref-java |
1.1 |
com.github.fommil.netlib |
native_ref-java |
1.1-natives |
com.github.fommil.netlib |
native_system-java |
1.1 |
com.github.fommil.netlib |
native_system-java |
1.1-natives |
com.github.fommil.netlib |
netlib-native_ref-linux-x86_64 |
1.1-natives |
com.github.fommil.netlib |
netlib-native_system-linux-x86_64 |
1.1-natives |
com.github.luben |
zstd-jni |
1.5.5-4 |
com.github.wendykierp |
JTransforms |
3.1 |
com.google.code.findbugs |
jsr305 |
3.0.0 |
com.google.code.gson |
gson |
2.10.1 |
com.google.crypto.tink |
tink |
1.9.0 |
com.google.errorprone |
error_prone_annotations |
2.10.0 |
com.google.flatbuffers |
flatbuffers-java |
23.5.26 |
com.google.guava |
guava |
15.0 |
com.google.protobuf |
protobuf-java |
3.25.1 |
com.helger |
profiler |
1.1.1 |
com.ibm.icu |
icu4j |
75.1 |
com.jcraft |
jsch |
0.1.55 |
com.jolbox |
bonecp |
0.8.0.RELEASE |
com.lihaoyi |
sourcecode_2.12 |
0.1.9 |
com.microsoft.azure |
azure-data-lake-store-sdk |
2.3.9 |
com.microsoft.sqlserver |
mssql-jdbc |
11.2.2.jre8 |
com.microsoft.sqlserver |
mssql-jdbc |
11.2.3.jre8 |
com.ning |
compress-lzf |
1.1.2 |
com.sun.mail |
javax.mail |
1.5.2 |
com.sun.xml.bind |
jaxb-core |
2.2.11 |
com.sun.xml.bind |
jaxb-impl |
2.2.11 |
com.tdunning |
json |
1.8 |
com.thoughtworks.paranamer |
paranamer |
2.8 |
com.trueaccord.lenses |
lenses_2.12 |
0.4.12 |
com.twitter |
chill-java |
0.10.0 |
com.twitter |
chill_2.12 |
0.10.0 |
com.twitter |
util-app_2.12 |
7.1.0 |
com.twitter |
util-core_2.12 |
7.1.0 |
com.twitter |
util-function_2.12 |
7.1.0 |
com.twitter |
util-jvm_2.12 |
7.1.0 |
com.twitter |
util-lint_2.12 |
7.1.0 |
com.twitter |
util-registry_2.12 |
7.1.0 |
com.twitter |
util-stats_2.12 |
7.1.0 |
com.typesafe |
config |
1.4.3 |
com.typesafe.scala-logging |
scala-logging_2.12 |
3.7.2 |
com.uber |
h3 |
3.7.3 |
com.univocity |
univocity-parsers |
2.9.1 |
com.zaxxer |
HikariCP |
4.0.3 |
commons-cli |
commons-cli |
1.5.0 |
commons-codec |
commons-codec |
1.16.0 |
commons-collections |
commons-collections |
3.2.2 |
commons-dbcp |
commons-dbcp |
1.4 |
commons-fileupload |
commons-fileupload |
1.5 |
commons-httpclient |
commons-httpclient |
3.1 |
commons-io |
commons-io |
2.13.0 |
commons-lang |
commons-lang |
2.6 |
commons-logging |
commons-logging |
1.1.3 |
commons-pool |
commons-pool |
1.5.4 |
dev.ludovic.netlib |
arpack |
3.0.3 |
dev.ludovic.netlib |
blas |
3.0.3 |
dev.ludovic.netlib |
lapack |
3.0.3 |
info.ganglia.gmetric4j |
gmetric4j |
1.0.10 |
io.airlift |
aircompressor |
0.27 |
io.delta |
delta-sharing-client_2.12 |
1.2.2 |
io.dropwizard.metrics |
metrics-annotation |
4.2.19 |
io.dropwizard.metrics |
metrics-core |
4.2.19 |
io.dropwizard.metrics |
metrics-graphite |
4.2.19 |
io.dropwizard.metrics |
metrics-healthchecks |
4.2.19 |
io.dropwizard.metrics |
metrics-jetty9 |
4.2.19 |
io.dropwizard.metrics |
metrics-jmx |
4.2.19 |
io.dropwizard.metrics |
metrics-json |
4.2.19 |
io.dropwizard.metrics |
metrics-jvm |
4.2.19 |
io.dropwizard.metrics |
metrics-servlets |
4.2.19 |
io.netty |
netty-all |
4.1.108.Final |
io.netty |
netty-buffer |
4.1.108.Final |
io.netty |
netty-codec |
4.1.108.Final |
io.netty |
netty-codec-http |
4.1.108.Final |
io.netty |
netty-codec-http2 |
4.1.108.Final |
io.netty |
netty-codec-socks |
4.1.108.Final |
io.netty |
netty-common |
4.1.108.Final |
io.netty |
netty-handler |
4.1.108.Final |
io.netty |
netty-handler-proxy |
4.1.108.Final |
io.netty |
netty-resolver |
4.1.108.Final |
io.netty |
netty-tcnative-boringssl-static |
2.0.61.Final-db-r16 |
io.netty |
netty-tcnative-boringssl-static |
2.0.61.Final-db-r16-linux-aarch_64 |
io.netty |
netty-tcnative-boringssl-static |
2.0.61.Final-db-r16-linux-x86_64 |
io.netty |
netty-tcnative-boringssl-static |
2.0.61.Final-db-r16-osx-aarch_64 |
io.netty |
netty-tcnative-boringssl-static |
2.0.61.Final-db-r16-osx-x86_64 |
io.netty |
netty-tcnative-boringssl-static |
2.0.61.Final-db-r16-windows-x86_64 |
io.netty |
netty-tcnative-classes |
2.0.61.Final |
io.netty |
netty-transport |
4.1.108.Final |
io.netty |
netty-transport-classes-epoll |
4.1.108.Final |
io.netty |
netty-transport-classes-kqueue |
4.1.108.Final |
io.netty |
netty-transport-native-epoll |
4.1.108.Final |
io.netty |
netty-transport-native-epoll |
4.1.108.Final-linux-aarch_64 |
io.netty |
netty-transport-native-epoll |
4.1.108.Final-linux-riscv64 |
io.netty |
netty-transport-native-epoll |
4.1.108.Final-linux-x86_64 |
io.netty |
netty-transport-native-kqueue |
4.1.108.Final-osx-aarch_64 |
io.netty |
netty-transport-native-kqueue |
4.1.108.Final-osx-x86_64 |
io.netty |
netty-transport-native-unix-common |
4.1.108.Final |
io.prometheus |
simpleclient |
0.7.0 |
io.prometheus |
simpleclient_common |
0.7.0 |
io.prometheus |
simpleclient_dropwizard |
0.7.0 |
io.prometheus |
simpleclient_pushgateway |
0.7.0 |
io.prometheus |
simpleclient_servlet |
0.7.0 |
io.prometheus.jmx |
collector |
0.12.0 |
jakarta.annotation |
jakarta.annotation-api |
1.3.5 |
jakarta.servlet |
jakarta.servlet-api |
4.0.3 |
jakarta.validation |
jakarta.validation-api |
2.0.2 |
jakarta.ws.rs |
jakarta.ws.rs-api |
2.1.6 |
javax.activation |
activation |
1.1.1 |
javax.el |
javax.el-api |
2.2.4 |
javax.jdo |
jdo-api |
3.0.1 |
javax.transaction |
jta |
1.1 |
javax.transaction |
transaction-api |
1.1 |
javax.xml.bind |
jaxb-api |
2.2.11 |
javolution |
javolution |
5.5.1 |
jline |
jline |
2.14.6 |
joda-time |
joda-time |
2.12.1 |
net.java.dev.jna |
jna |
5.8.0 |
net.razorvine |
pickle |
1.3 |
net.sf.jpam |
jpam |
1.1 |
net.sf.opencsv |
opencsv |
2.3 |
net.sf.supercsv |
super-csv |
2.2.0 |
net.snowflake |
snowflake-ingest-sdk |
0.9.6 |
net.sourceforge.f2j |
arpack_combined_all |
0.1 |
org.acplt.remotetea |
remotetea-oncrpc |
1.1.2 |
org.antlr |
ST4 |
4.0.4 |
org.antlr |
antlr-runtime |
3.5.2 |
org.antlr |
antlr4-runtime |
4.9.3 |
org.antlr |
stringtemplate |
3.2.1 |
org.apache.ant |
ant |
1.10.11 |
org.apache.ant |
ant-jsch |
1.10.11 |
org.apache.ant |
ant-launcher |
1.10.11 |
org.apache.arrow |
arrow-format |
15.0.0 |
org.apache.arrow |
arrow-memory-core |
15.0.0 |
org.apache.arrow |
arrow-memory-netty |
15.0.0 |
org.apache.arrow |
arrow-vector |
15.0.0 |
org.apache.avro |
avro |
1.11.3 |
org.apache.avro |
avro-ipc |
1.11.3 |
org.apache.avro |
avro-mapred |
1.11.3 |
org.apache.commons |
commons-collections4 |
4.4 |
org.apache.commons |
commons-compress |
1.23.0 |
org.apache.commons |
commons-crypto |
1.1.0 |
org.apache.commons |
commons-lang3 |
3.12.0 |
org.apache.commons |
commons-math3 |
3.6.1 |
org.apache.commons |
commons-text |
1.10.0 |
org.apache.curator |
curator-client |
2.13.0 |
org.apache.curator |
curator-framework |
2.13.0 |
org.apache.curator |
curator-recipes |
2.13.0 |
org.apache.datasketches |
datasketches-java |
3.1.0 |
org.apache.datasketches |
datasketches-memory |
2.0.0 |
org.apache.derby |
derby |
10.14.2.0 |
org.apache.hadoop |
hadoop-client-runtime |
3.3.6 |
org.apache.hive |
hive-beeline |
2.3.9 |
org.apache.hive |
hive-cli |
2.3.9 |
org.apache.hive |
hive-jdbc |
2.3.9 |
org.apache.hive |
hive-llap-client |
2.3.9 |
org.apache.hive |
hive-llap-common |
2.3.9 |
org.apache.hive |
hive-serde |
2.3.9 |
org.apache.hive |
hive-shims |
2.3.9 |
org.apache.hive |
hive-storage-api |
2.8.1 |
org.apache.hive.shims |
hive-shims-0.23 |
2.3.9 |
org.apache.hive.shims |
hive-shims-common |
2.3.9 |
org.apache.hive.shims |
hive-shims-scheduler |
2.3.9 |
org.apache.httpcomponents |
httpclient |
4.5.14 |
org.apache.httpcomponents |
httpcore |
4.4.16 |
org.apache.ivy |
ivy |
2.5.2 |
org.apache.logging.log4j |
log4j-1.2-api |
2.22.1 |
org.apache.logging.log4j |
log4j-api |
2.22.1 |
org.apache.logging.log4j |
log4j-core |
2.22.1 |
org.apache.logging.log4j |
log4j-layout-template-json |
2.22.1 |
org.apache.logging.log4j |
log4j-slf4j2-impl |
2.22.1 |
org.apache.orc |
orc-core |
1.9.2-shaded-protobuf |
org.apache.orc |
orc-mapreduce |
1.9.2-shaded-protobuf |
org.apache.orc |
orc-shims |
1.9.2 |
org.apache.thrift |
libfb303 |
0.9.3 |
org.apache.thrift |
libthrift |
0.12.0 |
org.apache.ws.xmlschema |
xmlschema-core |
2.3.0 |
org.apache.xbean |
xbean-asm9-shaded |
4.23 |
org.apache.yetus |
audience-annotations |
0.13.0 |
org.apache.zookeeper |
zookeeper |
3.9.2 |
org.apache.zookeeper |
zookeeper-jute |
3.9.2 |
org.checkerframework |
checker-qual |
3.31.0 |
org.codehaus.jackson |
jackson-core-asl |
1.9.13 |
org.codehaus.jackson |
jackson-mapper-asl |
1.9.13 |
org.codehaus.janino |
commons-compiler |
3.0.16 |
org.codehaus.janino |
janino |
3.0.16 |
org.datanucleus |
datanucleus-api-jdo |
4.2.4 |
org.datanucleus |
datanucleus-core |
4.1.17 |
org.datanucleus |
datanucleus-rdbms |
4.1.19 |
org.datanucleus |
javax.jdo |
3.2.0-m3 |
org.eclipse.collections |
eclipse-collections |
11.1.0 |
org.eclipse.collections |
eclipse-collections-api |
11.1.0 |
org.eclipse.jetty |
jetty-client |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-continuation |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-http |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-io |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-jndi |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-plus |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-proxy |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-security |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-server |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-servlet |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-servlets |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-util |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-util-ajax |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-webapp |
9.4.52.v20230823 |
org.eclipse.jetty |
jetty-xml |
9.4.52.v20230823 |
org.eclipse.jetty.websocket |
websocket-api |
9.4.52.v20230823 |
org.eclipse.jetty.websocket |
websocket-client |
9.4.52.v20230823 |
org.eclipse.jetty.websocket |
websocket-common |
9.4.52.v20230823 |
org.eclipse.jetty.websocket |
websocket-server |
9.4.52.v20230823 |
org.eclipse.jetty.websocket |
websocket-servlet |
9.4.52.v20230823 |
org.fusesource.leveldbjni |
leveldbjni-all |
1.8 |
org.glassfish.hk2 |
hk2-api |
2.6.1 |
org.glassfish.hk2 |
hk2-locator |
2.6.1 |
org.glassfish.hk2 |
hk2-utils |
2.6.1 |
org.glassfish.hk2 |
osgi-resource-locator |
1.0.3 |
org.glassfish.hk2.external |
aopalliance-repackaged |
2.6.1 |
org.glassfish.hk2.external |
jakarta.inject |
2.6.1 |
org.glassfish.jersey.containers |
jersey-container-servlet |
2.40 |
org.glassfish.jersey.containers |
jersey-container-servlet-core |
2.40 |
org.glassfish.jersey.core |
jersey-client |
2.40 |
org.glassfish.jersey.core |
jersey-common |
2.40 |
org.glassfish.jersey.core |
jersey-server |
2.40 |
org.glassfish.jersey.inject |
jersey-hk2 |
2.40 |
org.hibernate.validator |
hibernate-validator |
6.1.7.Final |
org.ini4j |
ini4j |
0.5.4 |
org.javassist |
javassist |
3.29.2-GA |
org.jboss.logging |
jboss-logging |
3.3.2.Final |
org.jdbi |
jdbi |
2.63.1 |
org.jetbrains |
annotations |
17.0.0 |
org.joda |
joda-convert |
1.7 |
org.jodd |
jodd-core |
3.5.2 |
org.json4s |
json4s-ast_2.12 |
3.7.0-M11 |
org.json4s |
json4s-core_2.12 |
3.7.0-M11 |
org.json4s |
json4s-jackson_2.12 |
3.7.0-M11 |
org.json4s |
json4s-scalap_2.12 |
3.7.0-M11 |
org.lz4 |
lz4-java |
1.8.0-databricks-1 |
org.mlflow |
mlflow-spark_2.12 |
2.9.1 |
org.objenesis |
objenesis |
2.5.1 |
org.postgresql |
postgresql |
42.6.1 |
org.roaringbitmap |
RoaringBitmap |
0.9.45-databricks |
org.roaringbitmap |
shims |
0.9.45-databricks |
org.rocksdb |
rocksdbjni |
9.2.1 |
org.rosuda.REngine |
REngine |
2.1.0 |
org.scala-lang |
scala-compiler_2.12 |
2.12.15 |
org.scala-lang |
scala-library_2.12 |
2.12.15 |
org.scala-lang |
scala-reflect_2.12 |
2.12.15 |
org.scala-lang.modules |
scala-collection-compat_2.12 |
2.11.0 |
org.scala-lang.modules |
scala-java8-compat_2.12 |
0.9.1 |
org.scala-lang.modules |
scala-parser-combinators_2.12 |
1.1.2 |
org.scala-lang.modules |
scala-xml_2.12 |
1.2.0 |
org.scala-sbt |
test-interface |
1.0 |
org.scalacheck |
scalacheck_2.12 |
1.14.2 |
org.scalactic |
scalactic_2.12 |
3.2.16 |
org.scalanlp |
breeze-macros_2.12 |
2.1.0 |
org.scalanlp |
breeze_2.12 |
2.1.0 |
org.scalatest |
scalatest-compatible |
3.2.16 |
org.scalatest |
scalatest-core_2.12 |
3.2.16 |
org.scalatest |
scalatest-diagrams_2.12 |
3.2.16 |
org.scalatest |
scalatest-featurespec_2.12 |
3.2.16 |
org.scalatest |
scalatest-flatspec_2.12 |
3.2.16 |
org.scalatest |
scalatest-freespec_2.12 |
3.2.16 |
org.scalatest |
scalatest-funspec_2.12 |
3.2.16 |
org.scalatest |
scalatest-funsuite_2.12 |
3.2.16 |
org.scalatest |
scalatest-matchers-core_2.12 |
3.2.16 |
org.scalatest |
scalatest-mustmatchers_2.12 |
3.2.16 |
org.scalatest |
scalatest-propspec_2.12 |
3.2.16 |
org.scalatest |
scalatest-refspec_2.12 |
3.2.16 |
org.scalatest |
scalatest-shouldmatchers_2.12 |
3.2.16 |
org.scalatest |
scalatest-wordspec_2.12 |
3.2.16 |
org.scalatest |
scalatest_2.12 |
3.2.16 |
org.slf4j |
jcl-over-slf4j |
2.0.7 |
org.slf4j |
jul-to-slf4j |
2.0.7 |
org.slf4j |
slf4j-api |
2.0.7 |
org.slf4j |
slf4j-simple |
1.7.25 |
org.threeten |
threeten-extra |
1.7.1 |
org.tukaani |
xz |
1.9 |
org.typelevel |
algebra_2.12 |
2.0.1 |
org.typelevel |
cats-kernel_2.12 |
2.1.1 |
org.typelevel |
spire-macros_2.12 |
0.17.0 |
org.typelevel |
spire-platform_2.12 |
0.17.0 |
org.typelevel |
spire-util_2.12 |
0.17.0 |
org.typelevel |
spire_2.12 |
0.17.0 |
org.wildfly.openssl |
wildfly-openssl |
1.1.3.Final |
org.xerial |
sqlite-jdbc |
3.42.0.0 |
org.xerial.snappy |
snappy-java |
1.1.10.3 |
org.yaml |
snakeyaml |
2.0 |
oro |
oro |
2.0.8 |
pl.edu.icm |
JLargeArrays |
1.5 |
software.amazon.cryptools |
AmazonCorrettoCryptoProvider |
2.4.1-linux-x86_64 |
stax |
stax-api |
1.0.1 |