Skip to content

Custom JDBC Driver Requirements

  • Self-hosted

This page is the contract a custom driver must satisfy: what you ship, how it must be packaged, the YAML definition keys, the connection field vocabulary, and the validation rules the Dataplane enforces at startup. It closes with a complete, working example.

What You Ship

Each custom driver is two artifacts placed on the Dataplane classpath:

  1. The vendor JDBC driver JAR: the third-party .jar containing the JDBC driver class and its dependencies.
  2. The YAML driver definition: a single file at META-INF/jdbc-drivers/<prefix>.yaml.

The YAML can ride inside the vendor JAR or in its own JAR, as long as both are on the classpath when the Dataplane starts.

The JAR must record the META-INF/jdbc-drivers/ directory entry

Discovery only finds JARs whose archive lists the directory itself, not just the files inside it. The standard jar tool records directory entries by default, but some zip tools and build plugins write only the file entries, and such a JAR is silently invisible: the YAML is on the classpath, yet the driver never registers. Run jar tf my-driver.jar and confirm a META-INF/jdbc-drivers/ line ending in / is present.

The YAML Definition

The definition has three top-level keys:

Key Required Purpose
config Yes Driver identity, URL template, and the connection form fields.
sql Yes SQL capability declarations. sql: {} is valid and is the common case.
dialectClass No Fully qualified name of a dialect class shipped in the JAR, for databases whose identifier quoting or type mapping needs it. Most drivers do not need one. See The dialectClass Contract.

Inside config, the required keys are prefix, className, displayName, tableNameCasing, transactionIsolation, url.template, and connectionSpec. Useful optional keys:

Key Default Purpose
connectionTest SELECT 1 The statement used to test a connection, for databases that need something else (for example, SELECT 1 FROM DUAL).
defaultPort none The port pre-filled on the connection form when the driver declares a port field.
connectionSpec.supportsEnrichment true Whether the connector supports enrichment datastores. Set to false for read-only or query-only engines.

Connection Form Fields

Every entry under connectionSpec.fields describes one control on the connection form and one input available to the URL template. The vocabulary is closed: these are all the keys a field accepts.

Key Required Purpose
name Yes The field's identifier. It is how URL placeholders and property mappings reference the value.
label Yes The label shown on the connection form.
fieldType Yes One of string, integer, boolean, password, file, enum, or secret.
required No (default true) Whether the form demands a value.
defaultValue No The value used when the form leaves the field empty.
hint No Help text shown with the field.
options No The choices for an enum field.
dependsOn, dependsOnValue, dependsOnValues No Shows the field only when another field holds the given value (or one of the given values).
aliases No Alternative names accepted for the same value, for compatibility with existing payloads.

Fields named exactly username, password, database, and schema map to the four canonical connection inputs; every other field travels in the connection's parameters map and is referenced as parameters.<name> in property mappings. See What the Connection Form Sends.

Validation Rules

Validation is strict, and it protects the deployment rather than the file:

  • Unknown keys, missing required keys, and duplicate keys are rejected. There is no pass-through for unrecognized settings.
  • The YAML file name must match config.prefix (exampledb.yaml pairs with prefix: exampledb).
  • Every {placeholder} in the URL template must correspond to a field declared under connectionSpec.fields, and placeholders in the always-applied template must be required fields or carry a defaultValue.
  • A custom driver cannot reuse the prefix of a built-in connector, and two custom drivers cannot share a prefix.
  • A URL template that joins parameters with ; or , must declare that separator in url.paramSeparator.

When any definition fails validation, the Dataplane refuses to start instead of silently dropping the driver, and its startup log names the offending file, the key, and the reason. Fix every reported error and redeploy.

The dialectClass Contract

When a definition declares dialectClass, the class it names must:

  • ship inside a JAR on the same classpath as the driver;
  • extend org.apache.spark.sql.jdbc.JdbcDialect;
  • for a Scala object, be referenced with a trailing $ (for example, com.example.ExampleDialect$);
  • for a Java class, expose a public no-argument constructor.

A class that cannot be loaded, does not extend the required parent, or throws from its constructor fails registry startup like any other validation error.

Example

The definition below registers a fictional exampledb database with host, port, database, and schema fields, an optional token-based authentication variant, and standard username and password credentials:

config:
  prefix: exampledb
  className: com.example.ExampleDriver
  displayName: Example DB
  tableNameCasing: AS_IS
  transactionIsolation: READ_COMMITTED
  defaultPort: 1234
  url:
    template: "jdbc:exampledb://{host}:{port}/{database}"
    staticParams:
      - "ssl=true"
    conditionalParams:
      - key: schema
        param: "currentSchema={schema}"
    authVariants:
      TOKEN:
        urlTemplate: "jdbc:exampledb://{host}:{port}/{database}?authMethod=token"
        connectionPropertyMappings:
          access_token: parameters.access_token
  connectionSpec:
    fields:
      - name: host
        label: Host
        fieldType: string
      - name: port
        label: Port
        fieldType: integer
        required: false
        defaultValue: "1234"
      - name: database
        label: Database
        fieldType: string
      - name: schema
        label: Schema
        fieldType: string
        required: false
      - name: authentication_type
        label: Authentication
        fieldType: enum
        required: false
        options:
          - BASIC
          - TOKEN
      - name: access_token
        label: Access Token
        fieldType: secret
        required: false
        dependsOn: authentication_type
        dependsOnValue: TOKEN
      - name: username
        label: Username
        fieldType: string
        required: false
      - name: password
        label: Password
        fieldType: password
        required: false
sql: {}

With this definition, a connection for host db.example.com and database ORDERS that leaves the port on its default produces jdbc:exampledb://db.example.com:1234/ORDERS?ssl=true. Choosing the TOKEN authentication type instead switches the URL to the variant template and passes the access token as a connection property. The assembly rules behind this are on How It Works.