Dataplane Card
The Dataplane is the service that runs every data quality operation. It is an independent service that talks to the rest of the platform through RabbitMQ, and depending on your deployment it runs on Kubernetes, Databricks, or a local host.
The card has a Configuration subtab that everyone with access sees, and, for Admins only, a Diagnostics subtab and a Restart button in its header.
| SUBTAB | PERMISSION | WHAT IT SHOWS |
|---|---|---|
| Configuration | Manager, Admin | The engine's fixed allocation plus three live capacity readouts. |
| Diagnostics | Admin only | A captured report of what the engine is doing right now. See Dataplane Diagnostics. |
| HEADER ACTION | PERMISSION | WHAT IT DOES |
|---|---|---|
| Restart | Admin only | Sends a graceful shutdown signal, which triggers an immediate restart of the engine. Opens a confirmation dialog first. See Restart Dataplane. |
Note
The Dataplane reports unhealthy while it restarts. After triggering a restart, wait a few seconds before refreshing the status.
Configuration
These values come from the running engine and describe the resources it was allocated. They all display -- while the Dataplane is unreachable.
| FIELD | WHAT IT SHOWS |
|---|---|
| Implementation Version | The version of the Dataplane currently running, for example 2.8.1. Comparing it with Version on the Deployment card confirms that both layers are on compatible releases. |
| Spark Version | The version of Apache Spark used for processing, for example 3.5.1. |
| Max Executors | The ceiling on executor nodes available for parallel processing. Each executor is an independent worker that can run operations concurrently. |
| Cores Per Executor | The CPU cores assigned to each executor. More cores speed up compute-intensive operations. |
| Max Memory Per Executor | The memory ceiling for each executor, in MB. Higher values let a single executor process larger datasets before spilling to disk. |
| Max Dataframe Size | The largest single dataframe the engine will process, in MB. This bounds how much data one operation can load into memory at a time. |
Capacity Readouts
Three readouts sit below the configuration fields. Unlike the fields above, these change continuously with the workload.
| READOUT | VALUE | CAPTION | HOW TO READ IT |
|---|---|---|---|
| Thread Pool State | N running • M queued |
The IDs of the running operations, shown as pills. A +N more pill appears when they do not all fit, and hovering it lists the rest. | Running is what the engine is executing now. Queued is what is waiting for a free slot. A queue that stays long means the workload is outrunning the available executors. When nothing is running or queued, the caption reads Idle. |
| Executor Capacity | <live> / <max> running |
<pending> pending tasks vs <slots> capacity slots |
Live executors against the configured ceiling, with the work waiting against the slots available to absorb it. Pending tasks persistently above capacity slots is the signal behind the warnings below. |
| Sync Capacity | <in flight> / <granted> running |
driver CPU <N>% • <W> waiting • max <C> |
Sync operations get their own admission lane whose size is adjusted automatically based on CPU load. The granted value moves between a floor and the configured maximum, so it can read lower than the maximum while the engine is under CPU pressure. |
Note
Executor Capacity and Sync Capacity only appear when the running engine reports them. Deployments on Databricks, and Dataplane builds that predate these measurements, omit them rather than showing zeros.
Info
If the engine returns a Thread Pool State message the page does not recognize, the message is shown exactly as reported instead of being split into running and queued counts.
Capacity Warnings
When executor demand stays unmet for a sustained period, a warning appears above the readouts.
| WARNING | WHAT IT MEANS | WHAT TO DO |
|---|---|---|
| Cluster autoscaling may be misconfigured | The workload needs more executors than are running, and the Dataplane has not yet reached its configured maximum. Executors are being requested but are not arriving. | Check that node pool autoscaling is enabled and that nodes are schedulable. For managed deployments, contact the Qualytics support team. |
| Executor capacity may be undersized | The workload has more queued than the configured maximum number of executors can absorb. The ceiling itself is the limit, not the cluster. | Raise the maximum executors for the environment, or move to a larger deployment size. |
Tip
If the Dataplane reports unhealthy, refresh the status first. If it stays unhealthy, an Admin can restart it. See Restart Dataplane.