Types of Databricks APIs
Databricks provides a rich set of APIs to manage both the platform and workspace workloads. These APIs are categorized based on their scope and functionality, and they are critical for automation, CI/CD, governance, and onboarding of applications.
1. Account-Level APIs (Control Plane)
These APIs manage the Databricks account itself. They allow platform engineers to create and configure workspaces, set up Unity Catalog (metastores), manage networking, storage credentials, and service principals.
Official Databricks Account API Docs
2. Workspace-Level APIs (Data Plane)
These APIs operate inside a single workspace to manage data workloads such as:
- Clusters, Jobs, and Libraries
- DBFS file storage
- Secrets & instance pools
- SQL Warehouses
Official Workspace REST API Docs
3. Unity Catalog / Metastore APIs
These APIs manage metadata, governance, and data access across multiple workspaces:
- Create catalogs, schemas, tables, and external locations
- Grant permissions at table, column, or catalog level
- Attach or detach workspaces to a metastore
4. Repos API
Used to manage Git repositories integrated with Databricks (GitHub, GitLab, Azure DevOps). Enables CI/CD automation for notebooks.
5. Tokens & Authentication APIs
Used to manage personal access tokens (PATs) and service principal tokens for automation pipelines.
6. SCIM API
Manages users, groups, and service principals for identity management and enterprise compliance. SCIM v2.1 is standard.
7. SQL API
Enables programmatic execution of SQL queries and management of SQL endpoints / warehouses.
8. MLflow API
Manages the machine learning lifecycle including experiments, runs, and model registry.
Summary Table of Databricks APIs
| API Type | Scope | Purpose | Official Documentation |
|---|---|---|---|
| Account API | Account | Platform setup & governance: workspaces, metastore, network, credentials, service principals | Docs |
| Workspace REST API | Workspace | Data plane workloads: clusters, jobs, DBFS, libraries, secrets | Docs |
| SCIM API | Workspace / Account | Identity management: users, groups, service principals | Docs |
| Unity Catalog / Metastore API | Account + Workspaces | Data governance, catalogs, schemas, tables, permissions, external locations | Docs |
| Repos API | Workspace | Git repository integration for CI/CD | Docs |
| Tokens / Authentication API | Account / Workspace | Manage PATs & service principal tokens | Docs |
| SQL API | Workspace | Programmatic SQL execution & SQL endpoint management | Docs |
| MLflow API | Workspace | Machine learning lifecycle: experiments, runs, model registry | Docs |
For the full index of all Databricks APIs and SDKs: Databricks API Reference
No comments:
Post a Comment