Databricks Workspace Resources – Full Guide
A Databricks workspace provides an environment where you can create, organize, and manage compute resources, data objects, automation workflows, analytics assets, and machine learning components. The Databricks UI supports creating notebooks, queries, dashboards, jobs, pipelines, experiments, models, and more through the + New menu and the workspace sidebar [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). Workspace objects include notebooks, jobs, libraries, data files, experiments, and more [2](https://www.youtube.com/watch?v=cNFKzWpRvsw).
Summary Table of Creatable Databricks Workspace Resources
| Resource | Description | How to Create (UI Steps) |
|---|---|---|
| Notebook | Interactive document for Python, SQL, R, Scala code execution [2](https://www.youtube.com/watch?v=cNFKzWpRvsw). |
1. Click + New → Notebook. 2. Enter notebook name. 3. Choose language. 4. Select compute (cluster not covered). 5. Click Create [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). |
| Query | SQL query used for dashboards & alerts [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). |
1. Click + New → Query. 2. SQL Editor opens. 3. Select SQL Warehouse. 4. Write SQL and click Save. |
| Dashboard | Visual BI dashboard created from queries [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). |
1. Open a saved query. 2. Click Add to Dashboard. 3. Create new or choose existing. 4. Arrange visuals → Save. |
| Alert | Condition-based SQL alert [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). |
1. Open a SQL Query. 2. Click Create Alert. 3. Add condition + recipients. 4. Save. |
| Repo | Git-connected source repo [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). |
1. Click + New → Repo. 2. Choose Git provider. 3. Paste repository URL. 4. Authenticate and click Create. |
| File | Workspace-level file (CSV, Python script, config) [2](https://www.youtube.com/watch?v=cNFKzWpRvsw). |
1. Open Workspace browser. 2. Click Add → File Upload. 3. Upload file. |
| Library | Install Python/JAR packages for use in notebooks/jobs [2](https://www.youtube.com/watch?v=cNFKzWpRvsw). |
1. Go to Workspace → Libraries. 2. Click Install New. 3. Upload wheel/JAR or specify PyPI package. 4. Click Install. |
| Job | Automation for notebooks, scripts, JARs, pipelines [2](https://www.youtube.com/watch?v=cNFKzWpRvsw). |
1. Click Jobs in sidebar. 2. Click Create Job. 3. Name the job. 4. Click Add Task and choose task type. 5. Configure task details. 6. Assign compute (cluster selection only). 7. Add schedule if needed. 8. Click Create [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). |
| Pipeline | DLT / Lakeflow ETL pipelines (triggered or continuous) [3](https://docs.databricks.com/aws/en/getting-started/concepts). |
1. Click Jobs & Pipelines. 2. Click Create Pipeline. 3. Enter name. 4. Select pipeline mode. 5. Add SQL/Python pipeline code. 6. Select target catalog/schema. 7. Configure settings. 8. Click Create. |
| Experiment | MLflow tracking experiment for ML models [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). |
1. Click + New → Experiment. 2. Enter name and location. 3. Click Create. |
| Model | MLflow model stored in Model Registry [4](https://devstacktips.com/development/programming-languages/2025/06/06/mastering-databricks-jobs-api-build-and-orchestrate-complex-data-pipelines/). |
1. Open an MLflow run. 2. Click Register Model. 3. Select or create model name. 4. Register. |
| Serving Endpoint | Real-time inference endpoint for ML models [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). |
1. Click + New → Serving Endpoint. 2. Select model. 3. Configure autoscaling. 4. Click Create Endpoint. |
Visual Diagram of All Databricks Workspace Resources
The following diagram shows how notebooks, pipelines, jobs, dashboards, alerts, and ML workflows connect logically inside a Databricks workspace.
```mermaid flowchart TD A[Notebook] --> B[Job] B --> C[Pipeline] C --> D[Tables / Data Assets] A --> E[Experiment] E --> F[Model] F --> G[Serving Endpoint] B --> H[Dashboards] H --> I[Alerts] J[Repo] --> A K[Files / Libraries] --> A
No comments:
Post a Comment