Saturday, 21 March 2026

Databricks Workspace Resources – Complete Guide

Databricks Workspace Resources – Complete Guide

Databricks Workspace Resources – Full Guide

A Databricks workspace provides an environment where you can create, organize, and manage compute resources, data objects, automation workflows, analytics assets, and machine learning components. The Databricks UI supports creating notebooks, queries, dashboards, jobs, pipelines, experiments, models, and more through the + New menu and the workspace sidebar [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). Workspace objects include notebooks, jobs, libraries, data files, experiments, and more [2](https://www.youtube.com/watch?v=cNFKzWpRvsw).

Summary Table of Creatable Databricks Workspace Resources

Resource Description How to Create (UI Steps)
Notebook Interactive document for Python, SQL, R, Scala code execution [2](https://www.youtube.com/watch?v=cNFKzWpRvsw). 1. Click + NewNotebook.
2. Enter notebook name.
3. Choose language.
4. Select compute (cluster not covered).
5. Click Create [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline).
Query SQL query used for dashboards & alerts [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). 1. Click + NewQuery.
2. SQL Editor opens.
3. Select SQL Warehouse.
4. Write SQL and click Save.
Dashboard Visual BI dashboard created from queries [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). 1. Open a saved query.
2. Click Add to Dashboard.
3. Create new or choose existing.
4. Arrange visuals → Save.
Alert Condition-based SQL alert [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). 1. Open a SQL Query.
2. Click Create Alert.
3. Add condition + recipients.
4. Save.
Repo Git-connected source repo [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). 1. Click + NewRepo.
2. Choose Git provider.
3. Paste repository URL.
4. Authenticate and click Create.
File Workspace-level file (CSV, Python script, config) [2](https://www.youtube.com/watch?v=cNFKzWpRvsw). 1. Open Workspace browser.
2. Click Add → File Upload.
3. Upload file.
Library Install Python/JAR packages for use in notebooks/jobs [2](https://www.youtube.com/watch?v=cNFKzWpRvsw). 1. Go to Workspace → Libraries.
2. Click Install New.
3. Upload wheel/JAR or specify PyPI package.
4. Click Install.
Job Automation for notebooks, scripts, JARs, pipelines [2](https://www.youtube.com/watch?v=cNFKzWpRvsw). 1. Click Jobs in sidebar.
2. Click Create Job.
3. Name the job.
4. Click Add Task and choose task type.
5. Configure task details.
6. Assign compute (cluster selection only).
7. Add schedule if needed.
8. Click Create [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline).
Pipeline DLT / Lakeflow ETL pipelines (triggered or continuous) [3](https://docs.databricks.com/aws/en/getting-started/concepts). 1. Click Jobs & Pipelines.
2. Click Create Pipeline.
3. Enter name.
4. Select pipeline mode.
5. Add SQL/Python pipeline code.
6. Select target catalog/schema.
7. Configure settings.
8. Click Create.
Experiment MLflow tracking experiment for ML models [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). 1. Click + NewExperiment.
2. Enter name and location.
3. Click Create.
Model MLflow model stored in Model Registry [4](https://devstacktips.com/development/programming-languages/2025/06/06/mastering-databricks-jobs-api-build-and-orchestrate-complex-data-pipelines/). 1. Open an MLflow run.
2. Click Register Model.
3. Select or create model name.
4. Register.
Serving Endpoint Real-time inference endpoint for ML models [1](https://learn.microsoft.com/en-us/azure/databricks/jobs/pipeline). 1. Click + NewServing Endpoint.
2. Select model.
3. Configure autoscaling.
4. Click Create Endpoint.

Visual Diagram of All Databricks Workspace Resources

The following diagram shows how notebooks, pipelines, jobs, dashboards, alerts, and ML workflows connect logically inside a Databricks workspace.

```mermaid
flowchart TD

A[Notebook] --> B[Job]
B --> C[Pipeline]
C --> D[Tables / Data Assets]

A --> E[Experiment]
E --> F[Model]
F --> G[Serving Endpoint]

B --> H[Dashboards]
H --> I[Alerts]

J[Repo] --> A
K[Files / Libraries] --> A

No comments:

Post a Comment