An application can authenticate every user correctly and still expose the wrong data. If its backend queries through a powerful service principal, the database evaluates that principal’s permissions—not the permissions of the person clicking “Search”.
Our work at RIVM includes Unity Catalog governance and a self-service data onboarding platform, described in our onboarding write-up. That governance context makes the distinction between application access and user access particularly relevant.
We’ll turn on-behalf-of-user authorization, or OBO, into an application architecture: user-delegated queries, separate app-owned operations, narrow scopes and request-local tokens. We’ll also define the negative tests that prove a less-privileged user cannot inherit the app’s access.
What Changed
Databricks has announced general availability of OBO authorization for Databricks Apps. According to the official permission-aware Apps announcement, calls to supported Databricks APIs can execute using the signed-in user’s identity, with Unity Catalog enforcing that user’s permissions, including row filters and column masks.
This complements app authorization, which uses the app’s dedicated service principal. It does not replace it: shared configuration, application metrics and background maintenance remain legitimate app-owned operations.
Three boundaries govern a delegated query. None substitutes for the others.
| Boundary | What it controls |
|---|---|
| App API scopes | Which supported API capabilities the app can exercise for the user |
| User’s SQL warehouse permissions | Whether the user can use the target warehouse |
| User’s Unity Catalog permissions and policies | Which governed data the query can return |
For read-only SQL, the announcement identifies sql:restricted-query. It permits read-only SQL queries, not other SQL operations; the broader sql scope is unnecessary for that use case.
User authorization can be configured in the UI or a Declarative Automation Bundle. Users consent to the requested scopes when first accessing a user-authorized app, and the workspace’s allowed-scope policy constrains what developers can request.
The supplied announcement does not enumerate supported clouds or regions. We would not interpret GA as confirmation that every intended Azure deployment is eligible.
For Azure West Europe (westeurope), Microsoft’s regional availability table lists support for both serverless compute and Databricks Apps. This confirms documented regional availability, not successful validation of a particular workspace.
The workspace and development environment guide and Azure authorization guide describe the remaining deployment and authorization requirements:
- Permit outbound network access to
*.databricksapps.com. - Use Databricks CLI version
0.229.0or later, configured for the target workspace. The bundle tutorial requires version0.250.0or later for its workflow. - Configure user authorization with the required, administrator-permitted scopes. The GA announcement documents
sql:restricted-queryfor read-only SQL. - Ensure the user has access to the target SQL warehouse and the required Unity Catalog resources.
- Keep the SQL warehouse in the same workspace as the app when using the forwarded user token.
These are documented requirements, not evidence of a validated workspace. Before declaring a deployment ready, check outbound connectivity, workspace scope policy, CLI configuration, resource permissions and successful OBO execution in the target workspace. This does not establish coverage for other Azure regions.
The Replit and Lakebase enterprise-app write-up illustrates why this boundary matters: applications can read live analytical data while maintaining operational state. A builder connector may use a shared service principal or the builder’s identity, but builder access and production end-user authorization are separate design decisions.
Why It Matters for Platform Teams
Authentication is not data authorization. Knowing who opened the app does not help Unity Catalog enforce their permissions if the app queries as somebody else.
When a response should vary with the requesting user’s data permissions, execute the governed operation as that user. Do not query a superset as the app and then attempt to reproduce row filters, column masks and group rules in Python.
Scopes provide a second boundary. A user might legitimately possess write privileges elsewhere, while a read-only application should still be unable to modify data on their behalf. Conversely, granting sql:restricted-query does not give an otherwise unauthorized user warehouse or table access.
This changes the platform team’s deliverable from “a working service principal” to an identity contract for each operation. That contract covers code paths, grants, scope policy, credential lifetime and evidence of the identity actually used.
It also reduces the need for privileged application credentials. The app principal should not require broad access to sensitive analytical data merely because the application presents that data to authorized users.
Use Case: Regional Public Health Analysis
Consider a national public health institute running a hypothetical internal analysis app on Azure. Regional analysts should see their own reporting areas, while a national surveillance team has broader access; sensitive fields require governed masking. Curated reporting data flows into Unity Catalog-governed analytical views, which the app queries through a SQL warehouse.
The usual implementation gives the app principal access to every region and adds regional filters in application code. This breaks when a new endpoint omits the filter, an export follows a different path, or a cached response is reused across users.
With OBO, the browser request reaches Databricks Apps, and the application passes the forwarded user token to its SQL connection. The warehouse evaluates the analyst’s Unity Catalog permissions and policies; the response contains only the resulting authorized data.
A separate app-authorized path reads non-sensitive configuration and writes operational metrics. A maintenance worker uses app authorization without relying on an analyst’s session or retaining their token.
| Component | Identity and responsibility |
|---|---|
| Interactive analysis endpoint | Requesting user; query governed analytical views |
| Configuration reader | App principal; read shared, non-sensitive settings |
| Metrics writer and maintenance worker | App principal; perform explicitly app-owned work |
This is a hypothetical design, not a claim that NexOps deployed OBO at RIVM. It builds on the public-health scenario’s governance constraints without inventing an implementation history.
We would measure authorization failures by cause, identity-attribution coverage, cross-user isolation failures and query latency. We would also track whether access-policy changes affect subsequent requests as expected, rather than treating a successful login as proof of correct governance.
Implementation
1. Assign an identity to every operation
Start with an operation inventory before changing credentials. A single generic “Databricks client” is an architectural smell when some operations belong to the app and others belong to a user.
| Operation | Identity | Required boundary |
|---|---|---|
| Read regional analytical data | User | Restricted SQL scope, warehouse permission and Unity Catalog access |
| Read shared display configuration | App | Narrow app-principal permissions |
| Record aggregate application health metrics | App | Dedicated operational destination |
| Execute a user-triggered governed action | User | Supported API and scope appropriate to that action |
| Perform scheduled maintenance | App | Explicit maintenance permissions |
Make the distinction visible in module names and dependencies, such as user_queries.py and app_metrics.py. User-query modules should not import the app-authentication helper.
A queued export needs a separate decision. Persisting the initiating user’s token is not an acceptable way to turn a request into background work—either finish within the request model or design a separately authorized workflow with explicit ownership and access rules.
2. Configure scopes and resource permissions independently
For the read-only application, declare sql:restricted-query through the supported user-authorization configuration. We would use the UI for the first controlled deployment rather than guessing bundle syntax absent from the supplied material.
Workspace administrators configure the scope allowlist under Settings / Development / Apps. The announcement states that it defaults to all supported APIs, can be narrowed, or can be set to None to disable user authorization.
Treat that default as something to review, not an approved enterprise baseline. Add genie or ai-gateway only for corresponding capabilities; do not request unrelated scopes such as files, model-serving or vector-search.
Grant users the required warehouse access and Unity Catalog privileges. For a conventional governed view, review USE CATALOG, USE SCHEMA and SELECT, alongside applicable row filters and masks—a scope declaration grants none of these.
Use groups to manage access rather than accumulating individual grants. The supplied identity-management documentation describes centralized identities and workspace assignment, and recommends identity-provider synchronization.
3. Build a request-local user query path
The announcement specifies the x-forwarded-access-token header and passing its value through the SQL connector’s access_token argument. Retrieve it inside the request handler, never at application startup.
The following Flask example uses application-defined environment variables and an allowlisted view mapping. It intentionally contains no app credential and no fallback authentication path.
import os
from databricks import sql
from flask import Flask, abort, jsonify, request
app = Flask(__name__)
# Application settings, not Databricks product configuration keys.
VIEW_BY_ENV = {
"dev": "health_dev.analytics.regional_case_summary",
"test": "health_test.analytics.regional_case_summary",
"acceptance": "health_acceptance.analytics.regional_case_summary",
"prod": "health_prod.analytics.regional_case_summary",
}
SUMMARY_VIEW = VIEW_BY_ENV[os.environ["APP_ENV"]]
def require_user_token():
token = request.headers.get("x-forwarded-access-token")
if not token:
# This HTTP status is our app's choice; never fall back to app auth.
abort(401, description="User authorization is required.")
return token
@app.get("/api/regional-summary")
def regional_summary():
token = require_user_token()
try:
# Keep the connection and credential local to this request.
with sql.connect(
server_hostname=os.environ["APP_SQL_HOSTNAME"],
http_path=os.environ["APP_SQL_HTTP_PATH"],
access_token=token,
) as connection:
with connection.cursor() as cursor:
# The identifier is allowlisted, never taken from browser input.
cursor.execute(
f"""
SELECT region_code, SUM(case_count) AS total_cases
FROM {SUMMARY_VIEW}
GROUP BY region_code
ORDER BY region_code
"""
)
rows = cursor.fetchall()
except Exception:
# Do not expose connector diagnostics or credentials to the browser.
app.logger.warning("User-authorized query failed.")
abort(502, description="The data query could not be completed.")
response = jsonify([
{"region": row[0], "case_count": row[1]}
for row in rows
])
# Prevent ordinary HTTP caching of this user-specific response.
response.headers["Cache-Control"] = "no-store"
return response
The connector receives only the current request’s user token, so the route cannot silently switch to app permissions. The broad exception handler is a minimal fail-closed example; production error classification should distinguish resource denial from availability failures without logging tokens, sensitive results or unsanitized exception payloads.
Deploy behind the Databricks Apps request boundary and review any additional proxy path so browser-supplied identity headers cannot substitute for platform forwarding. Never accept a user token through a form field or query parameter as an alternative.
Do not retain tokens in sessions, global clients, connection pools shared between users, job payloads or telemetry. The source describes the forwarded token as short-lived and per-request—obtain a fresh request token rather than guessing its lifetime.
4. Promote permissions and tests through DTAP
Promote the identity contract with the code through development, test, acceptance and production—DTAP. Keep the scope requirement consistent, while changing warehouses, catalogs, app identities and test fixtures per environment.
| Environment | Data and identity setup | Release gate |
|---|---|---|
| Development | Synthetic regional data; separate user and app paths | Missing-token handling and basic isolation |
| Test | Contrasting users with deliberately different permissions | Scope, warehouse, data-access and cross-user negative tests |
| Acceptance | Representative policies and operational topology | Consent flow, policy changes and audit attribution |
| Production | Approved groups, scopes and app-principal grants | Reviewed evidence and smoke tests using designated identities |
Never resolve a failed acceptance test by giving the app principal the missing analytical grants. First establish which user was used, which scope was configured and which resource permission was absent.
We use the same promotion discipline described in our declarative CI/CD guide. Version configuration and review environment differences instead of relying on remembered UI changes.
5. Prove that broader access cannot leak
Positive tests establish functionality; negative tests establish the boundary. Use real user-authorized integration requests in an isolated environment, not only mocked header values.
| Test | Setup | Required outcome |
|---|---|---|
| Missing forwarded token | Invoke the user-query handler without the header | Reject before opening a SQL connection |
| Missing warehouse access | User has data privileges but cannot use the warehouse | Fail without app-identity retry |
| Missing data access | User can use the warehouse but lacks access to the target data | Return no protected data |
| Row and column restrictions | Compare restricted and broader users | Each receives the policy-appropriate result |
| Capability ceiling | User has write privileges on a disposable fixture; app has only restricted-query scope | Delegated write attempt is rejected |
| App has broader fixture access | App principal can read synthetic data that the test user cannot | User path still denies that data |
| Cross-user concurrency | Alternate and overlap requests from contrasting users | No credential or result crossover |
| Policy change | Change a test grant or policy between requests | Subsequent requests reflect the updated permission boundary |
The broader-app-access fixture belongs in an isolated test environment. It is a deliberate regression trap, not justification for broader production grants.
Probe write-scope enforcement through a dedicated test harness against disposable data. Do not add a production endpoint that accepts arbitrary SQL merely to make the test convenient.
6. Verify attribution, not just application logs
Application logs show what the app recorded; platform audit records help investigate the identity and operation evaluated by Databricks. Keep this evidence separate for user-authorized operations and app-owned work.
The Azure Databricks audit log system table reference documents system.access.audit and relevant identity and correlation metadata, including identity_metadata.run_by, identity_metadata.run_as, acting-resource metadata, request_id, event_id, session_id, workspace_id and event_time.
When investigating a user-authorized SQL request and a separate app-authorized operation, inspect the available records and compare service_name, action_name, identity metadata and correlation fields. Use the fields actually present in the records; do not assume an application request identifier necessarily matches request_id.
Keep application request identifiers and sanitized operation metadata for correlation. Never log forwarded access tokens or sensitive query results.
Pitfalls and When Not to Use It
Never fall back to the app principal. Missing tokens, expired credentials and permission failures are errors to handle—not opportunities to return a more privileged result.
OBO does not fix shared caches. An authorized query result can still leak through application memory, generated files or downstream services. Avoid shared user-data caching initially; any later cache needs an explicit isolation and policy-invalidation design.
Removing a scope is not an immediate kill switch. The announcement states that apps already running with a scope removed from the workspace allowlist can continue running, although they cannot be started, deployed or updated until that scope is removed from their configuration. Account administrators can also add scopes outside the workspace allowlist, so include those exceptions in governance reviews.
We would not use user authorization for shared configuration, application metrics or scheduled maintenance. Those operations need stable, narrowly permissioned app identities, not dependence on an employee’s browser session.
We would also reject a design that uses OBO for the first SQL query but sends its results to an unreviewed summarization service. Governance must cover the complete response path, not just the warehouse call.
Key Takeaways
- Assign an identity per operation. Use OBO for user-governed access and app authorization for explicitly app-owned work.
- Separate scopes from grants. Start with
sql:restricted-query, then verify warehouse access and Unity Catalog permissions independently. - Keep tokens request-local. Never persist them, share user clients or fall back to app credentials.
- Promote negative tests through DTAP. Prove that users cannot inherit broader app access or another user’s results.
- Verify platform attribution. Require observed audit evidence before claiming that the evaluated identity is confirmed.
- Treat GA as a reason to decide, not to skip review. Check target-workspace availability, administrator exceptions and every downstream data path.
Building RIVM’s Self-Service Data Onboarding Platform
How we approach self-service data onboarding at RIVM: validated configuration, governed access and automated deployment without removing human accountability.
AI Engineering · 13 minInference on Azure Databricks: What Drives Cost and Latency
Understand inference cost and latency, measure tokens and capacity, and choose batch, pay-per-token, provisioned or self-hosted serving on Azure Databricks.
Case study · Data Platform EngineeringHow we did this at RIVM Data Platform →
Centralised Azure & Databricks data platform with a self-service data onboarding UI for the Dutch National Institute for Public Health.