Commit Graph

6145 Commits

Author SHA1 Message Date
813da349ec fix(api,web): post-review hardening for OAuth device flow
- api: account-flow stores subject_issuer="dify:account" sentinel
  instead of NULL so the rotate-in-place unique index collides as
  intended (Postgres treats NULLs as distinct in unique indices).
  mint_oauth_token validates prefix-specific issuer rules.
- api: enterprise_only inverts to an allowlist (ACTIVE / EXPIRING) so
  any future LicenseStatus value defaults to denial.
- api: consume_on_poll moved to a single Lua script (GET + status-check
  + DEL) so concurrent pollers can't both observe APPROVED.
- web: typed DeviceFlowError + central error-copy mapping; page
  surfaces rate_limited / lookup_failed view states; URL params
  scrubbed after consumption (RFC 8628 §5.4).
2026-04-26 23:05:07 -07:00
fe8510ad1a feat(api,web): OAuth 2.0 device flow + bearer auth (RFC 8628)
Adds a CLI-friendly authorization flow so difyctl (and future
non-browser clients) can obtain user-scoped tokens without copy-
pasting cookies or raw API keys. Two grant paths share one device
flow surface:

  1. Account branch — user signs in via the existing /signin
     methods, /device page calls console-authed approve, mints a
     dfoa_ token tied to (account_id, tenant).
  2. External-SSO branch (EE) — /v1/oauth/device/sso-initiate signs
     an SSOState envelope, hands off to Enterprise's external ACS,
     receives a signed external-subject assertion, mints a dfoe_
     token tied to (subject_email, subject_issuer).

API surface (all under /v1, EE-only endpoints 404 on CE):

  POST   /v1/oauth/device/code              — RFC 8628 start
  POST   /v1/oauth/device/token             — RFC 8628 poll
  GET    /v1/oauth/device/lookup            — pre-validate user_code
  GET    /v1/oauth/device/sso-initiate      — SSO branch entry
  GET    /v1/device/sso-complete            — SSO callback sink
  GET    /v1/oauth/device/approval-context  — /device cookie probe
  POST   /v1/oauth/device/approve-external  — SSO approve
  GET    /v1/me                             — bearer subject lookup
  DELETE /v1/oauth/authorizations/self      — self-revoke
  POST   /console/api/oauth/device/approve  — account approve
  POST   /console/api/oauth/device/deny     — account deny

Core primitives:
- libs/oauth_bearer.py: prefix-keyed TokenKindRegistry +
  BearerAuthenticator + validate_bearer decorator. Two-tier scope
  (full vs apps:run) stamped from the registry, never from the DB.
- libs/jws.py: HS256 compact JWS keyed on the shared Dify
  SECRET_KEY — same key-set verifies the SSOState envelope, the
  external-subject assertion (minted by Enterprise), and the
  approval-grant cookie.
- libs/device_flow_security.py: enterprise_only gate, approval-
  grant cookie mint/verify/consume (Path=/v1/oauth/device,
  HttpOnly, SameSite=Lax, Secure follows is_secure()), anti-
  framing headers.
- libs/rate_limit.py: typed RateLimit / RateLimitScope dispatch
  with composite-key buckets; both decorator + imperative form.
- services/oauth_device_flow.py: Redis state machine (PENDING ->
  APPROVED|DENIED with atomic consume-on-poll), token mint via
  partial unique index uq_oauth_active_per_device (rotates in
  place), env-driven TTL policy.

Storage: oauth_access_tokens table with partial unique index on
(subject_email, subject_issuer, client_id, device_label) WHERE
revoked_at IS NULL. account_id NULL distinguishes external-SSO
rows. Hard-expire is CAS UPDATE (revoked_at + nullify token_hash)
so audit events keep their token_id. Retention pruner DELETEs
revoked + zombie-expired rows past OAUTH_ACCESS_TOKEN_RETENTION_DAYS.

Frontend: /device page with code-entry, chooser (account vs SSO),
authorize-account, authorize-sso views. SSO branch detaches from
the URL user_code and reads everything from the cookie via
/approval-context. Anti-framing headers on all responses.

Wiring: ENABLE_OAUTH_BEARER feature flag; ext_oauth_bearer binds
the authenticator at startup; clean_oauth_access_tokens_task
scheduled in ext_celery.

Spec: docs/specs/v1.0/server/{device-flow,tokens,middleware,security}.md
2026-04-26 20:06:43 -07:00
8f070f2190 refactor(api): flatten nested conditionals and clean up token helpers (#34835) 2026-04-20 03:39:54 +00:00
97bb338e7d fix: prevent double /v1 in MCP server URL causing 404 authorization failure (#34596)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-20 02:42:59 +00:00
ceec00e172 fix: accept icon type in app icon updates (#35360)
Co-authored-by: Stephen Zhou <hi@hyoban.cc>
2026-04-20 02:33:49 +00:00
059d605a6d ci: Update pyrefly dependency version to 0.61.1 (#35391)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-19 15:49:24 +00:00
f8a4205ac4 chore: Replace 'db' with 'sa' for SQLAlchemy compatibility (#35373)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-19 09:56:19 +00:00
99
3e876e173a chore(api): adapt Graphon 0.2.2 upgrade (#35377)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-18 11:16:24 +00:00
f980d5d3be feat: copy nodes cross apps (#33273)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-17 10:02:26 +00:00
b9c300d570 chore(api): migrate mail task and OAuth data source to use Session(db… (#35235)
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-17 08:52:27 +00:00
0020aa8f59 refactor(api): type pipeline template retrieval dicts with TypedDict (#34874) 2026-04-17 08:13:54 +00:00
e70e4fa41d chore(api): migrate file factory builders and account commands to use Session(db.engine) (#35236)
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-17 08:12:31 +00:00
881a9a1a08 refactor(api): move trace providers (#35144)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-17 07:53:35 +00:00
a74e12809b fix: move remote credential validation outside DB session to prevent … (#35350)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-17 07:42:29 +00:00
0c41d0bf51 fix: guard against KeyError in update_prompt_message_tool loop (#35150)
Co-authored-by: Crazywoola <100913391+crazywoola@users.noreply.github.com>
2026-04-17 07:09:40 +00:00
90e281c8da test: migrate dataset service document mock tests to testcontainers (#35191)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-17 07:03:30 +00:00
90d638fba3 refactor: migrate DocumentSegmentSummary to TypeBase (#34862) 2026-04-17 06:58:32 +00:00
de15e5b449 fix: scope plugin inner API end-user lookup by tenant (#35325) 2026-04-17 06:12:07 +00:00
af8ffa0654 refactor: use sessionmaker in api_tools_manage_service.py (#34892)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-17 05:07:12 +00:00
af21dc7df8 refactor(api): migrate dataset document response schemas to BaseModel (#35298)
Co-authored-by: ai-hpc <ai-hpc@users.noreply.github.com>
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-17 05:02:04 +00:00
13a9359191 refactor(api): migrate console mcp-server responses to BaseModel (#35219)
Co-authored-by: ai-hpc <ai-hpc@users.noreply.github.com>
2026-04-17 04:16:14 +00:00
f07f9ee7a3 chore(api): migrate event handlers to use Session(db.engine) (#35234) 2026-04-17 03:59:41 +00:00
7396230223 chore(deps): bump pypdf from 6.10.1 to 6.10.2 in /api (#35339)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-17 03:41:02 +00:00
ed14f58697 chore(deps): bump mako from 1.3.10 to 1.3.11 in /api (#35340)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-17 03:40:58 +00:00
37714cb44e chore(deps): bump authlib from 1.6.9 to 1.6.11 in /api (#35341)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-17 03:40:49 +00:00
3193e8a712 chore: reorg imports (#35308)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-16 08:50:02 +00:00
e8af6a6b3b test: migrate conversation service mock tests to testcontainers (#35198)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-16 07:55:21 +00:00
7f4fe4d064 test: migrate clean notion task tests to SQLAlchemy 2.0 APIs (#35159) 2026-04-16 07:52:11 +00:00
54e51be665 fix: apply score threshold after reranking in hybrid search (#35263) 2026-04-16 06:21:11 +00:00
b665eaa015 refactor(api): migrate console conversation responses to BaseModel (#35294)
Co-authored-by: ai-hpc <ai-hpc@users.noreply.github.com>
2026-04-16 05:11:21 +00:00
25df902ec4 refactor(api): add BaseModel workflow field schemas (#35297)
Co-authored-by: ai-hpc <ai-hpc@users.noreply.github.com>
2026-04-16 04:39:59 +00:00
5956dd79df refactor(api): add BaseModel conversation variable schemas (#35296)
Co-authored-by: ai-hpc <ai-hpc@users.noreply.github.com>
2026-04-16 04:39:30 +00:00
9fa50774b4 test: migrate duplicate and vector index task integration tests to SQLAlchemy 2.0 APIs (#35292)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-16 04:36:33 +00:00
731414a44f chore(deps): bump pypdf from 6.10.0 to 6.10.1 in /api (#35273)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-16 04:36:03 +00:00
987b5f4bf4 chore(deps): bump langsmith from 0.7.30 to 0.7.31 in /api (#35288)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-16 04:35:21 +00:00
8baa864c35 fix: add miss celery queue (#35282) 2026-04-16 02:40:14 +00:00
53a22aa41b feat: collaboration (#30781)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: yyh <yuanyouhuilyz@gmail.com>
Co-authored-by: yyh <92089059+lyzno1@users.noreply.github.com>
2026-04-16 02:21:04 +00:00
99
cf4d7afb9c chore(api): prune redundant direct dependency declarations (#35272) 2026-04-15 16:53:41 +00:00
e6b5923ff1 test(types): replace Account/Tenant status string literals with enum values (#35267)
Co-authored-by: xr843 <xianren843@protonmail.com>
2026-04-15 15:52:38 +00:00
dbceb3067e refactor(api): migrate console tag responses from marshal_with to BaseModel (#35208)
Co-authored-by: ai-hpc <ai-hpc@users.noreply.github.com>
2026-04-15 09:57:27 +00:00
425457cb16 test: remove legacy workflow draft variable api test (#35226) 2026-04-15 09:53:53 +00:00
e5bd18132c test: remove legacy jinja2 code executor integration test (#35222) 2026-04-15 09:53:46 +00:00
2f33867d07 test: remove legacy trigger provider permissions test (#35227) 2026-04-15 09:53:42 +00:00
fd71c56f16 test: remove legacy storage key loader integration test (#35225) 2026-04-15 09:52:31 +00:00
e3c2116501 fix: remove enable for get (#35245)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Joel <iamjoel007@gmail.com>
2026-04-15 09:18:29 +00:00
9fd196642d feat: tidb endpoint (#35158)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-15 08:46:25 +00:00
98897a5379 test: migrate webhook service additional mock tests to testcontainers (#35199)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-15 08:14:36 +00:00
79332c0e5e fix: Change 'commit' to 'flush' to prevent subsequent transaction fai… (#35243) 2026-04-15 07:50:52 +00:00
3bccdd6c9a test: migrate Service API site controller tests to Testcontainers (#32454) (#35183)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-15 05:51:34 +00:00
76af80e332 fix: open restore version panel raise 500 (#35240) 2026-04-15 04:43:38 +00:00