Commit Graph

48 Commits

Author SHA1 Message Date
fe8510ad1a feat(api,web): OAuth 2.0 device flow + bearer auth (RFC 8628)
Adds a CLI-friendly authorization flow so difyctl (and future
non-browser clients) can obtain user-scoped tokens without copy-
pasting cookies or raw API keys. Two grant paths share one device
flow surface:

  1. Account branch — user signs in via the existing /signin
     methods, /device page calls console-authed approve, mints a
     dfoa_ token tied to (account_id, tenant).
  2. External-SSO branch (EE) — /v1/oauth/device/sso-initiate signs
     an SSOState envelope, hands off to Enterprise's external ACS,
     receives a signed external-subject assertion, mints a dfoe_
     token tied to (subject_email, subject_issuer).

API surface (all under /v1, EE-only endpoints 404 on CE):

  POST   /v1/oauth/device/code              — RFC 8628 start
  POST   /v1/oauth/device/token             — RFC 8628 poll
  GET    /v1/oauth/device/lookup            — pre-validate user_code
  GET    /v1/oauth/device/sso-initiate      — SSO branch entry
  GET    /v1/device/sso-complete            — SSO callback sink
  GET    /v1/oauth/device/approval-context  — /device cookie probe
  POST   /v1/oauth/device/approve-external  — SSO approve
  GET    /v1/me                             — bearer subject lookup
  DELETE /v1/oauth/authorizations/self      — self-revoke
  POST   /console/api/oauth/device/approve  — account approve
  POST   /console/api/oauth/device/deny     — account deny

Core primitives:
- libs/oauth_bearer.py: prefix-keyed TokenKindRegistry +
  BearerAuthenticator + validate_bearer decorator. Two-tier scope
  (full vs apps:run) stamped from the registry, never from the DB.
- libs/jws.py: HS256 compact JWS keyed on the shared Dify
  SECRET_KEY — same key-set verifies the SSOState envelope, the
  external-subject assertion (minted by Enterprise), and the
  approval-grant cookie.
- libs/device_flow_security.py: enterprise_only gate, approval-
  grant cookie mint/verify/consume (Path=/v1/oauth/device,
  HttpOnly, SameSite=Lax, Secure follows is_secure()), anti-
  framing headers.
- libs/rate_limit.py: typed RateLimit / RateLimitScope dispatch
  with composite-key buckets; both decorator + imperative form.
- services/oauth_device_flow.py: Redis state machine (PENDING ->
  APPROVED|DENIED with atomic consume-on-poll), token mint via
  partial unique index uq_oauth_active_per_device (rotates in
  place), env-driven TTL policy.

Storage: oauth_access_tokens table with partial unique index on
(subject_email, subject_issuer, client_id, device_label) WHERE
revoked_at IS NULL. account_id NULL distinguishes external-SSO
rows. Hard-expire is CAS UPDATE (revoked_at + nullify token_hash)
so audit events keep their token_id. Retention pruner DELETEs
revoked + zombie-expired rows past OAUTH_ACCESS_TOKEN_RETENTION_DAYS.

Frontend: /device page with code-entry, chooser (account vs SSO),
authorize-account, authorize-sso views. SSO branch detaches from
the URL user_code and reads everything from the cookie via
/approval-context. Anti-framing headers on all responses.

Wiring: ENABLE_OAUTH_BEARER feature flag; ext_oauth_bearer binds
the authenticator at startup; clean_oauth_access_tokens_task
scheduled in ext_celery.

Spec: docs/specs/v1.0/server/{device-flow,tokens,middleware,security}.md
2026-04-26 20:06:43 -07:00
736880e046 feat: support configurable redis key prefix (#35139) 2026-04-14 09:31:41 +00:00
a31c1d2c69 refactor(api): type Celery SSL options and Sentinel transport dicts with TypedDicts (#34871) 2026-04-10 00:33:23 +00:00
e83a4090ac fix: lighten the health checks for the Worker and Worker Beat services, and disable them by default (#34572) 2026-04-06 02:26:26 +00:00
5a8a68cab8 feat: enterprise otel exporter (#33138)
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
Co-authored-by: Yunlu Wen <yunlu.wen@dify.ai>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-27 07:56:31 +00:00
56593f20b0 refactor(api): continue decoupling dify_graph from API concerns (#33580)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: WH-2099 <wh2099@pm.me>
2026-03-25 20:32:24 +08:00
6015f23e79 feat: enhancement celery configuration (#32145) 2026-02-10 17:55:24 +08:00
a1fc280102 feat: Human Input Node (#32060)
The frontend and backend implementation for the human input node.

Co-authored-by: twwu <twwu@dify.ai>
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: yyh <92089059+lyzno1@users.noreply.github.com>
Co-authored-by: zhsama <torvalds@linux.do>
2026-02-09 14:57:23 +08:00
2c9430313d fix: redis for api token (#31861)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: hj24 <mambahj24@gmail.com>
2026-02-06 16:25:27 +08:00
90fe9abab7 revert: revert human input relevant code (#31766)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-01-30 19:18:49 +08:00
03e3acfc71 feat(api): Human Input Node (backend part) (#31646)
The backend part of the human in the loop (HITL) feature and relevant architecture / workflow engine changes.

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: 盐粒 Yanli <yanli@dify.ai>
Co-authored-by: CrabSAMA <40541269+CrabSAMA@users.noreply.github.com>
Co-authored-by: Stephen Zhou <38493346+hyoban@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: yihong <zouzou0208@gmail.com>
Co-authored-by: Joel <iamjoel007@gmail.com>
2026-01-30 10:18:49 +08:00
c2473d85dc feat: Add summary index for knowledge. (#31625)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jyong <76649700+JohnJyong@users.noreply.github.com>
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
Co-authored-by: Yansong Zhang <916125788@qq.com>
Co-authored-by: hj24 <mambahj24@gmail.com>
Co-authored-by: CodingOnStar <hanxujiang@dify.ai>
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-29 13:47:35 +08:00
51ea87ab85 feat: clear free plan workflow run logs (#29494)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2026-01-12 15:57:40 +08:00
5362f69083 feat(refactoring): Support Structured Logging (JSON) (#30170) 2026-01-04 11:46:46 +08:00
e6f3528bb0 fix: Incorrect REDIS ssl variable used for Celery causing Celery unable to start (#29605)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2025-12-31 10:26:28 +08:00
b76e17b25d feat: introduce trigger functionality (#27644)
Signed-off-by: lyzno1 <yuanyouhuilyz@gmail.com>
Co-authored-by: Stream <Stream_2@qq.com>
Co-authored-by: lyzno1 <92089059+lyzno1@users.noreply.github.com>
Co-authored-by: zhsama <torvalds@linux.do>
Co-authored-by: Harry <xh001x@hotmail.com>
Co-authored-by: lyzno1 <yuanyouhuilyz@gmail.com>
Co-authored-by: yessenia <yessenia.contact@gmail.com>
Co-authored-by: hjlarry <hjlarry@163.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: WTW0313 <twwu@dify.ai>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-12 17:59:37 +08:00
cba2b9b2ad fix: switch plugin auto upgrade cache to redis (#26356) 2025-09-29 12:57:30 +08:00
bdd85b36a4 ruff check preview (#25653)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-16 12:58:12 +08:00
bab4975809 chore: add ast-grep rule to convert Optional[T] to T | None (#25560)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-15 13:06:33 +08:00
d8000251ff chore: apply static type checks on celery async task dispatches and imports (#24418) 2025-08-24 23:07:22 +08:00
75199442c1 feat: Implements periodic deletion of workflow run logs that exceed t… (#23881)
Co-authored-by: shiyun.li973792 <shiyun.li@seres.cn>
Co-authored-by: 1wangshu <suewangswu@gmail.com>
Co-authored-by: Blackoutta <hyytez@gmail.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2025-08-19 09:47:34 +08:00
531e784a92 feat: no longer enable auto upgrade when marketplace is disabled (#24… (#24101) 2025-08-18 15:57:33 +08:00
052d0e059e feat: add Redis SSL/TLS certificate authentication support (#23624) 2025-08-18 10:59:07 +08:00
9c5b1e7012 fix: Update the scheduling method for timed tasks, (#22779) 2025-07-25 16:27:35 +08:00
eaae79a581 feat: plugin auto upgrade strategy (#19758)
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Novice <novice12185727@gmail.com>
2025-07-23 15:33:39 +08:00
28f5c37211 Add Env 'CELERY_SENTINEL_PASSWORD' for celery connect redis sentinel. (#21198) 2025-06-27 17:37:11 +08:00
92614765ff Feat/queue monitor (#20647) 2025-06-04 19:56:34 +08:00
cf00ee42f5 send knowledge base auto disable notification (#12126) 2024-12-26 18:14:08 +08:00
56e15d09a9 feat: mypy for all type check (#10921) 2024-12-24 18:38:51 +08:00
9b46b02717 refactor: assembling the app features in modular way (#9129)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
2024-11-30 23:05:22 +08:00
41c6bf5fe4 update the scheduler of update_tidb_serverless_status_task to 1/10min (#11135) 2024-11-27 17:41:00 +08:00
128efc3193 Feat/clean message records (#10588) 2024-11-18 16:57:39 +08:00
16db2c4e57 Fix: Set Celery LOG_File only when available, always log to console (#10563) 2024-11-11 23:53:12 +08:00
7c2a9b0744 celery worker log format following LOG_FORMAT env#9404 (#10016)
Signed-off-by: root <root@localhost.localdomain>
Co-authored-by: root <root@localhost.localdomain>
2024-11-08 17:12:09 +08:00
18106a4fc6 add tidb on qdrant type (#9831)
Co-authored-by: Zhaofeng Miao <522856232@qq.com>
2024-10-25 13:57:03 +08:00
4d9160ca9f refactor: use dify_config to replace legacy usage of flask app's config (#9089) 2024-10-22 11:01:32 +08:00
Zhi
d542b15cc0 feat: support redis sentinel mode (#7756) 2024-09-08 13:23:51 +08:00
3571292fbf chore(api): Introduce Ruff Formatter. (#7291) 2024-08-15 12:54:05 +08:00
7c397f5722 update celery beat scheduler time to env (#6352) 2024-07-17 02:31:30 +08:00
988aa4b5da update clean_unused_datasets_task timedelta (#6324) 2024-07-16 13:43:04 +08:00
28089c98c1 fix: skip Celery warning by setting broker_connection_retry_on_startup config (#3188) 2024-04-09 16:14:43 +08:00
5e66a60f1c add embedding cache and clean embedding cache job (#3087)
Co-authored-by: jyong <jyong@dify.ai>
2024-04-02 20:46:24 +08:00
1716ac562c add clean_unused_datasets_task (#3057)
Co-authored-by: jyong <jyong@dify.ai>
2024-04-01 01:34:21 +08:00
cc9e74123c improve: introduce isort for linting Python imports (#1983) 2024-01-12 12:34:01 +08:00
2757494265 alter schedule timedelta (#1923)
Co-authored-by: jyong <jyong@dify.ai>
2024-01-04 18:10:16 +08:00
595e9b25ba Add data clean schedule (#1859)
Co-authored-by: jyong <jyong@dify.ai>
2024-01-02 15:29:18 +08:00
f8eefa31fe feat: add redis ssl support (#65) 2023-05-17 15:40:21 +08:00
db896255d6 Initial commit 2023-05-15 08:51:32 +08:00