Commit Graph

328 Commits

Author SHA1 Message Date
0b3b0b5ce8 feat(api): retire legacy /v1/* and /console/api device-flow mounts (Phase F)
Web and CLI consumers now hit /openapi/v1/* directly, so the dual-mount
shims can go:
  - controllers/oauth_device_sso.py (legacy /v1/oauth/device/sso-* + /v1/device/sso-complete)
  - controllers/service_api/oauth.py (legacy /v1/oauth/device/*, /v1/me, /v1/oauth/authorizations/self)
  - controllers/console/auth/oauth_device.py (placeholder for legacy /console/api/oauth/device/{approve,deny})
  - the deferred _register_legacy_console_mount() inside openapi/oauth_device.py

Imports in controllers/console/__init__.py, controllers/service_api/__init__.py,
and extensions/ext_blueprints.py pruned. Tests rewritten to openapi-only.
2026-04-27 00:45:10 -07:00
2a38df2b7f refactor(api): consolidate openapi/oauth_device into per-domain modules
Match the existing api-group convention: one module per resource family
with multiple Resource classes per file (cf service_api/dataset/dataset.py
with 7 routes, console/auth/oauth_device.py with 2 before this branch).

The Phase B-D fragmentation (one file per route under
controllers/openapi/oauth_device/) was inconsistent with the codebase.
Collapse into:

  controllers/openapi/oauth_device.py        (5 routes: code, token,
                                              lookup, approve, deny —
                                              account branch)
  controllers/openapi/oauth_device_sso.py    (4 routes: sso-initiate,
                                              sso-complete,
                                              approval-context,
                                              approve-external —
                                              EE-only SSO branch)

The split mirrors the original pre-migration layout: account branch in
console/auth/oauth_device.py, SSO branch in controllers/oauth_device_sso.py
(root). Both legacy mount files updated to import from the new modules.

No behavior change; 59 tests still green. Test files updated to import
from the consolidated module paths.

Plan: docs/superpowers/plans/2026-04-26-openapi-migration.md (in difyctl repo).
2026-04-27 00:07:15 -07:00
b7bd9c19ed feat(api): lift identity + self-revoke to /openapi/v1/account (Phase C.9-10)
GET /v1/me moves to GET /openapi/v1/account. DELETE
/v1/oauth/authorizations/self moves to DELETE
/openapi/v1/account/sessions/self. Both classes (AccountApi,
AccountSessionsSelfApi) are now in controllers/openapi/account.py and
re-registered on service_api_ns at the legacy paths.

service_api/oauth.py is now nothing but legacy re-mount declarations
(20 lines). All in-place handler logic has moved to openapi/. Phase F
will delete the file and the legacy mounts together.

Helper functions (_load_memberships, _pick_default_workspace,
_workspace_payload, _account_payload) move with the AccountApi class.

Plan: docs/superpowers/plans/2026-04-26-openapi-migration.md (in difyctl repo).
2026-04-26 23:50:15 -07:00
e93821af46 feat(api): lift GET /oauth/device/lookup to /openapi/v1 (Phase B.8)
Same pattern as B.6 / B.7: OAuthDeviceLookupApi moves to
controllers/openapi/oauth_device/lookup.py and is re-registered on
service_api_ns to keep /v1/oauth/device/lookup serving until Phase F.

service_api/oauth.py is now down to /me + /oauth/authorizations/self
plus three legacy mounts; remaining handlers move in Phase C.
Now-unused imports (LIMIT_LOOKUP_PUBLIC, rate_limit, reqparse, request,
DEVICE_FLOW_TTL_SECONDS, DeviceFlowRedis, DeviceFlowStatus) removed.

Plan: docs/superpowers/plans/2026-04-26-openapi-migration.md (in difyctl repo).
2026-04-26 23:44:05 -07:00
9408759954 feat(api): lift POST /oauth/device/token to /openapi/v1 (Phase B.7)
Same pattern as B.6: OAuthDeviceTokenApi moves to
controllers/openapi/oauth_device/token.py and is re-registered on
service_api_ns to keep /v1/oauth/device/token serving until Phase F.

_audit_cross_ip_if_needed helper moves with the handler. Now-unused
imports removed from service_api/oauth.py.

Plan: docs/superpowers/plans/2026-04-26-openapi-migration.md (in difyctl repo).
2026-04-26 23:42:27 -07:00
fe9412af5d feat(api): lift POST /oauth/device/code to /openapi/v1 (Phase B.6)
Canonical class OAuthDeviceCodeApi now lives in
controllers/openapi/oauth_device/code.py and is registered on
openapi_ns at /openapi/v1/oauth/device/code. service_api/oauth.py
re-registers the same class object on service_api_ns at
/v1/oauth/device/code so existing callers keep working until Phase F.

KNOWN_CLIENT_IDS literal moves to dify_config.OPENAPI_KNOWN_CLIENT_IDS
(CSV-parsed, default "difyctl") so new CLIs / SDKs can be admitted
without code changes (CLAUDE.md rule 8 — no magic strings).

_verification_uri helper moves with the handler. Single source of
truth — no duplicated logic between the two mounts.

Plan: docs/superpowers/plans/2026-04-26-openapi-migration.md (in difyctl repo).
2026-04-26 23:40:58 -07:00
fe8510ad1a feat(api,web): OAuth 2.0 device flow + bearer auth (RFC 8628)
Adds a CLI-friendly authorization flow so difyctl (and future
non-browser clients) can obtain user-scoped tokens without copy-
pasting cookies or raw API keys. Two grant paths share one device
flow surface:

  1. Account branch — user signs in via the existing /signin
     methods, /device page calls console-authed approve, mints a
     dfoa_ token tied to (account_id, tenant).
  2. External-SSO branch (EE) — /v1/oauth/device/sso-initiate signs
     an SSOState envelope, hands off to Enterprise's external ACS,
     receives a signed external-subject assertion, mints a dfoe_
     token tied to (subject_email, subject_issuer).

API surface (all under /v1, EE-only endpoints 404 on CE):

  POST   /v1/oauth/device/code              — RFC 8628 start
  POST   /v1/oauth/device/token             — RFC 8628 poll
  GET    /v1/oauth/device/lookup            — pre-validate user_code
  GET    /v1/oauth/device/sso-initiate      — SSO branch entry
  GET    /v1/device/sso-complete            — SSO callback sink
  GET    /v1/oauth/device/approval-context  — /device cookie probe
  POST   /v1/oauth/device/approve-external  — SSO approve
  GET    /v1/me                             — bearer subject lookup
  DELETE /v1/oauth/authorizations/self      — self-revoke
  POST   /console/api/oauth/device/approve  — account approve
  POST   /console/api/oauth/device/deny     — account deny

Core primitives:
- libs/oauth_bearer.py: prefix-keyed TokenKindRegistry +
  BearerAuthenticator + validate_bearer decorator. Two-tier scope
  (full vs apps:run) stamped from the registry, never from the DB.
- libs/jws.py: HS256 compact JWS keyed on the shared Dify
  SECRET_KEY — same key-set verifies the SSOState envelope, the
  external-subject assertion (minted by Enterprise), and the
  approval-grant cookie.
- libs/device_flow_security.py: enterprise_only gate, approval-
  grant cookie mint/verify/consume (Path=/v1/oauth/device,
  HttpOnly, SameSite=Lax, Secure follows is_secure()), anti-
  framing headers.
- libs/rate_limit.py: typed RateLimit / RateLimitScope dispatch
  with composite-key buckets; both decorator + imperative form.
- services/oauth_device_flow.py: Redis state machine (PENDING ->
  APPROVED|DENIED with atomic consume-on-poll), token mint via
  partial unique index uq_oauth_active_per_device (rotates in
  place), env-driven TTL policy.

Storage: oauth_access_tokens table with partial unique index on
(subject_email, subject_issuer, client_id, device_label) WHERE
revoked_at IS NULL. account_id NULL distinguishes external-SSO
rows. Hard-expire is CAS UPDATE (revoked_at + nullify token_hash)
so audit events keep their token_id. Retention pruner DELETEs
revoked + zombie-expired rows past OAUTH_ACCESS_TOKEN_RETENTION_DAYS.

Frontend: /device page with code-entry, chooser (account vs SSO),
authorize-account, authorize-sso views. SSO branch detaches from
the URL user_code and reads everything from the cookie via
/approval-context. Anti-framing headers on all responses.

Wiring: ENABLE_OAUTH_BEARER feature flag; ext_oauth_bearer binds
the authenticator at startup; clean_oauth_access_tokens_task
scheduled in ext_celery.

Spec: docs/specs/v1.0/server/{device-flow,tokens,middleware,security}.md
2026-04-26 20:06:43 -07:00
99
3e876e173a chore(api): adapt Graphon 0.2.2 upgrade (#35377)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-18 11:16:24 +00:00
3193e8a712 chore: reorg imports (#35308)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-16 08:50:02 +00:00
b65a5fcd97 refactor(api): migrate service api workflow responses from marshal_with to BaseModel (#35195)
Co-authored-by: ai-hpc <ai-hpc@users.noreply.github.com>
2026-04-14 19:50:59 +00:00
800954f8ce refactor(api): migrate service conversation-variable responses to BaseModel (#35205)
Co-authored-by: ai-hpc <ai-hpc@users.noreply.github.com>
2026-04-14 19:49:21 +00:00
b0bf7ca486 refactor: replace bare dict with typed annotations in controllers (#35095) 2026-04-13 19:19:52 +00:00
5a446f8200 refactor(api): deduplicate dataset controller schemas into controller_schemas.py (#34756) 2026-04-13 05:33:20 +00:00
9121f24181 refactor(api): deduplicate TextToAudioPayload and MessageListQuery into controller_schemas.py (#34757) 2026-04-13 05:27:35 +00:00
8436470fcb refactor: replace bare dict with TypedDicts in annotation_service (#34998) 2026-04-13 03:46:33 +00:00
33bc58c9c2 refactor(api): migrate controllers to SQLAlchemy 2.0 select() API (#34960)
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-11 15:30:19 +00:00
0ba66ab155 refactor(api): deduplicate shared controller request schemas into controller_schemas.py (#34700)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-08 03:10:04 +00:00
624db69f12 refactor(api): remove duplicated RAG entities from services layer (#34689) 2026-04-07 23:36:59 +00:00
f67a811f7f refactor: replace dict params with BaseModel payloads in TagService (#34422)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-07 04:20:02 +00:00
9081c46565 refactor(api): type upload file serialization with UploadFileDict TypedDict (#34589) 2026-04-06 11:34:52 +00:00
99
318a3d0308 refactor(api): tighten login and wrapper typing (#34447) 2026-04-02 09:36:58 +00:00
99
8f9dbf269e chore(api): align Python support with 3.12 (#34419)
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
2026-04-02 05:07:32 +00:00
391007d02e refactor: migrate service_api and inner_api to sessionmaker pattern (#34379)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-01 14:53:41 +00:00
1aaba80211 fix: enrich Service API segment responses with summary content (#34221)
Co-authored-by: jigangz <jigangz@github.com>
Co-authored-by: FFXN <31929997+FFXN@users.noreply.github.com>
2026-03-30 10:09:50 +00:00
99
40591a7c50 refactor(api): use standalone graphon package (#34209)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-27 21:05:32 +00:00
a69b8c1e96 refactor: select in service API dataset document and segment controllers (#34101)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-26 13:24:54 +09:00
22dd0aa20c refactor: select in service API wraps, file_preview, and site controllers (#34086)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-25 23:01:05 +09:00
99
52e7492cbc refactor(api): rename dify_graph to graphon (#34095) 2026-03-25 21:58:56 +08:00
56593f20b0 refactor(api): continue decoupling dify_graph from API concerns (#33580)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: WH-2099 <wh2099@pm.me>
2026-03-25 20:32:24 +08:00
d87263f7c3 refactor: select in console datasets document controller (#34029)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-25 12:47:25 +09:00
9065d54f4a chore: bump pyrefly from 0.55.0 to 0.57.0 (#33755)
Co-authored-by: yuchengpersonal <yuchengpersonal@users.noreply.github.com>
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-24 23:22:18 +09:00
5b9cb55c45 refactor: use EnumText for MessageFeedback and MessageFile columns (#33738) 2026-03-20 01:13:26 +09:00
3870b2ad2d refactor: use EnumText for dataset and replace string literals 4 (#33606)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-18 09:18:08 +09:00
b09a75aae0 fix(api): resolve type errors in service API wraps tests (#33467) 2026-03-16 01:20:42 +09:00
0ab4e16335 feat: add batch download for dataset documents as ZIP and signed URL for single document download. (#33100) 2026-03-10 11:28:37 +08:00
741d48560d refactor(api): add TypedDict definitions to models/model.py (#32925)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-06 08:42:54 +09:00
92bde3503b fix: fix check workflow_run (#33028)
Co-authored-by: 非法操作 <hjlarry@163.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-05 17:13:35 +08:00
7d25415e4d feat: 204 http status code not return content (#33023) 2026-03-05 17:04:19 +08:00
2068640a4b fix: Add the missing validation of doc_form in the service API. (#32892) 2026-03-03 15:54:43 +08:00
4fd6b52808 refactor(api): move model_runtime into dify_graph (#32858) 2026-03-02 20:15:32 +08:00
c917838f9c refactor: move workflow package to dify_graph (#32844) 2026-03-02 18:42:30 +08:00
eea1cf17ef refactor(workflow): inject redis into graph engine manager (#32622) 2026-02-27 13:29:52 +08:00
d773096146 test: improve unit tests for controllers.service_api (#32073)
Co-authored-by: Rajat Agarwal <rajat.agarwal@infocusp.com>
2026-02-25 14:45:50 +08:00
4ac461d882 fix(api): serialize pipeline file-upload created_at (#32098)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-02-09 17:50:29 +08:00
fa763216d0 fix(api): register knowledge pipeline service API routes (#32097)
Co-authored-by: Crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: FFXN <31929997+FFXN@users.noreply.github.com>
2026-02-09 17:43:36 +08:00
a1fc280102 feat: Human Input Node (#32060)
The frontend and backend implementation for the human input node.

Co-authored-by: twwu <twwu@dify.ai>
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: yyh <92089059+lyzno1@users.noreply.github.com>
Co-authored-by: zhsama <torvalds@linux.do>
2026-02-09 14:57:23 +08:00
5b06203ef5 feat: Service API - add end-user lookup endpoint (#32015)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-02-09 14:01:22 +08:00
2c9430313d fix: redis for api token (#31861)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: hj24 <mambahj24@gmail.com>
2026-02-06 16:25:27 +08:00
c2a3f459c7 fix(api): return proper HTTP 204 status code in DELETE endpoints (#32012)
Signed-off-by: NeatGuyCoding <15627489+NeatGuyCoding@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-02-06 15:32:52 +08:00
47f8de3f8e refactor: port api/controllers/console/app/annotation.py api/controllers/console/explore/trial.py api/controllers/console/workspace/account.py api/controllers/console/workspace/members.py api/controllers/service_api/app/annotation.py to basemodel (#31833)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-03 09:59:00 +08:00