Commit Graph

1035 Commits

Author SHA1 Message Date
04ebf8a92f feat(openapi): /apps/permitted — external-subject app discovery (EE)
Split route for dfoe_ external-SSO discovery, separate from /apps
(dfoa_-only workspace catalog). Cross-tenant allow-list query: server
calls Enterprise inner-API POST /inner/api/webapp/permitted-apps and
hydrates app/tenant rows locally. New scope apps:read:permitted (no
dual-meaning with apps:read). Route gated by @enterprise_only — 404
on CE — and validate_bearer(accept=ACCEPT_USER_EXT_SSO) — 403 on dfoa_.
Query validator rejects workspace_id and tag (cross-tenant
unresolvable); mode/name supported.

EE inner-API wire-up depends on ee-2; the service-layer stub raises
ServiceUnavailable until that endpoint ships. CLI dispatches between
/apps and /apps/permitted client-side based on the bearer prefix in
hosts.yml — see docs/specs/v1.0/apps.md §Subject dispatch.

Verified via unit tests on AppPermittedListQuery and Scope wiring;
HTTP integration tests deferred to ee-2 once the inner-API ships.
2026-05-05 20:20:22 -07:00
8a62c1d915 chore(api): pyright + ruff cleanup for openapi/cli surface
Type and lint pass over the openapi controllers, auth pipeline, and
oauth bearer/device-flow plumbing. Down from 36 pyright errors and 16
ruff errors to 0/0; 93 openapi unit tests pass.

Logic fixes:
- libs/oauth_bearer.py: drop private-naming on the friend-API methods
  consumed by _VariantResolver (cache_get / cache_set_positive /
  cache_set_negative / hard_expire / session_factory). They were always
  cross-class accessors — leading underscore was misleading. Add public
  registry property on BearerAuthenticator. _hard_expire row_id widened
  to UUID | str (matches the StringUUID column type).
- libs/oauth_bearer.py: type validate_bearer / bearer_feature_required
  with ParamSpec / PEP-695 so wrapped routes preserve their signature.
- libs/rate_limit.py: same — typed rate_limit decorator.
- services/oauth_device_flow.py: mint_oauth_token / _upsert accept
  Session | scoped_session (Flask-SQLAlchemy proxy). Guard row-is-None
  after upsert.
- controllers/openapi/{chat,completion,workflow}_messages.py: tuple-vs-
  Mapping shape narrowing on AppGenerateService.generate return —
  production returns Mapping, tests mock as (body, status). Validate
  through Pydantic Response model in both shapes.
- controllers/openapi/oauth_device.py: replace flask_restx.reqparse (banned)
  with Pydantic Request/Query models — DeviceCodeRequest, DevicePollRequest,
  DeviceLookupQuery, DeviceMutateRequest. Two PEP-695 generic helpers
  (_validate_json / _validate_query) translate ValidationError to BadRequest.
- controllers/openapi/auth/strategies.py: Protocol param-name match
  (subject_type), Optional narrowing on app/tenant/account_id/subject_email.
- controllers/openapi/auth/steps.py: subject_type-is-None guard before
  mounter dispatch.
- core/app/apps/workflow/generate_task_pipeline.py + models/workflow.py:
  add WorkflowAppLogCreatedFrom.OPENAPI + matching match-case branch.
  Fixes match-exhaustiveness and possibly-unbound created_from.
- libs/device_flow_security.py: pyright ignore on flask after_request
  hook (registered by the framework, pyright sees as unused).
- services/oauth_device_flow.py: rename Exceptions to *Error suffix
  (StateNotFoundError / InvalidTransitionError / UserCodeExhaustedError);
  same for libs/oauth_bearer.py (InvalidBearerError / TokenExpiredError).
  Update all callers across openapi controllers.
- controllers/openapi/{oauth_device,oauth_device_sso}.py +
  services/oauth_device_flow.py: switch logger.error in except blocks
  to logger.exception (TRY400) — keeps the traceback for ops.
- configs/feature/__init__.py: OPENAPI_KNOWN_CLIENT_IDS computed_field
  needs an @property alongside for pyright to see it as a value, not a
  method. Matches the existing line-451 pattern.

Plus ruff format + import-sort across the openapi tree (pure formatting).
2026-04-28 21:44:54 -07:00
813da349ec fix(api,web): post-review hardening for OAuth device flow
- api: account-flow stores subject_issuer="dify:account" sentinel
  instead of NULL so the rotate-in-place unique index collides as
  intended (Postgres treats NULLs as distinct in unique indices).
  mint_oauth_token validates prefix-specific issuer rules.
- api: enterprise_only inverts to an allowlist (ACTIVE / EXPIRING) so
  any future LicenseStatus value defaults to denial.
- api: consume_on_poll moved to a single Lua script (GET + status-check
  + DEL) so concurrent pollers can't both observe APPROVED.
- web: typed DeviceFlowError + central error-copy mapping; page
  surfaces rate_limited / lookup_failed view states; URL params
  scrubbed after consumption (RFC 8628 §5.4).
2026-04-26 23:05:07 -07:00
fe8510ad1a feat(api,web): OAuth 2.0 device flow + bearer auth (RFC 8628)
Adds a CLI-friendly authorization flow so difyctl (and future
non-browser clients) can obtain user-scoped tokens without copy-
pasting cookies or raw API keys. Two grant paths share one device
flow surface:

  1. Account branch — user signs in via the existing /signin
     methods, /device page calls console-authed approve, mints a
     dfoa_ token tied to (account_id, tenant).
  2. External-SSO branch (EE) — /v1/oauth/device/sso-initiate signs
     an SSOState envelope, hands off to Enterprise's external ACS,
     receives a signed external-subject assertion, mints a dfoe_
     token tied to (subject_email, subject_issuer).

API surface (all under /v1, EE-only endpoints 404 on CE):

  POST   /v1/oauth/device/code              — RFC 8628 start
  POST   /v1/oauth/device/token             — RFC 8628 poll
  GET    /v1/oauth/device/lookup            — pre-validate user_code
  GET    /v1/oauth/device/sso-initiate      — SSO branch entry
  GET    /v1/device/sso-complete            — SSO callback sink
  GET    /v1/oauth/device/approval-context  — /device cookie probe
  POST   /v1/oauth/device/approve-external  — SSO approve
  GET    /v1/me                             — bearer subject lookup
  DELETE /v1/oauth/authorizations/self      — self-revoke
  POST   /console/api/oauth/device/approve  — account approve
  POST   /console/api/oauth/device/deny     — account deny

Core primitives:
- libs/oauth_bearer.py: prefix-keyed TokenKindRegistry +
  BearerAuthenticator + validate_bearer decorator. Two-tier scope
  (full vs apps:run) stamped from the registry, never from the DB.
- libs/jws.py: HS256 compact JWS keyed on the shared Dify
  SECRET_KEY — same key-set verifies the SSOState envelope, the
  external-subject assertion (minted by Enterprise), and the
  approval-grant cookie.
- libs/device_flow_security.py: enterprise_only gate, approval-
  grant cookie mint/verify/consume (Path=/v1/oauth/device,
  HttpOnly, SameSite=Lax, Secure follows is_secure()), anti-
  framing headers.
- libs/rate_limit.py: typed RateLimit / RateLimitScope dispatch
  with composite-key buckets; both decorator + imperative form.
- services/oauth_device_flow.py: Redis state machine (PENDING ->
  APPROVED|DENIED with atomic consume-on-poll), token mint via
  partial unique index uq_oauth_active_per_device (rotates in
  place), env-driven TTL policy.

Storage: oauth_access_tokens table with partial unique index on
(subject_email, subject_issuer, client_id, device_label) WHERE
revoked_at IS NULL. account_id NULL distinguishes external-SSO
rows. Hard-expire is CAS UPDATE (revoked_at + nullify token_hash)
so audit events keep their token_id. Retention pruner DELETEs
revoked + zombie-expired rows past OAUTH_ACCESS_TOKEN_RETENTION_DAYS.

Frontend: /device page with code-entry, chooser (account vs SSO),
authorize-account, authorize-sso views. SSO branch detaches from
the URL user_code and reads everything from the cookie via
/approval-context. Anti-framing headers on all responses.

Wiring: ENABLE_OAUTH_BEARER feature flag; ext_oauth_bearer binds
the authenticator at startup; clean_oauth_access_tokens_task
scheduled in ext_celery.

Spec: docs/specs/v1.0/server/{device-flow,tokens,middleware,security}.md
2026-04-26 20:06:43 -07:00
ceec00e172 fix: accept icon type in app icon updates (#35360)
Co-authored-by: Stephen Zhou <hi@hyoban.cc>
2026-04-20 02:33:49 +00:00
99
3e876e173a chore(api): adapt Graphon 0.2.2 upgrade (#35377)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-18 11:16:24 +00:00
f980d5d3be feat: copy nodes cross apps (#33273)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-17 10:02:26 +00:00
0020aa8f59 refactor(api): type pipeline template retrieval dicts with TypedDict (#34874) 2026-04-17 08:13:54 +00:00
90d638fba3 refactor: migrate DocumentSegmentSummary to TypeBase (#34862) 2026-04-17 06:58:32 +00:00
af8ffa0654 refactor: use sessionmaker in api_tools_manage_service.py (#34892)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-17 05:07:12 +00:00
3193e8a712 chore: reorg imports (#35308)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-16 08:50:02 +00:00
53a22aa41b feat: collaboration (#30781)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: yyh <yuanyouhuilyz@gmail.com>
Co-authored-by: yyh <92089059+lyzno1@users.noreply.github.com>
2026-04-16 02:21:04 +00:00
79332c0e5e fix: Change 'commit' to 'flush' to prevent subsequent transaction fai… (#35243) 2026-04-15 07:50:52 +00:00
9c90c1c455 refactor: replace bare dict with dict[str, Any] in services and hosti… (#35211)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-14 19:44:40 +00:00
1bcc7f78c7 refactor: replace bare dict with dict[str, Any] in models, providers, extensions, libs, and test utilities (#35186) 2026-04-14 17:51:25 +00:00
bd7a9b5fcf refactor: replace bare dict with dict[str, Any] in model provider service and core modules (#35122)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
2026-04-14 09:18:30 +00:00
f7c6270f74 refactor: use sessionmaker in tool_label_manager.py (#34895)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-14 07:23:29 +00:00
eeebedcfe8 refactor: replace bare dict with dict[str, Any] in core provider services and misc modules (#35124) 2026-04-14 03:03:08 +00:00
2f682780fa refactor: replace bare dict with dict[str, Any] in rag_pipeline and datasource_provider services (#35107) 2026-04-14 03:02:41 +00:00
ed83f5369e refactor: replace bare dict with dict[str, Any] in entities, workflow nodes, and tasks (#35109) 2026-04-14 03:02:39 +00:00
1c2bbed405 refactor: replace bare dict with dict[str, Any] in services grab-bag (#35112) 2026-04-14 03:02:34 +00:00
d8fbc00cb9 refactor: replace bare dict with dict[str, Any] in dataset and external_knowledge services (#35073) 2026-04-13 17:08:45 +00:00
57c5f0ec87 refactor: replace bare dict with dict[str, Any] in tools manage services (#35075) 2026-04-13 17:08:31 +00:00
e5bd80c719 refactor: replace bare dict with dict[str, Any] in website_service (#35074) 2026-04-13 17:07:59 +00:00
e1eb582bea refactor: replace bare dict with dict[str, Any] in pipeline_template module (#35071) 2026-04-13 14:05:59 +00:00
d412cddf39 refactor: replace bare dict with UtmInfo TypedDict in operation_service (#35055) 2026-04-13 13:05:47 +00:00
671c5cdd84 refactor: replace bare dict with WorkflowRunListArgs TypedDict (#35057) 2026-04-13 13:05:39 +00:00
554f060092 refactor: replace bare dict with AdvancedPromptTemplateArgs TypedDict (#35056) 2026-04-13 13:05:23 +00:00
dd50a68bf2 refactor: replace bare dict with dict[str, Any] in ops_service tracin… (#35064) 2026-04-13 13:01:00 +00:00
e8dd3461e8 refactor: replace bare dict with dict[str, Any] in plugin endpoint_service (#35065) 2026-04-13 13:00:27 +00:00
8dd4473432 refactor(auth): standardize failed login audit logging (#35054) 2026-04-13 12:26:13 +00:00
f0266e13c5 refactor: improve type annotations in HitTestingService (#27838)
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
2026-04-13 10:31:31 +00:00
815c536e05 fix: optimize trigger long running read transactions (#35046) 2026-04-13 08:22:54 +00:00
2108c44c8b refactor(api): consolidate duplicate RerankingModelConfig and WeightedScoreConfig definitions (#34747)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-13 05:53:45 +00:00
f4d5e2f43d refactor(api): improve type safety in MCPToolManageService.execute_auth_actions (#34824) 2026-04-13 05:29:10 +00:00
a111d56ea3 refactor: use sessionmaker in workflow_tools_manage_service.py (#34896)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-13 03:47:29 +00:00
8436470fcb refactor: replace bare dict with TypedDicts in annotation_service (#34998) 2026-04-13 03:46:33 +00:00
8c4ea5c898 fix: external dataset tenant checks for bound knowledge APIs (#34734) 2026-04-13 01:47:57 +00:00
0862fd74b0 refactor: migrate _model_to_insertion_dict return type to TypedDict (#34988) 2026-04-12 09:50:55 +00:00
0841b4c663 refactor(api): migrate tools, account, workflow and plugin services to SQLAlchemy 2.0 (#34966)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:45:27 +00:00
440602f52a refactor(services): migrate summary_index_service to SQLAlchemy 2.0 select() API (#34971)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:37:16 +00:00
510120410b refactor(services): migrate trigger_provider_service to SQLAlchemy 2.0 select() API (#34972)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:36:13 +00:00
4ef67fef3a refactor(services): migrate builtin_tools_manage_service to SQLAlchemy 2.0 select() API (#34973)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:25:51 +00:00
7bd5e80323 refactor(services): migrate datasource_provider_service to SQLAlchemy 2.0 select() API (#34974) 2026-04-12 01:23:24 +00:00
7515eee0a8 refactor(services): migrate dataset_service and clear_free_plan_tenant_expired_logs to SQLAlchemy 2.0 select() API (#34970)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:21:52 +00:00
859920a81f refactor: migrate verify_subscription_credentials return type to TypedDict (#34967) 2026-04-11 16:41:40 +00:00
65d66768c1 fix: fix tool output duplicate (#34962) 2026-04-11 15:07:31 +00:00
2dc015b360 fix(api): default parent_mode to paragraph for hierarchical chunking via API (#34635)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-10 10:55:40 +00:00
98eedf14dc refactor(services): replace Union with | syntax in service layer (#34905) 2026-04-10 07:44:47 +00:00
cd3ee5bd5d fix: sqlalchemy.orm.exc.DetachedInstanceError (#34910) 2026-04-10 07:44:22 +00:00