mirror of
https://github.com/langgenius/dify.git
synced 2026-06-09 18:07:36 +08:00
Compare commits
33 Commits
dependabot
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 34f3591d4c | |||
| c88a38b8b5 | |||
| 0019e6a6f3 | |||
| 1502a57381 | |||
| 686e643632 | |||
| 8e37d95760 | |||
| 11db079428 | |||
| eb3b12fa70 | |||
| 5bec8eb33a | |||
| d11e4eeaf7 | |||
| bbdf3d7634 | |||
| a80bba2c35 | |||
| 789698cddd | |||
| a8977be999 | |||
| 22e67b4673 | |||
| f948e442e0 | |||
| 8a1c0cf5ab | |||
| 47b58a34ef | |||
| d80bd2a135 | |||
| 5d814ca8c1 | |||
| 0239b81cca | |||
| a15ecf6bec | |||
| d0b376d31a | |||
| 9c24b7bac5 | |||
| 6291452020 | |||
| d46a4c05b1 | |||
| f15a8f02ef | |||
| 0c4b36b3f5 | |||
| 37e1d452b8 | |||
| db1aa683bc | |||
| a88c15c906 | |||
| 12bd8d2aa8 | |||
| 813bfea730 |
@ -25,6 +25,7 @@ Before reviewing, read the relevant local contracts:
|
||||
- `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md` when code uses or changes `@langgenius/dify-ui/*`.
|
||||
- `web/docs/overlay.md` when reviewing dialogs, drawers, popovers, tooltips, menus, selects, comboboxes, or other floating UI.
|
||||
- `web/docs/test.md` and the `frontend-testing` skill when reviewing tests or testability.
|
||||
- `karpathy-guidelines` for scope control and focused, verifiable changes.
|
||||
- `how-to-write-component` when reviewing React component structure, ownership, effects, query/mutation contracts, or memoization.
|
||||
|
||||
For any UI, UX, or accessibility review, fetch the latest Web Interface Guidelines before finalizing findings. Treat them as a required baseline, not the complete source of accessibility truth:
|
||||
|
||||
@ -29,10 +29,11 @@ Prefer:
|
||||
Flag:
|
||||
|
||||
- New CSS modules or ad hoc CSS when Tailwind utilities and Dify tokens cover the need.
|
||||
- Component-level plain `.css` files or component CSS imported through `globals.css`; use scoped `*.module.css` only when Tailwind and component variants cannot express the style.
|
||||
- Generic color utilities where Dify semantic tokens exist.
|
||||
- Hardcoded magic class values for colors, spacing, radius, shadow, z-index, or typography when Dify tokens, component variants, or documented radius mappings exist.
|
||||
- `!` important modifiers or important CSS overrides without a narrow, documented reason.
|
||||
- Manual string concatenation for conditional classes.
|
||||
- Manual string concatenation, template strings, array `.join(' ')`, or custom ternaries for conditional or multi-line classes.
|
||||
- JS conditional class branches for primitive visual states already exposed by Dify UI/Base UI `data-*` selectors.
|
||||
- Incoming `className` placed before default classes in `cn(...)`, preventing call-site overrides.
|
||||
- Arbitrary z-index or one-off layering fixes on overlays.
|
||||
@ -59,8 +60,9 @@ Flag:
|
||||
Flag:
|
||||
|
||||
- User-facing hardcoded strings in `web/`.
|
||||
- Added or renamed i18n keys that are not present in every supported locale file for the touched namespace.
|
||||
- Translation namespace drift, especially using unrelated module namespaces for local feature copy.
|
||||
- Generic button labels like `Continue` where the action is specific.
|
||||
- Error messages that state only the failure and not the next step.
|
||||
|
||||
Use feature-local translation keys by default. Alias only when crossing namespaces.
|
||||
Use feature-local translation keys by default. Alias only when crossing namespaces. `pnpm i18n:check --file <name>` should pass for any touched translation namespace.
|
||||
|
||||
@ -18,6 +18,7 @@ Accept repeated TanStack Query calls in siblings when each component independent
|
||||
|
||||
Flag:
|
||||
|
||||
- React component files over 300 lines when the file mixes multiple responsibilities that can be split into focused colocated components, hooks, or utilities.
|
||||
- Shallow wrappers that only rename props or hide the real primitive.
|
||||
- Extra DOM wrappers that do not provide layout, semantics, accessibility, state ownership, or library integration.
|
||||
- Dialog/dropdown/popover hidden surfaces that obscure the parent flow when they should be extracted into a small local component.
|
||||
@ -29,6 +30,7 @@ Prefer colocated components split by actual data and state needs.
|
||||
|
||||
Flag:
|
||||
|
||||
- Refactors of existing navigation, sidebar, dropdown, webapp list, or app-switching UI that do not preserve behavior-sensitive interactions such as expand/collapse arrows, hover persistence, pin/delete controls, routing, keyboard/focus handling, or open-state ownership.
|
||||
- Components that mix data fetching, mutation side effects, popup state, form validation, layout, and row rendering without a clear owner.
|
||||
- Generic components with many boolean props that encode one feature's workflow.
|
||||
- A shared component that imports feature-specific copy, routes, or API contracts.
|
||||
@ -38,6 +40,8 @@ Flag:
|
||||
- A component that exposes controlled props but still keeps a competing private state for the same value.
|
||||
- A component that cannot render empty, loading, or missing optional API fields without caller-side preprocessing.
|
||||
|
||||
When existing components already own interaction logic, prefer reusing or extending them. If a refactor is necessary, preserve the old interaction contract and add or update focused tests for changed behavior.
|
||||
|
||||
## Props And Types
|
||||
|
||||
Flag:
|
||||
|
||||
@ -91,6 +91,7 @@ Use:
|
||||
|
||||
Flag:
|
||||
|
||||
- Manually recreating UI behavior or chrome already owned by `@langgenius/dify-ui/*` or `web/app/components/base/*`, such as buttons, inputs, toggle groups, popovers, dropdown menus, alert dialogs, switches, avatars, scroll areas, toasts, borders, focus states, disabled states, segmented controls, or existing feature components.
|
||||
- Styling a raw Base UI primitive directly in `web/` when a Dify UI primitive exists.
|
||||
- Wrapping a Dify UI primitive in a feature component that hides its label, error, disabled, or focus contract.
|
||||
- Replacing a semantic primitive with a generic `div` plus classes to match a screenshot.
|
||||
@ -121,3 +122,13 @@ Use `!` only for a tightly scoped compatibility override after confirming the pr
|
||||
## Focus Details
|
||||
|
||||
Flag focus rings attached to the wrong element. For example, Base UI `Slider.Thumb` focuses an internal `input[type=range]`, so the visible thumb wrapper needs `has-[:focus-visible]` rather than direct wrapper `focus-visible`.
|
||||
|
||||
## Custom SVG Icons
|
||||
|
||||
Flag:
|
||||
|
||||
- New generated React icon components or JSON files under `web/app/components/base/icons/src/...` for custom SVG icons.
|
||||
- Custom SVG icons consumed outside the Tailwind `i-custom-*` icon class pipeline.
|
||||
- Generated `packages/iconify-collections/custom-*/icons.json` diffs where unrelated existing icons lost or changed intrinsic `width` or `height`.
|
||||
|
||||
New custom SVG icons belong in `packages/iconify-collections/assets/...`. Regenerate with `pnpm --filter @dify/iconify-collections generate`, validate with `pnpm --filter @dify/iconify-collections check:dimensions`, and consume the generated icon with Tailwind `i-custom-*` classes.
|
||||
|
||||
33
.agents/skills/karpathy-guidelines/SKILL.md
Normal file
33
.agents/skills/karpathy-guidelines/SKILL.md
Normal file
@ -0,0 +1,33 @@
|
||||
---
|
||||
name: karpathy-guidelines
|
||||
description: Lightweight coding guardrails for making focused, simple, and verifiable changes in this repo. Use for all coding work.
|
||||
---
|
||||
|
||||
# Karpathy Guidelines
|
||||
|
||||
Use this skill whenever you touch code in this repository.
|
||||
|
||||
## Principles
|
||||
|
||||
- Keep the change small and directly tied to the user request.
|
||||
- Prefer the simplest implementation that fits the existing codebase.
|
||||
- Read the nearby code first, then match its patterns.
|
||||
- Avoid unrelated refactors, broad rewrites, or style churn.
|
||||
- Preserve existing behavior unless the user explicitly asked to change it.
|
||||
- Treat regressions as a signal to narrow the change, not to add workaround layers.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Inspect the current implementation and tests around the change.
|
||||
2. Make the smallest coherent edit.
|
||||
3. Add or update focused tests when the behavior changes or the risk is non-trivial.
|
||||
4. Run the narrowest relevant verification first.
|
||||
5. Report exactly what was verified and anything left unverified.
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- Does this change solve the stated problem without expanding scope?
|
||||
- Did it preserve existing route/component/data-flow semantics?
|
||||
- Are new abstractions justified by real complexity?
|
||||
- Are tests focused on the behavior that could regress?
|
||||
- Are unrelated files and generated artifacts left alone?
|
||||
1
.claude/skills/karpathy-guidelines
Symbolic link
1
.claude/skills/karpathy-guidelines
Symbolic link
@ -0,0 +1 @@
|
||||
../../.agents/skills/karpathy-guidelines
|
||||
14
.github/workflows/api-tests.yml
vendored
14
.github/workflows/api-tests.yml
vendored
@ -29,13 +29,13 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: ${{ matrix.python-version }}
|
||||
@ -91,13 +91,13 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: ${{ matrix.python-version }}
|
||||
@ -142,13 +142,13 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: "3.12"
|
||||
@ -195,7 +195,7 @@ jobs:
|
||||
|
||||
- name: Report coverage
|
||||
if: ${{ env.CODECOV_TOKEN != '' }}
|
||||
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
|
||||
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
|
||||
with:
|
||||
files: ./coverage.xml
|
||||
disable_search: true
|
||||
|
||||
4
.github/workflows/autofix.yml
vendored
4
.github/workflows/autofix.yml
vendored
@ -20,7 +20,7 @@ jobs:
|
||||
run: echo "autofix.ci updates pull request branches, not merge group refs."
|
||||
|
||||
- if: github.event_name != 'merge_group'
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Check Docker Compose inputs
|
||||
if: github.event_name != 'merge_group'
|
||||
@ -66,7 +66,7 @@ jobs:
|
||||
python-version: "3.11"
|
||||
|
||||
- if: github.event_name != 'merge_group'
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
|
||||
- name: Generate Docker Compose
|
||||
if: github.event_name != 'merge_group' && steps.docker-compose-changes.outputs.any_changed == 'true'
|
||||
|
||||
14
.github/workflows/build-push.yml
vendored
14
.github/workflows/build-push.yml
vendored
@ -68,7 +68,7 @@ jobs:
|
||||
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
|
||||
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
|
||||
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
|
||||
with:
|
||||
username: ${{ env.DOCKERHUB_USER }}
|
||||
password: ${{ env.DOCKERHUB_TOKEN }}
|
||||
@ -78,13 +78,13 @@ jobs:
|
||||
|
||||
- name: Extract metadata for Docker
|
||||
id: meta
|
||||
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
|
||||
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
|
||||
with:
|
||||
images: ${{ env[matrix.image_name_env] }}
|
||||
|
||||
- name: Build Docker image
|
||||
id: build
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.18.0
|
||||
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.17.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
context: ${{ matrix.build_context }}
|
||||
@ -124,10 +124,10 @@ jobs:
|
||||
file: "web/Dockerfile"
|
||||
steps:
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
- name: Validate Docker image
|
||||
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
|
||||
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
|
||||
with:
|
||||
push: false
|
||||
context: ${{ matrix.build_context }}
|
||||
@ -156,14 +156,14 @@ jobs:
|
||||
merge-multiple: true
|
||||
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
|
||||
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
|
||||
with:
|
||||
username: ${{ env.DOCKERHUB_USER }}
|
||||
password: ${{ env.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Extract metadata for Docker
|
||||
id: meta
|
||||
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
|
||||
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
|
||||
with:
|
||||
images: ${{ env[matrix.image_name_env] }}
|
||||
tags: |
|
||||
|
||||
415
.github/workflows/cli-e2e.yml
vendored
Normal file
415
.github/workflows/cli-e2e.yml
vendored
Normal file
@ -0,0 +1,415 @@
|
||||
name: CLI E2E Tests
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
cli_ref:
|
||||
description: "Git ref (default: current branch)"
|
||||
type: string
|
||||
required: false
|
||||
|
||||
edition:
|
||||
description: "Dify edition"
|
||||
type: choice
|
||||
required: false
|
||||
default: ee
|
||||
options: [ee, ce]
|
||||
|
||||
test_scope:
|
||||
description: "smoke = [P0] only / full = all cases"
|
||||
type: choice
|
||||
required: false
|
||||
default: full
|
||||
options: [smoke, full]
|
||||
|
||||
# ── Suite on/off ────────────────────────────────────────────────────────
|
||||
suite_framework_output_error:
|
||||
description: "framework + output + error-handling suites"
|
||||
type: boolean
|
||||
default: true
|
||||
suite_discovery:
|
||||
description: "discovery suite (get app / describe app)"
|
||||
type: boolean
|
||||
default: true
|
||||
suite_run:
|
||||
description: "run suite (basic / streaming / conversation / file / hitl)"
|
||||
type: boolean
|
||||
default: true
|
||||
suite_auth:
|
||||
description: "auth suite (login / status / whoami / use / devices / logout)"
|
||||
type: boolean
|
||||
default: true
|
||||
suite_agent:
|
||||
description: "agent suite"
|
||||
type: boolean
|
||||
default: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
# ── Shared env injected into every E2E job ───────────────────────────────────
|
||||
# Each job reads DIFY_E2E_TOKEN + app IDs from the provision job outputs,
|
||||
# so global-setup skips minting and finds existing apps in < 10 s.
|
||||
env:
|
||||
DIFY_E2E_NO_KEYRING: "1" # Linux CI has no keychain; skip probe
|
||||
VITEST_RETRY: "2" # Retry flaky staging responses
|
||||
|
||||
jobs:
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 0. PROVISION — mint token + import DSL fixtures (runs once, outputs IDs)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
provision:
|
||||
name: "Provision: mint token + DSL apps"
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
token: ${{ steps.out.outputs.DIFY_E2E_TOKEN }}
|
||||
workspace_id: ${{ steps.out.outputs.DIFY_E2E_WORKSPACE_ID }}
|
||||
workspace_name: ${{ steps.out.outputs.DIFY_E2E_WORKSPACE_NAME }}
|
||||
ws2_id: ${{ steps.out.outputs.DIFY_E2E_WS2_ID }}
|
||||
chat_app_id: ${{ steps.out.outputs.DIFY_E2E_CHAT_APP_ID }}
|
||||
workflow_app_id: ${{ steps.out.outputs.DIFY_E2E_WORKFLOW_APP_ID }}
|
||||
file_app_id: ${{ steps.out.outputs.DIFY_E2E_FILE_APP_ID }}
|
||||
file_chat_app_id: ${{ steps.out.outputs.DIFY_E2E_FILE_CHAT_APP_ID }}
|
||||
hitl_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_APP_ID }}
|
||||
hitl_external_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_EXTERNAL_APP_ID }}
|
||||
hitl_single_action_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_SINGLE_ACTION_APP_ID }}
|
||||
hitl_multi_node_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_MULTI_NODE_APP_ID }}
|
||||
ws2_app_id: ${{ steps.out.outputs.DIFY_E2E_WS2_APP_ID }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: latest
|
||||
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with:
|
||||
package_json_field: packageManager
|
||||
run_install: false
|
||||
|
||||
- name: Install CLI dependencies
|
||||
working-directory: cli
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Mint token & provision apps
|
||||
id: out
|
||||
working-directory: cli
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_TOKEN: ${{ secrets.DIFY_E2E_TOKEN }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
run: bun scripts/e2e-provision.ts
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-B. framework + output + error-handling (parallel with run/discovery)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
suite-framework-output-error:
|
||||
name: "Suite: framework + output + error-handling"
|
||||
if: ${{ inputs.suite_framework_output_error != 'false' }}
|
||||
needs: provision
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
defaults:
|
||||
run:
|
||||
working-directory: cli
|
||||
shell: bash
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
|
||||
- uses: ./.github/actions/setup-web
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with: { bun-version: latest }
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with: { package_json_field: packageManager, run_install: false }
|
||||
- run: pnpm install --frozen-lockfile
|
||||
- run: pnpm tree:gen
|
||||
|
||||
- name: Run framework + output + error-handling
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_WORKFLOW_APP_ID: ${{ needs.provision.outputs.workflow_app_id }}
|
||||
DIFY_E2E_INCLUDE: "test/e2e/suites/framework/**/*.e2e.ts,test/e2e/suites/output/**/*.e2e.ts,test/e2e/suites/error-handling/**/*.e2e.ts"
|
||||
run: |
|
||||
if [ "${{ inputs.test_scope }}" = "smoke" ]; then
|
||||
pnpm test:e2e -- -t "\[P0\]"
|
||||
else
|
||||
pnpm test:e2e
|
||||
fi
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-C. Discovery (parallel)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
suite-discovery:
|
||||
name: "Suite: discovery"
|
||||
if: ${{ inputs.suite_discovery != 'false' }}
|
||||
needs: provision
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
defaults:
|
||||
run:
|
||||
working-directory: cli
|
||||
shell: bash
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
|
||||
- uses: ./.github/actions/setup-web
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with: { bun-version: latest }
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with: { package_json_field: packageManager, run_install: false }
|
||||
- run: pnpm install --frozen-lockfile
|
||||
- run: pnpm tree:gen
|
||||
|
||||
- name: Run discovery suite
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_WS2_ID: ${{ needs.provision.outputs.ws2_id }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_WORKFLOW_APP_ID: ${{ needs.provision.outputs.workflow_app_id }}
|
||||
DIFY_E2E_INCLUDE: "test/e2e/suites/discovery/**/*.e2e.ts"
|
||||
run: |
|
||||
if [ "${{ inputs.test_scope }}" = "smoke" ]; then
|
||||
pnpm test:e2e -- -t "\[P0\]"
|
||||
else
|
||||
pnpm test:e2e
|
||||
fi
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-D. Run suite — 5 files in matrix (parallel)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
suite-run:
|
||||
name: "Suite: run / ${{ matrix.name }}"
|
||||
if: ${{ inputs.suite_run != 'false' }}
|
||||
needs: provision
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- name: basic
|
||||
file: run-app-basic.e2e.ts
|
||||
- name: streaming
|
||||
file: run-app-streaming.e2e.ts
|
||||
- name: conversation
|
||||
file: run-app-conversation.e2e.ts
|
||||
- name: file
|
||||
file: run-app-file.e2e.ts
|
||||
- name: hitl
|
||||
file: run-app-hitl.e2e.ts
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: cli
|
||||
shell: bash
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
|
||||
- uses: ./.github/actions/setup-web
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with: { bun-version: latest }
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with: { package_json_field: packageManager, run_install: false }
|
||||
- run: pnpm install --frozen-lockfile
|
||||
- run: pnpm tree:gen
|
||||
|
||||
- name: "Run run/${{ matrix.name }}"
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_SSO_TOKEN: ${{ secrets.DIFY_E2E_SSO_TOKEN }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_WORKFLOW_APP_ID: ${{ needs.provision.outputs.workflow_app_id }}
|
||||
DIFY_E2E_FILE_APP_ID: ${{ needs.provision.outputs.file_app_id }}
|
||||
DIFY_E2E_FILE_CHAT_APP_ID: ${{ needs.provision.outputs.file_chat_app_id }}
|
||||
DIFY_E2E_HITL_APP_ID: ${{ needs.provision.outputs.hitl_app_id }}
|
||||
DIFY_E2E_HITL_EXTERNAL_APP_ID: ${{ needs.provision.outputs.hitl_external_app_id }}
|
||||
DIFY_E2E_HITL_SINGLE_ACTION_APP_ID: ${{ needs.provision.outputs.hitl_single_action_app_id }}
|
||||
DIFY_E2E_HITL_MULTI_NODE_APP_ID: ${{ needs.provision.outputs.hitl_multi_node_app_id }}
|
||||
DIFY_E2E_INCLUDE: "test/e2e/suites/run/${{ matrix.file }}"
|
||||
run: |
|
||||
if [ "${{ inputs.test_scope }}" = "smoke" ]; then
|
||||
pnpm test:e2e -- -t "\[P0\]"
|
||||
else
|
||||
pnpm test:e2e
|
||||
fi
|
||||
|
||||
- name: Upload results on failure
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-run-${{ matrix.name }}-${{ github.run_id }}
|
||||
path: cli/test-results/
|
||||
retention-days: 3
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-E. auth/login + status + whoami (parallel, read-only, safe)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
suite-auth-safe:
|
||||
name: "Suite: auth (login / status / whoami)"
|
||||
if: ${{ inputs.suite_auth != 'false' }}
|
||||
needs: provision
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
defaults:
|
||||
run:
|
||||
working-directory: cli
|
||||
shell: bash
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
|
||||
- uses: ./.github/actions/setup-web
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with: { bun-version: latest }
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with: { package_json_field: packageManager, run_install: false }
|
||||
- run: pnpm install --frozen-lockfile
|
||||
- run: pnpm tree:gen
|
||||
|
||||
- name: Run auth/login + status + whoami
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_WS2_ID: ${{ needs.provision.outputs.ws2_id }}
|
||||
DIFY_E2E_INCLUDE: "test/e2e/suites/auth/login.e2e.ts,test/e2e/suites/auth/status.e2e.ts,test/e2e/suites/auth/whoami.e2e.ts"
|
||||
run: |
|
||||
if [ "${{ inputs.test_scope }}" = "smoke" ]; then
|
||||
pnpm test:e2e -- -t "\[P0\]"
|
||||
else
|
||||
pnpm test:e2e
|
||||
fi
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 2. DESTRUCTIVE — auth/use + devices + logout + agent (serial, runs LAST)
|
||||
# Must wait for ALL parallel suites to finish to avoid token revocation
|
||||
# invalidating other in-flight requests.
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
suite-last:
|
||||
name: "Suite: auth-use + devices + logout + agent (last, serial)"
|
||||
# Runs when auth is selected; also runs after all parallel jobs finish
|
||||
if: ${{ inputs.suite_auth != 'false' || inputs.suite_agent != 'false' }}
|
||||
needs:
|
||||
- provision
|
||||
- suite-framework-output-error
|
||||
- suite-discovery
|
||||
- suite-run
|
||||
- suite-auth-safe
|
||||
# `needs` on a skipped job is treated as success — safe to proceed even if
|
||||
# some suites were disabled via toggle.
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 25
|
||||
defaults:
|
||||
run:
|
||||
working-directory: cli
|
||||
shell: bash
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
|
||||
- uses: ./.github/actions/setup-web
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with: { bun-version: latest }
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with: { package_json_field: packageManager, run_install: false }
|
||||
- run: pnpm install --frozen-lockfile
|
||||
- run: pnpm tree:gen
|
||||
|
||||
- name: Run use / devices / logout / agent (serial)
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_WS2_ID: ${{ needs.provision.outputs.ws2_id }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_WORKFLOW_APP_ID: ${{ needs.provision.outputs.workflow_app_id }}
|
||||
DIFY_E2E_HITL_APP_ID: ${{ needs.provision.outputs.hitl_app_id }}
|
||||
DIFY_E2E_HITL_EXTERNAL_APP_ID: ${{ needs.provision.outputs.hitl_external_app_id }}
|
||||
DIFY_E2E_HITL_SINGLE_ACTION_APP_ID: ${{ needs.provision.outputs.hitl_single_action_app_id }}
|
||||
DIFY_E2E_HITL_MULTI_NODE_APP_ID: ${{ needs.provision.outputs.hitl_multi_node_app_id }}
|
||||
run: |
|
||||
# Collect files in safe order: use → devices → logout (revokes last) → agent
|
||||
FILES=()
|
||||
if [ "${{ inputs.suite_auth }}" = "true" ]; then
|
||||
FILES+=(
|
||||
test/e2e/suites/auth/use.e2e.ts
|
||||
test/e2e/suites/auth/devices.e2e.ts
|
||||
test/e2e/suites/auth/logout.e2e.ts
|
||||
)
|
||||
fi
|
||||
if [ "${{ inputs.suite_agent }}" = "true" ]; then
|
||||
while IFS= read -r f; do FILES+=("$f"); done \
|
||||
< <(find test/e2e/suites/agent -name '*.e2e.ts' | sort)
|
||||
fi
|
||||
|
||||
[ ${#FILES[@]} -eq 0 ] && { echo "Nothing to run."; exit 0; }
|
||||
|
||||
# Pass files via DIFY_E2E_INCLUDE (comma-separated) so vitest
|
||||
# config's include list is overridden instead of ANDed.
|
||||
INCLUDE=$(IFS=,; echo "${FILES[*]}")
|
||||
if [ "${{ inputs.test_scope }}" = "smoke" ]; then
|
||||
DIFY_E2E_INCLUDE="$INCLUDE" pnpm test:e2e -- -t "\[P0\]"
|
||||
else
|
||||
DIFY_E2E_INCLUDE="$INCLUDE" pnpm test:e2e
|
||||
fi
|
||||
|
||||
- name: Upload results on failure
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-last-${{ github.run_id }}
|
||||
path: cli/test-results/
|
||||
retention-days: 3
|
||||
6
.github/workflows/cli-release.yml
vendored
6
.github/workflows/cli-release.yml
vendored
@ -35,7 +35,7 @@ jobs:
|
||||
dify_tag: ${{ steps.resolve.outputs.dify_tag }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -98,7 +98,7 @@ jobs:
|
||||
DIFY_TAG: ${{ needs.validate.outputs.dify_tag }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 1
|
||||
@ -114,7 +114,7 @@ jobs:
|
||||
run: node scripts/release-naming.mjs github-env >> "$GITHUB_ENV"
|
||||
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.0.2
|
||||
uses: oven-sh/setup-bun@4bc047ad259df6fc24a6c9b0f9a0cb08cf17fbe5 # v2.0.2
|
||||
with:
|
||||
bun-version-file: cli/.bun-version
|
||||
|
||||
|
||||
2
.github/workflows/cli-smoke.yml
vendored
2
.github/workflows/cli-smoke.yml
vendored
@ -24,7 +24,7 @@ jobs:
|
||||
shell: bash
|
||||
steps:
|
||||
- name: Checkout cli ref
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
|
||||
4
.github/workflows/cli-tests.yml
vendored
4
.github/workflows/cli-tests.yml
vendored
@ -30,7 +30,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -46,7 +46,7 @@ jobs:
|
||||
|
||||
- name: Report coverage
|
||||
if: ${{ env.CODECOV_TOKEN != '' && matrix.os == 'depot-ubuntu-24.04' }}
|
||||
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
|
||||
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
|
||||
with:
|
||||
directory: cli/coverage
|
||||
flags: cli
|
||||
|
||||
12
.github/workflows/db-migration-test.yml
vendored
12
.github/workflows/db-migration-test.yml
vendored
@ -13,13 +13,13 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: "3.12"
|
||||
@ -40,7 +40,7 @@ jobs:
|
||||
cp envs/middleware.env.example middleware.env
|
||||
|
||||
- name: Set up Middlewares
|
||||
uses: hoverkraft-tech/compose-action@11beaa1c2dae4e8ed7b1665aa074723b6cecb0e4 # v3.0.0
|
||||
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0
|
||||
with:
|
||||
compose-file: |
|
||||
docker/docker-compose.middleware.yaml
|
||||
@ -63,13 +63,13 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: "3.12"
|
||||
@ -94,7 +94,7 @@ jobs:
|
||||
sed -i 's/DB_USERNAME=postgres/DB_USERNAME=mysql/' middleware.env
|
||||
|
||||
- name: Set up Middlewares
|
||||
uses: hoverkraft-tech/compose-action@11beaa1c2dae4e8ed7b1665aa074723b6cecb0e4 # v3.0.0
|
||||
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0
|
||||
with:
|
||||
compose-file: |
|
||||
docker/docker-compose.middleware.yaml
|
||||
|
||||
6
.github/workflows/docker-build.yml
vendored
6
.github/workflows/docker-build.yml
vendored
@ -53,7 +53,7 @@ jobs:
|
||||
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.7.1
|
||||
|
||||
- name: Build Docker Image
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.18.0
|
||||
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.17.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
push: false
|
||||
@ -77,10 +77,10 @@ jobs:
|
||||
file: "web/Dockerfile"
|
||||
steps:
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
- name: Build Docker Image
|
||||
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
|
||||
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
|
||||
with:
|
||||
push: false
|
||||
context: ${{ matrix.context }}
|
||||
|
||||
2
.github/workflows/hotfix-cherry-pick.yml
vendored
2
.github/workflows/hotfix-cherry-pick.yml
vendored
@ -24,7 +24,7 @@ jobs:
|
||||
name: Require cherry-pick provenance
|
||||
runs-on: depot-ubuntu-24.04
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
|
||||
2
.github/workflows/main-ci.yml
vendored
2
.github/workflows/main-ci.yml
vendored
@ -48,7 +48,7 @@ jobs:
|
||||
vdb-changed: ${{ steps.changes.outputs.vdb }}
|
||||
migration-changed: ${{ steps.changes.outputs.migration }}
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
- uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
|
||||
4
.github/workflows/pyrefly-diff.yml
vendored
4
.github/workflows/pyrefly-diff.yml
vendored
@ -17,12 +17,12 @@ jobs:
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Checkout PR branch
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Setup Python & UV
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
|
||||
@ -21,10 +21,10 @@ jobs:
|
||||
if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.pull_requests[0].head.repo.full_name != github.repository }}
|
||||
steps:
|
||||
- name: Checkout default branch (trusted code)
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Setup Python & UV
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
|
||||
4
.github/workflows/pyrefly-type-coverage.yml
vendored
4
.github/workflows/pyrefly-type-coverage.yml
vendored
@ -17,12 +17,12 @@ jobs:
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Checkout PR branch
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Setup Python & UV
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
|
||||
2
.github/workflows/stale.yml
vendored
2
.github/workflows/stale.yml
vendored
@ -18,7 +18,7 @@ jobs:
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- uses: actions/stale@eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899 # v10.3.0
|
||||
- uses: actions/stale@b5d41d4e1d5dceea10e7104786b73624c18a190f # v10.2.0
|
||||
with:
|
||||
days-before-issue-stale: 15
|
||||
days-before-issue-close: 3
|
||||
|
||||
10
.github/workflows/style.yml
vendored
10
.github/workflows/style.yml
vendored
@ -19,7 +19,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -33,7 +33,7 @@ jobs:
|
||||
|
||||
- name: Setup UV and Python
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: false
|
||||
python-version: "3.12"
|
||||
@ -71,7 +71,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -114,7 +114,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -171,7 +171,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
2
.github/workflows/tool-test-sdks.yaml
vendored
2
.github/workflows/tool-test-sdks.yaml
vendored
@ -24,7 +24,7 @@ jobs:
|
||||
working-directory: sdks/nodejs-client
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
|
||||
4
.github/workflows/translate-i18n-claude.yml
vendored
4
.github/workflows/translate-i18n-claude.yml
vendored
@ -40,7 +40,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
@ -158,7 +158,7 @@ jobs:
|
||||
|
||||
- name: Run Claude Code for Translation Sync
|
||||
if: steps.context.outputs.CHANGED_FILES != ''
|
||||
uses: anthropics/claude-code-action@fbda2eb1bdc90d319b8d853f5deb53bca199a7c1 # v1.0.140
|
||||
uses: anthropics/claude-code-action@1dc994ee7a008f0ecc866d9ac23ef036b7229f84 # v1.0.127
|
||||
with:
|
||||
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
2
.github/workflows/trigger-i18n-sync.yml
vendored
2
.github/workflows/trigger-i18n-sync.yml
vendored
@ -21,7 +21,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
|
||||
4
.github/workflows/vdb-tests-full.yml
vendored
4
.github/workflows/vdb-tests-full.yml
vendored
@ -24,7 +24,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -36,7 +36,7 @@ jobs:
|
||||
remove_tool_cache: true
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
4
.github/workflows/vdb-tests.yml
vendored
4
.github/workflows/vdb-tests.yml
vendored
@ -21,7 +21,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -33,7 +33,7 @@ jobs:
|
||||
remove_tool_cache: true
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
4
.github/workflows/web-e2e.yml
vendored
4
.github/workflows/web-e2e.yml
vendored
@ -20,7 +20,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -28,7 +28,7 @@ jobs:
|
||||
uses: ./.github/actions/setup-web
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: "3.12"
|
||||
|
||||
10
.github/workflows/web-tests.yml
vendored
10
.github/workflows/web-tests.yml
vendored
@ -31,7 +31,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -64,7 +64,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -83,7 +83,7 @@ jobs:
|
||||
|
||||
- name: Report coverage
|
||||
if: ${{ env.CODECOV_TOKEN != '' }}
|
||||
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
|
||||
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
|
||||
with:
|
||||
directory: web/coverage
|
||||
flags: web
|
||||
@ -102,7 +102,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -117,7 +117,7 @@ jobs:
|
||||
|
||||
- name: Report coverage
|
||||
if: ${{ env.CODECOV_TOKEN != '' }}
|
||||
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
|
||||
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
|
||||
with:
|
||||
directory: packages/dify-ui/coverage
|
||||
flags: dify-ui
|
||||
|
||||
@ -7,7 +7,6 @@ consumes injected context managers when it needs to preserve thread-local state.
|
||||
|
||||
import contextvars
|
||||
import threading
|
||||
from abc import ABC, abstractmethod
|
||||
from collections.abc import Callable, Generator
|
||||
from contextlib import AbstractContextManager, contextmanager
|
||||
from typing import Any, Protocol, final, override, runtime_checkable
|
||||
@ -15,28 +14,25 @@ from typing import Any, Protocol, final, override, runtime_checkable
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class AppContext(ABC):
|
||||
class AppContext(Protocol):
|
||||
"""
|
||||
Abstract application context interface.
|
||||
Application context interface.
|
||||
|
||||
Application adapters can implement this to restore framework-specific state
|
||||
such as Flask app context around worker execution.
|
||||
"""
|
||||
|
||||
@abstractmethod
|
||||
def get_config(self, key: str, default: Any = None) -> Any:
|
||||
"""Get configuration value by key."""
|
||||
raise NotImplementedError
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
def get_extension(self, name: str) -> Any:
|
||||
"""Get application extension by name."""
|
||||
raise NotImplementedError
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
def enter(self) -> AbstractContextManager[None]:
|
||||
"""Enter the application context."""
|
||||
raise NotImplementedError
|
||||
...
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
|
||||
@ -5,11 +5,17 @@ from pydantic import BaseModel, Field, field_validator
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
from controllers.console.wraps import account_initialization_required, setup_required
|
||||
from controllers.console.wraps import account_initialization_required, setup_required, with_current_user
|
||||
from extensions.ext_database import db
|
||||
from libs.helper import uuid_value
|
||||
from libs.login import login_required
|
||||
from models import Account
|
||||
from models.model import App, AppMode
|
||||
from services.agent.skill_package_service import SkillPackageError, SkillPackageService
|
||||
from services.agent.skill_standardize_service import SkillStandardizeService
|
||||
from services.agent_drive_service import AgentDriveError
|
||||
from services.agent_service import AgentService
|
||||
from services.file_service import FileService
|
||||
|
||||
|
||||
class AgentLogQuery(BaseModel):
|
||||
@ -44,3 +50,80 @@ class AgentLogApi(Resource):
|
||||
args = AgentLogQuery.model_validate(request.args.to_dict(flat=True))
|
||||
|
||||
return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/agent/skills/upload")
|
||||
class AgentSkillUploadApi(Resource):
|
||||
@console_ns.doc("upload_agent_skill")
|
||||
@console_ns.doc(description="Upload + validate a Skill package (.zip/.skill) and extract its manifest")
|
||||
@console_ns.doc(params={"app_id": "Application ID"})
|
||||
@console_ns.response(201, "Skill validated")
|
||||
@console_ns.response(400, "Invalid skill package")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.AGENT])
|
||||
@with_current_user
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""Validate an uploaded Skill package and persist the archive.
|
||||
|
||||
Returns a validated skill ref (to bind into the Agent soul config on save)
|
||||
plus its manifest. Standardizing into the agent drive is ENG-594.
|
||||
"""
|
||||
if "file" not in request.files:
|
||||
return {"code": "no_file", "message": "no skill file uploaded"}, 400
|
||||
if len(request.files) > 1:
|
||||
return {"code": "too_many_files", "message": "only one skill file is allowed"}, 400
|
||||
|
||||
upload = request.files["file"]
|
||||
content = upload.stream.read()
|
||||
try:
|
||||
manifest = SkillPackageService().validate_and_extract(content=content, filename=upload.filename or "")
|
||||
except SkillPackageError as exc:
|
||||
return {"code": exc.code, "message": exc.message}, exc.status_code
|
||||
|
||||
upload_file = FileService(db.engine).upload_file(
|
||||
filename=upload.filename or "skill.zip",
|
||||
content=content,
|
||||
mimetype=upload.mimetype or "application/zip",
|
||||
user=current_user,
|
||||
)
|
||||
skill_ref = manifest.to_skill_ref(file_id=upload_file.id)
|
||||
return {"skill": skill_ref.model_dump(exclude_none=True), "manifest": manifest.model_dump()}, 201
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/agent/skills/standardize")
|
||||
class AgentSkillStandardizeApi(Resource):
|
||||
@console_ns.doc("standardize_agent_skill")
|
||||
@console_ns.doc(description="Validate + standardize a Skill into the agent drive (ENG-594)")
|
||||
@console_ns.doc(params={"app_id": "Application ID"})
|
||||
@console_ns.response(201, "Skill standardized into drive")
|
||||
@console_ns.response(400, "Invalid skill package or no bound agent")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.AGENT])
|
||||
@with_current_user
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""Upload a Skill, validate it, and standardize it into the app agent's drive."""
|
||||
agent_id = app_model.bound_agent_id
|
||||
if not agent_id:
|
||||
return {"code": "no_bound_agent", "message": "app has no bound agent"}, 400
|
||||
if "file" not in request.files:
|
||||
return {"code": "no_file", "message": "no skill file uploaded"}, 400
|
||||
if len(request.files) > 1:
|
||||
return {"code": "too_many_files", "message": "only one skill file is allowed"}, 400
|
||||
|
||||
upload = request.files["file"]
|
||||
content = upload.stream.read()
|
||||
try:
|
||||
result = SkillStandardizeService().standardize(
|
||||
content=content,
|
||||
filename=upload.filename or "",
|
||||
tenant_id=app_model.tenant_id,
|
||||
user_id=current_user.id,
|
||||
agent_id=agent_id,
|
||||
)
|
||||
except (SkillPackageError, AgentDriveError) as exc:
|
||||
return {"code": exc.code, "message": exc.message}, exc.status_code
|
||||
return result, 201
|
||||
|
||||
@ -21,7 +21,12 @@ from controllers.common.schema import (
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.error import ConversationCompletedError, DraftWorkflowNotExist, DraftWorkflowNotSync
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
|
||||
from controllers.console.wraps import (
|
||||
account_initialization_required,
|
||||
edit_permission_required,
|
||||
setup_required,
|
||||
with_current_user,
|
||||
)
|
||||
from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpError
|
||||
from core.app.app_config.features.file_upload.manager import FileUploadConfigManager
|
||||
from core.app.apps.base_app_queue_manager import AppQueueManager
|
||||
@ -54,7 +59,7 @@ from libs import helper
|
||||
from libs.datetime_utils import naive_utc_now
|
||||
from libs.helper import TimestampField, dump_response, to_timestamp, uuid_value
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from models import App
|
||||
from models import Account, App
|
||||
from models.model import AppMode
|
||||
from models.workflow import Workflow
|
||||
from repositories.workflow_collaboration_repository import WORKFLOW_ONLINE_USERS_PREFIX
|
||||
@ -401,13 +406,12 @@ class DraftWorkflowApi(Resource):
|
||||
)
|
||||
@console_ns.response(400, "Invalid workflow configuration")
|
||||
@console_ns.response(403, "Permission denied")
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App):
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""
|
||||
Sync draft workflow
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
content_type = request.headers.get("Content-Type", "")
|
||||
|
||||
if "application/json" in content_type:
|
||||
@ -468,13 +472,12 @@ class AdvancedChatDraftWorkflowRunApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App):
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""
|
||||
Run draft workflow
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
args_model = AdvancedChatWorkflowRunPayload.model_validate(console_ns.payload or {})
|
||||
args = args_model.model_dump(exclude_none=True)
|
||||
|
||||
@ -514,12 +517,12 @@ class AdvancedChatDraftRunIterationNodeApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, node_id: str):
|
||||
def post(self, current_user: Account, app_model: App, node_id: str):
|
||||
"""
|
||||
Run draft workflow iteration node
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
args = IterationNodeRunPayload.model_validate(console_ns.payload or {}).model_dump(exclude_none=True)
|
||||
|
||||
try:
|
||||
@ -552,12 +555,12 @@ class WorkflowDraftRunIterationNodeApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, node_id: str):
|
||||
def post(self, current_user: Account, app_model: App, node_id: str):
|
||||
"""
|
||||
Run draft workflow iteration node
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
args = IterationNodeRunPayload.model_validate(console_ns.payload or {}).model_dump(exclude_none=True)
|
||||
|
||||
try:
|
||||
@ -590,12 +593,12 @@ class AdvancedChatDraftRunLoopNodeApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, node_id: str):
|
||||
def post(self, current_user: Account, app_model: App, node_id: str):
|
||||
"""
|
||||
Run draft workflow loop node
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
args = LoopNodeRunPayload.model_validate(console_ns.payload or {})
|
||||
|
||||
try:
|
||||
@ -628,12 +631,12 @@ class WorkflowDraftRunLoopNodeApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, node_id: str):
|
||||
def post(self, current_user: Account, app_model: App, node_id: str):
|
||||
"""
|
||||
Run draft workflow loop node
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
args = LoopNodeRunPayload.model_validate(console_ns.payload or {})
|
||||
|
||||
try:
|
||||
@ -695,12 +698,12 @@ class AdvancedChatDraftHumanInputFormPreviewApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, node_id: str):
|
||||
def post(self, current_user: Account, app_model: App, node_id: str):
|
||||
"""
|
||||
Preview human input form content and placeholders
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
args = HumanInputFormPreviewPayload.model_validate(console_ns.payload or {})
|
||||
inputs = args.inputs
|
||||
|
||||
@ -724,12 +727,12 @@ class AdvancedChatDraftHumanInputFormRunApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, node_id: str):
|
||||
def post(self, current_user: Account, app_model: App, node_id: str):
|
||||
"""
|
||||
Submit human input form preview
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
args = HumanInputFormSubmitPayload.model_validate(console_ns.payload or {})
|
||||
workflow_service = WorkflowService()
|
||||
result = workflow_service.submit_human_input_form_preview(
|
||||
@ -753,12 +756,12 @@ class WorkflowDraftHumanInputFormPreviewApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, node_id: str):
|
||||
def post(self, current_user: Account, app_model: App, node_id: str):
|
||||
"""
|
||||
Preview human input form content and placeholders
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
args = HumanInputFormPreviewPayload.model_validate(console_ns.payload or {})
|
||||
inputs = args.inputs
|
||||
|
||||
@ -782,12 +785,12 @@ class WorkflowDraftHumanInputFormRunApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, node_id: str):
|
||||
def post(self, current_user: Account, app_model: App, node_id: str):
|
||||
"""
|
||||
Submit human input form preview
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
workflow_service = WorkflowService()
|
||||
args = HumanInputFormSubmitPayload.model_validate(console_ns.payload or {})
|
||||
result = workflow_service.submit_human_input_form_preview(
|
||||
@ -811,12 +814,12 @@ class WorkflowDraftHumanInputDeliveryTestApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, node_id: str):
|
||||
def post(self, current_user: Account, app_model: App, node_id: str):
|
||||
"""
|
||||
Test human input delivery
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
workflow_service = WorkflowService()
|
||||
args = HumanInputDeliveryTestPayload.model_validate(console_ns.payload or {})
|
||||
workflow_service.test_human_input_delivery(
|
||||
@ -841,12 +844,12 @@ class DraftWorkflowRunApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App):
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""
|
||||
Run draft workflow
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
args = DraftWorkflowRunPayload.model_validate(console_ns.payload or {}).model_dump(exclude_none=True)
|
||||
|
||||
external_trace_id = get_external_trace_id(request)
|
||||
@ -911,12 +914,12 @@ class DraftWorkflowNodeRunApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, node_id: str):
|
||||
def post(self, current_user: Account, app_model: App, node_id: str):
|
||||
"""
|
||||
Run draft workflow node
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
args_model = DraftWorkflowNodeRunPayload.model_validate(console_ns.payload or {})
|
||||
args = args_model.model_dump(exclude_none=True)
|
||||
|
||||
@ -981,12 +984,12 @@ class PublishedWorkflowApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App):
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""
|
||||
Publish workflow
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
args = PublishWorkflowPayload.model_validate(console_ns.payload or {})
|
||||
|
||||
@ -1083,14 +1086,14 @@ class ConvertToWorkflowApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.CHAT, AppMode.COMPLETION])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App):
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""
|
||||
Convert basic mode of chatbot app to workflow mode
|
||||
Convert expert mode of chatbot app to workflow mode
|
||||
Convert Completion App to Workflow App
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
payload = console_ns.payload or {}
|
||||
args = ConvertToWorkflowPayload.model_validate(payload).model_dump(exclude_none=True)
|
||||
@ -1122,9 +1125,9 @@ class WorkflowFeaturesApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
|
||||
args = WorkflowFeaturesPayload.model_validate(console_ns.payload or {})
|
||||
features = args.features
|
||||
@ -1150,12 +1153,12 @@ class PublishedAllWorkflowApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def get(self, app_model: App):
|
||||
def get(self, current_user: Account, app_model: App):
|
||||
"""
|
||||
Get published workflows
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
args = WorkflowListQuery.model_validate(request.args.to_dict(flat=True))
|
||||
page = args.page
|
||||
@ -1199,9 +1202,9 @@ class DraftWorkflowRestoreApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, workflow_id: str):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
def post(self, current_user: Account, app_model: App, workflow_id: str):
|
||||
workflow_service = WorkflowService()
|
||||
|
||||
try:
|
||||
@ -1237,12 +1240,12 @@ class WorkflowByIdApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def patch(self, app_model: App, workflow_id: str):
|
||||
def patch(self, current_user: Account, app_model: App, workflow_id: str):
|
||||
"""
|
||||
Update workflow attributes
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
args = WorkflowUpdatePayload.model_validate(console_ns.payload or {})
|
||||
|
||||
# Prepare update data
|
||||
@ -1355,12 +1358,12 @@ class DraftWorkflowTriggerRunApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App):
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""
|
||||
Poll for trigger events and execute full workflow when event arrives
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
args = DraftWorkflowTriggerRunPayload.model_validate(console_ns.payload or {})
|
||||
node_id = args.node_id
|
||||
workflow_service = WorkflowService()
|
||||
@ -1419,12 +1422,12 @@ class DraftWorkflowTriggerNodeApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App, node_id: str):
|
||||
def post(self, current_user: Account, app_model: App, node_id: str):
|
||||
"""
|
||||
Poll for trigger events and execute single node when event arrives
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
workflow_service = WorkflowService()
|
||||
draft_workflow = workflow_service.get_draft_workflow(app_model)
|
||||
@ -1499,12 +1502,12 @@ class DraftWorkflowTriggerRunAllApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
def post(self, app_model: App):
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""
|
||||
Full workflow debug when the start node is a trigger
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
args = DraftWorkflowTriggerRunAllPayload.model_validate(console_ns.payload or {})
|
||||
node_ids = args.node_ids
|
||||
|
||||
@ -30,6 +30,7 @@ from uuid import UUID
|
||||
from flask import Response
|
||||
from flask_restx import Resource
|
||||
|
||||
from controllers.common.schema import register_response_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
from controllers.console.wraps import account_initialization_required, setup_required
|
||||
@ -38,8 +39,13 @@ from libs.login import login_required
|
||||
from models import App, AppMode
|
||||
from services.workflow import inspector_events
|
||||
from services.workflow.node_output_inspector_service import (
|
||||
CheckResultView,
|
||||
NodeOutputInspectorError,
|
||||
NodeOutputInspectorService,
|
||||
NodeOutputsView,
|
||||
NodeOutputView,
|
||||
OutputPreviewView,
|
||||
WorkflowRunSnapshotView,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@ -54,6 +60,15 @@ _HEARTBEAT_EVERY_TICKS = 15
|
||||
# many ticks (= seconds).
|
||||
_STREAM_HARD_TIMEOUT_TICKS = 1800 # 30 min
|
||||
|
||||
register_response_schema_models(
|
||||
console_ns,
|
||||
CheckResultView,
|
||||
NodeOutputView,
|
||||
NodeOutputsView,
|
||||
WorkflowRunSnapshotView,
|
||||
OutputPreviewView,
|
||||
)
|
||||
|
||||
|
||||
def _service() -> NodeOutputInspectorService:
|
||||
"""One-line factory so tests can monkeypatch a stub if needed."""
|
||||
@ -124,6 +139,7 @@ class WorkflowDraftRunNodeOutputsApi(Resource):
|
||||
@console_ns.doc("get_workflow_draft_run_node_outputs")
|
||||
@console_ns.doc(description="Snapshot of every node's declared outputs for a draft workflow run.")
|
||||
@console_ns.doc(params={"app_id": "Application ID", "run_id": "Workflow run ID"})
|
||||
@console_ns.response(200, "Workflow run node outputs", console_ns.models[WorkflowRunSnapshotView.__name__])
|
||||
@console_ns.response(404, "Workflow run not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
@ -146,6 +162,7 @@ class WorkflowDraftRunNodeOutputDetailApi(Resource):
|
||||
"node_id": "Node ID inside the workflow graph",
|
||||
}
|
||||
)
|
||||
@console_ns.response(200, "Workflow run node output detail", console_ns.models[NodeOutputsView.__name__])
|
||||
@console_ns.response(404, "Workflow run / node not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
@ -171,6 +188,7 @@ class WorkflowDraftRunNodeOutputPreviewApi(Resource):
|
||||
"output_name": "Declared output name as exposed by Composer",
|
||||
}
|
||||
)
|
||||
@console_ns.response(200, "Workflow run node output preview", console_ns.models[OutputPreviewView.__name__])
|
||||
@console_ns.response(404, "Workflow run / node / output not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
@ -309,6 +327,7 @@ class WorkflowDraftRunNodeOutputEventsApi(Resource):
|
||||
@console_ns.doc("stream_workflow_draft_run_node_output_events")
|
||||
@console_ns.doc(description="Server-Sent Events stream of inspector deltas for a draft workflow run.")
|
||||
@console_ns.doc(params={"app_id": "Application ID", "run_id": "Workflow run ID"})
|
||||
@console_ns.response(200, "Workflow run node output event stream")
|
||||
@console_ns.response(404, "Workflow run not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
@ -338,6 +357,7 @@ class WorkflowPublishedRunNodeOutputsApi(Resource):
|
||||
@console_ns.doc("get_workflow_published_run_node_outputs")
|
||||
@console_ns.doc(description="Snapshot of every node's declared outputs for a published workflow run.")
|
||||
@console_ns.doc(params={"app_id": "Application ID", "run_id": "Workflow run ID"})
|
||||
@console_ns.response(200, "Workflow run node outputs", console_ns.models[WorkflowRunSnapshotView.__name__])
|
||||
@console_ns.response(404, "Workflow run not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
@ -360,6 +380,7 @@ class WorkflowPublishedRunNodeOutputDetailApi(Resource):
|
||||
"node_id": "Node ID inside the workflow graph",
|
||||
}
|
||||
)
|
||||
@console_ns.response(200, "Workflow run node output detail", console_ns.models[NodeOutputsView.__name__])
|
||||
@console_ns.response(404, "Workflow run / node not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
@ -386,6 +407,7 @@ class WorkflowPublishedRunNodeOutputPreviewApi(Resource):
|
||||
"output_name": "Declared output name as exposed by Composer",
|
||||
}
|
||||
)
|
||||
@console_ns.response(200, "Workflow run node output preview", console_ns.models[OutputPreviewView.__name__])
|
||||
@console_ns.response(404, "Workflow run / node / output not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
@ -402,6 +424,7 @@ class WorkflowPublishedRunNodeOutputEventsApi(Resource):
|
||||
@console_ns.doc("stream_workflow_published_run_node_output_events")
|
||||
@console_ns.doc(description="Server-Sent Events stream of inspector deltas for a published workflow run.")
|
||||
@console_ns.doc(params={"app_id": "Application ID", "run_id": "Workflow run ID"})
|
||||
@console_ns.response(200, "Workflow run node output event stream")
|
||||
@console_ns.response(404, "Workflow run not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
|
||||
@ -5,7 +5,7 @@ from typing import Any
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, computed_field, field_validator
|
||||
from sqlalchemy import and_, select
|
||||
from sqlalchemy import and_, exists, or_, select
|
||||
from werkzeug.exceptions import BadRequest, Forbidden, NotFound
|
||||
|
||||
from controllers.common.fields import SimpleMessageResponse, SimpleResultMessageResponse
|
||||
@ -24,8 +24,8 @@ from graphon.file import helpers as file_helpers
|
||||
from libs.datetime_utils import naive_utc_now
|
||||
from libs.helper import to_timestamp
|
||||
from libs.login import login_required
|
||||
from models import Account, App, InstalledApp, RecommendedApp
|
||||
from models.model import IconType
|
||||
from models import Account, App, AppModelConfig, InstalledApp, RecommendedApp, Workflow
|
||||
from models.model import AppMode, IconType
|
||||
from services.account_service import TenantService
|
||||
from services.enterprise.enterprise_service import EnterpriseService
|
||||
from services.feature_service import FeatureService
|
||||
@ -61,6 +61,24 @@ def _safe_primitive(value: Any) -> Any:
|
||||
return None
|
||||
|
||||
|
||||
def _published_app_filter():
|
||||
"""Return the SQL predicate for installed-app web API availability.
|
||||
|
||||
The installed-app parameters endpoint reads the published workflow for
|
||||
workflow-style apps and the published app model config for easy UI apps.
|
||||
Keep the list endpoint aligned in SQL so it does not return entries that
|
||||
will immediately fail with app_unavailable when opened.
|
||||
"""
|
||||
workflow_app_modes = (AppMode.ADVANCED_CHAT, AppMode.WORKFLOW)
|
||||
has_published_workflow = exists(select(Workflow.id).where(Workflow.id == App.workflow_id))
|
||||
has_published_model_config = exists(select(AppModelConfig.id).where(AppModelConfig.id == App.app_model_config_id))
|
||||
|
||||
return or_(
|
||||
and_(App.mode.in_(workflow_app_modes), App.workflow_id.isnot(None), has_published_workflow),
|
||||
and_(~App.mode.in_(workflow_app_modes), App.app_model_config_id.isnot(None), has_published_model_config),
|
||||
)
|
||||
|
||||
|
||||
class InstalledAppInfoResponse(ResponseModel):
|
||||
id: str
|
||||
name: str | None = None
|
||||
@ -141,33 +159,32 @@ class InstalledAppsListApi(Resource):
|
||||
def get(self, current_tenant_id: str, current_user: Account):
|
||||
query = InstalledAppsListQuery.model_validate(request.args.to_dict())
|
||||
|
||||
stmt = (
|
||||
select(InstalledApp, App)
|
||||
.join(App, App.id == InstalledApp.app_id)
|
||||
.where(InstalledApp.tenant_id == current_tenant_id, _published_app_filter())
|
||||
)
|
||||
if query.app_id:
|
||||
installed_apps = db.session.scalars(
|
||||
select(InstalledApp).where(
|
||||
and_(InstalledApp.tenant_id == current_tenant_id, InstalledApp.app_id == query.app_id)
|
||||
)
|
||||
).all()
|
||||
else:
|
||||
installed_apps = db.session.scalars(
|
||||
select(InstalledApp).where(InstalledApp.tenant_id == current_tenant_id)
|
||||
).all()
|
||||
stmt = stmt.where(InstalledApp.app_id == query.app_id)
|
||||
|
||||
installed_apps = db.session.execute(stmt).all()
|
||||
|
||||
if current_user.current_tenant is None:
|
||||
raise ValueError("current_user.current_tenant must not be None")
|
||||
current_user.role = TenantService.get_user_role(current_user, current_user.current_tenant)
|
||||
installed_app_list: list[dict[str, Any]] = [
|
||||
{
|
||||
"id": installed_app.id,
|
||||
"app": installed_app.app,
|
||||
"app_owner_tenant_id": installed_app.app_owner_tenant_id,
|
||||
"is_pinned": installed_app.is_pinned,
|
||||
"last_used_at": installed_app.last_used_at,
|
||||
"editable": current_user.role in {"owner", "admin"},
|
||||
"uninstallable": current_tenant_id == installed_app.app_owner_tenant_id,
|
||||
}
|
||||
for installed_app in installed_apps
|
||||
if installed_app.app is not None
|
||||
]
|
||||
installed_app_list: list[dict[str, Any]] = []
|
||||
for installed_app, app_model in installed_apps:
|
||||
installed_app_list.append(
|
||||
{
|
||||
"id": installed_app.id,
|
||||
"app": app_model,
|
||||
"app_owner_tenant_id": installed_app.app_owner_tenant_id,
|
||||
"is_pinned": installed_app.is_pinned,
|
||||
"last_used_at": installed_app.last_used_at,
|
||||
"editable": current_user.role in {"owner", "admin"},
|
||||
"uninstallable": current_tenant_id == installed_app.app_owner_tenant_id,
|
||||
}
|
||||
)
|
||||
|
||||
# filter out apps that user doesn't have access to
|
||||
if FeatureService.get_system_features().webapp_auth.enabled:
|
||||
|
||||
@ -20,7 +20,7 @@ from controllers.console.wraps import (
|
||||
setup_required,
|
||||
)
|
||||
from core.db.session_factory import session_factory
|
||||
from core.entities.mcp_provider import MCPAuthentication, MCPConfiguration
|
||||
from core.entities.mcp_provider import IdentityMode, MCPAuthentication, MCPConfiguration
|
||||
from core.mcp.auth.auth_flow import auth, handle_callback
|
||||
from core.mcp.error import MCPAuthError, MCPError, MCPRefreshTokenError
|
||||
from core.mcp.mcp_client import MCPClient
|
||||
@ -210,6 +210,30 @@ class MCPProviderBasePayload(BaseModel):
|
||||
configuration: dict[str, Any] | None = Field(default_factory=dict)
|
||||
headers: dict[str, Any] | None = Field(default_factory=dict)
|
||||
authentication: dict[str, Any] | None = Field(default_factory=dict)
|
||||
# None means "leave unchanged" on update; the controller resolves it to a
|
||||
# concrete IdentityMode before calling the service (see _resolve_identity_mode).
|
||||
identity_mode: IdentityMode | None = None
|
||||
|
||||
|
||||
def _resolve_identity_mode(requested: IdentityMode | None, *, current: IdentityMode) -> IdentityMode:
|
||||
"""Resolve the effective MCP identity_mode for a create/update request.
|
||||
|
||||
Keeps two API-layer concerns out of the service so the service always
|
||||
receives a concrete value:
|
||||
|
||||
* ``None`` means "leave unchanged" (update semantics) — fall back to
|
||||
``current`` (``IdentityMode.OFF`` for a brand-new provider).
|
||||
* Identity forwarding is an enterprise-only capability. On non-enterprise
|
||||
deployments any non-OFF value is coerced back to OFF so a persisted row
|
||||
can never imply forwarding that the runtime won't perform. This gates the
|
||||
API surface to match the backend gate in
|
||||
``MCPTool._forwarding_requested`` — both the API and the backend
|
||||
invocation must be gated on ``dify_config.ENTERPRISE_ENABLED``.
|
||||
"""
|
||||
mode = current if requested is None else requested
|
||||
if mode != IdentityMode.OFF and not dify_config.ENTERPRISE_ENABLED:
|
||||
return IdentityMode.OFF
|
||||
return mode
|
||||
|
||||
|
||||
class MCPProviderCreatePayload(MCPProviderBasePayload):
|
||||
@ -1000,6 +1024,7 @@ class ToolProviderMCPApi(Resource):
|
||||
headers=payload.headers or {},
|
||||
configuration=configuration,
|
||||
authentication=authentication,
|
||||
identity_mode=_resolve_identity_mode(payload.identity_mode, current=IdentityMode.OFF),
|
||||
)
|
||||
|
||||
# 2) Try to fetch tools immediately after creation so they appear without a second save.
|
||||
@ -1054,6 +1079,11 @@ class ToolProviderMCPApi(Resource):
|
||||
# Step 3: Perform database update in a transaction
|
||||
with sessionmaker(db.engine).begin() as session:
|
||||
service = MCPToolManageService(session=session)
|
||||
# Resolve "leave unchanged" (None) against the stored value, and gate
|
||||
# the result on ENTERPRISE_ENABLED — both are API-layer concerns, so
|
||||
# the service receives a concrete IdentityMode.
|
||||
existing = service.get_provider(provider_id=payload.provider_id, tenant_id=current_tenant_id)
|
||||
identity_mode = _resolve_identity_mode(payload.identity_mode, current=IdentityMode(existing.identity_mode))
|
||||
service.update_provider(
|
||||
tenant_id=current_tenant_id,
|
||||
provider_id=payload.provider_id,
|
||||
@ -1067,6 +1097,7 @@ class ToolProviderMCPApi(Resource):
|
||||
configuration=configuration,
|
||||
authentication=authentication,
|
||||
validation_result=validation_result,
|
||||
identity_mode=identity_mode,
|
||||
)
|
||||
|
||||
return {"result": "success"}
|
||||
|
||||
@ -17,12 +17,14 @@ inner_api_ns = Namespace("inner_api", description="Internal API operations", pat
|
||||
|
||||
from . import mail as _mail
|
||||
from .app import dsl as _app_dsl
|
||||
from .plugin import agent_drive as _agent_drive
|
||||
from .plugin import plugin as _plugin
|
||||
from .workspace import workspace as _workspace
|
||||
|
||||
api.add_namespace(inner_api_ns)
|
||||
|
||||
__all__ = [
|
||||
"_agent_drive",
|
||||
"_app_dsl",
|
||||
"_mail",
|
||||
"_plugin",
|
||||
|
||||
80
api/controllers/inner_api/plugin/agent_drive.py
Normal file
80
api/controllers/inner_api/plugin/agent_drive.py
Normal file
@ -0,0 +1,80 @@
|
||||
"""Inner API for the agent drive (agent 网盘) control plane — ENG-591.
|
||||
|
||||
Two endpoints, called by the dify-agent server (not the sandbox) with the inner
|
||||
API key. The drive ref is the URL segment ``agent-<agent_id>``; the path-like
|
||||
file key travels in the query/body, never as a URL path segment (so its ``/``
|
||||
characters do not collide with routing). Drive-owned semantics: tenant scoped,
|
||||
no user-level FileAccessScope.
|
||||
"""
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, ValidationError
|
||||
|
||||
from controllers.console.wraps import setup_required
|
||||
from controllers.inner_api import inner_api_ns
|
||||
from controllers.inner_api.wraps import plugin_inner_api_only
|
||||
from services.agent_drive_service import (
|
||||
AgentDriveError,
|
||||
AgentDriveService,
|
||||
DriveCommitItem,
|
||||
parse_agent_drive_ref,
|
||||
)
|
||||
|
||||
|
||||
class _CommitRequest(BaseModel):
|
||||
tenant_id: str
|
||||
user_id: str
|
||||
items: list[DriveCommitItem]
|
||||
|
||||
|
||||
def _error_response(exc: AgentDriveError) -> tuple[dict[str, str], int]:
|
||||
return {"code": exc.code, "message": exc.message}, exc.status_code
|
||||
|
||||
|
||||
@inner_api_ns.route("/drive/<string:drive_ref>/manifest")
|
||||
class AgentDriveManifestApi(Resource):
|
||||
@setup_required
|
||||
@plugin_inner_api_only
|
||||
@inner_api_ns.doc("agent_drive_manifest")
|
||||
@inner_api_ns.doc(description="List an agent drive (optionally with download URLs)")
|
||||
def get(self, drive_ref: str):
|
||||
try:
|
||||
agent_id = parse_agent_drive_ref(drive_ref)
|
||||
tenant_id = (request.args.get("tenant_id") or "").strip()
|
||||
if not tenant_id:
|
||||
raise AgentDriveError("missing_tenant_id", "tenant_id is required", status_code=400)
|
||||
include_download_url = (request.args.get("include_download_url") or "").lower() in ("1", "true", "yes")
|
||||
items = AgentDriveService().manifest(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
prefix=request.args.get("prefix", ""),
|
||||
include_download_url=include_download_url,
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return _error_response(exc)
|
||||
return {"items": items}
|
||||
|
||||
|
||||
@inner_api_ns.route("/drive/<string:drive_ref>/commit")
|
||||
class AgentDriveCommitApi(Resource):
|
||||
@setup_required
|
||||
@plugin_inner_api_only
|
||||
@inner_api_ns.doc("agent_drive_commit")
|
||||
@inner_api_ns.doc(description="Commit a batch of file refs into an agent drive")
|
||||
def post(self, drive_ref: str):
|
||||
try:
|
||||
agent_id = parse_agent_drive_ref(drive_ref)
|
||||
try:
|
||||
body = _CommitRequest.model_validate(request.get_json(silent=True) or {})
|
||||
except ValidationError as exc:
|
||||
raise AgentDriveError("invalid_request", str(exc), status_code=400) from exc
|
||||
items = AgentDriveService().commit(
|
||||
tenant_id=body.tenant_id,
|
||||
user_id=body.user_id,
|
||||
agent_id=agent_id,
|
||||
items=body.items,
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return _error_response(exc)
|
||||
return {"items": items}
|
||||
@ -25,6 +25,7 @@ from core.plugin.entities.request import (
|
||||
RequestInvokeTextEmbedding,
|
||||
RequestInvokeTool,
|
||||
RequestInvokeTTS,
|
||||
RequestRequestDownloadFile,
|
||||
RequestRequestUploadFile,
|
||||
)
|
||||
from core.tools.entities.tool_entities import ToolProviderType
|
||||
@ -33,6 +34,7 @@ from graphon.model_runtime.utils.encoders import jsonable_encoder
|
||||
from libs.helper import length_prefixed_response
|
||||
from models import Account, Tenant
|
||||
from models.model import EndUser
|
||||
from services.agent_file_request_service import AgentFileDownloadRequestService, FileDownloadRequestError
|
||||
|
||||
|
||||
@inner_api_ns.route("/invoke/llm")
|
||||
@ -429,6 +431,36 @@ class PluginUploadFileRequestApi(Resource):
|
||||
return BaseBackwardsInvocationResponse(data={"url": url}).model_dump()
|
||||
|
||||
|
||||
@inner_api_ns.route("/download/file/request")
|
||||
class PluginDownloadFileRequestApi(Resource):
|
||||
@get_user_tenant
|
||||
@setup_required
|
||||
@plugin_inner_api_only
|
||||
@plugin_data(payload_type=RequestRequestDownloadFile)
|
||||
@inner_api_ns.doc("plugin_download_file_request")
|
||||
@inner_api_ns.doc(description="Request a signed download URL for a workflow file ref")
|
||||
@inner_api_ns.doc(
|
||||
responses={
|
||||
200: "Signed download URL generated successfully",
|
||||
400: "Invalid access context or file mapping",
|
||||
401: "Unauthorized - invalid API key",
|
||||
404: "File not accessible to the tenant/user",
|
||||
}
|
||||
)
|
||||
def post(self, user_model: Account | EndUser, tenant_model: Tenant, payload: RequestRequestDownloadFile):
|
||||
try:
|
||||
data = AgentFileDownloadRequestService.resolve(
|
||||
tenant_id=tenant_model.id,
|
||||
user_id=user_model.id,
|
||||
user_from=payload.user_from,
|
||||
invoke_from=payload.invoke_from,
|
||||
file_mapping=payload.file,
|
||||
)
|
||||
except FileDownloadRequestError as exc:
|
||||
return BaseBackwardsInvocationResponse(error=exc.message).model_dump(), exc.status_code
|
||||
return BaseBackwardsInvocationResponse(data=data).model_dump()
|
||||
|
||||
|
||||
@inner_api_ns.route("/fetch/app/info")
|
||||
class PluginFetchAppInfoApi(Resource):
|
||||
@get_user_tenant
|
||||
|
||||
@ -37,6 +37,8 @@ from controllers.openapi._models import (
|
||||
DeviceMutateRequest,
|
||||
DeviceMutateResponse,
|
||||
DevicePollRequest,
|
||||
FormSubmitResponse,
|
||||
HealthResponse,
|
||||
MemberActionResponse,
|
||||
MemberInvitePayload,
|
||||
MemberInviteResponse,
|
||||
@ -49,9 +51,11 @@ from controllers.openapi._models import (
|
||||
PermittedExternalAppsListResponse,
|
||||
RevokeResponse,
|
||||
ServerVersionResponse,
|
||||
SessionListQuery,
|
||||
SessionListResponse,
|
||||
SessionRow,
|
||||
TagItem,
|
||||
TaskStopResponse,
|
||||
UsageInfo,
|
||||
WorkflowRunData,
|
||||
WorkspaceDetailResponse,
|
||||
@ -74,6 +78,7 @@ register_schema_models(
|
||||
MemberListQuery,
|
||||
MemberRoleUpdatePayload,
|
||||
PermittedExternalAppsListQuery,
|
||||
SessionListQuery,
|
||||
)
|
||||
register_response_schema_models(
|
||||
openapi_ns,
|
||||
@ -100,11 +105,14 @@ register_response_schema_models(
|
||||
MemberListResponse,
|
||||
MemberInviteResponse,
|
||||
MemberActionResponse,
|
||||
TaskStopResponse,
|
||||
FormSubmitResponse,
|
||||
DeviceCodeResponse,
|
||||
DeviceLookupResponse,
|
||||
DeviceMutateResponse,
|
||||
FileResponse,
|
||||
ServerVersionResponse,
|
||||
HealthResponse,
|
||||
)
|
||||
|
||||
from . import (
|
||||
|
||||
@ -6,7 +6,7 @@ from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
from libs.helper import EmailStr, UUIDStrOrEmpty, uuid_value
|
||||
from libs.helper import EmailStr, UUIDStr, UUIDStrOrEmpty, uuid_value
|
||||
from models.model import AppMode
|
||||
|
||||
# Server-side cap on `limit` query param for /openapi/v1/* list endpoints.
|
||||
@ -87,8 +87,12 @@ class AppDescribeInfo(AppInfoResponse):
|
||||
|
||||
class AppDescribeResponse(BaseModel):
|
||||
info: AppDescribeInfo | None = None
|
||||
parameters: dict[str, Any] | None = None
|
||||
input_schema: dict[str, Any] | None = None
|
||||
# `parameters` (the app-config blob) and `input_schema` (a Draft 2020-12 JSON Schema derived
|
||||
# per-app) are deliberately open JSON, not under-annotated. The `x-dify-opaque` marker tells the
|
||||
# contract generator's readiness detector to treat them as intentional, so the route is not
|
||||
# flagged "annotations incomplete". CLI/web consume them as opaque objects either way.
|
||||
parameters: dict[str, Any] | None = Field(default=None, json_schema_extra={"x-dify-opaque": True})
|
||||
input_schema: dict[str, Any] | None = Field(default=None, json_schema_extra={"x-dify-opaque": True})
|
||||
|
||||
|
||||
class ChatMessageResponse(BaseModel):
|
||||
@ -173,6 +177,15 @@ class SessionListResponse(BaseModel):
|
||||
data: list[SessionRow]
|
||||
|
||||
|
||||
class SessionListQuery(BaseModel):
|
||||
"""Pagination for GET /account/sessions. Strict (extra='forbid')."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
page: int = Field(1, ge=1)
|
||||
limit: int = Field(100, ge=1, le=MAX_PAGE_LIMIT)
|
||||
|
||||
|
||||
class RevokeResponse(BaseModel):
|
||||
status: str
|
||||
|
||||
@ -223,6 +236,23 @@ class ServerVersionResponse(BaseModel):
|
||||
edition: Literal["SELF_HOSTED", "CLOUD"]
|
||||
|
||||
|
||||
class HealthResponse(BaseModel):
|
||||
"""Liveness payload for `GET /openapi/v1/_health` — no auth required."""
|
||||
|
||||
ok: bool
|
||||
|
||||
|
||||
def _csv_string_query_schema(schema: dict[str, Any]) -> None:
|
||||
"""Re-shape a set/list field's query schema to a comma-separated string — the wire form the
|
||||
handler actually accepts (`request.args` is flat + the validator splits on ','). Without this
|
||||
the generated contract would type it as an array and serialize `fields[0]=…&fields[1]=…`,
|
||||
which `extra='forbid'` rejects. Runtime `set[str]` validation is unaffected."""
|
||||
schema.pop("anyOf", None)
|
||||
schema.pop("items", None)
|
||||
schema.pop("uniqueItems", None)
|
||||
schema["type"] = "string"
|
||||
|
||||
|
||||
class AppDescribeQuery(BaseModel):
|
||||
"""`?fields=` allow-list for GET /apps/<id>/describe.
|
||||
|
||||
@ -231,23 +261,7 @@ class AppDescribeQuery(BaseModel):
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
fields: set[str] | None = None
|
||||
workspace_id: str | None = None
|
||||
|
||||
@field_validator("workspace_id", mode="before")
|
||||
@classmethod
|
||||
def _validate_workspace_id(cls, v: object) -> str | None:
|
||||
if v is None or v == "":
|
||||
return None
|
||||
if not isinstance(v, str):
|
||||
raise ValueError("workspace_id must be a string")
|
||||
try:
|
||||
import uuid as _uuid
|
||||
|
||||
_uuid.UUID(v)
|
||||
except ValueError:
|
||||
raise ValueError("workspace_id must be a valid UUID")
|
||||
return v
|
||||
fields: set[str] | None = Field(default=None, json_schema_extra=_csv_string_query_schema)
|
||||
|
||||
@field_validator("fields", mode="before")
|
||||
@classmethod
|
||||
@ -267,7 +281,7 @@ class AppDescribeQuery(BaseModel):
|
||||
class AppListQuery(BaseModel):
|
||||
"""mode is a closed enum."""
|
||||
|
||||
workspace_id: str
|
||||
workspace_id: UUIDStr
|
||||
page: int = Field(1, ge=1)
|
||||
limit: int = Field(20, ge=1, le=MAX_PAGE_LIMIT)
|
||||
mode: AppMode | None = None
|
||||
@ -400,3 +414,19 @@ class MemberInviteResponse(BaseModel):
|
||||
|
||||
class MemberActionResponse(BaseModel):
|
||||
result: Literal["success"] = "success"
|
||||
|
||||
|
||||
class TaskStopResponse(BaseModel):
|
||||
"""200 body for POST /apps/<id>/tasks/<task_id>/stop. The handler always returns
|
||||
{"result": "success"}, so `result` is required (no default) — the generated contract
|
||||
types it as a required `'success'` rather than an optional field."""
|
||||
|
||||
result: Literal["success"]
|
||||
|
||||
|
||||
class FormSubmitResponse(BaseModel):
|
||||
"""Empty 200 body for POST /apps/<id>/form/human_input/<token>. `extra='forbid'`
|
||||
pins `additionalProperties: false` so the generated contract is an exact `{}` rather
|
||||
than an under-annotated open object."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
@ -4,15 +4,17 @@ from datetime import UTC, datetime
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from werkzeug.exceptions import NotFound
|
||||
from pydantic import ValidationError
|
||||
from werkzeug.exceptions import NotFound, UnprocessableEntity
|
||||
|
||||
from controllers.common.schema import query_params_from_model
|
||||
from controllers.openapi import openapi_ns
|
||||
from controllers.openapi._models import (
|
||||
MAX_PAGE_LIMIT,
|
||||
AccountPayload,
|
||||
AccountResponse,
|
||||
PaginationEnvelope,
|
||||
RevokeResponse,
|
||||
SessionListQuery,
|
||||
SessionListResponse,
|
||||
SessionRow,
|
||||
WorkspacePayload,
|
||||
@ -70,13 +72,21 @@ class AccountSessionsSelfApi(Resource):
|
||||
|
||||
@openapi_ns.route("/account/sessions")
|
||||
class AccountSessionsApi(Resource):
|
||||
@openapi_ns.doc(params=query_params_from_model(SessionListQuery))
|
||||
@openapi_ns.response(200, "Session list", openapi_ns.models[SessionListResponse.__name__])
|
||||
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
|
||||
def get(self, *, auth_data: AuthData):
|
||||
# Validate page/limit through the same model the contract advertises (extra='forbid',
|
||||
# page>=1, 1<=limit<=MAX_PAGE_LIMIT) so the server actually enforces those bounds rather
|
||||
# than silently coercing (e.g. page=0 -> empty slice). Mirrors AppDescribeQuery.
|
||||
try:
|
||||
query = SessionListQuery.model_validate(request.args.to_dict(flat=True))
|
||||
except ValidationError as exc:
|
||||
raise UnprocessableEntity(exc.json())
|
||||
ctx = get_auth_ctx()
|
||||
now = datetime.now(UTC)
|
||||
page = int(request.args.get("page", "1"))
|
||||
limit = min(int(request.args.get("limit", "100")), MAX_PAGE_LIMIT)
|
||||
page = query.page
|
||||
limit = query.limit
|
||||
|
||||
all_rows = list_active_sessions(db.session, ctx, now)
|
||||
|
||||
|
||||
@ -15,7 +15,7 @@ from werkzeug.exceptions import BadRequest, HTTPException, InternalServerError,
|
||||
import services
|
||||
from controllers.openapi import openapi_ns
|
||||
from controllers.openapi._audit import emit_app_run
|
||||
from controllers.openapi._models import AppRunRequest
|
||||
from controllers.openapi._models import AppRunRequest, TaskStopResponse
|
||||
from controllers.openapi.auth.composition import auth_router
|
||||
from controllers.openapi.auth.data import AuthData
|
||||
from controllers.service_api.app.error import (
|
||||
@ -159,7 +159,7 @@ class AppRunApi(Resource):
|
||||
|
||||
@openapi_ns.route("/apps/<string:app_id>/tasks/<string:task_id>/stop")
|
||||
class AppRunTaskStopApi(Resource):
|
||||
@openapi_ns.response(200, "Task stopped")
|
||||
@openapi_ns.response(200, "Task stopped", openapi_ns.models[TaskStopResponse.__name__])
|
||||
@auth_router.guard(scope=Scope.APPS_RUN)
|
||||
def post(self, app_id: str, task_id: str, *, auth_data: AuthData):
|
||||
app_model, caller, caller_kind = auth_data.require_app_context()
|
||||
|
||||
@ -97,7 +97,7 @@ class AppDescribeApi(AppReadResource):
|
||||
except ValidationError as exc:
|
||||
raise UnprocessableEntity(exc.json())
|
||||
|
||||
app = self._load(app_id, workspace_id=query.workspace_id)
|
||||
app = self._load(app_id)
|
||||
|
||||
requested = query.fields
|
||||
want_info = requested is None or "info" in requested
|
||||
|
||||
@ -19,6 +19,10 @@ def load_app(data: AuthData) -> None:
|
||||
if data.app is not None:
|
||||
return
|
||||
app_id = data.path_params["app_id"]
|
||||
try:
|
||||
uuid.UUID(app_id)
|
||||
except ValueError:
|
||||
raise NotFound("app not found")
|
||||
app = AppService.get_app_by_id(db.session, app_id)
|
||||
if not app or app.status != "normal":
|
||||
raise NotFound("app not found")
|
||||
|
||||
@ -17,6 +17,7 @@ from werkzeug.exceptions import BadRequest, NotFound
|
||||
from controllers.common.human_input import HumanInputFormSubmitPayload, stringify_form_default_values
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.openapi import openapi_ns
|
||||
from controllers.openapi._models import FormSubmitResponse
|
||||
from controllers.openapi.auth.composition import auth_router
|
||||
from controllers.openapi.auth.data import AuthData
|
||||
from core.workflow.human_input_policy import HumanInputSurface, is_recipient_type_allowed_for_surface
|
||||
@ -70,7 +71,7 @@ class OpenApiWorkflowHumanInputFormApi(Resource):
|
||||
return _jsonify_form_definition(form)
|
||||
|
||||
@openapi_ns.expect(openapi_ns.models[HumanInputFormSubmitPayload.__name__])
|
||||
@openapi_ns.response(200, "Form submitted")
|
||||
@openapi_ns.response(200, "Form submitted", openapi_ns.models[FormSubmitResponse.__name__])
|
||||
@auth_router.guard(scope=Scope.APPS_RUN)
|
||||
def post(self, app_id: str, form_token: str, *, auth_data: AuthData):
|
||||
app_model, caller, caller_kind = auth_data.require_app_context()
|
||||
|
||||
@ -1,9 +1,11 @@
|
||||
from flask_restx import Resource
|
||||
|
||||
from controllers.openapi import openapi_ns
|
||||
from controllers.openapi._models import HealthResponse
|
||||
|
||||
|
||||
@openapi_ns.route("/_health")
|
||||
class HealthApi(Resource):
|
||||
@openapi_ns.response(200, "Health check", openapi_ns.models[HealthResponse.__name__])
|
||||
def get(self):
|
||||
return {"ok": True}
|
||||
|
||||
@ -13,7 +13,11 @@ from dataclasses import dataclass
|
||||
from typing import Any, Protocol, cast
|
||||
|
||||
from agenton.compositor import CompositorSessionSnapshot
|
||||
from dify_agent.layers.execution_context import DifyExecutionContextLayerConfig
|
||||
from dify_agent.layers.execution_context import (
|
||||
DifyExecutionContextInvokeFrom,
|
||||
DifyExecutionContextLayerConfig,
|
||||
DifyExecutionContextUserFrom,
|
||||
)
|
||||
from dify_agent.protocol import CreateRunRequest
|
||||
|
||||
from clients.agent_backend import (
|
||||
@ -126,7 +130,10 @@ class AgentAppRuntimeRequestBuilder:
|
||||
conversation_id=context.conversation_id,
|
||||
agent_id=context.agent_id,
|
||||
agent_config_version_id=context.agent_config_snapshot_id,
|
||||
invoke_from="agent_app",
|
||||
# Agent Files §1.3: real Dify access context + agent run mode.
|
||||
user_from=cast(DifyExecutionContextUserFrom, context.dify_context.user_from.value),
|
||||
invoke_from=cast(DifyExecutionContextInvokeFrom, context.dify_context.invoke_from.value),
|
||||
agent_mode="agent_app",
|
||||
),
|
||||
agent_soul_prompt=agent_soul.prompt.system_prompt or None,
|
||||
user_prompt=context.user_query,
|
||||
|
||||
@ -37,6 +37,13 @@ class MCPSupportGrantType(StrEnum):
|
||||
REFRESH_TOKEN = "refresh_token"
|
||||
|
||||
|
||||
class IdentityMode(StrEnum):
|
||||
"""How Dify forwards the end-user's identity to an MCP server."""
|
||||
|
||||
OFF = "off"
|
||||
IDP_TOKEN = "idp_token"
|
||||
|
||||
|
||||
class MCPAuthentication(BaseModel):
|
||||
client_id: str
|
||||
client_secret: str | None = None
|
||||
@ -76,6 +83,8 @@ class MCPProviderEntity(BaseModel):
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
identity_mode: IdentityMode = IdentityMode.OFF
|
||||
|
||||
@classmethod
|
||||
def from_db_model(cls, db_provider: MCPToolProvider) -> MCPProviderEntity:
|
||||
"""Create entity from database model with decryption"""
|
||||
@ -96,6 +105,7 @@ class MCPProviderEntity(BaseModel):
|
||||
icon=db_provider.icon or "",
|
||||
created_at=db_provider.created_at,
|
||||
updated_at=db_provider.updated_at,
|
||||
identity_mode=IdentityMode(db_provider.identity_mode),
|
||||
)
|
||||
|
||||
@property
|
||||
@ -170,6 +180,7 @@ class MCPProviderEntity(BaseModel):
|
||||
"updated_at": int(self.updated_at.timestamp()),
|
||||
"label": I18nObject(en_US=self.name, zh_Hans=self.name).to_dict(),
|
||||
"description": I18nObject(en_US="", zh_Hans="").to_dict(),
|
||||
"identity_mode": self.identity_mode,
|
||||
}
|
||||
|
||||
# Add configuration
|
||||
|
||||
@ -316,6 +316,7 @@ class IndexingRunner:
|
||||
qa_preview_texts: list[QAPreviewDetail] = []
|
||||
|
||||
total_segments = 0
|
||||
deleted_preview_images = False
|
||||
# doc_form represents the segmentation method (general, parent-child, QA)
|
||||
index_type = doc_form
|
||||
index_processor = IndexProcessorFactory(index_type).init_index_processor()
|
||||
@ -368,6 +369,10 @@ class IndexingRunner:
|
||||
upload_file_id,
|
||||
)
|
||||
db.session.delete(image_file)
|
||||
deleted_preview_images = True
|
||||
|
||||
if deleted_preview_images:
|
||||
db.session.commit()
|
||||
|
||||
if doc_form and doc_form == "qa_model":
|
||||
return IndexingEstimate(total_segments=total_segments * 20, qa_preview=qa_preview_texts, preview=[])
|
||||
|
||||
@ -40,6 +40,7 @@ class MCPClientWithAuthRetry(MCPClient):
|
||||
provider_entity: MCPProviderEntity | None = None,
|
||||
authorization_code: str | None = None,
|
||||
by_server_id: bool = False,
|
||||
forward_identity_active: bool = False,
|
||||
):
|
||||
"""
|
||||
Initialize the MCP client with auth retry capability.
|
||||
@ -52,12 +53,15 @@ class MCPClientWithAuthRetry(MCPClient):
|
||||
provider_entity: Provider entity for authentication
|
||||
authorization_code: Optional authorization code for initial auth
|
||||
by_server_id: Whether to look up provider by server ID
|
||||
forward_identity_active: If True, suppress the static-OAuth retry
|
||||
on 401 — the forwarded identity must propagate as-is.
|
||||
"""
|
||||
super().__init__(server_url, headers, timeout, sse_read_timeout)
|
||||
|
||||
self.provider_entity = provider_entity
|
||||
self.authorization_code = authorization_code
|
||||
self.by_server_id = by_server_id
|
||||
self.forward_identity_active = forward_identity_active
|
||||
self._has_retried = False
|
||||
|
||||
def _handle_auth_error(self, error: MCPAuthError) -> None:
|
||||
@ -73,6 +77,8 @@ class MCPClientWithAuthRetry(MCPClient):
|
||||
Raises:
|
||||
MCPAuthError: If authentication fails or max retries reached
|
||||
"""
|
||||
if self.forward_identity_active:
|
||||
raise error
|
||||
if not self.provider_entity:
|
||||
raise error
|
||||
if self._has_retried:
|
||||
|
||||
@ -7,7 +7,7 @@ import threading
|
||||
import time
|
||||
from collections.abc import Mapping
|
||||
from datetime import timedelta
|
||||
from typing import TYPE_CHECKING, Any, TypedDict
|
||||
from typing import TYPE_CHECKING, Any, TypedDict, override
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
from cachetools import LRUCache
|
||||
@ -221,6 +221,7 @@ class TracingProviderConfigEntry(TypedDict):
|
||||
|
||||
|
||||
class OpsTraceProviderConfigMap(collections.UserDict[str, TracingProviderConfigEntry]):
|
||||
@override
|
||||
def __getitem__(self, key: str) -> TracingProviderConfigEntry:
|
||||
try:
|
||||
match key:
|
||||
|
||||
@ -231,6 +231,20 @@ class RequestRequestUploadFile(BaseModel):
|
||||
mimetype: str
|
||||
|
||||
|
||||
class RequestRequestDownloadFile(BaseModel):
|
||||
"""Request a signed download URL for a workflow file ref (Agent Files §3.1.1).
|
||||
|
||||
``user_from`` / ``invoke_from`` are the flattened Dify file-access context (the
|
||||
dify-agent server reads them from the execution context). ``file`` is a standard
|
||||
file mapping: ``transfer_method`` plus ``reference`` (local_file / tool_file /
|
||||
datasource_file) or ``url`` (remote_url).
|
||||
"""
|
||||
|
||||
user_from: str
|
||||
invoke_from: str
|
||||
file: Mapping[str, Any]
|
||||
|
||||
|
||||
class RequestFetchAppInfo(BaseModel):
|
||||
"""
|
||||
Request to fetch app info
|
||||
|
||||
@ -862,15 +862,20 @@ class RetrievalService:
|
||||
str(dataset.tenant_id), reranking_mode, reranking_model, weights, False
|
||||
)
|
||||
|
||||
query = query or attachment_id
|
||||
if not query:
|
||||
if query:
|
||||
rerank_query = query
|
||||
query_type = QueryType.TEXT_QUERY
|
||||
elif attachment_id:
|
||||
rerank_query = attachment_id
|
||||
query_type = QueryType.IMAGE_QUERY
|
||||
else:
|
||||
return
|
||||
all_documents_item = data_post_processor.invoke(
|
||||
query=query,
|
||||
query=rerank_query,
|
||||
documents=all_documents_item,
|
||||
score_threshold=score_threshold,
|
||||
top_n=top_k,
|
||||
query_type=QueryType.TEXT_QUERY if query else QueryType.IMAGE_QUERY,
|
||||
query_type=query_type,
|
||||
)
|
||||
if not data_post_processor.rerank_runner and score_threshold:
|
||||
all_documents_item = self._filter_documents_by_vector_score_threshold(
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
import base64
|
||||
import logging
|
||||
import pickle
|
||||
from typing import Any, cast
|
||||
from typing import Any, cast, override
|
||||
|
||||
import numpy as np
|
||||
from sqlalchemy import select
|
||||
@ -25,6 +25,7 @@ class CacheEmbedding(Embeddings):
|
||||
def __init__(self, model_instance: ModelInstance):
|
||||
self._model_instance = model_instance
|
||||
|
||||
@override
|
||||
def embed_documents(self, texts: list[str]) -> list[list[float]]:
|
||||
"""Embed search docs in batches of 10."""
|
||||
# use doc embedding cache or store if not exists
|
||||
@ -106,6 +107,7 @@ class CacheEmbedding(Embeddings):
|
||||
|
||||
return text_embeddings
|
||||
|
||||
@override
|
||||
def embed_multimodal_documents(self, multimodel_documents: list[dict[str, Any]]) -> list[list[float]]:
|
||||
"""Embed file documents."""
|
||||
# use doc embedding cache or store if not exists
|
||||
@ -189,6 +191,7 @@ class CacheEmbedding(Embeddings):
|
||||
|
||||
return multimodel_embeddings
|
||||
|
||||
@override
|
||||
def embed_query(self, text: str) -> list[float]:
|
||||
"""Embed query text."""
|
||||
# use doc embedding cache or store if not exists
|
||||
@ -232,6 +235,7 @@ class CacheEmbedding(Embeddings):
|
||||
|
||||
return embedding_results # type: ignore
|
||||
|
||||
@override
|
||||
def embed_multimodal_query(self, multimodel_document: dict[str, Any]) -> list[float]:
|
||||
"""Embed multimodal documents."""
|
||||
# use doc embedding cache or store if not exists
|
||||
|
||||
@ -1,13 +1,32 @@
|
||||
"""Abstract interface for document loader implementations."""
|
||||
"""Excel document extractor used for RAG ingestion.
|
||||
|
||||
Supports cell hyperlinks for both `.xls` and `.xlsx`, and embedded worksheet images
|
||||
for `.xlsx` files by converting them into markdown image links. Embedded images are
|
||||
stored with deterministic keys derived from the source upload file and anchor cell so
|
||||
retries can safely reuse the same assets.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import mimetypes
|
||||
import os
|
||||
from typing import TypedDict, override
|
||||
|
||||
import pandas as pd
|
||||
from openpyxl import load_workbook
|
||||
from sqlalchemy import select
|
||||
|
||||
from configs import dify_config
|
||||
from core.db.session_factory import session_factory
|
||||
from core.rag.extractor.extractor_base import BaseExtractor
|
||||
from core.rag.models.document import Document
|
||||
from extensions.ext_storage import storage
|
||||
from extensions.storage.storage_type import StorageType
|
||||
from libs.datetime_utils import naive_utc_now
|
||||
from models.enums import CreatorUserRole
|
||||
from models.model import UploadFile
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Candidate(TypedDict):
|
||||
@ -16,17 +35,42 @@ class Candidate(TypedDict):
|
||||
map: dict[int, str]
|
||||
|
||||
|
||||
class SheetImageCandidate(TypedDict):
|
||||
anchor: tuple[int, int]
|
||||
content_hash: str
|
||||
file_key: str
|
||||
image_bytes: bytes
|
||||
image_ext: str
|
||||
|
||||
|
||||
class ExcelExtractor(BaseExtractor):
|
||||
"""Load Excel files.
|
||||
|
||||
|
||||
Args:
|
||||
file_path: Path to the file to load.
|
||||
"""
|
||||
|
||||
def __init__(self, file_path: str, encoding: str | None = None, autodetect_encoding: bool = False):
|
||||
_file_path: str
|
||||
_encoding: str | None
|
||||
_autodetect_encoding: bool
|
||||
_tenant_id: str | None
|
||||
_user_id: str | None
|
||||
_source_file_id: str | None
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
file_path: str,
|
||||
tenant_id: str | None = None,
|
||||
user_id: str | None = None,
|
||||
source_file_id: str | None = None,
|
||||
encoding: str | None = None,
|
||||
autodetect_encoding: bool = False,
|
||||
):
|
||||
"""Initialize with file path."""
|
||||
self._file_path = file_path
|
||||
self._tenant_id = tenant_id
|
||||
self._user_id = user_id
|
||||
self._source_file_id = source_file_id
|
||||
self._encoding = encoding
|
||||
self._autodetect_encoding = autodetect_encoding
|
||||
|
||||
@ -37,7 +81,8 @@ class ExcelExtractor(BaseExtractor):
|
||||
file_extension = os.path.splitext(self._file_path)[-1].lower()
|
||||
|
||||
if file_extension == ".xlsx":
|
||||
wb = load_workbook(self._file_path, read_only=True, data_only=True)
|
||||
# Worksheet drawing objects, including embedded images, are not available in read-only mode.
|
||||
wb = load_workbook(self._file_path, data_only=True)
|
||||
try:
|
||||
for sheet_name in wb.sheetnames:
|
||||
sheet = wb[sheet_name]
|
||||
@ -45,10 +90,15 @@ class ExcelExtractor(BaseExtractor):
|
||||
if not column_map:
|
||||
continue
|
||||
start_row = header_row_idx + 1
|
||||
sheet_image_map = self._extract_images_from_sheet(
|
||||
sheet_name=sheet_name,
|
||||
sheet=sheet,
|
||||
valid_columns={column_idx + 1 for column_idx in column_map},
|
||||
min_row=start_row,
|
||||
)
|
||||
for row in sheet.iter_rows(min_row=start_row, max_col=max_col_idx, values_only=False):
|
||||
if all(cell.value is None for cell in row):
|
||||
continue
|
||||
page_content = []
|
||||
row_has_content = False
|
||||
for col_idx, cell in enumerate(row):
|
||||
value = cell.value
|
||||
if col_idx in column_map:
|
||||
@ -56,14 +106,27 @@ class ExcelExtractor(BaseExtractor):
|
||||
if hasattr(cell, "hyperlink") and cell.hyperlink:
|
||||
target = getattr(cell.hyperlink, "target", None)
|
||||
if target:
|
||||
value = f"[{value}]({target})"
|
||||
display_value = value if value is not None and str(value).strip() else target
|
||||
value = f"[{display_value}]({target})"
|
||||
cell_row = getattr(cell, "row", None)
|
||||
cell_column = getattr(cell, "column", None)
|
||||
image_links = (
|
||||
sheet_image_map.get((cell_row, cell_column), [])
|
||||
if isinstance(cell_row, int) and isinstance(cell_column, int)
|
||||
else []
|
||||
)
|
||||
if value is None:
|
||||
value = ""
|
||||
elif not isinstance(value, str):
|
||||
value = str(value)
|
||||
value = value.strip().replace('"', '\\"')
|
||||
if image_links:
|
||||
value = " ".join(filter(None, [value, " ".join(image_links)]))
|
||||
value = value.strip()
|
||||
if value:
|
||||
row_has_content = True
|
||||
value = value.replace('"', '\\"')
|
||||
page_content.append(f'"{col_name}":"{value}"')
|
||||
if page_content:
|
||||
if row_has_content and page_content:
|
||||
documents.append(
|
||||
Document(page_content=";".join(page_content), metadata={"source": self._file_path})
|
||||
)
|
||||
@ -89,6 +152,166 @@ class ExcelExtractor(BaseExtractor):
|
||||
|
||||
return documents
|
||||
|
||||
def _extract_images_from_sheet(
|
||||
self, sheet_name: str, sheet, valid_columns: set[int], min_row: int
|
||||
) -> dict[tuple[int, int], list[str]]:
|
||||
"""
|
||||
Extract embedded worksheet images and map them to their anchor cell.
|
||||
|
||||
Images are stored with deterministic keys derived from the source upload file,
|
||||
sheet, anchor cell, and content hash so retried tasks can reuse the same
|
||||
UploadFile rows and storage objects.
|
||||
"""
|
||||
if not self._tenant_id or not self._user_id or not self._source_file_id:
|
||||
return {}
|
||||
|
||||
images = getattr(sheet, "_images", None) or []
|
||||
image_candidates: list[SheetImageCandidate] = []
|
||||
|
||||
for image in images:
|
||||
marker = getattr(getattr(image, "anchor", None), "_from", None)
|
||||
row_idx = getattr(marker, "row", None)
|
||||
col_idx = getattr(marker, "col", None)
|
||||
if row_idx is None or col_idx is None:
|
||||
continue
|
||||
if row_idx + 1 < min_row or col_idx + 1 not in valid_columns:
|
||||
continue
|
||||
|
||||
image_bytes = self._get_image_bytes(image)
|
||||
if not image_bytes:
|
||||
continue
|
||||
|
||||
image_ext = self._get_image_extension(image)
|
||||
if not image_ext:
|
||||
continue
|
||||
|
||||
anchor_row = row_idx + 1
|
||||
anchor_column = col_idx + 1
|
||||
content_hash = self._hash_image_bytes(image_bytes)
|
||||
image_candidates.append(
|
||||
{
|
||||
"anchor": (anchor_row, anchor_column),
|
||||
"content_hash": content_hash,
|
||||
"file_key": self._build_image_file_key(
|
||||
sheet_name=sheet_name,
|
||||
anchor_row=anchor_row,
|
||||
anchor_column=anchor_column,
|
||||
content_hash=content_hash,
|
||||
image_ext=image_ext,
|
||||
),
|
||||
"image_bytes": image_bytes,
|
||||
"image_ext": image_ext,
|
||||
}
|
||||
)
|
||||
|
||||
if not image_candidates:
|
||||
return {}
|
||||
|
||||
image_map: dict[tuple[int, int], list[str]] = {}
|
||||
base_url = dify_config.FILES_URL
|
||||
candidate_keys = sorted({candidate["file_key"] for candidate in image_candidates})
|
||||
|
||||
with session_factory.create_session() as session:
|
||||
existing_upload_files = session.scalars(
|
||||
select(UploadFile).where(
|
||||
UploadFile.tenant_id == self._tenant_id,
|
||||
UploadFile.key.in_(candidate_keys),
|
||||
)
|
||||
).all()
|
||||
upload_files_by_key = {upload_file.key: upload_file for upload_file in existing_upload_files}
|
||||
new_upload_files: list[UploadFile] = []
|
||||
|
||||
for candidate in image_candidates:
|
||||
upload_file = upload_files_by_key.get(candidate["file_key"])
|
||||
if upload_file is None:
|
||||
storage.save(candidate["file_key"], candidate["image_bytes"])
|
||||
mime_type, _ = mimetypes.guess_type(candidate["file_key"])
|
||||
upload_file = UploadFile(
|
||||
tenant_id=self._tenant_id,
|
||||
storage_type=StorageType(dify_config.STORAGE_TYPE),
|
||||
key=candidate["file_key"],
|
||||
name=candidate["file_key"],
|
||||
size=len(candidate["image_bytes"]),
|
||||
extension=candidate["image_ext"],
|
||||
mime_type=mime_type or "",
|
||||
created_by=self._user_id,
|
||||
created_by_role=CreatorUserRole.ACCOUNT,
|
||||
created_at=naive_utc_now(),
|
||||
used=True,
|
||||
used_by=self._user_id,
|
||||
used_at=naive_utc_now(),
|
||||
hash=candidate["content_hash"],
|
||||
)
|
||||
upload_files_by_key[candidate["file_key"]] = upload_file
|
||||
new_upload_files.append(upload_file)
|
||||
|
||||
image_map.setdefault(candidate["anchor"], []).append(
|
||||
f""
|
||||
)
|
||||
|
||||
if new_upload_files:
|
||||
session.add_all(new_upload_files)
|
||||
session.commit()
|
||||
|
||||
return image_map
|
||||
|
||||
@staticmethod
|
||||
def _hash_image_bytes(image_bytes: bytes) -> str:
|
||||
"""Return a stable content hash for extracted image bytes."""
|
||||
return hashlib.sha256(image_bytes).hexdigest()
|
||||
|
||||
def _build_image_file_key(
|
||||
self,
|
||||
*,
|
||||
sheet_name: str,
|
||||
anchor_row: int,
|
||||
anchor_column: int,
|
||||
content_hash: str,
|
||||
image_ext: str,
|
||||
) -> str:
|
||||
"""Build a deterministic storage key for an embedded worksheet image."""
|
||||
assert self._tenant_id is not None, "tenant_id is required for image extraction"
|
||||
assert self._source_file_id is not None, "source_file_id is required for image extraction"
|
||||
|
||||
normalized_ext = image_ext.strip().lower()
|
||||
sheet_hash = hashlib.sha256(sheet_name.encode("utf-8")).hexdigest()[:16]
|
||||
return (
|
||||
f"image_files/{self._tenant_id}/{self._source_file_id}/"
|
||||
f"{sheet_hash}_r{anchor_row}_c{anchor_column}_{content_hash}.{normalized_ext}"
|
||||
)
|
||||
|
||||
def _get_image_bytes(self, image) -> bytes | None:
|
||||
"""Return embedded image bytes from an openpyxl image object."""
|
||||
data_loader = getattr(image, "_data", None)
|
||||
if not callable(data_loader):
|
||||
return None
|
||||
|
||||
try:
|
||||
data = data_loader()
|
||||
if isinstance(data, bytes):
|
||||
return data
|
||||
if isinstance(data, bytearray):
|
||||
return bytes(data)
|
||||
logger.warning("Unexpected embedded image payload type: %s", type(data).__name__)
|
||||
return None
|
||||
except Exception:
|
||||
logger.warning("Failed to read embedded image bytes from Excel sheet", exc_info=True)
|
||||
return None
|
||||
|
||||
def _get_image_extension(self, image) -> str | None:
|
||||
"""Resolve an image extension from openpyxl metadata."""
|
||||
image_format = getattr(image, "format", None)
|
||||
if isinstance(image_format, str) and image_format.strip():
|
||||
return image_format.strip().lower()
|
||||
|
||||
image_path = getattr(image, "path", None)
|
||||
if isinstance(image_path, str):
|
||||
_, extension = os.path.splitext(image_path)
|
||||
if extension:
|
||||
return extension.lstrip(".").lower()
|
||||
|
||||
return None
|
||||
|
||||
def _find_header_and_columns(self, sheet, scan_rows=10) -> tuple[int, dict[int, str], int]:
|
||||
"""
|
||||
Scan first N rows to find the most likely header row.
|
||||
|
||||
@ -113,7 +113,12 @@ class ExtractProcessor:
|
||||
unstructured_api_key = dify_config.UNSTRUCTURED_API_KEY or ""
|
||||
|
||||
if file_extension in {".xlsx", ".xls"}:
|
||||
extractor = ExcelExtractor(file_path)
|
||||
extractor = ExcelExtractor(
|
||||
file_path,
|
||||
upload_file.tenant_id,
|
||||
upload_file.created_by,
|
||||
upload_file.id,
|
||||
)
|
||||
elif file_extension == ".pdf":
|
||||
assert upload_file is not None
|
||||
extractor = PdfExtractor(file_path, upload_file.tenant_id, upload_file.created_by)
|
||||
@ -151,7 +156,12 @@ class ExtractProcessor:
|
||||
extractor = TextExtractor(file_path, autodetect_encoding=True)
|
||||
else:
|
||||
if file_extension in {".xlsx", ".xls"}:
|
||||
extractor = ExcelExtractor(file_path)
|
||||
extractor = ExcelExtractor(
|
||||
file_path,
|
||||
upload_file.tenant_id,
|
||||
upload_file.created_by,
|
||||
upload_file.id,
|
||||
)
|
||||
elif file_extension == ".pdf":
|
||||
assert upload_file is not None
|
||||
extractor = PdfExtractor(file_path, upload_file.tenant_id, upload_file.created_by)
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
import logging
|
||||
import re
|
||||
import uuid
|
||||
from typing import Any, TypedDict, cast
|
||||
from typing import Any, TypedDict, cast, override
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@ -61,6 +61,7 @@ class ParagraphFormatPreviewDict(TypedDict):
|
||||
|
||||
|
||||
class ParagraphIndexProcessor(BaseIndexProcessor):
|
||||
@override
|
||||
def extract(self, extract_setting: ExtractSetting, **kwargs) -> list[Document]:
|
||||
text_docs = ExtractProcessor.extract(
|
||||
extract_setting=extract_setting,
|
||||
@ -71,6 +72,7 @@ class ParagraphIndexProcessor(BaseIndexProcessor):
|
||||
|
||||
return text_docs
|
||||
|
||||
@override
|
||||
def transform(self, documents: list[Document], current_user: Account | None = None, **kwargs) -> list[Document]:
|
||||
process_rule = kwargs.get("process_rule")
|
||||
if not process_rule:
|
||||
@ -120,6 +122,7 @@ class ParagraphIndexProcessor(BaseIndexProcessor):
|
||||
all_documents.extend(split_documents)
|
||||
return all_documents
|
||||
|
||||
@override
|
||||
def load(
|
||||
self,
|
||||
dataset: Dataset,
|
||||
@ -142,6 +145,7 @@ class ParagraphIndexProcessor(BaseIndexProcessor):
|
||||
else:
|
||||
keyword.add_texts(documents)
|
||||
|
||||
@override
|
||||
def clean(self, dataset: Dataset, node_ids: list[str] | None, with_keywords: bool = True, **kwargs) -> None:
|
||||
# Note: Summary indexes are now disabled (not deleted) when segments are disabled.
|
||||
# This method is called for actual deletion scenarios (e.g., when segment is deleted).
|
||||
@ -178,6 +182,7 @@ class ParagraphIndexProcessor(BaseIndexProcessor):
|
||||
else:
|
||||
keyword.delete()
|
||||
|
||||
@override
|
||||
def retrieve(
|
||||
self,
|
||||
retrieval_method: RetrievalMethod,
|
||||
@ -206,6 +211,7 @@ class ParagraphIndexProcessor(BaseIndexProcessor):
|
||||
docs.append(doc)
|
||||
return docs
|
||||
|
||||
@override
|
||||
def index(self, dataset: Dataset, document: DatasetDocument, chunks: Any) -> None:
|
||||
documents: list[Any] = []
|
||||
all_multimodal_documents: list[Any] = []
|
||||
@ -271,6 +277,7 @@ class ParagraphIndexProcessor(BaseIndexProcessor):
|
||||
keyword = Keyword(dataset)
|
||||
keyword.add_texts(documents)
|
||||
|
||||
@override
|
||||
def format_preview(self, chunks: Any) -> ParagraphFormatPreviewDict:
|
||||
if isinstance(chunks, list):
|
||||
preview = []
|
||||
@ -285,6 +292,7 @@ class ParagraphIndexProcessor(BaseIndexProcessor):
|
||||
else:
|
||||
raise ValueError("Chunks is not a list")
|
||||
|
||||
@override
|
||||
def generate_summary_preview(
|
||||
self,
|
||||
tenant_id: str,
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
import json
|
||||
import logging
|
||||
import uuid
|
||||
from typing import Any, TypedDict
|
||||
from typing import Any, TypedDict, override
|
||||
|
||||
from sqlalchemy import delete, select
|
||||
|
||||
@ -44,6 +44,7 @@ class ParentChildFormatPreviewDict(TypedDict):
|
||||
|
||||
|
||||
class ParentChildIndexProcessor(BaseIndexProcessor):
|
||||
@override
|
||||
def extract(self, extract_setting: ExtractSetting, **kwargs) -> list[Document]:
|
||||
text_docs = ExtractProcessor.extract(
|
||||
extract_setting=extract_setting,
|
||||
@ -54,6 +55,7 @@ class ParentChildIndexProcessor(BaseIndexProcessor):
|
||||
|
||||
return text_docs
|
||||
|
||||
@override
|
||||
def transform(self, documents: list[Document], current_user: Account | None = None, **kwargs) -> list[Document]:
|
||||
process_rule = kwargs.get("process_rule")
|
||||
if not process_rule:
|
||||
@ -129,6 +131,7 @@ class ParentChildIndexProcessor(BaseIndexProcessor):
|
||||
|
||||
return all_documents
|
||||
|
||||
@override
|
||||
def load(
|
||||
self,
|
||||
dataset: Dataset,
|
||||
@ -149,6 +152,7 @@ class ParentChildIndexProcessor(BaseIndexProcessor):
|
||||
if multimodal_documents and dataset.is_multimodal:
|
||||
vector.create_multimodal(multimodal_documents)
|
||||
|
||||
@override
|
||||
def clean(self, dataset: Dataset, node_ids: list[str] | None, with_keywords: bool = True, **kwargs) -> None:
|
||||
# node_ids is segment's node_ids
|
||||
# Note: Summary indexes are now disabled (not deleted) when segments are disabled.
|
||||
@ -219,6 +223,7 @@ class ParentChildIndexProcessor(BaseIndexProcessor):
|
||||
)
|
||||
db.session.commit()
|
||||
|
||||
@override
|
||||
def retrieve(
|
||||
self,
|
||||
retrieval_method: RetrievalMethod,
|
||||
@ -283,6 +288,7 @@ class ParentChildIndexProcessor(BaseIndexProcessor):
|
||||
child_nodes.append(child_document)
|
||||
return child_nodes
|
||||
|
||||
@override
|
||||
def index(self, dataset: Dataset, document: DatasetDocument, chunks: Any) -> None:
|
||||
parent_childs = ParentChildStructureChunk.model_validate(chunks)
|
||||
documents = []
|
||||
@ -356,6 +362,7 @@ class ParentChildIndexProcessor(BaseIndexProcessor):
|
||||
if all_multimodal_documents and dataset.is_multimodal:
|
||||
vector.create_multimodal(all_multimodal_documents)
|
||||
|
||||
@override
|
||||
def format_preview(self, chunks: Any) -> ParentChildFormatPreviewDict:
|
||||
parent_childs = ParentChildStructureChunk.model_validate(chunks)
|
||||
preview = []
|
||||
@ -369,6 +376,7 @@ class ParentChildIndexProcessor(BaseIndexProcessor):
|
||||
}
|
||||
return result
|
||||
|
||||
@override
|
||||
def generate_summary_preview(
|
||||
self,
|
||||
tenant_id: str,
|
||||
|
||||
@ -4,7 +4,7 @@ import logging
|
||||
import re
|
||||
import threading
|
||||
import uuid
|
||||
from typing import Any, TypedDict
|
||||
from typing import Any, TypedDict, override
|
||||
|
||||
import pandas as pd
|
||||
from flask import Flask, current_app
|
||||
@ -43,6 +43,7 @@ class QAFormatPreviewDict(TypedDict):
|
||||
|
||||
|
||||
class QAIndexProcessor(BaseIndexProcessor):
|
||||
@override
|
||||
def extract(self, extract_setting: ExtractSetting, **kwargs) -> list[Document]:
|
||||
text_docs = ExtractProcessor.extract(
|
||||
extract_setting=extract_setting,
|
||||
@ -52,6 +53,7 @@ class QAIndexProcessor(BaseIndexProcessor):
|
||||
)
|
||||
return text_docs
|
||||
|
||||
@override
|
||||
def transform(self, documents: list[Document], current_user: Account | None = None, **kwargs) -> list[Document]:
|
||||
preview = kwargs.get("preview")
|
||||
process_rule = kwargs.get("process_rule")
|
||||
@ -139,6 +141,7 @@ class QAIndexProcessor(BaseIndexProcessor):
|
||||
raise ValueError(str(e))
|
||||
return text_docs
|
||||
|
||||
@override
|
||||
def load(
|
||||
self,
|
||||
dataset: Dataset,
|
||||
@ -153,6 +156,7 @@ class QAIndexProcessor(BaseIndexProcessor):
|
||||
if multimodal_documents and dataset.is_multimodal:
|
||||
vector.create_multimodal(multimodal_documents)
|
||||
|
||||
@override
|
||||
def clean(self, dataset: Dataset, node_ids: list[str] | None, with_keywords: bool = True, **kwargs) -> None:
|
||||
# Note: Summary indexes are now disabled (not deleted) when segments are disabled.
|
||||
# This method is called for actual deletion scenarios (e.g., when segment is deleted).
|
||||
@ -183,6 +187,7 @@ class QAIndexProcessor(BaseIndexProcessor):
|
||||
else:
|
||||
vector.delete()
|
||||
|
||||
@override
|
||||
def retrieve(
|
||||
self,
|
||||
retrieval_method: RetrievalMethod,
|
||||
@ -211,6 +216,7 @@ class QAIndexProcessor(BaseIndexProcessor):
|
||||
docs.append(doc)
|
||||
return docs
|
||||
|
||||
@override
|
||||
def index(self, dataset: Dataset, document: DatasetDocument, chunks: Any) -> None:
|
||||
qa_chunks = QAStructureChunk.model_validate(chunks)
|
||||
documents = []
|
||||
@ -234,6 +240,7 @@ class QAIndexProcessor(BaseIndexProcessor):
|
||||
else:
|
||||
raise ValueError("Indexing technique must be high quality.")
|
||||
|
||||
@override
|
||||
def format_preview(self, chunks: Any) -> QAFormatPreviewDict:
|
||||
qa_chunks = QAStructureChunk.model_validate(chunks)
|
||||
preview = []
|
||||
@ -246,6 +253,7 @@ class QAIndexProcessor(BaseIndexProcessor):
|
||||
}
|
||||
return result
|
||||
|
||||
@override
|
||||
def generate_summary_preview(
|
||||
self,
|
||||
tenant_id: str,
|
||||
|
||||
@ -1,4 +1,5 @@
|
||||
import base64
|
||||
from typing import override
|
||||
|
||||
from core.model_manager import ModelInstance, ModelManager
|
||||
from core.rag.index_processor.constant.doc_type import DocType
|
||||
@ -16,6 +17,7 @@ class RerankModelRunner(BaseRerankRunner):
|
||||
def __init__(self, rerank_model_instance: ModelInstance):
|
||||
self.rerank_model_instance = rerank_model_instance
|
||||
|
||||
@override
|
||||
def run(
|
||||
self,
|
||||
query: str,
|
||||
|
||||
@ -1,5 +1,6 @@
|
||||
import math
|
||||
from collections import Counter
|
||||
from typing import override
|
||||
|
||||
import numpy as np
|
||||
|
||||
@ -19,6 +20,7 @@ class WeightRerankRunner(BaseRerankRunner):
|
||||
self.tenant_id = tenant_id
|
||||
self.weights = weights
|
||||
|
||||
@override
|
||||
def run(
|
||||
self,
|
||||
query: str,
|
||||
|
||||
@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
import codecs
|
||||
import re
|
||||
from collections.abc import Set as AbstractSet
|
||||
from typing import Any, Literal
|
||||
from typing import Any, Literal, override
|
||||
|
||||
from core.model_manager import ModelInstance
|
||||
from core.rag.splitter.text_splitter import RecursiveCharacterTextSplitter
|
||||
@ -51,6 +51,7 @@ class FixedRecursiveCharacterTextSplitter(EnhanceRecursiveCharacterTextSplitter)
|
||||
self._fixed_separator = codecs.decode(fixed_separator, "unicode_escape")
|
||||
self._separators = separators or ["\n\n", "\n", "。", ". ", " ", ""]
|
||||
|
||||
@override
|
||||
def split_text(self, text: str) -> list[str]:
|
||||
"""Split incoming text and return chunks."""
|
||||
if self._fixed_separator:
|
||||
|
||||
@ -7,7 +7,7 @@ from abc import ABC, abstractmethod
|
||||
from collections.abc import Callable, Iterable, Sequence
|
||||
from collections.abc import Set as AbstractSet
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Literal
|
||||
from typing import Any, Literal, override
|
||||
|
||||
from core.rag.models.document import BaseDocumentTransformer, Document
|
||||
|
||||
@ -148,10 +148,12 @@ class TextSplitter(BaseDocumentTransformer, ABC):
|
||||
)
|
||||
return cls(length_function=lambda x: [_huggingface_tokenizer_length(text) for text in x], **kwargs)
|
||||
|
||||
@override
|
||||
def transform_documents(self, documents: Sequence[Document], **kwargs: Any) -> Sequence[Document]:
|
||||
"""Transform sequence of documents by splitting them."""
|
||||
return self.split_documents(list(documents))
|
||||
|
||||
@override
|
||||
async def atransform_documents(self, documents: Sequence[Document], **kwargs: Any) -> Sequence[Document]:
|
||||
"""Asynchronously transform a sequence of documents by splitting them."""
|
||||
raise NotImplementedError
|
||||
@ -211,6 +213,7 @@ class TokenTextSplitter(TextSplitter):
|
||||
self._allowed_special: Literal["all"] | AbstractSet[str] = allowed_special
|
||||
self._disallowed_special: Literal["all"] | AbstractSet[str] = disallowed_special
|
||||
|
||||
@override
|
||||
def split_text(self, text: str) -> list[str]:
|
||||
def _encode(_text: str) -> list[int]:
|
||||
return self._tokenizer.encode(
|
||||
@ -287,5 +290,6 @@ class RecursiveCharacterTextSplitter(TextSplitter):
|
||||
|
||||
return final_chunks
|
||||
|
||||
@override
|
||||
def split_text(self, text: str) -> list[str]:
|
||||
return self._split_text(text, self._separators)
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
from abc import abstractmethod
|
||||
from os import listdir, path
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.entities.provider_entities import ProviderConfig
|
||||
from core.helper.module_import_helper import load_single_subclass_from_source
|
||||
@ -105,6 +105,7 @@ class BuiltinToolProviderController(ToolProviderController):
|
||||
"""
|
||||
return self.tools
|
||||
|
||||
@override
|
||||
def get_credentials_schema(self) -> list[ProviderConfig]:
|
||||
"""
|
||||
returns the credentials schema of the provider
|
||||
@ -182,6 +183,7 @@ class BuiltinToolProviderController(ToolProviderController):
|
||||
)
|
||||
|
||||
@property
|
||||
@override
|
||||
def provider_type(self) -> ToolProviderType:
|
||||
"""
|
||||
returns the type of the provider
|
||||
|
||||
@ -1,8 +1,9 @@
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.tools.builtin_tool.provider import BuiltinToolProviderController
|
||||
|
||||
|
||||
class AudioToolProvider(BuiltinToolProviderController):
|
||||
@override
|
||||
def _validate_credentials(self, user_id: str, credentials: dict[str, Any]):
|
||||
pass
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
import io
|
||||
from collections.abc import Generator
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.model_manager import ModelManager
|
||||
from core.plugin.entities.parameters import PluginParameterOption
|
||||
@ -14,6 +14,7 @@ from services.model_provider_service import ModelProviderService
|
||||
|
||||
|
||||
class ASRTool(BuiltinTool):
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
@ -56,6 +57,7 @@ class ASRTool(BuiltinTool):
|
||||
items.append((provider, model.model))
|
||||
return items
|
||||
|
||||
@override
|
||||
def get_runtime_parameters(
|
||||
self,
|
||||
conversation_id: str | None = None,
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
import io
|
||||
from collections.abc import Generator
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.model_manager import ModelManager
|
||||
from core.plugin.entities.parameters import PluginParameterOption
|
||||
@ -12,6 +12,7 @@ from services.model_provider_service import ModelProviderService
|
||||
|
||||
|
||||
class TTSTool(BuiltinTool):
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
@ -66,6 +67,7 @@ class TTSTool(BuiltinTool):
|
||||
items.append((provider, model.model, voices))
|
||||
return items
|
||||
|
||||
@override
|
||||
def get_runtime_parameters(
|
||||
self,
|
||||
conversation_id: str | None = None,
|
||||
|
||||
@ -1,8 +1,9 @@
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.tools.builtin_tool.provider import BuiltinToolProviderController
|
||||
|
||||
|
||||
class CodeToolProvider(BuiltinToolProviderController):
|
||||
@override
|
||||
def _validate_credentials(self, user_id: str, credentials: dict[str, Any]):
|
||||
pass
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
from collections.abc import Generator
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.helper.code_executor.code_executor import CodeExecutor, CodeLanguage
|
||||
from core.tools.builtin_tool.tool import BuiltinTool
|
||||
@ -8,6 +8,7 @@ from core.tools.errors import ToolInvokeError
|
||||
|
||||
|
||||
class SimpleCode(BuiltinTool):
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
|
||||
@ -1,8 +1,9 @@
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.tools.builtin_tool.provider import BuiltinToolProviderController
|
||||
|
||||
|
||||
class WikiPediaProvider(BuiltinToolProviderController):
|
||||
@override
|
||||
def _validate_credentials(self, user_id: str, credentials: dict[str, Any]):
|
||||
pass
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
from collections.abc import Generator
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from pytz import timezone as pytz_timezone # type: ignore[import-untyped]
|
||||
|
||||
@ -9,6 +9,7 @@ from core.tools.entities.tool_entities import ToolInvokeMessage
|
||||
|
||||
|
||||
class CurrentTimeTool(BuiltinTool):
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
from collections.abc import Generator
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
import pytz # type: ignore[import-untyped]
|
||||
|
||||
@ -10,6 +10,7 @@ from core.tools.errors import ToolInvokeError
|
||||
|
||||
|
||||
class LocaltimeToTimestampTool(BuiltinTool):
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
from collections.abc import Generator
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
import pytz # type: ignore[import-untyped]
|
||||
|
||||
@ -10,6 +10,7 @@ from core.tools.errors import ToolInvokeError
|
||||
|
||||
|
||||
class TimestampToLocaltimeTool(BuiltinTool):
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
from collections.abc import Generator
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
import pytz # type: ignore[import-untyped]
|
||||
|
||||
@ -10,6 +10,7 @@ from core.tools.errors import ToolInvokeError
|
||||
|
||||
|
||||
class TimezoneConversionTool(BuiltinTool):
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
|
||||
@ -1,13 +1,14 @@
|
||||
import calendar
|
||||
from collections.abc import Generator
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.tools.builtin_tool.tool import BuiltinTool
|
||||
from core.tools.entities.tool_entities import ToolInvokeMessage
|
||||
|
||||
|
||||
class WeekdayTool(BuiltinTool):
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
from collections.abc import Generator
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.tools.builtin_tool.tool import BuiltinTool
|
||||
from core.tools.entities.tool_entities import ToolInvokeMessage
|
||||
@ -8,6 +8,7 @@ from core.tools.utils.web_reader_tool import get_url
|
||||
|
||||
|
||||
class WebscraperTool(BuiltinTool):
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
|
||||
@ -1,9 +1,10 @@
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.tools.builtin_tool.provider import BuiltinToolProviderController
|
||||
|
||||
|
||||
class WebscraperProvider(BuiltinToolProviderController):
|
||||
@override
|
||||
def _validate_credentials(self, user_id: str, credentials: dict[str, Any]):
|
||||
"""
|
||||
Validate credentials
|
||||
|
||||
@ -1,5 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import override
|
||||
|
||||
from core.tools.__base.tool import Tool
|
||||
from core.tools.__base.tool_runtime import ToolRuntime
|
||||
from core.tools.entities.tool_entities import ToolProviderType
|
||||
@ -26,6 +28,7 @@ class BuiltinTool(Tool):
|
||||
super().__init__(**kwargs)
|
||||
self.provider = provider
|
||||
|
||||
@override
|
||||
def fork_tool_runtime(self, runtime: ToolRuntime) -> BuiltinTool:
|
||||
"""
|
||||
fork a new tool with metadata
|
||||
@ -56,6 +59,7 @@ class BuiltinTool(Tool):
|
||||
caller_user_id=self.runtime.user_id,
|
||||
)
|
||||
|
||||
@override
|
||||
def tool_provider_type(self) -> ToolProviderType:
|
||||
return ToolProviderType.BUILT_IN
|
||||
|
||||
|
||||
@ -1,5 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import override
|
||||
|
||||
from pydantic import Field
|
||||
from sqlalchemy import select
|
||||
|
||||
@ -122,6 +124,7 @@ class ApiToolProviderController(ToolProviderController):
|
||||
)
|
||||
|
||||
@property
|
||||
@override
|
||||
def provider_type(self) -> ToolProviderType:
|
||||
return ToolProviderType.API
|
||||
|
||||
@ -194,6 +197,7 @@ class ApiToolProviderController(ToolProviderController):
|
||||
self.tools = tools
|
||||
return tools
|
||||
|
||||
@override
|
||||
def get_tool(self, tool_name: str) -> ApiTool:
|
||||
"""
|
||||
get tool by name
|
||||
|
||||
@ -2,7 +2,7 @@ import json
|
||||
from collections.abc import Generator
|
||||
from dataclasses import dataclass
|
||||
from os import getenv
|
||||
from typing import Any, Union
|
||||
from typing import Any, Union, override
|
||||
from urllib.parse import urlencode
|
||||
|
||||
import httpx
|
||||
@ -45,6 +45,7 @@ class ApiTool(Tool):
|
||||
self.api_bundle = api_bundle
|
||||
self.provider_id = provider_id
|
||||
|
||||
@override
|
||||
def fork_tool_runtime(self, runtime: ToolRuntime):
|
||||
"""
|
||||
fork a new tool with metadata
|
||||
@ -77,6 +78,7 @@ class ApiTool(Tool):
|
||||
# For credential validation, always return as string
|
||||
return parsed_response.to_string()
|
||||
|
||||
@override
|
||||
def tool_provider_type(self) -> ToolProviderType:
|
||||
return ToolProviderType.API
|
||||
|
||||
@ -373,6 +375,7 @@ class ApiTool(Tool):
|
||||
except ValueError:
|
||||
return value
|
||||
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
|
||||
@ -54,6 +54,9 @@ class ToolProviderApiEntity(BaseModel):
|
||||
configuration: MCPConfiguration | None = Field(
|
||||
default=None, description="The timeout and sse_read_timeout of the MCP tool"
|
||||
)
|
||||
# M3 — user-identity forwarding selector. Round-tripped through the
|
||||
# console API so the create/edit modal can hydrate the toggle state.
|
||||
identity_mode: str = Field(default="off", description="Identity-forwarding mechanism: 'off' or 'idp_token'")
|
||||
# Workflow
|
||||
workflow_app_id: str | None = Field(default=None, description="The app id of the workflow tool")
|
||||
|
||||
@ -92,6 +95,9 @@ class ToolProviderApiEntity(BaseModel):
|
||||
optional_fields.update(self.optional_field("is_dynamic_registration", self.is_dynamic_registration))
|
||||
optional_fields.update(self.optional_field("masked_headers", self.masked_headers))
|
||||
optional_fields.update(self.optional_field("original_headers", self.original_headers))
|
||||
# M3 — forwarding selector. Always emit ("off" is a valid
|
||||
# value that the UI must hydrate, not skip).
|
||||
optional_fields["identity_mode"] = self.identity_mode
|
||||
case ToolProviderType.WORKFLOW:
|
||||
optional_fields.update(self.optional_field("workflow_app_id", self.workflow_app_id))
|
||||
case _:
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
from typing import Any, Self
|
||||
from typing import Any, Self, override
|
||||
|
||||
from core.entities.mcp_provider import MCPProviderEntity
|
||||
from core.entities.mcp_provider import IdentityMode, MCPProviderEntity
|
||||
from core.mcp.types import Tool as RemoteMCPTool
|
||||
from core.tools.__base.tool_provider import ToolProviderController
|
||||
from core.tools.__base.tool_runtime import ToolRuntime
|
||||
@ -28,6 +28,7 @@ class MCPToolProviderController(ToolProviderController):
|
||||
headers: dict[str, str] | None = None,
|
||||
timeout: float | None = None,
|
||||
sse_read_timeout: float | None = None,
|
||||
identity_mode: IdentityMode = IdentityMode.OFF,
|
||||
):
|
||||
super().__init__(entity)
|
||||
self.entity: ToolProviderEntityWithPlugin = entity
|
||||
@ -37,8 +38,10 @@ class MCPToolProviderController(ToolProviderController):
|
||||
self.headers = headers or {}
|
||||
self.timeout = timeout
|
||||
self.sse_read_timeout = sse_read_timeout
|
||||
self.identity_mode: IdentityMode = identity_mode
|
||||
|
||||
@property
|
||||
@override
|
||||
def provider_type(self) -> ToolProviderType:
|
||||
"""
|
||||
returns the type of the provider
|
||||
@ -105,6 +108,7 @@ class MCPToolProviderController(ToolProviderController):
|
||||
headers=entity.headers,
|
||||
timeout=entity.timeout,
|
||||
sse_read_timeout=entity.sse_read_timeout,
|
||||
identity_mode=entity.identity_mode,
|
||||
)
|
||||
|
||||
def _validate_credentials(self, user_id: str, credentials: dict[str, Any]):
|
||||
@ -113,6 +117,7 @@ class MCPToolProviderController(ToolProviderController):
|
||||
"""
|
||||
pass
|
||||
|
||||
@override
|
||||
def get_tool(self, tool_name: str) -> MCPTool:
|
||||
"""
|
||||
return tool with given name
|
||||
@ -134,6 +139,7 @@ class MCPToolProviderController(ToolProviderController):
|
||||
headers=self.headers,
|
||||
timeout=self.timeout,
|
||||
sse_read_timeout=self.sse_read_timeout,
|
||||
identity_mode=self.identity_mode,
|
||||
)
|
||||
|
||||
def get_tools(self) -> list[MCPTool]:
|
||||
@ -151,6 +157,7 @@ class MCPToolProviderController(ToolProviderController):
|
||||
headers=self.headers,
|
||||
timeout=self.timeout,
|
||||
sse_read_timeout=self.sse_read_timeout,
|
||||
identity_mode=self.identity_mode,
|
||||
)
|
||||
for tool_entity in self.entity.tools
|
||||
]
|
||||
|
||||
@ -4,8 +4,10 @@ import base64
|
||||
import json
|
||||
import logging
|
||||
from collections.abc import Generator, Mapping
|
||||
from typing import Any, cast
|
||||
from typing import Any, cast, override
|
||||
|
||||
from configs import dify_config
|
||||
from core.entities.mcp_provider import IdentityMode
|
||||
from core.mcp.auth_client import MCPClientWithAuthRetry
|
||||
from core.mcp.error import MCPConnectionError
|
||||
from core.mcp.types import (
|
||||
@ -25,6 +27,11 @@ from graphon.model_runtime.entities.llm_entities import LLMUsage, LLMUsageMetada
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Custom header used to carry the forwarded SSO access token. Picked to avoid
|
||||
# stomping on the workspace-scoped Authorization header (provider OAuth /
|
||||
# user-supplied custom credentials), which would silently break those flows.
|
||||
FORWARDED_IDENTITY_HEADER = "X-Dify-SSO-Access-Token"
|
||||
|
||||
|
||||
class MCPTool(Tool):
|
||||
def __init__(
|
||||
@ -38,6 +45,7 @@ class MCPTool(Tool):
|
||||
headers: dict[str, str] | None = None,
|
||||
timeout: float | None = None,
|
||||
sse_read_timeout: float | None = None,
|
||||
identity_mode: IdentityMode = IdentityMode.OFF,
|
||||
):
|
||||
super().__init__(entity, runtime)
|
||||
self.tenant_id = tenant_id
|
||||
@ -47,11 +55,14 @@ class MCPTool(Tool):
|
||||
self.headers = headers or {}
|
||||
self.timeout = timeout
|
||||
self.sse_read_timeout = sse_read_timeout
|
||||
self.identity_mode: IdentityMode = identity_mode
|
||||
self._latest_usage = LLMUsage.empty_usage()
|
||||
|
||||
@override
|
||||
def tool_provider_type(self) -> ToolProviderType:
|
||||
return ToolProviderType.MCP
|
||||
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
@ -60,7 +71,7 @@ class MCPTool(Tool):
|
||||
app_id: str | None = None,
|
||||
message_id: str | None = None,
|
||||
) -> Generator[ToolInvokeMessage, None, None]:
|
||||
result = self.invoke_remote_mcp_tool(tool_parameters)
|
||||
result = self.invoke_remote_mcp_tool(tool_parameters, user_id=user_id, app_id=app_id)
|
||||
|
||||
# Extract usage metadata from MCP protocol's _meta field
|
||||
self._latest_usage = self._derive_usage_from_result(result)
|
||||
@ -223,6 +234,7 @@ class MCPTool(Tool):
|
||||
return found
|
||||
return None
|
||||
|
||||
@override
|
||||
def fork_tool_runtime(self, runtime: ToolRuntime) -> MCPTool:
|
||||
return MCPTool(
|
||||
entity=self.entity,
|
||||
@ -234,6 +246,7 @@ class MCPTool(Tool):
|
||||
headers=self.headers,
|
||||
timeout=self.timeout,
|
||||
sse_read_timeout=self.sse_read_timeout,
|
||||
identity_mode=self.identity_mode,
|
||||
)
|
||||
|
||||
def _handle_none_parameter(self, parameter: dict[str, Any]) -> dict[str, Any]:
|
||||
@ -246,7 +259,26 @@ class MCPTool(Tool):
|
||||
if value is not None and not (isinstance(value, str) and value.strip() == "")
|
||||
}
|
||||
|
||||
def invoke_remote_mcp_tool(self, tool_parameters: dict[str, Any]) -> CallToolResult:
|
||||
@property
|
||||
def _forwarding_requested(self) -> bool:
|
||||
"""True only when the configured identity_mode wants forwarding AND
|
||||
the deployment actually has the enterprise side that can mint tokens.
|
||||
Non-enterprise installs treat the DB value as a no-op — a stale row
|
||||
won't trigger a 5xx against a missing inner-API endpoint."""
|
||||
return self.identity_mode != IdentityMode.OFF and dify_config.ENTERPRISE_ENABLED
|
||||
|
||||
def invoke_remote_mcp_tool(
|
||||
self,
|
||||
tool_parameters: dict[str, Any],
|
||||
user_id: str | None = None,
|
||||
app_id: str | None = None,
|
||||
) -> CallToolResult:
|
||||
# Fail closed: forwarding requires user_id (refuse before any DB I/O).
|
||||
if self._forwarding_requested and not user_id:
|
||||
raise ToolInvokeError(
|
||||
"Forward-user-identity is enabled for this MCP provider but no end-user context was supplied."
|
||||
)
|
||||
|
||||
headers = self.headers.copy() if self.headers else {}
|
||||
tool_parameters = self._handle_none_parameter(tool_parameters)
|
||||
|
||||
@ -271,6 +303,15 @@ class MCPTool(Tool):
|
||||
if tokens and tokens.access_token:
|
||||
headers["Authorization"] = f"{tokens.token_type.capitalize()} {tokens.access_token}"
|
||||
|
||||
# Forwarded identity rides in a custom header so workspace-scoped
|
||||
# provider credentials (Authorization / custom Headers) keep working
|
||||
# untouched. The MCP server is expected to read X-Dify-SSO-Access-Token
|
||||
# when identity forwarding is configured.
|
||||
forward_identity_active = False
|
||||
if self._forwarding_requested and user_id:
|
||||
self._inject_forwarded_identity(headers, user_id=user_id, app_id=app_id, audience=server_url)
|
||||
forward_identity_active = True
|
||||
|
||||
# Step 2: Session is now closed, perform network operations without holding database connection
|
||||
# MCPClientWithAuthRetry will create a new session lazily only if auth retry is needed
|
||||
try:
|
||||
@ -280,9 +321,44 @@ class MCPTool(Tool):
|
||||
timeout=self.timeout,
|
||||
sse_read_timeout=self.sse_read_timeout,
|
||||
provider_entity=provider_entity,
|
||||
forward_identity_active=forward_identity_active,
|
||||
) as mcp_client:
|
||||
return mcp_client.invoke_tool(tool_name=self.entity.identity.name, tool_args=tool_parameters)
|
||||
except MCPConnectionError as e:
|
||||
raise ToolInvokeError(f"Failed to connect to MCP server: {e}") from e
|
||||
except Exception as e:
|
||||
raise ToolInvokeError(f"Failed to invoke tool: {e}") from e
|
||||
|
||||
def _inject_forwarded_identity(
|
||||
self,
|
||||
headers: dict[str, str],
|
||||
*,
|
||||
user_id: str,
|
||||
app_id: str | None,
|
||||
audience: str,
|
||||
) -> None:
|
||||
"""Call the enterprise IssueMCPToken endpoint and stamp the issued
|
||||
token into X-Dify-SSO-Access-Token.
|
||||
|
||||
A custom header is used (rather than Authorization) so it composes
|
||||
with workspace-scoped provider credentials — the user may have OAuth
|
||||
tokens or a custom Authorization header configured on the MCP
|
||||
provider, and forwarding must not silently overwrite them.
|
||||
|
||||
Errors are surfaced as ToolInvokeError so the workflow halts with a
|
||||
clear message instead of silently dropping identity and hitting the
|
||||
MCP server unauthenticated.
|
||||
"""
|
||||
from services.enterprise.base import MCPTokenError
|
||||
from services.enterprise.enterprise_service import EnterpriseService
|
||||
|
||||
try:
|
||||
token, _expires_at = EnterpriseService.issue_mcp_token(
|
||||
user_id=user_id,
|
||||
tenant_id=self.tenant_id,
|
||||
app_id=app_id,
|
||||
audience=audience,
|
||||
)
|
||||
except MCPTokenError as e:
|
||||
raise ToolInvokeError(f"Failed to obtain forwarded identity token: {e}") from e
|
||||
headers[FORWARDED_IDENTITY_HEADER] = token
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.plugin.impl.tool import PluginToolManager
|
||||
from core.tools.__base.tool_runtime import ToolRuntime
|
||||
@ -23,6 +23,7 @@ class PluginToolProviderController(BuiltinToolProviderController):
|
||||
self.plugin_unique_identifier = plugin_unique_identifier
|
||||
|
||||
@property
|
||||
@override
|
||||
def provider_type(self) -> ToolProviderType:
|
||||
"""
|
||||
returns the type of the provider
|
||||
@ -31,6 +32,7 @@ class PluginToolProviderController(BuiltinToolProviderController):
|
||||
"""
|
||||
return ToolProviderType.PLUGIN
|
||||
|
||||
@override
|
||||
def _validate_credentials(self, user_id: str, credentials: dict[str, Any]):
|
||||
"""
|
||||
validate the credentials of the provider
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Generator
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.plugin.impl.tool import PluginToolManager
|
||||
from core.plugin.utils.converter import convert_parameters_to_plugin_format
|
||||
@ -20,9 +20,11 @@ class PluginTool(Tool):
|
||||
self.plugin_unique_identifier = plugin_unique_identifier
|
||||
self.runtime_parameters: list[ToolParameter] | None = None
|
||||
|
||||
@override
|
||||
def tool_provider_type(self) -> ToolProviderType:
|
||||
return ToolProviderType.PLUGIN
|
||||
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
@ -48,6 +50,7 @@ class PluginTool(Tool):
|
||||
message_id=message_id,
|
||||
)
|
||||
|
||||
@override
|
||||
def fork_tool_runtime(self, runtime: ToolRuntime) -> PluginTool:
|
||||
return PluginTool(
|
||||
entity=self.entity,
|
||||
@ -57,6 +60,7 @@ class PluginTool(Tool):
|
||||
plugin_unique_identifier=self.plugin_unique_identifier,
|
||||
)
|
||||
|
||||
@override
|
||||
def get_runtime_parameters(
|
||||
self,
|
||||
conversation_id: str | None = None,
|
||||
|
||||
@ -1,4 +1,5 @@
|
||||
import threading
|
||||
from typing import override
|
||||
|
||||
from flask import Flask, current_app
|
||||
from pydantic import BaseModel, Field
|
||||
@ -46,6 +47,7 @@ class DatasetMultiRetrieverTool(DatasetRetrieverBaseTool):
|
||||
name=f"dataset_{tenant_id.replace('-', '_')}", tenant_id=tenant_id, dataset_ids=dataset_ids, **kwargs
|
||||
)
|
||||
|
||||
@override
|
||||
def _run(self, query: str) -> str:
|
||||
threads = []
|
||||
all_documents: list[RagDocument] = []
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
from typing import Any, cast
|
||||
from typing import Any, cast, override
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy import select
|
||||
@ -56,6 +56,7 @@ class DatasetRetrieverTool(DatasetRetrieverBaseTool):
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
@override
|
||||
def _run(self, query: str) -> str:
|
||||
dataset_stmt = select(Dataset).where(Dataset.tenant_id == self.tenant_id, Dataset.id == self.dataset_id)
|
||||
dataset = db.session.scalar(dataset_stmt)
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
from collections.abc import Generator
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from core.app.app_config.entities import DatasetRetrieveConfigEntity
|
||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||
@ -85,6 +85,7 @@ class DatasetRetrieverTool(Tool):
|
||||
|
||||
return tools
|
||||
|
||||
@override
|
||||
def get_runtime_parameters(
|
||||
self,
|
||||
conversation_id: str | None = None,
|
||||
@ -105,9 +106,11 @@ class DatasetRetrieverTool(Tool):
|
||||
),
|
||||
]
|
||||
|
||||
@override
|
||||
def tool_provider_type(self) -> ToolProviderType:
|
||||
return ToolProviderType.DATASET_RETRIEVAL
|
||||
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from typing import override
|
||||
|
||||
from pydantic import Field
|
||||
from sqlalchemy import select
|
||||
@ -80,6 +81,7 @@ class WorkflowToolProviderController(ToolProviderController):
|
||||
return controller
|
||||
|
||||
@property
|
||||
@override
|
||||
def provider_type(self) -> ToolProviderType:
|
||||
return ToolProviderType.WORKFLOW
|
||||
|
||||
|
||||
@ -3,7 +3,7 @@ from __future__ import annotations
|
||||
import json
|
||||
import logging
|
||||
from collections.abc import Generator, Mapping, Sequence
|
||||
from typing import Any, cast
|
||||
from typing import Any, cast, override
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
@ -67,6 +67,7 @@ class WorkflowTool(Tool):
|
||||
|
||||
super().__init__(entity=entity, runtime=runtime)
|
||||
|
||||
@override
|
||||
def tool_provider_type(self) -> ToolProviderType:
|
||||
"""
|
||||
get the tool provider type
|
||||
@ -75,6 +76,7 @@ class WorkflowTool(Tool):
|
||||
"""
|
||||
return ToolProviderType.WORKFLOW
|
||||
|
||||
@override
|
||||
def _invoke(
|
||||
self,
|
||||
user_id: str,
|
||||
@ -206,6 +208,7 @@ class WorkflowTool(Tool):
|
||||
return found
|
||||
return None
|
||||
|
||||
@override
|
||||
def fork_tool_runtime(self, runtime: ToolRuntime) -> WorkflowTool:
|
||||
"""
|
||||
fork a new tool with metadata
|
||||
|
||||
@ -6,7 +6,7 @@ import time
|
||||
from abc import ABC, abstractmethod
|
||||
from collections.abc import Mapping
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from typing import Any, override
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
@ -62,6 +62,7 @@ class TriggerDebugEventPoller(ABC):
|
||||
|
||||
|
||||
class PluginTriggerDebugEventPoller(TriggerDebugEventPoller):
|
||||
@override
|
||||
def poll(self) -> TriggerDebugEvent | None:
|
||||
from services.trigger.trigger_service import TriggerService
|
||||
|
||||
@ -103,6 +104,7 @@ class PluginTriggerDebugEventPoller(TriggerDebugEventPoller):
|
||||
|
||||
|
||||
class WebhookTriggerDebugEventPoller(TriggerDebugEventPoller):
|
||||
@override
|
||||
def poll(self) -> TriggerDebugEvent | None:
|
||||
pool_key = build_webhook_pool_key(
|
||||
tenant_id=self.tenant_id,
|
||||
@ -190,6 +192,7 @@ class ScheduleTriggerDebugEventPoller(TriggerDebugEventPoller):
|
||||
inputs={},
|
||||
)
|
||||
|
||||
@override
|
||||
def poll(self) -> TriggerDebugEvent | None:
|
||||
schedule_debug_runtime = self.get_or_create_schedule_debug_runtime()
|
||||
if schedule_debug_runtime.next_run_at > naive_utc_now():
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
from collections.abc import Mapping
|
||||
from typing import Union
|
||||
from typing import Union, override
|
||||
|
||||
from core.entities.provider_entities import BasicProviderConfig, ProviderConfig
|
||||
from core.helper.provider_cache import ProviderCredentialsCache
|
||||
@ -16,6 +16,7 @@ class TriggerProviderCredentialsCache(ProviderCredentialsCache):
|
||||
def __init__(self, tenant_id: str, provider_id: str, credential_id: str):
|
||||
super().__init__(tenant_id=tenant_id, provider_id=provider_id, credential_id=credential_id)
|
||||
|
||||
@override
|
||||
def _generate_cache_key(self, **kwargs) -> str:
|
||||
tenant_id = kwargs["tenant_id"]
|
||||
provider_id = kwargs["provider_id"]
|
||||
@ -29,6 +30,7 @@ class TriggerProviderOAuthClientParamsCache(ProviderCredentialsCache):
|
||||
def __init__(self, tenant_id: str, provider_id: str):
|
||||
super().__init__(tenant_id=tenant_id, provider_id=provider_id)
|
||||
|
||||
@override
|
||||
def _generate_cache_key(self, **kwargs) -> str:
|
||||
tenant_id = kwargs["tenant_id"]
|
||||
provider_id = kwargs["provider_id"]
|
||||
@ -41,6 +43,7 @@ class TriggerProviderPropertiesCache(ProviderCredentialsCache):
|
||||
def __init__(self, tenant_id: str, provider_id: str, subscription_id: str):
|
||||
super().__init__(tenant_id=tenant_id, provider_id=provider_id, subscription_id=subscription_id)
|
||||
|
||||
@override
|
||||
def _generate_cache_key(self, **kwargs) -> str:
|
||||
tenant_id = kwargs["tenant_id"]
|
||||
provider_id = kwargs["provider_id"]
|
||||
|
||||
@ -12,6 +12,7 @@ examples accurate or the LLM will invent fields.
|
||||
"""
|
||||
|
||||
import json
|
||||
from collections.abc import Iterable
|
||||
from typing import Any
|
||||
|
||||
# Per-node-type configuration cheatsheet.
|
||||
@ -22,11 +23,24 @@ from typing import Any
|
||||
# both ``WorkflowService.sync_draft_workflow``'s structural checks and the
|
||||
# runtime entity validation each node performs when the workflow runs.
|
||||
#
|
||||
# The cheatsheet is assembled DYNAMICALLY per request: the planner decides
|
||||
# which node types the workflow needs, and ``build_node_config_cheatsheet``
|
||||
# stitches together only the snippets for those types (plus the always-needed
|
||||
# wrapper / shared-field / edge-handle preamble, and the containers section
|
||||
# when an iteration / loop is planned). This keeps the builder prompt tight —
|
||||
# a 3-node summariser no longer carries the schema for 12 unrelated node
|
||||
# types — and lets each snippet document its FULL schema (e.g. a "file" start
|
||||
# variable's required ``allowed_file_types``) without bloating every prompt.
|
||||
#
|
||||
# The postprocessor in ``runner.py`` fills missing wrapper fields (``type``,
|
||||
# ``positionAbsolute``, ``width``, ``height``, ``sourcePosition`` /
|
||||
# ``targetPosition``, edge ``data.sourceType`` / ``data.targetType``), so the
|
||||
# LLM only needs to emit semantically meaningful fields.
|
||||
NODE_CONFIG_CHEATSHEET = """\
|
||||
|
||||
# Always-included preamble: the node/edge wrapper shape and the shared
|
||||
# ``data`` fields that apply to every node type, plus the "## Per type" header
|
||||
# the per-type snippets slot under.
|
||||
_CHEATSHEET_PREAMBLE = """\
|
||||
## Node wrapper (every node, top-level)
|
||||
|
||||
{"id": "node1" (digits + letters only — see "Node IDs" below),
|
||||
@ -46,14 +60,26 @@ Children of iteration / loop containers additionally need
|
||||
"desc": "<one-liner>",
|
||||
"selected": false}
|
||||
|
||||
## Per type — additional "data" fields
|
||||
## Per type — additional "data" fields (only the node types in your plan are shown)"""
|
||||
|
||||
|
||||
# node_type → its per-type schema snippet. Keyed by the exact ``node_type``
|
||||
# string the planner emits so ``build_node_config_cheatsheet`` can look each
|
||||
# one up directly. Iteration / loop are documented in the Containers section
|
||||
# (they are subgraphs, not leaf nodes) rather than here.
|
||||
_NODE_SNIPPETS: dict[str, str] = {
|
||||
"start": """\
|
||||
- start:
|
||||
{"variables": [
|
||||
{"variable": "url", "label": "URL", "type": "text-input",
|
||||
"required": true, "max_length": 256, "options": []},
|
||||
{"variable": "topic", "label": "Topic", "type": "paragraph",
|
||||
"required": false, "max_length": 4096, "options": []}
|
||||
"required": false, "max_length": 4096, "options": []},
|
||||
{"variable": "doc", "label": "Document", "type": "file",
|
||||
"required": true,
|
||||
"allowed_file_types": ["document"],
|
||||
"allowed_file_upload_methods": ["local_file", "remote_url"],
|
||||
"allowed_file_extensions": []}
|
||||
]}
|
||||
EVERY user-supplied value referenced by a downstream node
|
||||
(``{{#node-id.var#}}`` in a prompt / answer / template, or
|
||||
@ -62,19 +88,29 @@ Children of iteration / loop containers additionally need
|
||||
If the planner's ``start_inputs`` list is non-empty, use it verbatim
|
||||
(the user prompt section "Start inputs" surfaces it). Types:
|
||||
text-input | paragraph | select | number | file | file-list.
|
||||
For a "file" or "file-list" variable you MUST also set
|
||||
``allowed_file_types`` to a NON-EMPTY subset of
|
||||
["document", "image", "audio", "video", "custom"] — it is a REQUIRED
|
||||
field and the draft fails to load (showing "supported file types is
|
||||
required") without it. Choose by purpose: ["document"] for text
|
||||
extraction (PDF / Word / PPT / Markdown / …), ["image"] for vision,
|
||||
etc. Always set ``allowed_file_upload_methods`` to
|
||||
["local_file", "remote_url"]. Only when you include "custom" must you
|
||||
also set ``allowed_file_extensions`` to a non-empty list like
|
||||
[".epub", ".rtf"]; otherwise leave it [].
|
||||
In Advanced-Chat mode ``sys.query`` and ``sys.files`` are automatic
|
||||
system variables — downstream nodes may reference them; do NOT add
|
||||
them to ``variables``.
|
||||
|
||||
them to ``variables``.""",
|
||||
"end": """\
|
||||
- end (Workflow mode only):
|
||||
{"outputs": [
|
||||
{"variable": "result", "value_selector": ["<src-node-id>", "<out-var>"]}
|
||||
]}
|
||||
|
||||
]}""",
|
||||
"answer": """\
|
||||
- answer (Advanced Chat mode only):
|
||||
{"variables": [],
|
||||
"answer": "<text with {{#<src>.<var>#}} placeholders>"}
|
||||
|
||||
"answer": "<text with {{#<src>.<var>#}} placeholders>"}""",
|
||||
"llm": """\
|
||||
- llm:
|
||||
{"model": {"provider": "<provider>", "name": "<model>", "mode": "chat",
|
||||
"completion_params": {"temperature": 0.7}},
|
||||
@ -100,26 +136,26 @@ Children of iteration / loop containers additionally need
|
||||
values are the translations.
|
||||
Input: {{#node1.text#}}
|
||||
* Each placeholder only resolves the variable from its source node —
|
||||
it cannot be a Jinja template or call a function.
|
||||
|
||||
it cannot be a Jinja template or call a function.""",
|
||||
"knowledge-retrieval": """\
|
||||
- knowledge-retrieval:
|
||||
{"query_variable_selector": ["<src>", "<var>"],
|
||||
"query_attachment_selector": [],
|
||||
"dataset_ids": [],
|
||||
"retrieval_mode": "multiple",
|
||||
"multiple_retrieval_config": {"top_k": 4, "score_threshold": null,
|
||||
"reranking_enable": false}}
|
||||
|
||||
"reranking_enable": false}}""",
|
||||
"code": """\
|
||||
- code (escape hatch — only if no installed tool fits):
|
||||
{"code_language": "python3",
|
||||
"code": "def main(arg1: str) -> dict:\\n return {'result': arg1}",
|
||||
"variables": [{"variable": "arg1", "value_selector": ["<src>", "<var>"]}],
|
||||
"outputs": {"result": {"type": "string", "children": null}}}
|
||||
|
||||
"outputs": {"result": {"type": "string", "children": null}}}""",
|
||||
"template-transform": """\
|
||||
- template-transform:
|
||||
{"template": "Hello {{ name }}",
|
||||
"variables": [{"variable": "name", "value_selector": ["<src>", "<var>"]}]}
|
||||
|
||||
"variables": [{"variable": "name", "value_selector": ["<src>", "<var>"]}]}""",
|
||||
"http-request": """\
|
||||
- http-request (escape hatch — only if no installed tool fits):
|
||||
{"variables": [], "method": "get", "url": "https://example.com",
|
||||
"authorization": {"type": "no-auth", "config": null},
|
||||
@ -129,8 +165,8 @@ Children of iteration / loop containers additionally need
|
||||
"timeout": {"max_connect_timeout": 0, "max_read_timeout": 0,
|
||||
"max_write_timeout": 0},
|
||||
"retry_config": {"retry_enabled": true, "max_retries": 3,
|
||||
"retry_interval": 100}}
|
||||
|
||||
"retry_interval": 100}}""",
|
||||
"tool": """\
|
||||
- tool (PREFERRED for external actions when listed in Available tools):
|
||||
{"provider_id": "<provider>", # provider portion of provider/tool
|
||||
"provider_type": "builtin", # exact value from catalogue
|
||||
@ -144,8 +180,8 @@ Children of iteration / loop containers additionally need
|
||||
Parameter ``type`` is one of:
|
||||
"mixed" — string template referencing variables ({{#...#}})
|
||||
"variable" — direct reference, value is ["<src>", "<var>"]
|
||||
"constant" — literal value
|
||||
|
||||
"constant" — literal value""",
|
||||
"if-else": """\
|
||||
- if-else:
|
||||
{"_targetBranches": [{"id": "true", "name": "IF"},
|
||||
{"id": "false", "name": "ELSE"}],
|
||||
@ -158,8 +194,8 @@ Children of iteration / loop containers additionally need
|
||||
"comparison_operator": "is",
|
||||
"value": "<value>"}]}
|
||||
]}
|
||||
Source handle for downstream edges = the case_id ("true" / "false").
|
||||
|
||||
Source handle for downstream edges = the case_id ("true" / "false").""",
|
||||
"question-classifier": """\
|
||||
- question-classifier:
|
||||
{"query_variable_selector": ["<src>", "<var>"],
|
||||
"model": {"provider": "<p>", "name": "<m>", "mode": "chat",
|
||||
@ -169,8 +205,8 @@ Children of iteration / loop containers additionally need
|
||||
"_targetBranches": [{"id": "1", "name": ""}, {"id": "2", "name": ""}],
|
||||
"vision": {"enabled": false},
|
||||
"instruction": ""}
|
||||
Source handle for downstream edges = the class_id ("1" / "2" / ...).
|
||||
|
||||
Source handle for downstream edges = the class_id ("1" / "2" / ...).""",
|
||||
"parameter-extractor": """\
|
||||
- parameter-extractor:
|
||||
{"query": [["<src>", "<var>"]], # array of value_selector arrays
|
||||
"model": {"provider": "<p>", "name": "<m>", "mode": "chat",
|
||||
@ -179,8 +215,8 @@ Children of iteration / loop containers additionally need
|
||||
"description": "<purpose>", "required": true}],
|
||||
"reasoning_mode": "prompt",
|
||||
"vision": {"enabled": false},
|
||||
"instruction": ""}
|
||||
|
||||
"instruction": ""}""",
|
||||
"document-extractor": """\
|
||||
- document-extractor:
|
||||
{"variable_selector": ["<src>", "<file-var>"], # a file / file-list input
|
||||
"is_array_file": false} # true when the input is a
|
||||
@ -188,8 +224,9 @@ Children of iteration / loop containers additionally need
|
||||
Single output variable ``text``: a string when ``is_array_file`` is false,
|
||||
an array of strings (one per file) when it is true. ``variable_selector``
|
||||
MUST point at a ``start`` variable declared with type "file" / "file-list"
|
||||
(or ``sys.files`` in Advanced-Chat mode).
|
||||
|
||||
(or ``sys.files`` in Advanced-Chat mode). That start variable MUST set a
|
||||
non-empty ``allowed_file_types`` (use ["document"] for document text).""",
|
||||
"variable-aggregator": """\
|
||||
- variable-aggregator (merge mutually-exclusive branches into one output):
|
||||
{"output_type": "string", # VarType of the merged value — one of
|
||||
# string | number | object | array[string] |
|
||||
@ -200,8 +237,8 @@ Children of iteration / loop containers additionally need
|
||||
Output variable: ``output`` (the first branch that actually ran). Place it
|
||||
after an ``if-else`` / ``question-classifier`` to rejoin paths before the
|
||||
``end`` / ``answer`` node. Each entry of ``variables`` is a value_selector
|
||||
array, NOT a placeholder string.
|
||||
|
||||
array, NOT a placeholder string.""",
|
||||
"list-operator": """\
|
||||
- list-operator (filter / sort / slice an array variable):
|
||||
{"variable": ["<src>", "<array-var>"],
|
||||
"filter_by": {"enabled": false, "conditions": []},
|
||||
@ -210,8 +247,12 @@ Children of iteration / loop containers additionally need
|
||||
"limit": {"enabled": false, "size": 10}}
|
||||
Enable only the sub-features you need; ``conditions`` reuse the if-else
|
||||
condition shape (key / comparison_operator / value). Outputs: ``result``
|
||||
(the processed array), ``first_record``, ``last_record``.
|
||||
(the processed array), ``first_record``, ``last_record``.""",
|
||||
}
|
||||
|
||||
|
||||
# Pulled into the cheatsheet only when an iteration / loop appears in the plan.
|
||||
_CONTAINERS_SECTION = """\
|
||||
## Containers — iteration / loop
|
||||
|
||||
These are SUBGRAPH nodes. To use one you MUST emit, in order:
|
||||
@ -270,16 +311,59 @@ These are SUBGRAPH nodes. To use one you MUST emit, in order:
|
||||
|
||||
5. The container's incoming/outgoing edges connect to the container's id
|
||||
(``nodeK``), NOT to inner nodes. The first inner edge connects from
|
||||
``nodeKstart``.
|
||||
``nodeKstart``."""
|
||||
|
||||
|
||||
# Always-included trailer: edge handle conventions for every graph.
|
||||
_EDGE_HANDLES_SECTION = """\
|
||||
## Edge handles
|
||||
|
||||
- Most nodes: sourceHandle "source", targetHandle "target".
|
||||
- if-else cases: sourceHandle is the case_id ("true" / "false" / ...).
|
||||
- question-classifier: sourceHandle is the class_id ("1" / "2" / ...).
|
||||
- iteration-start / sourceHandle "source"; the edge from the *start node
|
||||
loop-start: is what kicks off the first inner step.
|
||||
"""
|
||||
loop-start: is what kicks off the first inner step."""
|
||||
|
||||
|
||||
# Container node types are described in ``_CONTAINERS_SECTION`` rather than as
|
||||
# leaf snippets; their presence in a plan pulls that section in.
|
||||
_CONTAINER_NODE_TYPES = frozenset({"iteration", "loop"})
|
||||
|
||||
|
||||
def build_node_config_cheatsheet(node_types: Iterable[str] | None = None) -> str:
|
||||
"""
|
||||
Assemble the builder cheatsheet for exactly the node types in the plan.
|
||||
|
||||
``node_types`` is the set of ``node_type`` strings the planner chose. We
|
||||
emit the always-on preamble (wrapper / shared fields), then only the
|
||||
per-type snippets for the requested types (``start`` is always included —
|
||||
every graph has one), the Containers section when an iteration / loop is
|
||||
planned, and the edge-handles trailer. Unknown / unrecognised type strings
|
||||
are ignored (the runtime / structural validator catches genuinely bogus
|
||||
types).
|
||||
|
||||
``None`` returns the FULL cheatsheet (every snippet + containers) — used to
|
||||
build the static back-compat constants below and as a safe fallback.
|
||||
"""
|
||||
if node_types is None:
|
||||
requested: set[str] = set(_NODE_SNIPPETS) | set(_CONTAINER_NODE_TYPES)
|
||||
else:
|
||||
requested = {str(t).strip() for t in node_types if str(t).strip()}
|
||||
requested.add("start") # every workflow has exactly one start node
|
||||
|
||||
parts: list[str] = [_CHEATSHEET_PREAMBLE]
|
||||
# Iterate _NODE_SNIPPETS (not ``requested``) to keep a stable, readable order.
|
||||
parts.extend(snippet for node_type, snippet in _NODE_SNIPPETS.items() if node_type in requested)
|
||||
if requested & _CONTAINER_NODE_TYPES:
|
||||
parts.append(_CONTAINERS_SECTION)
|
||||
parts.append(_EDGE_HANDLES_SECTION)
|
||||
return "\n\n".join(parts) + "\n"
|
||||
|
||||
|
||||
# Full cheatsheet (all node types) — retained as a module constant so callers
|
||||
# and tests that want the complete reference can import it directly. The
|
||||
# dynamic per-request prompt is built by ``get_builder_system_prompt``.
|
||||
NODE_CONFIG_CHEATSHEET = build_node_config_cheatsheet()
|
||||
|
||||
|
||||
_BASE_SYSTEM_PROMPT_HEAD = """You are a Dify workflow builder.
|
||||
@ -402,21 +486,24 @@ _ADVANCED_CHAT_MODE_RULES = """# Mode-specific rules — Advanced Chat (Chatflow
|
||||
"""
|
||||
|
||||
|
||||
BUILDER_SYSTEM_PROMPT_WORKFLOW = (
|
||||
_BASE_SYSTEM_PROMPT_HEAD
|
||||
+ _WORKFLOW_MODE_RULES
|
||||
+ _BASE_SYSTEM_PROMPT_TAIL
|
||||
+ NODE_CONFIG_CHEATSHEET
|
||||
+ _BASE_SYSTEM_PROMPT_FOOTER
|
||||
)
|
||||
def _assemble_builder_system_prompt(mode: str, node_types: Iterable[str] | None) -> str:
|
||||
"""Stitch the builder system prompt for ``mode`` around a cheatsheet built
|
||||
for ``node_types`` (``None`` → full cheatsheet)."""
|
||||
mode_rules = _ADVANCED_CHAT_MODE_RULES if mode == "advanced-chat" else _WORKFLOW_MODE_RULES
|
||||
return (
|
||||
_BASE_SYSTEM_PROMPT_HEAD
|
||||
+ mode_rules
|
||||
+ _BASE_SYSTEM_PROMPT_TAIL
|
||||
+ build_node_config_cheatsheet(node_types)
|
||||
+ _BASE_SYSTEM_PROMPT_FOOTER
|
||||
)
|
||||
|
||||
BUILDER_SYSTEM_PROMPT_ADVANCED_CHAT = (
|
||||
_BASE_SYSTEM_PROMPT_HEAD
|
||||
+ _ADVANCED_CHAT_MODE_RULES
|
||||
+ _BASE_SYSTEM_PROMPT_TAIL
|
||||
+ NODE_CONFIG_CHEATSHEET
|
||||
+ _BASE_SYSTEM_PROMPT_FOOTER
|
||||
)
|
||||
|
||||
# Static full-cheatsheet prompts — the back-compat default returned by
|
||||
# ``get_builder_system_prompt`` when the caller doesn't pin a node-type set.
|
||||
BUILDER_SYSTEM_PROMPT_WORKFLOW = _assemble_builder_system_prompt("workflow", None)
|
||||
|
||||
BUILDER_SYSTEM_PROMPT_ADVANCED_CHAT = _assemble_builder_system_prompt("advanced-chat", None)
|
||||
|
||||
|
||||
BUILDER_USER_PROMPT = """# User instruction
|
||||
@ -546,8 +633,16 @@ def format_plan_block(plan_nodes: list[dict[str, Any]]) -> str:
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def get_builder_system_prompt(mode: str) -> str:
|
||||
"""Pick the system prompt branch for Workflow vs Advanced Chat."""
|
||||
if mode == "advanced-chat":
|
||||
return BUILDER_SYSTEM_PROMPT_ADVANCED_CHAT
|
||||
return BUILDER_SYSTEM_PROMPT_WORKFLOW
|
||||
def get_builder_system_prompt(mode: str, node_types: Iterable[str] | None = None) -> str:
|
||||
"""
|
||||
Build the builder system prompt for ``mode``, with a cheatsheet scoped to
|
||||
``node_types`` (the planner's chosen node types).
|
||||
|
||||
When ``node_types`` is ``None`` we return the cached full-cheatsheet
|
||||
constant (back-compat default). When the runner passes the plan's node-type
|
||||
set we assemble a fresh prompt carrying only the relevant per-type schemas,
|
||||
so the builder isn't handed config for node types the workflow never uses.
|
||||
"""
|
||||
if node_types is None:
|
||||
return BUILDER_SYSTEM_PROMPT_ADVANCED_CHAT if mode == "advanced-chat" else BUILDER_SYSTEM_PROMPT_WORKFLOW
|
||||
return _assemble_builder_system_prompt(mode, node_types)
|
||||
|
||||
@ -74,6 +74,21 @@ _DEFAULT_VIEWPORT: GraphViewportDict = {"x": 0.0, "y": 0.0, "zoom": 0.7}
|
||||
_DEFAULT_NODE_WIDTH = 244
|
||||
_DEFAULT_NODE_HEIGHT = 100
|
||||
|
||||
# Start-node input variable types that carry file uploads. Mirrors
|
||||
# ``graphon.variables.input_entities.VariableEntityType.FILE / FILE_LIST``.
|
||||
_FILE_VARIABLE_TYPES = frozenset({"file", "file-list"})
|
||||
|
||||
# Backstop defaults for a file / file-list start variable when the builder
|
||||
# omits the required upload config. ``allowed_file_types`` is a REQUIRED field
|
||||
# (Studio rejects the draft with "supported file types is required" when it's
|
||||
# empty — see ``config-var/config-modal/utils.ts``); we default to every
|
||||
# standard type so no valid upload is rejected. ``custom`` is intentionally
|
||||
# excluded because it would in turn require a non-empty
|
||||
# ``allowed_file_extensions``. The real fix is the builder now documenting and
|
||||
# emitting these fields; this is the safety net that guarantees a loadable draft.
|
||||
_DEFAULT_ALLOWED_FILE_TYPES = ("document", "image", "audio", "video")
|
||||
_DEFAULT_FILE_UPLOAD_METHODS = ("local_file", "remote_url")
|
||||
|
||||
# Token ceiling for the planner call when the caller didn't pin one. The plan
|
||||
# is a short JSON node list (a handful of nodes with labels/purposes), so this
|
||||
# is generous headroom while still bounding a runaway response. The builder is
|
||||
@ -512,8 +527,15 @@ class WorkflowGenerator:
|
||||
tool_catalogue_section=format_builder_tool_catalogue_section(tool_catalogue_text),
|
||||
start_inputs_section=format_start_inputs_section(start_inputs or []),
|
||||
)
|
||||
# Scope the builder cheatsheet to exactly the node types the planner
|
||||
# chose, so the prompt carries each type's FULL schema (e.g. a file
|
||||
# start variable's required ``allowed_file_types``) without dragging in
|
||||
# config for unrelated node types.
|
||||
plan_node_types = {
|
||||
str(node.get("node_type") or "").strip() for node in plan_nodes if str(node.get("node_type") or "").strip()
|
||||
}
|
||||
messages = [
|
||||
SystemPromptMessage(content=get_builder_system_prompt(mode)),
|
||||
SystemPromptMessage(content=get_builder_system_prompt(mode, plan_node_types)),
|
||||
UserPromptMessage(content=user_prompt),
|
||||
]
|
||||
parsed = cls._invoke_and_parse_json(
|
||||
@ -658,6 +680,13 @@ class WorkflowGenerator:
|
||||
# variables before we surface them as errors.
|
||||
cls._reconcile_variable_references(nodes=nodes, mode=mode)
|
||||
|
||||
# Schema backstop: a "file" / "file-list" start variable MUST carry a
|
||||
# non-empty ``allowed_file_types`` or Studio refuses to load the draft
|
||||
# ("supported file types is required"). The builder is now told to set
|
||||
# it, but we fill safe defaults for any variable that still lacks it so
|
||||
# the generated workflow always loads and runs.
|
||||
cls._normalize_start_file_variables(nodes=nodes)
|
||||
|
||||
return cast(GraphDict, {"nodes": nodes, "edges": deduped_edges, "viewport": viewport})
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@ -693,6 +722,21 @@ class WorkflowGenerator:
|
||||
# remapping when we defensively strip hyphens out of LLM-emitted ids.
|
||||
_ID_FIELDS: ClassVar = frozenset({"start_node_id", "iteration_id", "loop_id", "parentId"})
|
||||
|
||||
# ``data`` keys whose value is a plain string list, never a
|
||||
# ``[node_id, var]`` value-selector — so the reference walker must not read
|
||||
# a 2-element one as a selector. ``default`` holds an input's default value;
|
||||
# ``options`` holds select choices; the ``allowed_file_*`` keys hold a file
|
||||
# variable's upload config (types / extensions / methods).
|
||||
_NON_SELECTOR_LIST_KEYS: ClassVar = frozenset(
|
||||
{
|
||||
"default",
|
||||
"options",
|
||||
"allowed_file_types",
|
||||
"allowed_file_extensions",
|
||||
"allowed_file_upload_methods",
|
||||
}
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def _reconcile_variable_references(cls, *, nodes: list[dict[str, Any]], mode: WorkflowGenerationMode) -> None:
|
||||
"""
|
||||
@ -747,12 +791,16 @@ class WorkflowGenerator:
|
||||
# Known selector shapes: 2-element [node_id, var] lists.
|
||||
for k, v in value.items():
|
||||
# ``value_selector`` / ``query_variable_selector`` / etc.: a
|
||||
# flat 2-element list of strings.
|
||||
# flat 2-element list of strings. Skip keys whose value is a
|
||||
# plain string list that merely HAPPENS to have two entries —
|
||||
# a 2-option ``select`` or a file variable's two allowed upload
|
||||
# methods are NOT ``[node_id, var]`` selectors and must not be
|
||||
# mistaken for references.
|
||||
if (
|
||||
isinstance(v, list)
|
||||
and len(v) == 2
|
||||
and all(isinstance(x, str) for x in v)
|
||||
and k != "default" # default values for input variables are not selectors
|
||||
and k not in cls._NON_SELECTOR_LIST_KEYS
|
||||
):
|
||||
node_id, var = v[0].strip(), v[1].strip()
|
||||
if node_id and var:
|
||||
@ -923,6 +971,96 @@ class WorkflowGenerator:
|
||||
}
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def _normalize_start_file_variables(cls, *, nodes: list[dict[str, Any]]) -> None:
|
||||
"""
|
||||
Fill the required upload config on every file / file-list start variable.
|
||||
|
||||
A start variable of type ``file`` / ``file-list`` is invalid without a
|
||||
non-empty ``allowed_file_types`` — Studio rejects the draft with
|
||||
"supported file types is required" (see the front-end validator in
|
||||
``config-var/config-modal/utils.ts``) and the workflow never runs. The
|
||||
builder prompt now documents these fields, but LLMs still drop them, so
|
||||
we backfill safe defaults here:
|
||||
|
||||
* a start variable a ``document-extractor`` consumes but that wasn't
|
||||
declared as a file type → promoted to ``file`` (or ``file-list``
|
||||
when the extractor's ``is_array_file`` is set), defaulting its
|
||||
allowed types to ``["document"]`` (what extraction needs);
|
||||
* empty / missing ``allowed_file_types`` → every standard file type;
|
||||
* ``custom`` present without ``allowed_file_extensions`` → drop
|
||||
``custom`` (it would otherwise require a non-empty extension list);
|
||||
* empty / missing ``allowed_file_upload_methods`` → local + remote;
|
||||
* ensure ``allowed_file_extensions`` is at least an empty list.
|
||||
|
||||
Idempotent: a variable that already declares valid file config is left
|
||||
untouched.
|
||||
"""
|
||||
start_node = next(
|
||||
(n for n in nodes if (n.get("data") or {}).get("type") == BuiltinNodeTypes.START),
|
||||
None,
|
||||
)
|
||||
if start_node is None:
|
||||
return
|
||||
variables = (start_node.get("data") or {}).get("variables")
|
||||
if not isinstance(variables, list):
|
||||
return
|
||||
|
||||
# Start variables a document-extractor reads → whether it wants an
|
||||
# array (file-list). These MUST be file inputs even if the builder
|
||||
# mistyped them (e.g. declared "paragraph"), or the extractor fails at
|
||||
# run time. ``["document"]`` is the right default for text extraction.
|
||||
extractor_file_vars = cls._document_extractor_start_vars(nodes=nodes, start_id=start_node.get("id", ""))
|
||||
|
||||
for var in variables:
|
||||
if not isinstance(var, dict):
|
||||
continue
|
||||
name = var.get("variable")
|
||||
if name in extractor_file_vars and var.get("type") not in _FILE_VARIABLE_TYPES:
|
||||
var["type"] = "file-list" if extractor_file_vars[name] else "file"
|
||||
var.setdefault("allowed_file_types", ["document"])
|
||||
if var.get("type") not in _FILE_VARIABLE_TYPES:
|
||||
continue
|
||||
allowed_types = var.get("allowed_file_types")
|
||||
if not isinstance(allowed_types, list) or not allowed_types:
|
||||
allowed_types = list(_DEFAULT_ALLOWED_FILE_TYPES)
|
||||
var["allowed_file_types"] = allowed_types
|
||||
# ``custom`` demands a non-empty extension list; without one, drop it
|
||||
# so the variable doesn't trip the "file extensions required" check.
|
||||
extensions = var.get("allowed_file_extensions")
|
||||
has_extensions = isinstance(extensions, list) and bool(extensions)
|
||||
if "custom" in allowed_types and not has_extensions:
|
||||
pruned = [t for t in allowed_types if t != "custom"]
|
||||
var["allowed_file_types"] = pruned or list(_DEFAULT_ALLOWED_FILE_TYPES)
|
||||
methods = var.get("allowed_file_upload_methods")
|
||||
if not isinstance(methods, list) or not methods:
|
||||
var["allowed_file_upload_methods"] = list(_DEFAULT_FILE_UPLOAD_METHODS)
|
||||
if not isinstance(var.get("allowed_file_extensions"), list):
|
||||
var["allowed_file_extensions"] = []
|
||||
|
||||
@classmethod
|
||||
def _document_extractor_start_vars(cls, *, nodes: list[dict[str, Any]], start_id: str) -> dict[str, bool]:
|
||||
"""
|
||||
Map start-variable name → ``is_array_file`` for every start variable a
|
||||
``document-extractor`` node reads via its ``variable_selector``.
|
||||
|
||||
When two extractors read the same variable we keep ``True`` (file-list)
|
||||
if any of them wants an array, since a file-list also satisfies a
|
||||
single-file read.
|
||||
"""
|
||||
out: dict[str, bool] = {}
|
||||
if not start_id:
|
||||
return out
|
||||
for node in nodes:
|
||||
data = node.get("data") or {}
|
||||
if data.get("type") != BuiltinNodeTypes.DOCUMENT_EXTRACTOR:
|
||||
continue
|
||||
selector = data.get("variable_selector")
|
||||
if isinstance(selector, list) and len(selector) == 2 and selector[0] == start_id:
|
||||
var_name = selector[1]
|
||||
out[var_name] = out.get(var_name, False) or bool(data.get("is_array_file"))
|
||||
return out
|
||||
|
||||
@classmethod
|
||||
def _fill_node_defaults(cls, node: dict[str, Any]) -> None:
|
||||
"""Ensure every node has the wrapper-level fields the Studio canvas needs."""
|
||||
|
||||
@ -10,7 +10,7 @@ from __future__ import annotations
|
||||
import enum
|
||||
import uuid
|
||||
from collections.abc import Mapping, Sequence
|
||||
from typing import Annotated, Any, ClassVar, Literal
|
||||
from typing import Annotated, Any, ClassVar, Literal, override
|
||||
|
||||
import bleach
|
||||
import markdown
|
||||
@ -158,6 +158,7 @@ class EmailDeliveryMethod(_DeliveryMethodBase):
|
||||
type: Literal[DeliveryMethodType.EMAIL] = DeliveryMethodType.EMAIL
|
||||
config: EmailDeliveryConfig
|
||||
|
||||
@override
|
||||
def extract_variable_selectors(self) -> Sequence[Sequence[str]]:
|
||||
variable_template_parser = VariableTemplateParser(template=self.config.body)
|
||||
selectors: list[Sequence[str]] = []
|
||||
|
||||
@ -195,13 +195,16 @@ class _LazyNodeTypeClassesMapping(MutableMapping[NodeType, Mapping[str, type[Nod
|
||||
snapshot.update(self._overrides)
|
||||
return snapshot
|
||||
|
||||
@override
|
||||
def __getitem__(self, key: NodeType) -> Mapping[str, type[Node]]:
|
||||
return self._snapshot()[key]
|
||||
|
||||
@override
|
||||
def __setitem__(self, key: NodeType, value: Mapping[str, type[Node]]) -> None:
|
||||
self._deleted.discard(key)
|
||||
self._overrides[key] = value
|
||||
|
||||
@override
|
||||
def __delitem__(self, key: NodeType) -> None:
|
||||
if key in self._overrides:
|
||||
del self._overrides[key]
|
||||
@ -211,9 +214,11 @@ class _LazyNodeTypeClassesMapping(MutableMapping[NodeType, Mapping[str, type[Nod
|
||||
return
|
||||
raise KeyError(key)
|
||||
|
||||
@override
|
||||
def __iter__(self) -> Iterator[NodeType]:
|
||||
return iter(self._snapshot())
|
||||
|
||||
@override
|
||||
def __len__(self) -> int:
|
||||
return len(self._snapshot())
|
||||
|
||||
@ -474,8 +479,9 @@ class DifyNodeFactory(NodeFactory):
|
||||
if issubclass(node_class, DifyAgentNode):
|
||||
from clients.agent_backend import AgentBackendRunEventAdapter, AgentBackendRunRequestBuilder
|
||||
from clients.agent_backend.factory import create_agent_backend_run_client
|
||||
from core.workflow.nodes.agent_v2.file_tenant_validator import UploadFileTenantValidator
|
||||
from core.workflow.nodes.agent_v2.file_tenant_validator import AgentOutputFileTenantValidator
|
||||
from core.workflow.nodes.agent_v2.output_failure_orchestrator import OutputFailureOrchestrator
|
||||
from core.workflow.nodes.agent_v2.output_file_rebacker import reback_tool_file_output
|
||||
from core.workflow.nodes.agent_v2.output_type_checker import PerOutputTypeChecker
|
||||
from core.workflow.nodes.agent_v2.session_store import WorkflowAgentRuntimeSessionStore
|
||||
|
||||
@ -491,11 +497,12 @@ class DifyNodeFactory(NodeFactory):
|
||||
fake_scenario=dify_config.AGENT_BACKEND_FAKE_SCENARIO,
|
||||
),
|
||||
"event_adapter": AgentBackendRunEventAdapter(),
|
||||
"output_adapter": WorkflowAgentOutputAdapter(),
|
||||
# Agent Files §4.6: reback file outputs from the ToolFile row so
|
||||
# downstream metadata is authoritative, not sandbox-provided.
|
||||
"output_adapter": WorkflowAgentOutputAdapter(tool_file_rebacker=reback_tool_file_output),
|
||||
# Stage 4 §5/§7: per-output validation + failure orchestration. The
|
||||
# tenant validator queries upload_files so it stays cheap when
|
||||
# outputs contain no file refs.
|
||||
"type_checker": PerOutputTypeChecker(file_validator=UploadFileTenantValidator()),
|
||||
# tenant validator resolves ToolFile (canonical) + UploadFile refs.
|
||||
"type_checker": PerOutputTypeChecker(file_validator=AgentOutputFileTenantValidator()),
|
||||
"failure_orchestrator": OutputFailureOrchestrator(),
|
||||
"session_store": WorkflowAgentRuntimeSessionStore(),
|
||||
}
|
||||
|
||||
@ -2,7 +2,7 @@ from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Generator, Mapping, Sequence
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any, Literal, cast, overload
|
||||
from typing import TYPE_CHECKING, Any, Literal, cast, overload, override
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
@ -136,6 +136,7 @@ class DifyFileReferenceFactory(FileReferenceFactoryProtocol):
|
||||
def __init__(self, run_context: Mapping[str, Any] | DifyRunContext) -> None:
|
||||
self._run_context = resolve_dify_run_context(run_context)
|
||||
|
||||
@override
|
||||
def build_from_mapping(self, *, mapping: Mapping[str, Any]):
|
||||
return file_factory.build_from_mapping(
|
||||
mapping=mapping,
|
||||
@ -151,25 +152,31 @@ class DifyPreparedLLM(LLMProtocol):
|
||||
self._model_instance = model_instance
|
||||
|
||||
@property
|
||||
@override
|
||||
def provider(self) -> str:
|
||||
return self._model_instance.provider
|
||||
|
||||
@property
|
||||
@override
|
||||
def model_name(self) -> str:
|
||||
return self._model_instance.model_name
|
||||
|
||||
@property
|
||||
@override
|
||||
def parameters(self) -> Mapping[str, Any]:
|
||||
return self._model_instance.parameters
|
||||
|
||||
@parameters.setter
|
||||
@override
|
||||
def parameters(self, value: Mapping[str, Any]) -> None:
|
||||
self._model_instance.parameters = value
|
||||
|
||||
@property
|
||||
@override
|
||||
def stop(self) -> Sequence[str] | None:
|
||||
return self._model_instance.stop
|
||||
|
||||
@override
|
||||
def get_model_schema(self) -> AIModelEntity:
|
||||
model_schema = cast(LargeLanguageModel, self._model_instance.model_type_instance).get_model_schema(
|
||||
self._model_instance.model_name,
|
||||
@ -179,6 +186,7 @@ class DifyPreparedLLM(LLMProtocol):
|
||||
raise ValueError(f"Model schema not found for {self._model_instance.model_name}")
|
||||
return model_schema
|
||||
|
||||
@override
|
||||
def get_llm_num_tokens(self, prompt_messages: Sequence[PromptMessage]) -> int:
|
||||
return self._model_instance.get_llm_num_tokens(prompt_messages)
|
||||
|
||||
@ -204,6 +212,7 @@ class DifyPreparedLLM(LLMProtocol):
|
||||
stream: Literal[True],
|
||||
) -> Generator[LLMResultChunk, None, None]: ...
|
||||
|
||||
@override
|
||||
def invoke_llm(
|
||||
self,
|
||||
*,
|
||||
@ -243,6 +252,7 @@ class DifyPreparedLLM(LLMProtocol):
|
||||
stream: Literal[True],
|
||||
) -> Generator[LLMResultChunkWithStructuredOutput, None, None]: ...
|
||||
|
||||
@override
|
||||
def invoke_llm_with_structured_output(
|
||||
self,
|
||||
*,
|
||||
@ -263,11 +273,13 @@ class DifyPreparedLLM(LLMProtocol):
|
||||
stream=stream,
|
||||
)
|
||||
|
||||
@override
|
||||
def is_structured_output_parse_error(self, error: Exception) -> bool:
|
||||
return isinstance(error, OutputParserError)
|
||||
|
||||
|
||||
class DifyPromptMessageSerializer(PromptMessageSerializerProtocol):
|
||||
@override
|
||||
def serialize(
|
||||
self,
|
||||
*,
|
||||
@ -294,6 +306,7 @@ class DifyRetrieverAttachmentLoader(RetrieverAttachmentLoaderProtocol):
|
||||
self._file_reference_factory = file_reference_factory
|
||||
self._segment_access_checker = segment_access_checker
|
||||
|
||||
@override
|
||||
def load(self, *, segment_id: str) -> Sequence[File]:
|
||||
if not is_retriever_segment_access_granted(segment_id):
|
||||
return []
|
||||
@ -341,6 +354,7 @@ class DifyToolFileManager(ToolFileManagerProtocol):
|
||||
self._manager = ToolFileManager()
|
||||
self._conversation_id_getter = conversation_id_getter
|
||||
|
||||
@override
|
||||
def create_file_by_raw(
|
||||
self,
|
||||
*,
|
||||
@ -358,6 +372,7 @@ class DifyToolFileManager(ToolFileManagerProtocol):
|
||||
filename=filename,
|
||||
)
|
||||
|
||||
@override
|
||||
def get_file_generator_by_tool_file_id(self, tool_file_id: str):
|
||||
return self._manager.get_file_generator_by_tool_file_id(tool_file_id)
|
||||
|
||||
@ -394,9 +409,11 @@ class DifyToolNodeRuntime(ToolNodeRuntimeProtocol):
|
||||
def file_reference_factory(self) -> FileReferenceFactoryProtocol:
|
||||
return self._file_reference_factory
|
||||
|
||||
@override
|
||||
def build_file_reference(self, *, mapping: Mapping[str, Any]):
|
||||
return self._file_reference_factory.build_from_mapping(mapping=mapping)
|
||||
|
||||
@override
|
||||
def get_runtime(
|
||||
self,
|
||||
*,
|
||||
@ -447,6 +464,7 @@ class DifyToolNodeRuntime(ToolNodeRuntimeProtocol):
|
||||
)
|
||||
)
|
||||
|
||||
@override
|
||||
def get_runtime_parameters(
|
||||
self,
|
||||
*,
|
||||
@ -458,6 +476,7 @@ class DifyToolNodeRuntime(ToolNodeRuntimeProtocol):
|
||||
for parameter in (tool.get_merged_runtime_parameters() or [])
|
||||
]
|
||||
|
||||
@override
|
||||
def invoke(
|
||||
self,
|
||||
*,
|
||||
@ -503,6 +522,7 @@ class DifyToolNodeRuntime(ToolNodeRuntimeProtocol):
|
||||
|
||||
return self._adapt_messages(transformed_messages, provider_name=provider_name)
|
||||
|
||||
@override
|
||||
def get_usage(
|
||||
self,
|
||||
*,
|
||||
@ -745,6 +765,7 @@ class DifyHumanInputNodeRuntime(HumanInputNodeRuntimeProtocol):
|
||||
form_repository=form_repository,
|
||||
)
|
||||
|
||||
@override
|
||||
def get_form(self, *, node_id: str) -> HumanInputFormStateProtocol | None:
|
||||
repo = self.build_form_repository()
|
||||
return repo.get_form(node_id)
|
||||
@ -766,6 +787,7 @@ class DifyHumanInputNodeRuntime(HumanInputNodeRuntimeProtocol):
|
||||
)
|
||||
return restored_data
|
||||
|
||||
@override
|
||||
def create_form(
|
||||
self,
|
||||
*,
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Generator, Mapping, Sequence
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from typing import TYPE_CHECKING, Any, override
|
||||
|
||||
from core.app.entities.app_invoke_entities import DIFY_RUN_CONTEXT_KEY, DifyRunContext
|
||||
from core.workflow.system_variables import SystemVariableKey, get_system_text
|
||||
@ -56,9 +56,11 @@ class AgentNode(Node[AgentNodeData]):
|
||||
self._message_transformer = message_transformer
|
||||
|
||||
@classmethod
|
||||
@override
|
||||
def version(cls) -> str:
|
||||
return "1"
|
||||
|
||||
@override
|
||||
def populate_start_event(self, event) -> None:
|
||||
dify_ctx = DifyRunContext.model_validate(self.require_run_context_value(DIFY_RUN_CONTEXT_KEY))
|
||||
event.extras["agent_strategy"] = {
|
||||
@ -69,6 +71,7 @@ class AgentNode(Node[AgentNodeData]):
|
||||
),
|
||||
}
|
||||
|
||||
@override
|
||||
def _run(self) -> Generator[NodeEventBase, None, None]:
|
||||
from core.plugin.impl.exc import PluginDaemonClientSideError
|
||||
|
||||
@ -167,6 +170,7 @@ class AgentNode(Node[AgentNodeData]):
|
||||
)
|
||||
|
||||
@classmethod
|
||||
@override
|
||||
def _extract_variable_selector_to_variable_mapping(
|
||||
cls,
|
||||
*,
|
||||
|
||||
@ -1,11 +1,14 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import override
|
||||
|
||||
from factories.agent_factory import get_plugin_agent_strategy
|
||||
|
||||
from .strategy_protocols import AgentStrategyPresentationProvider, AgentStrategyResolver, ResolvedAgentStrategy
|
||||
|
||||
|
||||
class PluginAgentStrategyResolver(AgentStrategyResolver):
|
||||
@override
|
||||
def resolve(
|
||||
self,
|
||||
*,
|
||||
@ -21,6 +24,7 @@ class PluginAgentStrategyResolver(AgentStrategyResolver):
|
||||
|
||||
|
||||
class PluginAgentStrategyPresentationProvider(AgentStrategyPresentationProvider):
|
||||
@override
|
||||
def get_icon(self, *, tenant_id: str, agent_strategy_provider_name: str) -> str | None:
|
||||
from core.plugin.impl.plugin import PluginInstaller
|
||||
|
||||
|
||||
@ -2,7 +2,7 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from collections.abc import Generator, Mapping, Sequence
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from typing import TYPE_CHECKING, Any, override
|
||||
|
||||
from agenton.compositor import CompositorSessionSnapshot
|
||||
|
||||
@ -101,12 +101,15 @@ class DifyAgentNode(Node[DifyAgentNodeData]):
|
||||
self._session_store = session_store
|
||||
|
||||
@classmethod
|
||||
@override
|
||||
def version(cls) -> str:
|
||||
return "2"
|
||||
|
||||
@override
|
||||
def populate_start_event(self, event) -> None:
|
||||
event.extras["agent_node"] = {"version": "2", "agent_node_kind": self.node_data.agent_node_kind}
|
||||
|
||||
@override
|
||||
def _run(self) -> Generator[NodeEventBase, None, None]:
|
||||
dify_ctx = DifyRunContext.model_validate(self.require_run_context_value(DIFY_RUN_CONTEXT_KEY))
|
||||
workflow_id = self.graph_init_params.workflow_id
|
||||
@ -309,6 +312,7 @@ class DifyAgentNode(Node[DifyAgentNodeData]):
|
||||
inputs=inputs,
|
||||
process_data=process_data,
|
||||
metadata=metadata,
|
||||
tenant_id=dify_ctx.tenant_id,
|
||||
)
|
||||
)
|
||||
return
|
||||
@ -339,6 +343,7 @@ class DifyAgentNode(Node[DifyAgentNodeData]):
|
||||
inputs=inputs,
|
||||
process_data=process_data,
|
||||
metadata=metadata,
|
||||
tenant_id=dify_ctx.tenant_id,
|
||||
)
|
||||
)
|
||||
return
|
||||
@ -577,6 +582,7 @@ class DifyAgentNode(Node[DifyAgentNodeData]):
|
||||
metadata["agent_backend"] = agent_backend
|
||||
|
||||
@classmethod
|
||||
@override
|
||||
def _extract_variable_selector_to_variable_mapping(
|
||||
cls,
|
||||
*,
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user