chore: split components

.agents/skills/frontend-code-review/SKILL.md

@@ -1,73 +0,0 @@
----
-name: frontend-code-review
-description: "Trigger when the user requests a review of frontend files (e.g., `.tsx`, `.ts`, `.js`). Support both pending-change reviews and focused file reviews while applying the checklist rules."
----
-
-# Frontend Code Review
-
-## Intent
-Use this skill whenever the user asks to review frontend code (especially `.tsx`, `.ts`, or `.js` files). Support two review modes:
-
-1. **Pending-change review** – inspect staged/working-tree files slated for commit and flag checklist violations before submission.
-2. **File-targeted review** – review the specific file(s) the user names and report the relevant checklist findings.
-
-Stick to the checklist below for every applicable file and mode.
-
-## Checklist
-See [references/code-quality.md](references/code-quality.md), [references/performance.md](references/performance.md), [references/business-logic.md](references/business-logic.md) for the living checklist split by category—treat it as the canonical set of rules to follow.
-
-Flag each rule violation with urgency metadata so future reviewers can prioritize fixes.
-
-## Review Process
-1. Open the relevant component/module. Gather lines that relate to class names, React Flow hooks, prop memoization, and styling.
-2. For each rule in the review point, note where the code deviates and capture a representative snippet.
-3. Compose the review section per the template below. Group violations first by **Urgent** flag, then by category order (Code Quality, Performance, Business Logic).
-
-## Required output
-When invoked, the response must exactly follow one of the two templates:
-
-### Template A (any findings)
-```
-# Code review
-Found <N> urgent issues need to be fixed:
-
-## 1 <brief description of bug>
-FilePath: <path> line <line>
-<relevant code snippet or pointer>
-
-
-### Suggested fix
-<brief description of suggested fix>
-
----
-... (repeat for each urgent issue) ...
-
-Found <M> suggestions for improvement:
-
-## 1 <brief description of suggestion>
-FilePath: <path> line <line>
-<relevant code snippet or pointer>
-
-
-### Suggested fix
-<brief description of suggested fix>
-
----
-
-... (repeat for each suggestion) ...
-```
-
-If there are no urgent issues, omit that section. If there are no suggestions, omit that section.
-
-If the issue number is more than 10, summarize as "10+ urgent issues" or "10+ suggestions" and just output the first 10 issues.
-
-Don't compress the blank lines between sections; keep them as-is for readability.
-
-If you use Template A (i.e., there are issues to fix) and at least one issue requires code changes, append a brief follow-up question after the structured output asking whether the user wants you to apply the suggested fix(es). For example: "Would you like me to use the Suggested fix section to address these issues?"
-
-### Template B (no issues)
-```
-## Code review
-No issues found.
-```
-

.agents/skills/frontend-code-review/references/business-logic.md

@@ -1,15 +0,0 @@
-# Rule Catalog — Business Logic
-
-## Can't use workflowStore in Node components
-
-IsUrgent: True
-
-### Description
-
-File path pattern of node components: `web/app/components/workflow/nodes/[nodeName]/node.tsx`
-
-Node components are also used when creating a RAG Pipe from a template, but in that context there is no workflowStore Provider, which results in a blank screen. [This Issue](https://github.com/langgenius/dify/issues/29168) was caused by exactly this reason.
-
-### Suggested Fix
-
-Use `import { useNodes } from 'reactflow'` instead of `import useNodes from '@/app/components/workflow/store/workflow/use-nodes'`.

.agents/skills/frontend-code-review/references/code-quality.md

@@ -1,44 +0,0 @@
-# Rule Catalog — Code Quality
-
-## Conditional class names use utility function
-
-IsUrgent: True
-Category: Code Quality
-
-### Description
-
-Ensure conditional CSS is handled via the shared `classNames` instead of custom ternaries, string concatenation, or template strings. Centralizing class logic keeps components consistent and easier to maintain.
-
-### Suggested Fix
-
-```ts
-import { cn } from '@/utils/classnames'
-const classNames = cn(isActive ? 'text-primary-600' : 'text-gray-500')
-```
-
-## Tailwind-first styling
-
-IsUrgent: True
-Category: Code Quality
-
-### Description
-
-Favor Tailwind CSS utility classes instead of adding new `.module.css` files unless a Tailwind combination cannot achieve the required styling. Keeping styles in Tailwind improves consistency and reduces maintenance overhead.
-
-Update this file when adding, editing, or removing Code Quality rules so the catalog remains accurate.
-
-## Classname ordering for easy overrides
-
-### Description
-
-When writing components, always place the incoming `className` prop after the component’s own class values so that downstream consumers can override or extend the styling. This keeps your component’s defaults but still lets external callers change or remove specific styles.
-
-Example:
-
-```tsx
-import { cn } from '@/utils/classnames'
-
-const Button = ({ className }) => {
-  return <div className={cn('bg-primary-600', className)}></div>
-}
-```

.agents/skills/frontend-code-review/references/performance.md

@@ -1,45 +0,0 @@
-# Rule Catalog — Performance
-
-## React Flow data usage
-
-IsUrgent: True
-Category: Performance
-
-### Description
-
-When rendering React Flow, prefer `useNodes`/`useEdges` for UI consumption and rely on `useStoreApi` inside callbacks that mutate or read node/edge state. Avoid manually pulling Flow data outside of these hooks.
-
-## Complex prop memoization
-
-IsUrgent: True
-Category: Performance
-
-### Description
-
-Wrap complex prop values (objects, arrays, maps) in `useMemo` prior to passing them into child components to guarantee stable references and prevent unnecessary renders.
-
-Update this file when adding, editing, or removing Performance rules so the catalog remains accurate.
-
-Wrong:
-
-```tsx
-<HeavyComp
-    config={{
-        provider: ...,
-        detail: ...
-    }}
-/>
-```
-
-Right:
-
-```tsx
-const config = useMemo(() => ({
-    provider: ...,
-    detail: ...
-}), [provider, detail]);
-
-<HeavyComp
-    config={config}
-/>
-```

.agents/skills/orpc-contract-first/SKILL.md

@@ -1,46 +0,0 @@
----
-name: orpc-contract-first
-description: Guide for implementing oRPC contract-first API patterns in Dify frontend. Triggers when creating new API contracts, adding service endpoints, integrating TanStack Query with typed contracts, or migrating legacy service calls to oRPC. Use for all API layer work in web/contract and web/service directories.
----
-
-# oRPC Contract-First Development
-
-## Project Structure
-
-```
-web/contract/
-├── base.ts           # Base contract (inputStructure: 'detailed')
-├── router.ts         # Router composition & type exports
-├── marketplace.ts    # Marketplace contracts
-└── console/          # Console contracts by domain
-    ├── system.ts
-    └── billing.ts
-```
-
-## Workflow
-
-1. **Create contract** in `web/contract/console/{domain}.ts`
-   - Import `base` from `../base` and `type` from `@orpc/contract`
-   - Define route with `path`, `method`, `input`, `output`
-
-2. **Register in router** at `web/contract/router.ts`
-   - Import directly from domain file (no barrel files)
-   - Nest by API prefix: `billing: { invoices, bindPartnerStack }`
-
-3. **Create hooks** in `web/service/use-{domain}.ts`
-   - Use `consoleQuery.{group}.{contract}.queryKey()` for query keys
-   - Use `consoleClient.{group}.{contract}()` for API calls
-
-## Key Rules
-
-- **Input structure**: Always use `{ params, query?, body? }` format
-- **Path params**: Use `{paramName}` in path, match in `params` object
-- **Router nesting**: Group by API prefix (e.g., `/billing/*` → `billing: {}`)
-- **No barrel files**: Import directly from specific files
-- **Types**: Import from `@/types/`, use `type<T>()` helper
-
-## Type Export
-
-```typescript
-export type ConsoleInputs = InferContractRouterInputs<typeof consoleRouterContract>
-```

.claude/settings.json

@@ -1,15 +1,8 @@
 {
-  "hooks": {
-    "PreToolUse": [
-      {
-        "matcher": "Bash",
-        "hooks": [
-          {
-            "type": "command",
-            "command": "npx -y block-no-verify@1.1.1"
-          }
-        ]
-      }
-    ]
+  "enabledPlugins": {
+    "feature-dev@claude-plugins-official": true,
+    "context7@claude-plugins-official": true,
+    "typescript-lsp@claude-plugins-official": true,
+    "pyright-lsp@claude-plugins-official": true
   }
 }

.claude/skills/component-refactoring

@@ -1 +0,0 @@
-../../.agents/skills/component-refactoring

.claude/skills/component-refactoring/SKILL.md

@@ -480,4 +480,4 @@ const useButtonState = () => {
 ### Related Skills
 
 - `frontend-testing` - For testing refactored components
-- `web/docs/test.md` - Testing specification
+- `web/testing/testing.md` - Testing specification

.claude/skills/frontend-code-review

@@ -1 +0,0 @@
-../../.agents/skills/frontend-code-review

.claude/skills/frontend-testing

@@ -1 +0,0 @@
-../../.agents/skills/frontend-testing

.claude/skills/frontend-testing/SKILL.md

@@ -7,7 +7,7 @@ description: Generate Vitest + React Testing Library tests for Dify frontend com
 
 This skill enables Claude to generate high-quality, comprehensive frontend tests for the Dify project following established conventions and best practices.
 
-> **⚠️ Authoritative Source**: This skill is derived from `web/docs/test.md`. Use Vitest mock/timer APIs (`vi.*`).
+> **⚠️ Authoritative Source**: This skill is derived from `web/testing/testing.md`. Use Vitest mock/timer APIs (`vi.*`).
 
 ## When to Apply This Skill
 
@@ -83,9 +83,6 @@ vi.mock('next/navigation', () => ({
   usePathname: () => '/test',
 }))
 
-// ✅ Zustand stores: Use real stores (auto-mocked globally)
-// Set test state with: useAppStore.setState({ ... })
-
 // Shared state for mocks (if needed)
 let mockSharedState = false
 
@@ -299,7 +296,7 @@ For each test file generated, aim for:
 For more detailed information, refer to:
 
 - `references/workflow.md` - **Incremental testing workflow** (MUST READ for multi-file testing)
-- `references/mocking.md` - Mock patterns, Zustand store testing, and best practices
+- `references/mocking.md` - Mock patterns and best practices
 - `references/async-testing.md` - Async operations and API calls
 - `references/domain-components.md` - Workflow, Dataset, Configuration testing
 - `references/common-patterns.md` - Frequently used testing patterns
@@ -309,7 +306,7 @@ For more detailed information, refer to:
 
 ### Primary Specification (MUST follow)
 
-- **`web/docs/test.md`** - The canonical testing specification. This skill is derived from this document.
+- **`web/testing/testing.md`** - The canonical testing specification. This skill is derived from this document.
 
 ### Reference Examples in Codebase
 

.claude/skills/frontend-testing/assets/component-test.template.tsx

@@ -28,14 +28,17 @@ import userEvent from '@testing-library/user-event'
 
 // i18n (automatically mocked)
 // WHY: Global mock in web/vitest.setup.ts is auto-loaded by Vitest setup
-// The global mock provides: useTranslation, Trans, useMixedTranslation, useGetLanguage
-// No explicit mock needed for most tests
-//
+// No explicit mock needed - it returns translation keys as-is
 // Override only if custom translations are required:
-// import { createReactI18nextMock } from '@/test/i18n-mock'
-// vi.mock('react-i18next', () => createReactI18nextMock({
-//   'my.custom.key': 'Custom Translation',
-//   'button.save': 'Save',
+// vi.mock('react-i18next', () => ({
+//   useTranslation: () => ({
+//     t: (key: string) => {
+//       const customTranslations: Record<string, string> = {
+//         'my.custom.key': 'Custom Translation',
+//       }
+//       return customTranslations[key] || key
+//     },
+//   }),
 // }))
 
 // Router (if component uses useRouter, usePathname, useSearchParams)

.claude/skills/frontend-testing/references/mocking.md

@@ -37,64 +37,38 @@ Only mock these categories:
 1. **Third-party libraries with side effects** - `next/navigation`, external SDKs
 1. **i18n** - Always mock to return keys
 
-### Zustand Stores - DO NOT Mock Manually
-
-**Zustand is globally mocked** in `web/vitest.setup.ts`. Use real stores with `setState()`:
-
-```typescript
-// ✅ CORRECT: Use real store, set test state
-import { useAppStore } from '@/app/components/app/store'
-
-useAppStore.setState({ appDetail: { id: 'test', name: 'Test' } })
-render(<MyComponent />)
-
-// ❌ WRONG: Don't mock the store module
-vi.mock('@/app/components/app/store', () => ({ ... }))
-```
-
-See [Zustand Store Testing](#zustand-store-testing) section for full details.
-
 ## Mock Placement
 
 | Location | Purpose |
 |----------|---------|
-| `web/vitest.setup.ts` | Global mocks shared by all tests (`react-i18next`, `next/image`, `zustand`) |
-| `web/__mocks__/zustand.ts` | Zustand mock implementation (auto-resets stores after each test) |
+| `web/vitest.setup.ts` | Global mocks shared by all tests (for example `react-i18next`, `next/image`) |
 | `web/__mocks__/` | Reusable mock factories shared across multiple test files |
 | Test file | Test-specific mocks, inline with `vi.mock()` |
 
 Modules are not mocked automatically. Use `vi.mock` in test files, or add global mocks in `web/vitest.setup.ts`.
 
-**Note**: Zustand is special - it's globally mocked but you should NOT mock store modules manually. See [Zustand Store Testing](#zustand-store-testing).
-
 ## Essential Mocks
 
 ### 1. i18n (Auto-loaded via Global Mock)
 
 A global mock is defined in `web/vitest.setup.ts` and is auto-loaded by Vitest setup.
+**No explicit mock needed** for most tests - it returns translation keys as-is.
 
-The global mock provides:
-
-- `useTranslation` - returns translation keys with namespace prefix
-- `Trans` component - renders i18nKey and components
-- `useMixedTranslation` (from `@/app/components/plugins/marketplace/hooks`)
-- `useGetLanguage` (from `@/context/i18n`) - returns `'en-US'`
-
-**Default behavior**: Most tests should use the global mock (no local override needed).
-
-**For custom translations**: Use the helper function from `@/test/i18n-mock`:
+For tests requiring custom translations, override the mock:
 
 ```typescript
-import { createReactI18nextMock } from '@/test/i18n-mock'
-
-vi.mock('react-i18next', () => createReactI18nextMock({
-  'my.custom.key': 'Custom translation',
-  'button.save': 'Save',
+vi.mock('react-i18next', () => ({
+  useTranslation: () => ({
+    t: (key: string) => {
+      const translations: Record<string, string> = {
+        'my.custom.key': 'Custom translation',
+      }
+      return translations[key] || key
+    },
+  }),
 }))
 ```
 
-**Avoid**: Manually defining `useTranslation` mocks that just return the key - the global mock already does this.
-
 ### 2. Next.js Router
 
 ```typescript
@@ -296,7 +270,6 @@ const renderWithQueryClient = (ui: React.ReactElement) => {
 
 1. **Use real base components** - Import from `@/app/components/base/` directly
 1. **Use real project components** - Prefer importing over mocking
-1. **Use real Zustand stores** - Set test state via `store.setState()`
 1. **Reset mocks in `beforeEach`**, not `afterEach`
 1. **Match actual component behavior** in mocks (when mocking is necessary)
 1. **Use factory functions** for complex mock data
@@ -306,7 +279,6 @@ const renderWithQueryClient = (ui: React.ReactElement) => {
 ### ❌ DON'T
 
 1. **Don't mock base components** (`Loading`, `Button`, `Tooltip`, etc.)
-1. **Don't mock Zustand store modules** - Use real stores with `setState()`
 1. Don't mock components you can import directly
 1. Don't create overly simplified mocks that miss conditional logic
 1. Don't forget to clean up nock after each test
@@ -330,151 +302,10 @@ Need to use a component in test?
 ├─ Is it a third-party lib with side effects?
 │  └─ YES → Mock it (next/navigation, external SDKs)
 │
-├─ Is it a Zustand store?
-│  └─ YES → DO NOT mock the module!
-│           Use real store + setState() to set test state
-│           (Global mock handles auto-reset)
-│
 └─ Is it i18n?
    └─ YES → Uses shared mock (auto-loaded). Override only for custom translations
 ```
 
-## Zustand Store Testing
-
-### Global Zustand Mock (Auto-loaded)
-
-Zustand is globally mocked in `web/vitest.setup.ts` following the [official Zustand testing guide](https://zustand.docs.pmnd.rs/guides/testing). The mock in `web/__mocks__/zustand.ts` provides:
-
-- Real store behavior with `getState()`, `setState()`, `subscribe()` methods
-- Automatic store reset after each test via `afterEach`
-- Proper test isolation between tests
-
-### ✅ Recommended: Use Real Stores (Official Best Practice)
-
-**DO NOT mock store modules manually.** Import and use the real store, then use `setState()` to set test state:
-
-```typescript
-// ✅ CORRECT: Use real store with setState
-import { useAppStore } from '@/app/components/app/store'
-
-describe('MyComponent', () => {
-  it('should render app details', () => {
-    // Arrange: Set test state via setState
-    useAppStore.setState({
-      appDetail: {
-        id: 'test-app',
-        name: 'Test App',
-        mode: 'chat',
-      },
-    })
-
-    // Act
-    render(<MyComponent />)
-
-    // Assert
-    expect(screen.getByText('Test App')).toBeInTheDocument()
-    // Can also verify store state directly
-    expect(useAppStore.getState().appDetail?.name).toBe('Test App')
-  })
-
-  // No cleanup needed - global mock auto-resets after each test
-})
-```
-
-### ❌ Avoid: Manual Store Module Mocking
-
-Manual mocking conflicts with the global Zustand mock and loses store functionality:
-
-```typescript
-// ❌ WRONG: Don't mock the store module
-vi.mock('@/app/components/app/store', () => ({
-  useStore: (selector) => mockSelector(selector),  // Missing getState, setState!
-}))
-
-// ❌ WRONG: This conflicts with global zustand mock
-vi.mock('@/app/components/workflow/store', () => ({
-  useWorkflowStore: vi.fn(() => mockState),
-}))
-```
-
-**Problems with manual mocking:**
-
-1. Loses `getState()`, `setState()`, `subscribe()` methods
-1. Conflicts with global Zustand mock behavior
-1. Requires manual maintenance of store API
-1. Tests don't reflect actual store behavior
-
-### When Manual Store Mocking is Necessary
-
-In rare cases where the store has complex initialization or side effects, you can mock it, but ensure you provide the full store API:
-
-```typescript
-// If you MUST mock (rare), include full store API
-const mockStore = {
-  appDetail: { id: 'test', name: 'Test' },
-  setAppDetail: vi.fn(),
-}
-
-vi.mock('@/app/components/app/store', () => ({
-  useStore: Object.assign(
-    (selector: (state: typeof mockStore) => unknown) => selector(mockStore),
-    {
-      getState: () => mockStore,
-      setState: vi.fn(),
-      subscribe: vi.fn(),
-    },
-  ),
-}))
-```
-
-### Store Testing Decision Tree
-
-```
-Need to test a component using Zustand store?
-│
-├─ Can you use the real store?
-│  └─ YES → Use real store + setState (RECOMMENDED)
-│           useAppStore.setState({ ... })
-│
-├─ Does the store have complex initialization/side effects?
-│  └─ YES → Consider mocking, but include full API
-│           (getState, setState, subscribe)
-│
-└─ Are you testing the store itself (not a component)?
-   └─ YES → Test store directly with getState/setState
-            const store = useMyStore
-            store.setState({ count: 0 })
-            store.getState().increment()
-            expect(store.getState().count).toBe(1)
-```
-
-### Example: Testing Store Actions
-
-```typescript
-import { useCounterStore } from '@/stores/counter'
-
-describe('Counter Store', () => {
-  it('should increment count', () => {
-    // Initial state (auto-reset by global mock)
-    expect(useCounterStore.getState().count).toBe(0)
-
-    // Call action
-    useCounterStore.getState().increment()
-
-    // Verify state change
-    expect(useCounterStore.getState().count).toBe(1)
-  })
-
-  it('should reset to initial state', () => {
-    // Set some state
-    useCounterStore.setState({ count: 100 })
-    expect(useCounterStore.getState().count).toBe(100)
-
-    // After this test, global mock will reset to initial state
-  })
-})
-```
-
 ## Factory Function Pattern
 
 ```typescript

.claude/skills/frontend-testing/references/workflow.md

@@ -4,7 +4,7 @@ This guide defines the workflow for generating tests, especially for complex com
 
 ## Scope Clarification
 
-This guide addresses **multi-file workflow** (how to process multiple test files). For coverage requirements within a single test file, see `web/docs/test.md` § Coverage Goals.
+This guide addresses **multi-file workflow** (how to process multiple test files). For coverage requirements within a single test file, see `web/testing/testing.md` § Coverage Goals.
 
 | Scope | Rule |
 |-------|------|

.claude/skills/orpc-contract-first

@@ -1 +0,0 @@
-../../.agents/skills/orpc-contract-first

.codex/skills

@@ -0,0 +1 @@
+../.claude/skills

.codex/skills/component-refactoring

@@ -1 +0,0 @@
-../../.agents/skills/component-refactoring

.codex/skills/frontend-code-review

@@ -1 +0,0 @@
-../../.agents/skills/frontend-code-review

.codex/skills/frontend-testing

@@ -1 +0,0 @@
-../../.agents/skills/frontend-testing

.codex/skills/orpc-contract-first

@@ -1 +0,0 @@
-../../.agents/skills/orpc-contract-first

.devcontainer/post_create_command.sh

@@ -8,7 +8,7 @@ pipx install uv
 
 echo "alias start-api=\"cd $WORKSPACE_ROOT/api && uv run python -m flask run --host 0.0.0.0 --port=5001 --debug\"" >> ~/.bashrc
 echo "alias start-worker=\"cd $WORKSPACE_ROOT/api && uv run python -m celery -A app.celery worker -P threads -c 1 --loglevel INFO -Q dataset,priority_dataset,priority_pipeline,pipeline,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation,workflow,schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executor,retention\"" >> ~/.bashrc
-echo "alias start-web=\"cd $WORKSPACE_ROOT/web && pnpm dev:inspect\"" >> ~/.bashrc
+echo "alias start-web=\"cd $WORKSPACE_ROOT/web && pnpm dev\"" >> ~/.bashrc
 echo "alias start-web-prod=\"cd $WORKSPACE_ROOT/web && pnpm build && pnpm start\"" >> ~/.bashrc
 echo "alias start-containers=\"cd $WORKSPACE_ROOT/docker && docker-compose -f docker-compose.middleware.yaml -p dify --env-file middleware.env up -d\"" >> ~/.bashrc
 echo "alias stop-containers=\"cd $WORKSPACE_ROOT/docker && docker-compose -f docker-compose.middleware.yaml -p dify --env-file middleware.env down\"" >> ~/.bashrc

.github/labeler.yml

@@ -1,3 +0,0 @@
-web:
-  - changed-files:
-      - any-glob-to-any-file: 'web/**'

.github/pull_request_template.md

@@ -20,4 +20,4 @@
 - [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
 - [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
 - [x] I've updated the documentation accordingly.
-- [x] I ran `make lint` and `make type-check` (backend) and `cd web && npx lint-staged` (frontend) to appease the lint gods
+- [x] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods

.github/workflows/api-tests.yml

@@ -39,6 +39,12 @@ jobs:
       - name: Install dependencies
         run: uv sync --project api --dev
 
+      - name: Run pyrefly check
+        run: |
+          cd api
+          uv add --dev pyrefly
+          uv run pyrefly check || true
+
       - name: Run dify config tests
         run: uv run --project api dev/pytest/pytest_config_tests.py
 
@@ -72,7 +78,6 @@ jobs:
           OPENDAL_FS_ROOT: /tmp/dify-storage
         run: |
           uv run --project api pytest \
-            -n auto \
             --timeout "${PYTEST_TIMEOUT:-180}" \
             api/tests/integration_tests/workflow \
             api/tests/integration_tests/tools \

.github/workflows/autofix.yml

@@ -16,14 +16,14 @@ jobs:
 
       - name: Check Docker Compose inputs
         id: docker-compose-changes
-        uses: tj-actions/changed-files@v47
+        uses: tj-actions/changed-files@v46
         with:
           files: |
             docker/generate_docker_compose
             docker/.env.example
             docker/docker-compose-template.yaml
             docker/docker-compose.yaml
-      - uses: actions/setup-python@v6
+      - uses: actions/setup-python@v5
         with:
           python-version: "3.11"
 
@@ -79,32 +79,9 @@ jobs:
           find . -name "*.py" -type f -exec sed -i.bak -E 's/"([^"]+)" \| None/Optional["\1"]/g; s/'"'"'([^'"'"']+)'"'"' \| None/Optional['"'"'\1'"'"']/g' {} \;
           find . -name "*.py.bak" -type f -delete
 
-      - name: Install pnpm
-        uses: pnpm/action-setup@v4
-        with:
-          package_json_file: web/package.json
-          run_install: false
-
-      - name: Setup Node.js
-        uses: actions/setup-node@v6
-        with:
-          node-version: 24
-          cache: pnpm
-          cache-dependency-path: ./web/pnpm-lock.yaml
-
-      - name: Install web dependencies
-        run: |
-          cd web
-          pnpm install --frozen-lockfile
-
-      - name: ESLint autofix
-        run: |
-          cd web
-          pnpm lint:fix || true
-
       # mdformat breaks YAML front matter in markdown files. Add --exclude for directories containing YAML front matter.
       - name: mdformat
         run: |
-          uvx --python 3.13 mdformat . --exclude ".agents/skills/**"
+          uvx --python 3.13 mdformat . --exclude ".claude/skills/**/SKILL.md"
 
       - uses: autofix-ci/action@635ffb0c9798bd160680f18fd73371e355b85f27

.github/workflows/build-push.yml

@@ -75,9 +75,7 @@ jobs:
         with:
           context: "{{defaultContext}}:${{ matrix.context }}"
           platforms: ${{ matrix.platform }}
-          build-args: |
-            COMMIT_SHA=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.revision'] }}
-            ENABLE_PROD_SOURCEMAP=${{ matrix.context == 'web' && github.ref_name == 'deploy/dev' }}
+          build-args: COMMIT_SHA=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.revision'] }}
           labels: ${{ steps.meta.outputs.labels }}
           outputs: type=image,name=${{ env[matrix.image_name_env] }},push-by-digest=true,name-canonical=true,push=true
           cache-from: type=gha,scope=${{ matrix.service_name }}
@@ -114,7 +112,7 @@ jobs:
             context: "web"
     steps:
       - name: Download digests
-        uses: actions/download-artifact@v7
+        uses: actions/download-artifact@v4
         with:
           path: /tmp/digests
           pattern: digests-${{ matrix.context }}-*

.github/workflows/deploy-dev.yml

@@ -16,7 +16,7 @@ jobs:
       github.event.workflow_run.head_branch == 'deploy/dev'
     steps:
       - name: Deploy to server
-        uses: appleboy/ssh-action@v1
+        uses: appleboy/ssh-action@v0.1.8
         with:
           host: ${{ secrets.SSH_HOST }}
           username: ${{ secrets.SSH_USER }}

.github/workflows/deploy-hitl.yml

@@ -1,29 +0,0 @@
-name: Deploy HITL
-
-on:
-  workflow_run:
-    workflows: ["Build and Push API & Web"]
-    branches:
-      - "feat/hitl-frontend"
-      - "feat/hitl-backend"
-    types:
-      - completed
-
-jobs:
-  deploy:
-    runs-on: ubuntu-latest
-    if: |
-      github.event.workflow_run.conclusion == 'success' &&
-      (
-        github.event.workflow_run.head_branch == 'feat/hitl-frontend' ||
-        github.event.workflow_run.head_branch == 'feat/hitl-backend'
-      )
-    steps:
-      - name: Deploy to server
-        uses: appleboy/ssh-action@v1
-        with:
-          host: ${{ secrets.HITL_SSH_HOST }}
-          username: ${{ secrets.SSH_USER }}
-          key: ${{ secrets.SSH_PRIVATE_KEY }}
-          script: |
-            ${{ vars.SSH_SCRIPT || secrets.SSH_SCRIPT }}

.github/workflows/deploy-trigger-dev.yml

@@ -1,4 +1,4 @@
-name: Deploy Agent Dev
+name: Deploy Trigger Dev
 
 permissions:
   contents: read
@@ -7,7 +7,7 @@ on:
   workflow_run:
     workflows: ["Build and Push API & Web"]
     branches:
-      - "deploy/agent-dev"
+      - "deploy/trigger-dev"
     types:
       - completed
 
@@ -16,12 +16,12 @@ jobs:
     runs-on: ubuntu-latest
     if: |
       github.event.workflow_run.conclusion == 'success' &&
-      github.event.workflow_run.head_branch == 'deploy/agent-dev'
+      github.event.workflow_run.head_branch == 'deploy/trigger-dev'
     steps:
       - name: Deploy to server
-        uses: appleboy/ssh-action@v1
+        uses: appleboy/ssh-action@v0.1.8
         with:
-          host: ${{ secrets.AGENT_DEV_SSH_HOST }}
+          host: ${{ secrets.TRIGGER_SSH_HOST }}
           username: ${{ secrets.SSH_USER }}
           key: ${{ secrets.SSH_PRIVATE_KEY }}
           script: |

.github/workflows/labeler.yml

@@ -1,14 +0,0 @@
-name: "Pull Request Labeler"
-on:
-  pull_request_target:
-
-jobs:
-  labeler:
-    permissions:
-      contents: read
-      pull-requests: write
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/labeler@v6
-        with:
-          sync-labels: true

.github/workflows/stale.yml

@@ -18,7 +18,7 @@ jobs:
       pull-requests: write
 
     steps:
-      - uses: actions/stale@v10
+      - uses: actions/stale@v5
         with:
           days-before-issue-stale: 15
           days-before-issue-close: 3

.github/workflows/style.yml

@@ -47,9 +47,13 @@ jobs:
         if: steps.changed-files.outputs.any_changed == 'true'
         run: uv run --directory api --dev lint-imports
 
-      - name: Run Type Checks
+      - name: Run Basedpyright Checks
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: make type-check
+        run: dev/basedpyright-check
+
+      - name: Run Mypy Type Checks
+        if: steps.changed-files.outputs.any_changed == 'true'
+        run: uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --check-untyped-defs --disable-error-code=import-untyped .
 
       - name: Dotenv check
         if: steps.changed-files.outputs.any_changed == 'true'
@@ -61,9 +65,6 @@ jobs:
     defaults:
       run:
         working-directory: ./web
-    permissions:
-      checks: write
-      pull-requests: read
 
     steps:
       - name: Checkout code
@@ -89,7 +90,7 @@ jobs:
         uses: actions/setup-node@v6
         if: steps.changed-files.outputs.any_changed == 'true'
         with:
-          node-version: 24
+          node-version: 22
           cache: pnpm
           cache-dependency-path: ./web/pnpm-lock.yaml
 
@@ -102,31 +103,12 @@ jobs:
         if: steps.changed-files.outputs.any_changed == 'true'
         working-directory: ./web
         run: |
-          pnpm run lint:ci
-        # pnpm run lint:report
-        # continue-on-error: true
-
-      # - name: Annotate Code
-      #   if: steps.changed-files.outputs.any_changed == 'true' && github.event_name == 'pull_request'
-      #   uses: DerLev/eslint-annotations@51347b3a0abfb503fc8734d5ae31c4b151297fae
-      #   with:
-      #     eslint-report: web/eslint_report.json
-      #     github-token: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Web tsslint
-        if: steps.changed-files.outputs.any_changed == 'true'
-        working-directory: ./web
-        run: pnpm run lint:tss
+          pnpm run lint
 
       - name: Web type check
         if: steps.changed-files.outputs.any_changed == 'true'
         working-directory: ./web
-        run: pnpm run type-check
-
-      - name: Web dead code check
-        if: steps.changed-files.outputs.any_changed == 'true'
-        working-directory: ./web
-        run: pnpm run knip
+        run: pnpm run type-check:tsgo
 
   superlinter:
     name: SuperLinter

.github/workflows/tool-test-sdks.yaml

@@ -16,6 +16,10 @@ jobs:
     name: unit test for Node.js SDK
     runs-on: ubuntu-latest
 
+    strategy:
+      matrix:
+        node-version: [16, 18, 20, 22]
+
     defaults:
       run:
         working-directory: sdks/nodejs-client
@@ -25,10 +29,10 @@ jobs:
         with:
           persist-credentials: false
 
-      - name: Use Node.js
+      - name: Use Node.js ${{ matrix.node-version }}
         uses: actions/setup-node@v6
         with:
-          node-version: 24
+          node-version: ${{ matrix.node-version }}
           cache: ''
           cache-dependency-path: 'pnpm-lock.yaml'
 

.github/workflows/translate-i18n-base-on-english.yml

@@ -0,0 +1,85 @@
+name: Translate i18n Files Based on English
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - 'web/i18n/en-US/*.json'
+
+permissions:
+  contents: write
+  pull-requests: write
+
+jobs:
+  check-and-update:
+    if: github.repository == 'langgenius/dify'
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: web
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Check for file changes in i18n/en-US
+        id: check_files
+        run: |
+          git fetch origin "${{ github.event.before }}" || true
+          git fetch origin "${{ github.sha }}" || true
+          changed_files=$(git diff --name-only "${{ github.event.before }}" "${{ github.sha }}" -- 'i18n/en-US/*.json')
+          echo "Changed files: $changed_files"
+          if [ -n "$changed_files" ]; then
+            echo "FILES_CHANGED=true" >> $GITHUB_ENV
+            file_args=""
+            for file in $changed_files; do
+              filename=$(basename "$file" .json)
+              file_args="$file_args --file $filename"
+            done
+            echo "FILE_ARGS=$file_args" >> $GITHUB_ENV
+            echo "File arguments: $file_args"
+          else
+            echo "FILES_CHANGED=false" >> $GITHUB_ENV
+          fi
+
+      - name: Install pnpm
+        uses: pnpm/action-setup@v4
+        with:
+          package_json_file: web/package.json
+          run_install: false
+
+      - name: Set up Node.js
+        if: env.FILES_CHANGED == 'true'
+        uses: actions/setup-node@v6
+        with:
+          node-version: 'lts/*'
+          cache: pnpm
+          cache-dependency-path: ./web/pnpm-lock.yaml
+
+      - name: Install dependencies
+        if: env.FILES_CHANGED == 'true'
+        working-directory: ./web
+        run: pnpm install --frozen-lockfile
+
+      - name: Generate i18n translations
+        if: env.FILES_CHANGED == 'true'
+        working-directory: ./web
+        run: pnpm run auto-gen-i18n ${{ env.FILE_ARGS }}
+
+      - name: Create Pull Request
+        if: env.FILES_CHANGED == 'true'
+        uses: peter-evans/create-pull-request@v6
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          commit-message: 'chore(i18n): update translations based on en-US changes'
+          title: 'chore(i18n): translate i18n files based on en-US changes'
+          body: |
+            This PR was automatically created to update i18n translation files based on changes in en-US locale.
+
+            **Triggered by:** ${{ github.sha }}
+
+            **Changes included:**
+            - Updated translation files for all locales
+          branch: chore/automated-i18n-updates-${{ github.sha }}
+          delete-branch: true

.github/workflows/translate-i18n-claude.yml

@@ -1,440 +0,0 @@
-name: Translate i18n Files with Claude Code
-
-# Note: claude-code-action doesn't support push events directly.
-# Push events are handled by trigger-i18n-sync.yml which sends repository_dispatch.
-# See: https://github.com/langgenius/dify/issues/30743
-
-on:
-  repository_dispatch:
-    types: [i18n-sync]
-  workflow_dispatch:
-    inputs:
-      files:
-        description: 'Specific files to translate (space-separated, e.g., "app common"). Leave empty for all files.'
-        required: false
-        type: string
-      languages:
-        description: 'Specific languages to translate (space-separated, e.g., "zh-Hans ja-JP"). Leave empty for all supported languages.'
-        required: false
-        type: string
-      mode:
-        description: 'Sync mode: incremental (only changes) or full (re-check all keys)'
-        required: false
-        default: 'incremental'
-        type: choice
-        options:
-          - incremental
-          - full
-
-permissions:
-  contents: write
-  pull-requests: write
-
-jobs:
-  translate:
-    if: github.repository == 'langgenius/dify'
-    runs-on: ubuntu-latest
-    timeout-minutes: 60
-
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v6
-        with:
-          fetch-depth: 0
-          token: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Configure Git
-        run: |
-          git config --global user.name "github-actions[bot]"
-          git config --global user.email "github-actions[bot]@users.noreply.github.com"
-
-      - name: Install pnpm
-        uses: pnpm/action-setup@v4
-        with:
-          package_json_file: web/package.json
-          run_install: false
-
-      - name: Set up Node.js
-        uses: actions/setup-node@v6
-        with:
-          node-version: 24
-          cache: pnpm
-          cache-dependency-path: ./web/pnpm-lock.yaml
-
-      - name: Detect changed files and generate diff
-        id: detect_changes
-        run: |
-          if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
-            # Manual trigger
-            if [ -n "${{ github.event.inputs.files }}" ]; then
-              echo "CHANGED_FILES=${{ github.event.inputs.files }}" >> $GITHUB_OUTPUT
-            else
-              # Get all JSON files in en-US directory
-              files=$(ls web/i18n/en-US/*.json 2>/dev/null | xargs -n1 basename | sed 's/.json$//' | tr '\n' ' ')
-              echo "CHANGED_FILES=$files" >> $GITHUB_OUTPUT
-            fi
-            echo "TARGET_LANGS=${{ github.event.inputs.languages }}" >> $GITHUB_OUTPUT
-            echo "SYNC_MODE=${{ github.event.inputs.mode || 'incremental' }}" >> $GITHUB_OUTPUT
-
-            # For manual trigger with incremental mode, get diff from last commit
-            # For full mode, we'll do a complete check anyway
-            if [ "${{ github.event.inputs.mode }}" == "full" ]; then
-              echo "Full mode: will check all keys" > /tmp/i18n-diff.txt
-              echo "DIFF_AVAILABLE=false" >> $GITHUB_OUTPUT
-            else
-              git diff HEAD~1..HEAD -- 'web/i18n/en-US/*.json' > /tmp/i18n-diff.txt 2>/dev/null || echo "" > /tmp/i18n-diff.txt
-              if [ -s /tmp/i18n-diff.txt ]; then
-                echo "DIFF_AVAILABLE=true" >> $GITHUB_OUTPUT
-              else
-                echo "DIFF_AVAILABLE=false" >> $GITHUB_OUTPUT
-              fi
-            fi
-          elif [ "${{ github.event_name }}" == "repository_dispatch" ]; then
-            # Triggered by push via trigger-i18n-sync.yml workflow
-            # Validate required payload fields
-            if [ -z "${{ github.event.client_payload.changed_files }}" ]; then
-              echo "Error: repository_dispatch payload missing required 'changed_files' field" >&2
-              exit 1
-            fi
-            echo "CHANGED_FILES=${{ github.event.client_payload.changed_files }}" >> $GITHUB_OUTPUT
-            echo "TARGET_LANGS=" >> $GITHUB_OUTPUT
-            echo "SYNC_MODE=${{ github.event.client_payload.sync_mode || 'incremental' }}" >> $GITHUB_OUTPUT
-
-            # Decode the base64-encoded diff from the trigger workflow
-            if [ -n "${{ github.event.client_payload.diff_base64 }}" ]; then
-              if ! echo "${{ github.event.client_payload.diff_base64 }}" | base64 -d > /tmp/i18n-diff.txt 2>&1; then
-                echo "Warning: Failed to decode base64 diff payload" >&2
-                echo "" > /tmp/i18n-diff.txt
-                echo "DIFF_AVAILABLE=false" >> $GITHUB_OUTPUT
-              elif [ -s /tmp/i18n-diff.txt ]; then
-                echo "DIFF_AVAILABLE=true" >> $GITHUB_OUTPUT
-              else
-                echo "DIFF_AVAILABLE=false" >> $GITHUB_OUTPUT
-              fi
-            else
-              echo "" > /tmp/i18n-diff.txt
-              echo "DIFF_AVAILABLE=false" >> $GITHUB_OUTPUT
-            fi
-          else
-            echo "Unsupported event type: ${{ github.event_name }}"
-            exit 1
-          fi
-
-          # Truncate diff if too large (keep first 50KB)
-          if [ -f /tmp/i18n-diff.txt ]; then
-            head -c 50000 /tmp/i18n-diff.txt > /tmp/i18n-diff-truncated.txt
-            mv /tmp/i18n-diff-truncated.txt /tmp/i18n-diff.txt
-          fi
-
-          echo "Detected files: $(cat $GITHUB_OUTPUT | grep CHANGED_FILES || echo 'none')"
-
-      - name: Run Claude Code for Translation Sync
-        if: steps.detect_changes.outputs.CHANGED_FILES != ''
-        uses: anthropics/claude-code-action@v1
-        with:
-          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
-          github_token: ${{ secrets.GITHUB_TOKEN }}
-          # Allow github-actions bot to trigger this workflow via repository_dispatch
-          # See: https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
-          allowed_bots: 'github-actions[bot]'
-          prompt: |
-            You are a professional i18n synchronization engineer for the Dify project.
-            Your task is to keep all language translations in sync with the English source (en-US).
-
-            ## CRITICAL TOOL RESTRICTIONS
-            - Use **Read** tool to read files (NOT cat or bash)
-            - Use **Edit** tool to modify JSON files (NOT node, jq, or bash scripts)
-            - Use **Bash** ONLY for: git commands, gh commands, pnpm commands
-            - Run bash commands ONE BY ONE, never combine with && or ||
-            - NEVER use `$()` command substitution - it's not supported. Split into separate commands instead.
-
-            ## WORKING DIRECTORY & ABSOLUTE PATHS
-            Claude Code sandbox working directory may vary. Always use absolute paths:
-            - For pnpm: `pnpm --dir ${{ github.workspace }}/web <command>`
-            - For git: `git -C ${{ github.workspace }} <command>`
-            - For gh: `gh --repo ${{ github.repository }} <command>`
-            - For file paths: `${{ github.workspace }}/web/i18n/`
-
-            ## EFFICIENCY RULES
-            - **ONE Edit per language file** - batch all key additions into a single Edit
-            - Insert new keys at the beginning of JSON (after `{`), lint:fix will sort them
-            - Translate ALL keys for a language mentally first, then do ONE Edit
-
-            ## Context
-            - Changed/target files: ${{ steps.detect_changes.outputs.CHANGED_FILES }}
-            - Target languages (empty means all supported): ${{ steps.detect_changes.outputs.TARGET_LANGS }}
-            - Sync mode: ${{ steps.detect_changes.outputs.SYNC_MODE }}
-            - Translation files are located in: ${{ github.workspace }}/web/i18n/{locale}/{filename}.json
-            - Language configuration is in: ${{ github.workspace }}/web/i18n-config/languages.ts
-            - Git diff is available: ${{ steps.detect_changes.outputs.DIFF_AVAILABLE }}
-
-            ## CRITICAL DESIGN: Verify First, Then Sync
-
-            You MUST follow this three-phase approach:
-
-            ═══════════════════════════════════════════════════════════════
-            ║  PHASE 1: VERIFY - Analyze and Generate Change Report       ║
-            ═══════════════════════════════════════════════════════════════
-
-            ### Step 1.1: Analyze Git Diff (for incremental mode)
-            Use the Read tool to read `/tmp/i18n-diff.txt` to see the git diff.
-
-            Parse the diff to categorize changes:
-            - Lines with `+` (not `+++`): Added or modified values
-            - Lines with `-` (not `---`): Removed or old values
-            - Identify specific keys for each category:
-              * ADD: Keys that appear only in `+` lines (new keys)
-              * UPDATE: Keys that appear in both `-` and `+` lines (value changed)
-              * DELETE: Keys that appear only in `-` lines (removed keys)
-
-            ### Step 1.2: Read Language Configuration
-            Use the Read tool to read `${{ github.workspace }}/web/i18n-config/languages.ts`.
-            Extract all languages with `supported: true`.
-
-            ### Step 1.3: Run i18n:check for Each Language
-            ```bash
-            pnpm --dir ${{ github.workspace }}/web install --frozen-lockfile
-            ```
-            ```bash
-            pnpm --dir ${{ github.workspace }}/web run i18n:check
-            ```
-
-            This will report:
-            - Missing keys (need to ADD)
-            - Extra keys (need to DELETE)
-
-            ### Step 1.4: Generate Change Report
-
-            Create a structured report identifying:
-            ```
-            ╔══════════════════════════════════════════════════════════════╗
-            ║                    I18N SYNC CHANGE REPORT                   ║
-            ╠══════════════════════════════════════════════════════════════╣
-            ║ Files to process: [list]                                     ║
-            ║ Languages to sync: [list]                                    ║
-            ╠══════════════════════════════════════════════════════════════╣
-            ║ ADD (New Keys):                                              ║
-            ║   - [filename].[key]: "English value"                        ║
-            ║   ...                                                        ║
-            ╠══════════════════════════════════════════════════════════════╣
-            ║ UPDATE (Modified Keys - MUST re-translate):                  ║
-            ║   - [filename].[key]: "Old value" → "New value"              ║
-            ║   ...                                                        ║
-            ╠══════════════════════════════════════════════════════════════╣
-            ║ DELETE (Extra Keys):                                         ║
-            ║   - [language]/[filename].[key]                              ║
-            ║   ...                                                        ║
-            ╚══════════════════════════════════════════════════════════════╝
-            ```
-
-            **IMPORTANT**: For UPDATE detection, compare git diff to find keys where
-            the English value changed. These MUST be re-translated even if target
-            language already has a translation (it's now stale!).
-
-            ═══════════════════════════════════════════════════════════════
-            ║  PHASE 2: SYNC - Execute Changes Based on Report            ║
-            ═══════════════════════════════════════════════════════════════
-
-            ### Step 2.1: Process ADD Operations (BATCH per language file)
-
-            **CRITICAL WORKFLOW for efficiency:**
-            1. First, translate ALL new keys for ALL languages mentally
-            2. Then, for EACH language file, do ONE Edit operation:
-               - Read the file once
-               - Insert ALL new keys at the beginning (right after the opening `{`)
-               - Don't worry about alphabetical order - lint:fix will sort them later
-
-            Example Edit (adding 3 keys to zh-Hans/app.json):
-            ```
-            old_string: '{\n  "accessControl"'
-            new_string: '{\n  "newKey1": "translation1",\n  "newKey2": "translation2",\n  "newKey3": "translation3",\n  "accessControl"'
-            ```
-
-            **IMPORTANT**:
-            - ONE Edit per language file (not one Edit per key!)
-            - Always use the Edit tool. NEVER use bash scripts, node, or jq.
-
-            ### Step 2.2: Process UPDATE Operations
-
-            **IMPORTANT: Special handling for zh-Hans and ja-JP**
-            If zh-Hans or ja-JP files were ALSO modified in the same push:
-            - Run: `git -C ${{ github.workspace }} diff HEAD~1 --name-only` and check for zh-Hans or ja-JP files
-            - If found, it means someone manually translated them. Apply these rules:
-
-            1. **Missing keys**: Still ADD them (completeness required)
-            2. **Existing translations**: Compare with the NEW English value:
-               - If translation is **completely wrong** or **unrelated** → Update it
-               - If translation is **roughly correct** (captures the meaning) → Keep it, respect manual work
-               - When in doubt, **keep the manual translation**
-
-            Example:
-            - English changed: "Save" → "Save Changes"
-            - Manual translation: "保存更改" → Keep it (correct meaning)
-            - Manual translation: "删除" → Update it (completely wrong)
-
-            For other languages:
-            Use Edit tool to replace the old value with the new translation.
-            You can batch multiple updates in one Edit if they are adjacent.
-
-            ### Step 2.3: Process DELETE Operations
-            For extra keys reported by i18n:check:
-            - Run: `pnpm --dir ${{ github.workspace }}/web run i18n:check --auto-remove`
-            - Or manually remove from target language JSON files
-
-            ## Translation Guidelines
-
-            - PRESERVE all placeholders exactly as-is:
-              - `{{variable}}` - Mustache interpolation
-              - `${variable}` - Template literal
-              - `<tag>content</tag>` - HTML tags
-              - `_one`, `_other` - Pluralization suffixes (these are KEY suffixes, not values)
-
-              **CRITICAL: Variable names and tag names MUST stay in English - NEVER translate them**
-
-              ✅ CORRECT examples:
-              - English: "{{count}} items" → Japanese: "{{count}} 個のアイテム"
-              - English: "{{name}} updated" → Korean: "{{name}} 업데이트됨"
-              - English: "<email>{{email}}</email>" → Chinese: "<email>{{email}}</email>"
-              - English: "<CustomLink>Marketplace</CustomLink>" → Japanese: "<CustomLink>マーケットプレイス</CustomLink>"
-
-              ❌ WRONG examples (NEVER do this - will break the application):
-              - "{{count}}" → "{{カウント}}" ❌ (variable name translated to Japanese)
-              - "{{name}}" → "{{이름}}" ❌ (variable name translated to Korean)
-              - "{{email}}" → "{{邮箱}}" ❌ (variable name translated to Chinese)
-              - "<email>" → "<メール>" ❌ (tag name translated)
-              - "<CustomLink>" → "<自定义链接>" ❌ (component name translated)
-
-            - Use appropriate language register (formal/informal) based on existing translations
-            - Match existing translation style in each language
-            - Technical terms: check existing conventions per language
-            - For CJK languages: no spaces between characters unless necessary
-            - For RTL languages (ar-TN, fa-IR): ensure proper text handling
-
-            ## Output Format Requirements
-            - Alphabetical key ordering (if original file uses it)
-            - 2-space indentation
-            - Trailing newline at end of file
-            - Valid JSON (use proper escaping for special characters)
-
-            ═══════════════════════════════════════════════════════════════
-            ║  PHASE 3: RE-VERIFY - Confirm All Issues Resolved           ║
-            ═══════════════════════════════════════════════════════════════
-
-            ### Step 3.1: Run Lint Fix (IMPORTANT!)
-            ```bash
-            pnpm --dir ${{ github.workspace }}/web lint:fix --quiet -- 'i18n/**/*.json'
-            ```
-            This ensures:
-            - JSON keys are sorted alphabetically (jsonc/sort-keys rule)
-            - Valid i18n keys (dify-i18n/valid-i18n-keys rule)
-            - No extra keys (dify-i18n/no-extra-keys rule)
-
-            ### Step 3.2: Run Final i18n Check
-            ```bash
-            pnpm --dir ${{ github.workspace }}/web run i18n:check
-            ```
-
-            ### Step 3.3: Fix Any Remaining Issues
-            If check reports issues:
-            - Go back to PHASE 2 for unresolved items
-            - Repeat until check passes
-
-            ### Step 3.4: Generate Final Summary
-            ```
-            ╔══════════════════════════════════════════════════════════════╗
-            ║                    SYNC COMPLETED SUMMARY                    ║
-            ╠══════════════════════════════════════════════════════════════╣
-            ║ Language      │ Added │ Updated │ Deleted │ Status          ║
-            ╠══════════════════════════════════════════════════════════════╣
-            ║ zh-Hans       │  5    │   2     │    1    │ ✓ Complete      ║
-            ║ ja-JP         │  5    │   2     │    1    │ ✓ Complete      ║
-            ║ ...           │ ...   │  ...    │   ...   │ ...             ║
-            ╠══════════════════════════════════════════════════════════════╣
-            ║ i18n:check    │ PASSED - All keys in sync                   ║
-            ╚══════════════════════════════════════════════════════════════╝
-            ```
-
-            ## Mode-Specific Behavior
-
-            **SYNC_MODE = "incremental"** (default):
-            - Focus on keys identified from git diff
-            - Also check i18n:check output for any missing/extra keys
-            - Efficient for small changes
-
-            **SYNC_MODE = "full"**:
-            - Compare ALL keys between en-US and each language
-            - Run i18n:check to identify all discrepancies
-            - Use for first-time sync or fixing historical issues
-
-            ## Important Notes
-
-            1. Always run i18n:check BEFORE and AFTER making changes
-            2. The check script is the source of truth for missing/extra keys
-            3. For UPDATE scenario: git diff is the source of truth for changed values
-            4. Create a single commit with all translation changes
-            5. If any translation fails, continue with others and report failures
-
-            ═══════════════════════════════════════════════════════════════
-            ║  PHASE 4: COMMIT AND CREATE PR                              ║
-            ═══════════════════════════════════════════════════════════════
-
-            After all translations are complete and verified:
-
-            ### Step 4.1: Check for changes
-            ```bash
-            git -C ${{ github.workspace }} status --porcelain
-            ```
-
-            If there are changes:
-
-            ### Step 4.2: Create a new branch and commit
-            Run these git commands ONE BY ONE (not combined with &&).
-            **IMPORTANT**: Do NOT use `$()` command substitution. Use two separate commands:
-
-            1. First, get the timestamp:
-            ```bash
-            date +%Y%m%d-%H%M%S
-            ```
-            (Note the output, e.g., "20260115-143052")
-
-            2. Then create branch using the timestamp value:
-            ```bash
-            git -C ${{ github.workspace }} checkout -b chore/i18n-sync-20260115-143052
-            ```
-            (Replace "20260115-143052" with the actual timestamp from step 1)
-
-            3. Stage changes:
-            ```bash
-            git -C ${{ github.workspace }} add web/i18n/
-            ```
-
-            4. Commit:
-            ```bash
-            git -C ${{ github.workspace }} commit -m "chore(i18n): sync translations with en-US - Mode: ${{ steps.detect_changes.outputs.SYNC_MODE }}"
-            ```
-
-            5. Push:
-            ```bash
-            git -C ${{ github.workspace }} push origin HEAD
-            ```
-
-            ### Step 4.3: Create Pull Request
-            ```bash
-            gh pr create --repo ${{ github.repository }} --title "chore(i18n): sync translations with en-US" --body "## Summary
-
-            This PR was automatically generated to sync i18n translation files.
-
-            ### Changes
-            - Mode: ${{ steps.detect_changes.outputs.SYNC_MODE }}
-            - Files processed: ${{ steps.detect_changes.outputs.CHANGED_FILES }}
-
-            ### Verification
-            - [x] \`i18n:check\` passed
-            - [x] \`lint:fix\` applied
-
-            🤖 Generated with Claude Code GitHub Action" --base main
-            ```
-
-          claude_args: |
-            --max-turns 150
-            --allowedTools "Read,Write,Edit,Bash(git *),Bash(git:*),Bash(gh *),Bash(gh:*),Bash(pnpm *),Bash(pnpm:*),Bash(date *),Bash(date:*),Glob,Grep"

.github/workflows/trigger-i18n-sync.yml

@@ -1,66 +0,0 @@
-name: Trigger i18n Sync on Push
-
-# This workflow bridges the push event to repository_dispatch
-# because claude-code-action doesn't support push events directly.
-# See: https://github.com/langgenius/dify/issues/30743
-
-on:
-  push:
-    branches: [main]
-    paths:
-      - 'web/i18n/en-US/*.json'
-
-permissions:
-  contents: write
-
-jobs:
-  trigger:
-    if: github.repository == 'langgenius/dify'
-    runs-on: ubuntu-latest
-    timeout-minutes: 5
-
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v6
-        with:
-          fetch-depth: 0
-
-      - name: Detect changed files and generate diff
-        id: detect
-        run: |
-          BEFORE_SHA="${{ github.event.before }}"
-          # Handle edge case: force push may have null/zero SHA
-          if [ -z "$BEFORE_SHA" ] || [ "$BEFORE_SHA" = "0000000000000000000000000000000000000000" ]; then
-            BEFORE_SHA="HEAD~1"
-          fi
-
-          # Detect changed i18n files
-          changed=$(git diff --name-only "$BEFORE_SHA" "${{ github.sha }}" -- 'web/i18n/en-US/*.json' 2>/dev/null | xargs -n1 basename 2>/dev/null | sed 's/.json$//' | tr '\n' ' ' || echo "")
-          echo "changed_files=$changed" >> $GITHUB_OUTPUT
-
-          # Generate diff for context
-          git diff "$BEFORE_SHA" "${{ github.sha }}" -- 'web/i18n/en-US/*.json' > /tmp/i18n-diff.txt 2>/dev/null || echo "" > /tmp/i18n-diff.txt
-
-          # Truncate if too large (keep first 50KB to match receiving workflow)
-          head -c 50000 /tmp/i18n-diff.txt > /tmp/i18n-diff-truncated.txt
-          mv /tmp/i18n-diff-truncated.txt /tmp/i18n-diff.txt
-
-          # Base64 encode the diff for safe JSON transport (portable, single-line)
-          diff_base64=$(base64 < /tmp/i18n-diff.txt | tr -d '\n')
-          echo "diff_base64=$diff_base64" >> $GITHUB_OUTPUT
-
-          if [ -n "$changed" ]; then
-            echo "has_changes=true" >> $GITHUB_OUTPUT
-            echo "Detected changed files: $changed"
-          else
-            echo "has_changes=false" >> $GITHUB_OUTPUT
-            echo "No i18n changes detected"
-          fi
-
-      - name: Trigger i18n sync workflow
-        if: steps.detect.outputs.has_changes == 'true'
-        uses: peter-evans/repository-dispatch@v3
-        with:
-          token: ${{ secrets.GITHUB_TOKEN }}
-          event-type: i18n-sync
-          client-payload: '{"changed_files": "${{ steps.detect.outputs.changed_files }}", "diff_base64": "${{ steps.detect.outputs.diff_base64 }}", "sync_mode": "incremental", "trigger_sha": "${{ github.sha }}"}'

.github/workflows/web-tests.yml

@@ -31,7 +31,7 @@ jobs:
       - name: Setup Node.js
         uses: actions/setup-node@v6
         with:
-          node-version: 24
+          node-version: 22
           cache: pnpm
           cache-dependency-path: ./web/pnpm-lock.yaml
 
@@ -366,48 +366,3 @@ jobs:
           path: web/coverage
           retention-days: 30
           if-no-files-found: error
-
-  web-build:
-    name: Web Build
-    runs-on: ubuntu-latest
-    defaults:
-      run:
-        working-directory: ./web
-
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@v6
-        with:
-          persist-credentials: false
-
-      - name: Check changed files
-        id: changed-files
-        uses: tj-actions/changed-files@v47
-        with:
-          files: |
-            web/**
-            .github/workflows/web-tests.yml
-
-      - name: Install pnpm
-        uses: pnpm/action-setup@v4
-        with:
-          package_json_file: web/package.json
-          run_install: false
-
-      - name: Setup NodeJS
-        uses: actions/setup-node@v6
-        if: steps.changed-files.outputs.any_changed == 'true'
-        with:
-          node-version: 24
-          cache: pnpm
-          cache-dependency-path: ./web/pnpm-lock.yaml
-
-      - name: Web dependencies
-        if: steps.changed-files.outputs.any_changed == 'true'
-        working-directory: ./web
-        run: pnpm install --frozen-lockfile
-
-      - name: Web build check
-        if: steps.changed-files.outputs.any_changed == 'true'
-        working-directory: ./web
-        run: pnpm run build

.gitignore

@@ -209,7 +209,6 @@ api/.vscode
 .history
 
 .idea/
-web/migration/
 
 # pnpm
 /.pnpm-store
@@ -236,4 +235,3 @@ scripts/stress-test/reports/
 
 # settings
 *.local.json
-*.local.md

.nvmrc

@@ -0,0 +1 @@
+22.11.0

AGENTS.md

@@ -7,18 +7,27 @@ Dify is an open-source platform for developing LLM applications with an intuitiv
 The codebase is split into:
 
 - **Backend API** (`/api`): Python Flask application organized with Domain-Driven Design
-- **Frontend Web** (`/web`): Next.js application using TypeScript and React
+- **Frontend Web** (`/web`): Next.js 15 application using TypeScript and React 19
 - **Docker deployment** (`/docker`): Containerized deployment configurations
 
 ## Backend Workflow
 
-- Read `api/AGENTS.md` for details
 - Run backend CLI commands through `uv run --project api <command>`.
+
+- Before submission, all backend modifications must pass local checks: `make lint`, `make type-check`, and `uv run --project api --dev dev/pytest/pytest_unit_tests.sh`.
+
+- Use Makefile targets for linting and formatting; `make lint` and `make type-check` cover the required checks.
+
 - Integration tests are CI-only and are not expected to run in the local environment.
 
 ## Frontend Workflow
 
-- Read `web/AGENTS.md` for details
+```bash
+cd web
+pnpm lint:fix
+pnpm type-check:tsgo
+pnpm test
+```
 
 ## Testing & Quality Practices
 

CONTRIBUTING.md

@@ -77,7 +77,7 @@ How we prioritize:
 
 For setting up the frontend service, please refer to our comprehensive [guide](https://github.com/langgenius/dify/blob/main/web/README.md) in the `web/README.md` file. This document provides detailed instructions to help you set up the frontend environment properly.
 
-**Testing**: All React components must have comprehensive test coverage. See [web/docs/test.md](https://github.com/langgenius/dify/blob/main/web/docs/test.md) for the canonical frontend testing guidelines and follow every requirement described there.
+**Testing**: All React components must have comprehensive test coverage. See [web/testing/testing.md](https://github.com/langgenius/dify/blob/main/web/testing/testing.md) for the canonical frontend testing guidelines and follow every requirement described there.
 
 #### Backend
 

Makefile

@@ -60,28 +60,19 @@ check:
 	@echo "✅ Code check complete"
 
 lint:
-	@echo "🔧 Running ruff format, check with fixes, import linter, and dotenv-linter..."
-	@uv run --project api --dev ruff format ./api
-	@uv run --project api --dev ruff check --fix ./api
+	@echo "🔧 Running ruff format, check with fixes, and import linter..."
+	@uv run --project api --dev sh -c 'ruff format ./api && ruff check --fix ./api'
 	@uv run --directory api --dev lint-imports
-	@uv run --project api --dev dotenv-linter ./api/.env.example ./web/.env.example
 	@echo "✅ Linting complete"
 
 type-check:
-	@echo "📝 Running type checks (basedpyright + mypy + ty)..."
-	@./dev/basedpyright-check $(PATH_TO_CHECK)
-	@uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --check-untyped-defs --disable-error-code=import-untyped .
-	@cd api && uv run ty check
-	@echo "✅ Type checks complete"
+	@echo "📝 Running type check with basedpyright..."
+	@uv run --directory api --dev basedpyright
+	@echo "✅ Type check complete"
 
 test:
 	@echo "🧪 Running backend unit tests..."
-	@if [ -n "$(TARGET_TESTS)" ]; then \
-		echo "Target: $(TARGET_TESTS)"; \
-		uv run --project api --dev pytest $(TARGET_TESTS); \
-	else \
-		PYTEST_XDIST_ARGS="-n auto" uv run --project api --dev dev/pytest/pytest_unit_tests.sh; \
-	fi
+	@uv run --project api --dev dev/pytest/pytest_unit_tests.sh
 	@echo "✅ Tests complete"
 
 # Build Docker images
@@ -131,9 +122,9 @@ help:
 	@echo "Backend Code Quality:"
 	@echo "  make format         - Format code with ruff"
 	@echo "  make check          - Check code with ruff"
-	@echo "  make lint           - Format, fix, and lint code (ruff, imports, dotenv)"
-	@echo "  make type-check     - Run type checks (basedpyright, mypy, ty)"
-	@echo "  make test           - Run backend unit tests (or TARGET_TESTS=./api/tests/<target_tests>)"
+	@echo "  make lint           - Format and fix code with ruff"
+	@echo "  make type-check     - Run type checking with basedpyright"
+	@echo "  make test           - Run backend unit tests"
 	@echo ""
 	@echo "Docker Build Targets:"
 	@echo "  make build-web      - Build web Docker image"

api/.env.example

@@ -33,9 +33,6 @@ TRIGGER_URL=http://localhost:5001
 # The time in seconds after the signature is rejected
 FILES_ACCESS_TIMEOUT=300
 
-# Collaboration mode toggle
-ENABLE_COLLABORATION_MODE=false
-
 # Access token expiration time in minutes
 ACCESS_TOKEN_EXPIRE_MINUTES=60
 
@@ -104,15 +101,6 @@ S3_ACCESS_KEY=your-access-key
 S3_SECRET_KEY=your-secret-key
 S3_REGION=your-region
 
-# Workflow run and Conversation archive storage (S3-compatible)
-ARCHIVE_STORAGE_ENABLED=false
-ARCHIVE_STORAGE_ENDPOINT=
-ARCHIVE_STORAGE_ARCHIVE_BUCKET=
-ARCHIVE_STORAGE_EXPORT_BUCKET=
-ARCHIVE_STORAGE_ACCESS_KEY=
-ARCHIVE_STORAGE_SECRET_KEY=
-ARCHIVE_STORAGE_REGION=auto
-
 # Azure Blob Storage configuration
 AZURE_BLOB_ACCOUNT_NAME=your-account-name
 AZURE_BLOB_ACCOUNT_KEY=your-account-key
@@ -420,8 +408,6 @@ SMTP_USERNAME=123
 SMTP_PASSWORD=abc
 SMTP_USE_TLS=true
 SMTP_OPPORTUNISTIC_TLS=false
-# Optional: override the local hostname used for SMTP HELO/EHLO
-SMTP_LOCAL_HOSTNAME=
 # Sendgid configuration
 SENDGRID_API_KEY=
 # Sentry configuration
@@ -507,8 +493,6 @@ LOG_FILE_BACKUP_COUNT=5
 LOG_DATEFORMAT=%Y-%m-%d %H:%M:%S
 # Log Timezone
 LOG_TZ=UTC
-# Log output format: text or json
-LOG_OUTPUT_FORMAT=text
 # Log format
 LOG_FORMAT=%(asctime)s,%(msecs)d %(levelname)-2s [%(filename)s:%(lineno)d] %(req_id)s %(message)s
 
@@ -580,10 +564,6 @@ LOGSTORE_DUAL_WRITE_ENABLED=false
 # Enable dual-read fallback to SQL database when LogStore returns no results (default: true)
 # Useful for migration scenarios where historical data exists only in SQL database
 LOGSTORE_DUAL_READ_ENABLED=true
-# Control flag for whether to write the `graph` field to LogStore.
-# If LOGSTORE_ENABLE_PUT_GRAPH_FIELD is "true", write the full `graph` field;
-# otherwise write an empty {} instead. Defaults to writing the `graph` field.
-LOGSTORE_ENABLE_PUT_GRAPH_FIELD=true
 
 # Celery beat configuration
 CELERY_BEAT_SCHEDULER_TIME=1
@@ -594,7 +574,6 @@ ENABLE_CLEAN_UNUSED_DATASETS_TASK=false
 ENABLE_CREATE_TIDB_SERVERLESS_TASK=false
 ENABLE_UPDATE_TIDB_SERVERLESS_STATUS_TASK=false
 ENABLE_CLEAN_MESSAGES=false
-ENABLE_WORKFLOW_RUN_CLEANUP_TASK=false
 ENABLE_MAIL_CLEAN_DOCUMENT_NOTIFY_TASK=false
 ENABLE_DATASETS_QUEUE_MONITOR=false
 ENABLE_CHECK_UPGRADABLE_PLUGIN_TASK=true
@@ -620,7 +599,6 @@ PLUGIN_DAEMON_URL=http://127.0.0.1:5002
 PLUGIN_REMOTE_INSTALL_PORT=5003
 PLUGIN_REMOTE_INSTALL_HOST=localhost
 PLUGIN_MAX_PACKAGE_SIZE=15728640
-PLUGIN_MODEL_SCHEMA_CACHE_TTL=3600
 INNER_API_KEY_FOR_PLUGIN=QaHbTe77CtuXmsfyhR7+vRjI/+XbV1AaFy691iy+kGDv2Jvy0/eAh8Y1
 
 # Marketplace configuration
@@ -719,15 +697,3 @@ ANNOTATION_IMPORT_MAX_CONCURRENT=5
 SANDBOX_EXPIRED_RECORDS_CLEAN_GRACEFUL_PERIOD=21
 SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_SIZE=1000
 SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS=30
-SANDBOX_EXPIRED_RECORDS_CLEAN_TASK_LOCK_TTL=90000
-
-# Sandbox Dify CLI configuration
-# Directory containing dify CLI binaries (dify-cli-<os>-<arch>). Defaults to api/bin when unset.
-SANDBOX_DIFY_CLI_ROOT=
-
-# CLI API URL for sandbox (dify-sandbox or e2b) to call back to Dify API.
-# This URL must be accessible from the sandbox environment.
-# For local development: use http://localhost:5001 or http://127.0.0.1:5001
-# For Docker deployment: use http://api:5001 (internal Docker network)
-# For external sandbox (e.g., e2b): use a publicly accessible URL
-CLI_API_URL=http://localhost:5001

api/.importlinter

@@ -3,11 +3,9 @@ root_packages =
     core
     configs
     controllers
-    extensions
     models
     tasks
     services
-include_external_packages = True
 
 [importlinter:contract:workflow]
 name = Workflow
@@ -27,9 +25,7 @@ ignore_imports =
     core.workflow.nodes.iteration.iteration_node -> core.workflow.graph_events
     core.workflow.nodes.loop.loop_node -> core.workflow.graph_events
 
-    core.workflow.nodes.iteration.iteration_node -> core.app.workflow.node_factory
-    core.workflow.nodes.loop.loop_node -> core.app.workflow.node_factory
-
+    core.workflow.nodes.node_factory -> core.workflow.graph
     core.workflow.nodes.iteration.iteration_node -> core.workflow.graph_engine
     core.workflow.nodes.iteration.iteration_node -> core.workflow.graph
     core.workflow.nodes.iteration.iteration_node -> core.workflow.graph_engine.command_channels
@@ -37,324 +33,6 @@ ignore_imports =
     core.workflow.nodes.loop.loop_node -> core.workflow.graph
     core.workflow.nodes.loop.loop_node -> core.workflow.graph_engine.command_channels
 
-[importlinter:contract:workflow-infrastructure-dependencies]
-name = Workflow Infrastructure Dependencies
-type = forbidden
-source_modules =
-    core.workflow
-forbidden_modules =
-    extensions.ext_database
-    extensions.ext_redis
-allow_indirect_imports = True
-ignore_imports =
-    core.workflow.nodes.agent.agent_node -> extensions.ext_database
-    core.workflow.nodes.datasource.datasource_node -> extensions.ext_database
-    core.workflow.nodes.knowledge_index.knowledge_index_node -> extensions.ext_database
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> extensions.ext_database
-    core.workflow.nodes.llm.file_saver -> extensions.ext_database
-    core.workflow.nodes.llm.llm_utils -> extensions.ext_database
-    core.workflow.nodes.llm.node -> extensions.ext_database
-    core.workflow.nodes.tool.tool_node -> extensions.ext_database
-    core.workflow.graph_engine.command_channels.redis_channel -> extensions.ext_redis
-    core.workflow.graph_engine.manager -> extensions.ext_redis
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> extensions.ext_redis
-
-[importlinter:contract:workflow-external-imports]
-name = Workflow External Imports
-type = forbidden
-source_modules =
-    core.workflow
-forbidden_modules =
-    configs
-    controllers
-    extensions
-    models
-    services
-    tasks
-    core.agent
-    core.app
-    core.base
-    core.callback_handler
-    core.datasource
-    core.db
-    core.entities
-    core.errors
-    core.extension
-    core.external_data_tool
-    core.file
-    core.helper
-    core.hosting_configuration
-    core.indexing_runner
-    core.llm_generator
-    core.logging
-    core.mcp
-    core.memory
-    core.model_manager
-    core.moderation
-    core.ops
-    core.plugin
-    core.prompt
-    core.provider_manager
-    core.rag
-    core.repositories
-    core.schemas
-    core.tools
-    core.trigger
-    core.variables
-ignore_imports =
-    core.workflow.nodes.loop.loop_node -> core.app.workflow.node_factory
-    core.workflow.graph_engine.command_channels.redis_channel -> extensions.ext_redis
-    core.workflow.workflow_entry -> core.app.workflow.layers.observability
-    core.workflow.nodes.agent.agent_node -> core.model_manager
-    core.workflow.nodes.agent.agent_node -> core.provider_manager
-    core.workflow.nodes.agent.agent_node -> core.tools.tool_manager
-    core.workflow.nodes.code.code_node -> core.helper.code_executor.code_executor
-    core.workflow.nodes.datasource.datasource_node -> models.model
-    core.workflow.nodes.datasource.datasource_node -> models.tools
-    core.workflow.nodes.datasource.datasource_node -> services.datasource_provider_service
-    core.workflow.nodes.document_extractor.node -> configs
-    core.workflow.nodes.document_extractor.node -> core.file.file_manager
-    core.workflow.nodes.document_extractor.node -> core.helper.ssrf_proxy
-    core.workflow.nodes.http_request.entities -> configs
-    core.workflow.nodes.http_request.executor -> configs
-    core.workflow.nodes.http_request.executor -> core.file.file_manager
-    core.workflow.nodes.http_request.node -> configs
-    core.workflow.nodes.http_request.node -> core.tools.tool_file_manager
-    core.workflow.nodes.iteration.iteration_node -> core.app.workflow.node_factory
-    core.workflow.nodes.knowledge_index.knowledge_index_node -> core.rag.index_processor.index_processor_factory
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.rag.datasource.retrieval_service
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.rag.retrieval.dataset_retrieval
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> models.dataset
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> services.feature_service
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.model_runtime.model_providers.__base.large_language_model
-    core.workflow.nodes.llm.llm_utils -> configs
-    core.workflow.nodes.llm.llm_utils -> core.app.entities.app_invoke_entities
-    core.workflow.nodes.llm.llm_utils -> core.file.models
-    core.workflow.nodes.llm.llm_utils -> core.model_manager
-    core.workflow.nodes.llm.llm_utils -> core.model_runtime.model_providers.__base.large_language_model
-    core.workflow.nodes.llm.llm_utils -> models.model
-    core.workflow.nodes.llm.llm_utils -> models.provider
-    core.workflow.nodes.llm.llm_utils -> services.credit_pool_service
-    core.workflow.nodes.llm.node -> core.tools.signature
-    core.workflow.nodes.template_transform.template_transform_node -> configs
-    core.workflow.nodes.tool.tool_node -> core.callback_handler.workflow_tool_callback_handler
-    core.workflow.nodes.tool.tool_node -> core.tools.tool_engine
-    core.workflow.nodes.tool.tool_node -> core.tools.tool_manager
-    core.workflow.workflow_entry -> configs
-    core.workflow.workflow_entry -> models.workflow
-    core.workflow.nodes.agent.agent_node -> core.agent.entities
-    core.workflow.nodes.agent.agent_node -> core.agent.plugin_entities
-    core.workflow.nodes.base.node -> core.app.entities.app_invoke_entities
-    core.workflow.nodes.knowledge_index.knowledge_index_node -> core.app.entities.app_invoke_entities
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.app.app_config.entities
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.app.entities.app_invoke_entities
-    core.workflow.nodes.llm.node -> core.app.entities.app_invoke_entities
-    core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.app.entities.app_invoke_entities
-    core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.advanced_prompt_transform
-    core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.simple_prompt_transform
-    core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.model_runtime.model_providers.__base.large_language_model
-    core.workflow.nodes.question_classifier.question_classifier_node -> core.app.entities.app_invoke_entities
-    core.workflow.nodes.question_classifier.question_classifier_node -> core.prompt.advanced_prompt_transform
-    core.workflow.nodes.question_classifier.question_classifier_node -> core.prompt.simple_prompt_transform
-    core.workflow.nodes.start.entities -> core.app.app_config.entities
-    core.workflow.nodes.start.start_node -> core.app.app_config.entities
-    core.workflow.workflow_entry -> core.app.apps.exc
-    core.workflow.workflow_entry -> core.app.entities.app_invoke_entities
-    core.workflow.workflow_entry -> core.app.workflow.node_factory
-    core.workflow.nodes.datasource.datasource_node -> core.datasource.datasource_manager
-    core.workflow.nodes.datasource.datasource_node -> core.datasource.utils.message_transformer
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.entities.agent_entities
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.entities.model_entities
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.model_manager
-    core.workflow.nodes.llm.llm_utils -> core.entities.provider_entities
-    core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.model_manager
-    core.workflow.nodes.question_classifier.question_classifier_node -> core.model_manager
-    core.workflow.node_events.node -> core.file
-    core.workflow.nodes.agent.agent_node -> core.file
-    core.workflow.nodes.datasource.datasource_node -> core.file
-    core.workflow.nodes.datasource.datasource_node -> core.file.enums
-    core.workflow.nodes.document_extractor.node -> core.file
-    core.workflow.nodes.http_request.executor -> core.file.enums
-    core.workflow.nodes.http_request.node -> core.file
-    core.workflow.nodes.http_request.node -> core.file.file_manager
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.file.models
-    core.workflow.nodes.list_operator.node -> core.file
-    core.workflow.nodes.llm.file_saver -> core.file
-    core.workflow.nodes.llm.llm_utils -> core.variables.segments
-    core.workflow.nodes.llm.node -> core.file
-    core.workflow.nodes.llm.node -> core.file.file_manager
-    core.workflow.nodes.llm.node -> core.file.models
-    core.workflow.nodes.loop.entities -> core.variables.types
-    core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.file
-    core.workflow.nodes.protocols -> core.file
-    core.workflow.nodes.question_classifier.question_classifier_node -> core.file.models
-    core.workflow.nodes.tool.tool_node -> core.file
-    core.workflow.nodes.tool.tool_node -> core.tools.utils.message_transformer
-    core.workflow.nodes.tool.tool_node -> models
-    core.workflow.nodes.trigger_webhook.node -> core.file
-    core.workflow.runtime.variable_pool -> core.file
-    core.workflow.runtime.variable_pool -> core.file.file_manager
-    core.workflow.system_variable -> core.file.models
-    core.workflow.utils.condition.processor -> core.file
-    core.workflow.utils.condition.processor -> core.file.file_manager
-    core.workflow.workflow_entry -> core.file.models
-    core.workflow.workflow_type_encoder -> core.file.models
-    core.workflow.nodes.agent.agent_node -> models.model
-    core.workflow.nodes.code.code_node -> core.helper.code_executor.code_node_provider
-    core.workflow.nodes.code.code_node -> core.helper.code_executor.javascript.javascript_code_provider
-    core.workflow.nodes.code.code_node -> core.helper.code_executor.python3.python3_code_provider
-    core.workflow.nodes.code.entities -> core.helper.code_executor.code_executor
-    core.workflow.nodes.datasource.datasource_node -> core.variables.variables
-    core.workflow.nodes.http_request.executor -> core.helper.ssrf_proxy
-    core.workflow.nodes.http_request.node -> core.helper.ssrf_proxy
-    core.workflow.nodes.llm.file_saver -> core.helper.ssrf_proxy
-    core.workflow.nodes.llm.node -> core.helper.code_executor
-    core.workflow.nodes.template_transform.template_renderer -> core.helper.code_executor.code_executor
-    core.workflow.nodes.llm.node -> core.llm_generator.output_parser.errors
-    core.workflow.nodes.llm.node -> core.llm_generator.output_parser.structured_output
-    core.workflow.nodes.llm.node -> core.model_manager
-    core.workflow.nodes.agent.entities -> core.prompt.entities.advanced_prompt_entities
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.prompt.simple_prompt_transform
-    core.workflow.nodes.llm.entities -> core.prompt.entities.advanced_prompt_entities
-    core.workflow.nodes.llm.llm_utils -> core.prompt.entities.advanced_prompt_entities
-    core.workflow.nodes.llm.node -> core.prompt.entities.advanced_prompt_entities
-    core.workflow.nodes.llm.node -> core.prompt.utils.prompt_message_util
-    core.workflow.nodes.parameter_extractor.entities -> core.prompt.entities.advanced_prompt_entities
-    core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.entities.advanced_prompt_entities
-    core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.utils.prompt_message_util
-    core.workflow.nodes.question_classifier.entities -> core.prompt.entities.advanced_prompt_entities
-    core.workflow.nodes.question_classifier.question_classifier_node -> core.prompt.utils.prompt_message_util
-    core.workflow.nodes.knowledge_index.entities -> core.rag.retrieval.retrieval_methods
-    core.workflow.nodes.knowledge_index.knowledge_index_node -> core.rag.retrieval.retrieval_methods
-    core.workflow.nodes.knowledge_index.knowledge_index_node -> models.dataset
-    core.workflow.nodes.knowledge_index.knowledge_index_node -> services.summary_index_service
-    core.workflow.nodes.knowledge_index.knowledge_index_node -> tasks.generate_summary_index_task
-    core.workflow.nodes.knowledge_index.knowledge_index_node -> core.rag.index_processor.processor.paragraph_index_processor
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.rag.retrieval.retrieval_methods
-    core.workflow.nodes.llm.node -> models.dataset
-    core.workflow.nodes.agent.agent_node -> core.tools.utils.message_transformer
-    core.workflow.nodes.llm.file_saver -> core.tools.signature
-    core.workflow.nodes.llm.file_saver -> core.tools.tool_file_manager
-    core.workflow.nodes.tool.tool_node -> core.tools.errors
-    core.workflow.conversation_variable_updater -> core.variables
-    core.workflow.graph_engine.entities.commands -> core.variables.variables
-    core.workflow.nodes.agent.agent_node -> core.variables.segments
-    core.workflow.nodes.answer.answer_node -> core.variables
-    core.workflow.nodes.code.code_node -> core.variables.segments
-    core.workflow.nodes.code.code_node -> core.variables.types
-    core.workflow.nodes.code.entities -> core.variables.types
-    core.workflow.nodes.datasource.datasource_node -> core.variables.segments
-    core.workflow.nodes.document_extractor.node -> core.variables
-    core.workflow.nodes.document_extractor.node -> core.variables.segments
-    core.workflow.nodes.http_request.executor -> core.variables.segments
-    core.workflow.nodes.http_request.node -> core.variables.segments
-    core.workflow.nodes.iteration.iteration_node -> core.variables
-    core.workflow.nodes.iteration.iteration_node -> core.variables.segments
-    core.workflow.nodes.iteration.iteration_node -> core.variables.variables
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.variables
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.variables.segments
-    core.workflow.nodes.list_operator.node -> core.variables
-    core.workflow.nodes.list_operator.node -> core.variables.segments
-    core.workflow.nodes.llm.node -> core.variables
-    core.workflow.nodes.loop.loop_node -> core.variables
-    core.workflow.nodes.parameter_extractor.entities -> core.variables.types
-    core.workflow.nodes.parameter_extractor.exc -> core.variables.types
-    core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.variables.types
-    core.workflow.nodes.tool.tool_node -> core.variables.segments
-    core.workflow.nodes.tool.tool_node -> core.variables.variables
-    core.workflow.nodes.trigger_webhook.node -> core.variables.types
-    core.workflow.nodes.trigger_webhook.node -> core.variables.variables
-    core.workflow.nodes.variable_aggregator.entities -> core.variables.types
-    core.workflow.nodes.variable_aggregator.variable_aggregator_node -> core.variables.segments
-    core.workflow.nodes.variable_assigner.common.helpers -> core.variables
-    core.workflow.nodes.variable_assigner.common.helpers -> core.variables.consts
-    core.workflow.nodes.variable_assigner.common.helpers -> core.variables.types
-    core.workflow.nodes.variable_assigner.v1.node -> core.variables
-    core.workflow.nodes.variable_assigner.v2.helpers -> core.variables
-    core.workflow.nodes.variable_assigner.v2.node -> core.variables
-    core.workflow.nodes.variable_assigner.v2.node -> core.variables.consts
-    core.workflow.runtime.graph_runtime_state_protocol -> core.variables.segments
-    core.workflow.runtime.read_only_wrappers -> core.variables.segments
-    core.workflow.runtime.variable_pool -> core.variables
-    core.workflow.runtime.variable_pool -> core.variables.consts
-    core.workflow.runtime.variable_pool -> core.variables.segments
-    core.workflow.runtime.variable_pool -> core.variables.variables
-    core.workflow.utils.condition.processor -> core.variables
-    core.workflow.utils.condition.processor -> core.variables.segments
-    core.workflow.variable_loader -> core.variables
-    core.workflow.variable_loader -> core.variables.consts
-    core.workflow.workflow_type_encoder -> core.variables
-    core.workflow.graph_engine.manager -> extensions.ext_redis
-    core.workflow.nodes.agent.agent_node -> extensions.ext_database
-    core.workflow.nodes.datasource.datasource_node -> extensions.ext_database
-    core.workflow.nodes.knowledge_index.knowledge_index_node -> extensions.ext_database
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> extensions.ext_database
-    core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> extensions.ext_redis
-    core.workflow.nodes.llm.file_saver -> extensions.ext_database
-    core.workflow.nodes.llm.llm_utils -> extensions.ext_database
-    core.workflow.nodes.llm.node -> extensions.ext_database
-    core.workflow.nodes.tool.tool_node -> extensions.ext_database
-    core.workflow.workflow_entry -> extensions.otel.runtime
-    core.workflow.nodes.agent.agent_node -> models
-    core.workflow.nodes.base.node -> models.enums
-    core.workflow.nodes.llm.llm_utils -> models.provider_ids
-    core.workflow.nodes.llm.node -> models.model
-    core.workflow.workflow_entry -> models.enums
-    core.workflow.nodes.agent.agent_node -> services
-    core.workflow.nodes.tool.tool_node -> services
-
-[importlinter:contract:model-runtime-no-internal-imports]
-name = Model Runtime Internal Imports
-type = forbidden
-source_modules =
-    core.model_runtime
-forbidden_modules =
-    configs
-    controllers
-    extensions
-    models
-    services
-    tasks
-    core.agent
-    core.app
-    core.base
-    core.callback_handler
-    core.datasource
-    core.db
-    core.entities
-    core.errors
-    core.extension
-    core.external_data_tool
-    core.file
-    core.helper
-    core.hosting_configuration
-    core.indexing_runner
-    core.llm_generator
-    core.logging
-    core.mcp
-    core.memory
-    core.model_manager
-    core.moderation
-    core.ops
-    core.plugin
-    core.prompt
-    core.provider_manager
-    core.rag
-    core.repositories
-    core.schemas
-    core.tools
-    core.trigger
-    core.variables
-    core.workflow
-ignore_imports =
-    core.model_runtime.model_providers.__base.ai_model -> configs
-    core.model_runtime.model_providers.__base.ai_model -> extensions.ext_redis
-    core.model_runtime.model_providers.__base.large_language_model -> configs
-    core.model_runtime.model_providers.__base.text_embedding_model -> core.entities.embedding_type
-    core.model_runtime.model_providers.model_provider_factory -> configs
-    core.model_runtime.model_providers.model_provider_factory -> extensions.ext_redis
-    core.model_runtime.model_providers.model_provider_factory -> models.provider_ids
-
 [importlinter:contract:rsc]
 name = RSC
 type = layers

api/.ruff.toml

@@ -1,8 +1,4 @@
-exclude = [
-    "migrations/*",
-    ".git",
-    ".git/**",
-]
+exclude = ["migrations/*"]
 line-length = 120
 
 [format]

api/AGENTS.md

@@ -1,186 +1,62 @@
-# API Agent Guide
+# Agent Skill Index
 
-## Notes for Agent (must-check)
+Start with the section that best matches your need. Each entry lists the problems it solves plus key files/concepts so you know what to expect before opening it.
 
-Before changing any backend code under `api/`, you MUST read the surrounding docstrings and comments. These notes contain required context (invariants, edge cases, trade-offs) and are treated as part of the spec.
+______________________________________________________________________
 
-Look for:
+## Platform Foundations
 
-- The module (file) docstring at the top of a source code file
-- Docstrings on classes and functions/methods
-- Paragraph/block comments for non-obvious logic
+- **[Infrastructure Overview](agent_skills/infra.md)**\
+  When to read this:
 
-### What to write where
+  - You need to understand where a feature belongs in the architecture.
+  - You’re wiring storage, Redis, vector stores, or OTEL.
+  - You’re about to add CLI commands or async jobs.\
+    What it covers: configuration stack (`configs/app_config.py`, remote settings), storage entry points (`extensions/ext_storage.py`, `core/file/file_manager.py`), Redis conventions (`extensions/ext_redis.py`), plugin runtime topology, vector-store factory (`core/rag/datasource/vdb/*`), observability hooks, SSRF proxy usage, and core CLI commands.
 
-- Keep notes scoped: module notes cover module-wide context, class notes cover class-wide context, function/method notes cover behavioural contracts, and paragraph/block comments cover local “why”. Avoid duplicating the same content across scopes unless repetition prevents misuse.
-- **Module (file) docstring**: purpose, boundaries, key invariants, and “gotchas” that a new reader must know before editing.
-  - Include cross-links to the key collaborators (modules/services) when discovery is otherwise hard.
-  - Prefer stable facts (invariants, contracts) over ephemeral “today we…” notes.
-- **Class docstring**: responsibility, lifecycle, invariants, and how it should be used (or not used).
-  - If the class is intentionally stateful, note what state exists and what methods mutate it.
-  - If concurrency/async assumptions matter, state them explicitly.
-- **Function/method docstring**: behavioural contract.
-  - Document arguments, return shape, side effects (DB writes, external I/O, task dispatch), and raised domain exceptions.
-  - Add examples only when they prevent misuse.
-- **Paragraph/block comments**: explain *why* (trade-offs, historical constraints, surprising edge cases), not what the code already states.
-  - Keep comments adjacent to the logic they justify; delete or rewrite comments that no longer match reality.
+- **[Coding Style](agent_skills/coding_style.md)**\
+  When to read this:
 
-### Rules (must follow)
+  - You’re writing or reviewing backend code and need the authoritative checklist.
+  - You’re unsure about Pydantic validators, SQLAlchemy session usage, or logging patterns.
+  - You want the exact lint/type/test commands used in PRs.\
+    Includes: Ruff & BasedPyright commands, no-annotation policy, session examples (`with Session(db.engine, ...)`), `@field_validator` usage, logging expectations, and the rule set for file size, helpers, and package management.
 
-In this section, “notes” means module/class/function docstrings plus any relevant paragraph/block comments.
+______________________________________________________________________
 
-- **Before working**
-  - Read the notes in the area you’ll touch; treat them as part of the spec.
-  - If a docstring or comment conflicts with the current code, treat the **code as the single source of truth** and update the docstring or comment to match reality.
-  - If important intent/invariants/edge cases are missing, add them in the closest docstring or comment (module for overall scope, function for behaviour).
-- **During working**
-  - Keep the notes in sync as you discover constraints, make decisions, or change approach.
-  - If you move/rename responsibilities across modules/classes, update the affected docstrings and comments so readers can still find the “why” and the invariants.
-  - Record non-obvious edge cases, trade-offs, and the test/verification plan in the nearest docstring or comment that will stay correct.
-  - Keep the notes **coherent**: integrate new findings into the relevant docstrings and comments; avoid append-only “recent fix” / changelog-style additions.
-- **When finishing**
-  - Update the notes to reflect what changed, why, and any new edge cases/tests.
-  - Remove or rewrite any comments that could be mistaken as current guidance but no longer apply.
-  - Keep docstrings and comments concise and accurate; they are meant to prevent repeated rediscovery.
+## Plugin & Extension Development
 
-## Coding Style
+- **[Plugin Systems](agent_skills/plugin.md)**\
+  When to read this:
 
-This is the default standard for backend code in this repo. Follow it for new code and use it as the checklist when reviewing changes.
+  - You’re building or debugging a marketplace plugin.
+  - You need to know how manifests, providers, daemons, and migrations fit together.\
+    What it covers: plugin manifests (`core/plugin/entities/plugin.py`), installation/upgrade flows (`services/plugin/plugin_service.py`, CLI commands), runtime adapters (`core/plugin/impl/*` for tool/model/datasource/trigger/endpoint/agent), daemon coordination (`core/plugin/entities/plugin_daemon.py`), and how provider registries surface capabilities to the rest of the platform.
 
-### Linting & Formatting
+- **[Plugin OAuth](agent_skills/plugin_oauth.md)**\
+  When to read this:
 
-- Use Ruff for formatting and linting (follow `.ruff.toml`).
-- Keep each line under 120 characters (including spaces).
+  - You must integrate OAuth for a plugin or datasource.
+  - You’re handling credential encryption or refresh flows.\
+    Topics: credential storage, encryption helpers (`core/helper/provider_encryption.py`), OAuth client bootstrap (`services/plugin/oauth_service.py`, `services/plugin/plugin_parameter_service.py`), and how console/API layers expose the flows.
 
-### Naming Conventions
+______________________________________________________________________
 
-- Use `snake_case` for variables and functions.
-- Use `PascalCase` for classes.
-- Use `UPPER_CASE` for constants.
+## Workflow Entry & Execution
 
-### Typing & Class Layout
+- **[Trigger Concepts](agent_skills/trigger.md)**\
+  When to read this:
+  - You’re debugging why a workflow didn’t start.
+  - You’re adding a new trigger type or hook.
+  - You need to trace async execution, draft debugging, or webhook/schedule pipelines.\
+    Details: Start-node taxonomy, webhook & schedule internals (`core/workflow/nodes/trigger_*`, `services/trigger/*`), async orchestration (`services/async_workflow_service.py`, Celery queues), debug event bus, and storage/logging interactions.
 
-- Code should usually include type annotations that match the repo’s current Python version (avoid untyped public APIs and “mystery” values).
-- Prefer modern typing forms (e.g. `list[str]`, `dict[str, int]`) and avoid `Any` unless there’s a strong reason.
-- For classes, declare member variables at the top of the class body (before `__init__`) so the class shape is obvious at a glance:
+______________________________________________________________________
 
-```python
-from datetime import datetime
+## Additional Notes for Agents
 
-
-class Example:
-    user_id: str
-    created_at: datetime
-
-    def __init__(self, user_id: str, created_at: datetime) -> None:
-        self.user_id = user_id
-        self.created_at = created_at
-```
-
-### General Rules
-
-- Use Pydantic v2 conventions.
-- Use `uv` for Python package management in this repo (usually with `--project api`).
-- Prefer simple functions over small “utility classes” for lightweight helpers.
-- Avoid implementing dunder methods unless it’s clearly needed and matches existing patterns.
-- Never start long-running services as part of agent work (`uv run app.py`, `flask run`, etc.); running tests is allowed.
-- Keep files below ~800 lines; split when necessary.
-- Keep code readable and explicit—avoid clever hacks.
-
-### Architecture & Boundaries
-
-- Mirror the layered architecture: controller → service → core/domain.
-- Reuse existing helpers in `core/`, `services/`, and `libs/` before creating new abstractions.
-- Optimise for observability: deterministic control flow, clear logging, actionable errors.
-
-### Logging & Errors
-
-- Never use `print`; use a module-level logger:
-  - `logger = logging.getLogger(__name__)`
-- Include tenant/app/workflow identifiers in log context when relevant.
-- Raise domain-specific exceptions (`services/errors`, `core/errors`) and translate them into HTTP responses in controllers.
-- Log retryable events at `warning`, terminal failures at `error`.
-
-### SQLAlchemy Patterns
-
-- Models inherit from `models.base.TypeBase`; do not create ad-hoc metadata or engines.
-- Open sessions with context managers:
-
-```python
-from sqlalchemy.orm import Session
-
-with Session(db.engine, expire_on_commit=False) as session:
-    stmt = select(Workflow).where(
-        Workflow.id == workflow_id,
-        Workflow.tenant_id == tenant_id,
-    )
-    workflow = session.execute(stmt).scalar_one_or_none()
-```
-
-- Prefer SQLAlchemy expressions; avoid raw SQL unless necessary.
-- Always scope queries by `tenant_id` and protect write paths with safeguards (`FOR UPDATE`, row counts, etc.).
-- Introduce repository abstractions only for very large tables (e.g., workflow executions) or when alternative storage strategies are required.
-
-### Storage & External I/O
-
-- Access storage via `extensions.ext_storage.storage`.
-- Use `core.helper.ssrf_proxy` for outbound HTTP fetches.
-- Background tasks that touch storage must be idempotent, and should log relevant object identifiers.
-
-### Pydantic Usage
-
-- Define DTOs with Pydantic v2 models and forbid extras by default.
-- Use `@field_validator` / `@model_validator` for domain rules.
-
-Example:
-
-```python
-from pydantic import BaseModel, ConfigDict, HttpUrl, field_validator
-
-
-class TriggerConfig(BaseModel):
-    endpoint: HttpUrl
-    secret: str
-
-    model_config = ConfigDict(extra="forbid")
-
-    @field_validator("secret")
-    def ensure_secret_prefix(cls, value: str) -> str:
-        if not value.startswith("dify_"):
-            raise ValueError("secret must start with dify_")
-        return value
-```
-
-### Generics & Protocols
-
-- Use `typing.Protocol` to define behavioural contracts (e.g., cache interfaces).
-- Apply generics (`TypeVar`, `Generic`) for reusable utilities like caches or providers.
-- Validate dynamic inputs at runtime when generics cannot enforce safety alone.
-
-### Tooling & Checks
-
-Quick checks while iterating:
-
-- Format: `make format`
-- Lint (includes auto-fix): `make lint`
-- Type check: `make type-check`
-- Targeted tests: `make test TARGET_TESTS=./api/tests/<target_tests>`
-
-Before opening a PR / submitting:
-
-- `make lint`
-- `make type-check`
-- `make test`
-
-### Controllers & Services
-
-- Controllers: parse input via Pydantic, invoke services, return serialised responses; no business logic.
-- Services: coordinate repositories, providers, background tasks; keep side effects explicit.
-- Document non-obvious behaviour with concise docstrings and comments.
-
-### Miscellaneous
-
-- Use `configs.dify_config` for configuration—never read environment variables directly.
-- Maintain tenant awareness end-to-end; `tenant_id` must flow through every layer touching shared resources.
-- Queue async work through `services/async_workflow_service`; implement tasks under `tasks/` with explicit queue selection.
-- Keep experimental scripts under `dev/`; do not ship them in production builds.
+- All skill docs assume you follow the coding style guide—run Ruff/BasedPyright/tests listed there before submitting changes.
+- When you cannot find an answer in these briefs, search the codebase using the paths referenced (e.g., `core/plugin/impl/tool.py`, `services/dataset_service.py`).
+- If you run into cross-cutting concerns (tenancy, configuration, storage), check the infrastructure guide first; it links to most supporting modules.
+- Keep multi-tenancy and configuration central: everything flows through `configs.dify_config` and `tenant_id`.
+- When touching plugins or triggers, consult both the system overview and the specialised doc to ensure you adjust lifecycle, storage, and observability consistently.

api/Dockerfile

@@ -50,33 +50,16 @@ WORKDIR /app/api
 
 # Create non-root user
 ARG dify_uid=1001
-ARG NODE_MAJOR=22
-ARG NODE_PACKAGE_VERSION=22.21.0-1nodesource1
-ARG NODESOURCE_KEY_FPR=6F71F525282841EEDAF851B42F59B5F99B1BE0B4
 RUN groupadd -r -g ${dify_uid} dify && \
     useradd -r -u ${dify_uid} -g ${dify_uid} -s /bin/bash dify && \
     chown -R dify:dify /app
 
 RUN \
     apt-get update \
-    && apt-get install -y --no-install-recommends \
-        ca-certificates \
-        curl \
-        gnupg \
-    && mkdir -p /etc/apt/keyrings \
-    && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key -o /tmp/nodesource.gpg \
-    && gpg --show-keys --with-colons /tmp/nodesource.gpg \
-        | awk -F: '/^fpr:/ {print $10}' \
-        | grep -Fx "${NODESOURCE_KEY_FPR}" \
-    && gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg /tmp/nodesource.gpg \
-    && rm -f /tmp/nodesource.gpg \
-    && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_${NODE_MAJOR}.x nodistro main" \
-        > /etc/apt/sources.list.d/nodesource.list \
-    && apt-get update \
     # Install dependencies
     && apt-get install -y --no-install-recommends \
         # basic environment
-        nodejs=${NODE_PACKAGE_VERSION} \
+        curl nodejs \
         # for gmpy2 \
         libgmp-dev libmpfr-dev libmpc-dev \
         # For Security
@@ -96,8 +79,7 @@ COPY --from=packages --chown=dify:dify ${VIRTUAL_ENV} ${VIRTUAL_ENV}
 ENV PATH="${VIRTUAL_ENV}/bin:${PATH}"
 
 # Download nltk data
-RUN mkdir -p /usr/local/share/nltk_data \
-    && NLTK_DATA=/usr/local/share/nltk_data python -c "import nltk; from unstructured.nlp.tokenize import download_nltk_packages; nltk.download('punkt'); nltk.download('averaged_perceptron_tagger'); nltk.download('stopwords'); download_nltk_packages()" \
+RUN mkdir -p /usr/local/share/nltk_data && NLTK_DATA=/usr/local/share/nltk_data python -c "import nltk; nltk.download('punkt'); nltk.download('averaged_perceptron_tagger'); nltk.download('stopwords')" \
     && chmod -R 755 /usr/local/share/nltk_data
 
 ENV TIKTOKEN_CACHE_DIR=/app/api/.tiktoken_cache

api/README.md

@@ -1,6 +1,6 @@
 # Dify Backend API
 
-## Setup and Run
+## Usage
 
 > [!IMPORTANT]
 >
@@ -8,77 +8,48 @@
 > [`uv`](https://docs.astral.sh/uv/) as the package manager
 > for Dify API backend service.
 
-`uv` and `pnpm` are required to run the setup and development commands below.
+1. Start the docker-compose stack
 
-### Using scripts (recommended)
-
-The scripts resolve paths relative to their location, so you can run them from anywhere.
-
-1. Run setup (copies env files and installs dependencies).
+   The backend require some middleware, including PostgreSQL, Redis, and Weaviate, which can be started together using `docker-compose`.
 
    ```bash
-   ./dev/setup
+   cd ../docker
+   cp middleware.env.example middleware.env
+   # change the profile to mysql if you are not using postgres,change the profile to other vector database if you are not using weaviate
+   docker compose -f docker-compose.middleware.yaml --profile postgresql --profile weaviate -p dify up -d
+   cd ../api
    ```
 
-1. Review `api/.env`, `web/.env.local`, and `docker/middleware.env` values (see the `SECRET_KEY` note below).
+1. Copy `.env.example` to `.env`
 
-1. Start middleware (PostgreSQL/Redis/Weaviate).
-
-   ```bash
-   ./dev/start-docker-compose
+   ```cli
+   cp .env.example .env
    ```
 
-1. Start backend (runs migrations first).
+> [!IMPORTANT]
+>
+> When the frontend and backend run on different subdomains, set COOKIE_DOMAIN to the site’s top-level domain (e.g., `example.com`). The frontend and backend must be under the same top-level domain in order to share authentication cookies.
 
-   ```bash
-   ./dev/start-api
+1. Generate a `SECRET_KEY` in the `.env` file.
+
+   bash for Linux
+
+   ```bash for Linux
+   sed -i "/^SECRET_KEY=/c\SECRET_KEY=$(openssl rand -base64 42)" .env
    ```
 
-1. Start Dify [web](../web) service.
+   bash for Mac
 
-   ```bash
-   ./dev/start-web
+   ```bash for Mac
+   secret_key=$(openssl rand -base64 42)
+   sed -i '' "/^SECRET_KEY=/c\\
+   SECRET_KEY=${secret_key}" .env
    ```
 
-1. Set up your application by visiting `http://localhost:3000`.
+1. Create environment.
 
-1. Optional: start the worker service (async tasks, runs from `api`).
-
-   ```bash
-   ./dev/start-worker
-   ```
-
-1. Optional: start Celery Beat (scheduled tasks).
-
-   ```bash
-   ./dev/start-beat
-   ```
-
-### Manual commands
-
-<details>
-<summary>Show manual setup and run steps</summary>
-
-These commands assume you start from the repository root.
-
-1. Start the docker-compose stack.
-
-   The backend requires middleware, including PostgreSQL, Redis, and Weaviate, which can be started together using `docker-compose`.
-
-   ```bash
-   cp docker/middleware.env.example docker/middleware.env
-   # Use mysql or another vector database profile if you are not using postgres/weaviate.
-   docker compose -f docker/docker-compose.middleware.yaml --profile postgresql --profile weaviate -p dify up -d
-   ```
-
-1. Copy env files.
-
-   ```bash
-   cp api/.env.example api/.env
-   cp web/.env.example web/.env.local
-   ```
-
-1. Install UV if needed.
+   Dify API service uses [UV](https://docs.astral.sh/uv/) to manage dependencies.
+   First, you need to add the uv package manager, if you don't have it already.
 
    ```bash
    pip install uv
@@ -86,96 +57,60 @@ These commands assume you start from the repository root.
    brew install uv
    ```
 
-1. Install API dependencies.
+1. Install dependencies
 
    ```bash
-   cd api
-   uv sync --group dev
+   uv sync --dev
    ```
 
-1. Install web dependencies.
+1. Run migrate
+
+   Before the first launch, migrate the database to the latest version.
 
    ```bash
-   cd web
-   pnpm install
-   cd ..
-   ```
-
-1. Start backend (runs migrations first, in a new terminal).
-
-   ```bash
-   cd api
    uv run flask db upgrade
+   ```
+
+1. Start backend
+
+   ```bash
    uv run flask run --host 0.0.0.0 --port=5001 --debug
    ```
 
-1. Start Dify [web](../web) service (in a new terminal).
+1. Start Dify [web](../web) service.
 
-   ```bash
-   cd web
-   pnpm dev:inspect
-   ```
+1. Setup your application by visiting `http://localhost:3000`.
 
-1. Set up your application by visiting `http://localhost:3000`.
+1. If you need to handle and debug the async tasks (e.g. dataset importing and documents indexing), please start the worker service.
 
-1. Optional: start the worker service (async tasks, in a new terminal).
+```bash
+uv run celery -A app.celery worker -P threads -c 2 --loglevel INFO -Q dataset,priority_dataset,priority_pipeline,pipeline,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation,workflow,schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executor,retention
+```
 
-   ```bash
-   cd api
-   uv run celery -A app.celery worker -P threads -c 2 --loglevel INFO -Q dataset,priority_dataset,priority_pipeline,pipeline,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation,workflow,schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executor,retention
-   ```
+Additionally, if you want to debug the celery scheduled tasks, you can run the following command in another terminal to start the beat service:
 
-1. Optional: start Celery Beat (scheduled tasks, in a new terminal).
-
-   ```bash
-   cd api
-   uv run celery -A app.celery beat
-   ```
-
-</details>
-
-### Environment notes
-
-> [!IMPORTANT]
->
-> When the frontend and backend run on different subdomains, set COOKIE_DOMAIN to the site’s top-level domain (e.g., `example.com`). The frontend and backend must be under the same top-level domain in order to share authentication cookies.
-
-- Generate a `SECRET_KEY` in the `.env` file.
-
-  bash for Linux
-
-  ```bash
-  sed -i "/^SECRET_KEY=/c\\SECRET_KEY=$(openssl rand -base64 42)" .env
-  ```
-
-  bash for Mac
-
-  ```bash
-  secret_key=$(openssl rand -base64 42)
-  sed -i '' "/^SECRET_KEY=/c\\
-  SECRET_KEY=${secret_key}" .env
-  ```
+```bash
+uv run celery -A app.celery beat
+```
 
 ## Testing
 
 1. Install dependencies for both the backend and the test environment
 
    ```bash
-   cd api
-   uv sync --group dev
+   uv sync --dev
    ```
 
 1. Run the tests locally with mocked system environment variables in `tool.pytest_env` section in `pyproject.toml`, more can check [Claude.md](../CLAUDE.md)
 
    ```bash
-   cd api
    uv run pytest                           # Run all tests
    uv run pytest tests/unit_tests/         # Unit tests only
    uv run pytest tests/integration_tests/  # Integration tests
 
    # Code quality
-   ./dev/reformat               # Run all formatters and linters
-   uv run ruff check --fix ./   # Fix linting issues
-   uv run ruff format ./        # Format code
-   uv run basedpyright .        # Type checking
+   ../dev/reformat               # Run all formatters and linters
+   uv run ruff check --fix ./    # Fix linting issues
+   uv run ruff format ./         # Format code
+   uv run basedpyright .         # Type checking
    ```

api/agent-notes/configs/feature/__init__.py.md

@@ -1,9 +0,0 @@
-Summary:
-Summary:
-- Application configuration definitions, including file access settings.
-
-Invariants:
-- File access settings drive signed URL expiration and base URLs.
-
-Tests:
-- Config parsing tests under tests/unit_tests/configs.

api/agent-notes/controllers/files/__init__.py.md

@@ -1,9 +0,0 @@
-Summary:
-- Registers file-related API namespaces and routes for files service.
-- Includes app-assets and sandbox archive proxy controllers.
-
-Invariants:
-- files_ns must include all file controller modules to register routes.
-
-Tests:
-- Coverage via controller unit tests and route registration smoke checks.

api/agent-notes/controllers/files/app_assets_download.py.md

@@ -1,14 +0,0 @@
-Summary:
-- App assets download proxy endpoint (signed URL verification, stream from storage).
-
-Invariants:
-- Validates AssetPath fields (UUIDs, asset_type allowlist).
-- Verifies tenant-scoped signature and expiration before reading storage.
-- URL uses expires_at/nonce/sign query params.
-
-Edge Cases:
-- Missing files return NotFound.
-- Invalid signature or expired link returns Forbidden.
-
-Tests:
-- Verify signature validation and invalid/expired cases.

api/agent-notes/controllers/files/app_assets_upload.py.md

@@ -1,13 +0,0 @@
-Summary:
-- App assets upload proxy endpoint (signed URL verification, upload to storage).
-
-Invariants:
-- Validates AssetPath fields (UUIDs, asset_type allowlist).
-- Verifies tenant-scoped signature and expiration before writing storage.
-- URL uses expires_at/nonce/sign query params.
-
-Edge Cases:
-- Invalid signature or expired link returns Forbidden.
-
-Tests:
-- Verify signature validation and invalid/expired cases.

api/agent-notes/controllers/files/sandbox_archive.py.md

@@ -1,14 +0,0 @@
-Summary:
-- Sandbox archive upload/download proxy endpoints (signed URL verification, stream to storage).
-
-Invariants:
-- Validates tenant_id and sandbox_id UUIDs.
-- Verifies tenant-scoped signature and expiration before storage access.
-- URL uses expires_at/nonce/sign query params.
-
-Edge Cases:
-- Missing archive returns NotFound.
-- Invalid signature or expired link returns Forbidden.
-
-Tests:
-- Add unit tests for signature validation if needed.

api/agent-notes/core/app_assets/builder/file_builder.py.md

@@ -1,9 +0,0 @@
-Summary:
-Summary:
-- Collects file assets and emits FileAsset entries with storage keys.
-
-Invariants:
-- Storage keys are derived via AppAssetStorage for draft files.
-
-Tests:
-- Covered by asset build pipeline tests.

api/agent-notes/core/app_assets/builder/skill_builder.py.md

@@ -1,14 +0,0 @@
-Summary:
-Summary:
-- Builds skill artifacts from markdown assets and uploads resolved outputs.
-
-Invariants:
-- Reads draft asset content via AppAssetStorage refs.
-- Writes resolved artifacts via AppAssetStorage refs.
-- FileAsset storage keys are derived via AppAssetStorage.
-
-Edge Cases:
-- Missing or invalid JSON content yields empty skill content/metadata.
-
-Tests:
-- Build pipeline unit tests covering compile/upload paths.

api/agent-notes/core/app_assets/converters.py.md

@@ -1,9 +0,0 @@
-Summary:
-Summary:
-- Converts AppAssetFileTree to FileAsset items for packaging.
-
-Invariants:
-- Storage keys for assets are derived via AppAssetStorage.
-
-Tests:
-- Used in packaging/service tests for asset bundles.

api/agent-notes/core/app_assets/packager/zip_packager.py.md

@@ -1,14 +0,0 @@
-# Zip Packager Notes
-
-## Purpose
-- Builds a ZIP archive of asset contents stored via the configured storage backend.
-
-## Key Decisions
-- Packaging writes assets into an in-memory zip buffer returned as bytes.
-- Asset fetch + zip writing are executed via a thread pool with a lock guarding `ZipFile` writes.
-
-## Edge Cases
-- ZIP writes are serialized by the lock; storage reads still run in parallel.
-
-## Tests/Verification
-- None yet.

api/agent-notes/core/app_assets/parser/asset_parser.py.md

@@ -1,9 +0,0 @@
-Summary:
-Summary:
-- Builds AssetItem entries for asset trees using AssetPath-derived storage keys.
-
-Invariants:
-- Uses AssetPath to compute draft storage keys.
-
-Tests:
-- Covered by asset parsing and packaging tests.

api/agent-notes/core/app_assets/storage.py.md

@@ -1,20 +0,0 @@
-Summary:
-- Defines AssetPath facade + typed asset path classes for app-asset storage access.
-- Maps asset paths to storage keys and generates presigned or signed-proxy URLs.
-- Signs proxy URLs using tenant private keys and enforces expiration.
-- Exposes app_asset_storage singleton for reuse.
-
-Invariants:
-- AssetPathBase fields (tenant_id/app_id/resource_id/node_id) must be UUIDs.
-- AssetPath.from_components enforces valid types and resolved node_id presence.
-- Storage keys are derived internally via AssetPathBase.get_storage_key; callers never supply raw paths.
-- AppAssetStorage.storage returns the cached presign wrapper (not the raw storage).
-
-Edge Cases:
-- Storage backends without presign support must fall back to signed proxy URLs.
-- Signed proxy verification enforces expiration and tenant-scoped signing keys.
-- Upload URLs also fall back to signed proxy endpoints when presign is unsupported.
-- load_or_none treats SilentStorage "File Not Found" bytes as missing.
-
-Tests:
-- Unit tests for ref validation, storage key mapping, and signed URL verification.

api/agent-notes/core/app_bundle/source_zip_extractor.py.md

@@ -1,10 +0,0 @@
-Summary:
-Summary:
-- Extracts asset files from a zip and persists them into app asset storage.
-
-Invariants:
-- Rejects path traversal/absolute/backslash paths.
-- Saves extracted files via AppAssetStorage draft refs.
-
-Tests:
-- Zip security edge cases and tree construction tests.

api/agent-notes/core/sandbox/initializer/app_assets_initializer.py.md

@@ -1,9 +0,0 @@
-Summary:
-Summary:
-- Downloads published app asset zip into sandbox and extracts it.
-
-Invariants:
-- Uses AppAssetStorage to generate download URLs for build zips (internal URL).
-
-Tests:
-- Sandbox initialization integration tests.

api/agent-notes/core/sandbox/initializer/draft_app_assets_initializer.py.md

@@ -1,12 +0,0 @@
-Summary:
-Summary:
-- Downloads draft/resolved assets into sandbox for draft execution.
-
-Invariants:
-- Uses AppAssetStorage to generate download URLs for draft/resolved refs (internal URL).
-
-Edge Cases:
-- No nodes -> returns early.
-
-Tests:
-- Sandbox draft initialization tests.

api/agent-notes/core/sandbox/sandbox.py.md

@@ -1,9 +0,0 @@
-Summary:
-- Sandbox lifecycle wrapper (ready/cancel/fail signals, mount/unmount, release).
-
-Invariants:
-- wait_ready raises with the original initialization error as the cause.
-- release always attempts unmount and environment release, logging failures.
-
-Tests:
-- Covered by sandbox lifecycle/unit tests and workflow execution error handling.

api/agent-notes/core/sandbox/security/__init__.py.md

@@ -1,2 +0,0 @@
-Summary:
-- Sandbox security helper modules.

api/agent-notes/core/sandbox/security/archive_signer.py.md

@@ -1,13 +0,0 @@
-Summary:
-- Generates and verifies signed URLs for sandbox archive upload/download.
-
-Invariants:
-- tenant_id and sandbox_id must be UUIDs.
-- Signatures are tenant-scoped and include operation, expiry, and nonce.
-
-Edge Cases:
-- Missing tenant private key raises ValueError.
-- Expired or tampered signatures are rejected.
-
-Tests:
-- Add unit tests if sandbox archive signature behavior expands.

api/agent-notes/core/sandbox/storage/archive_storage.py.md

@@ -1,12 +0,0 @@
-Summary:
-- Manages sandbox archive uploads/downloads for workspace persistence.
-
-Invariants:
-- Archive storage key is sandbox/<tenant_id>/<sandbox_id>.tar.gz.
-- Signed URLs are tenant-scoped and use external files URL.
-
-Edge Cases:
-- Missing archive skips mount.
-
-Tests:
-- Covered indirectly via sandbox integration tests.

api/agent-notes/core/skill/skill_manager.py.md

@@ -1,9 +0,0 @@
-Summary:
-Summary:
-- Loads/saves skill bundles to app asset storage.
-
-Invariants:
-- Skill bundles use AppAssetStorage refs and JSON serialization.
-
-Tests:
-- Covered by skill bundle build/load unit tests.

api/agent-notes/core/virtual_environment/providers/e2b_sandbox.py.md

@@ -1,16 +0,0 @@
-# E2B Sandbox Provider Notes
-
-## Purpose
-- Implements the E2B-backed `VirtualEnvironment` provider and bootstraps sandbox metadata, file I/O, and command execution.
-
-## Key Decisions
-- Sandbox metadata is gathered during `_construct_environment` using the E2B SDK before returning `Metadata`.
-- Architecture/OS detection uses a single `uname -m -s` call split by whitespace to reduce round-trips.
-- Command execution streams stdout/stderr through `QueueTransportReadCloser`; stdin is unsupported.
-
-## Edge Cases
-- `release_environment` raises when sandbox termination fails.
-- `execute_command` runs in a background thread; consumers must read stdout/stderr until EOF.
-
-## Tests/Verification
-- None yet. Add targeted service tests when behavior changes.

api/agent-notes/services/app_asset_service.py.md

@@ -1,14 +0,0 @@
-Summary:
-- App asset CRUD, publish/build pipeline, and presigned URL generation.
-
-Invariants:
-- Asset storage access goes through AppAssetStorage + AssetPath, using app_asset_storage singleton.
-- Tree operations require tenant/app scoping and lock for mutation.
-- Asset zips are packaged via raw storage with storage keys from AppAssetStorage.
-
-Edge Cases:
-- File nodes larger than preview limit are rejected.
-- Deletion runs asynchronously; storage failures are logged.
-
-Tests:
-- Unit tests for storage URL generation and publish/build flows.

api/agent-notes/services/app_bundle_service.py.md

@@ -1,10 +0,0 @@
-Summary:
-Summary:
-- Imports app bundles, including asset extraction into app asset storage.
-
-Invariants:
-- Asset imports respect zip security checks and tenant/app scoping.
-- Draft asset packaging uses AppAssetStorage for key mapping.
-
-Tests:
-- Bundle import unit tests and zip validation coverage.

api/agent-notes/tests/unit_tests/core/app_assets/test_storage.py.md

@@ -1,6 +0,0 @@
-Summary:
-Summary:
-- Unit tests for AppAssetStorage ref validation, key mapping, and signing.
-
-Tests:
-- Covers valid/invalid refs, signature verify, expiration handling, and proxy URL generation.

api/agent_skills/coding_style.md

@@ -0,0 +1,115 @@
+## Linter
+
+- Always follow `.ruff.toml`.
+- Run `uv run ruff check --fix --unsafe-fixes`.
+- Keep each line under 100 characters (including spaces).
+
+## Code Style
+
+- `snake_case` for variables and functions.
+- `PascalCase` for classes.
+- `UPPER_CASE` for constants.
+
+## Rules
+
+- Use Pydantic v2 standard.
+- Use `uv` for package management.
+- Do not override dunder methods like `__init__`, `__iadd__`, etc.
+- Never launch services (`uv run app.py`, `flask run`, etc.); running tests under `tests/` is allowed.
+- Prefer simple functions over classes for lightweight helpers.
+- Keep files below 800 lines; split when necessary.
+- Keep code readable—no clever hacks.
+- Never use `print`; log with `logger = logging.getLogger(__name__)`.
+
+## Guiding Principles
+
+- Mirror the project’s layered architecture: controller → service → core/domain.
+- Reuse existing helpers in `core/`, `services/`, and `libs/` before creating new abstractions.
+- Optimise for observability: deterministic control flow, clear logging, actionable errors.
+
+## SQLAlchemy Patterns
+
+- Models inherit from `models.base.Base`; never create ad-hoc metadata or engines.
+
+- Open sessions with context managers:
+
+  ```python
+  from sqlalchemy.orm import Session
+
+  with Session(db.engine, expire_on_commit=False) as session:
+      stmt = select(Workflow).where(
+          Workflow.id == workflow_id,
+          Workflow.tenant_id == tenant_id,
+      )
+      workflow = session.execute(stmt).scalar_one_or_none()
+  ```
+
+- Use SQLAlchemy expressions; avoid raw SQL unless necessary.
+
+- Introduce repository abstractions only for very large tables (e.g., workflow executions) to support alternative storage strategies.
+
+- Always scope queries by `tenant_id` and protect write paths with safeguards (`FOR UPDATE`, row counts, etc.).
+
+## Storage & External IO
+
+- Access storage via `extensions.ext_storage.storage`.
+- Use `core.helper.ssrf_proxy` for outbound HTTP fetches.
+- Background tasks that touch storage must be idempotent and log the relevant object identifiers.
+
+## Pydantic Usage
+
+- Define DTOs with Pydantic v2 models and forbid extras by default.
+
+- Use `@field_validator` / `@model_validator` for domain rules.
+
+- Example:
+
+  ```python
+  from pydantic import BaseModel, ConfigDict, HttpUrl, field_validator
+
+  class TriggerConfig(BaseModel):
+      endpoint: HttpUrl
+      secret: str
+
+      model_config = ConfigDict(extra="forbid")
+
+      @field_validator("secret")
+      def ensure_secret_prefix(cls, value: str) -> str:
+          if not value.startswith("dify_"):
+              raise ValueError("secret must start with dify_")
+          return value
+  ```
+
+## Generics & Protocols
+
+- Use `typing.Protocol` to define behavioural contracts (e.g., cache interfaces).
+- Apply generics (`TypeVar`, `Generic`) for reusable utilities like caches or providers.
+- Validate dynamic inputs at runtime when generics cannot enforce safety alone.
+
+## Error Handling & Logging
+
+- Raise domain-specific exceptions (`services/errors`, `core/errors`) and translate to HTTP responses in controllers.
+- Declare `logger = logging.getLogger(__name__)` at module top.
+- Include tenant/app/workflow identifiers in log context.
+- Log retryable events at `warning`, terminal failures at `error`.
+
+## Tooling & Checks
+
+- Format/lint: `uv run --project api --dev ruff format ./api` and `uv run --project api --dev ruff check --fix --unsafe-fixes ./api`.
+- Type checks: `uv run --directory api --dev basedpyright`.
+- Tests: `uv run --project api --dev dev/pytest/pytest_unit_tests.sh`.
+- Run all of the above before submitting your work.
+
+## Controllers & Services
+
+- Controllers: parse input via Pydantic, invoke services, return serialised responses; no business logic.
+- Services: coordinate repositories, providers, background tasks; keep side effects explicit.
+- Avoid repositories unless necessary; direct SQLAlchemy usage is preferred for typical tables.
+- Document non-obvious behaviour with concise comments.
+
+## Miscellaneous
+
+- Use `configs.dify_config` for configuration—never read environment variables directly.
+- Maintain tenant awareness end-to-end; `tenant_id` must flow through every layer touching shared resources.
+- Queue async work through `services/async_workflow_service`; implement tasks under `tasks/` with explicit queue selection.
+- Keep experimental scripts under `dev/`; do not ship them in production builds.

api/agent_skills/infra.md

@@ -0,0 +1,96 @@
+## Configuration
+
+- Import `configs.dify_config` for every runtime toggle. Do not read environment variables directly.
+- Add new settings to the proper mixin inside `configs/` (deployment, feature, middleware, etc.) so they load through `DifyConfig`.
+- Remote overrides come from the optional providers in `configs/remote_settings_sources`; keep defaults in code safe when the value is missing.
+- Example: logging pulls targets from `extensions/ext_logging.py`, and model provider URLs are assembled in `services/entities/model_provider_entities.py`.
+
+## Dependencies
+
+- Runtime dependencies live in `[project].dependencies` inside `pyproject.toml`. Optional clients go into the `storage`, `tools`, or `vdb` groups under `[dependency-groups]`.
+- Always pin versions and keep the list alphabetised. Shared tooling (lint, typing, pytest) belongs in the `dev` group.
+- When code needs a new package, explain why in the PR and run `uv lock` so the lockfile stays current.
+
+## Storage & Files
+
+- Use `extensions.ext_storage.storage` for all blob IO; it already respects the configured backend.
+- Convert files for workflows with helpers in `core/file/file_manager.py`; they handle signed URLs and multimodal payloads.
+- When writing controller logic, delegate upload quotas and metadata to `services/file_service.py` instead of touching storage directly.
+- All outbound HTTP fetches (webhooks, remote files) must go through the SSRF-safe client in `core/helper/ssrf_proxy.py`; it wraps `httpx` with the allow/deny rules configured for the platform.
+
+## Redis & Shared State
+
+- Access Redis through `extensions.ext_redis.redis_client`. For locking, reuse `redis_client.lock`.
+- Prefer higher-level helpers when available: rate limits use `libs.helper.RateLimiter`, provider metadata uses caches in `core/helper/provider_cache.py`.
+
+## Models
+
+- SQLAlchemy models sit in `models/` and inherit from the shared declarative `Base` defined in `models/base.py` (metadata configured via `models/engine.py`).
+- `models/__init__.py` exposes grouped aggregates: account/tenant models, app and conversation tables, datasets, providers, workflow runs, triggers, etc. Import from there to avoid deep path churn.
+- Follow the DDD boundary: persistence objects live in `models/`, repositories under `repositories/` translate them into domain entities, and services consume those repositories.
+- When adding a table, create the model class, register it in `models/__init__.py`, wire a repository if needed, and generate an Alembic migration as described below.
+
+## Vector Stores
+
+- Vector client implementations live in `core/rag/datasource/vdb/<provider>`, with a common factory in `core/rag/datasource/vdb/vector_factory.py` and enums in `core/rag/datasource/vdb/vector_type.py`.
+- Retrieval pipelines call these providers through `core/rag/datasource/retrieval_service.py` and dataset ingestion flows in `services/dataset_service.py`.
+- The CLI helper `flask vdb-migrate` orchestrates bulk migrations using routines in `commands.py`; reuse that pattern when adding new backend transitions.
+- To add another store, mirror the provider layout, register it with the factory, and include any schema changes in Alembic migrations.
+
+## Observability & OTEL
+
+- OpenTelemetry settings live under the observability mixin in `configs/observability`. Toggle exporters and sampling via `dify_config`, not ad-hoc env reads.
+- HTTP, Celery, Redis, SQLAlchemy, and httpx instrumentation is initialised in `extensions/ext_app_metrics.py` and `extensions/ext_request_logging.py`; reuse these hooks when adding new workers or entrypoints.
+- When creating background tasks or external calls, propagate tracing context with helpers in the existing instrumented clients (e.g. use the shared `httpx` session from `core/helper/http_client_pooling.py`).
+- If you add a new external integration, ensure spans and metrics are emitted by wiring the appropriate OTEL instrumentation package in `pyproject.toml` and configuring it in `extensions/`.
+
+## Ops Integrations
+
+- Langfuse support and other tracing bridges live under `core/ops/opik_trace`. Config toggles sit in `configs/observability`, while exporters are initialised in the OTEL extensions mentioned above.
+- External monitoring services should follow this pattern: keep client code in `core/ops`, expose switches via `dify_config`, and hook initialisation in `extensions/ext_app_metrics.py` or sibling modules.
+- Before instrumenting new code paths, check whether existing context helpers (e.g. `extensions/ext_request_logging.py`) already capture the necessary metadata.
+
+## Controllers, Services, Core
+
+- Controllers only parse HTTP input and call a service method. Keep business rules in `services/`.
+- Services enforce tenant rules, quotas, and orchestration, then call into `core/` engines (workflow execution, tools, LLMs).
+- When adding a new endpoint, search for an existing service to extend before introducing a new layer. Example: workflow APIs pipe through `services/workflow_service.py` into `core/workflow`.
+
+## Plugins, Tools, Providers
+
+- In Dify a plugin is a tenant-installable bundle that declares one or more providers (tool, model, datasource, trigger, endpoint, agent strategy) plus its resource needs and version metadata. The manifest (`core/plugin/entities/plugin.py`) mirrors what you see in the marketplace documentation.
+- Installation, upgrades, and migrations are orchestrated by `services/plugin/plugin_service.py` together with helpers such as `services/plugin/plugin_migration.py`.
+- Runtime loading happens through the implementations under `core/plugin/impl/*` (tool/model/datasource/trigger/endpoint/agent). These modules normalise plugin providers so that downstream systems (`core/tools/tool_manager.py`, `services/model_provider_service.py`, `services/trigger/*`) can treat builtin and plugin capabilities the same way.
+- For remote execution, plugin daemons (`core/plugin/entities/plugin_daemon.py`, `core/plugin/impl/plugin.py`) manage lifecycle hooks, credential forwarding, and background workers that keep plugin processes in sync with the main application.
+- Acquire tool implementations through `core/tools/tool_manager.py`; it resolves builtin, plugin, and workflow-as-tool providers uniformly, injecting the right context (tenant, credentials, runtime config).
+- To add a new plugin capability, extend the relevant `core/plugin/entities` schema and register the implementation in the matching `core/plugin/impl` module rather than importing the provider directly.
+
+## Async Workloads
+
+see `agent_skills/trigger.md` for more detailed documentation.
+
+- Enqueue background work through `services/async_workflow_service.py`. It routes jobs to the tiered Celery queues defined in `tasks/`.
+- Workers boot from `celery_entrypoint.py` and execute functions in `tasks/workflow_execution_tasks.py`, `tasks/trigger_processing_tasks.py`, etc.
+- Scheduled workflows poll from `schedule/workflow_schedule_tasks.py`. Follow the same pattern if you need new periodic jobs.
+
+## Database & Migrations
+
+- SQLAlchemy models live under `models/` and map directly to migration files in `migrations/versions`.
+- Generate migrations with `uv run --project api flask db revision --autogenerate -m "<summary>"`, then review the diff; never hand-edit the database outside Alembic.
+- Apply migrations locally using `uv run --project api flask db upgrade`; production deploys expect the same history.
+- If you add tenant-scoped data, confirm the upgrade includes tenant filters or defaults consistent with the service logic touching those tables.
+
+## CLI Commands
+
+- Maintenance commands from `commands.py` are registered on the Flask CLI. Run them via `uv run --project api flask <command>`.
+- Use the built-in `db` commands from Flask-Migrate for schema operations (`flask db upgrade`, `flask db stamp`, etc.). Only fall back to custom helpers if you need their extra behaviour.
+- Custom entries such as `flask reset-password`, `flask reset-email`, and `flask vdb-migrate` handle self-hosted account recovery and vector database migrations.
+- Before adding a new command, check whether an existing service can be reused and ensure the command guards edition-specific behaviour (many enforce `SELF_HOSTED`). Document any additions in the PR.
+- Ruff helpers are run directly with `uv`: `uv run --project api --dev ruff format ./api` for formatting and `uv run --project api --dev ruff check ./api` (add `--fix` if you want automatic fixes).
+
+## When You Add Features
+
+- Check for an existing helper or service before writing a new util.
+- Uphold tenancy: every service method should receive the tenant ID from controller wrappers such as `controllers/console/wraps.py`.
+- Update or create tests alongside behaviour changes (`tests/unit_tests` for fast coverage, `tests/integration_tests` when touching orchestrations).
+- Run `uv run --project api --dev ruff check ./api`, `uv run --directory api --dev basedpyright`, and `uv run --project api --dev dev/pytest/pytest_unit_tests.sh` before submitting changes.

api/agent_skills/plugin.md

@@ -0,0 +1 @@
+// TBD

api/agent_skills/plugin_oauth.md

@@ -0,0 +1 @@
+// TBD

api/agent_skills/trigger.md

@@ -0,0 +1,53 @@
+## Overview
+
+Trigger is a collection of nodes that we called `Start` nodes, also, the concept of `Start` is the same as `RootNode` in the workflow engine `core/workflow/graph_engine`, On the other hand, `Start` node is the entry point of workflows, every workflow run always starts from a `Start` node.
+
+## Trigger nodes
+
+- `UserInput`
+- `Trigger Webhook`
+- `Trigger Schedule`
+- `Trigger Plugin`
+
+### UserInput
+
+Before `Trigger` concept is introduced, it's what we called `Start` node, but now, to avoid confusion, it was renamed to `UserInput` node, has a strong relation with `ServiceAPI` in `controllers/service_api/app`
+
+1. `UserInput` node introduces a list of arguments that need to be provided by the user, finally it will be converted into variables in the workflow variable pool.
+1. `ServiceAPI` accept those arguments, and pass through them into `UserInput` node.
+1. For its detailed implementation, please refer to `core/workflow/nodes/start`
+
+### Trigger Webhook
+
+Inside Webhook Node, Dify provided a UI panel that allows user define a HTTP manifest `core/workflow/nodes/trigger_webhook/entities.py`.`WebhookData`, also, Dify generates a random webhook id for each `Trigger Webhook` node, the implementation was implemented in `core/trigger/utils/endpoint.py`, as you can see, `webhook-debug` is a debug mode for webhook, you may find it in `controllers/trigger/webhook.py`.
+
+Finally, requests to `webhook` endpoint will be converted into variables in workflow variable pool during workflow execution.
+
+### Trigger Schedule
+
+`Trigger Schedule` node is a node that allows user define a schedule to trigger the workflow, detailed manifest is here `core/workflow/nodes/trigger_schedule/entities.py`, we have a poller and executor to handle millions of schedules, see `docker/entrypoint.sh` / `schedule/workflow_schedule_task.py` for help.
+
+To Achieve this, a `WorkflowSchedulePlan` model was introduced in `models/trigger.py`, and a `events/event_handlers/sync_workflow_schedule_when_app_published.py` was used to sync workflow schedule plans when app is published.
+
+### Trigger Plugin
+
+`Trigger Plugin` node allows user define there own distributed trigger plugin, whenever a request was received, Dify forwards it to the plugin and wait for parsed variables from it.
+
+1. Requests were saved in storage by `services/trigger/trigger_request_service.py`, referenced by `services/trigger/trigger_service.py`.`TriggerService`.`process_endpoint`
+1. Plugins accept those requests and parse variables from it, see `core/plugin/impl/trigger.py` for details.
+
+A `subscription` concept was out here by Dify, it means an endpoint address from Dify was bound to thirdparty webhook service like `Github` `Slack` `Linear` `GoogleDrive` `Gmail` etc. Once a subscription was created, Dify continually receives requests from the platforms and handle them one by one.
+
+## Worker Pool / Async Task
+
+All the events that triggered a new workflow run is always in async mode, a unified entrypoint can be found here `services/async_workflow_service.py`.`AsyncWorkflowService`.`trigger_workflow_async`.
+
+The infrastructure we used is `celery`, we've already configured it in `docker/entrypoint.sh`, and the consumers are in `tasks/async_workflow_tasks.py`, 3 queues were used to handle different tiers of users, `PROFESSIONAL_QUEUE` `TEAM_QUEUE` `SANDBOX_QUEUE`.
+
+## Debug Strategy
+
+Dify divided users into 2 groups: builders / end users.
+
+Builders are the users who create workflows, in this stage, debugging a workflow becomes a critical part of the workflow development process, as the start node in workflows, trigger nodes can `listen` to the events from `WebhookDebug` `Schedule` `Plugin`, debugging process was created in `controllers/console/app/workflow.py`.`DraftWorkflowTriggerNodeApi`.
+
+A polling process can be considered as combine of few single `poll` operations, each `poll` operation fetches events cached in `Redis`, returns `None` if no event was found, more detailed implemented: `core/trigger/debug/event_bus.py` was used to handle the polling process, and `core/trigger/debug/event_selectors.py` was used to select the event poller based on the trigger type.

api/app.py

@@ -1,4 +1,3 @@
-import os
 import sys
 
 
@@ -9,15 +8,10 @@ def is_db_command() -> bool:
 
 
 # create app
-flask_app = None
-socketio_app = None
-
 if is_db_command():
     from app_factory import create_migrations_app
 
     app = create_migrations_app()
-    socketio_app = app
-    flask_app = app
 else:
     # Gunicorn and Celery handle monkey patching automatically in production by
     # specifying the `gevent` worker class. Manual monkey patching is not required here.
@@ -28,15 +22,8 @@ else:
 
     from app_factory import create_app
 
-    socketio_app, flask_app = create_app()
-    app = flask_app
-    celery = flask_app.extensions["celery"]
+    app = create_app()
+    celery = app.extensions["celery"]
 
 if __name__ == "__main__":
-    from gevent import pywsgi
-    from geventwebsocket.handler import WebSocketHandler  # type: ignore[reportMissingTypeStubs]
-
-    host = os.environ.get("HOST", "0.0.0.0")
-    port = int(os.environ.get("PORT", 5001))
-    server = pywsgi.WSGIServer((host, port), socketio_app, handler_class=WebSocketHandler)
-    server.serve_forever()
+    app.run(host="0.0.0.0", port=5001)

api/app_factory.py

@@ -1,15 +1,11 @@
 import logging
 import time
 
-import socketio  # type: ignore[reportMissingTypeStubs]
 from opentelemetry.trace import get_current_span
-from opentelemetry.trace.span import INVALID_SPAN_ID, INVALID_TRACE_ID
 
 from configs import dify_config
 from contexts.wrapper import RecyclableContextVar
-from core.logging.context import init_request_context
 from dify_app import DifyApp
-from extensions.ext_socketio import sio
 
 logger = logging.getLogger(__name__)
 
@@ -29,56 +25,43 @@ def create_flask_app_with_configs() -> DifyApp:
     # add before request hook
     @dify_app.before_request
     def before_request():
-        # Initialize logging context for this request
-        init_request_context()
+        # add an unique identifier to each request
         RecyclableContextVar.increment_thread_recycles()
 
-    # add after request hook for injecting trace headers from OpenTelemetry span context
-    # Only adds headers when OTEL is enabled and has valid context
+    # add after request hook for injecting X-Trace-Id header from OpenTelemetry span context
     @dify_app.after_request
-    def add_trace_headers(response):
+    def add_trace_id_header(response):
         try:
             span = get_current_span()
             ctx = span.get_span_context() if span else None
-
-            if not ctx or not ctx.is_valid:
-                return response
-
-            # Inject trace headers from OTEL context
-            if ctx.trace_id != INVALID_TRACE_ID and "X-Trace-Id" not in response.headers:
-                response.headers["X-Trace-Id"] = format(ctx.trace_id, "032x")
-            if ctx.span_id != INVALID_SPAN_ID and "X-Span-Id" not in response.headers:
-                response.headers["X-Span-Id"] = format(ctx.span_id, "016x")
-
+            if ctx and ctx.is_valid:
+                trace_id_hex = format(ctx.trace_id, "032x")
+                # Avoid duplicates if some middleware added it
+                if "X-Trace-Id" not in response.headers:
+                    response.headers["X-Trace-Id"] = trace_id_hex
         except Exception:
             # Never break the response due to tracing header injection
-            logger.warning("Failed to add trace headers to response", exc_info=True)
+            logger.warning("Failed to add trace ID to response header", exc_info=True)
         return response
 
     # Capture the decorator's return value to avoid pyright reportUnusedFunction
     _ = before_request
-    _ = add_trace_headers
+    _ = add_trace_id_header
 
     return dify_app
 
 
-def create_app() -> tuple[socketio.WSGIApp, DifyApp]:
+def create_app() -> DifyApp:
     start_time = time.perf_counter()
     app = create_flask_app_with_configs()
     initialize_extensions(app)
-
-    sio.app = app
-    socketio_app = socketio.WSGIApp(sio, app)
-
     end_time = time.perf_counter()
     if dify_config.DEBUG:
         logger.info("Finished create_app (%s ms)", round((end_time - start_time) * 1000, 2))
-    return socketio_app, app
+    return app
 
 
 def initialize_extensions(app: DifyApp):
-    # Initialize Flask context capture for workflow execution
-    from context.flask_app_context import init_flask_context
     from extensions import (
         ext_app_metrics,
         ext_blueprints,
@@ -87,7 +70,6 @@ def initialize_extensions(app: DifyApp):
         ext_commands,
         ext_compress,
         ext_database,
-        ext_fastopenapi,
         ext_forward_refs,
         ext_hosting_provider,
         ext_import_modules,
@@ -109,8 +91,6 @@ def initialize_extensions(app: DifyApp):
         ext_warnings,
     )
 
-    init_flask_context()
-
     extensions = [
         ext_timezone,
         ext_logging,
@@ -135,7 +115,6 @@ def initialize_extensions(app: DifyApp):
         ext_proxy_fix,
         ext_blueprints,
         ext_commands,
-        ext_fastopenapi,
         ext_otel,
         ext_request_logging,
         ext_session_factory,

api/commands.py

@@ -1,9 +1,7 @@
 import base64
-import datetime
 import json
 import logging
 import secrets
-import time
 from typing import Any
 
 import click
@@ -22,9 +20,8 @@ from core.plugin.impl.plugin import PluginInstaller
 from core.rag.datasource.vdb.vector_factory import Vector
 from core.rag.datasource.vdb.vector_type import VectorType
 from core.rag.index_processor.constant.built_in_field import BuiltInField
-from core.rag.models.document import ChildDocument, Document
-from core.sandbox import SandboxBuilder, SandboxType
-from core.tools.utils.system_encryption import encrypt_system_params
+from core.rag.models.document import Document
+from core.tools.utils.system_oauth_encryption import encrypt_system_oauth_params
 from events.app_event import app_was_created
 from extensions.ext_database import db
 from extensions.ext_redis import redis_client
@@ -37,7 +34,7 @@ from libs.rsa import generate_key_pair
 from models import Tenant
 from models.dataset import Dataset, DatasetCollectionBinding, DatasetMetadata, DatasetMetadataBinding, DocumentSegment
 from models.dataset import Document as DatasetDocument
-from models.model import App, AppAnnotationSetting, AppMode, Conversation, MessageAnnotation, UploadFile
+from models.model import Account, App, AppAnnotationSetting, AppMode, Conversation, MessageAnnotation, UploadFile
 from models.oauth import DatasourceOauthParamConfig, DatasourceProvider
 from models.provider import Provider, ProviderModel
 from models.provider_ids import DatasourceProviderID, ToolProviderID
@@ -48,9 +45,6 @@ from services.clear_free_plan_tenant_expired_logs import ClearFreePlanTenantExpi
 from services.plugin.data_migration import PluginDataMigration
 from services.plugin.plugin_migration import PluginMigration
 from services.plugin.plugin_service import PluginService
-from services.retention.conversation.messages_clean_policy import create_message_clean_policy
-from services.retention.conversation.messages_clean_service import MessagesCleanService
-from services.retention.workflow_run.clear_free_plan_expired_workflow_run_logs import WorkflowRunCleanup
 from tasks.remove_app_and_related_data_task import delete_draft_variables_batch
 
 logger = logging.getLogger(__name__)
@@ -68,10 +62,8 @@ def reset_password(email, new_password, password_confirm):
     if str(new_password).strip() != str(password_confirm).strip():
         click.echo(click.style("Passwords do not match.", fg="red"))
         return
-    normalized_email = email.strip().lower()
-
     with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
-        account = AccountService.get_account_by_email_with_case_fallback(email.strip(), session=session)
+        account = session.query(Account).where(Account.email == email).one_or_none()
 
         if not account:
             click.echo(click.style(f"Account not found for email: {email}", fg="red"))
@@ -92,7 +84,7 @@ def reset_password(email, new_password, password_confirm):
         base64_password_hashed = base64.b64encode(password_hashed).decode()
         account.password = base64_password_hashed
         account.password_salt = base64_salt
-        AccountService.reset_login_error_rate_limit(normalized_email)
+        AccountService.reset_login_error_rate_limit(email)
         click.echo(click.style("Password reset successfully.", fg="green"))
 
 
@@ -108,22 +100,20 @@ def reset_email(email, new_email, email_confirm):
     if str(new_email).strip() != str(email_confirm).strip():
         click.echo(click.style("New emails do not match.", fg="red"))
         return
-    normalized_new_email = new_email.strip().lower()
-
     with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
-        account = AccountService.get_account_by_email_with_case_fallback(email.strip(), session=session)
+        account = session.query(Account).where(Account.email == email).one_or_none()
 
         if not account:
             click.echo(click.style(f"Account not found for email: {email}", fg="red"))
             return
 
         try:
-            email_validate(normalized_new_email)
+            email_validate(new_email)
         except:
             click.echo(click.style(f"Invalid email: {new_email}", fg="red"))
             return
 
-        account.email = normalized_new_email
+        account.email = new_email
         click.echo(click.style("Email updated successfully.", fg="green"))
 
 
@@ -245,7 +235,7 @@ def migrate_annotation_vector_database():
                 if annotations:
                     for annotation in annotations:
                         document = Document(
-                            page_content=annotation.question_text,
+                            page_content=annotation.question,
                             metadata={"annotation_id": annotation.id, "app_id": app.id, "doc_id": annotation.id},
                         )
                         documents.append(document)
@@ -419,22 +409,6 @@ def migrate_knowledge_vector_database():
                                 "dataset_id": segment.dataset_id,
                             },
                         )
-                        if dataset_document.doc_form == "hierarchical_model":
-                            child_chunks = segment.get_child_chunks()
-                            if child_chunks:
-                                child_documents = []
-                                for child_chunk in child_chunks:
-                                    child_document = ChildDocument(
-                                        page_content=child_chunk.content,
-                                        metadata={
-                                            "doc_id": child_chunk.index_node_id,
-                                            "doc_hash": child_chunk.index_node_hash,
-                                            "document_id": segment.document_id,
-                                            "dataset_id": segment.dataset_id,
-                                        },
-                                    )
-                                    child_documents.append(child_document)
-                                document.children = child_documents
 
                         documents.append(document)
                         segments_count = segments_count + 1
@@ -448,13 +422,7 @@ def migrate_knowledge_vector_database():
                                 fg="green",
                             )
                         )
-                        all_child_documents = []
-                        for doc in documents:
-                            if doc.children:
-                                all_child_documents.extend(doc.children)
                         vector.create(documents)
-                        if all_child_documents:
-                            vector.create(all_child_documents)
                         click.echo(click.style(f"Created vector index for dataset {dataset.id}.", fg="green"))
                     except Exception as e:
                         click.echo(click.style(f"Failed to created vector index for dataset {dataset.id}.", fg="red"))
@@ -690,7 +658,7 @@ def create_tenant(email: str, language: str | None = None, name: str | None = No
         return
 
     # Create account
-    email = email.strip().lower()
+    email = email.strip()
 
     if "@" not in email:
         click.echo(click.style("Invalid email address.", fg="red"))
@@ -884,435 +852,6 @@ def clear_free_plan_tenant_expired_logs(days: int, batch: int, tenant_ids: list[
     click.echo(click.style("Clear free plan tenant expired logs completed.", fg="green"))
 
 
-@click.command("clean-workflow-runs", help="Clean expired workflow runs and related data for free tenants.")
-@click.option(
-    "--before-days",
-    "--days",
-    default=30,
-    show_default=True,
-    type=click.IntRange(min=0),
-    help="Delete workflow runs created before N days ago.",
-)
-@click.option("--batch-size", default=200, show_default=True, help="Batch size for selecting workflow runs.")
-@click.option(
-    "--from-days-ago",
-    default=None,
-    type=click.IntRange(min=0),
-    help="Lower bound in days ago (older). Must be paired with --to-days-ago.",
-)
-@click.option(
-    "--to-days-ago",
-    default=None,
-    type=click.IntRange(min=0),
-    help="Upper bound in days ago (newer). Must be paired with --from-days-ago.",
-)
-@click.option(
-    "--start-from",
-    type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
-    default=None,
-    help="Optional lower bound (inclusive) for created_at; must be paired with --end-before.",
-)
-@click.option(
-    "--end-before",
-    type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
-    default=None,
-    help="Optional upper bound (exclusive) for created_at; must be paired with --start-from.",
-)
-@click.option(
-    "--dry-run",
-    is_flag=True,
-    help="Preview cleanup results without deleting any workflow run data.",
-)
-def clean_workflow_runs(
-    before_days: int,
-    batch_size: int,
-    from_days_ago: int | None,
-    to_days_ago: int | None,
-    start_from: datetime.datetime | None,
-    end_before: datetime.datetime | None,
-    dry_run: bool,
-):
-    """
-    Clean workflow runs and related workflow data for free tenants.
-    """
-    if (start_from is None) ^ (end_before is None):
-        raise click.UsageError("--start-from and --end-before must be provided together.")
-
-    if (from_days_ago is None) ^ (to_days_ago is None):
-        raise click.UsageError("--from-days-ago and --to-days-ago must be provided together.")
-
-    if from_days_ago is not None and to_days_ago is not None:
-        if start_from or end_before:
-            raise click.UsageError("Choose either day offsets or explicit dates, not both.")
-        if from_days_ago <= to_days_ago:
-            raise click.UsageError("--from-days-ago must be greater than --to-days-ago.")
-        now = datetime.datetime.now()
-        start_from = now - datetime.timedelta(days=from_days_ago)
-        end_before = now - datetime.timedelta(days=to_days_ago)
-        before_days = 0
-
-    start_time = datetime.datetime.now(datetime.UTC)
-    click.echo(click.style(f"Starting workflow run cleanup at {start_time.isoformat()}.", fg="white"))
-
-    WorkflowRunCleanup(
-        days=before_days,
-        batch_size=batch_size,
-        start_from=start_from,
-        end_before=end_before,
-        dry_run=dry_run,
-    ).run()
-
-    end_time = datetime.datetime.now(datetime.UTC)
-    elapsed = end_time - start_time
-    click.echo(
-        click.style(
-            f"Workflow run cleanup completed. start={start_time.isoformat()} "
-            f"end={end_time.isoformat()} duration={elapsed}",
-            fg="green",
-        )
-    )
-
-
-@click.command(
-    "archive-workflow-runs",
-    help="Archive workflow runs for paid plan tenants to S3-compatible storage.",
-)
-@click.option("--tenant-ids", default=None, help="Optional comma-separated tenant IDs for grayscale rollout.")
-@click.option("--before-days", default=90, show_default=True, help="Archive runs older than N days.")
-@click.option(
-    "--from-days-ago",
-    default=None,
-    type=click.IntRange(min=0),
-    help="Lower bound in days ago (older). Must be paired with --to-days-ago.",
-)
-@click.option(
-    "--to-days-ago",
-    default=None,
-    type=click.IntRange(min=0),
-    help="Upper bound in days ago (newer). Must be paired with --from-days-ago.",
-)
-@click.option(
-    "--start-from",
-    type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
-    default=None,
-    help="Archive runs created at or after this timestamp (UTC if no timezone).",
-)
-@click.option(
-    "--end-before",
-    type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
-    default=None,
-    help="Archive runs created before this timestamp (UTC if no timezone).",
-)
-@click.option("--batch-size", default=100, show_default=True, help="Batch size for processing.")
-@click.option("--workers", default=1, show_default=True, type=int, help="Concurrent workflow runs to archive.")
-@click.option("--limit", default=None, type=int, help="Maximum number of runs to archive.")
-@click.option("--dry-run", is_flag=True, help="Preview without archiving.")
-@click.option("--delete-after-archive", is_flag=True, help="Delete runs and related data after archiving.")
-def archive_workflow_runs(
-    tenant_ids: str | None,
-    before_days: int,
-    from_days_ago: int | None,
-    to_days_ago: int | None,
-    start_from: datetime.datetime | None,
-    end_before: datetime.datetime | None,
-    batch_size: int,
-    workers: int,
-    limit: int | None,
-    dry_run: bool,
-    delete_after_archive: bool,
-):
-    """
-    Archive workflow runs for paid plan tenants older than the specified days.
-
-    This command archives the following tables to storage:
-    - workflow_node_executions
-    - workflow_node_execution_offload
-    - workflow_pauses
-    - workflow_pause_reasons
-    - workflow_trigger_logs
-
-    The workflow_runs and workflow_app_logs tables are preserved for UI listing.
-    """
-    from services.retention.workflow_run.archive_paid_plan_workflow_run import WorkflowRunArchiver
-
-    run_started_at = datetime.datetime.now(datetime.UTC)
-    click.echo(
-        click.style(
-            f"Starting workflow run archiving at {run_started_at.isoformat()}.",
-            fg="white",
-        )
-    )
-
-    if (start_from is None) ^ (end_before is None):
-        click.echo(click.style("start-from and end-before must be provided together.", fg="red"))
-        return
-
-    if (from_days_ago is None) ^ (to_days_ago is None):
-        click.echo(click.style("from-days-ago and to-days-ago must be provided together.", fg="red"))
-        return
-
-    if from_days_ago is not None and to_days_ago is not None:
-        if start_from or end_before:
-            click.echo(click.style("Choose either day offsets or explicit dates, not both.", fg="red"))
-            return
-        if from_days_ago <= to_days_ago:
-            click.echo(click.style("from-days-ago must be greater than to-days-ago.", fg="red"))
-            return
-        now = datetime.datetime.now()
-        start_from = now - datetime.timedelta(days=from_days_ago)
-        end_before = now - datetime.timedelta(days=to_days_ago)
-        before_days = 0
-
-    if start_from and end_before and start_from >= end_before:
-        click.echo(click.style("start-from must be earlier than end-before.", fg="red"))
-        return
-    if workers < 1:
-        click.echo(click.style("workers must be at least 1.", fg="red"))
-        return
-
-    archiver = WorkflowRunArchiver(
-        days=before_days,
-        batch_size=batch_size,
-        start_from=start_from,
-        end_before=end_before,
-        workers=workers,
-        tenant_ids=[tid.strip() for tid in tenant_ids.split(",")] if tenant_ids else None,
-        limit=limit,
-        dry_run=dry_run,
-        delete_after_archive=delete_after_archive,
-    )
-    summary = archiver.run()
-    click.echo(
-        click.style(
-            f"Summary: processed={summary.total_runs_processed}, archived={summary.runs_archived}, "
-            f"skipped={summary.runs_skipped}, failed={summary.runs_failed}, "
-            f"time={summary.total_elapsed_time:.2f}s",
-            fg="cyan",
-        )
-    )
-
-    run_finished_at = datetime.datetime.now(datetime.UTC)
-    elapsed = run_finished_at - run_started_at
-    click.echo(
-        click.style(
-            f"Workflow run archiving completed. start={run_started_at.isoformat()} "
-            f"end={run_finished_at.isoformat()} duration={elapsed}",
-            fg="green",
-        )
-    )
-
-
-@click.command(
-    "restore-workflow-runs",
-    help="Restore archived workflow runs from S3-compatible storage.",
-)
-@click.option(
-    "--tenant-ids",
-    required=False,
-    help="Tenant IDs (comma-separated).",
-)
-@click.option("--run-id", required=False, help="Workflow run ID to restore.")
-@click.option(
-    "--start-from",
-    type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
-    default=None,
-    help="Optional lower bound (inclusive) for created_at; must be paired with --end-before.",
-)
-@click.option(
-    "--end-before",
-    type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
-    default=None,
-    help="Optional upper bound (exclusive) for created_at; must be paired with --start-from.",
-)
-@click.option("--workers", default=1, show_default=True, type=int, help="Concurrent workflow runs to restore.")
-@click.option("--limit", type=int, default=100, show_default=True, help="Maximum number of runs to restore.")
-@click.option("--dry-run", is_flag=True, help="Preview without restoring.")
-def restore_workflow_runs(
-    tenant_ids: str | None,
-    run_id: str | None,
-    start_from: datetime.datetime | None,
-    end_before: datetime.datetime | None,
-    workers: int,
-    limit: int,
-    dry_run: bool,
-):
-    """
-    Restore an archived workflow run from storage to the database.
-
-    This restores the following tables:
-    - workflow_node_executions
-    - workflow_node_execution_offload
-    - workflow_pauses
-    - workflow_pause_reasons
-    - workflow_trigger_logs
-    """
-    from services.retention.workflow_run.restore_archived_workflow_run import WorkflowRunRestore
-
-    parsed_tenant_ids = None
-    if tenant_ids:
-        parsed_tenant_ids = [tid.strip() for tid in tenant_ids.split(",") if tid.strip()]
-        if not parsed_tenant_ids:
-            raise click.BadParameter("tenant-ids must not be empty")
-
-    if (start_from is None) ^ (end_before is None):
-        raise click.UsageError("--start-from and --end-before must be provided together.")
-    if run_id is None and (start_from is None or end_before is None):
-        raise click.UsageError("--start-from and --end-before are required for batch restore.")
-    if workers < 1:
-        raise click.BadParameter("workers must be at least 1")
-
-    start_time = datetime.datetime.now(datetime.UTC)
-    click.echo(
-        click.style(
-            f"Starting restore of workflow run {run_id} at {start_time.isoformat()}.",
-            fg="white",
-        )
-    )
-
-    restorer = WorkflowRunRestore(dry_run=dry_run, workers=workers)
-    if run_id:
-        results = [restorer.restore_by_run_id(run_id)]
-    else:
-        assert start_from is not None
-        assert end_before is not None
-        results = restorer.restore_batch(
-            parsed_tenant_ids,
-            start_date=start_from,
-            end_date=end_before,
-            limit=limit,
-        )
-
-    end_time = datetime.datetime.now(datetime.UTC)
-    elapsed = end_time - start_time
-
-    successes = sum(1 for result in results if result.success)
-    failures = len(results) - successes
-
-    if failures == 0:
-        click.echo(
-            click.style(
-                f"Restore completed successfully. success={successes} duration={elapsed}",
-                fg="green",
-            )
-        )
-    else:
-        click.echo(
-            click.style(
-                f"Restore completed with failures. success={successes} failed={failures} duration={elapsed}",
-                fg="red",
-            )
-        )
-
-
-@click.command(
-    "delete-archived-workflow-runs",
-    help="Delete archived workflow runs from the database.",
-)
-@click.option(
-    "--tenant-ids",
-    required=False,
-    help="Tenant IDs (comma-separated).",
-)
-@click.option("--run-id", required=False, help="Workflow run ID to delete.")
-@click.option(
-    "--start-from",
-    type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
-    default=None,
-    help="Optional lower bound (inclusive) for created_at; must be paired with --end-before.",
-)
-@click.option(
-    "--end-before",
-    type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
-    default=None,
-    help="Optional upper bound (exclusive) for created_at; must be paired with --start-from.",
-)
-@click.option("--limit", type=int, default=100, show_default=True, help="Maximum number of runs to delete.")
-@click.option("--dry-run", is_flag=True, help="Preview without deleting.")
-def delete_archived_workflow_runs(
-    tenant_ids: str | None,
-    run_id: str | None,
-    start_from: datetime.datetime | None,
-    end_before: datetime.datetime | None,
-    limit: int,
-    dry_run: bool,
-):
-    """
-    Delete archived workflow runs from the database.
-    """
-    from services.retention.workflow_run.delete_archived_workflow_run import ArchivedWorkflowRunDeletion
-
-    parsed_tenant_ids = None
-    if tenant_ids:
-        parsed_tenant_ids = [tid.strip() for tid in tenant_ids.split(",") if tid.strip()]
-        if not parsed_tenant_ids:
-            raise click.BadParameter("tenant-ids must not be empty")
-
-    if (start_from is None) ^ (end_before is None):
-        raise click.UsageError("--start-from and --end-before must be provided together.")
-    if run_id is None and (start_from is None or end_before is None):
-        raise click.UsageError("--start-from and --end-before are required for batch delete.")
-
-    start_time = datetime.datetime.now(datetime.UTC)
-    target_desc = f"workflow run {run_id}" if run_id else "workflow runs"
-    click.echo(
-        click.style(
-            f"Starting delete of {target_desc} at {start_time.isoformat()}.",
-            fg="white",
-        )
-    )
-
-    deleter = ArchivedWorkflowRunDeletion(dry_run=dry_run)
-    if run_id:
-        results = [deleter.delete_by_run_id(run_id)]
-    else:
-        assert start_from is not None
-        assert end_before is not None
-        results = deleter.delete_batch(
-            parsed_tenant_ids,
-            start_date=start_from,
-            end_date=end_before,
-            limit=limit,
-        )
-
-    for result in results:
-        if result.success:
-            click.echo(
-                click.style(
-                    f"{'[DRY RUN] Would delete' if dry_run else 'Deleted'} "
-                    f"workflow run {result.run_id} (tenant={result.tenant_id})",
-                    fg="green",
-                )
-            )
-        else:
-            click.echo(
-                click.style(
-                    f"Failed to delete workflow run {result.run_id}: {result.error}",
-                    fg="red",
-                )
-            )
-
-    end_time = datetime.datetime.now(datetime.UTC)
-    elapsed = end_time - start_time
-
-    successes = sum(1 for result in results if result.success)
-    failures = len(results) - successes
-
-    if failures == 0:
-        click.echo(
-            click.style(
-                f"Delete completed successfully. success={successes} duration={elapsed}",
-                fg="green",
-            )
-        )
-    else:
-        click.echo(
-            click.style(
-                f"Delete completed with failures. success={successes} failed={failures} duration={elapsed}",
-                fg="red",
-            )
-        )
-
-
 @click.option("-f", "--force", is_flag=True, help="Skip user confirmation and force the command to execute.")
 @click.command("clear-orphaned-file-records", help="Clear orphaned file records.")
 def clear_orphaned_file_records(force: bool):
@@ -1608,7 +1147,7 @@ def remove_orphaned_files_on_storage(force: bool):
             click.echo(click.style(f"- Scanning files on storage path {storage_path}", fg="white"))
             files = storage.scan(path=storage_path, files=True, directories=False)
             all_files_on_storage.extend(files)
-        except FileNotFoundError:
+        except FileNotFoundError as e:
             click.echo(click.style(f"  -> Skipping path {storage_path} as it does not exist.", fg="yellow"))
             continue
         except Exception as e:
@@ -1645,268 +1184,6 @@ def remove_orphaned_files_on_storage(force: bool):
         click.echo(click.style(f"Removed {removed_files} orphaned files, with {error_files} errors.", fg="yellow"))
 
 
-@click.command("file-usage", help="Query file usages and show where files are referenced.")
-@click.option("--file-id", type=str, default=None, help="Filter by file UUID.")
-@click.option("--key", type=str, default=None, help="Filter by storage key.")
-@click.option("--src", type=str, default=None, help="Filter by table.column pattern (e.g., 'documents.%' or '%.icon').")
-@click.option("--limit", type=int, default=100, help="Limit number of results (default: 100).")
-@click.option("--offset", type=int, default=0, help="Offset for pagination (default: 0).")
-@click.option("--json", "output_json", is_flag=True, help="Output results in JSON format.")
-def file_usage(
-    file_id: str | None,
-    key: str | None,
-    src: str | None,
-    limit: int,
-    offset: int,
-    output_json: bool,
-):
-    """
-    Query file usages and show where files are referenced in the database.
-
-    This command reuses the same reference checking logic as clear-orphaned-file-records
-    and displays detailed information about where each file is referenced.
-    """
-    # define tables and columns to process
-    files_tables = [
-        {"table": "upload_files", "id_column": "id", "key_column": "key"},
-        {"table": "tool_files", "id_column": "id", "key_column": "file_key"},
-    ]
-    ids_tables = [
-        {"type": "uuid", "table": "message_files", "column": "upload_file_id", "pk_column": "id"},
-        {"type": "text", "table": "documents", "column": "data_source_info", "pk_column": "id"},
-        {"type": "text", "table": "document_segments", "column": "content", "pk_column": "id"},
-        {"type": "text", "table": "messages", "column": "answer", "pk_column": "id"},
-        {"type": "text", "table": "workflow_node_executions", "column": "inputs", "pk_column": "id"},
-        {"type": "text", "table": "workflow_node_executions", "column": "process_data", "pk_column": "id"},
-        {"type": "text", "table": "workflow_node_executions", "column": "outputs", "pk_column": "id"},
-        {"type": "text", "table": "conversations", "column": "introduction", "pk_column": "id"},
-        {"type": "text", "table": "conversations", "column": "system_instruction", "pk_column": "id"},
-        {"type": "text", "table": "accounts", "column": "avatar", "pk_column": "id"},
-        {"type": "text", "table": "apps", "column": "icon", "pk_column": "id"},
-        {"type": "text", "table": "sites", "column": "icon", "pk_column": "id"},
-        {"type": "json", "table": "messages", "column": "inputs", "pk_column": "id"},
-        {"type": "json", "table": "messages", "column": "message", "pk_column": "id"},
-    ]
-
-    # Stream file usages with pagination to avoid holding all results in memory
-    paginated_usages = []
-    total_count = 0
-
-    # First, build a mapping of file_id -> storage_key from the base tables
-    file_key_map = {}
-    for files_table in files_tables:
-        query = f"SELECT {files_table['id_column']}, {files_table['key_column']} FROM {files_table['table']}"
-        with db.engine.begin() as conn:
-            rs = conn.execute(sa.text(query))
-            for row in rs:
-                file_key_map[str(row[0])] = f"{files_table['table']}:{row[1]}"
-
-    # If filtering by key or file_id, verify it exists
-    if file_id and file_id not in file_key_map:
-        if output_json:
-            click.echo(json.dumps({"error": f"File ID {file_id} not found in base tables"}))
-        else:
-            click.echo(click.style(f"File ID {file_id} not found in base tables.", fg="red"))
-        return
-
-    if key:
-        valid_prefixes = {f"upload_files:{key}", f"tool_files:{key}"}
-        matching_file_ids = [fid for fid, fkey in file_key_map.items() if fkey in valid_prefixes]
-        if not matching_file_ids:
-            if output_json:
-                click.echo(json.dumps({"error": f"Key {key} not found in base tables"}))
-            else:
-                click.echo(click.style(f"Key {key} not found in base tables.", fg="red"))
-            return
-
-    guid_regexp = "[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}"
-
-    # For each reference table/column, find matching file IDs and record the references
-    for ids_table in ids_tables:
-        src_filter = f"{ids_table['table']}.{ids_table['column']}"
-
-        # Skip if src filter doesn't match (use fnmatch for wildcard patterns)
-        if src:
-            if "%" in src or "_" in src:
-                import fnmatch
-
-                # Convert SQL LIKE wildcards to fnmatch wildcards (% -> *, _ -> ?)
-                pattern = src.replace("%", "*").replace("_", "?")
-                if not fnmatch.fnmatch(src_filter, pattern):
-                    continue
-            else:
-                if src_filter != src:
-                    continue
-
-        if ids_table["type"] == "uuid":
-            # Direct UUID match
-            query = (
-                f"SELECT {ids_table['pk_column']}, {ids_table['column']} "
-                f"FROM {ids_table['table']} WHERE {ids_table['column']} IS NOT NULL"
-            )
-            with db.engine.begin() as conn:
-                rs = conn.execute(sa.text(query))
-                for row in rs:
-                    record_id = str(row[0])
-                    ref_file_id = str(row[1])
-                    if ref_file_id not in file_key_map:
-                        continue
-                    storage_key = file_key_map[ref_file_id]
-
-                    # Apply filters
-                    if file_id and ref_file_id != file_id:
-                        continue
-                    if key and not storage_key.endswith(key):
-                        continue
-
-                    # Only collect items within the requested page range
-                    if offset <= total_count < offset + limit:
-                        paginated_usages.append(
-                            {
-                                "src": f"{ids_table['table']}.{ids_table['column']}",
-                                "record_id": record_id,
-                                "file_id": ref_file_id,
-                                "key": storage_key,
-                            }
-                        )
-                    total_count += 1
-
-        elif ids_table["type"] in ("text", "json"):
-            # Extract UUIDs from text/json content
-            column_cast = f"{ids_table['column']}::text" if ids_table["type"] == "json" else ids_table["column"]
-            query = (
-                f"SELECT {ids_table['pk_column']}, {column_cast} "
-                f"FROM {ids_table['table']} WHERE {ids_table['column']} IS NOT NULL"
-            )
-            with db.engine.begin() as conn:
-                rs = conn.execute(sa.text(query))
-                for row in rs:
-                    record_id = str(row[0])
-                    content = str(row[1])
-
-                    # Find all UUIDs in the content
-                    import re
-
-                    uuid_pattern = re.compile(guid_regexp, re.IGNORECASE)
-                    matches = uuid_pattern.findall(content)
-
-                    for ref_file_id in matches:
-                        if ref_file_id not in file_key_map:
-                            continue
-                        storage_key = file_key_map[ref_file_id]
-
-                        # Apply filters
-                        if file_id and ref_file_id != file_id:
-                            continue
-                        if key and not storage_key.endswith(key):
-                            continue
-
-                        # Only collect items within the requested page range
-                        if offset <= total_count < offset + limit:
-                            paginated_usages.append(
-                                {
-                                    "src": f"{ids_table['table']}.{ids_table['column']}",
-                                    "record_id": record_id,
-                                    "file_id": ref_file_id,
-                                    "key": storage_key,
-                                }
-                            )
-                        total_count += 1
-
-    # Output results
-    if output_json:
-        result = {
-            "total": total_count,
-            "offset": offset,
-            "limit": limit,
-            "usages": paginated_usages,
-        }
-        click.echo(json.dumps(result, indent=2))
-    else:
-        click.echo(
-            click.style(f"Found {total_count} file usages (showing {len(paginated_usages)} results)", fg="white")
-        )
-        click.echo("")
-
-        if not paginated_usages:
-            click.echo(click.style("No file usages found matching the specified criteria.", fg="yellow"))
-            return
-
-        # Print table header
-        click.echo(
-            click.style(
-                f"{'Src (Table.Column)':<50} {'Record ID':<40} {'File ID':<40} {'Storage Key':<60}",
-                fg="cyan",
-            )
-        )
-        click.echo(click.style("-" * 190, fg="white"))
-
-        # Print each usage
-        for usage in paginated_usages:
-            click.echo(f"{usage['src']:<50} {usage['record_id']:<40} {usage['file_id']:<40} {usage['key']:<60}")
-
-        # Show pagination info
-        if offset + limit < total_count:
-            click.echo("")
-            click.echo(
-                click.style(
-                    f"Showing {offset + 1}-{offset + len(paginated_usages)} of {total_count} results", fg="white"
-                )
-            )
-            click.echo(click.style(f"Use --offset {offset + limit} to see next page", fg="white"))
-
-
-@click.command("setup-sandbox-system-config", help="Setup system-level sandbox provider configuration.")
-@click.option(
-    "--provider-type", prompt=True, type=click.Choice(["e2b", "docker", "local"]), help="Sandbox provider type"
-)
-@click.option("--config", prompt=True, help='Configuration JSON (e.g., {"api_key": "xxx"} for e2b)')
-def setup_sandbox_system_config(provider_type: str, config: str):
-    """
-    Setup system-level sandbox provider configuration.
-
-    Examples:
-        flask setup-sandbox-system-config --provider-type e2b --config '{"api_key": "e2b_xxx"}'
-        flask setup-sandbox-system-config --provider-type docker --config '{"docker_sock": "unix:///var/run/docker.sock"}'
-        flask setup-sandbox-system-config --provider-type local --config '{}'
-    """
-    from models.sandbox import SandboxProviderSystemConfig
-
-    try:
-        click.echo(click.style(f"Validating config: {config}", fg="yellow"))
-        config_dict = TypeAdapter(dict[str, Any]).validate_json(config)
-        click.echo(click.style("Config validated successfully.", fg="green"))
-
-        click.echo(click.style(f"Validating config schema for provider type: {provider_type}", fg="yellow"))
-        SandboxBuilder.validate(SandboxType(provider_type), config_dict)
-        click.echo(click.style("Config schema validated successfully.", fg="green"))
-
-        click.echo(click.style("Encrypting config...", fg="yellow"))
-        click.echo(click.style(f"Using SECRET_KEY: `{dify_config.SECRET_KEY}`", fg="yellow"))
-        encrypted_config = encrypt_system_params(config_dict)
-        click.echo(click.style("Config encrypted successfully.", fg="green"))
-    except Exception as e:
-        click.echo(click.style(f"Error validating/encrypting config: {str(e)}", fg="red"))
-        return
-
-    deleted_count = db.session.query(SandboxProviderSystemConfig).filter_by(provider_type=provider_type).delete()
-    if deleted_count > 0:
-        click.echo(
-            click.style(
-                f"Deleted {deleted_count} existing system config for provider type: {provider_type}", fg="yellow"
-            )
-        )
-
-    system_config = SandboxProviderSystemConfig(
-        provider_type=provider_type,
-        encrypted_config=encrypted_config,
-    )
-    db.session.add(system_config)
-    db.session.commit()
-    click.echo(click.style(f"Sandbox system config setup successfully. id: {system_config.id}", fg="green"))
-    click.echo(click.style(f"Provider type: {provider_type}", fg="green"))
-
-
 @click.command("setup-system-tool-oauth-client", help="Setup system tool oauth client.")
 @click.option("--provider", prompt=True, help="Provider name")
 @click.option("--client-params", prompt=True, help="Client Params")
@@ -1926,7 +1203,7 @@ def setup_system_tool_oauth_client(provider, client_params):
 
         click.echo(click.style(f"Encrypting client params: {client_params}", fg="yellow"))
         click.echo(click.style(f"Using SECRET_KEY: `{dify_config.SECRET_KEY}`", fg="yellow"))
-        oauth_client_params = encrypt_system_params(client_params_dict)
+        oauth_client_params = encrypt_system_oauth_params(client_params_dict)
         click.echo(click.style("Client params encrypted successfully.", fg="green"))
     except Exception as e:
         click.echo(click.style(f"Error parsing client params: {str(e)}", fg="red"))
@@ -1975,7 +1252,7 @@ def setup_system_trigger_oauth_client(provider, client_params):
 
         click.echo(click.style(f"Encrypting client params: {client_params}", fg="yellow"))
         click.echo(click.style(f"Using SECRET_KEY: `{dify_config.SECRET_KEY}`", fg="yellow"))
-        oauth_client_params = encrypt_system_params(client_params_dict)
+        oauth_client_params = encrypt_system_oauth_params(client_params_dict)
         click.echo(click.style("Client params encrypted successfully.", fg="green"))
     except Exception as e:
         click.echo(click.style(f"Error parsing client params: {str(e)}", fg="red"))
@@ -2623,79 +1900,3 @@ def migrate_oss(
             except Exception as e:
                 db.session.rollback()
                 click.echo(click.style(f"Failed to update DB storage_type: {str(e)}", fg="red"))
-
-
-@click.command("clean-expired-messages", help="Clean expired messages.")
-@click.option(
-    "--start-from",
-    type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
-    required=True,
-    help="Lower bound (inclusive) for created_at.",
-)
-@click.option(
-    "--end-before",
-    type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
-    required=True,
-    help="Upper bound (exclusive) for created_at.",
-)
-@click.option("--batch-size", default=1000, show_default=True, help="Batch size for selecting messages.")
-@click.option(
-    "--graceful-period",
-    default=21,
-    show_default=True,
-    help="Graceful period in days after subscription expiration, will be ignored when billing is disabled.",
-)
-@click.option("--dry-run", is_flag=True, default=False, help="Show messages logs would be cleaned without deleting")
-def clean_expired_messages(
-    batch_size: int,
-    graceful_period: int,
-    start_from: datetime.datetime,
-    end_before: datetime.datetime,
-    dry_run: bool,
-):
-    """
-    Clean expired messages and related data for tenants based on clean policy.
-    """
-    click.echo(click.style("clean_messages: start clean messages.", fg="green"))
-
-    start_at = time.perf_counter()
-
-    try:
-        # Create policy based on billing configuration
-        # NOTE: graceful_period will be ignored when billing is disabled.
-        policy = create_message_clean_policy(graceful_period_days=graceful_period)
-
-        # Create and run the cleanup service
-        service = MessagesCleanService.from_time_range(
-            policy=policy,
-            start_from=start_from,
-            end_before=end_before,
-            batch_size=batch_size,
-            dry_run=dry_run,
-        )
-        stats = service.run()
-
-        end_at = time.perf_counter()
-        click.echo(
-            click.style(
-                f"clean_messages: completed successfully\n"
-                f"  - Latency: {end_at - start_at:.2f}s\n"
-                f"  - Batches processed: {stats['batches']}\n"
-                f"  - Total messages scanned: {stats['total_messages']}\n"
-                f"  - Messages filtered: {stats['filtered_messages']}\n"
-                f"  - Messages deleted: {stats['total_deleted']}",
-                fg="green",
-            )
-        )
-    except Exception as e:
-        end_at = time.perf_counter()
-        logger.exception("clean_messages failed")
-        click.echo(
-            click.style(
-                f"clean_messages: failed after {end_at - start_at:.2f}s - {str(e)}",
-                fg="red",
-            )
-        )
-        raise
-
-    click.echo(click.style("messages cleanup completed.", fg="green"))

api/configs/app_config.py

@@ -2,7 +2,6 @@ import logging
 from pathlib import Path
 from typing import Any
 
-from pydantic import Field
 from pydantic.fields import FieldInfo
 from pydantic_settings import BaseSettings, PydanticBaseSettingsSource, SettingsConfigDict, TomlConfigSettingsSource
 
@@ -83,17 +82,6 @@ class DifyConfig(
         extra="ignore",
     )
 
-    SANDBOX_DIFY_CLI_ROOT: str | None = Field(
-        default=None,
-        description=(
-            "Filesystem directory containing dify CLI binaries named dify-cli-<os>-<arch>. "
-            "Defaults to api/bin when unset."
-        ),
-    )
-    DIFY_PORT: int = Field(
-        default=5001,
-        description="Port used by Dify to communicate with the host machine.",
-    )
     # Before adding any config,
     # please consider to arrange it in the proper config group of existed or added
     # for better readability and maintainability.

api/configs/extra/__init__.py

@@ -1,11 +1,9 @@
-from configs.extra.archive_config import ArchiveStorageConfig
 from configs.extra.notion_config import NotionConfig
 from configs.extra.sentry_config import SentryConfig
 
 
 class ExtraServiceConfig(
     # place the configs in alphabet order
-    ArchiveStorageConfig,
     NotionConfig,
     SentryConfig,
 ):

api/configs/extra/archive_config.py

@@ -1,43 +0,0 @@
-from pydantic import Field
-from pydantic_settings import BaseSettings
-
-
-class ArchiveStorageConfig(BaseSettings):
-    """
-    Configuration settings for workflow run logs archiving storage.
-    """
-
-    ARCHIVE_STORAGE_ENABLED: bool = Field(
-        description="Enable workflow run logs archiving to S3-compatible storage",
-        default=False,
-    )
-
-    ARCHIVE_STORAGE_ENDPOINT: str | None = Field(
-        description="URL of the S3-compatible storage endpoint (e.g., 'https://storage.example.com')",
-        default=None,
-    )
-
-    ARCHIVE_STORAGE_ARCHIVE_BUCKET: str | None = Field(
-        description="Name of the bucket to store archived workflow logs",
-        default=None,
-    )
-
-    ARCHIVE_STORAGE_EXPORT_BUCKET: str | None = Field(
-        description="Name of the bucket to store exported workflow runs",
-        default=None,
-    )
-
-    ARCHIVE_STORAGE_ACCESS_KEY: str | None = Field(
-        description="Access key ID for authenticating with storage",
-        default=None,
-    )
-
-    ARCHIVE_STORAGE_SECRET_KEY: str | None = Field(
-        description="Secret access key for authenticating with storage",
-        default=None,
-    )
-
-    ARCHIVE_STORAGE_REGION: str = Field(
-        description="Region for storage (use 'auto' if the provider supports it)",
-        default="auto",
-    )

api/configs/feature/__init__.py

@@ -243,22 +243,6 @@ class PluginConfig(BaseSettings):
         default=15728640 * 12,
     )
 
-    PLUGIN_MODEL_SCHEMA_CACHE_TTL: PositiveInt = Field(
-        description="TTL in seconds for caching plugin model schemas in Redis",
-        default=60 * 60,
-    )
-
-
-class CliApiConfig(BaseSettings):
-    """
-    Configuration for CLI API (for dify-cli to call back from external sandbox environments)
-    """
-
-    CLI_API_URL: str = Field(
-        description="CLI API URL for external sandbox (e.g., e2b) to call back.",
-        default="http://localhost:5001",
-    )
-
 
 class MarketplaceConfig(BaseSettings):
     """
@@ -603,11 +587,6 @@ class LoggingConfig(BaseSettings):
         default="INFO",
     )
 
-    LOG_OUTPUT_FORMAT: Literal["text", "json"] = Field(
-        description="Log output format: 'text' for human-readable, 'json' for structured JSON logs.",
-        default="text",
-    )
-
     LOG_FILE: str | None = Field(
         description="File path for log output.",
         default=None,
@@ -965,12 +944,6 @@ class MailConfig(BaseSettings):
         default=False,
     )
 
-    SMTP_LOCAL_HOSTNAME: str | None = Field(
-        description="Override the local hostname used in SMTP HELO/EHLO. "
-        "Useful behind NAT or when the default hostname causes rejections.",
-        default=None,
-    )
-
     EMAIL_SEND_IP_LIMIT_PER_MINUTE: PositiveInt = Field(
         description="Maximum number of emails allowed to be sent from the same IP address in a minute",
         default=50,
@@ -981,16 +954,6 @@ class MailConfig(BaseSettings):
         default=None,
     )
 
-    ENABLE_TRIAL_APP: bool = Field(
-        description="Enable trial app",
-        default=False,
-    )
-
-    ENABLE_EXPLORE_BANNER: bool = Field(
-        description="Enable explore banner",
-        default=False,
-    )
-
 
 class RagEtlConfig(BaseSettings):
     """
@@ -1133,10 +1096,6 @@ class CeleryScheduleTasksConfig(BaseSettings):
         description="Enable clean messages task",
         default=False,
     )
-    ENABLE_WORKFLOW_RUN_CLEANUP_TASK: bool = Field(
-        description="Enable scheduled workflow run cleanup task",
-        default=False,
-    )
     ENABLE_MAIL_CLEAN_DOCUMENT_NOTIFY_TASK: bool = Field(
         description="Enable mail clean document notify task",
         default=False,
@@ -1245,13 +1204,6 @@ class PositionConfig(BaseSettings):
         return {item.strip() for item in self.POSITION_TOOL_EXCLUDES.split(",") if item.strip() != ""}
 
 
-class CollaborationConfig(BaseSettings):
-    ENABLE_COLLABORATION_MODE: bool = Field(
-        description="Whether to enable collaboration mode features across the workspace",
-        default=False,
-    )
-
-
 class LoginConfig(BaseSettings):
     ENABLE_EMAIL_CODE_LOGIN: bool = Field(
         description="whether to enable email code login",
@@ -1331,10 +1283,6 @@ class SandboxExpiredRecordsCleanConfig(BaseSettings):
         description="Retention days for sandbox expired workflow_run records and message records",
         default=30,
     )
-    SANDBOX_EXPIRED_RECORDS_CLEAN_TASK_LOCK_TTL: PositiveInt = Field(
-        description="Lock TTL for sandbox expired records clean task in seconds",
-        default=90000,
-    )
 
 
 class FeatureConfig(
@@ -1346,7 +1294,6 @@ class FeatureConfig(
     TriggerConfig,
     AsyncWorkflowConfig,
     PluginConfig,
-    CliApiConfig,
     MarketplaceConfig,
     DataSetConfig,
     EndpointConfig,
@@ -1371,7 +1318,6 @@ class FeatureConfig(
     WorkflowConfig,
     WorkflowNodeExecutionConfig,
     WorkspaceConfig,
-    CollaborationConfig,
     LoginConfig,
     AccountConfig,
     SwaggerUIConfig,

api/configs/feature/hosted_service/__init__.py

@@ -8,11 +8,6 @@ class HostedCreditConfig(BaseSettings):
         default="",
     )
 
-    HOSTED_POOL_CREDITS: int = Field(
-        description="Pool credits for hosted service",
-        default=200,
-    )
-
     def get_model_credits(self, model_name: str) -> int:
         """
         Get credit value for a specific model name.
@@ -65,46 +60,19 @@ class HostedOpenAiConfig(BaseSettings):
 
     HOSTED_OPENAI_TRIAL_MODELS: str = Field(
         description="Comma-separated list of available models for trial access",
-        default="gpt-4,"
-        "gpt-4-turbo-preview,"
-        "gpt-4-turbo-2024-04-09,"
-        "gpt-4-1106-preview,"
-        "gpt-4-0125-preview,"
-        "gpt-4-turbo,"
-        "gpt-4.1,"
-        "gpt-4.1-2025-04-14,"
-        "gpt-4.1-mini,"
-        "gpt-4.1-mini-2025-04-14,"
-        "gpt-4.1-nano,"
-        "gpt-4.1-nano-2025-04-14,"
-        "gpt-3.5-turbo,"
+        default="gpt-3.5-turbo,"
+        "gpt-3.5-turbo-1106,"
+        "gpt-3.5-turbo-instruct,"
         "gpt-3.5-turbo-16k,"
         "gpt-3.5-turbo-16k-0613,"
-        "gpt-3.5-turbo-1106,"
         "gpt-3.5-turbo-0613,"
         "gpt-3.5-turbo-0125,"
-        "gpt-3.5-turbo-instruct,"
-        "text-davinci-003,"
-        "chatgpt-4o-latest,"
-        "gpt-4o,"
-        "gpt-4o-2024-05-13,"
-        "gpt-4o-2024-08-06,"
-        "gpt-4o-2024-11-20,"
-        "gpt-4o-audio-preview,"
-        "gpt-4o-audio-preview-2025-06-03,"
-        "gpt-4o-mini,"
-        "gpt-4o-mini-2024-07-18,"
-        "o3-mini,"
-        "o3-mini-2025-01-31,"
-        "gpt-5-mini-2025-08-07,"
-        "gpt-5-mini,"
-        "o4-mini,"
-        "o4-mini-2025-04-16,"
-        "gpt-5-chat-latest,"
-        "gpt-5,"
-        "gpt-5-2025-08-07,"
-        "gpt-5-nano,"
-        "gpt-5-nano-2025-08-07",
+        "text-davinci-003",
+    )
+
+    HOSTED_OPENAI_QUOTA_LIMIT: NonNegativeInt = Field(
+        description="Quota limit for hosted OpenAI service usage",
+        default=200,
     )
 
     HOSTED_OPENAI_PAID_ENABLED: bool = Field(
@@ -119,13 +87,6 @@ class HostedOpenAiConfig(BaseSettings):
         "gpt-4-turbo-2024-04-09,"
         "gpt-4-1106-preview,"
         "gpt-4-0125-preview,"
-        "gpt-4-turbo,"
-        "gpt-4.1,"
-        "gpt-4.1-2025-04-14,"
-        "gpt-4.1-mini,"
-        "gpt-4.1-mini-2025-04-14,"
-        "gpt-4.1-nano,"
-        "gpt-4.1-nano-2025-04-14,"
         "gpt-3.5-turbo,"
         "gpt-3.5-turbo-16k,"
         "gpt-3.5-turbo-16k-0613,"
@@ -133,150 +94,7 @@ class HostedOpenAiConfig(BaseSettings):
         "gpt-3.5-turbo-0613,"
         "gpt-3.5-turbo-0125,"
         "gpt-3.5-turbo-instruct,"
-        "text-davinci-003,"
-        "chatgpt-4o-latest,"
-        "gpt-4o,"
-        "gpt-4o-2024-05-13,"
-        "gpt-4o-2024-08-06,"
-        "gpt-4o-2024-11-20,"
-        "gpt-4o-audio-preview,"
-        "gpt-4o-audio-preview-2025-06-03,"
-        "gpt-4o-mini,"
-        "gpt-4o-mini-2024-07-18,"
-        "o3-mini,"
-        "o3-mini-2025-01-31,"
-        "gpt-5-mini-2025-08-07,"
-        "gpt-5-mini,"
-        "o4-mini,"
-        "o4-mini-2025-04-16,"
-        "gpt-5-chat-latest,"
-        "gpt-5,"
-        "gpt-5-2025-08-07,"
-        "gpt-5-nano,"
-        "gpt-5-nano-2025-08-07",
-    )
-
-
-class HostedGeminiConfig(BaseSettings):
-    """
-    Configuration for fetching Gemini service
-    """
-
-    HOSTED_GEMINI_API_KEY: str | None = Field(
-        description="API key for hosted Gemini service",
-        default=None,
-    )
-
-    HOSTED_GEMINI_API_BASE: str | None = Field(
-        description="Base URL for hosted Gemini API",
-        default=None,
-    )
-
-    HOSTED_GEMINI_API_ORGANIZATION: str | None = Field(
-        description="Organization ID for hosted Gemini service",
-        default=None,
-    )
-
-    HOSTED_GEMINI_TRIAL_ENABLED: bool = Field(
-        description="Enable trial access to hosted Gemini service",
-        default=False,
-    )
-
-    HOSTED_GEMINI_TRIAL_MODELS: str = Field(
-        description="Comma-separated list of available models for trial access",
-        default="gemini-2.5-flash,gemini-2.0-flash,gemini-2.0-flash-lite,",
-    )
-
-    HOSTED_GEMINI_PAID_ENABLED: bool = Field(
-        description="Enable paid access to hosted gemini service",
-        default=False,
-    )
-
-    HOSTED_GEMINI_PAID_MODELS: str = Field(
-        description="Comma-separated list of available models for paid access",
-        default="gemini-2.5-flash,gemini-2.0-flash,gemini-2.0-flash-lite,",
-    )
-
-
-class HostedXAIConfig(BaseSettings):
-    """
-    Configuration for fetching XAI service
-    """
-
-    HOSTED_XAI_API_KEY: str | None = Field(
-        description="API key for hosted XAI service",
-        default=None,
-    )
-
-    HOSTED_XAI_API_BASE: str | None = Field(
-        description="Base URL for hosted XAI API",
-        default=None,
-    )
-
-    HOSTED_XAI_API_ORGANIZATION: str | None = Field(
-        description="Organization ID for hosted XAI service",
-        default=None,
-    )
-
-    HOSTED_XAI_TRIAL_ENABLED: bool = Field(
-        description="Enable trial access to hosted XAI service",
-        default=False,
-    )
-
-    HOSTED_XAI_TRIAL_MODELS: str = Field(
-        description="Comma-separated list of available models for trial access",
-        default="grok-3,grok-3-mini,grok-3-mini-fast",
-    )
-
-    HOSTED_XAI_PAID_ENABLED: bool = Field(
-        description="Enable paid access to hosted XAI service",
-        default=False,
-    )
-
-    HOSTED_XAI_PAID_MODELS: str = Field(
-        description="Comma-separated list of available models for paid access",
-        default="grok-3,grok-3-mini,grok-3-mini-fast",
-    )
-
-
-class HostedDeepseekConfig(BaseSettings):
-    """
-    Configuration for fetching Deepseek service
-    """
-
-    HOSTED_DEEPSEEK_API_KEY: str | None = Field(
-        description="API key for hosted Deepseek service",
-        default=None,
-    )
-
-    HOSTED_DEEPSEEK_API_BASE: str | None = Field(
-        description="Base URL for hosted Deepseek API",
-        default=None,
-    )
-
-    HOSTED_DEEPSEEK_API_ORGANIZATION: str | None = Field(
-        description="Organization ID for hosted Deepseek service",
-        default=None,
-    )
-
-    HOSTED_DEEPSEEK_TRIAL_ENABLED: bool = Field(
-        description="Enable trial access to hosted Deepseek service",
-        default=False,
-    )
-
-    HOSTED_DEEPSEEK_TRIAL_MODELS: str = Field(
-        description="Comma-separated list of available models for trial access",
-        default="deepseek-chat,deepseek-reasoner",
-    )
-
-    HOSTED_DEEPSEEK_PAID_ENABLED: bool = Field(
-        description="Enable paid access to hosted Deepseek service",
-        default=False,
-    )
-
-    HOSTED_DEEPSEEK_PAID_MODELS: str = Field(
-        description="Comma-separated list of available models for paid access",
-        default="deepseek-chat,deepseek-reasoner",
+        "text-davinci-003",
     )
 
 
@@ -326,66 +144,16 @@ class HostedAnthropicConfig(BaseSettings):
         default=False,
     )
 
+    HOSTED_ANTHROPIC_QUOTA_LIMIT: NonNegativeInt = Field(
+        description="Quota limit for hosted Anthropic service usage",
+        default=600000,
+    )
+
     HOSTED_ANTHROPIC_PAID_ENABLED: bool = Field(
         description="Enable paid access to hosted Anthropic service",
         default=False,
     )
 
-    HOSTED_ANTHROPIC_TRIAL_MODELS: str = Field(
-        description="Comma-separated list of available models for paid access",
-        default="claude-opus-4-20250514,"
-        "claude-sonnet-4-20250514,"
-        "claude-3-5-haiku-20241022,"
-        "claude-3-opus-20240229,"
-        "claude-3-7-sonnet-20250219,"
-        "claude-3-haiku-20240307",
-    )
-    HOSTED_ANTHROPIC_PAID_MODELS: str = Field(
-        description="Comma-separated list of available models for paid access",
-        default="claude-opus-4-20250514,"
-        "claude-sonnet-4-20250514,"
-        "claude-3-5-haiku-20241022,"
-        "claude-3-opus-20240229,"
-        "claude-3-7-sonnet-20250219,"
-        "claude-3-haiku-20240307",
-    )
-
-
-class HostedTongyiConfig(BaseSettings):
-    """
-    Configuration for hosted Tongyi service
-    """
-
-    HOSTED_TONGYI_API_KEY: str | None = Field(
-        description="API key for hosted Tongyi service",
-        default=None,
-    )
-
-    HOSTED_TONGYI_USE_INTERNATIONAL_ENDPOINT: bool = Field(
-        description="Use international endpoint for hosted Tongyi service",
-        default=False,
-    )
-
-    HOSTED_TONGYI_TRIAL_ENABLED: bool = Field(
-        description="Enable trial access to hosted Tongyi service",
-        default=False,
-    )
-
-    HOSTED_TONGYI_PAID_ENABLED: bool = Field(
-        description="Enable paid access to hosted Anthropic service",
-        default=False,
-    )
-
-    HOSTED_TONGYI_TRIAL_MODELS: str = Field(
-        description="Comma-separated list of available models for trial access",
-        default="",
-    )
-
-    HOSTED_TONGYI_PAID_MODELS: str = Field(
-        description="Comma-separated list of available models for paid access",
-        default="",
-    )
-
 
 class HostedMinmaxConfig(BaseSettings):
     """
@@ -478,13 +246,9 @@ class HostedServiceConfig(
     HostedOpenAiConfig,
     HostedSparkConfig,
     HostedZhipuAIConfig,
-    HostedTongyiConfig,
     # moderation
     HostedModerationConfig,
     # credit config
     HostedCreditConfig,
-    HostedGeminiConfig,
-    HostedXAIConfig,
-    HostedDeepseekConfig,
 ):
     pass

api/configs/middleware/storage/volcengine_tos_storage_config.py

@@ -4,7 +4,7 @@ from pydantic_settings import BaseSettings
 
 class VolcengineTOSStorageConfig(BaseSettings):
     """
-    Configuration settings for Volcengine Torch Object Storage (TOS)
+    Configuration settings for Volcengine Tinder Object Storage (TOS)
     """
 
     VOLCENGINE_TOS_BUCKET_NAME: str | None = Field(

api/configs/middleware/vdb/milvus_config.py

@@ -16,6 +16,7 @@ class MilvusConfig(BaseSettings):
         description="Authentication token for Milvus, if token-based authentication is enabled",
         default=None,
     )
+
     MILVUS_USER: str | None = Field(
         description="Username for authenticating with Milvus, if username/password authentication is enabled",
         default=None,

api/context/__init__.py

@@ -1,74 +0,0 @@
-"""
-Core Context - Framework-agnostic context management.
-
-This module provides context management that is independent of any specific
-web framework. Framework-specific implementations register their context
-capture functions at application initialization time.
-
-This ensures the workflow layer remains completely decoupled from Flask
-or any other web framework.
-"""
-
-import contextvars
-from collections.abc import Callable
-
-from core.workflow.context.execution_context import (
-    ExecutionContext,
-    IExecutionContext,
-    NullAppContext,
-)
-
-# Global capturer function - set by framework-specific modules
-_capturer: Callable[[], IExecutionContext] | None = None
-
-
-def register_context_capturer(capturer: Callable[[], IExecutionContext]) -> None:
-    """
-    Register a context capture function.
-
-    This should be called by framework-specific modules (e.g., Flask)
-    during application initialization.
-
-    Args:
-        capturer: Function that captures current context and returns IExecutionContext
-    """
-    global _capturer
-    _capturer = capturer
-
-
-def capture_current_context() -> IExecutionContext:
-    """
-    Capture current execution context.
-
-    This function uses the registered context capturer. If no capturer
-    is registered, it returns a minimal context with only contextvars
-    (suitable for non-framework environments like tests or standalone scripts).
-
-    Returns:
-        IExecutionContext with captured context
-    """
-    if _capturer is None:
-        # No framework registered - return minimal context
-        return ExecutionContext(
-            app_context=NullAppContext(),
-            context_vars=contextvars.copy_context(),
-        )
-
-    return _capturer()
-
-
-def reset_context_provider() -> None:
-    """
-    Reset the context capturer.
-
-    This is primarily useful for testing to ensure a clean state.
-    """
-    global _capturer
-    _capturer = None
-
-
-__all__ = [
-    "capture_current_context",
-    "register_context_capturer",
-    "reset_context_provider",
-]