Compare commits

..

40 Commits

Author SHA1 Message Date
e9ee897973 fix: resolve remaining CI failures for style checks and unit tests
- Add model_features property and build_execution_context method to
  AgentAppRunner to fix mypy attr-defined errors
- Export WorkflowComment, WorkflowCommentReply, WorkflowCommentMention
  from models/__init__.py to fix import errors
- Add NestedNodeGraphRequest, NestedNodeGraphResponse,
  NestedNodeParameterSchema to services/workflow/entities.py
- Update test_agent_chat_app_runner: tests for invalid LLM mode and
  invalid strategy now reflect unified AgentAppRunner behavior
  (no longer raises ValueError for these cases)

Made-with: Cursor
2026-04-13 16:07:38 +08:00
971828615e fix: resolve CI failures for Python style, DB migration, and unit tests
- Fix type errors in dify_graph/nodes/agent/agent_node.py:
  - Add missing user_id param to get_agent_tool_runtime call
  - Use create_plugin_provider_manager instead of bare ProviderManager()
  - Pass provider_manager to ModelManager constructor
  - Add access_controller param to file_factory.build_from_mapping
  - Fix return type annotation for _fetch_memory
- Fix DB migration chain: update workflow_comments migration to point
  to correct parent after sandbox migration removal
- Fix test_app_generate_service: set AGENT_V2_TRANSPARENT_UPGRADE=False
  in mock config to prevent transparent upgrade intercepting test flow
- Fix test_app_generator: add scalar method to mock db.session
- Fix test_app_models: add AppMode.AGENT to expected modes set
- Remove unnecessary db.session.close() from agent_chat app_runner

Made-with: Cursor
2026-04-13 15:07:16 +08:00
b804c7ed47 fix: restore SandboxExpiredRecordsCleanConfig, remove debug logs
- Restore SandboxExpiredRecordsCleanConfig (billing/ops config that
  was mistakenly removed with sandbox execution code)
- Remove [DEBUG-AGENT] logging from app_generate_service.py

Made-with: Cursor
2026-04-13 14:39:48 +08:00
c7a7c73034 Merge branch 'main' into feat/new-agent-node 2026-04-13 13:58:02 +08:00
94b3087b98 fix: resolve remaining CI failures
- app_model_config_service.py: add AppMode.AGENT to exhaustive match
- app_service.py: fix possibly unbound default_model_dict variable

Made-with: Cursor
2026-04-13 13:56:08 +08:00
3e0578a1c6 Merge branch 'main' into feat/new-agent-node 2026-04-13 13:43:47 +08:00
5f87239abc fix: resolve CI failures — unused imports, type errors, test updates
- Remove 12 unused imports across node.py, tool_manager.py,
  event_adapter.py, legacy_response_adapter.py
- Fix Sequence[str] → list[str] type annotation in node.py
- Update test_agent_chat_app_runner.py: CotChatAgentRunner →
  AgentAppRunner (old runner classes replaced by unified runner)

Made-with: Cursor
2026-04-13 13:10:08 +08:00
c03b25a940 merge: resolve conflicts with origin/main
Conflicts resolved:
- workflow_app_runner.py: adopt main's DifyGraphInitContext pattern
- token_buffer_memory.py: adopt main's match/case, add AppMode.AGENT
- app_dsl_service.py: adopt main's match/case, add AppMode.AGENT

Made-with: Cursor
2026-04-13 12:52:56 +08:00
90cce7693f revert: remove all sandbox and skill related code
Remove ~12,900 lines of sandbox/skill code that was ported from
feat/support-agent-sandbox. This reverts to direct tool execution
(the original behavior before sandbox integration).

Removed:
- core/sandbox/ (SandboxBuilder, bash tools, providers, initializers)
- core/skill/ (SkillManager, assembler, entities)
- core/virtual_environment/ (5 provider implementations)
- core/zip_sandbox/ (archive operations)
- core/app_assets/ (asset management)
- core/app_bundle/ (bundle management)
- controllers/cli_api/ (DifyCli callback endpoints)
- services/sandbox/ (provider service)
- services/skill_service, app_asset_service, app_bundle_service
- models/sandbox.py, app_asset.py
- bin/dify-cli-* (3 platform binaries)
- web sandbox-provider-page and service
- SandboxLayer, _resolve_sandbox_context, _invoke_tool_in_sandbox
- CliApiConfig, DIFY_SANDBOX_CONTEXT_KEY
- sandbox-related migrations

Preserved: All Agent V2 core functionality (agent-v2 node, strategy
engine, transparent upgrade, LLM remapping, memory, context, tools
via direct execution).

Made-with: Cursor
2026-04-13 10:42:36 +08:00
77c182f738 feat(api): propagate all app features in transparent upgrade
VirtualWorkflowSynthesizer._build_features() now extracts ALL legacy
app features from AppModelConfig into the synthesized workflow.features:

- opening_statement + suggested_questions
- sensitive_word_avoidance (keywords/API moderation)
- more_like_this
- speech_to_text / text_to_speech
- retriever_resource

Previously workflow.features was hardcoded to "{}", losing all these
features during transparent upgrade. Now AdvancedChatAppRunner's
moderation, opening text, and other feature layers work correctly
for transparently upgraded old apps.

Made-with: Cursor
2026-04-10 18:47:18 +08:00
e04f00d29b feat(api): add context injection and Jinja2 support to Agent V2 node
Agent V2 now fully covers all LLM node capabilities:
- Context injection: {{#context#}} placeholder replaced with upstream
  knowledge retrieval results via _build_context_string()
- Jinja2 template rendering via _render_jinja2() with variable pool
- Multi-variable references across upstream nodes

Compatibility verified (7/7):
- T1: Context injection ({{#context#}})
- T2: Variable template resolution ({{#start.var#}})
- T3: Multi-upstream variable refs
- T4: Old Chat app with opening_statement
- T5: Old app sensitive_word_avoidance
- T6: Old app more_like_this
- T7: Old Completion app with variable substitution

Made-with: Cursor
2026-04-10 17:05:48 +08:00
bbed99a4cb fix(web): add AGENT mode to AppPreview and AppScreenShot maps
Made-with: Cursor
2026-04-10 16:17:34 +08:00
df6c1064c6 fix(web): resolve all TypeScript errors in Agent V2 frontend
- Fix toast API: use toast.success()/toast.error() instead of object
- Fix panel: use native HTML elements instead of mismatched component APIs
- Add BlockEnum.AgentV2 to block-icon map (icon + color)
- Add BlockEnum.AgentV2 to use-last-run.ts form params maps
- Add i18n keys: blocks.agent-v2, blocksAbout.agent-v2 (en + zh)
- TypeScript: 0 errors

Made-with: Cursor
2026-04-10 16:00:16 +08:00
f4e04fc872 feat(web): add Agent V2 frontend — app creation, node editor, sandbox settings
P0 — Agent App can be created and routed:
- Add AppModeEnum.AGENT to types/app.ts
- Add Agent card to create-app-modal (primary row, with RiRobot2Fill icon)
- Route Agent apps to /workflow editor (same as workflow/advanced-chat)
- Update layout-main.tsx mode guards

P1 — Agent V2 workflow node:
- Add BlockEnum.AgentV2 = 'agent-v2' to workflow types
- Create agent-v2/node.tsx: displays model, strategy, tool count
- Create agent-v2/panel.tsx: model selector, strategy picker, tool list,
  max iterations, memory config, vision toggle
- Register in NodeComponentMap and PanelComponentMap

P2 — Sandbox Provider settings:
- Create sandbox-provider-page: list/configure/activate/delete providers
  (Docker, E2B, SSH, AWS CodeInterpreter)
- Create service/sandbox.ts: API client for sandbox provider endpoints
- Add "Sandbox Providers" to settings menu

i18n: Add en-US and zh-Hans translations for agent V2 description.
Made-with: Cursor
2026-04-10 15:31:48 +08:00
59b9221501 fix(api): fix AWS CodeInterpreter stdout capture failure
Root cause: _WORKDIR was hardcoded to "/home/user" which doesn't exist
in AWS AgentCore Code Interpreter environment (actual pwd is
/opt/amazon/genesis1p-tools/var). Every command was prefixed with
"cd /home/user && ..." which failed silently, producing empty stdout.

Fix:
- Default _WORKDIR to "/tmp" (universally available)
- Auto-detect actual working directory via "pwd" during
  _construct_environment and override _WORKDIR dynamically

Verified: echo, python3, uname all return correct stdout.
Made-with: Cursor
2026-04-10 14:21:06 +08:00
218c10ba4f feat(api): add SSH private key auth support and verify SSH/E2B providers
- SSH Provider: add automatic private key detection in ssh_password
  field (RSA/Ed25519/ECDSA) alongside existing password auth.
- SSH Provider verified end-to-end on EC2: connection, command exec,
  CLI binary upload via SFTP, dify init, tool symlink creation.
- E2B Provider verified: cloud sandbox creation, CLI binary upload,
  dify init with tool symlinks.
- Add linux/amd64 CLI binary for E2B (x86_64 cloud sandboxes).

Made-with: Cursor
2026-04-10 12:57:40 +08:00
4c878da9e6 feat(api): add linux/amd64 dify-cli binary for E2B cloud sandbox
E2B Provider verified end-to-end:
- Cloud sandbox creation/release via E2B API
- CLI binary upload + execution inside E2B
- dify init + symlink creation
- dify execute requires public CLI_API_URL (expected for cloud sandbox)

Made-with: Cursor
2026-04-10 11:40:53 +08:00
698af54c4f feat(api): complete end-to-end Docker sandbox auto tool execution
Full pipeline working: Agent V2 node → Docker container creation →
CLI binary upload (linux/arm64) → dify init (fetch tools from API) →
dify execute (tool callback via CLI API) → result returned.

Fixes:
- Use sandbox.id (not vm.metadata.id) for CLI paths
- Upload CLI binary to container during sandbox creation
- Resolve linux binary separately for Docker containers on macOS
- Save Docker provider config via SandboxProviderService (proper
  encryption) instead of raw DB insert
- Add verbose logging for sandbox tool execution path
- Fix NameError: binary not defined

Made-with: Cursor
2026-04-10 11:28:02 +08:00
10bb276e97 fix(api): complete Docker sandbox tool execution pipeline
- Add linux/arm64 dify-cli binary for Docker containers
- Add DIFY_PORT config field for Docker socat forwarding
- Fix InvokeFrom.AGENT (doesn't exist) → InvokeFrom.DEBUGGER
  in CLI API fetch/tools/batch endpoint

Full pipeline verified: Docker container → dify init → dify execute
→ CLI API callback → plugin invocation → result returned to stdout.

Made-with: Cursor
2026-04-10 11:06:54 +08:00
73fd439541 fix(api): resolve sandbox deadlock under gevent and refine integration
- Skip Local sandbox provider under gevent worker (subprocess pipes
  cause cooperative threading deadlock with Celery's gevent pool).
- Add non-blocking sandbox readiness check before tool execution.
- Add gevent timeout wrapper for sandbox bash session.
- Fix CLI binary resolution: add SANDBOX_DIFY_CLI_ROOT config field.
- Fix ExecutionContext.node_id propagation.
- Fix SkillInitializer to gracefully handle missing skill bundles.
- Update _invoke_tool_in_sandbox to use correct `dify execute` CLI
  subcommand format (not `invoke-tool`).

The full sandbox-in-agent pipeline works end-to-end for network-based
providers (Docker, E2B, SSH). Local provider is skipped under gevent
but works in non-gevent contexts.

Made-with: Cursor
2026-04-10 10:51:40 +08:00
5cdae671d5 feat(api): integrate Sandbox Provider into Agent V2 execution pipeline
Close 3 integration gaps between the ported Sandbox system and Agent V2:

1. Fix _invoke_tool_in_sandbox to use SandboxBashSession context manager
   API correctly (keyword args, bash_tool, ToolReference), with graceful
   fallback to direct invocation when DifyCli binary is unavailable.

2. Inject sandbox into run_context via _resolve_sandbox_context() in
   WorkflowBasedAppRunner — automatically creates a sandbox when a
   tenant has an active sandbox provider configured.

3. Register SandboxLayer in both advanced_chat and workflow app runners
   for proper sandbox lifecycle cleanup on graph end.

Also: make SkillInitializer non-fatal when no skill bundle exists,
add node_id to ExecutionContext for sandbox session scoping.

Made-with: Cursor
2026-04-10 10:14:42 +08:00
e50c36526e fix(api): fix transparent upgrade SSE channel mismatch and chat mode routing
- workflow_execute_task: add AppMode.CHAT/AGENT_CHAT/COMPLETION to the
  AdvancedChatAppGenerator routing branch so transparently upgraded old
  apps can execute through the workflow engine.
- app_generate_service: use app_model.mode (not hardcoded AppMode.AGENT)
  for SSE event subscription channel, ensuring the subscriber and
  Celery publisher use the same Redis channel key.

Made-with: Cursor
2026-04-09 17:27:41 +08:00
2de2a8fd3a fix(api): resolve multi-turn memory failure in Agent apps
- Auto-resolve parent_message_id when not provided by client,
  querying the latest message in the conversation to maintain
  the thread chain that extract_thread_messages() relies on.
- Add AppMode.AGENT to TokenBufferMemory mode checks so file
  attachments in memory are handled via the workflow branch.
- Add debug logging for memory injection in node_factory and node.

Made-with: Cursor
2026-04-09 16:27:38 +08:00
e2e16772a1 fix(api): fix DSL import, memory loading, and remaining test coverage
1. DSL Import fix: change self._session.commit() to self._session.flush()
   in app_dsl_service.py _create_or_update_app() to avoid "closed transaction"
   error. DSL import now works: export agent app -> import -> new app created.

2. Memory loading attempt: added _load_memory_messages() to AgentV2Node
   that loads TokenBufferMemory from conversation history. However, chatflow
   engine manages conversations differently from easy-UI (conversation may
   not be in DB at query time, or uses ConversationVariablePersistenceLayer
   instead of Message table). Memory needs further investigation.

Test results:
- Multi-turn memory: Turn 1 OK, Turn 2 LLM doesn't see history (needs deeper fix)
- Service API with API Key: PASSED (answer="Sixteen" for 8+8)
- DSL Import: PASSED (status=completed, new app created)
- Token aggregation: PASSED (node=49, workflow=49)

Known: memory in multi-turn chatflow needs to use graphon's built-in
memory mechanism (MemoryConfig on node + ConversationVariablePersistenceLayer)
rather than direct DB query.

Made-with: Cursor
2026-04-09 14:47:55 +08:00
b21a443d56 fix(api): resolve all remaining known issues
1. Fix workflow-level total_tokens=0:
   Call graph_runtime_state.add_tokens(usage.total_tokens) in both
   _run_without_tools and _run_with_tools paths after node execution.
   Previously only graphon's internal ModelInvokeCompletedEvent handler
   called add_tokens, which agent-v2 doesn't emit.

2. Fix Turn 2 SSE empty response:
   Set PUBSUB_REDIS_CHANNEL_TYPE=streams in .env. Redis Streams
   provides durable event delivery (consumers can replay past events),
   solving the pub/sub at-most-once timing issue.

3. Skill -> Agent runtime integration:
   SandboxBuilder.build() now auto-includes SkillInitializer if not
   already present. This ensures sandbox.attrs has the skill bundle
   loaded for downstream consumers (tool execution in sandbox).

4. LegacyResponseAdapter:
   New module at core/app/apps/common/legacy_response_adapter.py.
   Filters workflow-specific SSE events (workflow_started, node_started,
   node_finished, workflow_finished) from the stream, passing through
   only message/message_end/agent_log/error/ping events that old
   clients expect.

46 unit tests pass.

Made-with: Cursor
2026-04-09 12:53:11 +08:00
4f010cd4f5 fix(api): stop emitting StreamChunkEvent from tool path to prevent answer duplication
The EventAdapter was converting every LLMResultChunk from the agent
strategy into StreamChunkEvent. Combined with the answer node's
{{#agent.text#}} variable output, this caused the final answer to
appear twice (e.g., "It is 2026-04-09 04:27:45.It is 2026-04-09 04:27:45.").

Now LLMResultChunk from strategy output is silently consumed (text still
accumulates in AgentResult.text via the strategy). Only AgentLogEvent
(thought/tool_call/round) is forwarded to the pipeline.

Known remaining issues:
- workflow/message level total_tokens=0 (node level is correct at 33)
  because pipeline aggregation doesn't include agent-v2 node tokens
- Turn 2 SSE delivery timing with Redis pubsub (celery executes OK)

Made-with: Cursor
2026-04-09 12:31:49 +08:00
3d4be88d97 fix(api): remove unsupported 'user' param from FC/ReAct invoke_llm calls
FunctionCallStrategy and ReActStrategy were passing user=self.context.user_id
to ModelInstance.invoke_llm() which doesn't accept that parameter.
This caused tool-using agent runs to fail with:
  "ModelInstance.invoke_llm() got an unexpected keyword argument 'user'"

Verified: Agent V2 with current_time tool now works end-to-end:
  ROUND 1: LLM thought -> CALL current_time -> got time
  ROUND 2: LLM generates answer with time info
Made-with: Cursor
2026-04-09 12:18:07 +08:00
482a004efe fix(api): fix duplicate answer and completion app upgrade issues
1. Remove StreamChunkEvent from AgentV2Node._run_without_tools():
   The agent-v2 node was yielding StreamChunkEvent during LLM streaming,
   AND the downstream answer node was outputting the same text via
   {{#agent.text#}} variable reference, causing "FourFour" duplication.
   Now text only flows through outputs.text -> answer node (single path).

2. Map inputs to query for completion app transparent upgrade:
   Completion apps send {inputs: {query: "..."}} not {query: "..."}.
   VirtualWorkflowSynthesizer route now extracts query from inputs
   when the top-level query is missing.

Verified:
- Old chat app: "What is 2+2?" -> "Four" (was "FourFour")
- Old completion app: {inputs: {query: "What is 3+3?"}} -> "3 + 3 = 6" (was failing)
- Old agent-chat app: still works

Made-with: Cursor
2026-04-09 12:02:43 +08:00
7052257c8d fix(api): use lazy workflow persistence for transparent upgrade of old apps
VirtualWorkflowSynthesizer.ensure_workflow() creates a real draft
workflow on first call for a legacy app, persisting it to the database.
On subsequent calls, returns the existing draft.

This is needed because AdvancedChatAppGenerator's worker thread looks
up workflows from the database by ID. Instead of hacking the generator
to skip DB lookups, we treat this as a lazy one-time upgrade: the old
app gets a real workflow that can also be edited in the workflow editor.

Verified: old chat app created on main branch ("What is 2+2?" -> "Four")
and old agent-chat app ("Say hello" -> "Hello!") both successfully
execute through the Agent V2 engine with AGENT_V2_TRANSPARENT_UPGRADE=true.

Made-with: Cursor
2026-04-09 11:28:16 +08:00
edfcab6455 fix(api): add AGENT mode to app list filtering
Add AppMode.AGENT branch in get_paginate_apps() so that
filtering apps by mode=agent works correctly.
Discovered during comprehensive E2E testing.

14/14 E2E tests pass covering:
- A: New Agent app full lifecycle (create, draft, configs, publish, run)
- B: Old app creation compat (chat, completion, agent-chat, advanced-chat, workflow)
- C: App listing and filtering (all modes, agent filter)
- D: Workflow editor compat (block configs)
- E: DSL export

Made-with: Cursor
2026-04-09 10:54:05 +08:00
66212e3575 feat(api): implement zero-migration transparent upgrade (Phase 8)
Add two feature-flag-controlled upgrade paths that allow existing apps
and LLM nodes to transparently run through the Agent V2 engine without
any database migration:

1. AGENT_V2_TRANSPARENT_UPGRADE (default: off):
   When enabled, old apps (chat/completion/agent-chat) bypass legacy
   Easy-UI runners. VirtualWorkflowSynthesizer converts AppModelConfig
   to an in-memory Workflow (start -> agent-v2 -> answer) at runtime,
   then executes via AdvancedChatAppGenerator. Falls back to legacy
   path on any synthesis error.

   VirtualWorkflowSynthesizer maps:
   - model JSON -> ModelConfig
   - pre_prompt/chat_prompt_config -> prompt_template
   - agent_mode.tools -> ToolMetadata[]
   - agent_mode.strategy -> agent_strategy
   - dataset_configs -> context
   - file_upload -> vision

2. AGENT_V2_REPLACES_LLM (default: off):
   When enabled, DifyNodeFactory.create_node() transparently remaps
   nodes with type="llm" to type="agent-v2" before class resolution.
   Since AgentV2NodeData is a strict superset of LLMNodeData, the
   mapping is lossless. With tools=[], Agent V2 behaves identically
   to LLM Node.

Both flags default to False for safety. Turn off = instant rollback.
46 existing tests pass. Flask starts successfully.

Made-with: Cursor
2026-04-09 10:30:52 +08:00
96374d7f6a refactor(api): replace legacy agent runners with StrategyFactory in AgentChatAppRunner (Phase 4)
Replace the hardcoded FunctionCallAgentRunner / CotChatAgentRunner /
CotCompletionAgentRunner selection in AgentChatAppRunner with the new
AgentAppRunner class that uses StrategyFactory from Phase 1.

Before: AgentChatAppRunner manually selects FC/CoT runner class based on
model features and LLM mode, then instantiates it directly.

After: AgentChatAppRunner instantiates AgentAppRunner (from sandbox branch),
which internally uses StrategyFactory.create_strategy() to auto-select
the right strategy, and uses ToolInvokeHook for proper agent_invoke
with file handling and thought persistence.

This unifies the agent execution engine: both the new Agent V2 workflow
node and the legacy agent-chat app now use the same StrategyFactory
and AgentPattern implementations.

Also fix: command and file_upload nodes use string node_type instead of
BuiltinNodeTypes.COMMAND/FILE_UPLOAD (not in current graphon version).

46 tests pass. Flask starts successfully.

Made-with: Cursor
2026-04-09 09:42:23 +08:00
44491e427c feat(api): enable all sandbox/skill controller routes and resolve dependencies (P0)
Resolve the full dependency chain to enable all previously disabled controllers:

Enabled routes:
- sandbox_files: sandbox file browser API
- sandbox_providers: sandbox provider management API
- app_asset: app asset management API
- skills: skill extraction API
- CLI API blueprint: DifyCli callback endpoints (/cli/api/*)

Dependencies extracted (64 files, ~8000 lines):
- models/sandbox.py, models/app_asset.py: DB models
- core/zip_sandbox/: zip-based sandbox execution
- core/session/: CLI API session management
- core/memory/: base memory + node token buffer
- core/helper/creators.py: helper utilities
- core/llm_generator/: context models, output models, utils
- core/workflow/nodes/command/: command node type
- core/workflow/nodes/file_upload/: file upload node type
- core/app/entities/: app_asset_entities, app_bundle_entities, llm_generation_entities
- services/: asset_content, skill, workflow_collaboration, workflow_comment
- controllers/console/app/error.py: AppAsset error classes
- core/tools/utils/system_encryption.py

Import fixes:
- dify_graph.enums -> graphon.enums in skill_service.py
- get_signed_file_url_for_plugin -> get_signed_file_url in cli_api.py

All 5 controllers verified: import OK, Flask starts successfully.
46 existing tests still pass.

Made-with: Cursor
2026-04-09 09:36:16 +08:00
d3d9f21cdf feat(api): wire sandbox into Agent V2 node execution pipeline
Integrate the ported sandbox system with Agent V2 node:

- Add DIFY_SANDBOX_CONTEXT_KEY to app_invoke_entities for passing
  sandbox through run_context without modifying graphon
- DifyNodeFactory._resolve_sandbox() extracts sandbox from run_context
  and passes it to AgentV2Node constructor
- AgentV2Node accepts optional sandbox parameter
- AgentV2ToolManager supports dual execution paths:
  - _invoke_tool_directly(): standard ToolEngine.generic_invoke (no sandbox)
  - _invoke_tool_in_sandbox(): delegates to SandboxBashSession.run_tool()
    which uses DifyCli to call back to Dify API from inside the sandbox
- Graceful fallback: if sandbox execution fails, logs warning and returns
  error message (does not crash the agent loop)

To enable sandbox for an Agent workflow:
1. Create a Sandbox via SandboxBuilder
2. Add it to run_context under DIFY_SANDBOX_CONTEXT_KEY
3. Agent V2 nodes will automatically use sandbox for tool execution

46 existing tests still pass.

Made-with: Cursor
2026-04-08 17:46:34 +08:00
0c7e7e0c4e feat(api): port Sandbox + VirtualEnvironment + Skill system from feat/support-agent-sandbox (Phase 5-6)
Port the complete infrastructure for agent sandbox execution and skill system:

Sandbox & Virtual Environment (core/sandbox/, core/virtual_environment/):
- Sandbox entity with lifecycle management (ready/failed/cancelled states)
- SandboxBuilder with fluent API for configuring providers
- 5 VM providers: Local, SSH, Docker, E2B, AWS CodeInterpreter
- VirtualEnvironment base with command execution, file transfer, transport layers
- Channel transport: pipe, queue, socket implementations
- Bash session management and DifyCli binary integration
- Storage: archive storage, file storage, noop storage, presign storage
- Initializers: DifyCli, AppAssets, DraftAppAssets, Skills
- Inspector: file browser, archive/runtime source, script utils
- Security: encryption utils, debug helpers

Skill & App Assets (core/skill/, core/app_assets/, core/app_bundle/):
- Skill entity and manager
- App asset accessor, builder pipeline (file, skill builders)
- App bundle source zip extractor
- Storage and converter utilities

API Endpoints:
- CLI API blueprint (controllers/cli_api/) for sandbox callback
- Sandbox provider management (workspace/sandbox_providers)
- Sandbox file browser (console/sandbox_files)
- App asset management (console/app/app_asset)
- Skill management (console/app/skills)
- Storage file endpoints (controllers/files/storage_files)

Services:
- Sandbox service, provider service, file service
- App asset service, app bundle service

Config:
- CliApiConfig, CreatorsPlatformConfig, CollaborationConfig
- FILES_API_URL for sandbox file access

Note: Controller route registration temporarily commented out (marked TODO)
pending resolution of deep dependency chains (socketio, workflow_comment,
command node, etc.). Core sandbox modules are fully ported and syntax-validated.
110 files changed, 10,549 insertions.

Made-with: Cursor
2026-04-08 17:39:02 +08:00
d9d1e9b63a fix(api): resolve Agent V2 node E2E runtime issues
Fixes discovered during end-to-end testing of Agent workflow execution:

1. ModelManager instantiation: use ModelManager.for_tenant() instead of
   ModelManager() which requires a ProviderManager argument
2. Variable template resolution: use VariableTemplateParser(template).format()
   instead of non-existent resolve_template() static method
3. invoke_llm() signature: remove unsupported 'user' keyword argument
4. Event dispatch: remove ModelInvokeCompletedEvent from _run() yield
   (graphon base Node._dispatch doesn't support it via singledispatch)
5. NodeRunResult metadata: use WorkflowNodeExecutionMetadataKey enum keys
   (TOTAL_TOKENS, TOTAL_PRICE, CURRENCY) instead of arbitrary string keys
6. SSE topic mismatch: use AppMode.AGENT (not ADVANCED_CHAT) in
   retrieve_events() so publisher and subscriber share the same channel
7. Celery task routing: add AppMode.AGENT to workflow_execute_task._run_app()
   alongside ADVANCED_CHAT

All issues verified fixed: Agent V2 node successfully invokes LLM and
returns "Hello there!" through the full SSE streaming pipeline.

Made-with: Cursor
2026-04-08 16:21:12 +08:00
bebafaa346 fix(api): allow AGENT mode in console chat, message, and debug endpoints
Add AppMode.AGENT to mode checks discovered during E2E testing:
- Console chat-messages endpoint (ChatApi)
- Console chat stop endpoint (ChatMessageStopApi)
- Console message list and detail endpoints
- Advanced-chat debug run endpoints (5 in workflow.py)
- Advanced-chat workflow run endpoints (2 in workflow_run.py)

Made-with: Cursor
2026-04-08 13:27:42 +08:00
1835a1dc5d fix(api): allow AGENT mode in workflow features validation
Add AppMode.AGENT to validate_features_structure() match case
alongside ADVANCED_CHAT, fixing 'Invalid app mode: agent' error
when creating Agent apps (which auto-generate a workflow draft).

Discovered during E2E testing of the full create -> draft -> publish flow.

Made-with: Cursor
2026-04-08 13:19:59 +08:00
8f3a3ea03e feat(api): enable Agent mode in workflow/service APIs and add default config (Phase 7)
Ensure new Agent apps (AppMode.AGENT) can access all workflow-related
APIs and Service API chat endpoints:

- Add AppMode.AGENT to 13 workflow controller mode checks
- Add AppMode.AGENT to 4 workflow_run controller mode checks
- Add AppMode.AGENT to workflow_draft_variable controller
- Add AppMode.AGENT to Service API chat, conversation, message endpoints
- Add AgentV2Node.get_default_config() with prompt templates and strategy defaults
- 46 unit tests all passing (8 new Phase 7 tests)

Old agent/agent-chat paths remain completely unchanged.

Made-with: Cursor
2026-04-08 12:41:37 +08:00
96641a93f6 feat(api): add Agent V2 node and new Agent app type (Phase 1-3)
Introduce a new unified Agent V2 workflow node that combines LLM capabilities
with agent tool-calling loops, along with a new AppMode.AGENT for standalone
agent apps backed by single-node workflows.

Phase 1 — Agent Patterns:
- Add core/agent/patterns/ module (AgentPattern, FunctionCallStrategy,
  ReActStrategy, StrategyFactory) ported from feat/support-agent-sandbox
- Add ExecutionContext, AgentLog, AgentResult entities
- Add Tool.to_prompt_message_tool() for LLM-consumable tool conversion

Phase 2 — Agent V2 Workflow Node:
- Add core/workflow/nodes/agent_v2/ (AgentV2Node, AgentV2NodeData,
  AgentV2ToolManager, AgentV2EventAdapter)
- Register agent-v2 node type in DifyNodeFactory
- No-tools path: single LLM call (LLM Node equivalent)
- Tools path: FC/ReAct loop via StrategyFactory

Phase 3 — Agent App Type:
- Add AppMode.AGENT to model enum
- Add WorkflowGraphFactory for auto-generating start->agent_v2->answer graphs
- AppService.create_app() creates workflow draft for AGENT mode
- AppGenerateService.generate() routes AGENT to AdvancedChatAppGenerator
- Console API and DSL import/export support AGENT mode
- Default app template for AGENT mode

Old agent/agent-chat/LLM node paths are fully preserved.
38 unit tests all passing.

Made-with: Cursor
2026-04-08 12:31:23 +08:00
5475 changed files with 129384 additions and 259696 deletions

View File

@ -63,7 +63,7 @@ pnpm analyze-component <path> --json
```typescript ```typescript
// ❌ Before: Complex state logic in component // ❌ Before: Complex state logic in component
function Configuration() { const Configuration: FC = () => {
const [modelConfig, setModelConfig] = useState<ModelConfig>(...) const [modelConfig, setModelConfig] = useState<ModelConfig>(...)
const [datasetConfigs, setDatasetConfigs] = useState<DatasetConfigs>(...) const [datasetConfigs, setDatasetConfigs] = useState<DatasetConfigs>(...)
const [completionParams, setCompletionParams] = useState<FormValue>({}) const [completionParams, setCompletionParams] = useState<FormValue>({})
@ -85,7 +85,7 @@ export const useModelConfig = (appId: string) => {
} }
// Component becomes cleaner // Component becomes cleaner
function Configuration() { const Configuration: FC = () => {
const { modelConfig, setModelConfig } = useModelConfig(appId) const { modelConfig, setModelConfig } = useModelConfig(appId)
return <div>...</div> return <div>...</div>
} }
@ -189,6 +189,8 @@ const Template = useMemo(() => {
**Dify Convention**: **Dify Convention**:
- This skill is for component decomposition, not query/mutation design. - This skill is for component decomposition, not query/mutation design.
- When refactoring data fetching, follow `web/AGENTS.md`.
- Use `frontend-query-mutation` for contracts, query shape, data-fetching wrappers, query/mutation call-site patterns, conditional queries, invalidation, and mutation error handling.
- Do not introduce deprecated `useInvalid` / `useReset`. - Do not introduce deprecated `useInvalid` / `useReset`.
- Do not add thin passthrough `useQuery` wrappers during refactoring; only extract a custom hook when it truly orchestrates multiple queries/mutations or shared derived state. - Do not add thin passthrough `useQuery` wrappers during refactoring; only extract a custom hook when it truly orchestrates multiple queries/mutations or shared derived state.
@ -365,7 +367,7 @@ For each extraction:
┌────────────────────────────────────────┐ ┌────────────────────────────────────────┐
│ 1. Extract code │ │ 1. Extract code │
│ 2. Run: pnpm lint:fix │ │ 2. Run: pnpm lint:fix │
│ 3. Run: pnpm type-check │ 3. Run: pnpm type-check:tsgo
│ 4. Run: pnpm test │ │ 4. Run: pnpm test │
│ 5. Test functionality manually │ │ 5. Test functionality manually │
│ 6. PASS? → Next extraction │ │ 6. PASS? → Next extraction │

View File

@ -60,10 +60,8 @@ const Template = useMemo(() => {
**After** (complexity: ~3): **After** (complexity: ~3):
```typescript ```typescript
import type { ComponentType } from 'react'
// Define lookup table outside component // Define lookup table outside component
const TEMPLATE_MAP: Record<AppModeEnum, Record<string, ComponentType<TemplateProps>>> = { const TEMPLATE_MAP: Record<AppModeEnum, Record<string, FC<TemplateProps>>> = {
[AppModeEnum.CHAT]: { [AppModeEnum.CHAT]: {
[LanguagesSupported[1]]: TemplateChatZh, [LanguagesSupported[1]]: TemplateChatZh,
[LanguagesSupported[7]]: TemplateChatJa, [LanguagesSupported[7]]: TemplateChatJa,

View File

@ -65,10 +65,10 @@ interface ConfigurationHeaderProps {
onPublish: () => void onPublish: () => void
} }
function ConfigurationHeader({ const ConfigurationHeader: FC<ConfigurationHeaderProps> = ({
isAdvancedMode, isAdvancedMode,
onPublish, onPublish,
}: ConfigurationHeaderProps) { }) => {
const { t } = useTranslation() const { t } = useTranslation()
return ( return (
@ -136,7 +136,7 @@ const AppInfo = () => {
} }
// ✅ After: Separate view components // ✅ After: Separate view components
function AppInfoExpanded({ appDetail, onAction }: AppInfoViewProps) { const AppInfoExpanded: FC<AppInfoViewProps> = ({ appDetail, onAction }) => {
return ( return (
<div className="expanded"> <div className="expanded">
{/* Clean, focused expanded view */} {/* Clean, focused expanded view */}
@ -144,7 +144,7 @@ function AppInfoExpanded({ appDetail, onAction }: AppInfoViewProps) {
) )
} }
function AppInfoCollapsed({ appDetail, onAction }: AppInfoViewProps) { const AppInfoCollapsed: FC<AppInfoViewProps> = ({ appDetail, onAction }) => {
return ( return (
<div className="collapsed"> <div className="collapsed">
{/* Clean, focused collapsed view */} {/* Clean, focused collapsed view */}
@ -203,12 +203,12 @@ interface AppInfoModalsProps {
onSuccess: () => void onSuccess: () => void
} }
function AppInfoModals({ const AppInfoModals: FC<AppInfoModalsProps> = ({
appDetail, appDetail,
activeModal, activeModal,
onClose, onClose,
onSuccess, onSuccess,
}: AppInfoModalsProps) { }) => {
const handleEdit = async (data) => { /* logic */ } const handleEdit = async (data) => { /* logic */ }
const handleDuplicate = async (data) => { /* logic */ } const handleDuplicate = async (data) => { /* logic */ }
const handleDelete = async () => { /* logic */ } const handleDelete = async () => { /* logic */ }
@ -296,7 +296,7 @@ interface OperationItemProps {
onAction: (id: string) => void onAction: (id: string) => void
} }
function OperationItem({ operation, onAction }: OperationItemProps) { const OperationItem: FC<OperationItemProps> = ({ operation, onAction }) => {
return ( return (
<div className="operation-item"> <div className="operation-item">
<span className="icon">{operation.icon}</span> <span className="icon">{operation.icon}</span>
@ -435,7 +435,7 @@ interface ChildProps {
onSubmit: () => void onSubmit: () => void
} }
function Child({ value, onChange, onSubmit }: ChildProps) { const Child: FC<ChildProps> = ({ value, onChange, onSubmit }) => {
return ( return (
<div> <div>
<input value={value} onChange={e => onChange(e.target.value)} /> <input value={value} onChange={e => onChange(e.target.value)} />

View File

@ -112,13 +112,13 @@ export const useModelConfig = ({
```typescript ```typescript
// Before: 50+ lines of state management // Before: 50+ lines of state management
function Configuration() { const Configuration: FC = () => {
const [modelConfig, setModelConfig] = useState<ModelConfig>(...) const [modelConfig, setModelConfig] = useState<ModelConfig>(...)
// ... lots of related state and effects // ... lots of related state and effects
} }
// After: Clean component // After: Clean component
function Configuration() { const Configuration: FC = () => {
const { const {
modelConfig, modelConfig,
setModelConfig, setModelConfig,
@ -159,6 +159,8 @@ function Configuration() {
When hook extraction touches query or mutation code, do not use this reference as the source of truth for data-layer patterns. When hook extraction touches query or mutation code, do not use this reference as the source of truth for data-layer patterns.
- Follow `web/AGENTS.md` first.
- Use `frontend-query-mutation` for contracts, query shape, data-fetching wrappers, query/mutation call-site patterns, conditional queries, invalidation, and mutation error handling.
- Do not introduce deprecated `useInvalid` / `useReset`. - Do not introduce deprecated `useInvalid` / `useReset`.
- Do not extract thin passthrough `useQuery` hooks; only extract orchestration hooks. - Do not extract thin passthrough `useQuery` hooks; only extract orchestration hooks.

View File

@ -1,79 +0,0 @@
---
name: e2e-cucumber-playwright
description: Write, update, or review Dify end-to-end tests under `e2e/` that use Cucumber, Gherkin, and Playwright. Use when the task involves `.feature` files, `features/step-definitions/`, `features/support/`, `DifyWorld`, scenario tags, locator/assertion choices, or E2E testing best practices for this repository.
---
# Dify E2E Cucumber + Playwright
Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical guide for local architecture and conventions, then apply Playwright/Cucumber best practices only where they fit the current suite.
## Scope
- Use this skill for `.feature` files, Cucumber step definitions, `DifyWorld`, hooks, tags, and E2E review work under `e2e/`.
- Do not use this skill for Vitest or React Testing Library work under `web/`; use `frontend-testing` instead.
- Do not use this skill for backend test or API review tasks under `api/`.
## Read Order
1. Read [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) first.
2. Read only the files directly involved in the task:
- target `.feature` files under `e2e/features/`
- related step files under `e2e/features/step-definitions/`
- `e2e/features/support/hooks.ts` and `e2e/features/support/world.ts` when session lifecycle or shared state matters
- `e2e/scripts/run-cucumber.ts` and `e2e/cucumber.config.ts` when tags or execution flow matter
3. Read [`references/playwright-best-practices.md`](references/playwright-best-practices.md) only when locator, assertion, isolation, or waiting choices are involved.
4. Read [`references/cucumber-best-practices.md`](references/cucumber-best-practices.md) only when scenario wording, step granularity, tags, or expression design are involved.
5. Re-check official Playwright or Cucumber docs with the available documentation tools before introducing a new framework pattern.
## Local Rules
- `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
- `DifyWorld` is the per-scenario context object. Type `this` as `DifyWorld` and use `async function`, not arrow functions.
- Keep glue organized by capability under `e2e/features/step-definitions/`; use `common/` only for broadly reusable steps.
- Browser session behavior comes from `features/support/hooks.ts`:
- default: authenticated session with shared storage state
- `@unauthenticated`: clean browser context
- `@authenticated`: readability/selective-run tag only unless implementation changes
- `@fresh`: only for `e2e:full*` flows
- Do not import Playwright Test runner patterns that bypass the current Cucumber + `DifyWorld` architecture unless the task is explicitly about changing that architecture.
## Workflow
1. Rebuild local context.
- Inspect the target feature area.
- Reuse an existing step when wording and behavior already match.
- Add a new step only for a genuinely new user action or assertion.
- Keep edits close to the current capability folder unless the step is broadly reusable.
2. Write behavior-first scenarios.
- Describe user-observable behavior, not DOM mechanics.
- Keep each scenario focused on one workflow or outcome.
- Keep scenarios independent and re-runnable.
3. Write step definitions in the local style.
- Keep one step to one user-visible action or one assertion.
- Prefer Cucumber Expressions such as `{string}` and `{int}`.
- Scope locators to stable containers when the page has repeated elements.
- Avoid page-object layers or extra helper abstractions unless repeated complexity clearly justifies them.
4. Use Playwright in the local style.
- Prefer user-facing locators: `getByRole`, `getByLabel`, `getByPlaceholder`, `getByText`, then `getByTestId` for explicit contracts.
- Use web-first `expect(...)` assertions.
- Do not use `waitForTimeout`, manual polling, or raw visibility checks when a locator action or retrying assertion already expresses the behavior.
5. Validate narrowly.
- Run the narrowest tagged scenario or flow that exercises the change.
- Run `pnpm -C e2e check`.
- Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.
## Review Checklist
- Does the scenario describe behavior rather than implementation?
- Does it fit the current session model, tags, and `DifyWorld` usage?
- Should an existing step be reused instead of adding a new one?
- Are locators user-facing and assertions web-first?
- Does the change introduce hidden coupling across scenarios, tags, or instance state?
- Does it document or implement behavior that differs from the real hooks or configuration?
Lead findings with correctness, flake risk, and architecture drift.
## References
- [`references/playwright-best-practices.md`](references/playwright-best-practices.md)
- [`references/cucumber-best-practices.md`](references/cucumber-best-practices.md)

View File

@ -1,4 +0,0 @@
interface:
display_name: "E2E Cucumber + Playwright"
short_description: "Write and review Dify E2E scenarios."
default_prompt: "Use $e2e-cucumber-playwright to write or review a Dify E2E scenario under e2e/."

View File

@ -1,93 +0,0 @@
# Cucumber Best Practices For Dify E2E
Use this reference when writing or reviewing Gherkin scenarios, step definitions, parameter expressions, and step reuse in Dify's `e2e/` suite.
Official sources:
- https://cucumber.io/docs/guides/10-minute-tutorial/
- https://cucumber.io/docs/cucumber/step-definitions/
- https://cucumber.io/docs/cucumber/cucumber-expressions/
## What Matters Most
### 1. Treat scenarios as executable specifications
Cucumber scenarios should describe examples of behavior, not test implementation recipes.
Apply it like this:
- write what the user does and what should happen
- avoid UI-internal wording such as selector details, DOM structure, or component names
- keep language concrete enough that the scenario reads like living documentation
### 2. Keep scenarios focused
A scenario should usually prove one workflow or business outcome. If a scenario wanders across several unrelated behaviors, split it.
In Dify's suite, this means:
- one capability-focused scenario per feature path
- no long setup chains when existing bootstrap or reusable steps already cover them
- no hidden dependency on another scenario's side effects
### 3. Reuse steps, but only when behavior really matches
Good reuse reduces duplication. Bad reuse hides meaning.
Prefer reuse when:
- the user action is genuinely the same
- the expected outcome is genuinely the same
- the wording stays natural across features
Write a new step when:
- the behavior is materially different
- reusing the old wording would make the scenario misleading
- a supposedly generic step would become an implementation-detail wrapper
### 4. Prefer Cucumber Expressions
Use Cucumber Expressions for parameters unless regex is clearly necessary.
Common examples:
- `{string}` for labels, names, and visible text
- `{int}` for counts
- `{float}` for decimal values
- `{word}` only when the value is truly a single token
Keep expressions readable. If a step needs complicated parsing logic, first ask whether the scenario wording should be simpler.
### 5. Keep step definitions thin and meaningful
Step definitions are glue between Gherkin and automation, not a second abstraction language.
For Dify:
- type `this` as `DifyWorld`
- use `async function`
- keep each step to one user-visible action or assertion
- rely on `DifyWorld` and existing support code for shared context
- avoid leaking cross-scenario state
### 6. Use tags intentionally
Tags should communicate run scope or session semantics, not become ad hoc metadata.
In Dify's current suite:
- capability tags group related scenarios
- `@unauthenticated` changes session behavior
- `@authenticated` is descriptive/selective, not a behavior switch by itself
- `@fresh` belongs to reset/full-install flows only
If a proposed tag implies behavior, verify that hooks or runner configuration actually implement it.
## Review Questions
- Does the scenario read like a real example of product behavior?
- Are the steps behavior-oriented instead of implementation-oriented?
- Is a reused step still truthful in this feature?
- Is a new tag documenting real behavior, or inventing semantics that the suite does not implement?
- Would a new reader understand the outcome without opening the step-definition file?

View File

@ -1,96 +0,0 @@
# Playwright Best Practices For Dify E2E
Use this reference when writing or reviewing locator, assertion, isolation, or synchronization logic for Dify's Cucumber-based E2E suite.
Official sources:
- https://playwright.dev/docs/best-practices
- https://playwright.dev/docs/locators
- https://playwright.dev/docs/test-assertions
- https://playwright.dev/docs/browser-contexts
## What Matters Most
### 1. Keep scenarios isolated
Playwright's model is built around clean browser contexts so one test does not leak into another. In Dify's suite, that principle maps to per-scenario session setup in `features/support/hooks.ts` and `DifyWorld`.
Apply it like this:
- do not depend on another scenario having run first
- do not persist ad hoc scenario state outside `DifyWorld`
- do not couple ordinary scenarios to `@fresh` behavior
- when a flow needs special auth/session semantics, express that through the existing tag model or explicit hook changes
### 2. Prefer user-facing locators
Playwright recommends built-in locators that reflect what users perceive on the page.
Preferred order in this repository:
1. `getByRole`
2. `getByLabel`
3. `getByPlaceholder`
4. `getByText`
5. `getByTestId` when an explicit test contract is the most stable option
Avoid raw CSS/XPath selectors unless no stable user-facing contract exists and adding one is not practical.
Also remember:
- repeated content usually needs scoping to a stable container
- exact text matching is often too brittle when role/name or label already exists
- `getByTestId` is acceptable when semantics are weak but the contract is intentional
### 3. Use web-first assertions
Playwright assertions auto-wait and retry. Prefer them over manual state inspection.
Prefer:
- `await expect(page).toHaveURL(...)`
- `await expect(locator).toBeVisible()`
- `await expect(locator).toBeHidden()`
- `await expect(locator).toBeEnabled()`
- `await expect(locator).toHaveText(...)`
Avoid:
- `expect(await locator.isVisible()).toBe(true)`
- custom polling loops for DOM state
- `waitForTimeout` as synchronization
If a condition genuinely needs custom retry logic, use Playwright's polling/assertion tools deliberately and keep that choice local and explicit.
### 4. Let actions wait for actionability
Locator actions already wait for the element to be actionable. Do not preface every click/fill with extra timing logic unless the action needs a specific visible/ready assertion for clarity.
Good pattern:
- assert a meaningful visible state when that is part of the behavior
- then click/fill/select via locator APIs
Bad pattern:
- stack arbitrary waits before every action
- wait on unstable implementation details instead of the visible state the user cares about
### 5. Match debugging to the current suite
Playwright's wider ecosystem supports traces and rich debugging tools. Dify's current suite already captures:
- full-page screenshots
- page HTML
- console errors
- page errors
Use the existing artifact flow by default. If a task is specifically about improving diagnostics, confirm the change fits the current Cucumber architecture before importing broader Playwright tooling.
## Review Questions
- Would this locator survive DOM refactors that do not change user-visible behavior?
- Is this assertion using Playwright's retrying semantics?
- Is any explicit wait masking a real readiness problem?
- Does this code preserve per-scenario isolation?
- Is a new abstraction really needed, or does it bypass the existing `DifyWorld` + step-definition model?

View File

@ -9,18 +9,18 @@ Category: Performance
When rendering React Flow, prefer `useNodes`/`useEdges` for UI consumption and rely on `useStoreApi` inside callbacks that mutate or read node/edge state. Avoid manually pulling Flow data outside of these hooks. When rendering React Flow, prefer `useNodes`/`useEdges` for UI consumption and rely on `useStoreApi` inside callbacks that mutate or read node/edge state. Avoid manually pulling Flow data outside of these hooks.
## Complex prop stability ## Complex prop memoization
IsUrgent: False IsUrgent: True
Category: Performance Category: Performance
### Description ### Description
Only require stable object, array, or map props when there is a clear reason: the child is memoized, the value participates in effect/query dependencies, the value is part of a stable-reference API contract, or profiling/local behavior shows avoidable re-renders. Do not request `useMemo` for every inline object by default; `how-to-write-component` treats memoization as a targeted optimization. Wrap complex prop values (objects, arrays, maps) in `useMemo` prior to passing them into child components to guarantee stable references and prevent unnecessary renders.
Update this file when adding, editing, or removing Performance rules so the catalog remains accurate. Update this file when adding, editing, or removing Performance rules so the catalog remains accurate.
Risky: Wrong:
```tsx ```tsx
<HeavyComp <HeavyComp
@ -31,7 +31,7 @@ Risky:
/> />
``` ```
Better when stable identity matters: Right:
```tsx ```tsx
const config = useMemo(() => ({ const config = useMemo(() => ({

View File

@ -0,0 +1,44 @@
---
name: frontend-query-mutation
description: Guide for implementing Dify frontend query and mutation patterns with TanStack Query and oRPC. Trigger when creating or updating contracts in web/contract, wiring router composition, consuming consoleQuery or marketplaceQuery in components or services, deciding whether to call queryOptions() directly or extract a helper or use-* hook, handling conditional queries, cache invalidation, mutation error handling, or migrating legacy service calls to contract-first query and mutation helpers.
---
# Frontend Query & Mutation
## Intent
- Keep contract as the single source of truth in `web/contract/*`.
- Prefer contract-shaped `queryOptions()` and `mutationOptions()`.
- Keep invalidation and mutation flow knowledge in the service layer.
- Keep abstractions minimal to preserve TypeScript inference.
## Workflow
1. Identify the change surface.
- Read `references/contract-patterns.md` for contract files, router composition, client helpers, and query or mutation call-site shape.
- Read `references/runtime-rules.md` for conditional queries, invalidation, error handling, and legacy migrations.
- Read both references when a task spans contract shape and runtime behavior.
2. Implement the smallest abstraction that fits the task.
- Default to direct `useQuery(...)` or `useMutation(...)` calls with oRPC helpers at the call site.
- Extract a small shared query helper only when multiple call sites share the same extra options.
- Create `web/service/use-{domain}.ts` only for orchestration or shared domain behavior.
3. Preserve Dify conventions.
- Keep contract inputs in `{ params, query?, body? }` shape.
- Bind invalidation in the service-layer mutation definition.
- Prefer `mutate(...)`; use `mutateAsync(...)` only when Promise semantics are required.
## Files Commonly Touched
- `web/contract/console/*.ts`
- `web/contract/marketplace.ts`
- `web/contract/router.ts`
- `web/service/client.ts`
- `web/service/use-*.ts`
- component and hook call sites using `consoleQuery` or `marketplaceQuery`
## References
- Use `references/contract-patterns.md` for contract shape, router registration, query and mutation helpers, and anti-patterns that degrade inference.
- Use `references/runtime-rules.md` for conditional queries, invalidation, `mutate` versus `mutateAsync`, and legacy migration rules.
Treat this skill as the single query and mutation entry point for Dify frontend work. Keep detailed rules in the reference files instead of duplicating them in project docs.

View File

@ -0,0 +1,4 @@
interface:
display_name: "Frontend Query & Mutation"
short_description: "Dify TanStack Query and oRPC patterns"
default_prompt: "Use this skill when implementing or reviewing Dify frontend contracts, query and mutation call sites, conditional queries, invalidation, or legacy query/mutation migrations."

View File

@ -0,0 +1,98 @@
# Contract Patterns
## Table of Contents
- Intent
- Minimal structure
- Core workflow
- Query usage decision rule
- Mutation usage decision rule
- Anti-patterns
- Contract rules
- Type export
## Intent
- Keep contract as the single source of truth in `web/contract/*`.
- Default query usage to call-site `useQuery(consoleQuery|marketplaceQuery.xxx.queryOptions(...))` when endpoint behavior maps 1:1 to the contract.
- Keep abstractions minimal and preserve TypeScript inference.
## Minimal Structure
```text
web/contract/
├── base.ts
├── router.ts
├── marketplace.ts
└── console/
├── billing.ts
└── ...other domains
web/service/client.ts
```
## Core Workflow
1. Define contract in `web/contract/console/{domain}.ts` or `web/contract/marketplace.ts`.
- Use `base.route({...}).output(type<...>())` as the baseline.
- Add `.input(type<...>())` only when the request has `params`, `query`, or `body`.
- For `GET` without input, omit `.input(...)`; do not use `.input(type<unknown>())`.
2. Register contract in `web/contract/router.ts`.
- Import directly from domain files and nest by API prefix.
3. Consume from UI call sites via oRPC query utilities.
```typescript
import { useQuery } from '@tanstack/react-query'
import { consoleQuery } from '@/service/client'
const invoiceQuery = useQuery(consoleQuery.billing.invoices.queryOptions({
staleTime: 5 * 60 * 1000,
throwOnError: true,
select: invoice => invoice.url,
}))
```
## Query Usage Decision Rule
1. Default to direct `*.queryOptions(...)` usage at the call site.
2. If 3 or more call sites share the same extra options, extract a small query helper, not a `use-*` passthrough hook.
3. Create `web/service/use-{domain}.ts` only for orchestration.
- Combine multiple queries or mutations.
- Share domain-level derived state or invalidation helpers.
```typescript
const invoicesBaseQueryOptions = () =>
consoleQuery.billing.invoices.queryOptions({ retry: false })
const invoiceQuery = useQuery({
...invoicesBaseQueryOptions(),
throwOnError: true,
})
```
## Mutation Usage Decision Rule
1. Default to mutation helpers from `consoleQuery` or `marketplaceQuery`, for example `useMutation(consoleQuery.billing.bindPartnerStack.mutationOptions(...))`.
2. If the mutation flow is heavily custom, use oRPC clients as `mutationFn`, for example `consoleClient.xxx` or `marketplaceClient.xxx`, instead of handwritten non-oRPC mutation logic.
## Anti-Patterns
- Do not wrap `useQuery` with `options?: Partial<UseQueryOptions>`.
- Do not split local `queryKey` and `queryFn` when oRPC `queryOptions` already exists and fits the use case.
- Do not create thin `use-*` passthrough hooks for a single endpoint.
- These patterns can degrade inference, especially around `throwOnError` and `select`, and add unnecessary indirection.
## Contract Rules
- Input structure: always use `{ params, query?, body? }`.
- No-input `GET`: omit `.input(...)`; do not use `.input(type<unknown>())`.
- Path params: use `{paramName}` in the path and match it in the `params` object.
- Router nesting: group by API prefix, for example `/billing/*` becomes `billing: {}`.
- No barrel files: import directly from specific files.
- Types: import from `@/types/` and use the `type<T>()` helper.
- Mutations: prefer `mutationOptions`; use explicit `mutationKey` mainly for defaults, filtering, and devtools.
## Type Export
```typescript
export type ConsoleInputs = InferContractRouterInputs<typeof consoleRouterContract>
```

View File

@ -0,0 +1,130 @@
# Runtime Rules
## Table of Contents
- Conditional queries
- Cache invalidation
- Key API guide
- `mutate` vs `mutateAsync`
- Legacy migration
## Conditional Queries
Prefer contract-shaped `queryOptions(...)`.
When required input is missing, prefer `input: skipToken` instead of placeholder params or non-null assertions.
Use `enabled` only for extra business gating after the input itself is already valid.
```typescript
import { skipToken, useQuery } from '@tanstack/react-query'
// Disable the query by skipping input construction.
function useAccessMode(appId: string | undefined) {
return useQuery(consoleQuery.accessControl.appAccessMode.queryOptions({
input: appId
? { params: { appId } }
: skipToken,
}))
}
// Avoid runtime-only guards that bypass type checking.
function useBadAccessMode(appId: string | undefined) {
return useQuery(consoleQuery.accessControl.appAccessMode.queryOptions({
input: { params: { appId: appId! } },
enabled: !!appId,
}))
}
```
## Cache Invalidation
Bind invalidation in the service-layer mutation definition.
Components may add UI feedback in call-site callbacks, but they should not decide which queries to invalidate.
Use:
- `.key()` for namespace or prefix invalidation
- `.queryKey(...)` only for exact cache reads or writes such as `getQueryData` and `setQueryData`
- `queryClient.invalidateQueries(...)` in mutation `onSuccess`
Do not use deprecated `useInvalid` from `use-base.ts`.
```typescript
// Service layer owns cache invalidation.
export const useUpdateAccessMode = () => {
const queryClient = useQueryClient()
return useMutation(consoleQuery.accessControl.updateAccessMode.mutationOptions({
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: consoleQuery.accessControl.appWhitelistSubjects.key(),
})
},
}))
}
// Component only adds UI behavior.
updateAccessMode({ appId, mode }, {
onSuccess: () => toast.success('...'),
})
// Avoid putting invalidation knowledge in the component.
mutate({ appId, mode }, {
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: consoleQuery.accessControl.appWhitelistSubjects.key(),
})
},
})
```
## Key API Guide
- `.key(...)`
- Use for partial matching operations.
- Prefer it for invalidation, refetch, and cancel patterns.
- Example: `queryClient.invalidateQueries({ queryKey: consoleQuery.billing.key() })`
- `.queryKey(...)`
- Use for a specific query's full key.
- Prefer it for exact cache addressing and direct reads or writes.
- `.mutationKey(...)`
- Use for a specific mutation's full key.
- Prefer it for mutation defaults registration, mutation-status filtering, and devtools grouping.
## `mutate` vs `mutateAsync`
Prefer `mutate` by default.
Use `mutateAsync` only when Promise semantics are truly required, such as parallel mutations or sequential steps with result dependencies.
Rules:
- Event handlers should usually call `mutate(...)` with `onSuccess` or `onError`.
- Every `await mutateAsync(...)` must be wrapped in `try/catch`.
- Do not use `mutateAsync` when callbacks already express the flow clearly.
```typescript
// Default case.
mutation.mutate(data, {
onSuccess: result => router.push(result.url),
})
// Promise semantics are required.
try {
const order = await createOrder.mutateAsync(orderData)
await confirmPayment.mutateAsync({ orderId: order.id, token })
router.push(`/orders/${order.id}`)
}
catch (error) {
toast.error(error instanceof Error ? error.message : 'Unknown error')
}
```
## Legacy Migration
When touching old code, migrate it toward these rules:
| Old pattern | New pattern |
|---|---|
| `useInvalid(key)` in service layer | `queryClient.invalidateQueries(...)` inside mutation `onSuccess` |
| component-triggered invalidation after mutation | move invalidation into the service-layer mutation definition |
| imperative fetch plus manual invalidation | wrap it in `useMutation(...mutationOptions(...))` |
| `await mutateAsync()` without `try/catch` | switch to `mutate(...)` or add `try/catch` |

View File

@ -5,7 +5,7 @@ description: Generate Vitest + React Testing Library tests for Dify frontend com
# Dify Frontend Testing Skill # Dify Frontend Testing Skill
This skill enables Codex to generate high-quality, comprehensive frontend tests for the Dify project following established conventions and best practices. This skill enables Claude to generate high-quality, comprehensive frontend tests for the Dify project following established conventions and best practices.
> **⚠️ Authoritative Source**: This skill is derived from `web/docs/test.md`. Use Vitest mock/timer APIs (`vi.*`). > **⚠️ Authoritative Source**: This skill is derived from `web/docs/test.md`. Use Vitest mock/timer APIs (`vi.*`).
@ -24,27 +24,35 @@ Apply this skill when the user:
**Do NOT apply** when: **Do NOT apply** when:
- User is asking about backend/API tests (Python/pytest) - User is asking about backend/API tests (Python/pytest)
- User is asking about E2E tests (Cucumber + Playwright under `e2e/`) - User is asking about E2E tests (Playwright/Cypress)
- User is only asking conceptual questions without code context - User is only asking conceptual questions without code context
## Quick Reference ## Quick Reference
### Key Commands ### Tech Stack
Run these commands from `web/`. From the repository root, prefix them with `pnpm -C web`. | Tool | Version | Purpose |
|------|---------|---------|
| Vitest | 4.0.16 | Test runner |
| React Testing Library | 16.0 | Component testing |
| jsdom | - | Test environment |
| nock | 14.0 | HTTP mocking |
| TypeScript | 5.x | Type safety |
### Key Commands
```bash ```bash
# Run all tests # Run all tests
pnpm test pnpm test
# Watch mode # Watch mode
pnpm test --watch pnpm test:watch
# Run specific file # Run specific file
pnpm test path/to/file.spec.tsx pnpm test path/to/file.spec.tsx
# Generate coverage report # Generate coverage report
pnpm test --coverage pnpm test:coverage
# Analyze component complexity # Analyze component complexity
pnpm analyze-component <path> pnpm analyze-component <path>
@ -192,7 +200,7 @@ When assigned to test a directory/path, test **ALL content** within that path:
-**Import real project components** directly (including base components and siblings) -**Import real project components** directly (including base components and siblings)
-**Only mock**: API services (`@/service/*`), `next/navigation`, complex context providers -**Only mock**: API services (`@/service/*`), `next/navigation`, complex context providers
-**DO NOT mock** base components (`@/app/components/base/*`) or dify-ui primitives (`@langgenius/dify-ui/*`) -**DO NOT mock** base components (`@/app/components/base/*`)
-**DO NOT mock** sibling/child components in the same directory -**DO NOT mock** sibling/child components in the same directory
> See [Test Structure Template](#test-structure-template) for correct import/mock patterns. > See [Test Structure Template](#test-structure-template) for correct import/mock patterns.
@ -220,10 +228,7 @@ Every test should clearly separate:
### 2. Black-Box Testing ### 2. Black-Box Testing
- Test observable behavior, not implementation details - Test observable behavior, not implementation details
- Use semantic queries (`getByRole` with accessible `name`, `getByLabelText`, `getByPlaceholderText`, `getByText`, and scoped `within(...)`) - Use semantic queries (getByRole, getByLabelText)
- Treat `getByTestId` as a last resort. If a control cannot be found by role/name, label, landmark, or dialog scope, fix the component accessibility first instead of adding or relying on `data-testid`.
- Remove production `data-testid` attributes when semantic selectors can cover the behavior. Keep them only for non-visual mocked boundaries, editor/browser shims such as Monaco, canvas/chart output, or third-party widgets with no accessible DOM in the test environment.
- Do not assert decorative icons by test id. Assert the named control that contains them, or mark decorative icons `aria-hidden`.
- Avoid testing internal state directly - Avoid testing internal state directly
- **Prefer pattern matching over hardcoded strings** in assertions: - **Prefer pattern matching over hardcoded strings** in assertions:
@ -320,12 +325,12 @@ For more detailed information, refer to:
### Reference Examples in Codebase ### Reference Examples in Codebase
- `web/utils/classnames.spec.ts` - Utility function tests - `web/utils/classnames.spec.ts` - Utility function tests
- `web/app/components/base/radio/__tests__/index.spec.tsx` - Component tests - `web/app/components/base/button/index.spec.tsx` - Component tests
- `web/__mocks__/provider-context.ts` - Mock factory example - `web/__mocks__/provider-context.ts` - Mock factory example
### Project Configuration ### Project Configuration
- `web/vite.config.ts` - Vite/Vitest configuration - `web/vitest.config.ts` - Vitest configuration
- `web/vitest.setup.ts` - Test environment setup - `web/vitest.setup.ts` - Test environment setup
- `web/scripts/analyze-component.js` - Component analysis tool - `web/scripts/analyze-component.js` - Component analysis tool
- Modules are not mocked automatically. Global mocks live in `web/vitest.setup.ts` (for example `react-i18next`, `next/image`); mock other modules like `ky` or `mime` locally in test files. - Modules are not mocked automatically. Global mocks live in `web/vitest.setup.ts` (for example `react-i18next`, `next/image`); mock other modules like `ky` or `mime` locally in test files.

View File

@ -36,7 +36,7 @@ Use this checklist when generating or reviewing tests for Dify frontend componen
### Integration vs Mocking ### Integration vs Mocking
- [ ] **DO NOT mock base components or dify-ui primitives** (base `Loading`, `Input`, `Badge`; dify-ui `Button`, `Tooltip`, `Dialog`, etc.) - [ ] **DO NOT mock base components** (`Loading`, `Button`, `Tooltip`, etc.)
- [ ] Import real project components instead of mocking - [ ] Import real project components instead of mocking
- [ ] Only mock: API calls, complex context providers, third-party libs with side effects - [ ] Only mock: API calls, complex context providers, third-party libs with side effects
- [ ] Prefer integration testing when using single spec file - [ ] Prefer integration testing when using single spec file
@ -73,7 +73,7 @@ Use this checklist when generating or reviewing tests for Dify frontend componen
### Mocks ### Mocks
- [ ] **DO NOT mock base components or dify-ui primitives** (`@/app/components/base/*` or `@langgenius/dify-ui/*`) - [ ] **DO NOT mock base components** (`@/app/components/base/*`)
- [ ] `vi.clearAllMocks()` in `beforeEach` (not `afterEach`) - [ ] `vi.clearAllMocks()` in `beforeEach` (not `afterEach`)
- [ ] Shared mock state reset in `beforeEach` - [ ] Shared mock state reset in `beforeEach`
- [ ] i18n uses global mock (auto-loaded in `web/vitest.setup.ts`); only override locally for custom translations - [ ] i18n uses global mock (auto-loaded in `web/vitest.setup.ts`); only override locally for custom translations
@ -127,7 +127,7 @@ For the current file being tested:
- [ ] Run full directory test: `pnpm test path/to/directory/` - [ ] Run full directory test: `pnpm test path/to/directory/`
- [ ] Check coverage report: `pnpm test:coverage` - [ ] Check coverage report: `pnpm test:coverage`
- [ ] Run `pnpm lint:fix` on all test files - [ ] Run `pnpm lint:fix` on all test files
- [ ] Run `pnpm type-check` - [ ] Run `pnpm type-check:tsgo`
## Common Issues to Watch ## Common Issues to Watch

View File

@ -2,27 +2,29 @@
## ⚠️ Important: What NOT to Mock ## ⚠️ Important: What NOT to Mock
### DO NOT Mock Base Components or dify-ui Primitives ### DO NOT Mock Base Components
**Never mock components from `@/app/components/base/` or from `@langgenius/dify-ui/*`** such as: **Never mock components from `@/app/components/base/`** such as:
- Legacy base (`@/app/components/base/*`): `Loading`, `Spinner`, `Input`, `Badge`, `Tag` - `Loading`, `Spinner`
- dify-ui primitives (`@langgenius/dify-ui/*`): `Button`, `Tooltip`, `Dialog`, `Popover`, `DropdownMenu`, `ContextMenu`, `Select`, `AlertDialog`, `Toast` - `Button`, `Input`, `Select`
- `Tooltip`, `Modal`, `Dropdown`
- `Icon`, `Badge`, `Tag`
**Why?** **Why?**
- These components have their own dedicated tests - Base components will have their own dedicated tests
- Mocking them creates false positives (tests pass but real integration fails) - Mocking them creates false positives (tests pass but real integration fails)
- Using real components tests actual integration behavior - Using real components tests actual integration behavior
```typescript ```typescript
// ❌ WRONG: Don't mock base components or dify-ui primitives // ❌ WRONG: Don't mock base components
vi.mock('@/app/components/base/loading', () => () => <div>Loading</div>) vi.mock('@/app/components/base/loading', () => () => <div>Loading</div>)
vi.mock('@langgenius/dify-ui/button', () => ({ Button: ({ children }: any) => <button>{children}</button> })) vi.mock('@/app/components/base/button', () => ({ children }: any) => <button>{children}</button>)
// ✅ CORRECT: Import and use the real components // ✅ CORRECT: Import and use real base components
import Loading from '@/app/components/base/loading' import Loading from '@/app/components/base/loading'
import { Button } from '@langgenius/dify-ui/button' import Button from '@/app/components/base/button'
// They will render normally in tests // They will render normally in tests
``` ```
@ -56,7 +58,7 @@ See [Zustand Store Testing](#zustand-store-testing) section for full details.
| Location | Purpose | | Location | Purpose |
|----------|---------| |----------|---------|
| `web/vitest.setup.ts` | Global mocks shared by all tests (`react-i18next`, `zustand`, clipboard, FloatingPortal, Monaco, localStorage`) | | `web/vitest.setup.ts` | Global mocks shared by all tests (`react-i18next`, `next/image`, `zustand`) |
| `web/__mocks__/zustand.ts` | Zustand mock implementation (auto-resets stores after each test) | | `web/__mocks__/zustand.ts` | Zustand mock implementation (auto-resets stores after each test) |
| `web/__mocks__/` | Reusable mock factories shared across multiple test files | | `web/__mocks__/` | Reusable mock factories shared across multiple test files |
| Test file | Test-specific mocks, inline with `vi.mock()` | | Test file | Test-specific mocks, inline with `vi.mock()` |
@ -216,21 +218,28 @@ describe('Component', () => {
}) })
``` ```
### 5. HTTP and `fetch` Mocking ### 5. HTTP Mocking with Nock
```typescript ```typescript
import nock from 'nock'
const GITHUB_HOST = 'https://api.github.com'
const GITHUB_PATH = '/repos/owner/repo'
const mockGithubApi = (status: number, body: Record<string, unknown>, delayMs = 0) => {
return nock(GITHUB_HOST)
.get(GITHUB_PATH)
.delay(delayMs)
.reply(status, body)
}
describe('GithubComponent', () => { describe('GithubComponent', () => {
beforeEach(() => { afterEach(() => {
vi.clearAllMocks() nock.cleanAll()
}) })
it('should display repo info', async () => { it('should display repo info', async () => {
vi.mocked(globalThis.fetch).mockResolvedValueOnce( mockGithubApi(200, { name: 'dify', stars: 1000 })
new Response(JSON.stringify({ name: 'dify', stars: 1000 }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
}),
)
render(<GithubComponent />) render(<GithubComponent />)
@ -240,12 +249,7 @@ describe('GithubComponent', () => {
}) })
it('should handle API error', async () => { it('should handle API error', async () => {
vi.mocked(globalThis.fetch).mockResolvedValueOnce( mockGithubApi(500, { message: 'Server error' })
new Response(JSON.stringify({ message: 'Server error' }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
}),
)
render(<GithubComponent />) render(<GithubComponent />)
@ -256,8 +260,6 @@ describe('GithubComponent', () => {
}) })
``` ```
Prefer mocking `@/service/*` modules or spying on `global.fetch` / `ky` clients with deterministic responses. Do not introduce an HTTP interception dependency such as `nock` or MSW unless it is already declared in the workspace or adding it is part of the task.
### 6. Context Providers ### 6. Context Providers
```typescript ```typescript
@ -317,7 +319,7 @@ const renderWithQueryClient = (ui: React.ReactElement) => {
### ✅ DO ### ✅ DO
1. **Use real base components and dify-ui primitives** - Import from `@/app/components/base/` or `@langgenius/dify-ui/*` directly 1. **Use real base components** - Import from `@/app/components/base/` directly
1. **Use real project components** - Prefer importing over mocking 1. **Use real project components** - Prefer importing over mocking
1. **Use real Zustand stores** - Set test state via `store.setState()` 1. **Use real Zustand stores** - Set test state via `store.setState()`
1. **Reset mocks in `beforeEach`**, not `afterEach` 1. **Reset mocks in `beforeEach`**, not `afterEach`
@ -328,11 +330,11 @@ const renderWithQueryClient = (ui: React.ReactElement) => {
### ❌ DON'T ### ❌ DON'T
1. **Don't mock base components or dify-ui primitives** (`Loading`, `Input`, `Button`, `Tooltip`, `Dialog`, etc.) 1. **Don't mock base components** (`Loading`, `Button`, `Tooltip`, etc.)
1. **Don't mock Zustand store modules** - Use real stores with `setState()` 1. **Don't mock Zustand store modules** - Use real stores with `setState()`
1. Don't mock components you can import directly 1. Don't mock components you can import directly
1. Don't create overly simplified mocks that miss conditional logic 1. Don't create overly simplified mocks that miss conditional logic
1. Don't leave HTTP mocks or service mock state leaking between tests 1. Don't forget to clean up nock after each test
1. Don't use `any` types in mocks without necessity 1. Don't use `any` types in mocks without necessity
### Mock Decision Tree ### Mock Decision Tree
@ -340,7 +342,7 @@ const renderWithQueryClient = (ui: React.ReactElement) => {
``` ```
Need to use a component in test? Need to use a component in test?
├─ Is it from @/app/components/base/* or @langgenius/dify-ui/*? ├─ Is it from @/app/components/base/*?
│ └─ YES → Import real component, DO NOT mock │ └─ YES → Import real component, DO NOT mock
├─ Is it a project component? ├─ Is it a project component?

View File

@ -227,12 +227,12 @@ Failing tests compound:
**Fix failures immediately before proceeding.** **Fix failures immediately before proceeding.**
## Integration with Codex's Todo Feature ## Integration with Claude's Todo Feature
When using Codex for multi-file testing: When using Claude for multi-file testing:
1. **Create a todo list** before starting 1. **Ask Claude to create a todo list** before starting
1. **Process one file at a time** 1. **Request one file at a time** or ensure Claude processes incrementally
1. **Verify each test passes** before asking for the next 1. **Verify each test passes** before asking for the next
1. **Mark todos complete** as you progress 1. **Mark todos complete** as you progress

View File

@ -1,71 +0,0 @@
---
name: how-to-write-component
description: React/TypeScript component style guide. Use when writing, refactoring, or reviewing React components, especially around props typing, state boundaries, shared local state with Jotai atoms, API types, query/mutation contracts, navigation, memoization, wrappers, and empty-state handling.
---
# How To Write A Component
Use this as the decision guide for React/TypeScript component structure. Existing code is reference material, not automatic precedent; when it conflicts with these rules, adapt the approach instead of reproducing the violation.
## Core Defaults
- Search before adding UI, hooks, helpers, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
- Group code by feature workflow, route, or ownership area: components, hooks, local types, query helpers, atoms, constants, and small utilities should live near the code that changes with them.
- Promote code to shared only when multiple verticals need the same stable primitive. Otherwise keep it local and compose shared primitives inside the owning feature.
- Use Tailwind CSS v4.1+ rules via the `tailwind-css-rules` skill. Prefer v4 utilities, `gap`, `text-size/line-height`, `min-h-dvh`, and avoid deprecated utilities and `@apply`.
## Ownership
- Put local state, queries, mutations, handlers, and derived UI data in the lowest component that uses them. Extract a purpose-built owner component only when the logic has no natural home.
- Repeated TanStack query calls in sibling components are acceptable when each component independently consumes the data. Do not hoist a query only because it is duplicated; TanStack Query handles deduplication and cache sharing.
- Hoist state, queries, or callbacks to a parent only when the parent consumes the data, coordinates shared loading/error/empty UI, needs one consistent snapshot, or owns a workflow spanning children.
- Avoid prop drilling. One pass-through layer is acceptable; repeated forwarding means ownership should move down or into feature-scoped Jotai UI state. Keep server/cache state in query and API data flow.
- Keep callbacks in a parent only for workflow coordination such as form submission, shared selection, batch behavior, or navigation. Otherwise let the child or row own its action.
- Prefer uncontrolled DOM state and CSS variables before adding controlled props.
## Components, Props, And Types
- Type component signatures directly; do not use `FC` or `React.FC`.
- Prefer `function` for top-level components and module helpers. Use arrow functions for local callbacks, handlers, and lambda-style APIs.
- Prefer named exports. Use default exports only where the framework requires them, such as Next.js route files.
- Type simple one-off props inline. Use a named `Props` type only when reused, exported, complex, or clearer.
- Use API-generated or API-returned types at component boundaries. Keep small UI conversion helpers beside the component that needs them.
- Name values by their domain role and backend API contract, and keep that name stable across the call chain, especially IDs like `appInstanceId`. Normalize framework or route params at the boundary.
- Keep fallback and invariant checks at the lowest component that already handles that state; callers should pass raw values through instead of duplicating checks.
## Queries And Mutations
- Keep `web/contract/*` as the single source of truth for API shape; follow existing domain/router patterns and the `{ params, query?, body? }` input shape.
- Consume queries directly with `useQuery(consoleQuery.xxx.queryOptions(...))` or `useQuery(marketplaceQuery.xxx.queryOptions(...))`.
- Avoid pass-through hooks and thin `web/service/use-*` wrappers that only rename `queryOptions()` or `mutationOptions()`. Extract a small `queryOptions` helper only when repeated call-site options justify it.
- Keep feature hooks for real orchestration, workflow state, or shared domain behavior.
- For missing required query input, use `input: skipToken`; use `enabled` only for extra business gating after the input is valid.
- Consume mutations directly with `useMutation(consoleQuery.xxx.mutationOptions(...))` or `useMutation(marketplaceQuery.xxx.mutationOptions(...))`; use oRPC clients as `mutationFn` only for custom flows.
- Put shared cache behavior in `createTanstackQueryUtils(...experimental_defaults...)`; components may add UI feedback callbacks, but should not own shared invalidation rules.
- Do not use deprecated `useInvalid` or `useReset`.
- Prefer `mutate(...)`; use `mutateAsync(...)` only when Promise semantics are required, and wrap awaited calls in `try/catch`.
## Component Boundaries
- Use the first level below a page or tab to organize independent page sections when it adds real structure. This layer is layout/semantic first, not automatically the data owner.
- Split deeper components by the data and state each layer actually needs. Each component should access only necessary data, and ownership should stay at the lowest consumer.
- Keep cohesive forms, menu bodies, and one-off helpers local unless they need their own state, reuse, or semantic boundary.
- Separate hidden secondary surfaces from the trigger's main flow. For dialogs, dropdowns, popovers, and similar branches, extract a small local component that owns the trigger, open state, and hidden content when it would obscure the parent flow.
- Preserve composability by separating behavior ownership from layout ownership. A dropdown action may own its trigger, open state, and menu content; the caller owns placement such as slots, offsets, and alignment.
- Avoid unnecessary DOM hierarchy. Do not add wrapper elements unless they provide layout, semantics, accessibility, state ownership, or integration with a library API; prefer fragments or styling an existing element when possible.
- Avoid shallow wrappers and prop renaming unless the wrapper adds validation, orchestration, error handling, state ownership, or a real semantic boundary.
## You Might Not Need An Effect
- Use Effects only to synchronize with external systems such as browser APIs, non-React widgets, subscriptions, timers, analytics that must run because the component was shown, or imperative DOM integration.
- Do not use Effects to transform props or state for rendering. Calculate derived values during render, and use `useMemo` only when the calculation is actually expensive.
- Do not use Effects to handle user actions. Put action-specific logic in the event handler where the cause is known.
- Do not use Effects to copy one state value into another state value representing the same concept. Pick one source of truth and derive the rest during render.
- Do not reset or adjust state from props with an Effect. Prefer a `key` reset, storing a stable ID and deriving the selected object, or guarded same-component render-time adjustment when truly necessary.
- Prefer framework data APIs or TanStack Query for data fetching instead of writing request Effects in components.
- If an Effect still seems necessary, first name the external system it synchronizes with. If there is no external system, remove the Effect and restructure the state or event flow.
## Navigation And Performance
- Prefer `Link` for normal navigation. Use router APIs only for command-flow side effects such as mutation success, guarded redirects, or form submission.
- Avoid `memo`, `useMemo`, and `useCallback` unless there is a clear performance reason.

View File

@ -1,367 +0,0 @@
---
name: tailwind-css-rules
description: Tailwind CSS v4.1+ rules and best practices. Use when writing, reviewing, refactoring, or upgrading Tailwind CSS classes and styles, especially v4 utility migrations, layout spacing, typography, responsive variants, dark mode, gradients, CSS variables, and component styling.
---
# Tailwind CSS Rules and Best Practices
## Core Principles
- **Always use Tailwind CSS v4.1+** - Ensure the codebase is using the latest version
- **Do not use deprecated or removed utilities** - ALWAYS use the replacement
- **Never use `@apply`** - Use CSS variables, the `--spacing()` function, or framework components instead
- **Check for redundant classes** - Remove any classes that aren't necessary
- **Group elements logically** to simplify responsive tweaks later
## Upgrading to Tailwind CSS v4
### Before Upgrading
- **Always read the upgrade documentation first** - Read https://tailwindcss.com/docs/upgrade-guide and https://tailwindcss.com/blog/tailwindcss-v4 before starting an upgrade.
- Ensure the git repository is in a clean state before starting
### Upgrade Process
1. Run the upgrade command: `npx @tailwindcss/upgrade@latest` for both major and minor updates
2. The tool will convert JavaScript config files to the new CSS format
3. Review all changes extensively to clean up any false positives
4. Test thoroughly across your application
## Breaking Changes Reference
### Removed Utilities (NEVER use these in v4)
| ❌ Deprecated | ✅ Replacement |
| ----------------------- | ------------------------------------------------- |
| `bg-opacity-*` | Use opacity modifiers like `bg-black/50` |
| `text-opacity-*` | Use opacity modifiers like `text-black/50` |
| `border-opacity-*` | Use opacity modifiers like `border-black/50` |
| `divide-opacity-*` | Use opacity modifiers like `divide-black/50` |
| `ring-opacity-*` | Use opacity modifiers like `ring-black/50` |
| `placeholder-opacity-*` | Use opacity modifiers like `placeholder-black/50` |
| `flex-shrink-*` | `shrink-*` |
| `flex-grow-*` | `grow-*` |
| `overflow-ellipsis` | `text-ellipsis` |
| `decoration-slice` | `box-decoration-slice` |
| `decoration-clone` | `box-decoration-clone` |
### Renamed Utilities
Use the v4 name when migrating code that still carries Tailwind v3 semantics. Do not blanket-replace existing v4 classes: classes such as `rounded-sm`, `shadow-sm`, `ring-1`, and `ring-2` are valid in this codebase when they intentionally represent the current design scale.
| ❌ v3 pattern | ✅ v4 pattern |
| ------------------- | -------------------------------------------------- |
| `bg-gradient-*` | `bg-linear-*` |
| old shadow scale | verify against the current Tailwind/design scale |
| old blur scale | verify against the current Tailwind/design scale |
| old radius scale | use the Dify radius token mapping when applicable |
| `outline-none` | `outline-hidden` |
| bare `ring` utility | use an explicit ring width such as `ring-1`/`ring-2`/`ring-3` |
For Figma radius tokens, follow `packages/dify-ui/AGENTS.md`. For example, `--radius/xs` maps to `rounded-sm`; do not rewrite it to `rounded-xs`.
## Layout and Spacing Rules
### Flexbox and Grid Spacing
#### Always use gap utilities for internal spacing
Gap provides consistent spacing without edge cases (no extra space on last items). It's cleaner and more maintainable than margins on children.
```html
<!-- ❌ Don't do this -->
<div class="flex">
<div class="mr-4">Item 1</div>
<div class="mr-4">Item 2</div>
<div>Item 3</div>
<!-- No margin on last -->
</div>
<!-- ✅ Do this instead -->
<div class="flex gap-4">
<div>Item 1</div>
<div>Item 2</div>
<div>Item 3</div>
</div>
```
#### Gap vs Space utilities
- **Never use `space-x-*` or `space-y-*` in flex/grid layouts** - always use gap
- Space utilities add margins to children and have issues with wrapped items
- Gap works correctly with flex-wrap and all flex directions
```html
<!-- ❌ Avoid space utilities in flex containers -->
<div class="flex flex-wrap space-x-4">
<!-- Space utilities break with wrapped items -->
</div>
<!-- ✅ Use gap for consistent spacing -->
<div class="flex flex-wrap gap-4">
<!-- Gap works perfectly with wrapping -->
</div>
```
### General Spacing Guidelines
- **Prefer top and left margins** over bottom and right margins (unless conditionally rendered)
- **Use padding on parent containers** instead of bottom margins on the last child
- **Always use `min-h-dvh` instead of `min-h-screen`** - `min-h-screen` is buggy on mobile Safari
- **Prefer `size-*` utilities** over separate `w-*` and `h-*` when setting equal dimensions
- For max-widths, prefer the container scale (e.g., `max-w-2xs` over `max-w-72`)
## Typography Rules
### Line Heights
- **Never use `leading-*` classes** - Always use line height modifiers with text size
- **Always use fixed line heights from the spacing scale** - Don't use named values
```html
<!-- ❌ Don't do this -->
<p class="text-base leading-7">Text with separate line height</p>
<p class="text-lg leading-relaxed">Text with named line height</p>
<!-- ✅ Do this instead -->
<p class="text-base/7">Text with line height modifier</p>
<p class="text-lg/8">Text with specific line height</p>
```
### Font Size Reference
Be precise with font sizes - know the actual pixel values:
- `text-xs` = 12px
- `text-sm` = 14px
- `text-base` = 16px
- `text-lg` = 18px
- `text-xl` = 20px
## Color and Opacity
### Opacity Modifiers
**Never use `bg-opacity-*`, `text-opacity-*`, etc.** - use the opacity modifier syntax:
```html
<!-- ❌ Don't do this -->
<div class="bg-red-500 bg-opacity-60">Old opacity syntax</div>
<!-- ✅ Do this instead -->
<div class="bg-red-500/60">Modern opacity syntax</div>
```
## Responsive Design
### Breakpoint Optimization
- **Check for redundant classes across breakpoints**
- **Only add breakpoint variants when values change**
```html
<!-- ❌ Redundant breakpoint classes -->
<div class="px-4 md:px-4 lg:px-4">
<!-- md:px-4 and lg:px-4 are redundant -->
</div>
<!-- ✅ Efficient breakpoint usage -->
<div class="px-4 lg:px-8">
<!-- Only specify when value changes -->
</div>
```
## Dark Mode
### Dark Mode Best Practices
- Use the plain `dark:` variant pattern
- Put light mode styles first, then dark mode styles
- Ensure `dark:` variant comes before other variants
```html
<!-- ✅ Correct dark mode pattern -->
<div class="bg-white text-black dark:bg-black dark:text-white">
<button class="hover:bg-gray-100 dark:hover:bg-gray-800">Click me</button>
</div>
```
## Gradient Utilities
- **ALWAYS Use `bg-linear-*` instead of `bg-gradient-*` utilities** - The gradient utilities were renamed in v4
- Use the new `bg-radial` or `bg-radial-[<position>]` to create radial gradients
- Use the new `bg-conic` or `bg-conic-*` to create conic gradients
```html
<!-- ✅ Use the new gradient utilities -->
<div class="h-14 bg-linear-to-br from-violet-500 to-fuchsia-500"></div>
<div
class="size-18 bg-radial-[at_50%_75%] from-sky-200 via-blue-400 to-indigo-900 to-90%"
></div>
<div
class="size-24 bg-conic-180 from-indigo-600 via-indigo-50 to-indigo-600"
></div>
<!-- ❌ Do not use bg-gradient-* utilities -->
<div class="h-14 bg-gradient-to-br from-violet-500 to-fuchsia-500"></div>
```
## Working with CSS Variables
### Accessing Theme Values
Tailwind CSS v4 exposes all theme values as CSS variables:
```css
/* Access colors, and other theme values */
.custom-element {
background: var(--color-red-500);
border-radius: var(--radius-lg);
}
```
### The `--spacing()` Function
Use the dedicated `--spacing()` function for spacing calculations:
```css
.custom-class {
margin-top: calc(100vh - --spacing(16));
}
```
### Extending theme values
Use CSS to extend theme values:
```css
@import "tailwindcss";
@theme {
--color-mint-500: oklch(0.72 0.11 178);
}
```
```html
<div class="bg-mint-500">
<!-- ... -->
</div>
```
## New v4 Features
### Container Queries
Use the `@container` class and size variants:
```html
<article class="@container">
<div class="flex flex-col @md:flex-row @lg:gap-8">
<img class="w-full @md:w-48" />
<div class="mt-4 @md:mt-0">
<!-- Content adapts to container size -->
</div>
</div>
</article>
```
### Container Query Units
Use container-based units like `cqw` for responsive sizing:
```html
<div class="@container">
<h1 class="text-[50cqw]">Responsive to container width</h1>
</div>
```
### Text Shadows (v4.1)
Use text-shadow-\* utilities from text-shadow-2xs to text-shadow-lg:
```html
<!-- ✅ Text shadow examples -->
<h1 class="text-shadow-lg">Large shadow</h1>
<p class="text-shadow-sm/50">Small shadow with opacity</p>
```
### Masking (v4.1)
Use the new composable mask utilities for image and gradient masks:
```html
<!-- ✅ Linear gradient masks on specific sides -->
<div class="mask-t-from-50%">Top fade</div>
<div class="mask-b-from-20% mask-b-to-80%">Bottom gradient</div>
<div class="mask-linear-from-white mask-linear-to-black/60">
Fade from white to black
</div>
<!-- ✅ Radial gradient masks -->
<div class="mask-radial-[100%_100%] mask-radial-from-75% mask-radial-at-left">
Radial mask
</div>
```
## Component Patterns
### Avoiding Utility Inheritance
Don't add utilities to parents that you override in children:
```html
<!-- ❌ Avoid this pattern -->
<div class="text-center">
<h1>Centered Heading</h1>
<div class="text-left">Left-aligned content</div>
</div>
<!-- ✅ Better approach -->
<div>
<h1 class="text-center">Centered Heading</h1>
<div>Left-aligned content</div>
</div>
```
### Component Extraction
- Extract repeated patterns into framework components, not CSS classes
- Keep utility classes in templates/JSX
- Use data attributes for complex state-based styling
## CSS Best Practices
### Nesting Guidelines
- Use nesting when styling both parent and children
- Avoid empty parent selectors
```css
/* ✅ Good nesting - parent has styles */
.card {
padding: --spacing(4);
> .card-title {
font-weight: bold;
}
}
/* ❌ Avoid empty parents */
ul {
> li {
/* Parent has no styles */
}
}
```
## Common Pitfalls to Avoid
1. **Using old opacity utilities** - Always use `/opacity` syntax like `bg-red-500/60`
2. **Redundant breakpoint classes** - Only specify changes
3. **Space utilities in flex/grid** - Always use gap
4. **Leading utilities** - Use line-height modifiers like `text-sm/6`
5. **Arbitrary values** - Use the design scale
6. **@apply directive** - Use components or CSS variables
7. **min-h-screen on mobile** - Use min-h-dvh
8. **Separate width/height** - Use size utilities when equal
9. **Arbitrary values** - Always use Tailwind's predefined scale whenever possible (e.g., use `ml-4` over `ml-[16px]`)

View File

@ -1 +0,0 @@
../../.agents/skills/e2e-cucumber-playwright

View File

@ -7,7 +7,7 @@ cd web && pnpm install
pipx install uv pipx install uv
echo "alias start-api=\"cd $WORKSPACE_ROOT/api && uv run python -m flask run --host 0.0.0.0 --port=5001 --debug\"" >> ~/.bashrc echo "alias start-api=\"cd $WORKSPACE_ROOT/api && uv run python -m flask run --host 0.0.0.0 --port=5001 --debug\"" >> ~/.bashrc
echo "alias start-worker=\"cd $WORKSPACE_ROOT/api && uv run python -m celery -A app.celery worker -P threads -c 1 --loglevel INFO -Q dataset,dataset_summary,priority_dataset,priority_pipeline,pipeline,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation,workflow,schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_publisher,trigger_refresh_executor,retention\"" >> ~/.bashrc echo "alias start-worker=\"cd $WORKSPACE_ROOT/api && uv run python -m celery -A app.celery worker -P threads -c 1 --loglevel INFO -Q dataset,dataset_summary,priority_dataset,priority_pipeline,pipeline,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation,workflow,schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executor,retention\"" >> ~/.bashrc
echo "alias start-web=\"cd $WORKSPACE_ROOT/web && pnpm dev:inspect\"" >> ~/.bashrc echo "alias start-web=\"cd $WORKSPACE_ROOT/web && pnpm dev:inspect\"" >> ~/.bashrc
echo "alias start-web-prod=\"cd $WORKSPACE_ROOT/web && pnpm build && pnpm start\"" >> ~/.bashrc echo "alias start-web-prod=\"cd $WORKSPACE_ROOT/web && pnpm build && pnpm start\"" >> ~/.bashrc
echo "alias start-containers=\"cd $WORKSPACE_ROOT/docker && docker-compose -f docker-compose.middleware.yaml -p dify --env-file middleware.env up -d\"" >> ~/.bashrc echo "alias start-containers=\"cd $WORKSPACE_ROOT/docker && docker-compose -f docker-compose.middleware.yaml -p dify --env-file middleware.env up -d\"" >> ~/.bashrc

3
.github/CODEOWNERS vendored
View File

@ -6,9 +6,6 @@
* @crazywoola @laipz8200 @Yeuoly * @crazywoola @laipz8200 @Yeuoly
# ESLint suppression file is maintained by autofix.ci pruning.
/eslint-suppressions.json
# CODEOWNERS file # CODEOWNERS file
/.github/CODEOWNERS @laipz8200 @crazywoola /.github/CODEOWNERS @laipz8200 @crazywoola

View File

@ -4,7 +4,7 @@ runs:
using: composite using: composite
steps: steps:
- name: Setup Vite+ - name: Setup Vite+
uses: voidzero-dev/setup-vp@4f5aa3e38c781f1b01e78fb9255527cee8a6efa6 # v1.8.0 uses: voidzero-dev/setup-vp@20553a7a7429c429a74894104a2835d7fed28a72 # v1.3.0
with: with:
node-version-file: .nvmrc node-version-file: .nvmrc
cache: true cache: true

1
.github/labeler.yml vendored
View File

@ -6,4 +6,5 @@ web:
- 'package.json' - 'package.json'
- 'pnpm-lock.yaml' - 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml' - 'pnpm-workspace.yaml'
- '.npmrc'
- '.nvmrc' - '.nvmrc'

19
.github/workflows/anti-slop.yml vendored Normal file
View File

@ -0,0 +1,19 @@
name: Anti-Slop PR Check
on:
pull_request_target:
types: [opened, edited, synchronize]
permissions:
pull-requests: write
contents: read
jobs:
anti-slop:
runs-on: ubuntu-latest
steps:
- uses: peakoss/anti-slop@85daca1880e9e1af197fc06ea03349daf08f4202 # v0.2.1
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
close-pr: false
failure-add-pr-labels: "needs-revision"

View File

@ -16,7 +16,7 @@ concurrency:
jobs: jobs:
api-unit: api-unit:
name: API Unit Tests name: API Unit Tests
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
env: env:
COVERAGE_FILE: coverage-unit COVERAGE_FILE: coverage-unit
defaults: defaults:
@ -35,7 +35,7 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: true enable-cache: true
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
@ -62,7 +62,7 @@ jobs:
api-integration: api-integration:
name: API Integration Tests name: API Integration Tests
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
env: env:
COVERAGE_FILE: coverage-integration COVERAGE_FILE: coverage-integration
STORAGE_TYPE: opendal STORAGE_TYPE: opendal
@ -84,7 +84,7 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: true enable-cache: true
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
@ -99,13 +99,13 @@ jobs:
- name: Set up dotenvs - name: Set up dotenvs
run: | run: |
cp docker/.env.example docker/.env cp docker/.env.example docker/.env
cp docker/envs/middleware.env.example docker/middleware.env cp docker/middleware.env.example docker/middleware.env
- name: Expose Service Ports - name: Expose Service Ports
run: sh .github/workflows/expose_service_ports.sh run: sh .github/workflows/expose_service_ports.sh
- name: Set up Sandbox - name: Set up Sandbox
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0 uses: hoverkraft-tech/compose-action@4894d2492015c1774ee5a13a95b1072093087ec3 # v2.5.0
with: with:
compose-file: | compose-file: |
docker/docker-compose.middleware.yaml docker/docker-compose.middleware.yaml
@ -137,7 +137,7 @@ jobs:
api-coverage: api-coverage:
name: API Coverage name: API Coverage
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
needs: needs:
- api-unit - api-unit
- api-integration - api-integration
@ -156,7 +156,7 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: true enable-cache: true
python-version: "3.12" python-version: "3.12"

View File

@ -13,7 +13,7 @@ permissions:
jobs: jobs:
autofix: autofix:
if: github.repository == 'langgenius/dify' if: github.repository == 'langgenius/dify'
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Complete merge group check - name: Complete merge group check
if: github.event_name == 'merge_group' if: github.event_name == 'merge_group'
@ -25,7 +25,7 @@ jobs:
- name: Check Docker Compose inputs - name: Check Docker Compose inputs
if: github.event_name != 'merge_group' if: github.event_name != 'merge_group'
id: docker-compose-changes id: docker-compose-changes
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6 uses: tj-actions/changed-files@22103cc46bda19c2b464ffe86db46df6922fd323 # v47.0.5
with: with:
files: | files: |
docker/generate_docker_compose docker/generate_docker_compose
@ -35,7 +35,7 @@ jobs:
- name: Check web inputs - name: Check web inputs
if: github.event_name != 'merge_group' if: github.event_name != 'merge_group'
id: web-changes id: web-changes
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6 uses: tj-actions/changed-files@22103cc46bda19c2b464ffe86db46df6922fd323 # v47.0.5
with: with:
files: | files: |
web/** web/**
@ -43,11 +43,12 @@ jobs:
package.json package.json
pnpm-lock.yaml pnpm-lock.yaml
pnpm-workspace.yaml pnpm-workspace.yaml
.npmrc
.nvmrc .nvmrc
- name: Check api inputs - name: Check api inputs
if: github.event_name != 'merge_group' if: github.event_name != 'merge_group'
id: api-changes id: api-changes
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6 uses: tj-actions/changed-files@22103cc46bda19c2b464ffe86db46df6922fd323 # v47.0.5
with: with:
files: | files: |
api/** api/**
@ -57,7 +58,7 @@ jobs:
python-version: "3.11" python-version: "3.11"
- if: github.event_name != 'merge_group' - if: github.event_name != 'merge_group'
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
- name: Generate Docker Compose - name: Generate Docker Compose
if: github.event_name != 'merge_group' && steps.docker-compose-changes.outputs.any_changed == 'true' if: github.event_name != 'merge_group' && steps.docker-compose-changes.outputs.any_changed == 'true'
@ -113,19 +114,14 @@ jobs:
find . -name "*.py.bak" -type f -delete find . -name "*.py.bak" -type f -delete
- name: Setup web environment - name: Setup web environment
if: github.event_name != 'merge_group' if: github.event_name != 'merge_group' && steps.web-changes.outputs.any_changed == 'true'
uses: ./.github/actions/setup-web uses: ./.github/actions/setup-web
- name: Generate API docs
if: github.event_name != 'merge_group' && steps.api-changes.outputs.any_changed == 'true'
run: |
cd api
uv run dev/generate_swagger_markdown_docs.py --swagger-dir openapi --markdown-dir openapi/markdown
- name: ESLint autofix - name: ESLint autofix
if: github.event_name != 'merge_group' && steps.web-changes.outputs.any_changed == 'true' if: github.event_name != 'merge_group' && steps.web-changes.outputs.any_changed == 'true'
run: | run: |
cd web
vp exec eslint --concurrency=2 --prune-suppressions --quiet || true vp exec eslint --concurrency=2 --prune-suppressions --quiet || true
- if: github.event_name != 'merge_group' - if: github.event_name != 'merge_group'
uses: autofix-ci/action@c5b2d67aa2274e7b5a18224e8171550871fc7e4a # v1.3.4 uses: autofix-ci/action@7a166d7532b277f34e16238930461bf77f9d7ed8 # v1.3.3

View File

@ -26,9 +26,6 @@ jobs:
build: build:
runs-on: ${{ matrix.runs_on }} runs-on: ${{ matrix.runs_on }}
if: github.repository == 'langgenius/dify' if: github.repository == 'langgenius/dify'
permissions:
contents: read
id-token: write
strategy: strategy:
matrix: matrix:
include: include:
@ -38,28 +35,28 @@ jobs:
build_context: "{{defaultContext}}:api" build_context: "{{defaultContext}}:api"
file: "Dockerfile" file: "Dockerfile"
platform: linux/amd64 platform: linux/amd64
runs_on: depot-ubuntu-24.04-4 runs_on: ubuntu-latest
- service_name: "build-api-arm64" - service_name: "build-api-arm64"
image_name_env: "DIFY_API_IMAGE_NAME" image_name_env: "DIFY_API_IMAGE_NAME"
artifact_context: "api" artifact_context: "api"
build_context: "{{defaultContext}}:api" build_context: "{{defaultContext}}:api"
file: "Dockerfile" file: "Dockerfile"
platform: linux/arm64 platform: linux/arm64
runs_on: depot-ubuntu-24.04-4 runs_on: ubuntu-24.04-arm
- service_name: "build-web-amd64" - service_name: "build-web-amd64"
image_name_env: "DIFY_WEB_IMAGE_NAME" image_name_env: "DIFY_WEB_IMAGE_NAME"
artifact_context: "web" artifact_context: "web"
build_context: "{{defaultContext}}" build_context: "{{defaultContext}}"
file: "web/Dockerfile" file: "web/Dockerfile"
platform: linux/amd64 platform: linux/amd64
runs_on: depot-ubuntu-24.04-4 runs_on: ubuntu-latest
- service_name: "build-web-arm64" - service_name: "build-web-arm64"
image_name_env: "DIFY_WEB_IMAGE_NAME" image_name_env: "DIFY_WEB_IMAGE_NAME"
artifact_context: "web" artifact_context: "web"
build_context: "{{defaultContext}}" build_context: "{{defaultContext}}"
file: "web/Dockerfile" file: "web/Dockerfile"
platform: linux/arm64 platform: linux/arm64
runs_on: depot-ubuntu-24.04-4 runs_on: ubuntu-24.04-arm
steps: steps:
- name: Prepare - name: Prepare
@ -73,8 +70,8 @@ jobs:
username: ${{ env.DOCKERHUB_USER }} username: ${{ env.DOCKERHUB_USER }}
password: ${{ env.DOCKERHUB_TOKEN }} password: ${{ env.DOCKERHUB_TOKEN }}
- name: Set up Depot CLI - name: Set up Docker Buildx
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.7.1 uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Extract metadata for Docker - name: Extract metadata for Docker
id: meta id: meta
@ -84,15 +81,16 @@ jobs:
- name: Build Docker image - name: Build Docker image
id: build id: build
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.17.0 uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with: with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: ${{ matrix.build_context }} context: ${{ matrix.build_context }}
file: ${{ matrix.file }} file: ${{ matrix.file }}
platforms: ${{ matrix.platform }} platforms: ${{ matrix.platform }}
build-args: COMMIT_SHA=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.revision'] }} build-args: COMMIT_SHA=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.revision'] }}
labels: ${{ steps.meta.outputs.labels }} labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env[matrix.image_name_env] }},push-by-digest=true,name-canonical=true,push=true outputs: type=image,name=${{ env[matrix.image_name_env] }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=gha,scope=${{ matrix.service_name }}
cache-to: type=gha,mode=max,scope=${{ matrix.service_name }}
- name: Export digest - name: Export digest
env: env:
@ -110,33 +108,9 @@ jobs:
if-no-files-found: error if-no-files-found: error
retention-days: 1 retention-days: 1
fork-build-validate:
if: github.repository != 'langgenius/dify'
runs-on: ubuntu-24.04
strategy:
matrix:
include:
- service_name: "validate-api-amd64"
build_context: "{{defaultContext}}:api"
file: "Dockerfile"
- service_name: "validate-web-amd64"
build_context: "{{defaultContext}}"
file: "web/Dockerfile"
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Validate Docker image
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
push: false
context: ${{ matrix.build_context }}
file: ${{ matrix.file }}
platforms: linux/amd64
create-manifest: create-manifest:
needs: build needs: build
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
if: github.repository == 'langgenius/dify' if: github.repository == 'langgenius/dify'
strategy: strategy:
matrix: matrix:

View File

@ -9,7 +9,7 @@ concurrency:
jobs: jobs:
db-migration-test-postgres: db-migration-test-postgres:
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Checkout code - name: Checkout code
@ -19,7 +19,7 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: true enable-cache: true
python-version: "3.12" python-version: "3.12"
@ -37,10 +37,10 @@ jobs:
- name: Prepare middleware env - name: Prepare middleware env
run: | run: |
cd docker cd docker
cp envs/middleware.env.example middleware.env cp middleware.env.example middleware.env
- name: Set up Middlewares - name: Set up Middlewares
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0 uses: hoverkraft-tech/compose-action@4894d2492015c1774ee5a13a95b1072093087ec3 # v2.5.0
with: with:
compose-file: | compose-file: |
docker/docker-compose.middleware.yaml docker/docker-compose.middleware.yaml
@ -59,7 +59,7 @@ jobs:
run: uv run --directory api flask upgrade-db run: uv run --directory api flask upgrade-db
db-migration-test-mysql: db-migration-test-mysql:
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Checkout code - name: Checkout code
@ -69,7 +69,7 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: true enable-cache: true
python-version: "3.12" python-version: "3.12"
@ -87,14 +87,14 @@ jobs:
- name: Prepare middleware env for MySQL - name: Prepare middleware env for MySQL
run: | run: |
cd docker cd docker
cp envs/middleware.env.example middleware.env cp middleware.env.example middleware.env
sed -i 's/DB_TYPE=postgresql/DB_TYPE=mysql/' middleware.env sed -i 's/DB_TYPE=postgresql/DB_TYPE=mysql/' middleware.env
sed -i 's/DB_HOST=db_postgres/DB_HOST=db_mysql/' middleware.env sed -i 's/DB_HOST=db_postgres/DB_HOST=db_mysql/' middleware.env
sed -i 's/DB_PORT=5432/DB_PORT=3306/' middleware.env sed -i 's/DB_PORT=5432/DB_PORT=3306/' middleware.env
sed -i 's/DB_USERNAME=postgres/DB_USERNAME=mysql/' middleware.env sed -i 's/DB_USERNAME=postgres/DB_USERNAME=mysql/' middleware.env
- name: Set up Middlewares - name: Set up Middlewares
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0 uses: hoverkraft-tech/compose-action@4894d2492015c1774ee5a13a95b1072093087ec3 # v2.5.0
with: with:
compose-file: | compose-file: |
docker/docker-compose.middleware.yaml docker/docker-compose.middleware.yaml
@ -110,28 +110,6 @@ jobs:
sed -i 's/DB_PORT=5432/DB_PORT=3306/' .env sed -i 's/DB_PORT=5432/DB_PORT=3306/' .env
sed -i 's/DB_USERNAME=postgres/DB_USERNAME=root/' .env sed -i 's/DB_USERNAME=postgres/DB_USERNAME=root/' .env
# hoverkraft-tech/compose-action@v2.6.0 only waits for `docker compose up -d`
# to return (container processes started); it does not wait on healthcheck
# status. mysql:8.0's first-time init takes 15-30s, so without an explicit
# wait the migration runs while InnoDB is still initialising and gets
# killed with "Lost connection during query". Poll a real SELECT until it
# succeeds.
- name: Wait for MySQL to accept queries
run: |
set +e
for i in $(seq 1 60); do
if docker run --rm --network host mysql:8.0 \
mysql -h 127.0.0.1 -P 3306 -uroot -pdifyai123456 \
-e 'SELECT 1' >/dev/null 2>&1; then
echo "MySQL ready after ${i}s"
exit 0
fi
sleep 1
done
echo "MySQL not ready after 60s; dumping container logs:"
docker compose -f docker/docker-compose.middleware.yaml --profile mysql logs --tail=200 db_mysql
exit 1
- name: Run DB Migration - name: Run DB Migration
env: env:
DEBUG: true DEBUG: true

View File

@ -13,7 +13,7 @@ on:
jobs: jobs:
deploy: deploy:
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
if: | if: |
github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'deploy/agent-dev' github.event.workflow_run.head_branch == 'deploy/agent-dev'

View File

@ -10,7 +10,7 @@ on:
jobs: jobs:
deploy: deploy:
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
if: | if: |
github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'deploy/dev' github.event.workflow_run.head_branch == 'deploy/dev'

View File

@ -13,7 +13,7 @@ on:
jobs: jobs:
deploy: deploy:
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
if: | if: |
github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'deploy/enterprise' github.event.workflow_run.head_branch == 'deploy/enterprise'

View File

@ -10,7 +10,7 @@ on:
jobs: jobs:
deploy: deploy:
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
if: | if: |
github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'build/feat/hitl' github.event.workflow_run.head_branch == 'build/feat/hitl'

View File

@ -6,7 +6,14 @@ on:
- "main" - "main"
paths: paths:
- api/Dockerfile - api/Dockerfile
- web/docker/**
- web/Dockerfile - web/Dockerfile
- packages/**
- package.json
- pnpm-lock.yaml
- pnpm-workspace.yaml
- .npmrc
- .nvmrc
concurrency: concurrency:
group: docker-build-${{ github.head_ref || github.run_id }} group: docker-build-${{ github.head_ref || github.run_id }}
@ -14,59 +21,28 @@ concurrency:
jobs: jobs:
build-docker: build-docker:
if: github.event.pull_request.head.repo.full_name == github.repository
runs-on: ${{ matrix.runs_on }} runs-on: ${{ matrix.runs_on }}
permissions:
contents: read
id-token: write
strategy: strategy:
matrix: matrix:
include: include:
- service_name: "api-amd64" - service_name: "api-amd64"
platform: linux/amd64 platform: linux/amd64
runs_on: depot-ubuntu-24.04-4 runs_on: ubuntu-latest
context: "{{defaultContext}}:api" context: "{{defaultContext}}:api"
file: "Dockerfile" file: "Dockerfile"
- service_name: "api-arm64" - service_name: "api-arm64"
platform: linux/arm64 platform: linux/arm64
runs_on: depot-ubuntu-24.04-4 runs_on: ubuntu-24.04-arm
context: "{{defaultContext}}:api" context: "{{defaultContext}}:api"
file: "Dockerfile" file: "Dockerfile"
- service_name: "web-amd64" - service_name: "web-amd64"
platform: linux/amd64 platform: linux/amd64
runs_on: depot-ubuntu-24.04-4 runs_on: ubuntu-latest
context: "{{defaultContext}}" context: "{{defaultContext}}"
file: "web/Dockerfile" file: "web/Dockerfile"
- service_name: "web-arm64" - service_name: "web-arm64"
platform: linux/arm64 platform: linux/arm64
runs_on: depot-ubuntu-24.04-4 runs_on: ubuntu-24.04-arm
context: "{{defaultContext}}"
file: "web/Dockerfile"
steps:
- name: Set up Depot CLI
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.7.1
- name: Build Docker Image
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.17.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
push: false
context: ${{ matrix.context }}
file: ${{ matrix.file }}
platforms: ${{ matrix.platform }}
build-docker-fork:
if: github.event.pull_request.head.repo.full_name != github.repository
runs-on: ubuntu-24.04
permissions:
contents: read
strategy:
matrix:
include:
- service_name: "api-amd64"
context: "{{defaultContext}}:api"
file: "Dockerfile"
- service_name: "web-amd64"
context: "{{defaultContext}}" context: "{{defaultContext}}"
file: "web/Dockerfile" file: "web/Dockerfile"
steps: steps:
@ -79,4 +55,6 @@ jobs:
push: false push: false
context: ${{ matrix.context }} context: ${{ matrix.context }}
file: ${{ matrix.file }} file: ${{ matrix.file }}
platforms: linux/amd64 platforms: ${{ matrix.platform }}
cache-from: type=gha
cache-to: type=gha,mode=max

View File

@ -7,8 +7,8 @@ jobs:
permissions: permissions:
contents: read contents: read
pull-requests: write pull-requests: write
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- uses: actions/labeler@f27b608878404679385c85cfa523b85ccb86e213 # v6.1.0 - uses: actions/labeler@634933edcd8ababfe52f92936142cc22ac488b1b # v6.0.1
with: with:
sync-labels: true sync-labels: true

View File

@ -23,7 +23,7 @@ concurrency:
jobs: jobs:
pre_job: pre_job:
name: Skip Duplicate Checks name: Skip Duplicate Checks
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
outputs: outputs:
should_skip: ${{ steps.skip_check.outputs.should_skip || 'false' }} should_skip: ${{ steps.skip_check.outputs.should_skip || 'false' }}
steps: steps:
@ -39,7 +39,7 @@ jobs:
name: Check Changed Files name: Check Changed Files
needs: pre_job needs: pre_job
if: needs.pre_job.outputs.should_skip != 'true' if: needs.pre_job.outputs.should_skip != 'true'
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
outputs: outputs:
api-changed: ${{ steps.changes.outputs.api }} api-changed: ${{ steps.changes.outputs.api }}
e2e-changed: ${{ steps.changes.outputs.e2e }} e2e-changed: ${{ steps.changes.outputs.e2e }}
@ -57,7 +57,7 @@ jobs:
- '.github/workflows/api-tests.yml' - '.github/workflows/api-tests.yml'
- '.github/workflows/expose_service_ports.sh' - '.github/workflows/expose_service_ports.sh'
- 'docker/.env.example' - 'docker/.env.example'
- 'docker/envs/middleware.env.example' - 'docker/middleware.env.example'
- 'docker/docker-compose.middleware.yaml' - 'docker/docker-compose.middleware.yaml'
- 'docker/docker-compose-template.yaml' - 'docker/docker-compose-template.yaml'
- 'docker/generate_docker_compose' - 'docker/generate_docker_compose'
@ -69,6 +69,7 @@ jobs:
- 'package.json' - 'package.json'
- 'pnpm-lock.yaml' - 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml' - 'pnpm-workspace.yaml'
- '.npmrc'
- '.nvmrc' - '.nvmrc'
- '.github/workflows/web-tests.yml' - '.github/workflows/web-tests.yml'
- '.github/actions/setup-web/**' - '.github/actions/setup-web/**'
@ -82,19 +83,19 @@ jobs:
- 'package.json' - 'package.json'
- 'pnpm-lock.yaml' - 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml' - 'pnpm-workspace.yaml'
- '.npmrc'
- '.nvmrc' - '.nvmrc'
- 'docker/docker-compose.middleware.yaml' - 'docker/docker-compose.middleware.yaml'
- 'docker/envs/middleware.env.example' - 'docker/middleware.env.example'
- '.github/workflows/web-e2e.yml' - '.github/workflows/web-e2e.yml'
- '.github/actions/setup-web/**' - '.github/actions/setup-web/**'
vdb: vdb:
- 'api/core/rag/datasource/**' - 'api/core/rag/datasource/**'
- 'api/tests/integration_tests/vdb/**' - 'api/tests/integration_tests/vdb/**'
- 'api/providers/vdb/*/tests/**'
- '.github/workflows/vdb-tests.yml' - '.github/workflows/vdb-tests.yml'
- '.github/workflows/expose_service_ports.sh' - '.github/workflows/expose_service_ports.sh'
- 'docker/.env.example' - 'docker/.env.example'
- 'docker/envs/middleware.env.example' - 'docker/middleware.env.example'
- 'docker/docker-compose.yaml' - 'docker/docker-compose.yaml'
- 'docker/docker-compose-template.yaml' - 'docker/docker-compose-template.yaml'
- 'docker/generate_docker_compose' - 'docker/generate_docker_compose'
@ -116,7 +117,7 @@ jobs:
- '.github/workflows/db-migration-test.yml' - '.github/workflows/db-migration-test.yml'
- '.github/workflows/expose_service_ports.sh' - '.github/workflows/expose_service_ports.sh'
- 'docker/.env.example' - 'docker/.env.example'
- 'docker/envs/middleware.env.example' - 'docker/middleware.env.example'
- 'docker/docker-compose.middleware.yaml' - 'docker/docker-compose.middleware.yaml'
- 'docker/docker-compose-template.yaml' - 'docker/docker-compose-template.yaml'
- 'docker/generate_docker_compose' - 'docker/generate_docker_compose'
@ -139,7 +140,7 @@ jobs:
- pre_job - pre_job
- check-changes - check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.api-changed != 'true' if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.api-changed != 'true'
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Report skipped API tests - name: Report skipped API tests
run: echo "No API-related changes detected; skipping API tests." run: echo "No API-related changes detected; skipping API tests."
@ -152,7 +153,7 @@ jobs:
- check-changes - check-changes
- api-tests-run - api-tests-run
- api-tests-skip - api-tests-skip
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Finalize API Tests status - name: Finalize API Tests status
env: env:
@ -199,7 +200,7 @@ jobs:
- pre_job - pre_job
- check-changes - check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.web-changed != 'true' if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.web-changed != 'true'
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Report skipped web tests - name: Report skipped web tests
run: echo "No web-related changes detected; skipping web tests." run: echo "No web-related changes detected; skipping web tests."
@ -212,7 +213,7 @@ jobs:
- check-changes - check-changes
- web-tests-run - web-tests-run
- web-tests-skip - web-tests-skip
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Finalize Web Tests status - name: Finalize Web Tests status
env: env:
@ -258,7 +259,7 @@ jobs:
- pre_job - pre_job
- check-changes - check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed != 'true' if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed != 'true'
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Report skipped web full-stack e2e - name: Report skipped web full-stack e2e
run: echo "No E2E-related changes detected; skipping web full-stack E2E." run: echo "No E2E-related changes detected; skipping web full-stack E2E."
@ -271,7 +272,7 @@ jobs:
- check-changes - check-changes
- web-e2e-run - web-e2e-run
- web-e2e-skip - web-e2e-skip
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Finalize Web Full-Stack E2E status - name: Finalize Web Full-Stack E2E status
env: env:
@ -323,7 +324,7 @@ jobs:
- pre_job - pre_job
- check-changes - check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.vdb-changed != 'true' if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.vdb-changed != 'true'
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Report skipped VDB tests - name: Report skipped VDB tests
run: echo "No VDB-related changes detected; skipping VDB tests." run: echo "No VDB-related changes detected; skipping VDB tests."
@ -336,7 +337,7 @@ jobs:
- check-changes - check-changes
- vdb-tests-run - vdb-tests-run
- vdb-tests-skip - vdb-tests-skip
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Finalize VDB Tests status - name: Finalize VDB Tests status
env: env:
@ -382,7 +383,7 @@ jobs:
- pre_job - pre_job
- check-changes - check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.migration-changed != 'true' if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.migration-changed != 'true'
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Report skipped DB migration tests - name: Report skipped DB migration tests
run: echo "No migration-related changes detected; skipping DB migration tests." run: echo "No migration-related changes detected; skipping DB migration tests."
@ -395,7 +396,7 @@ jobs:
- check-changes - check-changes
- db-migration-test-run - db-migration-test-run
- db-migration-test-skip - db-migration-test-skip
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Finalize DB Migration Test status - name: Finalize DB Migration Test status
env: env:

View File

@ -12,7 +12,7 @@ permissions: {}
jobs: jobs:
comment: comment:
name: Comment PR with pyrefly diff name: Comment PR with pyrefly diff
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
permissions: permissions:
actions: read actions: read
contents: read contents: read
@ -76,29 +76,13 @@ jobs:
diff += '\\n\\n... (truncated) ...'; diff += '\\n\\n... (truncated) ...';
} }
if (diff.trim()) { const body = diff.trim()
const body = '### Pyrefly Diff\n<details>\n<summary>base → PR</summary>\n\n```diff\n' + diff + '\n```\n</details>'; ? '### Pyrefly Diff\n<details>\n<summary>base → PR</summary>\n\n```diff\n' + diff + '\n```\n</details>'
const marker = '### Pyrefly Diff'; : '### Pyrefly Diff\nNo changes detected.';
const { data: comments } = await github.rest.issues.listComments({
issue_number: prNumber,
owner: context.repo.owner,
repo: context.repo.repo,
});
const existing = comments.find((comment) => comment.body.startsWith(marker));
if (existing) { await github.rest.issues.createComment({
await github.rest.issues.updateComment({ issue_number: prNumber,
comment_id: existing.id, owner: context.repo.owner,
owner: context.repo.owner, repo: context.repo.repo,
repo: context.repo.repo, body,
body, });
});
} else {
await github.rest.issues.createComment({
issue_number: prNumber,
owner: context.repo.owner,
repo: context.repo.repo,
body,
});
}
}

View File

@ -10,7 +10,7 @@ permissions:
jobs: jobs:
pyrefly-diff: pyrefly-diff:
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
permissions: permissions:
contents: read contents: read
issues: write issues: write
@ -22,7 +22,7 @@ jobs:
fetch-depth: 0 fetch-depth: 0
- name: Setup Python & UV - name: Setup Python & UV
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: true enable-cache: true
@ -103,26 +103,9 @@ jobs:
].join('\n') ].join('\n')
: '### Pyrefly Diff\nNo changes detected.'; : '### Pyrefly Diff\nNo changes detected.';
const marker = '### Pyrefly Diff'; await github.rest.issues.createComment({
const { data: comments } = await github.rest.issues.listComments({
issue_number: prNumber, issue_number: prNumber,
owner: context.repo.owner, owner: context.repo.owner,
repo: context.repo.repo, repo: context.repo.repo,
body,
}); });
const existing = comments.find((comment) => comment.body.startsWith(marker));
if (existing) {
await github.rest.issues.updateComment({
comment_id: existing.id,
owner: context.repo.owner,
repo: context.repo.repo,
body,
});
} else {
await github.rest.issues.createComment({
issue_number: prNumber,
owner: context.repo.owner,
repo: context.repo.repo,
body,
});
}

View File

@ -12,7 +12,7 @@ permissions: {}
jobs: jobs:
comment: comment:
name: Comment PR with type coverage name: Comment PR with type coverage
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
permissions: permissions:
actions: read actions: read
contents: read contents: read
@ -24,7 +24,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Python & UV - name: Setup Python & UV
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: true enable-cache: true
@ -62,7 +62,7 @@ jobs:
- name: Render coverage markdown from structured data - name: Render coverage markdown from structured data
id: render id: render
run: | run: |
comment_body="$(uv run --directory api python libs/pyrefly_type_coverage.py \ comment_body="$(uv run --directory api python api/libs/pyrefly_type_coverage.py \
--base base_report.json \ --base base_report.json \
< pr_report.json)" < pr_report.json)"

View File

@ -10,7 +10,7 @@ permissions:
jobs: jobs:
pyrefly-type-coverage: pyrefly-type-coverage:
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
permissions: permissions:
contents: read contents: read
issues: write issues: write
@ -22,7 +22,7 @@ jobs:
fetch-depth: 0 fetch-depth: 0
- name: Setup Python & UV - name: Setup Python & UV
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: true enable-cache: true

View File

@ -16,7 +16,7 @@ jobs:
name: Validate PR title name: Validate PR title
permissions: permissions:
pull-requests: read pull-requests: read
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Complete merge group check - name: Complete merge group check
if: github.event_name == 'merge_group' if: github.event_name == 'merge_group'

View File

@ -12,7 +12,7 @@ on:
jobs: jobs:
stale: stale:
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
permissions: permissions:
issues: write issues: write
pull-requests: write pull-requests: write

View File

@ -15,7 +15,7 @@ permissions:
jobs: jobs:
python-style: python-style:
name: Python Style name: Python Style
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Checkout code - name: Checkout code
@ -25,7 +25,7 @@ jobs:
- name: Check changed files - name: Check changed files
id: changed-files id: changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6 uses: tj-actions/changed-files@22103cc46bda19c2b464ffe86db46df6922fd323 # v47.0.5
with: with:
files: | files: |
api/** api/**
@ -33,7 +33,7 @@ jobs:
- name: Setup UV and Python - name: Setup UV and Python
if: steps.changed-files.outputs.any_changed == 'true' if: steps.changed-files.outputs.any_changed == 'true'
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: false enable-cache: false
python-version: "3.12" python-version: "3.12"
@ -57,7 +57,7 @@ jobs:
web-style: web-style:
name: Web Style name: Web Style
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
defaults: defaults:
run: run:
working-directory: ./web working-directory: ./web
@ -73,16 +73,15 @@ jobs:
- name: Check changed files - name: Check changed files
id: changed-files id: changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6 uses: tj-actions/changed-files@22103cc46bda19c2b464ffe86db46df6922fd323 # v47.0.5
with: with:
files: | files: |
web/** web/**
e2e/**
sdks/nodejs-client/**
packages/** packages/**
package.json package.json
pnpm-lock.yaml pnpm-lock.yaml
pnpm-workspace.yaml pnpm-workspace.yaml
.npmrc
.nvmrc .nvmrc
.github/workflows/style.yml .github/workflows/style.yml
.github/actions/setup-web/** .github/actions/setup-web/**
@ -94,28 +93,26 @@ jobs:
- name: Restore ESLint cache - name: Restore ESLint cache
if: steps.changed-files.outputs.any_changed == 'true' if: steps.changed-files.outputs.any_changed == 'true'
id: eslint-cache-restore id: eslint-cache-restore
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
with: with:
path: .eslintcache path: web/.eslintcache
key: ${{ runner.os }}-eslint-${{ hashFiles('pnpm-lock.yaml', 'eslint.config.mjs', 'web/eslint.config.mjs', 'web/eslint.constants.mjs', 'web/plugins/eslint/**') }}-${{ github.sha }} key: ${{ runner.os }}-web-eslint-${{ hashFiles('web/package.json', 'pnpm-lock.yaml', 'web/eslint.config.mjs', 'web/eslint.constants.mjs', 'web/plugins/eslint/**') }}-${{ github.sha }}
restore-keys: | restore-keys: |
${{ runner.os }}-eslint-${{ hashFiles('pnpm-lock.yaml', 'eslint.config.mjs', 'web/eslint.config.mjs', 'web/eslint.constants.mjs', 'web/plugins/eslint/**') }}- ${{ runner.os }}-web-eslint-${{ hashFiles('web/package.json', 'pnpm-lock.yaml', 'web/eslint.config.mjs', 'web/eslint.constants.mjs', 'web/plugins/eslint/**') }}-
- name: Web style check - name: Web style check
if: steps.changed-files.outputs.any_changed == 'true' if: steps.changed-files.outputs.any_changed == 'true'
working-directory: . working-directory: ./web
run: vp run lint:ci run: vp run lint:ci
- name: Web tsslint - name: Web tsslint
if: steps.changed-files.outputs.any_changed == 'true' if: steps.changed-files.outputs.any_changed == 'true'
working-directory: ./web working-directory: ./web
env:
NODE_OPTIONS: --max-old-space-size=4096
run: vp run lint:tss run: vp run lint:tss
- name: Web type check - name: Web type check
if: steps.changed-files.outputs.any_changed == 'true' if: steps.changed-files.outputs.any_changed == 'true'
working-directory: . working-directory: ./web
run: vp run type-check run: vp run type-check
- name: Web dead code check - name: Web dead code check
@ -125,14 +122,14 @@ jobs:
- name: Save ESLint cache - name: Save ESLint cache
if: steps.changed-files.outputs.any_changed == 'true' && success() && steps.eslint-cache-restore.outputs.cache-hit != 'true' if: steps.changed-files.outputs.any_changed == 'true' && success() && steps.eslint-cache-restore.outputs.cache-hit != 'true'
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
with: with:
path: .eslintcache path: web/.eslintcache
key: ${{ steps.eslint-cache-restore.outputs.cache-primary-key }} key: ${{ steps.eslint-cache-restore.outputs.cache-primary-key }}
superlinter: superlinter:
name: SuperLinter name: SuperLinter
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
steps: steps:
- name: Checkout code - name: Checkout code
@ -143,7 +140,7 @@ jobs:
- name: Check changed files - name: Check changed files
id: changed-files id: changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6 uses: tj-actions/changed-files@22103cc46bda19c2b464ffe86db46df6922fd323 # v47.0.5
with: with:
files: | files: |
**.sh **.sh

View File

@ -9,6 +9,7 @@ on:
- package.json - package.json
- pnpm-lock.yaml - pnpm-lock.yaml
- pnpm-workspace.yaml - pnpm-workspace.yaml
- .npmrc
concurrency: concurrency:
group: sdk-tests-${{ github.head_ref || github.run_id }} group: sdk-tests-${{ github.head_ref || github.run_id }}
@ -17,7 +18,7 @@ concurrency:
jobs: jobs:
build: build:
name: unit test for Node.js SDK name: unit test for Node.js SDK
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
defaults: defaults:
run: run:
@ -29,7 +30,7 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Use Node.js - name: Use Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with: with:
node-version: 22 node-version: 22
cache: '' cache: ''

View File

@ -35,7 +35,7 @@ concurrency:
jobs: jobs:
translate: translate:
if: github.repository == 'langgenius/dify' if: github.repository == 'langgenius/dify'
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
timeout-minutes: 120 timeout-minutes: 120
steps: steps:
@ -158,7 +158,7 @@ jobs:
- name: Run Claude Code for Translation Sync - name: Run Claude Code for Translation Sync
if: steps.context.outputs.CHANGED_FILES != '' if: steps.context.outputs.CHANGED_FILES != ''
uses: anthropics/claude-code-action@476e359e6203e73dad705c8b322e333fabbd7416 # v1.0.119 uses: anthropics/claude-code-action@b47fd721da662d48c5680e154ad16a73ed74d2e0 # v1.0.93
with: with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ secrets.GITHUB_TOKEN }} github_token: ${{ secrets.GITHUB_TOKEN }}

View File

@ -16,7 +16,7 @@ concurrency:
jobs: jobs:
trigger: trigger:
if: github.repository == 'langgenius/dify' if: github.repository == 'langgenius/dify'
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
timeout-minutes: 5 timeout-minutes: 5
steps: steps:

View File

@ -16,7 +16,7 @@ jobs:
test: test:
name: Full VDB Tests name: Full VDB Tests
if: github.repository == 'langgenius/dify' if: github.repository == 'langgenius/dify'
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
strategy: strategy:
matrix: matrix:
python-version: python-version:
@ -36,7 +36,7 @@ jobs:
remove_tool_cache: true remove_tool_cache: true
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: true enable-cache: true
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
@ -51,7 +51,7 @@ jobs:
- name: Set up dotenvs - name: Set up dotenvs
run: | run: |
cp docker/.env.example docker/.env cp docker/.env.example docker/.env
cp docker/envs/middleware.env.example docker/middleware.env cp docker/middleware.env.example docker/middleware.env
- name: Expose Service Ports - name: Expose Service Ports
run: sh .github/workflows/expose_service_ports.sh run: sh .github/workflows/expose_service_ports.sh
@ -65,7 +65,7 @@ jobs:
# tiflash # tiflash
- name: Set up Full Vector Store Matrix - name: Set up Full Vector Store Matrix
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0 uses: hoverkraft-tech/compose-action@4894d2492015c1774ee5a13a95b1072093087ec3 # v2.5.0
with: with:
compose-file: | compose-file: |
docker/docker-compose.yaml docker/docker-compose.yaml
@ -89,7 +89,7 @@ jobs:
cp api/tests/integration_tests/.env.example api/tests/integration_tests/.env cp api/tests/integration_tests/.env.example api/tests/integration_tests/.env
# - name: Check VDB Ready (TiDB) # - name: Check VDB Ready (TiDB)
# run: uv run --project api python api/providers/vdb/tidb-vector/tests/integration_tests/check_tiflash_ready.py # run: uv run --project api python api/tests/integration_tests/vdb/tidb_vector/check_tiflash_ready.py
- name: Test Vector Stores - name: Test Vector Stores
run: uv run --project api bash dev/pytest/pytest_vdb.sh run: uv run --project api bash dev/pytest/pytest_vdb.sh

View File

@ -13,7 +13,7 @@ concurrency:
jobs: jobs:
test: test:
name: VDB Smoke Tests name: VDB Smoke Tests
runs-on: depot-ubuntu-24.04 runs-on: ubuntu-latest
strategy: strategy:
matrix: matrix:
python-version: python-version:
@ -33,7 +33,7 @@ jobs:
remove_tool_cache: true remove_tool_cache: true
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: true enable-cache: true
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
@ -48,7 +48,7 @@ jobs:
- name: Set up dotenvs - name: Set up dotenvs
run: | run: |
cp docker/.env.example docker/.env cp docker/.env.example docker/.env
cp docker/envs/middleware.env.example docker/middleware.env cp docker/middleware.env.example docker/middleware.env
- name: Expose Service Ports - name: Expose Service Ports
run: sh .github/workflows/expose_service_ports.sh run: sh .github/workflows/expose_service_ports.sh
@ -62,7 +62,7 @@ jobs:
# tiflash # tiflash
- name: Set up Vector Stores for Smoke Coverage - name: Set up Vector Stores for Smoke Coverage
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0 uses: hoverkraft-tech/compose-action@4894d2492015c1774ee5a13a95b1072093087ec3 # v2.5.0
with: with:
compose-file: | compose-file: |
docker/docker-compose.yaml docker/docker-compose.yaml
@ -81,12 +81,12 @@ jobs:
cp api/tests/integration_tests/.env.example api/tests/integration_tests/.env cp api/tests/integration_tests/.env.example api/tests/integration_tests/.env
# - name: Check VDB Ready (TiDB) # - name: Check VDB Ready (TiDB)
# run: uv run --project api python api/providers/vdb/tidb-vector/tests/integration_tests/check_tiflash_ready.py # run: uv run --project api python api/tests/integration_tests/vdb/tidb_vector/check_tiflash_ready.py
- name: Test Vector Stores - name: Test Vector Stores
run: | run: |
uv run --project api pytest --timeout "${PYTEST_TIMEOUT:-180}" \ uv run --project api pytest --timeout "${PYTEST_TIMEOUT:-180}" \
api/providers/vdb/vdb-chroma/tests/integration_tests \ api/tests/integration_tests/vdb/chroma \
api/providers/vdb/vdb-pgvector/tests/integration_tests \ api/tests/integration_tests/vdb/pgvector \
api/providers/vdb/vdb-qdrant/tests/integration_tests \ api/tests/integration_tests/vdb/qdrant \
api/providers/vdb/vdb-weaviate/tests/integration_tests api/tests/integration_tests/vdb/weaviate

View File

@ -13,7 +13,7 @@ concurrency:
jobs: jobs:
test: test:
name: Web Full-Stack E2E name: Web Full-Stack E2E
runs-on: depot-ubuntu-24.04-4 runs-on: ubuntu-latest
defaults: defaults:
run: run:
shell: bash shell: bash
@ -28,7 +28,7 @@ jobs:
uses: ./.github/actions/setup-web uses: ./.github/actions/setup-web
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with: with:
enable-cache: true enable-cache: true
python-version: "3.12" python-version: "3.12"

View File

@ -16,7 +16,7 @@ concurrency:
jobs: jobs:
test: test:
name: Web Tests (${{ matrix.shardIndex }}/${{ matrix.shardTotal }}) name: Web Tests (${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
runs-on: depot-ubuntu-24.04-4 runs-on: ubuntu-latest
env: env:
VITEST_COVERAGE_SCOPE: app-components VITEST_COVERAGE_SCOPE: app-components
strategy: strategy:
@ -54,7 +54,7 @@ jobs:
name: Merge Test Reports name: Merge Test Reports
if: ${{ !cancelled() }} if: ${{ !cancelled() }}
needs: [test] needs: [test]
runs-on: depot-ubuntu-24.04-4 runs-on: ubuntu-latest
env: env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
defaults: defaults:
@ -89,37 +89,3 @@ jobs:
flags: web flags: web
env: env:
CODECOV_TOKEN: ${{ env.CODECOV_TOKEN }} CODECOV_TOKEN: ${{ env.CODECOV_TOKEN }}
dify-ui-test:
name: dify-ui Tests
runs-on: depot-ubuntu-24.04-4
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
defaults:
run:
shell: bash
working-directory: ./packages/dify-ui
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Setup web environment
uses: ./.github/actions/setup-web
- name: Install Chromium for Browser Mode
run: vp exec playwright install --with-deps chromium
- name: Run dify-ui tests
run: vp test run --coverage --silent=passed-only
- name: Report coverage
if: ${{ env.CODECOV_TOKEN != '' }}
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
with:
directory: packages/dify-ui/coverage
flags: dify-ui
env:
CODECOV_TOKEN: ${{ env.CODECOV_TOKEN }}

10
.gitignore vendored
View File

@ -203,7 +203,6 @@ sdks/python-client/dify_client.egg-info
.vscode/* .vscode/*
!.vscode/launch.json.template !.vscode/launch.json.template
!.vscode/settings.example.json
!.vscode/README.md !.vscode/README.md
api/.vscode api/.vscode
# vscode Code History Extension # vscode Code History Extension
@ -219,9 +218,6 @@ node_modules
# plugin migrate # plugin migrate
plugins.jsonl plugins.jsonl
# generated API OpenAPI specs
packages/contracts/openapi/
# mise # mise
mise.toml mise.toml
@ -240,15 +236,9 @@ scripts/stress-test/reports/
.playwright-mcp/ .playwright-mcp/
.serena/ .serena/
# vitest browser mode attachments (failure screenshots, traces, etc.)
.vitest-attachments/
**/__screenshots__/
# settings # settings
*.local.json *.local.json
*.local.md *.local.md
# Code Agent Folder # Code Agent Folder
.qoder/* .qoder/*
.context/*
.eslintcache

1
.npmrc Normal file
View File

@ -0,0 +1 @@
save-exact=true

View File

@ -56,9 +56,44 @@ if $api_modified; then
fi fi
fi fi
if $skip_web_checks; then if $web_modified; then
echo "Git operation in progress, skipping web checks" if $skip_web_checks; then
exit 0 echo "Git operation in progress, skipping web checks"
fi exit 0
fi
vp staged echo "Running ESLint on web module"
if git diff --cached --quiet -- 'web/**/*.ts' 'web/**/*.tsx'; then
web_ts_modified=false
else
ts_diff_status=$?
if [ $ts_diff_status -eq 1 ]; then
web_ts_modified=true
else
echo "Unable to determine staged TypeScript changes (git exit code: $ts_diff_status)."
exit $ts_diff_status
fi
fi
cd ./web || exit 1
vp staged
if $web_ts_modified; then
echo "Running TypeScript type-check:tsgo"
if ! npm run type-check:tsgo; then
echo "Type check failed. Please run 'npm run type-check:tsgo' to fix the errors."
exit 1
fi
else
echo "No staged TypeScript changes detected, skipping type-check:tsgo"
fi
echo "Running knip"
if ! npm run knip; then
echo "Knip check failed. Please run 'npm run knip' to fix the errors."
exit 1
fi
cd ../
fi

View File

@ -2,10 +2,21 @@
"version": "0.2.0", "version": "0.2.0",
"configurations": [ "configurations": [
{ {
"name": "Python: API (gevent)", "name": "Python: Flask API",
"type": "debugpy", "type": "debugpy",
"request": "launch", "request": "launch",
"program": "${workspaceFolder}/api/app.py", "module": "flask",
"env": {
"FLASK_APP": "app.py",
"FLASK_ENV": "development"
},
"args": [
"run",
"--host=0.0.0.0",
"--port=5001",
"--no-debugger",
"--no-reload"
],
"jinja": true, "jinja": true,
"justMyCode": true, "justMyCode": true,
"cwd": "${workspaceFolder}/api", "cwd": "${workspaceFolder}/api",

View File

@ -9,7 +9,6 @@ The codebase is split into:
- **Backend API** (`/api`): Python Flask application organized with Domain-Driven Design - **Backend API** (`/api`): Python Flask application organized with Domain-Driven Design
- **Frontend Web** (`/web`): Next.js application using TypeScript and React - **Frontend Web** (`/web`): Next.js application using TypeScript and React
- **Docker deployment** (`/docker`): Containerized deployment configurations - **Docker deployment** (`/docker`): Containerized deployment configurations
- **Dify Agent Backend** (`/dify-agent`): Backend services for managing and executing agent
## Backend Workflow ## Backend Workflow
@ -31,7 +30,7 @@ The codebase is split into:
## Language Style ## Language Style
- **Python**: Keep type hints on functions and attributes, and implement relevant special methods (e.g., `__repr__`, `__str__`). Prefer `TypedDict` over `dict` or `Mapping` for type safety and better code documentation. - **Python**: Keep type hints on functions and attributes, and implement relevant special methods (e.g., `__repr__`, `__str__`). Prefer `TypedDict` over `dict` or `Mapping` for type safety and better code documentation.
- **TypeScript**: Use the strict config, rely on ESLint (`pnpm lint:fix` preferred) plus `pnpm type-check`, and avoid `any` types. - **TypeScript**: Use the strict config, rely on ESLint (`pnpm lint:fix` preferred) plus `pnpm type-check:tsgo`, and avoid `any` types.
## General Practices ## General Practices

View File

@ -3,10 +3,6 @@ DOCKER_REGISTRY=langgenius
WEB_IMAGE=$(DOCKER_REGISTRY)/dify-web WEB_IMAGE=$(DOCKER_REGISTRY)/dify-web
API_IMAGE=$(DOCKER_REGISTRY)/dify-api API_IMAGE=$(DOCKER_REGISTRY)/dify-api
VERSION=latest VERSION=latest
DOCKER_DIR=docker
DOCKER_MIDDLEWARE_ENV=$(DOCKER_DIR)/middleware.env
DOCKER_MIDDLEWARE_ENV_EXAMPLE=$(DOCKER_DIR)/envs/middleware.env.example
DOCKER_MIDDLEWARE_PROJECT=dify-middlewares-dev
# Default target - show help # Default target - show help
.DEFAULT_GOAL := help .DEFAULT_GOAL := help
@ -21,13 +17,8 @@ dev-setup: prepare-docker prepare-web prepare-api
# Step 1: Prepare Docker middleware # Step 1: Prepare Docker middleware
prepare-docker: prepare-docker:
@echo "🐳 Setting up Docker middleware..." @echo "🐳 Setting up Docker middleware..."
@if [ ! -f "$(DOCKER_MIDDLEWARE_ENV)" ]; then \ @cp -n docker/middleware.env.example docker/middleware.env 2>/dev/null || echo "Docker middleware.env already exists"
cp "$(DOCKER_MIDDLEWARE_ENV_EXAMPLE)" "$(DOCKER_MIDDLEWARE_ENV)"; \ @cd docker && docker compose -f docker-compose.middleware.yaml --env-file middleware.env -p dify-middlewares-dev up -d
echo "Docker middleware.env created"; \
else \
echo "Docker middleware.env already exists"; \
fi
@cd $(DOCKER_DIR) && docker compose -f docker-compose.middleware.yaml --env-file middleware.env -p $(DOCKER_MIDDLEWARE_PROJECT) up -d
@echo "✅ Docker middleware started" @echo "✅ Docker middleware started"
# Step 2: Prepare web environment # Step 2: Prepare web environment
@ -48,18 +39,12 @@ prepare-api:
# Clean dev environment # Clean dev environment
dev-clean: dev-clean:
@echo "⚠️ Stopping Docker containers..." @echo "⚠️ Stopping Docker containers..."
@if [ -f "$(DOCKER_MIDDLEWARE_ENV)" ]; then \ @cd docker && docker compose -f docker-compose.middleware.yaml --env-file middleware.env -p dify-middlewares-dev down
cd $(DOCKER_DIR) && docker compose -f docker-compose.middleware.yaml --env-file middleware.env -p $(DOCKER_MIDDLEWARE_PROJECT) down; \
else \
echo "Docker middleware.env does not exist, skipping compose down"; \
fi
@echo "🗑️ Removing volumes..." @echo "🗑️ Removing volumes..."
@rm -rf docker/volumes/db @rm -rf docker/volumes/db
@rm -rf docker/volumes/mysql
@rm -rf docker/volumes/redis @rm -rf docker/volumes/redis
@rm -rf docker/volumes/plugin_daemon @rm -rf docker/volumes/plugin_daemon
@rm -rf docker/volumes/weaviate @rm -rf docker/volumes/weaviate
@rm -rf docker/volumes/sandbox/dependencies
@rm -rf api/storage @rm -rf api/storage
@echo "✅ Cleanup complete" @echo "✅ Cleanup complete"
@ -83,15 +68,16 @@ lint:
@echo "✅ Linting complete" @echo "✅ Linting complete"
type-check: type-check:
@echo "📝 Running type checks (pyrefly + mypy)..." @echo "📝 Running type checks (basedpyright + pyrefly + mypy)..."
@./dev/pyrefly-check-local $(PATH_TO_CHECK) @./dev/basedpyright-check $(PATH_TO_CHECK)
@uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --exclude 'dev/generate_swagger_specs.py' --exclude 'dev/generate_fastopenapi_specs.py' --check-untyped-defs --disable-error-code=import-untyped . @./dev/pyrefly-check-local
@uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --check-untyped-defs --disable-error-code=import-untyped .
@echo "✅ Type checks complete" @echo "✅ Type checks complete"
type-check-core: type-check-core:
@echo "📝 Running core type checks (pyrefly + mypy)..." @echo "📝 Running core type checks (basedpyright + mypy)..."
@./dev/pyrefly-check-local $(PATH_TO_CHECK) @./dev/basedpyright-check $(PATH_TO_CHECK)
@uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --exclude 'dev/generate_swagger_specs.py' --exclude 'dev/generate_fastopenapi_specs.py' --check-untyped-defs --disable-error-code=import-untyped . @uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --check-untyped-defs --disable-error-code=import-untyped .
@echo "✅ Core type checks complete" @echo "✅ Core type checks complete"
test: test:
@ -146,14 +132,14 @@ help:
@echo " make prepare-docker - Set up Docker middleware" @echo " make prepare-docker - Set up Docker middleware"
@echo " make prepare-web - Set up web environment" @echo " make prepare-web - Set up web environment"
@echo " make prepare-api - Set up API environment" @echo " make prepare-api - Set up API environment"
@echo " make dev-clean - Stop Docker middleware containers and remove dev data" @echo " make dev-clean - Stop Docker middleware containers"
@echo "" @echo ""
@echo "Backend Code Quality:" @echo "Backend Code Quality:"
@echo " make format - Format code with ruff" @echo " make format - Format code with ruff"
@echo " make check - Check code with ruff" @echo " make check - Check code with ruff"
@echo " make lint - Format, fix, and lint code (ruff, imports, dotenv)" @echo " make lint - Format, fix, and lint code (ruff, imports, dotenv)"
@echo " make type-check - Run type checks (pyrefly, mypy)" @echo " make type-check - Run type checks (basedpyright, pyrefly, mypy)"
@echo " make type-check-core - Run core type checks (pyrefly, mypy)" @echo " make type-check-core - Run core type checks (basedpyright, mypy)"
@echo " make test - Run backend unit tests (or TARGET_TESTS=./api/tests/<target_tests>)" @echo " make test - Run backend unit tests (or TARGET_TESTS=./api/tests/<target_tests>)"
@echo "" @echo ""
@echo "Docker Build Targets:" @echo "Docker Build Targets:"

View File

@ -137,7 +137,20 @@ Star Dify on GitHub and be instantly notified of new releases.
### Custom configurations ### Custom configurations
If you need to customize the configuration, edit `docker/.env`. The essential startup defaults live in [`docker/.env.example`](docker/.env.example), and optional advanced variables are split under `docker/envs/` by theme. After making any changes, re-run `docker compose up -d` from the `docker` directory. You can find the full list of available environment variables [here](https://docs.dify.ai/getting-started/install-self-hosted/environments). If you need to customize the configuration, please refer to the comments in our [.env.example](docker/.env.example) file and update the corresponding values in your `.env` file. Additionally, you might need to make adjustments to the `docker-compose.yaml` file itself, such as changing image versions, port mappings, or volume mounts, based on your specific deployment environment and requirements. After making any changes, please re-run `docker compose up -d`. You can find the full list of available environment variables [here](https://docs.dify.ai/getting-started/install-self-hosted/environments).
#### Customizing Suggested Questions
You can now customize the "Suggested Questions After Answer" feature to better fit your use case. For example, to generate longer, more technical questions:
```bash
# In your .env file
SUGGESTED_QUESTIONS_PROMPT='Please help me predict the five most likely technical follow-up questions a developer would ask. Focus on implementation details, best practices, and architecture considerations. Keep each question between 40-60 characters. Output must be JSON array: ["question1","question2","question3","question4","question5"]'
SUGGESTED_QUESTIONS_MAX_TOKENS=512
SUGGESTED_QUESTIONS_TEMPERATURE=0.3
```
See the [Suggested Questions Configuration Guide](docs/suggested-questions-configuration.md) for detailed examples and usage instructions.
### Metrics Monitoring with Grafana ### Metrics Monitoring with Grafana
@ -147,7 +160,7 @@ Import the dashboard to Grafana, using Dify's PostgreSQL database as data source
### Deployment with Kubernetes ### Deployment with Kubernetes
If you'd like to configure a highly available setup, there are community-contributed [Helm Charts](https://helm.sh/) and YAML files which allow Dify to be deployed on Kubernetes. If you'd like to configure a highly-available setup, there are community-contributed [Helm Charts](https://helm.sh/) and YAML files which allow Dify to be deployed on Kubernetes.
- [Helm Chart by @LeoQuote](https://github.com/douban/charts/tree/master/charts/dify) - [Helm Chart by @LeoQuote](https://github.com/douban/charts/tree/master/charts/dify)
- [Helm Chart by @BorisPolonsky](https://github.com/BorisPolonsky/dify-helm) - [Helm Chart by @BorisPolonsky](https://github.com/BorisPolonsky/dify-helm)

View File

@ -33,9 +33,6 @@ TRIGGER_URL=http://localhost:5001
# The time in seconds after the signature is rejected # The time in seconds after the signature is rejected
FILES_ACCESS_TIMEOUT=300 FILES_ACCESS_TIMEOUT=300
# Collaboration mode toggle
ENABLE_COLLABORATION_MODE=true
# Access token expiration time in minutes # Access token expiration time in minutes
ACCESS_TOKEN_EXPIRE_MINUTES=60 ACCESS_TOKEN_EXPIRE_MINUTES=60
@ -60,9 +57,6 @@ REDIS_SSL_CERTFILE=
REDIS_SSL_KEYFILE= REDIS_SSL_KEYFILE=
# Path to client private key file for SSL authentication # Path to client private key file for SSL authentication
REDIS_DB=0 REDIS_DB=0
# Optional global prefix for Redis keys, topics, streams, and Celery Redis transport artifacts.
# Leave empty to preserve current unprefixed behavior.
REDIS_KEY_PREFIX=
# redis Sentinel configuration. # redis Sentinel configuration.
REDIS_USE_SENTINEL=false REDIS_USE_SENTINEL=false
@ -88,10 +82,6 @@ REDIS_HEALTH_CHECK_INTERVAL=30
CELERY_BROKER_URL=redis://:difyai123456@localhost:${REDIS_PORT}/1 CELERY_BROKER_URL=redis://:difyai123456@localhost:${REDIS_PORT}/1
CELERY_BACKEND=redis CELERY_BACKEND=redis
# Ops trace retry configuration
OPS_TRACE_RETRYABLE_DISPATCH_MAX_RETRIES=60
OPS_TRACE_RETRYABLE_DISPATCH_DELAY_SECONDS=5
# Database configuration # Database configuration
DB_TYPE=postgresql DB_TYPE=postgresql
DB_USERNAME=postgres DB_USERNAME=postgres
@ -102,8 +92,6 @@ DB_DATABASE=dify
SQLALCHEMY_POOL_PRE_PING=true SQLALCHEMY_POOL_PRE_PING=true
SQLALCHEMY_POOL_TIMEOUT=30 SQLALCHEMY_POOL_TIMEOUT=30
# Connection pool reset behavior on return
SQLALCHEMY_POOL_RESET_ON_RETURN=rollback
# Storage configuration # Storage configuration
# use for store upload files, private keys... # use for store upload files, private keys...
@ -387,7 +375,7 @@ VIKINGDB_ACCESS_KEY=your-ak
VIKINGDB_SECRET_KEY=your-sk VIKINGDB_SECRET_KEY=your-sk
VIKINGDB_REGION=cn-shanghai VIKINGDB_REGION=cn-shanghai
VIKINGDB_HOST=api-vikingdb.xxx.volces.com VIKINGDB_HOST=api-vikingdb.xxx.volces.com
VIKINGDB_SCHEME=http VIKINGDB_SCHEMA=http
VIKINGDB_CONNECTION_TIMEOUT=30 VIKINGDB_CONNECTION_TIMEOUT=30
VIKINGDB_SOCKET_TIMEOUT=30 VIKINGDB_SOCKET_TIMEOUT=30
@ -438,6 +426,8 @@ UPLOAD_FILE_EXTENSION_BLACKLIST=
# Model configuration # Model configuration
MULTIMODAL_SEND_FORMAT=base64 MULTIMODAL_SEND_FORMAT=base64
PROMPT_GENERATION_MAX_TOKENS=512
CODE_GENERATION_MAX_TOKENS=1024
PLUGIN_BASED_TOKEN_COUNTING_ENABLED=false PLUGIN_BASED_TOKEN_COUNTING_ENABLED=false
# Mail configuration, support: resend, smtp, sendgrid # Mail configuration, support: resend, smtp, sendgrid
@ -663,11 +653,6 @@ INNER_API_KEY_FOR_PLUGIN=QaHbTe77CtuXmsfyhR7+vRjI/+XbV1AaFy691iy+kGDv2Jvy0/eAh8Y
MARKETPLACE_ENABLED=true MARKETPLACE_ENABLED=true
MARKETPLACE_API_URL=https://marketplace.dify.ai MARKETPLACE_API_URL=https://marketplace.dify.ai
# Creators Platform configuration
CREATORS_PLATFORM_FEATURES_ENABLED=true
CREATORS_PLATFORM_API_URL=https://creators.dify.ai
CREATORS_PLATFORM_OAUTH_CLIENT_ID=
# Endpoint configuration # Endpoint configuration
ENDPOINT_URL_TEMPLATE=http://localhost:5002/e/{hook_id} ENDPOINT_URL_TEMPLATE=http://localhost:5002/e/{hook_id}
@ -718,6 +703,22 @@ SWAGGER_UI_PATH=/swagger-ui.html
# Set to false to export dataset IDs as plain text for easier cross-environment import # Set to false to export dataset IDs as plain text for easier cross-environment import
DSL_EXPORT_ENCRYPT_DATASET_ID=true DSL_EXPORT_ENCRYPT_DATASET_ID=true
# Suggested Questions After Answer Configuration
# These environment variables allow customization of the suggested questions feature
#
# Custom prompt for generating suggested questions (optional)
# If not set, uses the default prompt that generates 3 questions under 20 characters each
# Example: "Please help me predict the five most likely technical follow-up questions a developer would ask. Focus on implementation details, best practices, and architecture considerations. Keep each question between 40-60 characters. Output must be JSON array: [\"question1\",\"question2\",\"question3\",\"question4\",\"question5\"]"
# SUGGESTED_QUESTIONS_PROMPT=
# Maximum number of tokens for suggested questions generation (default: 256)
# Adjust this value for longer questions or more questions
# SUGGESTED_QUESTIONS_MAX_TOKENS=256
# Temperature for suggested questions generation (default: 0.0)
# Higher values (0.5-1.0) produce more creative questions, lower values (0.0-0.3) produce more focused questions
# SUGGESTED_QUESTIONS_TEMPERATURE=0
# Tenant isolated task queue configuration # Tenant isolated task queue configuration
TENANT_ISOLATED_TASK_CONCURRENCY=1 TENANT_ISOLATED_TASK_CONCURRENCY=1

View File

@ -69,6 +69,8 @@ ignore = [
"FURB152", # math-constant "FURB152", # math-constant
"UP007", # non-pep604-annotation "UP007", # non-pep604-annotation
"UP032", # f-string "UP032", # f-string
"UP045", # non-pep604-annotation-optional
"B005", # strip-with-multi-characters
"B006", # mutable-argument-default "B006", # mutable-argument-default
"B007", # unused-loop-control-variable "B007", # unused-loop-control-variable
"B026", # star-arg-unpacking-after-keyword-arg "B026", # star-arg-unpacking-after-keyword-arg
@ -82,6 +84,7 @@ ignore = [
"SIM102", # collapsible-if "SIM102", # collapsible-if
"SIM103", # needless-bool "SIM103", # needless-bool
"SIM105", # suppressible-exception "SIM105", # suppressible-exception
"SIM107", # return-in-try-except-finally
"SIM108", # if-else-block-instead-of-if-exp "SIM108", # if-else-block-instead-of-if-exp
"SIM113", # enumerate-for-loop "SIM113", # enumerate-for-loop
"SIM117", # multiple-with-statements "SIM117", # multiple-with-statements
@ -90,22 +93,32 @@ ignore = [
] ]
[lint.per-file-ignores] [lint.per-file-ignores]
"__init__.py" = [
"F401", # unused-import
"F811", # redefined-while-unused
]
"configs/*" = [ "configs/*" = [
"N802", # invalid-function-name "N802", # invalid-function-name
] ]
"graphon/model_runtime/callbacks/base_callback.py" = ["T201"]
"core/workflow/callbacks/workflow_logging_callback.py" = ["T201"]
"libs/gmpy2_pkcs10aep_cipher.py" = [ "libs/gmpy2_pkcs10aep_cipher.py" = [
"N803", # invalid-argument-name "N803", # invalid-argument-name
] ]
"tests/*" = [ "tests/*" = [
"F811", # redefined-while-unused
"T201", # allow print in tests, "T201", # allow print in tests,
"S110", # allow ignoring exceptions in tests code (currently) "S110", # allow ignoring exceptions in tests code (currently)
] ]
"controllers/console/explore/trial.py" = ["TID251"]
"controllers/console/human_input_form.py" = ["TID251"]
"controllers/web/human_input_form.py" = ["TID251"]
[lint.flake8-tidy-imports]
[lint.flake8-tidy-imports.banned-api."flask_restx.reqparse"] [lint.flake8-tidy-imports.banned-api."flask_restx.reqparse"]
msg = "Use Pydantic payload/query models instead of reqparse." msg = "Use Pydantic payload/query models instead of reqparse."
[lint.flake8-tidy-imports.banned-api."flask_restx.reqparse.RequestParser"] [lint.flake8-tidy-imports.banned-api."flask_restx.reqparse.RequestParser"]
msg = "Use Pydantic payload/query models instead of reqparse." msg = "Use Pydantic payload/query models instead of reqparse."
[lint.isort]
known-first-party = ["graphon"]

View File

@ -3,21 +3,29 @@
"compounds": [ "compounds": [
{ {
"name": "Launch Flask and Celery", "name": "Launch Flask and Celery",
"configurations": ["Python: API (gevent)", "Python: Celery"] "configurations": ["Python: Flask", "Python: Celery"]
} }
], ],
"configurations": [ "configurations": [
{ {
"name": "Python: API (gevent)", "name": "Python: Flask",
"consoleName": "API", "consoleName": "Flask",
"type": "debugpy", "type": "debugpy",
"request": "launch", "request": "launch",
"python": "${workspaceFolder}/.venv/bin/python", "python": "${workspaceFolder}/.venv/bin/python",
"cwd": "${workspaceFolder}", "cwd": "${workspaceFolder}",
"envFile": ".env", "envFile": ".env",
"program": "${workspaceFolder}/app.py", "module": "flask",
"justMyCode": true, "justMyCode": true,
"jinja": true "jinja": true,
"env": {
"FLASK_APP": "app.py",
"GEVENT_SUPPORT": "True"
},
"args": [
"run",
"--port=5001"
]
}, },
{ {
"name": "Python: Celery", "name": "Python: Celery",

View File

@ -193,10 +193,6 @@ Before opening a PR / submitting:
- Controllers: parse input via Pydantic, invoke services, return serialised responses; no business logic. - Controllers: parse input via Pydantic, invoke services, return serialised responses; no business logic.
- Services: coordinate repositories, providers, background tasks; keep side effects explicit. - Services: coordinate repositories, providers, background tasks; keep side effects explicit.
- Document non-obvious behaviour with concise docstrings and comments. - Document non-obvious behaviour with concise docstrings and comments.
- For Flask-RESTX controller request, query, and response schemas, follow `controllers/API_SCHEMA_GUIDE.md`.
In short: use Pydantic models, document GET query params with `query_params_from_model(...)`, register response
DTOs with `register_response_schema_models(...)`, serialize with `ResponseModel.model_validate(...).model_dump(...)`,
and avoid adding new legacy `ns.model(...)`, `@marshal_with(...)`, or GET `@ns.expect(...)` patterns.
### Miscellaneous ### Miscellaneous

View File

@ -21,9 +21,8 @@ RUN apt-get update \
# for building gmpy2 # for building gmpy2
libmpfr-dev libmpc-dev libmpfr-dev libmpc-dev
# Install Python dependencies (workspace members under providers/vdb/) # Install Python dependencies
COPY pyproject.toml uv.lock ./ COPY pyproject.toml uv.lock ./
COPY providers ./providers
RUN uv sync --locked --no-dev RUN uv sync --locked --no-dev
# production stage # production stage

View File

@ -99,13 +99,5 @@ The scripts resolve paths relative to their location, so you can run them from a
./dev/reformat # Run all formatters and linters ./dev/reformat # Run all formatters and linters
uv run ruff check --fix ./ # Fix linting issues uv run ruff check --fix ./ # Fix linting issues
uv run ruff format ./ # Format code uv run ruff format ./ # Format code
uv run pyrefly check # Type checking uv run basedpyright . # Type checking
``` ```
## Generate TS stub
```
uv run dev/generate_swagger_specs.py --output-dir openapi
```
use https://jsontotable.org/openapi-to-typescript to convert to typescript

View File

@ -1,6 +1,5 @@
from __future__ import annotations from __future__ import annotations
import logging
import sys import sys
from typing import TYPE_CHECKING, cast from typing import TYPE_CHECKING, cast
@ -10,35 +9,17 @@ if TYPE_CHECKING:
celery: Celery celery: Celery
HOST = "0.0.0.0"
PORT = 5001
logger = logging.getLogger(__name__)
def is_db_command() -> bool: def is_db_command() -> bool:
if len(sys.argv) > 1 and sys.argv[0].endswith("flask") and sys.argv[1] == "db": if len(sys.argv) > 1 and sys.argv[0].endswith("flask") and sys.argv[1] == "db":
return True return True
return False return False
def log_startup_banner(host: str, port: int) -> None:
debugger_attached = sys.gettrace() is not None
logger.info("Serving Dify API via gevent WebSocket server")
logger.info("Bound to http://%s:%s", host, port)
logger.info("Debugger attached: %s", "on" if debugger_attached else "off")
logger.info("Press CTRL+C to quit")
# create app # create app
flask_app = None
socketio_app = None
if is_db_command(): if is_db_command():
from app_factory import create_migrations_app from app_factory import create_migrations_app
app = create_migrations_app() app = create_migrations_app()
socketio_app = app
flask_app = app
else: else:
# Gunicorn and Celery handle monkey patching automatically in production by # Gunicorn and Celery handle monkey patching automatically in production by
# specifying the `gevent` worker class. Manual monkey patching is not required here. # specifying the `gevent` worker class. Manual monkey patching is not required here.
@ -49,14 +30,8 @@ else:
from app_factory import create_app from app_factory import create_app
socketio_app, flask_app = create_app() app = create_app()
app = flask_app
celery = cast("Celery", app.extensions["celery"]) celery = cast("Celery", app.extensions["celery"])
if __name__ == "__main__": if __name__ == "__main__":
from gevent import pywsgi app.run(host="0.0.0.0", port=5001)
from geventwebsocket.handler import WebSocketHandler # type: ignore[reportMissingTypeStubs]
log_startup_banner(HOST, PORT)
server = pywsgi.WSGIServer((HOST, PORT), socketio_app, handler_class=WebSocketHandler)
server.serve_forever()

View File

@ -1,7 +1,6 @@
import logging import logging
import time import time
import socketio # type: ignore[reportMissingTypeStubs]
from flask import request from flask import request
from opentelemetry.trace import get_current_span from opentelemetry.trace import get_current_span
from opentelemetry.trace.span import INVALID_SPAN_ID, INVALID_TRACE_ID from opentelemetry.trace.span import INVALID_SPAN_ID, INVALID_TRACE_ID
@ -11,7 +10,6 @@ from contexts.wrapper import RecyclableContextVar
from controllers.console.error import UnauthorizedAndForceLogout from controllers.console.error import UnauthorizedAndForceLogout
from core.logging.context import init_request_context from core.logging.context import init_request_context
from dify_app import DifyApp from dify_app import DifyApp
from extensions.ext_socketio import sio
from services.enterprise.enterprise_service import EnterpriseService from services.enterprise.enterprise_service import EnterpriseService
from services.feature_service import LicenseStatus from services.feature_service import LicenseStatus
@ -117,25 +115,21 @@ def create_flask_app_with_configs() -> DifyApp:
logger.warning("Failed to add trace headers to response", exc_info=True) logger.warning("Failed to add trace headers to response", exc_info=True)
return response return response
# Capture the decorator return values so static checkers do not treat the hooks as unused. # Capture the decorator's return value to avoid pyright reportUnusedFunction
_ = before_request _ = before_request
_ = add_trace_headers _ = add_trace_headers
return dify_app return dify_app
def create_app() -> tuple[socketio.WSGIApp, DifyApp]: def create_app() -> DifyApp:
start_time = time.perf_counter() start_time = time.perf_counter()
app = create_flask_app_with_configs() app = create_flask_app_with_configs()
initialize_extensions(app) initialize_extensions(app)
sio.app = app
socketio_app = socketio.WSGIApp(sio, app)
end_time = time.perf_counter() end_time = time.perf_counter()
if dify_config.DEBUG: if dify_config.DEBUG:
logger.info("Finished create_app (%s ms)", round((end_time - start_time) * 1000, 2)) logger.info("Finished create_app (%s ms)", round((end_time - start_time) * 1000, 2))
return socketio_app, app return app
def initialize_extensions(app: DifyApp): def initialize_extensions(app: DifyApp):
@ -181,6 +175,7 @@ def initialize_extensions(app: DifyApp):
ext_import_modules, ext_import_modules,
ext_orjson, ext_orjson,
ext_forward_refs, ext_forward_refs,
ext_set_secretkey,
ext_compress, ext_compress,
ext_code_based_extension, ext_code_based_extension,
ext_database, ext_database,
@ -188,7 +183,6 @@ def initialize_extensions(app: DifyApp):
ext_migrate, ext_migrate,
ext_redis, ext_redis,
ext_storage, ext_storage,
ext_set_secretkey,
ext_logstore, # Initialize logstore after storage, before celery ext_logstore, # Initialize logstore after storage, before celery
ext_celery, ext_celery,
ext_login, ext_login,

View File

@ -2,7 +2,6 @@ import base64
import secrets import secrets
import click import click
from sqlalchemy.orm import Session
from constants.languages import languages from constants.languages import languages
from extensions.ext_database import db from extensions.ext_database import db
@ -44,11 +43,10 @@ def reset_password(email, new_password, password_confirm):
# encrypt password with salt # encrypt password with salt
password_hashed = hash_password(new_password, salt) password_hashed = hash_password(new_password, salt)
base64_password_hashed = base64.b64encode(password_hashed).decode() base64_password_hashed = base64.b64encode(password_hashed).decode()
with Session(db.engine) as session: account = db.session.merge(account)
account = session.merge(account) account.password = base64_password_hashed
account.password = base64_password_hashed account.password_salt = base64_salt
account.password_salt = base64_salt db.session.commit()
session.commit()
AccountService.reset_login_error_rate_limit(normalized_email) AccountService.reset_login_error_rate_limit(normalized_email)
click.echo(click.style("Password reset successfully.", fg="green")) click.echo(click.style("Password reset successfully.", fg="green"))
@ -79,10 +77,9 @@ def reset_email(email, new_email, email_confirm):
click.echo(click.style(f"Invalid email: {new_email}", fg="red")) click.echo(click.style(f"Invalid email: {new_email}", fg="red"))
return return
with Session(db.engine) as session: account = db.session.merge(account)
account = session.merge(account) account.email = normalized_new_email
account.email = normalized_new_email db.session.commit()
session.commit()
click.echo(click.style("Email updated successfully.", fg="green")) click.echo(click.style("Email updated successfully.", fg="green"))
@ -113,18 +110,8 @@ def create_tenant(email: str, language: str | None = None, name: str | None = No
# Validates name encoding for non-Latin characters. # Validates name encoding for non-Latin characters.
name = name.strip().encode("utf-8").decode("utf-8") if name else None name = name.strip().encode("utf-8").decode("utf-8") if name else None
# Generate a random password that satisfies the password policy. # generate random password
# The iteration limit guards against infinite loops caused by unexpected bugs in valid_password. new_password = secrets.token_urlsafe(16)
for _ in range(100):
new_password = secrets.token_urlsafe(16)
try:
valid_password(new_password)
break
except Exception:
continue
else:
click.echo(click.style("Failed to generate a valid password. Please try again.", fg="red"))
return
# register account # register account
account = RegisterService.register( account = RegisterService.register(

View File

@ -11,7 +11,7 @@ from configs import dify_config
from core.helper import encrypter from core.helper import encrypter
from core.plugin.entities.plugin_daemon import CredentialType from core.plugin.entities.plugin_daemon import CredentialType
from core.plugin.impl.plugin import PluginInstaller from core.plugin.impl.plugin import PluginInstaller
from core.tools.utils.system_encryption import encrypt_system_params from core.tools.utils.system_oauth_encryption import encrypt_system_oauth_params
from extensions.ext_database import db from extensions.ext_database import db
from models import Tenant from models import Tenant
from models.oauth import DatasourceOauthParamConfig, DatasourceProvider from models.oauth import DatasourceOauthParamConfig, DatasourceProvider
@ -44,7 +44,7 @@ def setup_system_tool_oauth_client(provider, client_params):
click.echo(click.style(f"Encrypting client params: {client_params}", fg="yellow")) click.echo(click.style(f"Encrypting client params: {client_params}", fg="yellow"))
click.echo(click.style(f"Using SECRET_KEY: `{dify_config.SECRET_KEY}`", fg="yellow")) click.echo(click.style(f"Using SECRET_KEY: `{dify_config.SECRET_KEY}`", fg="yellow"))
oauth_client_params = encrypt_system_params(client_params_dict) oauth_client_params = encrypt_system_oauth_params(client_params_dict)
click.echo(click.style("Client params encrypted successfully.", fg="green")) click.echo(click.style("Client params encrypted successfully.", fg="green"))
except Exception as e: except Exception as e:
click.echo(click.style(f"Error parsing client params: {str(e)}", fg="red")) click.echo(click.style(f"Error parsing client params: {str(e)}", fg="red"))
@ -94,7 +94,7 @@ def setup_system_trigger_oauth_client(provider, client_params):
click.echo(click.style(f"Encrypting client params: {client_params}", fg="yellow")) click.echo(click.style(f"Encrypting client params: {client_params}", fg="yellow"))
click.echo(click.style(f"Using SECRET_KEY: `{dify_config.SECRET_KEY}`", fg="yellow")) click.echo(click.style(f"Using SECRET_KEY: `{dify_config.SECRET_KEY}`", fg="yellow"))
oauth_client_params = encrypt_system_params(client_params_dict) oauth_client_params = encrypt_system_oauth_params(client_params_dict)
click.echo(click.style("Client params encrypted successfully.", fg="green")) click.echo(click.style("Client params encrypted successfully.", fg="green"))
except Exception as e: except Exception as e:
click.echo(click.style(f"Error parsing client params: {str(e)}", fg="red")) click.echo(click.style(f"Error parsing client params: {str(e)}", fg="red"))
@ -185,9 +185,9 @@ def transform_datasource_credentials(environment: str):
firecrawl_plugin_id = "langgenius/firecrawl_datasource" firecrawl_plugin_id = "langgenius/firecrawl_datasource"
jina_plugin_id = "langgenius/jina_datasource" jina_plugin_id = "langgenius/jina_datasource"
if environment == "online": if environment == "online":
notion_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(notion_plugin_id) notion_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(notion_plugin_id) # pyright: ignore[reportPrivateUsage]
firecrawl_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(firecrawl_plugin_id) firecrawl_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(firecrawl_plugin_id) # pyright: ignore[reportPrivateUsage]
jina_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(jina_plugin_id) jina_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(jina_plugin_id) # pyright: ignore[reportPrivateUsage]
else: else:
notion_plugin_unique_identifier = None notion_plugin_unique_identifier = None
firecrawl_plugin_unique_identifier = None firecrawl_plugin_unique_identifier = None

View File

@ -341,10 +341,11 @@ def add_qdrant_index(field: str):
click.echo(click.style("No dataset collection bindings found.", fg="red")) click.echo(click.style("No dataset collection bindings found.", fg="red"))
return return
import qdrant_client import qdrant_client
from dify_vdb_qdrant.qdrant_vector import PathQdrantParams, QdrantConfig
from qdrant_client.http.exceptions import UnexpectedResponse from qdrant_client.http.exceptions import UnexpectedResponse
from qdrant_client.http.models import PayloadSchemaType from qdrant_client.http.models import PayloadSchemaType
from core.rag.datasource.vdb.qdrant.qdrant_vector import PathQdrantParams, QdrantConfig
for binding in bindings: for binding in bindings:
if dify_config.QDRANT_URL is None: if dify_config.QDRANT_URL is None:
raise ValueError("Qdrant URL is required.") raise ValueError("Qdrant URL is required.")

View File

@ -23,12 +23,6 @@ class EnterpriseFeatureConfig(BaseSettings):
ge=1, description="Maximum timeout in seconds for enterprise requests", default=5 ge=1, description="Maximum timeout in seconds for enterprise requests", default=5
) )
ENTERPRISE_DISABLE_RUNTIME_CREDENTIAL_CHECK: bool = Field(
default=False,
description="If disabled, credential policy check is only performed when saving workflows."
"This helps gain runtime performance by trading off consistency.",
)
class EnterpriseTelemetryConfig(BaseSettings): class EnterpriseTelemetryConfig(BaseSettings):
""" """

View File

@ -23,9 +23,9 @@ class SecurityConfig(BaseSettings):
""" """
SECRET_KEY: str = Field( SECRET_KEY: str = Field(
description="Secret key for secure session cookie signing. " description="Secret key for secure session cookie signing."
"Leave empty to let Dify generate a persistent key in the storage directory, " "Make sure you are changing this key for your deployment with a strong key."
"or set a strong value via the `SECRET_KEY` environment variable.", "Generate a strong key using `openssl rand -base64 42` or set via the `SECRET_KEY` environment variable.",
default="", default="",
) )
@ -289,11 +289,11 @@ class MarketplaceConfig(BaseSettings):
class CreatorsPlatformConfig(BaseSettings): class CreatorsPlatformConfig(BaseSettings):
""" """
Configuration for Creators Platform integration Configuration for creators platform
""" """
CREATORS_PLATFORM_FEATURES_ENABLED: bool = Field( CREATORS_PLATFORM_FEATURES_ENABLED: bool = Field(
description="Enable or disable Creators Platform features", description="Enable or disable creators platform features",
default=True, default=True,
) )
@ -303,7 +303,7 @@ class CreatorsPlatformConfig(BaseSettings):
) )
CREATORS_PLATFORM_OAUTH_CLIENT_ID: str = Field( CREATORS_PLATFORM_OAUTH_CLIENT_ID: str = Field(
description="OAuth client ID for Creators Platform integration", description="OAuth client_id for the Creators Platform app registered in Dify",
default="", default="",
) )
@ -362,6 +362,15 @@ class FileAccessConfig(BaseSettings):
default="", default="",
) )
FILES_API_URL: str = Field(
description="Base URL for storage file ticket API endpoints."
" Used by sandbox containers (internal or external like e2b) that need"
" an absolute, routable address to upload/download files via the API."
" For all-in-one Docker deployments, set to http://localhost."
" For public sandbox environments, set to a public domain or IP.",
default="",
)
FILES_ACCESS_TIMEOUT: int = Field( FILES_ACCESS_TIMEOUT: int = Field(
description="Expiration time in seconds for file access URLs", description="Expiration time in seconds for file access URLs",
default=300, default=300,
@ -1137,18 +1146,6 @@ class MultiModalTransferConfig(BaseSettings):
) )
class OpsTraceConfig(BaseSettings):
OPS_TRACE_RETRYABLE_DISPATCH_MAX_RETRIES: PositiveInt = Field(
description="Maximum retry attempts for transient ops trace provider dispatch failures.",
default=60,
)
OPS_TRACE_RETRYABLE_DISPATCH_DELAY_SECONDS: PositiveInt = Field(
description="Delay in seconds between transient ops trace provider dispatch retry attempts.",
default=5,
)
class CeleryBeatConfig(BaseSettings): class CeleryBeatConfig(BaseSettings):
CELERY_BEAT_SCHEDULER_TIME: int = Field( CELERY_BEAT_SCHEDULER_TIME: int = Field(
description="Interval in days for Celery Beat scheduler execution, default to 1 day", description="Interval in days for Celery Beat scheduler execution, default to 1 day",
@ -1310,7 +1307,46 @@ class PositionConfig(BaseSettings):
class CollaborationConfig(BaseSettings): class CollaborationConfig(BaseSettings):
ENABLE_COLLABORATION_MODE: bool = Field( ENABLE_COLLABORATION_MODE: bool = Field(
description="Whether to enable collaboration mode features across the workspace", description="Whether to enable collaboration mode features across the workspace",
default=True, default=False,
)
class SandboxExpiredRecordsCleanConfig(BaseSettings):
SANDBOX_EXPIRED_RECORDS_CLEAN_GRACEFUL_PERIOD: NonNegativeInt = Field(
description="Graceful period in days for sandbox records clean after subscription expiration",
default=21,
)
SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_SIZE: PositiveInt = Field(
description="Maximum number of records to process in each batch",
default=1000,
)
SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_MAX_INTERVAL: PositiveInt = Field(
description="Maximum interval in milliseconds between batches",
default=200,
)
SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS: PositiveInt = Field(
description="Retention days for sandbox expired workflow_run records and message records",
default=30,
)
SANDBOX_EXPIRED_RECORDS_CLEAN_TASK_LOCK_TTL: PositiveInt = Field(
description="Lock TTL for sandbox expired records clean task in seconds",
default=90000,
)
class AgentV2UpgradeConfig(BaseSettings):
"""Feature flags for transparent Agent V2 upgrade."""
AGENT_V2_TRANSPARENT_UPGRADE: bool = Field(
description="Transparently run old apps (chat/completion/agent-chat) through the Agent V2 workflow engine. "
"When enabled, old apps synthesize a virtual workflow at runtime instead of using legacy runners.",
default=False,
)
AGENT_V2_REPLACES_LLM: bool = Field(
description="Transparently replace LLM nodes in workflows with Agent V2 nodes at runtime. "
"LLMNodeData is remapped to AgentV2NodeData with tools=[] (identical behavior).",
default=False,
) )
@ -1383,40 +1419,17 @@ class TenantIsolatedTaskQueueConfig(BaseSettings):
) )
class SandboxExpiredRecordsCleanConfig(BaseSettings):
SANDBOX_EXPIRED_RECORDS_CLEAN_GRACEFUL_PERIOD: NonNegativeInt = Field(
description="Graceful period in days for sandbox records clean after subscription expiration",
default=21,
)
SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_SIZE: PositiveInt = Field(
description="Maximum number of records to process in each batch",
default=1000,
)
SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_MAX_INTERVAL: PositiveInt = Field(
description="Maximum interval in milliseconds between batches",
default=200,
)
SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS: PositiveInt = Field(
description="Retention days for sandbox expired workflow_run records and message records",
default=30,
)
SANDBOX_EXPIRED_RECORDS_CLEAN_TASK_LOCK_TTL: PositiveInt = Field(
description="Lock TTL for sandbox expired records clean task in seconds",
default=90000,
)
class FeatureConfig( class FeatureConfig(
# place the configs in alphabet order # place the configs in alphabet order
AppExecutionConfig, AppExecutionConfig,
AuthConfig, # Changed from OAuthConfig to AuthConfig AuthConfig, # Changed from OAuthConfig to AuthConfig
BillingConfig, BillingConfig,
CodeExecutionSandboxConfig, CodeExecutionSandboxConfig,
CreatorsPlatformConfig,
TriggerConfig, TriggerConfig,
AsyncWorkflowConfig, AsyncWorkflowConfig,
PluginConfig, PluginConfig,
MarketplaceConfig, MarketplaceConfig,
CreatorsPlatformConfig,
DataSetConfig, DataSetConfig,
EndpointConfig, EndpointConfig,
FileAccessConfig, FileAccessConfig,
@ -1429,11 +1442,9 @@ class FeatureConfig(
ModelLoadBalanceConfig, ModelLoadBalanceConfig,
ModerationConfig, ModerationConfig,
MultiModalTransferConfig, MultiModalTransferConfig,
OpsTraceConfig,
PositionConfig, PositionConfig,
RagEtlConfig, RagEtlConfig,
RepositoryConfig, RepositoryConfig,
SandboxExpiredRecordsCleanConfig,
SecurityConfig, SecurityConfig,
TenantIsolatedTaskQueueConfig, TenantIsolatedTaskQueueConfig,
ToolConfig, ToolConfig,
@ -1442,6 +1453,8 @@ class FeatureConfig(
WorkflowNodeExecutionConfig, WorkflowNodeExecutionConfig,
WorkspaceConfig, WorkspaceConfig,
CollaborationConfig, CollaborationConfig,
AgentV2UpgradeConfig,
SandboxExpiredRecordsCleanConfig,
LoginConfig, LoginConfig,
AccountConfig, AccountConfig,
SwaggerUIConfig, SwaggerUIConfig,

View File

@ -1,5 +1,5 @@
import os import os
from typing import Any, Literal, TypedDict, cast from typing import Any, Literal, TypedDict
from urllib.parse import parse_qsl, quote_plus from urllib.parse import parse_qsl, quote_plus
from pydantic import Field, NonNegativeFloat, NonNegativeInt, PositiveFloat, PositiveInt, computed_field from pydantic import Field, NonNegativeFloat, NonNegativeInt, PositiveFloat, PositiveInt, computed_field
@ -50,30 +50,28 @@ from .vdb.vastbase_vector_config import VastbaseVectorConfig
from .vdb.vikingdb_config import VikingDBConfig from .vdb.vikingdb_config import VikingDBConfig
from .vdb.weaviate_config import WeaviateConfig from .vdb.weaviate_config import WeaviateConfig
_VALID_STORAGE_TYPE = Literal[
"opendal",
"s3",
"aliyun-oss",
"azure-blob",
"baidu-obs",
"clickzetta-volume",
"google-storage",
"huawei-obs",
"oci-storage",
"tencent-cos",
"volcengine-tos",
"supabase",
"local",
]
class StorageConfig(BaseSettings): class StorageConfig(BaseSettings):
STORAGE_TYPE: _VALID_STORAGE_TYPE = Field( STORAGE_TYPE: Literal[
"opendal",
"s3",
"aliyun-oss",
"azure-blob",
"baidu-obs",
"clickzetta-volume",
"google-storage",
"huawei-obs",
"oci-storage",
"tencent-cos",
"volcengine-tos",
"supabase",
"local",
] = Field(
description="Type of storage to use." description="Type of storage to use."
" Options: 'opendal', '(deprecated) local', 's3', 'aliyun-oss', 'azure-blob', 'baidu-obs', " " Options: 'opendal', '(deprecated) local', 's3', 'aliyun-oss', 'azure-blob', 'baidu-obs', "
"'clickzetta-volume', 'google-storage', 'huawei-obs', 'oci-storage', 'tencent-cos', " "'clickzetta-volume', 'google-storage', 'huawei-obs', 'oci-storage', 'tencent-cos', "
"'volcengine-tos', 'supabase'. Default is 'opendal'.", "'volcengine-tos', 'supabase'. Default is 'opendal'.",
default=cast(_VALID_STORAGE_TYPE, "opendal"), default="opendal",
) )
STORAGE_LOCAL_PATH: str = Field( STORAGE_LOCAL_PATH: str = Field(
@ -116,7 +114,7 @@ class SQLAlchemyEngineOptionsDict(TypedDict):
pool_pre_ping: bool pool_pre_ping: bool
connect_args: dict[str, str] connect_args: dict[str, str]
pool_use_lifo: bool pool_use_lifo: bool
pool_reset_on_return: Literal["commit", "rollback", None] pool_reset_on_return: None
pool_timeout: int pool_timeout: int
@ -162,16 +160,6 @@ class DatabaseConfig(BaseSettings):
default="", default="",
) )
DB_SESSION_TIMEZONE_OVERRIDE: str = Field(
description=(
"PostgreSQL session timezone override injected via startup options."
" Default is 'UTC' for out-of-the-box consistency."
" Set to empty string to disable app-level timezone injection, for example when using RDS Proxy"
" together with a database-side default timezone."
),
default="UTC",
)
@computed_field # type: ignore[prop-decorator] @computed_field # type: ignore[prop-decorator]
@property @property
def SQLALCHEMY_DATABASE_URI_SCHEME(self) -> str: def SQLALCHEMY_DATABASE_URI_SCHEME(self) -> str:
@ -225,11 +213,6 @@ class DatabaseConfig(BaseSettings):
default=30, default=30,
) )
SQLALCHEMY_POOL_RESET_ON_RETURN: Literal["commit", "rollback", None] = Field(
description="Connection pool reset behavior on return. Options: 'commit', 'rollback', or None",
default="rollback",
)
RETRIEVAL_SERVICE_EXECUTORS: NonNegativeInt = Field( RETRIEVAL_SERVICE_EXECUTORS: NonNegativeInt = Field(
description="Number of processes for the retrieval service, default to CPU cores.", description="Number of processes for the retrieval service, default to CPU cores.",
default=os.cpu_count() or 1, default=os.cpu_count() or 1,
@ -244,13 +227,12 @@ class DatabaseConfig(BaseSettings):
connect_args: dict[str, str] = {} connect_args: dict[str, str] = {}
# Use the dynamic SQLALCHEMY_DATABASE_URI_SCHEME property # Use the dynamic SQLALCHEMY_DATABASE_URI_SCHEME property
if self.SQLALCHEMY_DATABASE_URI_SCHEME.startswith("postgresql"): if self.SQLALCHEMY_DATABASE_URI_SCHEME.startswith("postgresql"):
merged_options = options.strip() timezone_opt = "-c timezone=UTC"
session_timezone_override = self.DB_SESSION_TIMEZONE_OVERRIDE.strip() if options:
if session_timezone_override: merged_options = f"{options} {timezone_opt}"
timezone_opt = f"-c timezone={session_timezone_override}" else:
merged_options = f"{merged_options} {timezone_opt}".strip() if merged_options else timezone_opt merged_options = timezone_opt
if merged_options: connect_args = {"options": merged_options}
connect_args = {"options": merged_options}
result: SQLAlchemyEngineOptionsDict = { result: SQLAlchemyEngineOptionsDict = {
"pool_size": self.SQLALCHEMY_POOL_SIZE, "pool_size": self.SQLALCHEMY_POOL_SIZE,
@ -259,7 +241,7 @@ class DatabaseConfig(BaseSettings):
"pool_pre_ping": self.SQLALCHEMY_POOL_PRE_PING, "pool_pre_ping": self.SQLALCHEMY_POOL_PRE_PING,
"connect_args": connect_args, "connect_args": connect_args,
"pool_use_lifo": self.SQLALCHEMY_POOL_USE_LIFO, "pool_use_lifo": self.SQLALCHEMY_POOL_USE_LIFO,
"pool_reset_on_return": self.SQLALCHEMY_POOL_RESET_ON_RETURN, "pool_reset_on_return": None,
"pool_timeout": self.SQLALCHEMY_POOL_TIMEOUT, "pool_timeout": self.SQLALCHEMY_POOL_TIMEOUT,
} }
return result return result

View File

@ -32,11 +32,6 @@ class RedisConfig(BaseSettings):
default=0, default=0,
) )
REDIS_KEY_PREFIX: str = Field(
description="Optional global prefix for Redis keys, topics, and transport artifacts",
default="",
)
REDIS_USE_SSL: bool = Field( REDIS_USE_SSL: bool = Field(
description="Enable SSL/TLS for the Redis connection", description="Enable SSL/TLS for the Redis connection",
default=False, default=False,

View File

@ -1,3 +1,4 @@
from holo_search_sdk.types import BaseQuantizationType, DistanceType, TokenizerType
from pydantic import Field from pydantic import Field
from pydantic_settings import BaseSettings from pydantic_settings import BaseSettings
@ -41,17 +42,17 @@ class HologresConfig(BaseSettings):
default="public", default="public",
) )
HOLOGRES_TOKENIZER: str = Field( HOLOGRES_TOKENIZER: TokenizerType = Field(
description="Tokenizer for full-text search index (e.g., 'jieba', 'ik', 'standard', 'simple').", description="Tokenizer for full-text search index (e.g., 'jieba', 'ik', 'standard', 'simple').",
default="jieba", default="jieba",
) )
HOLOGRES_DISTANCE_METHOD: str = Field( HOLOGRES_DISTANCE_METHOD: DistanceType = Field(
description="Distance method for vector index (e.g., 'Cosine', 'Euclidean', 'InnerProduct').", description="Distance method for vector index (e.g., 'Cosine', 'Euclidean', 'InnerProduct').",
default="Cosine", default="Cosine",
) )
HOLOGRES_BASE_QUANTIZATION_TYPE: str = Field( HOLOGRES_BASE_QUANTIZATION_TYPE: BaseQuantizationType = Field(
description="Base quantization type for vector index (e.g., 'rabitq', 'sq8', 'fp16', 'fp32').", description="Base quantization type for vector index (e.g., 'rabitq', 'sq8', 'fp16', 'fp32').",
default="rabitq", default="rabitq",
) )

View File

@ -1,7 +1,5 @@
"""Configuration for InterSystems IRIS vector database.""" """Configuration for InterSystems IRIS vector database."""
from typing import Any
from pydantic import Field, PositiveInt, model_validator from pydantic import Field, PositiveInt, model_validator
from pydantic_settings import BaseSettings from pydantic_settings import BaseSettings
@ -66,7 +64,7 @@ class IrisVectorConfig(BaseSettings):
@model_validator(mode="before") @model_validator(mode="before")
@classmethod @classmethod
def validate_config(cls, values: dict[str, Any]) -> dict[str, Any]: def validate_config(cls, values: dict) -> dict:
"""Validate IRIS configuration values. """Validate IRIS configuration values.
Args: Args:

View File

@ -1,38 +0,0 @@
"""SECRET_KEY persistence helpers for runtime setup."""
from __future__ import annotations
import secrets
from extensions.ext_storage import storage
GENERATED_SECRET_KEY_FILENAME = ".dify_secret_key"
def resolve_secret_key(secret_key: str) -> str:
"""Return an explicit SECRET_KEY or a generated key persisted in storage."""
if secret_key:
return secret_key
return _load_or_create_secret_key()
def _load_or_create_secret_key() -> str:
try:
persisted_key = storage.load_once(GENERATED_SECRET_KEY_FILENAME).decode("utf-8").strip()
if persisted_key:
return persisted_key
except FileNotFoundError:
pass
generated_key = secrets.token_urlsafe(48)
try:
storage.save(GENERATED_SECRET_KEY_FILENAME, f"{generated_key}\n".encode())
except Exception as exc:
raise ValueError(
f"SECRET_KEY is not set and could not be generated at {GENERATED_SECRET_KEY_FILENAME}. "
"Set SECRET_KEY explicitly or make storage writable."
) from exc
return generated_key

View File

@ -1 +0,0 @@
CURRENT_APP_DSL_VERSION = "0.6.0"

View File

@ -81,4 +81,20 @@ default_app_templates: Mapping[AppMode, Mapping] = {
}, },
}, },
}, },
# agent default mode (new agent backed by single-node workflow)
AppMode.AGENT: {
"app": {
"mode": AppMode.AGENT,
"enable_site": True,
"enable_api": True,
},
"model_config": {
"model": {
"provider": "openai",
"name": "gpt-4o",
"mode": "chat",
"completion_params": {},
},
},
},
} }

View File

@ -19,7 +19,7 @@
"name": "Website Generator" "name": "Website Generator"
}, },
"app_id": "b53545b1-79ea-4da3-b31a-c39391c6f041", "app_id": "b53545b1-79ea-4da3-b31a-c39391c6f041",
"categories": ["Programming"], "category": "Programming",
"copyright": null, "copyright": null,
"description": null, "description": null,
"is_listed": true, "is_listed": true,
@ -35,7 +35,7 @@
"name": "Investment Analysis Report Copilot" "name": "Investment Analysis Report Copilot"
}, },
"app_id": "a23b57fa-85da-49c0-a571-3aff375976c1", "app_id": "a23b57fa-85da-49c0-a571-3aff375976c1",
"categories": ["Agent"], "category": "Agent",
"copyright": "Dify.AI", "copyright": "Dify.AI",
"description": "Welcome to your personalized Investment Analysis Copilot service, where we delve into the depths of stock analysis to provide you with comprehensive insights. \n", "description": "Welcome to your personalized Investment Analysis Copilot service, where we delve into the depths of stock analysis to provide you with comprehensive insights. \n",
"is_listed": true, "is_listed": true,
@ -51,7 +51,7 @@
"name": "Workflow Planning Assistant " "name": "Workflow Planning Assistant "
}, },
"app_id": "f3303a7d-a81c-404e-b401-1f8711c998c1", "app_id": "f3303a7d-a81c-404e-b401-1f8711c998c1",
"categories": ["Workflow"], "category": "Workflow",
"copyright": null, "copyright": null,
"description": "An assistant that helps you plan and select the right node for a workflow (V0.6.0). ", "description": "An assistant that helps you plan and select the right node for a workflow (V0.6.0). ",
"is_listed": true, "is_listed": true,
@ -67,7 +67,7 @@
"name": "Automated Email Reply " "name": "Automated Email Reply "
}, },
"app_id": "e9d92058-7d20-4904-892f-75d90bef7587", "app_id": "e9d92058-7d20-4904-892f-75d90bef7587",
"categories": ["Workflow"], "category": "Workflow",
"copyright": null, "copyright": null,
"description": "Reply emails using Gmail API. It will automatically retrieve email in your inbox and create a response in Gmail. \nConfigure your Gmail API in Google Cloud Console. ", "description": "Reply emails using Gmail API. It will automatically retrieve email in your inbox and create a response in Gmail. \nConfigure your Gmail API in Google Cloud Console. ",
"is_listed": true, "is_listed": true,
@ -83,7 +83,7 @@
"name": "Book Translation " "name": "Book Translation "
}, },
"app_id": "98b87f88-bd22-4d86-8b74-86beba5e0ed4", "app_id": "98b87f88-bd22-4d86-8b74-86beba5e0ed4",
"categories": ["Workflow"], "category": "Workflow",
"copyright": null, "copyright": null,
"description": "A workflow designed to translate a full book up to 15000 tokens per run. Uses Code node to separate text into chunks and Iteration to translate each chunk. ", "description": "A workflow designed to translate a full book up to 15000 tokens per run. Uses Code node to separate text into chunks and Iteration to translate each chunk. ",
"is_listed": true, "is_listed": true,
@ -99,7 +99,7 @@
"name": "Python bug fixer" "name": "Python bug fixer"
}, },
"app_id": "cae337e6-aec5-4c7b-beca-d6f1a808bd5e", "app_id": "cae337e6-aec5-4c7b-beca-d6f1a808bd5e",
"categories": ["Programming"], "category": "Programming",
"copyright": null, "copyright": null,
"description": null, "description": null,
"is_listed": true, "is_listed": true,
@ -115,7 +115,7 @@
"name": "Code Interpreter" "name": "Code Interpreter"
}, },
"app_id": "d077d587-b072-4f2c-b631-69ed1e7cdc0f", "app_id": "d077d587-b072-4f2c-b631-69ed1e7cdc0f",
"categories": ["Programming"], "category": "Programming",
"copyright": "Copyright 2023 Dify", "copyright": "Copyright 2023 Dify",
"description": "Code interpreter, clarifying the syntax and semantics of the code.", "description": "Code interpreter, clarifying the syntax and semantics of the code.",
"is_listed": true, "is_listed": true,
@ -131,7 +131,7 @@
"name": "SVG Logo Design " "name": "SVG Logo Design "
}, },
"app_id": "73fbb5f1-c15d-4d74-9cc8-46d9db9b2cca", "app_id": "73fbb5f1-c15d-4d74-9cc8-46d9db9b2cca",
"categories": ["Agent"], "category": "Agent",
"copyright": "Dify.AI", "copyright": "Dify.AI",
"description": "Hello, I am your creative partner in bringing ideas to vivid life! I can assist you in creating stunning designs by leveraging abilities of DALL·E 3. ", "description": "Hello, I am your creative partner in bringing ideas to vivid life! I can assist you in creating stunning designs by leveraging abilities of DALL·E 3. ",
"is_listed": true, "is_listed": true,
@ -147,7 +147,7 @@
"name": "Long Story Generator (Iteration) " "name": "Long Story Generator (Iteration) "
}, },
"app_id": "5efb98d7-176b-419c-b6ef-50767391ab62", "app_id": "5efb98d7-176b-419c-b6ef-50767391ab62",
"categories": ["Workflow"], "category": "Workflow",
"copyright": null, "copyright": null,
"description": "A workflow demonstrating how to use Iteration node to generate long article that is longer than the context length of LLMs. ", "description": "A workflow demonstrating how to use Iteration node to generate long article that is longer than the context length of LLMs. ",
"is_listed": true, "is_listed": true,
@ -163,7 +163,7 @@
"name": "Text Summarization Workflow" "name": "Text Summarization Workflow"
}, },
"app_id": "f00c4531-6551-45ee-808f-1d7903099515", "app_id": "f00c4531-6551-45ee-808f-1d7903099515",
"categories": ["Workflow"], "category": "Workflow",
"copyright": null, "copyright": null,
"description": "Based on users' choice, retrieve external knowledge to more accurately summarize articles.", "description": "Based on users' choice, retrieve external knowledge to more accurately summarize articles.",
"is_listed": true, "is_listed": true,
@ -179,7 +179,7 @@
"name": "YouTube Channel Data Analysis" "name": "YouTube Channel Data Analysis"
}, },
"app_id": "be591209-2ca8-410f-8f3b-ca0e530dd638", "app_id": "be591209-2ca8-410f-8f3b-ca0e530dd638",
"categories": ["Agent"], "category": "Agent",
"copyright": "Dify.AI", "copyright": "Dify.AI",
"description": "I am a YouTube Channel Data Analysis Copilot, I am here to provide expert data analysis tailored to your needs. ", "description": "I am a YouTube Channel Data Analysis Copilot, I am here to provide expert data analysis tailored to your needs. ",
"is_listed": true, "is_listed": true,
@ -195,7 +195,7 @@
"name": "Article Grading Bot" "name": "Article Grading Bot"
}, },
"app_id": "a747f7b4-c48b-40d6-b313-5e628232c05f", "app_id": "a747f7b4-c48b-40d6-b313-5e628232c05f",
"categories": ["Writing"], "category": "Writing",
"copyright": null, "copyright": null,
"description": "Assess the quality of articles and text based on user defined criteria. ", "description": "Assess the quality of articles and text based on user defined criteria. ",
"is_listed": true, "is_listed": true,
@ -211,7 +211,7 @@
"name": "SEO Blog Generator" "name": "SEO Blog Generator"
}, },
"app_id": "18f3bd03-524d-4d7a-8374-b30dbe7c69d5", "app_id": "18f3bd03-524d-4d7a-8374-b30dbe7c69d5",
"categories": ["Workflow"], "category": "Workflow",
"copyright": null, "copyright": null,
"description": "Workflow for retrieving information from the internet, followed by segmented generation of SEO blogs.", "description": "Workflow for retrieving information from the internet, followed by segmented generation of SEO blogs.",
"is_listed": true, "is_listed": true,
@ -227,7 +227,7 @@
"name": "SQL Creator" "name": "SQL Creator"
}, },
"app_id": "050ef42e-3e0c-40c1-a6b6-a64f2c49d744", "app_id": "050ef42e-3e0c-40c1-a6b6-a64f2c49d744",
"categories": ["Programming"], "category": "Programming",
"copyright": "Copyright 2023 Dify", "copyright": "Copyright 2023 Dify",
"description": "Write SQL from natural language by pasting in your schema with the request.Please describe your query requirements in natural language and select the target database type.", "description": "Write SQL from natural language by pasting in your schema with the request.Please describe your query requirements in natural language and select the target database type.",
"is_listed": true, "is_listed": true,
@ -243,7 +243,7 @@
"name": "Sentiment Analysis " "name": "Sentiment Analysis "
}, },
"app_id": "f06bf86b-d50c-4895-a942-35112dbe4189", "app_id": "f06bf86b-d50c-4895-a942-35112dbe4189",
"categories": ["Workflow"], "category": "Workflow",
"copyright": null, "copyright": null,
"description": "Batch sentiment analysis of text, followed by JSON output of sentiment classification along with scores.", "description": "Batch sentiment analysis of text, followed by JSON output of sentiment classification along with scores.",
"is_listed": true, "is_listed": true,
@ -259,7 +259,7 @@
"name": "Strategic Consulting Expert" "name": "Strategic Consulting Expert"
}, },
"app_id": "7e8ca1ae-02f2-4b5f-979e-62d19133bee2", "app_id": "7e8ca1ae-02f2-4b5f-979e-62d19133bee2",
"categories": ["Assistant"], "category": "Assistant",
"copyright": "Copyright 2023 Dify", "copyright": "Copyright 2023 Dify",
"description": "I can answer your questions related to strategic marketing.", "description": "I can answer your questions related to strategic marketing.",
"is_listed": true, "is_listed": true,
@ -275,7 +275,7 @@
"name": "Code Converter" "name": "Code Converter"
}, },
"app_id": "4006c4b2-0735-4f37-8dbb-fb1a8c5bd87a", "app_id": "4006c4b2-0735-4f37-8dbb-fb1a8c5bd87a",
"categories": ["Programming"], "category": "Programming",
"copyright": "Copyright 2023 Dify", "copyright": "Copyright 2023 Dify",
"description": "This is an application that provides the ability to convert code snippets in multiple programming languages. You can input the code you wish to convert, select the target programming language, and get the desired output.", "description": "This is an application that provides the ability to convert code snippets in multiple programming languages. You can input the code you wish to convert, select the target programming language, and get the desired output.",
"is_listed": true, "is_listed": true,
@ -291,7 +291,7 @@
"name": "Question Classifier + Knowledge + Chatbot " "name": "Question Classifier + Knowledge + Chatbot "
}, },
"app_id": "d9f6b733-e35d-4a40-9f38-ca7bbfa009f7", "app_id": "d9f6b733-e35d-4a40-9f38-ca7bbfa009f7",
"categories": ["Workflow"], "category": "Workflow",
"copyright": null, "copyright": null,
"description": "Basic Workflow Template, a chatbot capable of identifying intents alongside with a knowledge base.", "description": "Basic Workflow Template, a chatbot capable of identifying intents alongside with a knowledge base.",
"is_listed": true, "is_listed": true,
@ -307,7 +307,7 @@
"name": "AI Front-end interviewer" "name": "AI Front-end interviewer"
}, },
"app_id": "127efead-8944-4e20-ba9d-12402eb345e0", "app_id": "127efead-8944-4e20-ba9d-12402eb345e0",
"categories": ["HR"], "category": "HR",
"copyright": "Copyright 2023 Dify", "copyright": "Copyright 2023 Dify",
"description": "A simulated front-end interviewer that tests the skill level of front-end development through questioning.", "description": "A simulated front-end interviewer that tests the skill level of front-end development through questioning.",
"is_listed": true, "is_listed": true,
@ -323,7 +323,7 @@
"name": "Knowledge Retrieval + Chatbot " "name": "Knowledge Retrieval + Chatbot "
}, },
"app_id": "e9870913-dd01-4710-9f06-15d4180ca1ce", "app_id": "e9870913-dd01-4710-9f06-15d4180ca1ce",
"categories": ["Workflow"], "category": "Workflow",
"copyright": null, "copyright": null,
"description": "Basic Workflow Template, A chatbot with a knowledge base. ", "description": "Basic Workflow Template, A chatbot with a knowledge base. ",
"is_listed": true, "is_listed": true,
@ -339,7 +339,7 @@
"name": "Email Assistant Workflow " "name": "Email Assistant Workflow "
}, },
"app_id": "dd5b6353-ae9b-4bce-be6a-a681a12cf709", "app_id": "dd5b6353-ae9b-4bce-be6a-a681a12cf709",
"categories": ["Workflow"], "category": "Workflow",
"copyright": null, "copyright": null,
"description": "A multifunctional email assistant capable of summarizing, replying, composing, proofreading, and checking grammar.", "description": "A multifunctional email assistant capable of summarizing, replying, composing, proofreading, and checking grammar.",
"is_listed": true, "is_listed": true,
@ -355,7 +355,7 @@
"name": "Customer Review Analysis Workflow " "name": "Customer Review Analysis Workflow "
}, },
"app_id": "9c0cd31f-4b62-4005-adf5-e3888d08654a", "app_id": "9c0cd31f-4b62-4005-adf5-e3888d08654a",
"categories": ["Workflow"], "category": "Workflow",
"copyright": null, "copyright": null,
"description": "Utilize LLM (Large Language Models) to classify customer reviews and forward them to the internal system.", "description": "Utilize LLM (Large Language Models) to classify customer reviews and forward them to the internal system.",
"is_listed": true, "is_listed": true,

View File

@ -1,193 +0,0 @@
# API Schema Guide
This guide describes the expected Flask-RESTX + Pydantic pattern for controller request payloads, query
parameters, response schemas, and Swagger documentation.
## Principles
- Use Pydantic `BaseModel` for request bodies and query parameters.
- Use `fields.base.ResponseModel` for response DTOs.
- Keep runtime validation and Swagger documentation wired to the same Pydantic model.
- Prefer explicit validation and serialization in controller methods over Flask-RESTX marshalling.
- Do not add new Flask-RESTX `fields.*` dictionaries, `Namespace.model(...)` exports, or `@marshal_with(...)` for migrated or new endpoints.
- Do not use `@ns.expect(...)` for GET query parameters. Flask-RESTX documents that as a request body.
## Naming
- Request body models: use a `Payload` suffix.
- Example: `WorkflowRunPayload`, `DatasourceVariablesPayload`.
- Query parameter models: use a `Query` suffix.
- Example: `WorkflowRunListQuery`, `MessageListQuery`.
- Response models: use a `Response` suffix and inherit from `ResponseModel`.
- Example: `WorkflowRunDetailResponse`, `WorkflowRunNodeExecutionListResponse`.
- Use `ListResponse` or `PaginationResponse` for wrapper responses.
- Example: `WorkflowRunNodeExecutionListResponse`, `WorkflowRunPaginationResponse`.
- Keep these models near the controller when they are endpoint-specific. Move them to `fields/*_fields.py` only when shared by multiple controllers.
## Registering Models For Swagger
Use helpers from `controllers.common.schema`.
```python
from controllers.common.schema import (
query_params_from_model,
register_response_schema_models,
register_schema_models,
)
```
Register request payload and query models with `register_schema_models(...)`:
```python
register_schema_models(
console_ns,
WorkflowRunPayload,
WorkflowRunListQuery,
)
```
Register response models with `register_response_schema_models(...)`:
```python
register_response_schema_models(
console_ns,
WorkflowRunDetailResponse,
WorkflowRunPaginationResponse,
)
```
Response models are registered in Pydantic serialization mode. This matters when a response model uses
`validation_alias` to read internal object attributes but emits public API field names. For example, a response model
can validate from `inputs_dict` while documenting and serializing `inputs`.
## Request Bodies
For non-GET request bodies:
1. Define a Pydantic `Payload` model.
2. Register it with `register_schema_models(...)`.
3. Use `@ns.expect(ns.models[Payload.__name__])` for Swagger documentation.
4. Validate from `ns.payload or {}` inside the controller.
```python
class DraftWorkflowNodeRunPayload(BaseModel):
inputs: dict[str, Any]
query: str = ""
register_schema_models(console_ns, DraftWorkflowNodeRunPayload)
@console_ns.expect(console_ns.models[DraftWorkflowNodeRunPayload.__name__])
def post(self, app_model: App, node_id: str):
payload = DraftWorkflowNodeRunPayload.model_validate(console_ns.payload or {})
result = service.run(..., inputs=payload.inputs, query=payload.query)
return WorkflowRunNodeExecutionResponse.model_validate(result, from_attributes=True).model_dump(mode="json")
```
## Query Parameters
For GET query parameters:
1. Define a Pydantic `Query` model.
2. Register it with `register_schema_models(...)` if it is referenced elsewhere in docs, or only use
`query_params_from_model(...)` if a body schema is not needed.
3. Use `@ns.doc(params=query_params_from_model(QueryModel))`.
4. Validate from `request.args.to_dict(flat=True)` or an explicit dict when type coercion is needed.
```python
class WorkflowRunListQuery(BaseModel):
last_id: str | None = Field(default=None, description="Last run ID for pagination")
limit: int = Field(default=20, ge=1, le=100, description="Number of items per page (1-100)")
@console_ns.doc(params=query_params_from_model(WorkflowRunListQuery))
def get(self, app_model: App):
query = WorkflowRunListQuery.model_validate(request.args.to_dict(flat=True))
result = service.list(..., limit=query.limit, last_id=query.last_id)
return WorkflowRunPaginationResponse.model_validate(result, from_attributes=True).model_dump(mode="json")
```
Do not do this for GET query parameters:
```python
@console_ns.expect(console_ns.models[WorkflowRunListQuery.__name__])
def get(...):
...
```
That documents a GET request body and is not the expected contract.
## Responses
Response models should inherit from `ResponseModel`:
```python
class WorkflowRunNodeExecutionResponse(ResponseModel):
id: str
inputs: Any = Field(default=None, validation_alias="inputs_dict")
process_data: Any = Field(default=None, validation_alias="process_data_dict")
outputs: Any = Field(default=None, validation_alias="outputs_dict")
```
Document response models with `@ns.response(...)`:
```python
@console_ns.response(
200,
"Node run started successfully",
console_ns.models[WorkflowRunNodeExecutionResponse.__name__],
)
def post(...):
...
```
Serialize explicitly:
```python
return WorkflowRunNodeExecutionResponse.model_validate(
workflow_node_execution,
from_attributes=True,
).model_dump(mode="json")
```
If the service can return `None`, translate that into the expected HTTP error before validation:
```python
workflow_run = service.get_workflow_run(...)
if workflow_run is None:
raise NotFound("Workflow run not found")
return WorkflowRunDetailResponse.model_validate(workflow_run, from_attributes=True).model_dump(mode="json")
```
## Legacy Flask-RESTX Patterns
Avoid adding these patterns to new or migrated endpoints:
- `ns.model(...)` for new request/response DTOs.
- Module-level exported RESTX model objects such as `workflow_run_detail_model`.
- `fields.Nested({...})` with raw inline dict field maps.
- `@marshal_with(...)` for response serialization.
- `@ns.expect(...)` for GET query params.
Existing legacy field dictionaries may remain where an endpoint has not yet been migrated. Keep that compatibility local
to the legacy area and avoid importing RESTX model objects from controllers.
## Verifying Swagger
For schema and documentation changes, run focused tests and generate Swagger JSON:
```bash
uv run --project . pytest tests/unit_tests/controllers/common/test_schema.py
uv run --project . pytest tests/unit_tests/commands/test_generate_swagger_specs.py tests/unit_tests/controllers/test_swagger.py
uv run --project . dev/generate_swagger_specs.py --output-dir /tmp/dify-openapi-check
```
Inspect affected endpoints with `jq`. Check that:
- GET parameters are `in: query`.
- Request bodies appear only where the endpoint has a body.
- Responses reference the expected `*Response` schema.
- Response schemas use public serialized names, not internal validation aliases like `inputs_dict`.

View File

@ -2,9 +2,9 @@ from __future__ import annotations
from typing import Any from typing import Any
from graphon.file import helpers as file_helpers
from pydantic import BaseModel, ConfigDict, computed_field from pydantic import BaseModel, ConfigDict, computed_field
from graphon.file import helpers as file_helpers
from models.model import IconType from models.model import IconType
type JSONValue = str | int | float | bool | None | dict[str, Any] | list[Any] type JSONValue = str | int | float | bool | None | dict[str, Any] | list[Any]

View File

@ -41,8 +41,7 @@ def guess_file_info_from_response(response: httpx.Response):
# Try to extract filename from URL # Try to extract filename from URL
parsed_url = urllib.parse.urlparse(url) parsed_url = urllib.parse.urlparse(url)
url_path = parsed_url.path url_path = parsed_url.path
# Decode percent-encoded characters in the path segment filename = os.path.basename(url_path)
filename = urllib.parse.unquote(os.path.basename(url_path))
# If filename couldn't be extracted, use Content-Disposition header # If filename couldn't be extracted, use Content-Disposition header
if not filename: if not filename:

View File

@ -1,6 +0,0 @@
from pydantic import BaseModel, JsonValue
class HumanInputFormSubmitPayload(BaseModel):
inputs: dict[str, JsonValue]
action: str

View File

@ -1,14 +1,6 @@
"""Helpers for registering Pydantic models with Flask-RESTX namespaces. """Helpers for registering Pydantic models with Flask-RESTX namespaces."""
Flask-RESTX treats `SchemaModel` bodies as opaque JSON schemas; it does not
promote Pydantic's nested `$defs` into top-level Swagger `definitions`.
These helpers keep that translation centralized so models registered through
`register_schema_models` emit resolvable Swagger 2.0 references.
"""
from collections.abc import Mapping
from enum import StrEnum from enum import StrEnum
from typing import Any, Literal, NotRequired, TypedDict
from flask_restx import Namespace from flask_restx import Namespace
from pydantic import BaseModel, TypeAdapter from pydantic import BaseModel, TypeAdapter
@ -16,59 +8,10 @@ from pydantic import BaseModel, TypeAdapter
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}" DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
QueryParamDoc = TypedDict(
"QueryParamDoc",
{
"in": NotRequired[str],
"type": NotRequired[str],
"items": NotRequired[dict[str, object]],
"required": NotRequired[bool],
"description": NotRequired[str],
"enum": NotRequired[list[object]],
"default": NotRequired[object],
"minimum": NotRequired[int | float],
"maximum": NotRequired[int | float],
"minLength": NotRequired[int],
"maxLength": NotRequired[int],
"minItems": NotRequired[int],
"maxItems": NotRequired[int],
},
)
def _register_json_schema(namespace: Namespace, name: str, schema: dict) -> None:
"""Register a JSON schema and promote any nested Pydantic `$defs`."""
nested_definitions = schema.get("$defs")
schema_to_register = dict(schema)
if isinstance(nested_definitions, dict):
schema_to_register.pop("$defs")
namespace.schema_model(name, schema_to_register)
if not isinstance(nested_definitions, dict):
return
for nested_name, nested_schema in nested_definitions.items():
if isinstance(nested_schema, dict):
_register_json_schema(namespace, nested_name, nested_schema)
JsonSchemaMode = Literal["validation", "serialization"]
def _register_schema_model(namespace: Namespace, model: type[BaseModel], *, mode: JsonSchemaMode) -> None:
_register_json_schema(
namespace,
model.__name__,
model.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0, mode=mode),
)
def register_schema_model(namespace: Namespace, model: type[BaseModel]) -> None: def register_schema_model(namespace: Namespace, model: type[BaseModel]) -> None:
"""Register a BaseModel and its nested schema definitions for Swagger documentation.""" """Register a single BaseModel with a namespace for Swagger documentation."""
_register_schema_model(namespace, model, mode="validation") namespace.schema_model(model.__name__, model.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0))
def register_schema_models(namespace: Namespace, *models: type[BaseModel]) -> None: def register_schema_models(namespace: Namespace, *models: type[BaseModel]) -> None:
@ -78,19 +21,6 @@ def register_schema_models(namespace: Namespace, *models: type[BaseModel]) -> No
register_schema_model(namespace, model) register_schema_model(namespace, model)
def register_response_schema_model(namespace: Namespace, model: type[BaseModel]) -> None:
"""Register a BaseModel using its serialized response shape."""
_register_schema_model(namespace, model, mode="serialization")
def register_response_schema_models(namespace: Namespace, *models: type[BaseModel]) -> None:
"""Register multiple response BaseModels using their serialized response shape."""
for model in models:
register_response_schema_model(namespace, model)
def get_or_create_model(model_name: str, field_def): def get_or_create_model(model_name: str, field_def):
# Import lazily to avoid circular imports between console controllers and schema helpers. # Import lazily to avoid circular imports between console controllers and schema helpers.
from controllers.console import console_ns from controllers.console import console_ns
@ -104,114 +34,15 @@ def get_or_create_model(model_name: str, field_def):
def register_enum_models(namespace: Namespace, *models: type[StrEnum]) -> None: def register_enum_models(namespace: Namespace, *models: type[StrEnum]) -> None:
"""Register multiple StrEnum with a namespace.""" """Register multiple StrEnum with a namespace."""
for model in models: for model in models:
_register_json_schema( namespace.schema_model(
namespace, model.__name__, TypeAdapter(model).json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0)
model.__name__,
TypeAdapter(model).json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
) )
def query_params_from_model(model: type[BaseModel]) -> dict[str, QueryParamDoc]:
"""Build Flask-RESTX query parameter docs from a flat Pydantic model.
`Namespace.expect()` treats Pydantic schema models as request bodies, so GET
endpoints should keep runtime validation on the Pydantic model and feed this
derived mapping to `Namespace.doc(params=...)` for Swagger documentation.
"""
schema = model.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0)
properties = schema.get("properties", {})
if not isinstance(properties, Mapping):
return {}
required = schema.get("required", [])
required_names = set(required) if isinstance(required, list) else set()
params: dict[str, QueryParamDoc] = {}
for name, property_schema in properties.items():
if not isinstance(name, str) or not isinstance(property_schema, Mapping):
continue
params[name] = _query_param_from_property(property_schema, required=name in required_names)
return params
def _query_param_from_property(property_schema: Mapping[str, Any], *, required: bool) -> QueryParamDoc:
param_schema = _nullable_property_schema(property_schema)
param_doc: QueryParamDoc = {"in": "query", "required": required}
description = param_schema.get("description")
if isinstance(description, str):
param_doc["description"] = description
schema_type = param_schema.get("type")
if isinstance(schema_type, str) and schema_type in {"array", "boolean", "integer", "number", "string"}:
param_doc["type"] = schema_type
if schema_type == "array":
items = param_schema.get("items")
if isinstance(items, Mapping):
item_type = items.get("type")
if isinstance(item_type, str):
param_doc["items"] = {"type": item_type}
enum = param_schema.get("enum")
if isinstance(enum, list):
param_doc["enum"] = enum
default = param_schema.get("default")
if default is not None:
param_doc["default"] = default
minimum = param_schema.get("minimum")
if isinstance(minimum, int | float):
param_doc["minimum"] = minimum
maximum = param_schema.get("maximum")
if isinstance(maximum, int | float):
param_doc["maximum"] = maximum
min_length = param_schema.get("minLength")
if isinstance(min_length, int):
param_doc["minLength"] = min_length
max_length = param_schema.get("maxLength")
if isinstance(max_length, int):
param_doc["maxLength"] = max_length
min_items = param_schema.get("minItems")
if isinstance(min_items, int):
param_doc["minItems"] = min_items
max_items = param_schema.get("maxItems")
if isinstance(max_items, int):
param_doc["maxItems"] = max_items
return param_doc
def _nullable_property_schema(property_schema: Mapping[str, Any]) -> Mapping[str, Any]:
any_of = property_schema.get("anyOf")
if not isinstance(any_of, list):
return property_schema
non_null_candidates = [
candidate for candidate in any_of if isinstance(candidate, Mapping) and candidate.get("type") != "null"
]
if len(non_null_candidates) == 1:
return {**property_schema, **non_null_candidates[0]}
return property_schema
__all__ = [ __all__ = [
"DEFAULT_REF_TEMPLATE_SWAGGER_2_0", "DEFAULT_REF_TEMPLATE_SWAGGER_2_0",
"get_or_create_model", "get_or_create_model",
"query_params_from_model",
"register_enum_models", "register_enum_models",
"register_response_schema_model",
"register_response_schema_models",
"register_schema_model", "register_schema_model",
"register_schema_models", "register_schema_models",
] ]

View File

@ -33,6 +33,7 @@ for module_name in RESOURCE_MODULES:
# Ensure resource modules are imported so route decorators are evaluated. # Ensure resource modules are imported so route decorators are evaluated.
# Import other controllers # Import other controllers
from . import ( from . import (
admin,
apikey, apikey,
extension, extension,
feature, feature,
@ -64,7 +65,6 @@ from .app import (
statistic, statistic,
workflow, workflow,
workflow_app_log, workflow_app_log,
workflow_comment,
workflow_draft_variable, workflow_draft_variable,
workflow_run, workflow_run,
workflow_statistic, workflow_statistic,
@ -116,7 +116,6 @@ from .explore import (
saved_message, saved_message,
trial, trial,
) )
from .socketio import workflow as socketio_workflow
# Import tag controllers # Import tag controllers
from .tag import tags from .tag import tags
@ -141,6 +140,7 @@ api.add_namespace(console_ns)
__all__ = [ __all__ = [
"account", "account",
"activate", "activate",
"admin",
"advanced_prompt_template", "advanced_prompt_template",
"agent", "agent",
"agent_providers", "agent_providers",
@ -201,7 +201,6 @@ __all__ = [
"saved_message", "saved_message",
"setup", "setup",
"site", "site",
"socketio_workflow",
"spec", "spec",
"statistic", "statistic",
"tags", "tags",
@ -212,7 +211,6 @@ __all__ = [
"website", "website",
"workflow", "workflow",
"workflow_app_log", "workflow_app_log",
"workflow_comment",
"workflow_draft_variable", "workflow_draft_variable",
"workflow_run", "workflow_run",
"workflow_statistic", "workflow_statistic",

View File

@ -1,11 +1,72 @@
import csv
import io
from collections.abc import Callable from collections.abc import Callable
from functools import wraps from functools import wraps
from typing import cast
from flask import request from flask import request
from werkzeug.exceptions import Unauthorized from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from sqlalchemy import select
from werkzeug.exceptions import BadRequest, NotFound, Unauthorized
from configs import dify_config from configs import dify_config
from constants.languages import supported_language
from controllers.console import console_ns
from controllers.console.wraps import only_edition_cloud
from core.db.session_factory import session_factory
from extensions.ext_database import db
from libs.token import extract_access_token from libs.token import extract_access_token
from models.model import App, ExporleBanner, InstalledApp, RecommendedApp, TrialApp
from services.billing_service import BillingService, LangContentDict
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
class InsertExploreAppPayload(BaseModel):
app_id: str = Field(...)
desc: str | None = None
copyright: str | None = None
privacy_policy: str | None = None
custom_disclaimer: str | None = None
language: str = Field(...)
category: str = Field(...)
position: int = Field(...)
can_trial: bool = Field(default=False)
trial_limit: int = Field(default=0)
@field_validator("language")
@classmethod
def validate_language(cls, value: str) -> str:
return supported_language(value)
class InsertExploreBannerPayload(BaseModel):
category: str = Field(...)
title: str = Field(...)
description: str = Field(...)
img_src: str = Field(..., alias="img-src")
language: str = Field(default="en-US")
link: str = Field(...)
sort: int = Field(...)
@field_validator("language")
@classmethod
def validate_language(cls, value: str) -> str:
return supported_language(value)
model_config = {"populate_by_name": True}
console_ns.schema_model(
InsertExploreAppPayload.__name__,
InsertExploreAppPayload.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
)
console_ns.schema_model(
InsertExploreBannerPayload.__name__,
InsertExploreBannerPayload.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
)
def admin_required[**P, R](view: Callable[P, R]) -> Callable[P, R]: def admin_required[**P, R](view: Callable[P, R]) -> Callable[P, R]:
@ -23,3 +84,361 @@ def admin_required[**P, R](view: Callable[P, R]) -> Callable[P, R]:
return view(*args, **kwargs) return view(*args, **kwargs)
return decorated return decorated
@console_ns.route("/admin/insert-explore-apps")
class InsertExploreAppListApi(Resource):
@console_ns.doc("insert_explore_app")
@console_ns.doc(description="Insert or update an app in the explore list")
@console_ns.expect(console_ns.models[InsertExploreAppPayload.__name__])
@console_ns.response(200, "App updated successfully")
@console_ns.response(201, "App inserted successfully")
@console_ns.response(404, "App not found")
@only_edition_cloud
@admin_required
def post(self):
payload = InsertExploreAppPayload.model_validate(console_ns.payload)
app = db.session.execute(select(App).where(App.id == payload.app_id)).scalar_one_or_none()
if not app:
raise NotFound(f"App '{payload.app_id}' is not found")
site = app.site
if not site:
desc = payload.desc or ""
copy_right = payload.copyright or ""
privacy_policy = payload.privacy_policy or ""
custom_disclaimer = payload.custom_disclaimer or ""
else:
desc = site.description or payload.desc or ""
copy_right = site.copyright or payload.copyright or ""
privacy_policy = site.privacy_policy or payload.privacy_policy or ""
custom_disclaimer = site.custom_disclaimer or payload.custom_disclaimer or ""
with session_factory.create_session() as session:
recommended_app = session.execute(
select(RecommendedApp).where(RecommendedApp.app_id == payload.app_id)
).scalar_one_or_none()
if not recommended_app:
recommended_app = RecommendedApp(
app_id=app.id,
description=desc,
copyright=copy_right,
privacy_policy=privacy_policy,
custom_disclaimer=custom_disclaimer,
language=payload.language,
category=payload.category,
position=payload.position,
)
db.session.add(recommended_app)
if payload.can_trial:
trial_app = db.session.execute(
select(TrialApp).where(TrialApp.app_id == payload.app_id)
).scalar_one_or_none()
if not trial_app:
db.session.add(
TrialApp(
app_id=payload.app_id,
tenant_id=app.tenant_id,
trial_limit=payload.trial_limit,
)
)
else:
trial_app.trial_limit = payload.trial_limit
app.is_public = True
db.session.commit()
return {"result": "success"}, 201
else:
recommended_app.description = desc
recommended_app.copyright = copy_right
recommended_app.privacy_policy = privacy_policy
recommended_app.custom_disclaimer = custom_disclaimer
recommended_app.language = payload.language
recommended_app.category = payload.category
recommended_app.position = payload.position
if payload.can_trial:
trial_app = db.session.execute(
select(TrialApp).where(TrialApp.app_id == payload.app_id)
).scalar_one_or_none()
if not trial_app:
db.session.add(
TrialApp(
app_id=payload.app_id,
tenant_id=app.tenant_id,
trial_limit=payload.trial_limit,
)
)
else:
trial_app.trial_limit = payload.trial_limit
app.is_public = True
db.session.commit()
return {"result": "success"}, 200
@console_ns.route("/admin/insert-explore-apps/<uuid:app_id>")
class InsertExploreAppApi(Resource):
@console_ns.doc("delete_explore_app")
@console_ns.doc(description="Remove an app from the explore list")
@console_ns.doc(params={"app_id": "Application ID to remove"})
@console_ns.response(204, "App removed successfully")
@only_edition_cloud
@admin_required
def delete(self, app_id):
with session_factory.create_session() as session:
recommended_app = session.execute(
select(RecommendedApp).where(RecommendedApp.app_id == str(app_id))
).scalar_one_or_none()
if not recommended_app:
return {"result": "success"}, 204
with session_factory.create_session() as session:
app = session.execute(select(App).where(App.id == recommended_app.app_id)).scalar_one_or_none()
if app:
app.is_public = False
with session_factory.create_session() as session:
installed_apps = (
session.execute(
select(InstalledApp).where(
InstalledApp.app_id == recommended_app.app_id,
InstalledApp.tenant_id != InstalledApp.app_owner_tenant_id,
)
)
.scalars()
.all()
)
for installed_app in installed_apps:
session.delete(installed_app)
trial_app = session.execute(
select(TrialApp).where(TrialApp.app_id == recommended_app.app_id)
).scalar_one_or_none()
if trial_app:
session.delete(trial_app)
db.session.delete(recommended_app)
db.session.commit()
return {"result": "success"}, 204
@console_ns.route("/admin/insert-explore-banner")
class InsertExploreBannerApi(Resource):
@console_ns.doc("insert_explore_banner")
@console_ns.doc(description="Insert an explore banner")
@console_ns.expect(console_ns.models[InsertExploreBannerPayload.__name__])
@console_ns.response(201, "Banner inserted successfully")
@only_edition_cloud
@admin_required
def post(self):
payload = InsertExploreBannerPayload.model_validate(console_ns.payload)
banner = ExporleBanner(
content={
"category": payload.category,
"title": payload.title,
"description": payload.description,
"img-src": payload.img_src,
},
link=payload.link,
sort=payload.sort,
language=payload.language,
)
db.session.add(banner)
db.session.commit()
return {"result": "success"}, 201
@console_ns.route("/admin/delete-explore-banner/<uuid:banner_id>")
class DeleteExploreBannerApi(Resource):
@console_ns.doc("delete_explore_banner")
@console_ns.doc(description="Delete an explore banner")
@console_ns.doc(params={"banner_id": "Banner ID to delete"})
@console_ns.response(204, "Banner deleted successfully")
@only_edition_cloud
@admin_required
def delete(self, banner_id):
banner = db.session.execute(select(ExporleBanner).where(ExporleBanner.id == banner_id)).scalar_one_or_none()
if not banner:
raise NotFound(f"Banner '{banner_id}' is not found")
db.session.delete(banner)
db.session.commit()
return {"result": "success"}, 204
class LangContentPayload(BaseModel):
lang: str = Field(..., description="Language tag: 'zh' | 'en' | 'jp'")
title: str = Field(...)
subtitle: str | None = Field(default=None)
body: str = Field(...)
title_pic_url: str | None = Field(default=None)
class UpsertNotificationPayload(BaseModel):
notification_id: str | None = Field(default=None, description="Omit to create; supply UUID to update")
contents: list[LangContentPayload] = Field(..., min_length=1)
start_time: str | None = Field(default=None, description="RFC3339, e.g. 2026-03-01T00:00:00Z")
end_time: str | None = Field(default=None, description="RFC3339, e.g. 2026-03-20T23:59:59Z")
frequency: str = Field(default="once", description="'once' | 'every_page_load'")
status: str = Field(default="active", description="'active' | 'inactive'")
class BatchAddNotificationAccountsPayload(BaseModel):
notification_id: str = Field(...)
user_email: list[str] = Field(..., description="List of account email addresses")
console_ns.schema_model(
UpsertNotificationPayload.__name__,
UpsertNotificationPayload.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
)
console_ns.schema_model(
BatchAddNotificationAccountsPayload.__name__,
BatchAddNotificationAccountsPayload.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
)
@console_ns.route("/admin/upsert_notification")
class UpsertNotificationApi(Resource):
@console_ns.doc("upsert_notification")
@console_ns.doc(
description=(
"Create or update an in-product notification. "
"Supply notification_id to update an existing one; omit it to create a new one. "
"Pass at least one language variant in contents (zh / en / jp)."
)
)
@console_ns.expect(console_ns.models[UpsertNotificationPayload.__name__])
@console_ns.response(200, "Notification upserted successfully")
@only_edition_cloud
@admin_required
def post(self):
payload = UpsertNotificationPayload.model_validate(console_ns.payload)
result = BillingService.upsert_notification(
contents=[cast(LangContentDict, c.model_dump()) for c in payload.contents],
frequency=payload.frequency,
status=payload.status,
notification_id=payload.notification_id,
start_time=payload.start_time,
end_time=payload.end_time,
)
return {"result": "success", "notification_id": result.get("notificationId")}, 200
@console_ns.route("/admin/batch_add_notification_accounts")
class BatchAddNotificationAccountsApi(Resource):
@console_ns.doc("batch_add_notification_accounts")
@console_ns.doc(
description=(
"Register target accounts for a notification by email address. "
'JSON body: {"notification_id": "...", "user_email": ["a@example.com", ...]}. '
"File upload: multipart/form-data with a 'file' field (CSV or TXT, one email per line) "
"plus a 'notification_id' field. "
"Emails that do not match any account are silently skipped."
)
)
@console_ns.response(200, "Accounts added successfully")
@only_edition_cloud
@admin_required
def post(self):
from models.account import Account
if "file" in request.files:
notification_id = request.form.get("notification_id", "").strip()
if not notification_id:
raise BadRequest("notification_id is required.")
emails = self._parse_emails_from_file()
else:
payload = BatchAddNotificationAccountsPayload.model_validate(console_ns.payload)
notification_id = payload.notification_id
emails = payload.user_email
if not emails:
raise BadRequest("No valid email addresses provided.")
# Resolve emails → account IDs in chunks to avoid large IN-clause
account_ids: list[str] = []
chunk_size = 500
for i in range(0, len(emails), chunk_size):
chunk = emails[i : i + chunk_size]
rows = db.session.execute(select(Account.id, Account.email).where(Account.email.in_(chunk))).all()
account_ids.extend(str(row.id) for row in rows)
if not account_ids:
raise BadRequest("None of the provided emails matched an existing account.")
# Send to dify-saas in batches of 1000
total_count = 0
batch_size = 1000
for i in range(0, len(account_ids), batch_size):
batch = account_ids[i : i + batch_size]
result = BillingService.batch_add_notification_accounts(
notification_id=notification_id,
account_ids=batch,
)
total_count += result.get("count", 0)
return {
"result": "success",
"emails_provided": len(emails),
"accounts_matched": len(account_ids),
"count": total_count,
}, 200
@staticmethod
def _parse_emails_from_file() -> list[str]:
"""Parse email addresses from an uploaded CSV or TXT file."""
file = request.files["file"]
if not file.filename:
raise BadRequest("Uploaded file has no filename.")
filename_lower = file.filename.lower()
if not filename_lower.endswith((".csv", ".txt")):
raise BadRequest("Invalid file type. Only CSV (.csv) and TXT (.txt) files are allowed.")
try:
content = file.read().decode("utf-8")
except UnicodeDecodeError:
try:
file.seek(0)
content = file.read().decode("gbk")
except UnicodeDecodeError:
raise BadRequest("Unable to decode the file. Please use UTF-8 or GBK encoding.")
emails: list[str] = []
if filename_lower.endswith(".csv"):
reader = csv.reader(io.StringIO(content))
for row in reader:
for cell in row:
cell = cell.strip()
if cell:
emails.append(cell)
else:
for line in content.splitlines():
line = line.strip()
if line:
emails.append(line)
# Deduplicate while preserving order
seen: set[str] = set()
unique_emails: list[str] = []
for email in emails:
if email.lower() not in seen:
seen.add(email.lower())
unique_emails.append(email)
return unique_emails

View File

@ -5,7 +5,7 @@ from pydantic import BaseModel, Field
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
from libs.login import login_required from libs.login import login_required
from services.advanced_prompt_template_service import AdvancedPromptTemplateArgs, AdvancedPromptTemplateService from services.advanced_prompt_template_service import AdvancedPromptTemplateService
class AdvancedPromptTemplateQuery(BaseModel): class AdvancedPromptTemplateQuery(BaseModel):
@ -34,11 +34,6 @@ class AdvancedPromptTemplateList(Resource):
@login_required @login_required
@account_initialization_required @account_initialization_required
def get(self): def get(self):
args = AdvancedPromptTemplateQuery.model_validate(request.args.to_dict(flat=True)) args = AdvancedPromptTemplateQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
prompt_args: AdvancedPromptTemplateArgs = {
"app_mode": args.app_mode, return AdvancedPromptTemplateService.get_prompt(args.model_dump())
"model_mode": args.model_mode,
"model_name": args.model_name,
"has_context": args.has_context,
}
return AdvancedPromptTemplateService.get_prompt(prompt_args)

View File

@ -2,7 +2,6 @@ from flask import request
from flask_restx import Resource, fields from flask_restx import Resource, fields
from pydantic import BaseModel, Field, field_validator from pydantic import BaseModel, Field, field_validator
from controllers.common.schema import register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
@ -11,6 +10,8 @@ from libs.login import login_required
from models.model import AppMode from models.model import AppMode
from services.agent_service import AgentService from services.agent_service import AgentService
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
class AgentLogQuery(BaseModel): class AgentLogQuery(BaseModel):
message_id: str = Field(..., description="Message UUID") message_id: str = Field(..., description="Message UUID")
@ -22,7 +23,9 @@ class AgentLogQuery(BaseModel):
return uuid_value(value) return uuid_value(value)
register_schema_models(console_ns, AgentLogQuery) console_ns.schema_model(
AgentLogQuery.__name__, AgentLogQuery.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0)
)
@console_ns.route("/apps/<uuid:app_id>/agent/logs") @console_ns.route("/apps/<uuid:app_id>/agent/logs")
@ -41,6 +44,6 @@ class AgentLogApi(Resource):
@get_app_model(mode=[AppMode.AGENT_CHAT]) @get_app_model(mode=[AppMode.AGENT_CHAT])
def get(self, app_model): def get(self, app_model):
"""Get agent logs""" """Get agent logs"""
args = AgentLogQuery.model_validate(request.args.to_dict(flat=True)) args = AgentLogQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id) return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id)

View File

@ -1,5 +1,4 @@
from typing import Any, Literal from typing import Any, Literal
from uuid import UUID
from flask import abort, make_response, request from flask import abort, make_response, request
from flask_restx import Resource from flask_restx import Resource
@ -34,6 +33,8 @@ from services.annotation_service import (
UpsertAnnotationArgs, UpsertAnnotationArgs,
) )
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
class AnnotationReplyPayload(BaseModel): class AnnotationReplyPayload(BaseModel):
score_threshold: float = Field(..., description="Score threshold for annotation matching") score_threshold: float = Field(..., description="Score threshold for annotation matching")
@ -86,6 +87,17 @@ class AnnotationFilePayload(BaseModel):
return uuid_value(value) return uuid_value(value)
def reg(model: type[BaseModel]) -> None:
console_ns.schema_model(model.__name__, model.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0))
reg(AnnotationReplyPayload)
reg(AnnotationSettingUpdatePayload)
reg(AnnotationListQuery)
reg(CreateAnnotationPayload)
reg(UpdateAnnotationPayload)
reg(AnnotationReplyStatusQuery)
reg(AnnotationFilePayload)
register_schema_models( register_schema_models(
console_ns, console_ns,
Annotation, Annotation,
@ -93,13 +105,6 @@ register_schema_models(
AnnotationExportList, AnnotationExportList,
AnnotationHitHistory, AnnotationHitHistory,
AnnotationHitHistoryList, AnnotationHitHistoryList,
AnnotationReplyPayload,
AnnotationSettingUpdatePayload,
AnnotationListQuery,
CreateAnnotationPayload,
UpdateAnnotationPayload,
AnnotationReplyStatusQuery,
AnnotationFilePayload,
) )
@ -116,7 +121,8 @@ class AnnotationReplyActionApi(Resource):
@account_initialization_required @account_initialization_required
@cloud_edition_billing_resource_check("annotation") @cloud_edition_billing_resource_check("annotation")
@edit_permission_required @edit_permission_required
def post(self, app_id: UUID, action: Literal["enable", "disable"]): def post(self, app_id, action: Literal["enable", "disable"]):
app_id = str(app_id)
args = AnnotationReplyPayload.model_validate(console_ns.payload) args = AnnotationReplyPayload.model_validate(console_ns.payload)
match action: match action:
case "enable": case "enable":
@ -125,9 +131,9 @@ class AnnotationReplyActionApi(Resource):
"embedding_provider_name": args.embedding_provider_name, "embedding_provider_name": args.embedding_provider_name,
"embedding_model_name": args.embedding_model_name, "embedding_model_name": args.embedding_model_name,
} }
result = AppAnnotationService.enable_app_annotation(enable_args, str(app_id)) result = AppAnnotationService.enable_app_annotation(enable_args, app_id)
case "disable": case "disable":
result = AppAnnotationService.disable_app_annotation(str(app_id)) result = AppAnnotationService.disable_app_annotation(app_id)
return result, 200 return result, 200
@ -142,8 +148,9 @@ class AppAnnotationSettingDetailApi(Resource):
@login_required @login_required
@account_initialization_required @account_initialization_required
@edit_permission_required @edit_permission_required
def get(self, app_id: UUID): def get(self, app_id):
result = AppAnnotationService.get_app_annotation_setting_by_app_id(str(app_id)) app_id = str(app_id)
result = AppAnnotationService.get_app_annotation_setting_by_app_id(app_id)
return result, 200 return result, 200
@ -159,13 +166,14 @@ class AppAnnotationSettingUpdateApi(Resource):
@login_required @login_required
@account_initialization_required @account_initialization_required
@edit_permission_required @edit_permission_required
def post(self, app_id: UUID, annotation_setting_id): def post(self, app_id, annotation_setting_id):
app_id = str(app_id)
annotation_setting_id = str(annotation_setting_id) annotation_setting_id = str(annotation_setting_id)
args = AnnotationSettingUpdatePayload.model_validate(console_ns.payload) args = AnnotationSettingUpdatePayload.model_validate(console_ns.payload)
setting_args: UpdateAnnotationSettingArgs = {"score_threshold": args.score_threshold} setting_args: UpdateAnnotationSettingArgs = {"score_threshold": args.score_threshold}
result = AppAnnotationService.update_app_annotation_setting(str(app_id), annotation_setting_id, setting_args) result = AppAnnotationService.update_app_annotation_setting(app_id, annotation_setting_id, setting_args)
return result, 200 return result, 200
@ -181,7 +189,7 @@ class AnnotationReplyActionStatusApi(Resource):
@account_initialization_required @account_initialization_required
@cloud_edition_billing_resource_check("annotation") @cloud_edition_billing_resource_check("annotation")
@edit_permission_required @edit_permission_required
def get(self, app_id: UUID, job_id, action): def get(self, app_id, job_id, action):
job_id = str(job_id) job_id = str(job_id)
app_annotation_job_key = f"{action}_app_annotation_job_{str(job_id)}" app_annotation_job_key = f"{action}_app_annotation_job_{str(job_id)}"
cache_result = redis_client.get(app_annotation_job_key) cache_result = redis_client.get(app_annotation_job_key)
@ -209,13 +217,14 @@ class AnnotationApi(Resource):
@login_required @login_required
@account_initialization_required @account_initialization_required
@edit_permission_required @edit_permission_required
def get(self, app_id: UUID): def get(self, app_id):
args = AnnotationListQuery.model_validate(request.args.to_dict(flat=True)) args = AnnotationListQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
page = args.page page = args.page
limit = args.limit limit = args.limit
keyword = args.keyword keyword = args.keyword
annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(str(app_id), page, limit, keyword) app_id = str(app_id)
annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(app_id, page, limit, keyword)
annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True) annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
response = AnnotationList( response = AnnotationList(
data=annotation_models, data=annotation_models,
@ -237,7 +246,8 @@ class AnnotationApi(Resource):
@account_initialization_required @account_initialization_required
@cloud_edition_billing_resource_check("annotation") @cloud_edition_billing_resource_check("annotation")
@edit_permission_required @edit_permission_required
def post(self, app_id: UUID): def post(self, app_id):
app_id = str(app_id)
args = CreateAnnotationPayload.model_validate(console_ns.payload) args = CreateAnnotationPayload.model_validate(console_ns.payload)
upsert_args: UpsertAnnotationArgs = {} upsert_args: UpsertAnnotationArgs = {}
if args.answer is not None: if args.answer is not None:
@ -248,14 +258,15 @@ class AnnotationApi(Resource):
upsert_args["message_id"] = args.message_id upsert_args["message_id"] = args.message_id
if args.question is not None: if args.question is not None:
upsert_args["question"] = args.question upsert_args["question"] = args.question
annotation = AppAnnotationService.up_insert_app_annotation_from_message(upsert_args, str(app_id)) annotation = AppAnnotationService.up_insert_app_annotation_from_message(upsert_args, app_id)
return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json") return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@edit_permission_required @edit_permission_required
def delete(self, app_id: UUID): def delete(self, app_id):
app_id = str(app_id)
# Use request.args.getlist to get annotation_ids array directly # Use request.args.getlist to get annotation_ids array directly
annotation_ids = request.args.getlist("annotation_id") annotation_ids = request.args.getlist("annotation_id")
@ -269,11 +280,11 @@ class AnnotationApi(Resource):
"message": "annotation_ids are required if the parameter is provided.", "message": "annotation_ids are required if the parameter is provided.",
}, 400 }, 400
result = AppAnnotationService.delete_app_annotations_in_batch(str(app_id), annotation_ids) result = AppAnnotationService.delete_app_annotations_in_batch(app_id, annotation_ids)
return result, 204 return result, 204
# If no annotation_ids are provided, handle clearing all annotations # If no annotation_ids are provided, handle clearing all annotations
else: else:
AppAnnotationService.clear_all_annotations(str(app_id)) AppAnnotationService.clear_all_annotations(app_id)
return {"result": "success"}, 204 return {"result": "success"}, 204
@ -292,8 +303,9 @@ class AnnotationExportApi(Resource):
@login_required @login_required
@account_initialization_required @account_initialization_required
@edit_permission_required @edit_permission_required
def get(self, app_id: UUID): def get(self, app_id):
annotation_list = AppAnnotationService.export_annotation_list_by_app_id(str(app_id)) app_id = str(app_id)
annotation_list = AppAnnotationService.export_annotation_list_by_app_id(app_id)
annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True) annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
response_data = AnnotationExportList(data=annotation_models).model_dump(mode="json") response_data = AnnotationExportList(data=annotation_models).model_dump(mode="json")
@ -319,22 +331,26 @@ class AnnotationUpdateDeleteApi(Resource):
@account_initialization_required @account_initialization_required
@cloud_edition_billing_resource_check("annotation") @cloud_edition_billing_resource_check("annotation")
@edit_permission_required @edit_permission_required
def post(self, app_id: UUID, annotation_id: UUID): def post(self, app_id, annotation_id):
app_id = str(app_id)
annotation_id = str(annotation_id)
args = UpdateAnnotationPayload.model_validate(console_ns.payload) args = UpdateAnnotationPayload.model_validate(console_ns.payload)
update_args: UpdateAnnotationArgs = {} update_args: UpdateAnnotationArgs = {}
if args.answer is not None: if args.answer is not None:
update_args["answer"] = args.answer update_args["answer"] = args.answer
if args.question is not None: if args.question is not None:
update_args["question"] = args.question update_args["question"] = args.question
annotation = AppAnnotationService.update_app_annotation_directly(update_args, str(app_id), str(annotation_id)) annotation = AppAnnotationService.update_app_annotation_directly(update_args, app_id, annotation_id)
return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json") return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@edit_permission_required @edit_permission_required
def delete(self, app_id: UUID, annotation_id: UUID): def delete(self, app_id, annotation_id):
AppAnnotationService.delete_app_annotation(str(app_id), str(annotation_id)) app_id = str(app_id)
annotation_id = str(annotation_id)
AppAnnotationService.delete_app_annotation(app_id, annotation_id)
return {"result": "success"}, 204 return {"result": "success"}, 204
@ -355,9 +371,11 @@ class AnnotationBatchImportApi(Resource):
@annotation_import_rate_limit @annotation_import_rate_limit
@annotation_import_concurrency_limit @annotation_import_concurrency_limit
@edit_permission_required @edit_permission_required
def post(self, app_id: UUID): def post(self, app_id):
from configs import dify_config from configs import dify_config
app_id = str(app_id)
# check file # check file
if "file" not in request.files: if "file" not in request.files:
raise NoFileUploadedError() raise NoFileUploadedError()
@ -373,9 +391,9 @@ class AnnotationBatchImportApi(Resource):
raise ValueError("Invalid file type. Only CSV files are allowed") raise ValueError("Invalid file type. Only CSV files are allowed")
# Check file size before processing # Check file size before processing
file.stream.seek(0, 2) # Seek to end of file file.seek(0, 2) # Seek to end of file
file_size = file.stream.tell() file_size = file.tell()
file.stream.seek(0) # Reset to beginning file.seek(0) # Reset to beginning
max_size_bytes = dify_config.ANNOTATION_IMPORT_FILE_SIZE_LIMIT * 1024 * 1024 max_size_bytes = dify_config.ANNOTATION_IMPORT_FILE_SIZE_LIMIT * 1024 * 1024
if file_size > max_size_bytes: if file_size > max_size_bytes:
@ -388,7 +406,7 @@ class AnnotationBatchImportApi(Resource):
if file_size == 0: if file_size == 0:
raise ValueError("The uploaded file is empty") raise ValueError("The uploaded file is empty")
return AppAnnotationService.batch_import_app_annotations(str(app_id), file) return AppAnnotationService.batch_import_app_annotations(app_id, file)
@console_ns.route("/apps/<uuid:app_id>/annotations/batch-import-status/<uuid:job_id>") @console_ns.route("/apps/<uuid:app_id>/annotations/batch-import-status/<uuid:job_id>")
@ -403,7 +421,8 @@ class AnnotationBatchImportStatusApi(Resource):
@account_initialization_required @account_initialization_required
@cloud_edition_billing_resource_check("annotation") @cloud_edition_billing_resource_check("annotation")
@edit_permission_required @edit_permission_required
def get(self, app_id: UUID, job_id: UUID): def get(self, app_id, job_id):
job_id = str(job_id)
indexing_cache_key = f"app_annotation_batch_import_{str(job_id)}" indexing_cache_key = f"app_annotation_batch_import_{str(job_id)}"
cache_result = redis_client.get(indexing_cache_key) cache_result = redis_client.get(indexing_cache_key)
if cache_result is None: if cache_result is None:
@ -437,11 +456,13 @@ class AnnotationHitHistoryListApi(Resource):
@login_required @login_required
@account_initialization_required @account_initialization_required
@edit_permission_required @edit_permission_required
def get(self, app_id: UUID, annotation_id: UUID): def get(self, app_id, annotation_id):
page = request.args.get("page", default=1, type=int) page = request.args.get("page", default=1, type=int)
limit = request.args.get("limit", default=20, type=int) limit = request.args.get("limit", default=20, type=int)
app_id = str(app_id)
annotation_id = str(annotation_id)
annotation_hit_history_list, total = AppAnnotationService.get_annotation_hit_histories( annotation_hit_history_list, total = AppAnnotationService.get_annotation_hit_histories(
str(app_id), str(annotation_id), page, limit app_id, annotation_id, page, limit
) )
history_models = TypeAdapter(list[AnnotationHitHistory]).validate_python( history_models = TypeAdapter(list[AnnotationHitHistory]).validate_python(
annotation_hit_history_list, from_attributes=True annotation_hit_history_list, from_attributes=True

View File

@ -1,16 +1,15 @@
import logging import logging
import re
import uuid import uuid
from datetime import datetime from datetime import datetime
from typing import Any, Literal from typing import Any, Literal
from uuid import UUID
from flask import request from flask import request
from flask_restx import Resource from flask_restx import Resource
from graphon.enums import WorkflowExecutionStatus
from graphon.file import helpers as file_helpers
from pydantic import AliasChoices, BaseModel, Field, computed_field, field_validator from pydantic import AliasChoices, BaseModel, Field, computed_field, field_validator
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.orm import Session from sqlalchemy.orm import sessionmaker
from werkzeug.datastructures import MultiDict
from werkzeug.exceptions import BadRequest from werkzeug.exceptions import BadRequest
from controllers.common.helpers import FileInfo from controllers.common.helpers import FileInfo
@ -26,22 +25,19 @@ from controllers.console.wraps import (
is_admin_or_owner_required, is_admin_or_owner_required,
setup_required, setup_required,
) )
from core.db.session_factory import session_factory
from core.ops.ops_trace_manager import OpsTraceManager from core.ops.ops_trace_manager import OpsTraceManager
from core.rag.entities import PreProcessingRule, Rule, Segmentation from core.rag.entities import PreProcessingRule, Rule, Segmentation
from core.rag.retrieval.retrieval_methods import RetrievalMethod from core.rag.retrieval.retrieval_methods import RetrievalMethod
from core.trigger.constants import TRIGGER_NODE_TYPES from core.trigger.constants import TRIGGER_NODE_TYPES
from extensions.ext_database import db from extensions.ext_database import db
from fields.base import ResponseModel from fields.base import ResponseModel
from graphon.enums import WorkflowExecutionStatus
from libs.helper import build_icon_url
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models import App, DatasetPermissionEnum, Workflow from models import App, DatasetPermissionEnum, Workflow
from models.model import IconType from models.model import IconType
from services.app_dsl_service import AppDslService from services.app_dsl_service import AppDslService
from services.app_service import AppListParams, AppService, CreateAppParams from services.app_service import AppService
from services.enterprise.enterprise_service import EnterpriseService from services.enterprise.enterprise_service import EnterpriseService
from services.entities.dsl_entities import ImportMode, ImportStatus from services.entities.dsl_entities import ImportMode
from services.entities.knowledge_entities.knowledge_entities import ( from services.entities.knowledge_entities.knowledge_entities import (
DataSource, DataSource,
InfoList, InfoList,
@ -56,34 +52,36 @@ from services.entities.knowledge_entities.knowledge_entities import (
) )
from services.feature_service import FeatureService from services.feature_service import FeatureService
ALLOW_CREATE_APP_MODES = ["chat", "agent-chat", "advanced-chat", "workflow", "completion"] ALLOW_CREATE_APP_MODES = ["chat", "agent-chat", "advanced-chat", "workflow", "completion", "agent"]
register_enum_models(console_ns, IconType) register_enum_models(console_ns, IconType)
_logger = logging.getLogger(__name__) _logger = logging.getLogger(__name__)
_TAG_IDS_BRACKET_PATTERN = re.compile(r"^tag_ids\[(\d+)\]$")
class AppListQuery(BaseModel): class AppListQuery(BaseModel):
page: int = Field(default=1, ge=1, le=99999, description="Page number (1-99999)") page: int = Field(default=1, ge=1, le=99999, description="Page number (1-99999)")
limit: int = Field(default=20, ge=1, le=100, description="Page size (1-100)") limit: int = Field(default=20, ge=1, le=100, description="Page size (1-100)")
mode: Literal["completion", "chat", "advanced-chat", "workflow", "agent-chat", "channel", "all"] = Field( mode: Literal["completion", "chat", "advanced-chat", "workflow", "agent-chat", "agent", "channel", "all"] = Field(
default="all", description="App mode filter" default="all", description="App mode filter"
) )
name: str | None = Field(default=None, description="Filter by app name") name: str | None = Field(default=None, description="Filter by app name")
tag_ids: list[str] | None = Field(default=None, description="Filter by tag IDs") tag_ids: list[str] | None = Field(default=None, description="Comma-separated tag IDs")
is_created_by_me: bool | None = Field(default=None, description="Filter by creator") is_created_by_me: bool | None = Field(default=None, description="Filter by creator")
@field_validator("tag_ids", mode="before") @field_validator("tag_ids", mode="before")
@classmethod @classmethod
def validate_tag_ids(cls, value: list[str] | None) -> list[str] | None: def validate_tag_ids(cls, value: str | list[str] | None) -> list[str] | None:
if not value: if not value:
return None return None
if not isinstance(value, list): if isinstance(value, str):
raise ValueError("Unsupported tag_ids type.") items = [item.strip() for item in value.split(",") if item.strip()]
elif isinstance(value, list):
items = [str(item).strip() for item in value if item and str(item).strip()]
else:
raise TypeError("Unsupported tag_ids type.")
items = [str(item).strip() for item in value if item and str(item).strip()]
if not items: if not items:
return None return None
@ -93,30 +91,12 @@ class AppListQuery(BaseModel):
raise ValueError("Invalid UUID format in tag_ids.") from exc raise ValueError("Invalid UUID format in tag_ids.") from exc
def _normalize_app_list_query_args(query_args: MultiDict[str, str]) -> dict[str, str | list[str]]:
normalized: dict[str, str | list[str]] = {}
indexed_tag_ids: list[tuple[int, str]] = []
for key in query_args:
match = _TAG_IDS_BRACKET_PATTERN.fullmatch(key)
if match:
indexed_tag_ids.extend((int(match.group(1)), value) for value in query_args.getlist(key))
continue
value = query_args.get(key)
if value is not None:
normalized[key] = value
if indexed_tag_ids:
normalized["tag_ids"] = [value for _, value in sorted(indexed_tag_ids)]
return normalized
class CreateAppPayload(BaseModel): class CreateAppPayload(BaseModel):
name: str = Field(..., min_length=1, description="App name") name: str = Field(..., min_length=1, description="App name")
description: str | None = Field(default=None, description="App description (max 400 chars)", max_length=400) description: str | None = Field(default=None, description="App description (max 400 chars)", max_length=400)
mode: Literal["chat", "agent-chat", "advanced-chat", "workflow", "completion"] = Field(..., description="App mode") mode: Literal["chat", "agent-chat", "advanced-chat", "workflow", "completion", "agent"] = Field(
..., description="App mode"
)
icon_type: IconType | None = Field(default=None, description="Icon type") icon_type: IconType | None = Field(default=None, description="Icon type")
icon: str | None = Field(default=None, description="Icon") icon: str | None = Field(default=None, description="Icon")
icon_background: str | None = Field(default=None, description="Icon background color") icon_background: str | None = Field(default=None, description="Icon background color")
@ -151,7 +131,6 @@ class AppNamePayload(BaseModel):
class AppIconPayload(BaseModel): class AppIconPayload(BaseModel):
icon: str | None = Field(default=None, description="Icon data") icon: str | None = Field(default=None, description="Icon data")
icon_type: IconType | None = Field(default=None, description="Icon type")
icon_background: str | None = Field(default=None, description="Icon background color") icon_background: str | None = Field(default=None, description="Icon background color")
@ -184,6 +163,15 @@ def _to_timestamp(value: datetime | int | None) -> int | None:
return value return value
def _build_icon_url(icon_type: str | IconType | None, icon: str | None) -> str | None:
if icon is None or icon_type is None:
return None
icon_type_value = icon_type.value if isinstance(icon_type, IconType) else str(icon_type)
if icon_type_value.lower() != IconType.IMAGE:
return None
return file_helpers.get_signed_file_url(icon)
class Tag(ResponseModel): class Tag(ResponseModel):
id: str id: str
name: str name: str
@ -306,7 +294,7 @@ class Site(ResponseModel):
@computed_field(return_type=str | None) # type: ignore @computed_field(return_type=str | None) # type: ignore
@property @property
def icon_url(self) -> str | None: def icon_url(self) -> str | None:
return build_icon_url(self.icon_type, self.icon) return _build_icon_url(self.icon_type, self.icon)
@field_validator("icon_type", mode="before") @field_validator("icon_type", mode="before")
@classmethod @classmethod
@ -356,7 +344,7 @@ class AppPartial(ResponseModel):
@computed_field(return_type=str | None) # type: ignore @computed_field(return_type=str | None) # type: ignore
@property @property
def icon_url(self) -> str | None: def icon_url(self) -> str | None:
return build_icon_url(self.icon_type, self.icon) return _build_icon_url(self.icon_type, self.icon)
@field_validator("created_at", "updated_at", mode="before") @field_validator("created_at", "updated_at", mode="before")
@classmethod @classmethod
@ -404,7 +392,7 @@ class AppDetailWithSite(AppDetail):
@computed_field(return_type=str | None) # type: ignore @computed_field(return_type=str | None) # type: ignore
@property @property
def icon_url(self) -> str | None: def icon_url(self) -> str | None:
return build_icon_url(self.icon_type, self.icon) return _build_icon_url(self.icon_type, self.icon)
class AppPagination(ResponseModel): class AppPagination(ResponseModel):
@ -477,19 +465,12 @@ class AppListApi(Resource):
"""Get app list""" """Get app list"""
current_user, current_tenant_id = current_account_with_tenant() current_user, current_tenant_id = current_account_with_tenant()
args = AppListQuery.model_validate(_normalize_app_list_query_args(request.args)) args = AppListQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
params = AppListParams( args_dict = args.model_dump()
page=args.page,
limit=args.limit,
mode=args.mode,
name=args.name,
tag_ids=args.tag_ids,
is_created_by_me=args.is_created_by_me,
)
# get app list # get app list
app_service = AppService() app_service = AppService()
app_pagination = app_service.get_paginate_apps(current_user.id, current_tenant_id, params) app_pagination = app_service.get_paginate_apps(current_user.id, current_tenant_id, args_dict)
if not app_pagination: if not app_pagination:
empty = AppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[]) empty = AppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
return empty.model_dump(mode="json"), 200 return empty.model_dump(mode="json"), 200
@ -553,17 +534,9 @@ class AppListApi(Resource):
"""Create app""" """Create app"""
current_user, current_tenant_id = current_account_with_tenant() current_user, current_tenant_id = current_account_with_tenant()
args = CreateAppPayload.model_validate(console_ns.payload) args = CreateAppPayload.model_validate(console_ns.payload)
params = CreateAppParams(
name=args.name,
description=args.description,
mode=args.mode,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
)
app_service = AppService() app_service = AppService()
app = app_service.create_app(current_tenant_id, params, current_user) app = app_service.create_app(current_tenant_id, args.model_dump(), current_user)
app_detail = AppDetail.model_validate(app, from_attributes=True) app_detail = AppDetail.model_validate(app, from_attributes=True)
return app_detail.model_dump(mode="json"), 201 return app_detail.model_dump(mode="json"), 201
@ -661,7 +634,7 @@ class AppCopyApi(Resource):
args = CopyAppPayload.model_validate(console_ns.payload or {}) args = CopyAppPayload.model_validate(console_ns.payload or {})
with Session(db.engine, expire_on_commit=False) as session: with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
import_service = AppDslService(session) import_service = AppDslService(session)
yaml_content = import_service.export_dsl(app_model=app_model, include_secret=True) yaml_content = import_service.export_dsl(app_model=app_model, include_secret=True)
result = import_service.import_app( result = import_service.import_app(
@ -674,13 +647,6 @@ class AppCopyApi(Resource):
icon=args.icon, icon=args.icon,
icon_background=args.icon_background, icon_background=args.icon_background,
) )
if result.status == ImportStatus.FAILED:
session.rollback()
return result.model_dump(mode="json"), 400
if result.status == ImportStatus.PENDING:
session.rollback()
return result.model_dump(mode="json"), 202
session.commit()
# Inherit web app permission from original app # Inherit web app permission from original app
if result.app_id and FeatureService.get_system_features().webapp_auth.enabled: if result.app_id and FeatureService.get_system_features().webapp_auth.enabled:
@ -717,7 +683,7 @@ class AppExportApi(Resource):
@edit_permission_required @edit_permission_required
def get(self, app_model): def get(self, app_model):
"""Export app""" """Export app"""
args = AppExportQuery.model_validate(request.args.to_dict(flat=True)) args = AppExportQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
payload = AppExportResponse( payload = AppExportResponse(
data=AppDslService.export_dsl( data=AppDslService.export_dsl(
@ -729,32 +695,6 @@ class AppExportApi(Resource):
return payload.model_dump(mode="json") return payload.model_dump(mode="json")
@console_ns.route("/apps/<uuid:app_id>/publish-to-creators-platform")
class AppPublishToCreatorsPlatformApi(Resource):
@setup_required
@login_required
@account_initialization_required
@get_app_model(mode=None)
@edit_permission_required
def post(self, app_model):
"""Publish app to Creators Platform"""
from configs import dify_config
from core.helper.creators import get_redirect_url, upload_dsl
if not dify_config.CREATORS_PLATFORM_FEATURES_ENABLED:
return {"error": "Creators Platform features are not enabled"}, 403
current_user, _ = current_account_with_tenant()
dsl_content = AppDslService.export_dsl(app_model=app_model, include_secret=False)
dsl_bytes = dsl_content.encode("utf-8")
claim_code = upload_dsl(dsl_bytes)
redirect_url = get_redirect_url(str(current_user.id), claim_code)
return {"redirect_url": redirect_url}
@console_ns.route("/apps/<uuid:app_id>/name") @console_ns.route("/apps/<uuid:app_id>/name")
class AppNameApi(Resource): class AppNameApi(Resource):
@console_ns.doc("check_app_name") @console_ns.doc("check_app_name")
@ -793,12 +733,7 @@ class AppIconApi(Resource):
args = AppIconPayload.model_validate(console_ns.payload or {}) args = AppIconPayload.model_validate(console_ns.payload or {})
app_service = AppService() app_service = AppService()
app_model = app_service.update_app_icon( app_model = app_service.update_app_icon(app_model, args.icon or "", args.icon_background or "")
app_model,
args.icon or "",
args.icon_background or "",
args.icon_type,
)
response_model = AppDetail.model_validate(app_model, from_attributes=True) response_model = AppDetail.model_validate(app_model, from_attributes=True)
return response_model.model_dump(mode="json") return response_model.model_dump(mode="json")
@ -856,10 +791,9 @@ class AppTraceApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
def get(self, app_id: UUID): def get(self, app_id):
"""Get app trace""" """Get app trace"""
with session_factory.create_session() as session: app_trace_config = OpsTraceManager.get_app_tracing_config(app_id=app_id)
app_trace_config = OpsTraceManager.get_app_tracing_config(str(app_id), session)
return app_trace_config return app_trace_config
@ -873,12 +807,12 @@ class AppTraceApi(Resource):
@login_required @login_required
@account_initialization_required @account_initialization_required
@edit_permission_required @edit_permission_required
def post(self, app_id: UUID): def post(self, app_id):
# add app trace # add app trace
args = AppTracePayload.model_validate(console_ns.payload) args = AppTracePayload.model_validate(console_ns.payload)
OpsTraceManager.update_app_tracing_config( OpsTraceManager.update_app_tracing_config(
app_id=str(app_id), app_id=app_id,
enabled=args.enabled, enabled=args.enabled,
tracing_provider=args.tracing_provider, tracing_provider=args.tracing_provider,
) )

View File

@ -1,8 +1,8 @@
from flask_restx import Resource from flask_restx import Resource
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from sqlalchemy.orm import Session from sqlalchemy.orm import sessionmaker
from controllers.common.schema import register_enum_models, register_schema_models from controllers.common.schema import register_schema_models
from controllers.console.app.wraps import get_app_model from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import ( from controllers.console.wraps import (
account_initialization_required, account_initialization_required,
@ -33,7 +33,6 @@ class AppImportPayload(BaseModel):
app_id: str | None = Field(None) app_id: str | None = Field(None)
register_enum_models(console_ns, ImportStatus)
register_schema_models(console_ns, AppImportPayload, Import, CheckDependenciesResult) register_schema_models(console_ns, AppImportPayload, Import, CheckDependenciesResult)
@ -53,9 +52,8 @@ class AppImportApi(Resource):
current_user, _ = current_account_with_tenant() current_user, _ = current_account_with_tenant()
args = AppImportPayload.model_validate(console_ns.payload) args = AppImportPayload.model_validate(console_ns.payload)
# AppDslService performs internal commits for some creation paths, so use a plain # Create service with session
# Session here instead of nesting it inside sessionmaker(...).begin(). with sessionmaker(db.engine).begin() as session:
with Session(db.engine, expire_on_commit=False) as session:
import_service = AppDslService(session) import_service = AppDslService(session)
# Import app # Import app
account = current_user account = current_user
@ -71,10 +69,6 @@ class AppImportApi(Resource):
icon_background=args.icon_background, icon_background=args.icon_background,
app_id=args.app_id, app_id=args.app_id,
) )
if result.status == ImportStatus.FAILED:
session.rollback()
else:
session.commit()
if result.app_id and FeatureService.get_system_features().webapp_auth.enabled: if result.app_id and FeatureService.get_system_features().webapp_auth.enabled:
# update web app setting as private # update web app setting as private
EnterpriseService.WebAppAuth.update_app_access_mode(result.app_id, "private") EnterpriseService.WebAppAuth.update_app_access_mode(result.app_id, "private")
@ -101,15 +95,12 @@ class AppImportConfirmApi(Resource):
# Check user role first # Check user role first
current_user, _ = current_account_with_tenant() current_user, _ = current_account_with_tenant()
with Session(db.engine, expire_on_commit=False) as session: # Create service with session
with sessionmaker(db.engine).begin() as session:
import_service = AppDslService(session) import_service = AppDslService(session)
# Confirm import # Confirm import
account = current_user account = current_user
result = import_service.confirm_import(import_id=import_id, account=account) result = import_service.confirm_import(import_id=import_id, account=account)
if result.status == ImportStatus.FAILED:
session.rollback()
else:
session.commit()
# Return appropriate status code based on result # Return appropriate status code based on result
if result.status == ImportStatus.FAILED: if result.status == ImportStatus.FAILED:
@ -126,7 +117,7 @@ class AppImportCheckDependenciesApi(Resource):
@account_initialization_required @account_initialization_required
@edit_permission_required @edit_permission_required
def get(self, app_model: App): def get(self, app_model: App):
with Session(db.engine, expire_on_commit=False) as session: with sessionmaker(db.engine).begin() as session:
import_service = AppDslService(session) import_service = AppDslService(session)
result = import_service.check_dependencies(app_model=app_model) result = import_service.check_dependencies(app_model=app_model)

View File

@ -2,6 +2,7 @@ import logging
from flask import request from flask import request
from flask_restx import Resource, fields from flask_restx import Resource, fields
from graphon.model_runtime.errors.invoke import InvokeError
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from werkzeug.exceptions import InternalServerError from werkzeug.exceptions import InternalServerError
@ -22,7 +23,6 @@ from controllers.console.app.error import (
from controllers.console.app.wraps import get_app_model from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from graphon.model_runtime.errors.invoke import InvokeError
from libs.login import login_required from libs.login import login_required
from models import App, AppMode from models import App, AppMode
from services.audio_service import AudioService from services.audio_service import AudioService
@ -173,7 +173,7 @@ class TextModesApi(Resource):
@account_initialization_required @account_initialization_required
def get(self, app_model): def get(self, app_model):
try: try:
args = TextToSpeechVoiceQuery.model_validate(request.args.to_dict(flat=True)) args = TextToSpeechVoiceQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
response = AudioService.transcript_tts_voices( response = AudioService.transcript_tts_voices(
tenant_id=app_model.tenant_id, tenant_id=app_model.tenant_id,

View File

@ -3,11 +3,11 @@ from typing import Any, Literal
from flask import request from flask import request
from flask_restx import Resource from flask_restx import Resource
from graphon.model_runtime.errors.invoke import InvokeError
from pydantic import BaseModel, Field, field_validator from pydantic import BaseModel, Field, field_validator
from werkzeug.exceptions import InternalServerError, NotFound from werkzeug.exceptions import InternalServerError, NotFound
import services import services
from controllers.common.schema import register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.app.error import ( from controllers.console.app.error import (
AppUnavailableError, AppUnavailableError,
@ -27,7 +27,6 @@ from core.errors.error import (
QuotaExceededError, QuotaExceededError,
) )
from core.helper.trace_id_helper import get_external_trace_id from core.helper.trace_id_helper import get_external_trace_id
from graphon.model_runtime.errors.invoke import InvokeError
from libs import helper from libs import helper
from libs.helper import uuid_value from libs.helper import uuid_value
from libs.login import current_user, login_required from libs.login import current_user, login_required
@ -38,6 +37,7 @@ from services.app_task_service import AppTaskService
from services.errors.llm import InvokeRateLimitError from services.errors.llm import InvokeRateLimitError
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
class BaseMessagePayload(BaseModel): class BaseMessagePayload(BaseModel):
@ -65,7 +65,13 @@ class ChatMessagePayload(BaseMessagePayload):
return uuid_value(value) return uuid_value(value)
register_schema_models(console_ns, CompletionMessagePayload, ChatMessagePayload) console_ns.schema_model(
CompletionMessagePayload.__name__,
CompletionMessagePayload.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
)
console_ns.schema_model(
ChatMessagePayload.__name__, ChatMessagePayload.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0)
)
# define completion message api for user # define completion message api for user
@ -155,7 +161,7 @@ class ChatMessageApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT]) @get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def post(self, app_model): def post(self, app_model):
args_model = ChatMessagePayload.model_validate(console_ns.payload) args_model = ChatMessagePayload.model_validate(console_ns.payload)
@ -209,7 +215,7 @@ class ChatMessageStopApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT]) @get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
def post(self, app_model, task_id): def post(self, app_model, task_id):
if not isinstance(current_user, Account): if not isinstance(current_user, Account):
raise ValueError("current_user must be an Account instance") raise ValueError("current_user must be an Account instance")

View File

@ -2,43 +2,28 @@ from typing import Literal
import sqlalchemy as sa import sqlalchemy as sa
from flask import abort, request from flask import abort, request
from flask_restx import Resource from flask_restx import Resource, fields, marshal_with
from pydantic import BaseModel, Field, field_validator from pydantic import BaseModel, Field, field_validator
from sqlalchemy import func, or_ from sqlalchemy import func, or_
from sqlalchemy.orm import selectinload from sqlalchemy.orm import selectinload
from werkzeug.exceptions import NotFound from werkzeug.exceptions import NotFound
from controllers.common.schema import register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
from core.app.entities.app_invoke_entities import InvokeFrom from core.app.entities.app_invoke_entities import InvokeFrom
from extensions.ext_database import db from extensions.ext_database import db
from fields.conversation_fields import ( from fields.raws import FilesContainedField
Conversation as ConversationResponse,
)
from fields.conversation_fields import (
ConversationDetail as ConversationDetailResponse,
)
from fields.conversation_fields import (
ConversationMessageDetail as ConversationMessageDetailResponse,
)
from fields.conversation_fields import (
ConversationPagination as ConversationPaginationResponse,
)
from fields.conversation_fields import (
ConversationWithSummaryPagination as ConversationWithSummaryPaginationResponse,
)
from fields.conversation_fields import (
ResultResponse,
)
from libs.datetime_utils import naive_utc_now, parse_time_range from libs.datetime_utils import naive_utc_now, parse_time_range
from libs.helper import TimestampField
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models import Conversation, EndUser, Message, MessageAnnotation from models import Conversation, EndUser, Message, MessageAnnotation
from models.model import AppMode from models.model import AppMode
from services.conversation_service import ConversationService from services.conversation_service import ConversationService
from services.errors.conversation import ConversationNotExistsError from services.errors.conversation import ConversationNotExistsError
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
class BaseConversationQuery(BaseModel): class BaseConversationQuery(BaseModel):
keyword: str | None = Field(default=None, description="Search keyword") keyword: str | None = Field(default=None, description="Search keyword")
@ -68,18 +53,276 @@ class ChatConversationQuery(BaseConversationQuery):
) )
register_schema_models( console_ns.schema_model(
console_ns, CompletionConversationQuery.__name__,
CompletionConversationQuery, CompletionConversationQuery.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
ChatConversationQuery, )
ConversationResponse, console_ns.schema_model(
ConversationPaginationResponse, ChatConversationQuery.__name__,
ConversationMessageDetailResponse, ChatConversationQuery.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
ConversationWithSummaryPaginationResponse, )
ConversationDetailResponse,
ResultResponse, # Register models for flask_restx to avoid dict type issues in Swagger
CompletionConversationQuery, # Register in dependency order: base models first, then dependent models
ChatConversationQuery,
# Base models
simple_account_model = console_ns.model(
"SimpleAccount",
{
"id": fields.String,
"name": fields.String,
"email": fields.String,
},
)
feedback_stat_model = console_ns.model(
"FeedbackStat",
{
"like": fields.Integer,
"dislike": fields.Integer,
},
)
status_count_model = console_ns.model(
"StatusCount",
{
"success": fields.Integer,
"failed": fields.Integer,
"partial_success": fields.Integer,
"paused": fields.Integer,
},
)
message_file_model = console_ns.model(
"MessageFile",
{
"id": fields.String,
"filename": fields.String,
"type": fields.String,
"url": fields.String,
"mime_type": fields.String,
"size": fields.Integer,
"transfer_method": fields.String,
"belongs_to": fields.String(default="user"),
"upload_file_id": fields.String(default=None),
},
)
agent_thought_model = console_ns.model(
"AgentThought",
{
"id": fields.String,
"chain_id": fields.String,
"message_id": fields.String,
"position": fields.Integer,
"thought": fields.String,
"tool": fields.String,
"tool_labels": fields.Raw,
"tool_input": fields.String,
"created_at": TimestampField,
"observation": fields.String,
"files": fields.List(fields.String),
},
)
simple_model_config_model = console_ns.model(
"SimpleModelConfig",
{
"model": fields.Raw(attribute="model_dict"),
"pre_prompt": fields.String,
},
)
model_config_model = console_ns.model(
"ModelConfig",
{
"opening_statement": fields.String,
"suggested_questions": fields.Raw,
"model": fields.Raw,
"user_input_form": fields.Raw,
"pre_prompt": fields.String,
"agent_mode": fields.Raw,
},
)
# Models that depend on simple_account_model
feedback_model = console_ns.model(
"Feedback",
{
"rating": fields.String,
"content": fields.String,
"from_source": fields.String,
"from_end_user_id": fields.String,
"from_account": fields.Nested(simple_account_model, allow_null=True),
},
)
annotation_model = console_ns.model(
"Annotation",
{
"id": fields.String,
"question": fields.String,
"content": fields.String,
"account": fields.Nested(simple_account_model, allow_null=True),
"created_at": TimestampField,
},
)
annotation_hit_history_model = console_ns.model(
"AnnotationHitHistory",
{
"annotation_id": fields.String(attribute="id"),
"annotation_create_account": fields.Nested(simple_account_model, allow_null=True),
"created_at": TimestampField,
},
)
class MessageTextField(fields.Raw):
def format(self, value):
return value[0]["text"] if value else ""
# Simple message detail model
simple_message_detail_model = console_ns.model(
"SimpleMessageDetail",
{
"inputs": FilesContainedField,
"query": fields.String,
"message": MessageTextField,
"answer": fields.String,
},
)
# Message detail model that depends on multiple models
message_detail_model = console_ns.model(
"MessageDetail",
{
"id": fields.String,
"conversation_id": fields.String,
"inputs": FilesContainedField,
"query": fields.String,
"message": fields.Raw,
"message_tokens": fields.Integer,
"answer": fields.String(attribute="re_sign_file_url_answer"),
"answer_tokens": fields.Integer,
"provider_response_latency": fields.Float,
"from_source": fields.String,
"from_end_user_id": fields.String,
"from_account_id": fields.String,
"feedbacks": fields.List(fields.Nested(feedback_model)),
"workflow_run_id": fields.String,
"annotation": fields.Nested(annotation_model, allow_null=True),
"annotation_hit_history": fields.Nested(annotation_hit_history_model, allow_null=True),
"created_at": TimestampField,
"agent_thoughts": fields.List(fields.Nested(agent_thought_model)),
"message_files": fields.List(fields.Nested(message_file_model)),
"metadata": fields.Raw(attribute="message_metadata_dict"),
"status": fields.String,
"error": fields.String,
"parent_message_id": fields.String,
},
)
# Conversation models
conversation_fields_model = console_ns.model(
"Conversation",
{
"id": fields.String,
"status": fields.String,
"from_source": fields.String,
"from_end_user_id": fields.String,
"from_end_user_session_id": fields.String(),
"from_account_id": fields.String,
"from_account_name": fields.String,
"read_at": TimestampField,
"created_at": TimestampField,
"updated_at": TimestampField,
"annotation": fields.Nested(annotation_model, allow_null=True),
"model_config": fields.Nested(simple_model_config_model),
"user_feedback_stats": fields.Nested(feedback_stat_model),
"admin_feedback_stats": fields.Nested(feedback_stat_model),
"message": fields.Nested(simple_message_detail_model, attribute="first_message"),
},
)
conversation_pagination_model = console_ns.model(
"ConversationPagination",
{
"page": fields.Integer,
"limit": fields.Integer(attribute="per_page"),
"total": fields.Integer,
"has_more": fields.Boolean(attribute="has_next"),
"data": fields.List(fields.Nested(conversation_fields_model), attribute="items"),
},
)
conversation_message_detail_model = console_ns.model(
"ConversationMessageDetail",
{
"id": fields.String,
"status": fields.String,
"from_source": fields.String,
"from_end_user_id": fields.String,
"from_account_id": fields.String,
"created_at": TimestampField,
"model_config": fields.Nested(model_config_model),
"message": fields.Nested(message_detail_model, attribute="first_message"),
},
)
conversation_with_summary_model = console_ns.model(
"ConversationWithSummary",
{
"id": fields.String,
"status": fields.String,
"from_source": fields.String,
"from_end_user_id": fields.String,
"from_end_user_session_id": fields.String,
"from_account_id": fields.String,
"from_account_name": fields.String,
"name": fields.String,
"summary": fields.String(attribute="summary_or_query"),
"read_at": TimestampField,
"created_at": TimestampField,
"updated_at": TimestampField,
"annotated": fields.Boolean,
"model_config": fields.Nested(simple_model_config_model),
"message_count": fields.Integer,
"user_feedback_stats": fields.Nested(feedback_stat_model),
"admin_feedback_stats": fields.Nested(feedback_stat_model),
"status_count": fields.Nested(status_count_model),
},
)
conversation_with_summary_pagination_model = console_ns.model(
"ConversationWithSummaryPagination",
{
"page": fields.Integer,
"limit": fields.Integer(attribute="per_page"),
"total": fields.Integer,
"has_more": fields.Boolean(attribute="has_next"),
"data": fields.List(fields.Nested(conversation_with_summary_model), attribute="items"),
},
)
conversation_detail_model = console_ns.model(
"ConversationDetail",
{
"id": fields.String,
"status": fields.String,
"from_source": fields.String,
"from_end_user_id": fields.String,
"from_account_id": fields.String,
"created_at": TimestampField,
"updated_at": TimestampField,
"annotated": fields.Boolean,
"introduction": fields.String,
"model_config": fields.Nested(model_config_model),
"message_count": fields.Integer,
"user_feedback_stats": fields.Nested(feedback_stat_model),
"admin_feedback_stats": fields.Nested(feedback_stat_model),
},
) )
@ -89,16 +332,17 @@ class CompletionConversationApi(Resource):
@console_ns.doc(description="Get completion conversations with pagination and filtering") @console_ns.doc(description="Get completion conversations with pagination and filtering")
@console_ns.doc(params={"app_id": "Application ID"}) @console_ns.doc(params={"app_id": "Application ID"})
@console_ns.expect(console_ns.models[CompletionConversationQuery.__name__]) @console_ns.expect(console_ns.models[CompletionConversationQuery.__name__])
@console_ns.response(200, "Success", console_ns.models[ConversationPaginationResponse.__name__]) @console_ns.response(200, "Success", conversation_pagination_model)
@console_ns.response(403, "Insufficient permissions") @console_ns.response(403, "Insufficient permissions")
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=AppMode.COMPLETION) @get_app_model(mode=AppMode.COMPLETION)
@marshal_with(conversation_pagination_model)
@edit_permission_required @edit_permission_required
def get(self, app_model): def get(self, app_model):
current_user, _ = current_account_with_tenant() current_user, _ = current_account_with_tenant()
args = CompletionConversationQuery.model_validate(request.args.to_dict(flat=True)) args = CompletionConversationQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
query = sa.select(Conversation).where( query = sa.select(Conversation).where(
Conversation.app_id == app_model.id, Conversation.mode == "completion", Conversation.is_deleted.is_(False) Conversation.app_id == app_model.id, Conversation.mode == "completion", Conversation.is_deleted.is_(False)
@ -150,9 +394,7 @@ class CompletionConversationApi(Resource):
conversations = db.paginate(query, page=args.page, per_page=args.limit, error_out=False) conversations = db.paginate(query, page=args.page, per_page=args.limit, error_out=False)
return ConversationPaginationResponse.model_validate(conversations, from_attributes=True).model_dump( return conversations
mode="json"
)
@console_ns.route("/apps/<uuid:app_id>/completion-conversations/<uuid:conversation_id>") @console_ns.route("/apps/<uuid:app_id>/completion-conversations/<uuid:conversation_id>")
@ -160,19 +402,19 @@ class CompletionConversationDetailApi(Resource):
@console_ns.doc("get_completion_conversation") @console_ns.doc("get_completion_conversation")
@console_ns.doc(description="Get completion conversation details with messages") @console_ns.doc(description="Get completion conversation details with messages")
@console_ns.doc(params={"app_id": "Application ID", "conversation_id": "Conversation ID"}) @console_ns.doc(params={"app_id": "Application ID", "conversation_id": "Conversation ID"})
@console_ns.response(200, "Success", console_ns.models[ConversationMessageDetailResponse.__name__]) @console_ns.response(200, "Success", conversation_message_detail_model)
@console_ns.response(403, "Insufficient permissions") @console_ns.response(403, "Insufficient permissions")
@console_ns.response(404, "Conversation not found") @console_ns.response(404, "Conversation not found")
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=AppMode.COMPLETION) @get_app_model(mode=AppMode.COMPLETION)
@marshal_with(conversation_message_detail_model)
@edit_permission_required @edit_permission_required
def get(self, app_model, conversation_id): def get(self, app_model, conversation_id):
conversation_id = str(conversation_id) conversation_id = str(conversation_id)
return ConversationMessageDetailResponse.model_validate(
_get_conversation(app_model, conversation_id), from_attributes=True return _get_conversation(app_model, conversation_id)
).model_dump(mode="json")
@console_ns.doc("delete_completion_conversation") @console_ns.doc("delete_completion_conversation")
@console_ns.doc(description="Delete a completion conversation") @console_ns.doc(description="Delete a completion conversation")
@ -194,7 +436,7 @@ class CompletionConversationDetailApi(Resource):
except ConversationNotExistsError: except ConversationNotExistsError:
raise NotFound("Conversation Not Exists.") raise NotFound("Conversation Not Exists.")
return ResultResponse(result="success").model_dump(mode="json"), 204 return {"result": "success"}, 204
@console_ns.route("/apps/<uuid:app_id>/chat-conversations") @console_ns.route("/apps/<uuid:app_id>/chat-conversations")
@ -203,16 +445,17 @@ class ChatConversationApi(Resource):
@console_ns.doc(description="Get chat conversations with pagination, filtering and summary") @console_ns.doc(description="Get chat conversations with pagination, filtering and summary")
@console_ns.doc(params={"app_id": "Application ID"}) @console_ns.doc(params={"app_id": "Application ID"})
@console_ns.expect(console_ns.models[ChatConversationQuery.__name__]) @console_ns.expect(console_ns.models[ChatConversationQuery.__name__])
@console_ns.response(200, "Success", console_ns.models[ConversationWithSummaryPaginationResponse.__name__]) @console_ns.response(200, "Success", conversation_with_summary_pagination_model)
@console_ns.response(403, "Insufficient permissions") @console_ns.response(403, "Insufficient permissions")
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT]) @get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT])
@marshal_with(conversation_with_summary_pagination_model)
@edit_permission_required @edit_permission_required
def get(self, app_model): def get(self, app_model):
current_user, _ = current_account_with_tenant() current_user, _ = current_account_with_tenant()
args = ChatConversationQuery.model_validate(request.args.to_dict(flat=True)) args = ChatConversationQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
subquery = ( subquery = (
sa.select(Conversation.id.label("conversation_id"), EndUser.session_id.label("from_end_user_session_id")) sa.select(Conversation.id.label("conversation_id"), EndUser.session_id.label("from_end_user_session_id"))
@ -303,9 +546,7 @@ class ChatConversationApi(Resource):
conversations = db.paginate(query, page=args.page, per_page=args.limit, error_out=False) conversations = db.paginate(query, page=args.page, per_page=args.limit, error_out=False)
return ConversationWithSummaryPaginationResponse.model_validate(conversations, from_attributes=True).model_dump( return conversations
mode="json"
)
@console_ns.route("/apps/<uuid:app_id>/chat-conversations/<uuid:conversation_id>") @console_ns.route("/apps/<uuid:app_id>/chat-conversations/<uuid:conversation_id>")
@ -313,19 +554,19 @@ class ChatConversationDetailApi(Resource):
@console_ns.doc("get_chat_conversation") @console_ns.doc("get_chat_conversation")
@console_ns.doc(description="Get chat conversation details") @console_ns.doc(description="Get chat conversation details")
@console_ns.doc(params={"app_id": "Application ID", "conversation_id": "Conversation ID"}) @console_ns.doc(params={"app_id": "Application ID", "conversation_id": "Conversation ID"})
@console_ns.response(200, "Success", console_ns.models[ConversationDetailResponse.__name__]) @console_ns.response(200, "Success", conversation_detail_model)
@console_ns.response(403, "Insufficient permissions") @console_ns.response(403, "Insufficient permissions")
@console_ns.response(404, "Conversation not found") @console_ns.response(404, "Conversation not found")
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT]) @get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT])
@marshal_with(conversation_detail_model)
@edit_permission_required @edit_permission_required
def get(self, app_model, conversation_id): def get(self, app_model, conversation_id):
conversation_id = str(conversation_id) conversation_id = str(conversation_id)
return ConversationDetailResponse.model_validate(
_get_conversation(app_model, conversation_id), from_attributes=True return _get_conversation(app_model, conversation_id)
).model_dump(mode="json")
@console_ns.doc("delete_chat_conversation") @console_ns.doc("delete_chat_conversation")
@console_ns.doc(description="Delete a chat conversation") @console_ns.doc(description="Delete a chat conversation")
@ -347,7 +588,7 @@ class ChatConversationDetailApi(Resource):
except ConversationNotExistsError: except ConversationNotExistsError:
raise NotFound("Conversation Not Exists.") raise NotFound("Conversation Not Exists.")
return ResultResponse(result="success").model_dump(mode="json"), 204 return {"result": "success"}, 204
def _get_conversation(app_model, conversation_id): def _get_conversation(app_model, conversation_id):

View File

@ -1,86 +1,44 @@
from __future__ import annotations
from datetime import datetime
from typing import Any
from flask import request from flask import request
from flask_restx import Resource from flask_restx import Resource, fields, marshal_with
from pydantic import BaseModel, Field, field_validator from pydantic import BaseModel, Field
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.orm import sessionmaker from sqlalchemy.orm import sessionmaker
from controllers.common.schema import register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
from extensions.ext_database import db from extensions.ext_database import db
from fields._value_type_serializer import serialize_value_type from fields.conversation_variable_fields import (
from fields.base import ResponseModel conversation_variable_fields,
paginated_conversation_variable_fields,
)
from libs.login import login_required from libs.login import login_required
from models import ConversationVariable from models import ConversationVariable
from models.model import AppMode from models.model import AppMode
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
class ConversationVariablesQuery(BaseModel): class ConversationVariablesQuery(BaseModel):
conversation_id: str = Field(..., description="Conversation ID to filter variables") conversation_id: str = Field(..., description="Conversation ID to filter variables")
def _to_timestamp(value: datetime | int | None) -> int | None: console_ns.schema_model(
if isinstance(value, datetime): ConversationVariablesQuery.__name__,
return int(value.timestamp()) ConversationVariablesQuery.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
return value )
# Register models for flask_restx to avoid dict type issues in Swagger
# Register base model first
conversation_variable_model = console_ns.model("ConversationVariable", conversation_variable_fields)
class ConversationVariableResponse(ResponseModel): # For nested models, need to replace nested dict with registered model
id: str paginated_conversation_variable_fields_copy = paginated_conversation_variable_fields.copy()
name: str paginated_conversation_variable_fields_copy["data"] = fields.List(
value_type: str fields.Nested(conversation_variable_model), attribute="data"
value: str | None = None )
description: str | None = None paginated_conversation_variable_model = console_ns.model(
created_at: int | None = None "PaginatedConversationVariable", paginated_conversation_variable_fields_copy
updated_at: int | None = None
@field_validator("value_type", mode="before")
@classmethod
def _normalize_value_type(cls, value: Any) -> str:
exposed_type = getattr(value, "exposed_type", None)
if callable(exposed_type):
return str(exposed_type())
if isinstance(value, str):
return value
try:
return serialize_value_type(value)
except Exception:
return serialize_value_type({"value_type": value})
@field_validator("value", mode="before")
@classmethod
def _normalize_value(cls, value: Any | None) -> str | None:
if value is None:
return None
if isinstance(value, str):
return value
return str(value)
@field_validator("created_at", "updated_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
return _to_timestamp(value)
class PaginatedConversationVariableResponse(ResponseModel):
page: int
limit: int
total: int
has_more: bool
data: list[ConversationVariableResponse]
register_schema_models(
console_ns,
ConversationVariablesQuery,
ConversationVariableResponse,
PaginatedConversationVariableResponse,
) )
@ -90,17 +48,14 @@ class ConversationVariablesApi(Resource):
@console_ns.doc(description="Get conversation variables for an application") @console_ns.doc(description="Get conversation variables for an application")
@console_ns.doc(params={"app_id": "Application ID"}) @console_ns.doc(params={"app_id": "Application ID"})
@console_ns.expect(console_ns.models[ConversationVariablesQuery.__name__]) @console_ns.expect(console_ns.models[ConversationVariablesQuery.__name__])
@console_ns.response( @console_ns.response(200, "Conversation variables retrieved successfully", paginated_conversation_variable_model)
200,
"Conversation variables retrieved successfully",
console_ns.models[PaginatedConversationVariableResponse.__name__],
)
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=AppMode.ADVANCED_CHAT) @get_app_model(mode=AppMode.ADVANCED_CHAT)
@marshal_with(paginated_conversation_variable_model)
def get(self, app_model): def get(self, app_model):
args = ConversationVariablesQuery.model_validate(request.args.to_dict(flat=True)) args = ConversationVariablesQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
stmt = ( stmt = (
select(ConversationVariable) select(ConversationVariable)
@ -117,22 +72,17 @@ class ConversationVariablesApi(Resource):
with sessionmaker(db.engine, expire_on_commit=False).begin() as session: with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
rows = session.scalars(stmt).all() rows = session.scalars(stmt).all()
response = PaginatedConversationVariableResponse.model_validate( return {
{ "page": page,
"page": page, "limit": page_size,
"limit": page_size, "total": len(rows),
"total": len(rows), "has_more": False,
"has_more": False, "data": [
"data": [ {
ConversationVariableResponse.model_validate( "created_at": row.created_at,
{ "updated_at": row.updated_at,
"created_at": row.created_at, **row.to_variable().model_dump(),
"updated_at": row.updated_at, }
**row.to_variable().model_dump(), for row in rows
} ],
) }
for row in rows
],
}
)
return response.model_dump(mode="json")

View File

@ -1,9 +1,9 @@
from collections.abc import Sequence from collections.abc import Sequence
from flask_restx import Resource from flask_restx import Resource
from graphon.model_runtime.errors.invoke import InvokeError
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from controllers.common.schema import register_enum_models, register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.app.error import ( from controllers.console.app.error import (
CompletionRequestError, CompletionRequestError,
@ -20,12 +20,12 @@ from core.helper.code_executor.python3.python3_code_provider import Python3CodeP
from core.llm_generator.entities import RuleCodeGeneratePayload, RuleGeneratePayload, RuleStructuredOutputPayload from core.llm_generator.entities import RuleCodeGeneratePayload, RuleGeneratePayload, RuleStructuredOutputPayload
from core.llm_generator.llm_generator import LLMGenerator from core.llm_generator.llm_generator import LLMGenerator
from extensions.ext_database import db from extensions.ext_database import db
from graphon.model_runtime.entities.llm_entities import LLMMode
from graphon.model_runtime.errors.invoke import InvokeError
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models import App from models import App
from services.workflow_service import WorkflowService from services.workflow_service import WorkflowService
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
class InstructionGeneratePayload(BaseModel): class InstructionGeneratePayload(BaseModel):
flow_id: str = Field(..., description="Workflow/Flow ID") flow_id: str = Field(..., description="Workflow/Flow ID")
@ -41,16 +41,16 @@ class InstructionTemplatePayload(BaseModel):
type: str = Field(..., description="Instruction template type") type: str = Field(..., description="Instruction template type")
register_enum_models(console_ns, LLMMode) def reg(cls: type[BaseModel]):
register_schema_models( console_ns.schema_model(cls.__name__, cls.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0))
console_ns,
RuleGeneratePayload,
RuleCodeGeneratePayload, reg(RuleGeneratePayload)
RuleStructuredOutputPayload, reg(RuleCodeGeneratePayload)
InstructionGeneratePayload, reg(RuleStructuredOutputPayload)
InstructionTemplatePayload, reg(InstructionGeneratePayload)
ModelConfig, reg(InstructionTemplatePayload)
) reg(ModelConfig)
@console_ns.route("/rule-generate") @console_ns.route("/rule-generate")

View File

@ -18,37 +18,37 @@ from models.enums import AppMCPServerStatus
from models.model import AppMCPServer from models.model import AppMCPServer
def _to_timestamp(value: datetime | int | None) -> int | None:
if isinstance(value, datetime):
return int(value.timestamp())
return value
class MCPServerCreatePayload(BaseModel): class MCPServerCreatePayload(BaseModel):
description: str | None = Field(default=None, description="Server description") description: str | None = Field(default=None, description="Server description")
parameters: dict[str, Any] = Field(..., description="Server parameters configuration") parameters: dict = Field(..., description="Server parameters configuration")
class MCPServerUpdatePayload(BaseModel): class MCPServerUpdatePayload(BaseModel):
id: str = Field(..., description="Server ID") id: str = Field(..., description="Server ID")
description: str | None = Field(default=None, description="Server description") description: str | None = Field(default=None, description="Server description")
parameters: dict[str, Any] = Field(..., description="Server parameters configuration") parameters: dict = Field(..., description="Server parameters configuration")
status: str | None = Field(default=None, description="Server status") status: str | None = Field(default=None, description="Server status")
def _to_timestamp(value: datetime | int | None) -> int | None:
if isinstance(value, datetime):
return int(value.timestamp())
return value
class AppMCPServerResponse(ResponseModel): class AppMCPServerResponse(ResponseModel):
id: str id: str
name: str name: str
server_code: str server_code: str
description: str description: str
status: AppMCPServerStatus status: str
parameters: dict[str, Any] | list[Any] | str parameters: dict[str, Any] | list[Any] | str
created_at: int | None = None created_at: int | None = None
updated_at: int | None = None updated_at: int | None = None
@field_validator("parameters", mode="before") @field_validator("parameters", mode="before")
@classmethod @classmethod
def _normalize_parameters(cls, value: Any) -> Any: def _parse_json_string(cls, value: Any) -> Any:
if isinstance(value, str): if isinstance(value, str):
try: try:
return json.loads(value) return json.loads(value)
@ -70,9 +70,7 @@ class AppMCPServerController(Resource):
@console_ns.doc("get_app_mcp_server") @console_ns.doc("get_app_mcp_server")
@console_ns.doc(description="Get MCP server configuration for an application") @console_ns.doc(description="Get MCP server configuration for an application")
@console_ns.doc(params={"app_id": "Application ID"}) @console_ns.doc(params={"app_id": "Application ID"})
@console_ns.response( @console_ns.response(200, "Server configuration", console_ns.models[AppMCPServerResponse.__name__])
200, "MCP server configuration retrieved successfully", console_ns.models[AppMCPServerResponse.__name__]
)
@login_required @login_required
@account_initialization_required @account_initialization_required
@setup_required @setup_required
@ -87,9 +85,7 @@ class AppMCPServerController(Resource):
@console_ns.doc(description="Create MCP server configuration for an application") @console_ns.doc(description="Create MCP server configuration for an application")
@console_ns.doc(params={"app_id": "Application ID"}) @console_ns.doc(params={"app_id": "Application ID"})
@console_ns.expect(console_ns.models[MCPServerCreatePayload.__name__]) @console_ns.expect(console_ns.models[MCPServerCreatePayload.__name__])
@console_ns.response( @console_ns.response(200, "Server created", console_ns.models[AppMCPServerResponse.__name__])
201, "MCP server configuration created successfully", console_ns.models[AppMCPServerResponse.__name__]
)
@console_ns.response(403, "Insufficient permissions") @console_ns.response(403, "Insufficient permissions")
@account_initialization_required @account_initialization_required
@get_app_model @get_app_model
@ -115,15 +111,13 @@ class AppMCPServerController(Resource):
) )
db.session.add(server) db.session.add(server)
db.session.commit() db.session.commit()
return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json"), 201 return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
@console_ns.doc("update_app_mcp_server") @console_ns.doc("update_app_mcp_server")
@console_ns.doc(description="Update MCP server configuration for an application") @console_ns.doc(description="Update MCP server configuration for an application")
@console_ns.doc(params={"app_id": "Application ID"}) @console_ns.doc(params={"app_id": "Application ID"})
@console_ns.expect(console_ns.models[MCPServerUpdatePayload.__name__]) @console_ns.expect(console_ns.models[MCPServerUpdatePayload.__name__])
@console_ns.response( @console_ns.response(200, "Server updated", console_ns.models[AppMCPServerResponse.__name__])
200, "MCP server configuration updated successfully", console_ns.models[AppMCPServerResponse.__name__]
)
@console_ns.response(403, "Insufficient permissions") @console_ns.response(403, "Insufficient permissions")
@console_ns.response(404, "Server not found") @console_ns.response(404, "Server not found")
@get_app_model @get_app_model
@ -160,7 +154,7 @@ class AppMCPServerRefreshController(Resource):
@console_ns.doc("refresh_app_mcp_server") @console_ns.doc("refresh_app_mcp_server")
@console_ns.doc(description="Refresh MCP server configuration and regenerate server code") @console_ns.doc(description="Refresh MCP server configuration and regenerate server code")
@console_ns.doc(params={"server_id": "Server ID"}) @console_ns.doc(params={"server_id": "Server ID"})
@console_ns.response(200, "MCP server refreshed successfully", console_ns.models[AppMCPServerResponse.__name__]) @console_ns.response(200, "Server refreshed", console_ns.models[AppMCPServerResponse.__name__])
@console_ns.response(403, "Insufficient permissions") @console_ns.response(403, "Insufficient permissions")
@console_ns.response(404, "Server not found") @console_ns.response(404, "Server not found")
@setup_required @setup_required

View File

@ -1,9 +1,9 @@
import logging import logging
from datetime import datetime
from typing import Literal from typing import Literal
from flask import request from flask import request
from flask_restx import Resource from flask_restx import Resource, fields, marshal_with
from graphon.model_runtime.errors.invoke import InvokeError
from pydantic import BaseModel, Field, field_validator from pydantic import BaseModel, Field, field_validator
from sqlalchemy import exists, func, select from sqlalchemy import exists, func, select
from werkzeug.exceptions import InternalServerError, NotFound from werkzeug.exceptions import InternalServerError, NotFound
@ -25,22 +25,10 @@ from controllers.console.wraps import (
setup_required, setup_required,
) )
from core.app.entities.app_invoke_entities import InvokeFrom from core.app.entities.app_invoke_entities import InvokeFrom
from core.entities.execution_extra_content import ExecutionExtraContentDomainModel
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from extensions.ext_database import db from extensions.ext_database import db
from fields.base import ResponseModel from fields.raws import FilesContainedField
from fields.conversation_fields import ( from libs.helper import TimestampField, uuid_value
AgentThought,
ConversationAnnotation,
ConversationAnnotationHitHistory,
Feedback,
JSONValue,
MessageFile,
format_files_contained,
to_timestamp,
)
from graphon.model_runtime.errors.invoke import InvokeError
from libs.helper import uuid_value
from libs.infinite_scroll_pagination import InfiniteScrollPagination from libs.infinite_scroll_pagination import InfiniteScrollPagination
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models.enums import FeedbackFromSource, FeedbackRating from models.enums import FeedbackFromSource, FeedbackRating
@ -110,51 +98,6 @@ class SuggestedQuestionsResponse(BaseModel):
data: list[str] = Field(description="Suggested question") data: list[str] = Field(description="Suggested question")
class MessageDetailResponse(ResponseModel):
id: str
conversation_id: str
inputs: dict[str, JSONValue]
query: str
message: JSONValue | None = None
message_tokens: int | None = None
answer: str = Field(validation_alias="re_sign_file_url_answer")
answer_tokens: int | None = None
provider_response_latency: float | None = None
from_source: str
from_end_user_id: str | None = None
from_account_id: str | None = None
feedbacks: list[Feedback] = Field(default_factory=list)
workflow_run_id: str | None = None
annotation: ConversationAnnotation | None = None
annotation_hit_history: ConversationAnnotationHitHistory | None = None
created_at: int | None = None
agent_thoughts: list[AgentThought] = Field(default_factory=list)
message_files: list[MessageFile] = Field(default_factory=list)
extra_contents: list[ExecutionExtraContentDomainModel] = Field(default_factory=list)
metadata: JSONValue | None = Field(default=None, validation_alias="message_metadata_dict")
status: str
error: str | None = None
parent_message_id: str | None = None
@field_validator("inputs", mode="before")
@classmethod
def _normalize_inputs(cls, value: JSONValue) -> JSONValue:
return format_files_contained(value)
@field_validator("created_at", mode="before")
@classmethod
def _normalize_created_at(cls, value: datetime | int | None) -> int | None:
if isinstance(value, datetime):
return to_timestamp(value)
return value
class MessageInfiniteScrollPaginationResponse(ResponseModel):
limit: int
has_more: bool
data: list[MessageDetailResponse]
register_schema_models( register_schema_models(
console_ns, console_ns,
ChatMessagesQuery, ChatMessagesQuery,
@ -162,8 +105,124 @@ register_schema_models(
FeedbackExportQuery, FeedbackExportQuery,
AnnotationCountResponse, AnnotationCountResponse,
SuggestedQuestionsResponse, SuggestedQuestionsResponse,
MessageDetailResponse, )
MessageInfiniteScrollPaginationResponse,
# Register models for flask_restx to avoid dict type issues in Swagger
# Register in dependency order: base models first, then dependent models
# Base models
simple_account_model = console_ns.model(
"SimpleAccount",
{
"id": fields.String,
"name": fields.String,
"email": fields.String,
},
)
message_file_model = console_ns.model(
"MessageFile",
{
"id": fields.String,
"filename": fields.String,
"type": fields.String,
"url": fields.String,
"mime_type": fields.String,
"size": fields.Integer,
"transfer_method": fields.String,
"belongs_to": fields.String(default="user"),
"upload_file_id": fields.String(default=None),
},
)
agent_thought_model = console_ns.model(
"AgentThought",
{
"id": fields.String,
"chain_id": fields.String,
"message_id": fields.String,
"position": fields.Integer,
"thought": fields.String,
"tool": fields.String,
"tool_labels": fields.Raw,
"tool_input": fields.String,
"created_at": TimestampField,
"observation": fields.String,
"files": fields.List(fields.String),
},
)
# Models that depend on simple_account_model
feedback_model = console_ns.model(
"Feedback",
{
"rating": fields.String,
"content": fields.String,
"from_source": fields.String,
"from_end_user_id": fields.String,
"from_account": fields.Nested(simple_account_model, allow_null=True),
},
)
annotation_model = console_ns.model(
"Annotation",
{
"id": fields.String,
"question": fields.String,
"content": fields.String,
"account": fields.Nested(simple_account_model, allow_null=True),
"created_at": TimestampField,
},
)
annotation_hit_history_model = console_ns.model(
"AnnotationHitHistory",
{
"annotation_id": fields.String(attribute="id"),
"annotation_create_account": fields.Nested(simple_account_model, allow_null=True),
"created_at": TimestampField,
},
)
# Message detail model that depends on multiple models
message_detail_model = console_ns.model(
"MessageDetail",
{
"id": fields.String,
"conversation_id": fields.String,
"inputs": FilesContainedField,
"query": fields.String,
"message": fields.Raw,
"message_tokens": fields.Integer,
"answer": fields.String(attribute="re_sign_file_url_answer"),
"answer_tokens": fields.Integer,
"provider_response_latency": fields.Float,
"from_source": fields.String,
"from_end_user_id": fields.String,
"from_account_id": fields.String,
"feedbacks": fields.List(fields.Nested(feedback_model)),
"workflow_run_id": fields.String,
"annotation": fields.Nested(annotation_model, allow_null=True),
"annotation_hit_history": fields.Nested(annotation_hit_history_model, allow_null=True),
"created_at": TimestampField,
"agent_thoughts": fields.List(fields.Nested(agent_thought_model)),
"message_files": fields.List(fields.Nested(message_file_model)),
"extra_contents": fields.List(fields.Raw),
"metadata": fields.Raw(attribute="message_metadata_dict"),
"status": fields.String,
"error": fields.String,
"parent_message_id": fields.String,
},
)
# Message infinite scroll pagination model
message_infinite_scroll_pagination_model = console_ns.model(
"MessageInfiniteScrollPagination",
{
"limit": fields.Integer,
"has_more": fields.Boolean,
"data": fields.List(fields.Nested(message_detail_model)),
},
) )
@ -173,12 +232,13 @@ class ChatMessageListApi(Resource):
@console_ns.doc(description="Get chat messages for a conversation with pagination") @console_ns.doc(description="Get chat messages for a conversation with pagination")
@console_ns.doc(params={"app_id": "Application ID"}) @console_ns.doc(params={"app_id": "Application ID"})
@console_ns.expect(console_ns.models[ChatMessagesQuery.__name__]) @console_ns.expect(console_ns.models[ChatMessagesQuery.__name__])
@console_ns.response(200, "Success", console_ns.models[MessageInfiniteScrollPaginationResponse.__name__]) @console_ns.response(200, "Success", message_infinite_scroll_pagination_model)
@console_ns.response(404, "Conversation not found") @console_ns.response(404, "Conversation not found")
@login_required @login_required
@account_initialization_required @account_initialization_required
@setup_required @setup_required
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT]) @get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
@marshal_with(message_infinite_scroll_pagination_model)
@edit_permission_required @edit_permission_required
def get(self, app_model): def get(self, app_model):
args = ChatMessagesQuery.model_validate(request.args.to_dict()) args = ChatMessagesQuery.model_validate(request.args.to_dict())
@ -238,10 +298,7 @@ class ChatMessageListApi(Resource):
history_messages = list(reversed(history_messages)) history_messages = list(reversed(history_messages))
attach_message_extra_contents(history_messages) attach_message_extra_contents(history_messages)
return MessageInfiniteScrollPaginationResponse.model_validate( return InfiniteScrollPagination(data=history_messages, limit=args.limit, has_more=has_more)
InfiniteScrollPagination(data=history_messages, limit=args.limit, has_more=has_more),
from_attributes=True,
).model_dump(mode="json")
@console_ns.route("/apps/<uuid:app_id>/feedbacks") @console_ns.route("/apps/<uuid:app_id>/feedbacks")
@ -336,7 +393,7 @@ class MessageSuggestedQuestionApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT]) @get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
def get(self, app_model, message_id): def get(self, app_model, message_id):
current_user, _ = current_account_with_tenant() current_user, _ = current_account_with_tenant()
message_id = str(message_id) message_id = str(message_id)
@ -411,12 +468,13 @@ class MessageApi(Resource):
@console_ns.doc("get_message") @console_ns.doc("get_message")
@console_ns.doc(description="Get message details by ID") @console_ns.doc(description="Get message details by ID")
@console_ns.doc(params={"app_id": "Application ID", "message_id": "Message ID"}) @console_ns.doc(params={"app_id": "Application ID", "message_id": "Message ID"})
@console_ns.response(200, "Message retrieved successfully", console_ns.models[MessageDetailResponse.__name__]) @console_ns.response(200, "Message retrieved successfully", message_detail_model)
@console_ns.response(404, "Message not found") @console_ns.response(404, "Message not found")
@get_app_model @get_app_model
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@marshal_with(message_detail_model)
def get(self, app_model, message_id: str): def get(self, app_model, message_id: str):
message_id = str(message_id) message_id = str(message_id)
@ -428,4 +486,4 @@ class MessageApi(Resource):
raise NotFound("Message Not Exists.") raise NotFound("Message Not Exists.")
attach_message_extra_contents([message]) attach_message_extra_contents([message])
return MessageDetailResponse.model_validate(message, from_attributes=True).model_dump(mode="json") return message

View File

@ -1,18 +1,18 @@
from typing import Any from typing import Any
from uuid import UUID
from flask import request from flask import request
from flask_restx import Resource, fields from flask_restx import Resource, fields
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from werkzeug.exceptions import BadRequest from werkzeug.exceptions import BadRequest
from controllers.common.schema import register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.app.error import TracingConfigCheckError, TracingConfigIsExist, TracingConfigNotExist from controllers.console.app.error import TracingConfigCheckError, TracingConfigIsExist, TracingConfigNotExist
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
from libs.login import login_required from libs.login import login_required
from services.ops_service import OpsService from services.ops_service import OpsService
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
class TraceProviderQuery(BaseModel): class TraceProviderQuery(BaseModel):
tracing_provider: str = Field(..., description="Tracing provider name") tracing_provider: str = Field(..., description="Tracing provider name")
@ -23,7 +23,13 @@ class TraceConfigPayload(BaseModel):
tracing_config: dict[str, Any] = Field(..., description="Tracing configuration data") tracing_config: dict[str, Any] = Field(..., description="Tracing configuration data")
register_schema_models(console_ns, TraceProviderQuery, TraceConfigPayload) console_ns.schema_model(
TraceProviderQuery.__name__,
TraceProviderQuery.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
)
console_ns.schema_model(
TraceConfigPayload.__name__, TraceConfigPayload.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0)
)
@console_ns.route("/apps/<uuid:app_id>/trace-config") @console_ns.route("/apps/<uuid:app_id>/trace-config")
@ -43,11 +49,11 @@ class TraceAppConfigApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
def get(self, app_id: UUID): def get(self, app_id):
args = TraceProviderQuery.model_validate(request.args.to_dict(flat=True)) args = TraceProviderQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
try: try:
trace_config = OpsService.get_tracing_app_config(app_id=str(app_id), tracing_provider=args.tracing_provider) trace_config = OpsService.get_tracing_app_config(app_id=app_id, tracing_provider=args.tracing_provider)
if not trace_config: if not trace_config:
return {"has_not_configured": True} return {"has_not_configured": True}
return trace_config return trace_config
@ -65,13 +71,13 @@ class TraceAppConfigApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
def post(self, app_id: UUID): def post(self, app_id):
"""Create a new trace app configuration""" """Create a new trace app configuration"""
args = TraceConfigPayload.model_validate(console_ns.payload) args = TraceConfigPayload.model_validate(console_ns.payload)
try: try:
result = OpsService.create_tracing_app_config( result = OpsService.create_tracing_app_config(
app_id=str(app_id), tracing_provider=args.tracing_provider, tracing_config=args.tracing_config app_id=app_id, tracing_provider=args.tracing_provider, tracing_config=args.tracing_config
) )
if not result: if not result:
raise TracingConfigIsExist() raise TracingConfigIsExist()
@ -90,13 +96,13 @@ class TraceAppConfigApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
def patch(self, app_id: UUID): def patch(self, app_id):
"""Update an existing trace app configuration""" """Update an existing trace app configuration"""
args = TraceConfigPayload.model_validate(console_ns.payload) args = TraceConfigPayload.model_validate(console_ns.payload)
try: try:
result = OpsService.update_tracing_app_config( result = OpsService.update_tracing_app_config(
app_id=str(app_id), tracing_provider=args.tracing_provider, tracing_config=args.tracing_config app_id=app_id, tracing_provider=args.tracing_provider, tracing_config=args.tracing_config
) )
if not result: if not result:
raise TracingConfigNotExist() raise TracingConfigNotExist()
@ -113,12 +119,12 @@ class TraceAppConfigApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
def delete(self, app_id: UUID): def delete(self, app_id):
"""Delete an existing trace app configuration""" """Delete an existing trace app configuration"""
args = TraceProviderQuery.model_validate(request.args.to_dict(flat=True)) args = TraceProviderQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
try: try:
result = OpsService.delete_tracing_app_config(app_id=str(app_id), tracing_provider=args.tracing_provider) result = OpsService.delete_tracing_app_config(app_id=app_id, tracing_provider=args.tracing_provider)
if not result: if not result:
raise TracingConfigNotExist() raise TracingConfigNotExist()
return {"result": "success"}, 204 return {"result": "success"}, 204

View File

@ -5,7 +5,6 @@ from flask import abort, jsonify, request
from flask_restx import Resource, fields from flask_restx import Resource, fields
from pydantic import BaseModel, Field, field_validator from pydantic import BaseModel, Field, field_validator
from controllers.common.schema import register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
@ -16,6 +15,8 @@ from libs.helper import convert_datetime_to_date
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models import AppMode from models import AppMode
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
class StatisticTimeRangeQuery(BaseModel): class StatisticTimeRangeQuery(BaseModel):
start: str | None = Field(default=None, description="Start date (YYYY-MM-DD HH:MM)") start: str | None = Field(default=None, description="Start date (YYYY-MM-DD HH:MM)")
@ -29,7 +30,10 @@ class StatisticTimeRangeQuery(BaseModel):
return value return value
register_schema_models(console_ns, StatisticTimeRangeQuery) console_ns.schema_model(
StatisticTimeRangeQuery.__name__,
StatisticTimeRangeQuery.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
)
@console_ns.route("/apps/<uuid:app_id>/statistics/daily-messages") @console_ns.route("/apps/<uuid:app_id>/statistics/daily-messages")
@ -50,7 +54,7 @@ class DailyMessageStatistic(Resource):
def get(self, app_model): def get(self, app_model):
account, _ = current_account_with_tenant() account, _ = current_account_with_tenant()
args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
converted_created_at = convert_datetime_to_date("created_at") converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT sql_query = f"""SELECT
@ -107,7 +111,7 @@ class DailyConversationStatistic(Resource):
def get(self, app_model): def get(self, app_model):
account, _ = current_account_with_tenant() account, _ = current_account_with_tenant()
args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
converted_created_at = convert_datetime_to_date("created_at") converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT sql_query = f"""SELECT
@ -163,7 +167,7 @@ class DailyTerminalsStatistic(Resource):
def get(self, app_model): def get(self, app_model):
account, _ = current_account_with_tenant() account, _ = current_account_with_tenant()
args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
converted_created_at = convert_datetime_to_date("created_at") converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT sql_query = f"""SELECT
@ -220,7 +224,7 @@ class DailyTokenCostStatistic(Resource):
def get(self, app_model): def get(self, app_model):
account, _ = current_account_with_tenant() account, _ = current_account_with_tenant()
args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
converted_created_at = convert_datetime_to_date("created_at") converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT sql_query = f"""SELECT
@ -280,7 +284,7 @@ class AverageSessionInteractionStatistic(Resource):
def get(self, app_model): def get(self, app_model):
account, _ = current_account_with_tenant() account, _ = current_account_with_tenant()
args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
converted_created_at = convert_datetime_to_date("c.created_at") converted_created_at = convert_datetime_to_date("c.created_at")
sql_query = f"""SELECT sql_query = f"""SELECT
@ -356,7 +360,7 @@ class UserSatisfactionRateStatistic(Resource):
def get(self, app_model): def get(self, app_model):
account, _ = current_account_with_tenant() account, _ = current_account_with_tenant()
args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
converted_created_at = convert_datetime_to_date("m.created_at") converted_created_at = convert_datetime_to_date("m.created_at")
sql_query = f"""SELECT sql_query = f"""SELECT
@ -422,7 +426,7 @@ class AverageResponseTimeStatistic(Resource):
def get(self, app_model): def get(self, app_model):
account, _ = current_account_with_tenant() account, _ = current_account_with_tenant()
args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
converted_created_at = convert_datetime_to_date("created_at") converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT sql_query = f"""SELECT
@ -478,7 +482,7 @@ class TokensPerSecondStatistic(Resource):
@account_initialization_required @account_initialization_required
def get(self, app_model): def get(self, app_model):
account, _ = current_account_with_tenant() account, _ = current_account_with_tenant()
args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) args = StatisticTimeRangeQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
converted_created_at = convert_datetime_to_date("created_at") converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT sql_query = f"""SELECT

View File

@ -4,16 +4,20 @@ from collections.abc import Sequence
from typing import Any from typing import Any
from flask import abort, request from flask import abort, request
from flask_restx import Resource, fields, marshal, marshal_with from flask_restx import Resource, fields, marshal_with
from graphon.enums import NodeType
from graphon.file import File
from graphon.graph_engine.manager import GraphEngineManager
from graphon.model_runtime.utils.encoders import jsonable_encoder
from pydantic import BaseModel, Field, ValidationError, field_validator from pydantic import BaseModel, Field, ValidationError, field_validator
from sqlalchemy.orm import sessionmaker from sqlalchemy.orm import sessionmaker
from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotFound from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotFound
import services import services
from controllers.common.controller_schemas import DefaultBlockConfigQuery, WorkflowListQuery, WorkflowUpdatePayload from controllers.common.controller_schemas import DefaultBlockConfigQuery, WorkflowListQuery, WorkflowUpdatePayload
from controllers.common.schema import register_response_schema_model, register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.app.error import ConversationCompletedError, DraftWorkflowNotExist, DraftWorkflowNotSync from controllers.console.app.error import ConversationCompletedError, DraftWorkflowNotExist, DraftWorkflowNotSync
from controllers.console.app.workflow_run import workflow_run_node_execution_model
from controllers.console.app.wraps import get_app_model from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpError from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpError
@ -35,14 +39,7 @@ from extensions.ext_database import db
from extensions.ext_redis import redis_client from extensions.ext_redis import redis_client
from factories import file_factory, variable_factory from factories import file_factory, variable_factory
from fields.member_fields import simple_account_fields from fields.member_fields import simple_account_fields
from fields.online_user_fields import online_user_list_fields
from fields.workflow_fields import workflow_fields, workflow_pagination_fields from fields.workflow_fields import workflow_fields, workflow_pagination_fields
from fields.workflow_run_fields import WorkflowRunNodeExecutionResponse
from graphon.enums import NodeType
from graphon.file import File
from graphon.file import helpers as file_helpers
from graphon.graph_engine.manager import GraphEngineManager
from graphon.model_runtime.utils.encoders import jsonable_encoder
from libs import helper from libs import helper
from libs.datetime_utils import naive_utc_now from libs.datetime_utils import naive_utc_now
from libs.helper import TimestampField, uuid_value from libs.helper import TimestampField, uuid_value
@ -50,20 +47,16 @@ from libs.login import current_account_with_tenant, login_required
from models import App from models import App
from models.model import AppMode from models.model import AppMode
from models.workflow import Workflow from models.workflow import Workflow
from repositories.workflow_collaboration_repository import WORKFLOW_ONLINE_USERS_PREFIX
from services.app_generate_service import AppGenerateService from services.app_generate_service import AppGenerateService
from services.errors.app import IsDraftWorkflowError, WorkflowHashNotEqualError, WorkflowNotFoundError from services.errors.app import IsDraftWorkflowError, WorkflowHashNotEqualError, WorkflowNotFoundError
from services.errors.llm import InvokeRateLimitError from services.errors.llm import InvokeRateLimitError
from services.workflow_service import DraftWorkflowDeletionError, WorkflowInUseError, WorkflowService from services.workflow_service import DraftWorkflowDeletionError, WorkflowInUseError, WorkflowService
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
_file_access_controller = DatabaseFileAccessController() _file_access_controller = DatabaseFileAccessController()
LISTENING_RETRY_IN = 2000 LISTENING_RETRY_IN = 2000
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
RESTORE_SOURCE_WORKFLOW_MUST_BE_PUBLISHED_MESSAGE = "source workflow must be published" RESTORE_SOURCE_WORKFLOW_MUST_BE_PUBLISHED_MESSAGE = "source workflow must be published"
MAX_WORKFLOW_ONLINE_USERS_REQUEST_IDS = 1000
WORKFLOW_ONLINE_USERS_REDIS_BATCH_SIZE = 50
# Register models for flask_restx to avoid dict type issues in Swagger # Register models for flask_restx to avoid dict type issues in Swagger
# Register in dependency order: base models first, then dependent models # Register in dependency order: base models first, then dependent models
@ -157,19 +150,6 @@ class ConvertToWorkflowPayload(BaseModel):
icon_background: str | None = None icon_background: str | None = None
class WorkflowFeaturesPayload(BaseModel):
features: dict[str, Any] = Field(..., description="Workflow feature configuration")
class WorkflowOnlineUsersPayload(BaseModel):
app_ids: list[str] = Field(default_factory=list, description="App IDs")
@field_validator("app_ids")
@classmethod
def normalize_app_ids(cls, app_ids: list[str]) -> list[str]:
return list(dict.fromkeys(app_id.strip() for app_id in app_ids if app_id.strip()))
class DraftWorkflowTriggerRunPayload(BaseModel): class DraftWorkflowTriggerRunPayload(BaseModel):
node_id: str node_id: str
@ -178,25 +158,23 @@ class DraftWorkflowTriggerRunAllPayload(BaseModel):
node_ids: list[str] node_ids: list[str]
register_schema_models( def reg(cls: type[BaseModel]):
console_ns, console_ns.schema_model(cls.__name__, cls.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0))
SyncDraftWorkflowPayload,
AdvancedChatWorkflowRunPayload,
IterationNodeRunPayload, reg(SyncDraftWorkflowPayload)
LoopNodeRunPayload, reg(AdvancedChatWorkflowRunPayload)
DraftWorkflowRunPayload, reg(IterationNodeRunPayload)
DraftWorkflowNodeRunPayload, reg(LoopNodeRunPayload)
PublishWorkflowPayload, reg(DraftWorkflowRunPayload)
DefaultBlockConfigQuery, reg(DraftWorkflowNodeRunPayload)
ConvertToWorkflowPayload, reg(PublishWorkflowPayload)
WorkflowListQuery, reg(DefaultBlockConfigQuery)
WorkflowUpdatePayload, reg(ConvertToWorkflowPayload)
WorkflowFeaturesPayload, reg(WorkflowListQuery)
WorkflowOnlineUsersPayload, reg(WorkflowUpdatePayload)
DraftWorkflowTriggerRunPayload, reg(DraftWorkflowTriggerRunPayload)
DraftWorkflowTriggerRunAllPayload, reg(DraftWorkflowTriggerRunAllPayload)
)
register_response_schema_model(console_ns, WorkflowRunNodeExecutionResponse)
# TODO(QuantumGhost): Refactor existing node run API to handle file parameter parsing # TODO(QuantumGhost): Refactor existing node run API to handle file parameter parsing
@ -228,7 +206,7 @@ class DraftWorkflowApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@marshal_with(workflow_model) @marshal_with(workflow_model)
@edit_permission_required @edit_permission_required
def get(self, app_model: App): def get(self, app_model: App):
@ -248,7 +226,7 @@ class DraftWorkflowApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@console_ns.doc("sync_draft_workflow") @console_ns.doc("sync_draft_workflow")
@console_ns.doc(description="Sync draft workflow configuration") @console_ns.doc(description="Sync draft workflow configuration")
@console_ns.expect(console_ns.models[SyncDraftWorkflowPayload.__name__]) @console_ns.expect(console_ns.models[SyncDraftWorkflowPayload.__name__])
@ -332,7 +310,7 @@ class AdvancedChatDraftWorkflowRunApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def post(self, app_model: App): def post(self, app_model: App):
""" """
@ -378,7 +356,7 @@ class AdvancedChatDraftRunIterationNodeApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def post(self, app_model: App, node_id: str): def post(self, app_model: App, node_id: str):
""" """
@ -454,7 +432,7 @@ class AdvancedChatDraftRunLoopNodeApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def post(self, app_model: App, node_id: str): def post(self, app_model: App, node_id: str):
""" """
@ -542,12 +520,9 @@ class HumanInputDeliveryTestPayload(BaseModel):
) )
register_schema_models( reg(HumanInputFormPreviewPayload)
console_ns, reg(HumanInputFormSubmitPayload)
HumanInputFormPreviewPayload, reg(HumanInputDeliveryTestPayload)
HumanInputFormSubmitPayload,
HumanInputDeliveryTestPayload,
)
@console_ns.route("/apps/<uuid:app_id>/advanced-chat/workflows/draft/human-input/nodes/<string:node_id>/form/preview") @console_ns.route("/apps/<uuid:app_id>/advanced-chat/workflows/draft/human-input/nodes/<string:node_id>/form/preview")
@ -559,7 +534,7 @@ class AdvancedChatDraftHumanInputFormPreviewApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def post(self, app_model: App, node_id: str): def post(self, app_model: App, node_id: str):
""" """
@ -588,7 +563,7 @@ class AdvancedChatDraftHumanInputFormRunApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def post(self, app_model: App, node_id: str): def post(self, app_model: App, node_id: str):
""" """
@ -743,7 +718,7 @@ class WorkflowTaskStopApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def post(self, app_model: App, task_id: str): def post(self, app_model: App, task_id: str):
""" """
@ -765,17 +740,14 @@ class DraftWorkflowNodeRunApi(Resource):
@console_ns.doc(description="Run draft workflow node") @console_ns.doc(description="Run draft workflow node")
@console_ns.doc(params={"app_id": "Application ID", "node_id": "Node ID"}) @console_ns.doc(params={"app_id": "Application ID", "node_id": "Node ID"})
@console_ns.expect(console_ns.models[DraftWorkflowNodeRunPayload.__name__]) @console_ns.expect(console_ns.models[DraftWorkflowNodeRunPayload.__name__])
@console_ns.response( @console_ns.response(200, "Node run started successfully", workflow_run_node_execution_model)
200,
"Node run started successfully",
console_ns.models[WorkflowRunNodeExecutionResponse.__name__],
)
@console_ns.response(403, "Permission denied") @console_ns.response(403, "Permission denied")
@console_ns.response(404, "Node not found") @console_ns.response(404, "Node not found")
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@marshal_with(workflow_run_node_execution_model)
@edit_permission_required @edit_permission_required
def post(self, app_model: App, node_id: str): def post(self, app_model: App, node_id: str):
""" """
@ -807,9 +779,7 @@ class DraftWorkflowNodeRunApi(Resource):
files=files, files=files,
) )
return WorkflowRunNodeExecutionResponse.model_validate( return workflow_node_execution
workflow_node_execution, from_attributes=True
).model_dump(mode="json")
@console_ns.route("/apps/<uuid:app_id>/workflows/publish") @console_ns.route("/apps/<uuid:app_id>/workflows/publish")
@ -822,7 +792,7 @@ class PublishedWorkflowApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@marshal_with(workflow_model) @marshal_with(workflow_model)
@edit_permission_required @edit_permission_required
def get(self, app_model: App): def get(self, app_model: App):
@ -840,7 +810,7 @@ class PublishedWorkflowApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def post(self, app_model: App): def post(self, app_model: App):
""" """
@ -884,7 +854,7 @@ class DefaultBlockConfigsApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def get(self, app_model: App): def get(self, app_model: App):
""" """
@ -906,13 +876,13 @@ class DefaultBlockConfigApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def get(self, app_model: App, block_type: str): def get(self, app_model: App, block_type: str):
""" """
Get default block config Get default block config
""" """
args = DefaultBlockConfigQuery.model_validate(request.args.to_dict(flat=True)) args = DefaultBlockConfigQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
filters = None filters = None
if args.q: if args.q:
@ -961,32 +931,6 @@ class ConvertToWorkflowApi(Resource):
} }
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/features")
class WorkflowFeaturesApi(Resource):
"""Update draft workflow features."""
@console_ns.expect(console_ns.models[WorkflowFeaturesPayload.__name__])
@console_ns.doc("update_workflow_features")
@console_ns.doc(description="Update draft workflow features")
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.response(200, "Workflow features updated successfully")
@setup_required
@login_required
@account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
@edit_permission_required
def post(self, app_model: App):
current_user, _ = current_account_with_tenant()
args = WorkflowFeaturesPayload.model_validate(console_ns.payload or {})
features = args.features
workflow_service = WorkflowService()
workflow_service.update_draft_workflow_features(app_model=app_model, features=features, account=current_user)
return {"result": "success"}
@console_ns.route("/apps/<uuid:app_id>/workflows") @console_ns.route("/apps/<uuid:app_id>/workflows")
class PublishedAllWorkflowApi(Resource): class PublishedAllWorkflowApi(Resource):
@console_ns.expect(console_ns.models[WorkflowListQuery.__name__]) @console_ns.expect(console_ns.models[WorkflowListQuery.__name__])
@ -997,7 +941,8 @@ class PublishedAllWorkflowApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@marshal_with(workflow_pagination_model)
@edit_permission_required @edit_permission_required
def get(self, app_model: App): def get(self, app_model: App):
""" """
@ -1005,7 +950,7 @@ class PublishedAllWorkflowApi(Resource):
""" """
current_user, _ = current_account_with_tenant() current_user, _ = current_account_with_tenant()
args = WorkflowListQuery.model_validate(request.args.to_dict(flat=True)) args = WorkflowListQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
page = args.page page = args.page
limit = args.limit limit = args.limit
user_id = args.user_id user_id = args.user_id
@ -1025,10 +970,9 @@ class PublishedAllWorkflowApi(Resource):
user_id=user_id, user_id=user_id,
named_only=named_only, named_only=named_only,
) )
serialized_workflows = marshal(workflows, workflow_fields_copy)
return { return {
"items": serialized_workflows, "items": workflows,
"page": page, "page": page,
"limit": limit, "limit": limit,
"has_more": has_more, "has_more": has_more,
@ -1046,7 +990,7 @@ class DraftWorkflowRestoreApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def post(self, app_model: App, workflow_id: str): def post(self, app_model: App, workflow_id: str):
current_user, _ = current_account_with_tenant() current_user, _ = current_account_with_tenant()
@ -1084,7 +1028,7 @@ class WorkflowByIdApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@marshal_with(workflow_model) @marshal_with(workflow_model)
@edit_permission_required @edit_permission_required
def patch(self, app_model: App, workflow_id: str): def patch(self, app_model: App, workflow_id: str):
@ -1124,7 +1068,7 @@ class WorkflowByIdApi(Resource):
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@edit_permission_required @edit_permission_required
def delete(self, app_model: App, workflow_id: str): def delete(self, app_model: App, workflow_id: str):
""" """
@ -1153,17 +1097,14 @@ class DraftWorkflowNodeLastRunApi(Resource):
@console_ns.doc("get_draft_workflow_node_last_run") @console_ns.doc("get_draft_workflow_node_last_run")
@console_ns.doc(description="Get last run result for draft workflow node") @console_ns.doc(description="Get last run result for draft workflow node")
@console_ns.doc(params={"app_id": "Application ID", "node_id": "Node ID"}) @console_ns.doc(params={"app_id": "Application ID", "node_id": "Node ID"})
@console_ns.response( @console_ns.response(200, "Node last run retrieved successfully", workflow_run_node_execution_model)
200,
"Node last run retrieved successfully",
console_ns.models[WorkflowRunNodeExecutionResponse.__name__],
)
@console_ns.response(404, "Node last run not found") @console_ns.response(404, "Node last run not found")
@console_ns.response(403, "Permission denied") @console_ns.response(403, "Permission denied")
@setup_required @setup_required
@login_required @login_required
@account_initialization_required @account_initialization_required
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW]) @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW, AppMode.AGENT])
@marshal_with(workflow_run_node_execution_model)
def get(self, app_model: App, node_id: str): def get(self, app_model: App, node_id: str):
srv = WorkflowService() srv = WorkflowService()
workflow = srv.get_draft_workflow(app_model) workflow = srv.get_draft_workflow(app_model)
@ -1176,7 +1117,7 @@ class DraftWorkflowNodeLastRunApi(Resource):
) )
if node_exec is None: if node_exec is None:
raise NotFound("last run not found") raise NotFound("last run not found")
return WorkflowRunNodeExecutionResponse.model_validate(node_exec, from_attributes=True).model_dump(mode="json") return node_exec
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/trigger/run") @console_ns.route("/apps/<uuid:app_id>/workflows/draft/trigger/run")
@ -1399,73 +1340,3 @@ class DraftWorkflowTriggerRunAllApi(Resource):
"status": "error", "status": "error",
} }
), 400 ), 400
@console_ns.route("/apps/workflows/online-users")
class WorkflowOnlineUsersApi(Resource):
@console_ns.expect(console_ns.models[WorkflowOnlineUsersPayload.__name__])
@console_ns.doc("get_workflow_online_users")
@console_ns.doc(description="Get workflow online users")
@setup_required
@login_required
@account_initialization_required
@marshal_with(online_user_list_fields)
def post(self):
args = WorkflowOnlineUsersPayload.model_validate(console_ns.payload or {})
app_ids = args.app_ids
if len(app_ids) > MAX_WORKFLOW_ONLINE_USERS_REQUEST_IDS:
raise BadRequest(f"Maximum {MAX_WORKFLOW_ONLINE_USERS_REQUEST_IDS} app_ids are allowed per request.")
if not app_ids:
return {"data": []}
_, current_tenant_id = current_account_with_tenant()
workflow_service = WorkflowService()
accessible_app_ids = workflow_service.get_accessible_app_ids(app_ids, current_tenant_id)
ordered_accessible_app_ids = [app_id for app_id in app_ids if app_id in accessible_app_ids]
users_json_by_app_id: dict[str, Any] = {}
for start_index in range(0, len(ordered_accessible_app_ids), WORKFLOW_ONLINE_USERS_REDIS_BATCH_SIZE):
app_id_batch = ordered_accessible_app_ids[
start_index : start_index + WORKFLOW_ONLINE_USERS_REDIS_BATCH_SIZE
]
pipe = redis_client.pipeline(transaction=False)
for app_id in app_id_batch:
pipe.hgetall(f"{WORKFLOW_ONLINE_USERS_PREFIX}{app_id}")
users_json_batch = pipe.execute()
for app_id, users_json in zip(app_id_batch, users_json_batch):
users_json_by_app_id[app_id] = users_json
results = []
for app_id in ordered_accessible_app_ids:
users_json = users_json_by_app_id.get(app_id, {})
users = []
for _, user_info_json in users_json.items():
try:
user_info = json.loads(user_info_json)
except Exception:
continue
if not isinstance(user_info, dict):
continue
avatar = user_info.get("avatar")
if isinstance(avatar, str) and avatar and not avatar.startswith(("http://", "https://")):
try:
user_info["avatar"] = file_helpers.get_signed_file_url(avatar)
except Exception as exc:
logger.warning(
"Failed to sign workflow online user avatar; using original value. "
"app_id=%s avatar=%s error=%s",
app_id,
avatar,
exc,
)
users.append(user_info)
results.append({"app_id": app_id, "users": users})
return {"data": results}

Some files were not shown because too many files have changed in this diff Show More