mirror of
https://github.com/langgenius/dify.git
synced 2026-05-24 19:07:53 +08:00
Compare commits
3 Commits
feat/rbac
...
laipz8200/
| Author | SHA1 | Date | |
|---|---|---|---|
| 4fa49b27f8 | |||
| 19b334e5ca | |||
| dce25a3909 |
@ -12,7 +12,7 @@ Use this as the decision guide for React/TypeScript component structure. Existin
|
||||
- Search before adding UI, hooks, helpers, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
|
||||
- Group code by feature workflow, route, or ownership area: components, hooks, local types, query helpers, atoms, constants, and small utilities should live near the code that changes with them.
|
||||
- Promote code to shared only when multiple verticals need the same stable primitive. Otherwise keep it local and compose shared primitives inside the owning feature.
|
||||
- Follow Dify's CSS-first Tailwind v4 contract from `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md`. Prefer design-system tokens, utilities, and radius mappings over generic Tailwind guidance.
|
||||
- Use Tailwind CSS v4.1+ rules via the `tailwind-css-rules` skill. Prefer v4 utilities, `gap`, `text-size/line-height`, `min-h-dvh`, and avoid deprecated utilities and `@apply`.
|
||||
|
||||
## Ownership
|
||||
|
||||
|
||||
367
.agents/skills/tailwind-css-rules/SKILL.md
Normal file
367
.agents/skills/tailwind-css-rules/SKILL.md
Normal file
@ -0,0 +1,367 @@
|
||||
---
|
||||
name: tailwind-css-rules
|
||||
description: Tailwind CSS v4.1+ rules and best practices. Use when writing, reviewing, refactoring, or upgrading Tailwind CSS classes and styles, especially v4 utility migrations, layout spacing, typography, responsive variants, dark mode, gradients, CSS variables, and component styling.
|
||||
---
|
||||
|
||||
# Tailwind CSS Rules and Best Practices
|
||||
|
||||
## Core Principles
|
||||
|
||||
- **Always use Tailwind CSS v4.1+** - Ensure the codebase is using the latest version
|
||||
- **Do not use deprecated or removed utilities** - ALWAYS use the replacement
|
||||
- **Never use `@apply`** - Use CSS variables, the `--spacing()` function, or framework components instead
|
||||
- **Check for redundant classes** - Remove any classes that aren't necessary
|
||||
- **Group elements logically** to simplify responsive tweaks later
|
||||
|
||||
## Upgrading to Tailwind CSS v4
|
||||
|
||||
### Before Upgrading
|
||||
|
||||
- **Always read the upgrade documentation first** - Read https://tailwindcss.com/docs/upgrade-guide and https://tailwindcss.com/blog/tailwindcss-v4 before starting an upgrade.
|
||||
- Ensure the git repository is in a clean state before starting
|
||||
|
||||
### Upgrade Process
|
||||
|
||||
1. Run the upgrade command: `npx @tailwindcss/upgrade@latest` for both major and minor updates
|
||||
2. The tool will convert JavaScript config files to the new CSS format
|
||||
3. Review all changes extensively to clean up any false positives
|
||||
4. Test thoroughly across your application
|
||||
|
||||
## Breaking Changes Reference
|
||||
|
||||
### Removed Utilities (NEVER use these in v4)
|
||||
|
||||
| ❌ Deprecated | ✅ Replacement |
|
||||
| ----------------------- | ------------------------------------------------- |
|
||||
| `bg-opacity-*` | Use opacity modifiers like `bg-black/50` |
|
||||
| `text-opacity-*` | Use opacity modifiers like `text-black/50` |
|
||||
| `border-opacity-*` | Use opacity modifiers like `border-black/50` |
|
||||
| `divide-opacity-*` | Use opacity modifiers like `divide-black/50` |
|
||||
| `ring-opacity-*` | Use opacity modifiers like `ring-black/50` |
|
||||
| `placeholder-opacity-*` | Use opacity modifiers like `placeholder-black/50` |
|
||||
| `flex-shrink-*` | `shrink-*` |
|
||||
| `flex-grow-*` | `grow-*` |
|
||||
| `overflow-ellipsis` | `text-ellipsis` |
|
||||
| `decoration-slice` | `box-decoration-slice` |
|
||||
| `decoration-clone` | `box-decoration-clone` |
|
||||
|
||||
### Renamed Utilities
|
||||
|
||||
Use the v4 name when migrating code that still carries Tailwind v3 semantics. Do not blanket-replace existing v4 classes: classes such as `rounded-sm`, `shadow-sm`, `ring-1`, and `ring-2` are valid in this codebase when they intentionally represent the current design scale.
|
||||
|
||||
| ❌ v3 pattern | ✅ v4 pattern |
|
||||
| ------------------- | -------------------------------------------------- |
|
||||
| `bg-gradient-*` | `bg-linear-*` |
|
||||
| old shadow scale | verify against the current Tailwind/design scale |
|
||||
| old blur scale | verify against the current Tailwind/design scale |
|
||||
| old radius scale | use the Dify radius token mapping when applicable |
|
||||
| `outline-none` | `outline-hidden` |
|
||||
| bare `ring` utility | use an explicit ring width such as `ring-1`/`ring-2`/`ring-3` |
|
||||
|
||||
For Figma radius tokens, follow `packages/dify-ui/AGENTS.md`. For example, `--radius/xs` maps to `rounded-sm`; do not rewrite it to `rounded-xs`.
|
||||
|
||||
## Layout and Spacing Rules
|
||||
|
||||
### Flexbox and Grid Spacing
|
||||
|
||||
#### Always use gap utilities for internal spacing
|
||||
|
||||
Gap provides consistent spacing without edge cases (no extra space on last items). It's cleaner and more maintainable than margins on children.
|
||||
|
||||
```html
|
||||
<!-- ❌ Don't do this -->
|
||||
<div class="flex">
|
||||
<div class="mr-4">Item 1</div>
|
||||
<div class="mr-4">Item 2</div>
|
||||
<div>Item 3</div>
|
||||
<!-- No margin on last -->
|
||||
</div>
|
||||
|
||||
<!-- ✅ Do this instead -->
|
||||
<div class="flex gap-4">
|
||||
<div>Item 1</div>
|
||||
<div>Item 2</div>
|
||||
<div>Item 3</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
#### Gap vs Space utilities
|
||||
|
||||
- **Never use `space-x-*` or `space-y-*` in flex/grid layouts** - always use gap
|
||||
- Space utilities add margins to children and have issues with wrapped items
|
||||
- Gap works correctly with flex-wrap and all flex directions
|
||||
|
||||
```html
|
||||
<!-- ❌ Avoid space utilities in flex containers -->
|
||||
<div class="flex flex-wrap space-x-4">
|
||||
<!-- Space utilities break with wrapped items -->
|
||||
</div>
|
||||
|
||||
<!-- ✅ Use gap for consistent spacing -->
|
||||
<div class="flex flex-wrap gap-4">
|
||||
<!-- Gap works perfectly with wrapping -->
|
||||
</div>
|
||||
```
|
||||
|
||||
### General Spacing Guidelines
|
||||
|
||||
- **Prefer top and left margins** over bottom and right margins (unless conditionally rendered)
|
||||
- **Use padding on parent containers** instead of bottom margins on the last child
|
||||
- **Always use `min-h-dvh` instead of `min-h-screen`** - `min-h-screen` is buggy on mobile Safari
|
||||
- **Prefer `size-*` utilities** over separate `w-*` and `h-*` when setting equal dimensions
|
||||
- For max-widths, prefer the container scale (e.g., `max-w-2xs` over `max-w-72`)
|
||||
|
||||
## Typography Rules
|
||||
|
||||
### Line Heights
|
||||
|
||||
- **Never use `leading-*` classes** - Always use line height modifiers with text size
|
||||
- **Always use fixed line heights from the spacing scale** - Don't use named values
|
||||
|
||||
```html
|
||||
<!-- ❌ Don't do this -->
|
||||
<p class="text-base leading-7">Text with separate line height</p>
|
||||
<p class="text-lg leading-relaxed">Text with named line height</p>
|
||||
|
||||
<!-- ✅ Do this instead -->
|
||||
<p class="text-base/7">Text with line height modifier</p>
|
||||
<p class="text-lg/8">Text with specific line height</p>
|
||||
```
|
||||
|
||||
### Font Size Reference
|
||||
|
||||
Be precise with font sizes - know the actual pixel values:
|
||||
|
||||
- `text-xs` = 12px
|
||||
- `text-sm` = 14px
|
||||
- `text-base` = 16px
|
||||
- `text-lg` = 18px
|
||||
- `text-xl` = 20px
|
||||
|
||||
## Color and Opacity
|
||||
|
||||
### Opacity Modifiers
|
||||
|
||||
**Never use `bg-opacity-*`, `text-opacity-*`, etc.** - use the opacity modifier syntax:
|
||||
|
||||
```html
|
||||
<!-- ❌ Don't do this -->
|
||||
<div class="bg-red-500 bg-opacity-60">Old opacity syntax</div>
|
||||
|
||||
<!-- ✅ Do this instead -->
|
||||
<div class="bg-red-500/60">Modern opacity syntax</div>
|
||||
```
|
||||
|
||||
## Responsive Design
|
||||
|
||||
### Breakpoint Optimization
|
||||
|
||||
- **Check for redundant classes across breakpoints**
|
||||
- **Only add breakpoint variants when values change**
|
||||
|
||||
```html
|
||||
<!-- ❌ Redundant breakpoint classes -->
|
||||
<div class="px-4 md:px-4 lg:px-4">
|
||||
<!-- md:px-4 and lg:px-4 are redundant -->
|
||||
</div>
|
||||
|
||||
<!-- ✅ Efficient breakpoint usage -->
|
||||
<div class="px-4 lg:px-8">
|
||||
<!-- Only specify when value changes -->
|
||||
</div>
|
||||
```
|
||||
|
||||
## Dark Mode
|
||||
|
||||
### Dark Mode Best Practices
|
||||
|
||||
- Use the plain `dark:` variant pattern
|
||||
- Put light mode styles first, then dark mode styles
|
||||
- Ensure `dark:` variant comes before other variants
|
||||
|
||||
```html
|
||||
<!-- ✅ Correct dark mode pattern -->
|
||||
<div class="bg-white text-black dark:bg-black dark:text-white">
|
||||
<button class="hover:bg-gray-100 dark:hover:bg-gray-800">Click me</button>
|
||||
</div>
|
||||
```
|
||||
|
||||
## Gradient Utilities
|
||||
|
||||
- **ALWAYS Use `bg-linear-*` instead of `bg-gradient-*` utilities** - The gradient utilities were renamed in v4
|
||||
- Use the new `bg-radial` or `bg-radial-[<position>]` to create radial gradients
|
||||
- Use the new `bg-conic` or `bg-conic-*` to create conic gradients
|
||||
|
||||
```html
|
||||
<!-- ✅ Use the new gradient utilities -->
|
||||
<div class="h-14 bg-linear-to-br from-violet-500 to-fuchsia-500"></div>
|
||||
<div
|
||||
class="size-18 bg-radial-[at_50%_75%] from-sky-200 via-blue-400 to-indigo-900 to-90%"
|
||||
></div>
|
||||
<div
|
||||
class="size-24 bg-conic-180 from-indigo-600 via-indigo-50 to-indigo-600"
|
||||
></div>
|
||||
|
||||
<!-- ❌ Do not use bg-gradient-* utilities -->
|
||||
<div class="h-14 bg-gradient-to-br from-violet-500 to-fuchsia-500"></div>
|
||||
```
|
||||
|
||||
## Working with CSS Variables
|
||||
|
||||
### Accessing Theme Values
|
||||
|
||||
Tailwind CSS v4 exposes all theme values as CSS variables:
|
||||
|
||||
```css
|
||||
/* Access colors, and other theme values */
|
||||
.custom-element {
|
||||
background: var(--color-red-500);
|
||||
border-radius: var(--radius-lg);
|
||||
}
|
||||
```
|
||||
|
||||
### The `--spacing()` Function
|
||||
|
||||
Use the dedicated `--spacing()` function for spacing calculations:
|
||||
|
||||
```css
|
||||
.custom-class {
|
||||
margin-top: calc(100vh - --spacing(16));
|
||||
}
|
||||
```
|
||||
|
||||
### Extending theme values
|
||||
|
||||
Use CSS to extend theme values:
|
||||
|
||||
```css
|
||||
@import "tailwindcss";
|
||||
|
||||
@theme {
|
||||
--color-mint-500: oklch(0.72 0.11 178);
|
||||
}
|
||||
```
|
||||
|
||||
```html
|
||||
<div class="bg-mint-500">
|
||||
<!-- ... -->
|
||||
</div>
|
||||
```
|
||||
|
||||
## New v4 Features
|
||||
|
||||
### Container Queries
|
||||
|
||||
Use the `@container` class and size variants:
|
||||
|
||||
```html
|
||||
<article class="@container">
|
||||
<div class="flex flex-col @md:flex-row @lg:gap-8">
|
||||
<img class="w-full @md:w-48" />
|
||||
<div class="mt-4 @md:mt-0">
|
||||
<!-- Content adapts to container size -->
|
||||
</div>
|
||||
</div>
|
||||
</article>
|
||||
```
|
||||
|
||||
### Container Query Units
|
||||
|
||||
Use container-based units like `cqw` for responsive sizing:
|
||||
|
||||
```html
|
||||
<div class="@container">
|
||||
<h1 class="text-[50cqw]">Responsive to container width</h1>
|
||||
</div>
|
||||
```
|
||||
|
||||
### Text Shadows (v4.1)
|
||||
|
||||
Use text-shadow-\* utilities from text-shadow-2xs to text-shadow-lg:
|
||||
|
||||
```html
|
||||
<!-- ✅ Text shadow examples -->
|
||||
<h1 class="text-shadow-lg">Large shadow</h1>
|
||||
<p class="text-shadow-sm/50">Small shadow with opacity</p>
|
||||
```
|
||||
|
||||
### Masking (v4.1)
|
||||
|
||||
Use the new composable mask utilities for image and gradient masks:
|
||||
|
||||
```html
|
||||
<!-- ✅ Linear gradient masks on specific sides -->
|
||||
<div class="mask-t-from-50%">Top fade</div>
|
||||
<div class="mask-b-from-20% mask-b-to-80%">Bottom gradient</div>
|
||||
<div class="mask-linear-from-white mask-linear-to-black/60">
|
||||
Fade from white to black
|
||||
</div>
|
||||
|
||||
<!-- ✅ Radial gradient masks -->
|
||||
<div class="mask-radial-[100%_100%] mask-radial-from-75% mask-radial-at-left">
|
||||
Radial mask
|
||||
</div>
|
||||
```
|
||||
|
||||
## Component Patterns
|
||||
|
||||
### Avoiding Utility Inheritance
|
||||
|
||||
Don't add utilities to parents that you override in children:
|
||||
|
||||
```html
|
||||
<!-- ❌ Avoid this pattern -->
|
||||
<div class="text-center">
|
||||
<h1>Centered Heading</h1>
|
||||
<div class="text-left">Left-aligned content</div>
|
||||
</div>
|
||||
|
||||
<!-- ✅ Better approach -->
|
||||
<div>
|
||||
<h1 class="text-center">Centered Heading</h1>
|
||||
<div>Left-aligned content</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
### Component Extraction
|
||||
|
||||
- Extract repeated patterns into framework components, not CSS classes
|
||||
- Keep utility classes in templates/JSX
|
||||
- Use data attributes for complex state-based styling
|
||||
|
||||
## CSS Best Practices
|
||||
|
||||
### Nesting Guidelines
|
||||
|
||||
- Use nesting when styling both parent and children
|
||||
- Avoid empty parent selectors
|
||||
|
||||
```css
|
||||
/* ✅ Good nesting - parent has styles */
|
||||
.card {
|
||||
padding: --spacing(4);
|
||||
|
||||
> .card-title {
|
||||
font-weight: bold;
|
||||
}
|
||||
}
|
||||
|
||||
/* ❌ Avoid empty parents */
|
||||
ul {
|
||||
> li {
|
||||
/* Parent has no styles */
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Common Pitfalls to Avoid
|
||||
|
||||
1. **Using old opacity utilities** - Always use `/opacity` syntax like `bg-red-500/60`
|
||||
2. **Redundant breakpoint classes** - Only specify changes
|
||||
3. **Space utilities in flex/grid** - Always use gap
|
||||
4. **Leading utilities** - Use line-height modifiers like `text-sm/6`
|
||||
5. **Arbitrary values** - Use the design scale
|
||||
6. **@apply directive** - Use components or CSS variables
|
||||
7. **min-h-screen on mobile** - Use min-h-dvh
|
||||
8. **Separate width/height** - Use size utilities when equal
|
||||
9. **Arbitrary values** - Always use Tailwind's predefined scale whenever possible (e.g., use `ml-4` over `ml-[16px]`)
|
||||
@ -1,6 +1,5 @@
|
||||
[run]
|
||||
omit =
|
||||
api/conftest.py
|
||||
api/tests/*
|
||||
api/migrations/*
|
||||
api/core/rag/datasource/vdb/*
|
||||
|
||||
60
.github/CODEOWNERS
vendored
60
.github/CODEOWNERS
vendored
@ -4,7 +4,7 @@
|
||||
# Owners can be @username, @org/team-name, or email addresses.
|
||||
# For more information, see: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
|
||||
|
||||
* @crazywoola @laipz8200
|
||||
* @crazywoola @laipz8200 @Yeuoly
|
||||
|
||||
# ESLint suppression file is maintained by autofix.ci pruning.
|
||||
/eslint-suppressions.json
|
||||
@ -85,39 +85,39 @@
|
||||
/api/tasks/deal_dataset_vector_index_task.py @JohnJyong
|
||||
|
||||
# Backend - Plugins
|
||||
/api/core/plugin/ @WH-2099
|
||||
/api/services/plugin/ @WH-2099
|
||||
/api/controllers/console/workspace/plugin.py @WH-2099
|
||||
/api/controllers/inner_api/plugin/ @WH-2099
|
||||
/api/tasks/process_tenant_plugin_autoupgrade_check_task.py @WH-2099
|
||||
/api/core/plugin/ @Mairuis @Yeuoly @Stream29
|
||||
/api/services/plugin/ @Mairuis @Yeuoly @Stream29
|
||||
/api/controllers/console/workspace/plugin.py @Mairuis @Yeuoly @Stream29
|
||||
/api/controllers/inner_api/plugin/ @Mairuis @Yeuoly @Stream29
|
||||
/api/tasks/process_tenant_plugin_autoupgrade_check_task.py @Mairuis @Yeuoly @Stream29
|
||||
|
||||
# Backend - Trigger/Schedule/Webhook
|
||||
/api/controllers/trigger/ @CourTeous33
|
||||
/api/controllers/console/app/workflow_trigger.py @CourTeous33
|
||||
/api/controllers/console/workspace/trigger_providers.py @CourTeous33
|
||||
/api/core/trigger/ @CourTeous33
|
||||
/api/core/app/layers/trigger_post_layer.py @CourTeous33
|
||||
/api/services/trigger/ @CourTeous33
|
||||
/api/models/trigger.py @CourTeous33
|
||||
/api/fields/workflow_trigger_fields.py @CourTeous33
|
||||
/api/repositories/workflow_trigger_log_repository.py @CourTeous33
|
||||
/api/repositories/sqlalchemy_workflow_trigger_log_repository.py @CourTeous33
|
||||
/api/libs/schedule_utils.py @CourTeous33
|
||||
/api/services/workflow/scheduler.py @CourTeous33
|
||||
/api/schedule/trigger_provider_refresh_task.py @CourTeous33
|
||||
/api/schedule/workflow_schedule_task.py @CourTeous33
|
||||
/api/tasks/trigger_processing_tasks.py @CourTeous33
|
||||
/api/tasks/trigger_subscription_refresh_tasks.py @CourTeous33
|
||||
/api/tasks/workflow_schedule_tasks.py @CourTeous33
|
||||
/api/tasks/workflow_cfs_scheduler/ @CourTeous33
|
||||
/api/events/event_handlers/sync_plugin_trigger_when_app_created.py @CourTeous33
|
||||
/api/events/event_handlers/update_app_triggers_when_app_published_workflow_updated.py @CourTeous33
|
||||
/api/events/event_handlers/sync_workflow_schedule_when_app_published.py @CourTeous33
|
||||
/api/events/event_handlers/sync_webhook_when_app_created.py @CourTeous33
|
||||
/api/controllers/trigger/ @Mairuis @Yeuoly
|
||||
/api/controllers/console/app/workflow_trigger.py @Mairuis @Yeuoly
|
||||
/api/controllers/console/workspace/trigger_providers.py @Mairuis @Yeuoly
|
||||
/api/core/trigger/ @Mairuis @Yeuoly
|
||||
/api/core/app/layers/trigger_post_layer.py @Mairuis @Yeuoly
|
||||
/api/services/trigger/ @Mairuis @Yeuoly
|
||||
/api/models/trigger.py @Mairuis @Yeuoly
|
||||
/api/fields/workflow_trigger_fields.py @Mairuis @Yeuoly
|
||||
/api/repositories/workflow_trigger_log_repository.py @Mairuis @Yeuoly
|
||||
/api/repositories/sqlalchemy_workflow_trigger_log_repository.py @Mairuis @Yeuoly
|
||||
/api/libs/schedule_utils.py @Mairuis @Yeuoly
|
||||
/api/services/workflow/scheduler.py @Mairuis @Yeuoly
|
||||
/api/schedule/trigger_provider_refresh_task.py @Mairuis @Yeuoly
|
||||
/api/schedule/workflow_schedule_task.py @Mairuis @Yeuoly
|
||||
/api/tasks/trigger_processing_tasks.py @Mairuis @Yeuoly
|
||||
/api/tasks/trigger_subscription_refresh_tasks.py @Mairuis @Yeuoly
|
||||
/api/tasks/workflow_schedule_tasks.py @Mairuis @Yeuoly
|
||||
/api/tasks/workflow_cfs_scheduler/ @Mairuis @Yeuoly
|
||||
/api/events/event_handlers/sync_plugin_trigger_when_app_created.py @Mairuis @Yeuoly
|
||||
/api/events/event_handlers/update_app_triggers_when_app_published_workflow_updated.py @Mairuis @Yeuoly
|
||||
/api/events/event_handlers/sync_workflow_schedule_when_app_published.py @Mairuis @Yeuoly
|
||||
/api/events/event_handlers/sync_webhook_when_app_created.py @Mairuis @Yeuoly
|
||||
|
||||
# Backend - Async Workflow
|
||||
/api/services/async_workflow_service.py @Mairuis
|
||||
/api/tasks/async_workflow_tasks.py @Mairuis
|
||||
/api/services/async_workflow_service.py @Mairuis @Yeuoly
|
||||
/api/tasks/async_workflow_tasks.py @Mairuis @Yeuoly
|
||||
|
||||
# Backend - Billing
|
||||
/api/services/billing_service.py @hj24 @zyssyz123
|
||||
|
||||
4
.github/actions/setup-web/action.yml
vendored
4
.github/actions/setup-web/action.yml
vendored
@ -5,11 +5,11 @@ runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Setup pnpm
|
||||
uses: pnpm/action-setup@0e279bb959325dab635dd2c09392533439d90093 # v6.0.8
|
||||
uses: pnpm/action-setup@8912a9102ac27614460f54aedde9e1e7f9aec20d # v6.0.5
|
||||
with:
|
||||
run_install: false
|
||||
- name: Setup Vite+
|
||||
uses: voidzero-dev/setup-vp@ca1c46663915d6c1042ae23bd39ab85718bfb0fa # v1.10.0
|
||||
uses: voidzero-dev/setup-vp@4f5aa3e38c781f1b01e78fb9255527cee8a6efa6 # v1.8.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: true
|
||||
|
||||
73
.github/scripts/check-hotfix-cherry-picks.sh
vendored
73
.github/scripts/check-hotfix-cherry-picks.sh
vendored
@ -1,73 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
BASE_SHA=${BASE_SHA:-}
|
||||
HEAD_SHA=${HEAD_SHA:-}
|
||||
MAIN_REF=${MAIN_REF:-origin/main}
|
||||
REMEDIATION_HINT="Changes should be made from the main branch using git cherry-pick -x."
|
||||
|
||||
error() {
|
||||
printf 'ERROR: %s\n' "$1" >&2
|
||||
}
|
||||
|
||||
if [[ -z "$BASE_SHA" || -z "$HEAD_SHA" ]]; then
|
||||
error "BASE_SHA and HEAD_SHA are required. $REMEDIATION_HINT"
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if ! git rev-parse --verify "$BASE_SHA^{commit}" > /dev/null 2>&1; then
|
||||
error "Base commit '$BASE_SHA' is not available in the local git checkout."
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if ! git rev-parse --verify "$HEAD_SHA^{commit}" > /dev/null 2>&1; then
|
||||
error "Head commit '$HEAD_SHA' is not available in the local git checkout."
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if ! git rev-parse --verify "$MAIN_REF^{commit}" > /dev/null 2>&1; then
|
||||
error "Main ref '$MAIN_REF' is not available in the local git checkout. $REMEDIATION_HINT"
|
||||
exit 2
|
||||
fi
|
||||
|
||||
failed=0
|
||||
checked=0
|
||||
|
||||
while IFS= read -r commit_sha; do
|
||||
[[ -n "$commit_sha" ]] || continue
|
||||
|
||||
checked=$((checked + 1))
|
||||
subject=$(git log -1 --format=%s "$commit_sha")
|
||||
source_sha=$(
|
||||
git log -1 --format=%B "$commit_sha" \
|
||||
| sed -nE 's/^\(cherry picked from commit ([0-9a-fA-F]{7,64})\)$/\1/p' \
|
||||
| tail -n 1
|
||||
)
|
||||
|
||||
if [[ -z "$source_sha" ]]; then
|
||||
error "Commit $commit_sha ($subject) is missing cherry-pick provenance. $REMEDIATION_HINT"
|
||||
failed=1
|
||||
continue
|
||||
fi
|
||||
|
||||
if ! git cat-file -e "$source_sha^{commit}" 2> /dev/null; then
|
||||
error "Commit $commit_sha ($subject) references source $source_sha, but that commit is not available locally. $REMEDIATION_HINT"
|
||||
failed=1
|
||||
continue
|
||||
fi
|
||||
|
||||
if ! git merge-base --is-ancestor "$source_sha" "$MAIN_REF"; then
|
||||
error "Commit $commit_sha ($subject) references source $source_sha, but that source is not reachable from main ($MAIN_REF). $REMEDIATION_HINT"
|
||||
failed=1
|
||||
fi
|
||||
done < <(git rev-list --reverse "$BASE_SHA..$HEAD_SHA")
|
||||
|
||||
if [[ "$failed" -ne 0 ]]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "$checked" -eq 0 ]]; then
|
||||
echo "No PR commits to check."
|
||||
else
|
||||
echo "Verified $checked PR commit(s) include cherry-pick provenance from main."
|
||||
fi
|
||||
44
.github/workflows/api-tests.yml
vendored
44
.github/workflows/api-tests.yml
vendored
@ -48,23 +48,10 @@ jobs:
|
||||
run: uv sync --project api --dev
|
||||
|
||||
- name: Run dify config tests
|
||||
run: uv run --project api pytest api/tests/unit_tests/configs/test_env_consistency.py
|
||||
run: uv run --project api dev/pytest/pytest_config_tests.py
|
||||
|
||||
- name: Run Unit Tests
|
||||
run: |
|
||||
uv run --project api pytest \
|
||||
-p no:benchmark \
|
||||
--timeout "${PYTEST_TIMEOUT:-20}" \
|
||||
-n auto \
|
||||
api/tests/unit_tests \
|
||||
api/providers/vdb/*/tests/unit_tests \
|
||||
api/providers/trace/*/tests/unit_tests \
|
||||
--ignore=api/tests/unit_tests/controllers
|
||||
# Controller tests register Flask routes at import time, so keep them out of xdist.
|
||||
uv run --project api pytest \
|
||||
--timeout "${PYTEST_TIMEOUT:-20}" \
|
||||
--cov-append \
|
||||
api/tests/unit_tests/controllers
|
||||
run: uv run --project api bash dev/pytest/pytest_unit_tests.sh
|
||||
|
||||
- name: Upload unit coverage data
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
@ -109,11 +96,32 @@ jobs:
|
||||
- name: Install dependencies
|
||||
run: uv sync --project api --dev
|
||||
|
||||
- name: Set up dotenvs
|
||||
run: |
|
||||
cp docker/.env.example docker/.env
|
||||
cp docker/envs/middleware.env.example docker/middleware.env
|
||||
|
||||
- name: Expose Service Ports
|
||||
run: sh .github/workflows/expose_service_ports.sh
|
||||
|
||||
- name: Set up Sandbox
|
||||
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0
|
||||
with:
|
||||
compose-file: |
|
||||
docker/docker-compose.middleware.yaml
|
||||
services: |
|
||||
db_postgres
|
||||
redis
|
||||
sandbox
|
||||
ssrf_proxy
|
||||
|
||||
- name: setup test config
|
||||
run: |
|
||||
cp api/tests/integration_tests/.env.example api/tests/integration_tests/.env
|
||||
|
||||
- name: Run Integration Tests
|
||||
run: |
|
||||
uv run --project api pytest \
|
||||
-p no:benchmark \
|
||||
--start-middleware \
|
||||
-n auto \
|
||||
--timeout "${PYTEST_TIMEOUT:-180}" \
|
||||
api/tests/integration_tests/workflow \
|
||||
@ -195,7 +203,7 @@ jobs:
|
||||
|
||||
- name: Report coverage
|
||||
if: ${{ env.CODECOV_TOKEN != '' }}
|
||||
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
|
||||
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
|
||||
with:
|
||||
files: ./coverage.xml
|
||||
disable_search: true
|
||||
|
||||
13
.github/workflows/build-push.yml
vendored
13
.github/workflows/build-push.yml
vendored
@ -9,7 +9,6 @@ on:
|
||||
- "release/e-*"
|
||||
- "hotfix/**"
|
||||
- "feat/hitl-backend"
|
||||
- "feat/rbac"
|
||||
tags:
|
||||
- "*"
|
||||
|
||||
@ -36,15 +35,15 @@ jobs:
|
||||
- service_name: "build-api-amd64"
|
||||
image_name_env: "DIFY_API_IMAGE_NAME"
|
||||
artifact_context: "api"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
build_context: "{{defaultContext}}:api"
|
||||
file: "Dockerfile"
|
||||
platform: linux/amd64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
- service_name: "build-api-arm64"
|
||||
image_name_env: "DIFY_API_IMAGE_NAME"
|
||||
artifact_context: "api"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
build_context: "{{defaultContext}}:api"
|
||||
file: "Dockerfile"
|
||||
platform: linux/arm64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
- service_name: "build-web-amd64"
|
||||
@ -118,8 +117,8 @@ jobs:
|
||||
matrix:
|
||||
include:
|
||||
- service_name: "validate-api-amd64"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
build_context: "{{defaultContext}}:api"
|
||||
file: "Dockerfile"
|
||||
- service_name: "validate-web-amd64"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "web/Dockerfile"
|
||||
|
||||
22
.github/workflows/docker-build.yml
vendored
22
.github/workflows/docker-build.yml
vendored
@ -6,12 +6,6 @@ on:
|
||||
- "main"
|
||||
paths:
|
||||
- api/Dockerfile
|
||||
- api/Dockerfile.dockerignore
|
||||
- api/pyproject.toml
|
||||
- api/uv.lock
|
||||
- dify-agent/pyproject.toml
|
||||
- dify-agent/README.md
|
||||
- dify-agent/src/**
|
||||
- web/Dockerfile
|
||||
|
||||
concurrency:
|
||||
@ -31,13 +25,13 @@ jobs:
|
||||
- service_name: "api-amd64"
|
||||
platform: linux/amd64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
context: "{{defaultContext}}:api"
|
||||
file: "Dockerfile"
|
||||
- service_name: "api-arm64"
|
||||
platform: linux/arm64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
context: "{{defaultContext}}:api"
|
||||
file: "Dockerfile"
|
||||
- service_name: "web-amd64"
|
||||
platform: linux/amd64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
@ -70,17 +64,17 @@ jobs:
|
||||
matrix:
|
||||
include:
|
||||
- service_name: "api-amd64"
|
||||
context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
context: "{{defaultContext}}:api"
|
||||
file: "Dockerfile"
|
||||
- service_name: "web-amd64"
|
||||
context: "{{defaultContext}}"
|
||||
file: "web/Dockerfile"
|
||||
steps:
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@98e3b2c9eab4f4f98a95c0c0a3ea5e5e672fd2a8 # v3.10.0
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
- name: Build Docker Image
|
||||
uses: docker/build-push-action@5cd29d66b4a8d8e6f4d5dfe2e9329f0b1d446289 # v6.18.0
|
||||
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
|
||||
with:
|
||||
push: false
|
||||
context: ${{ matrix.context }}
|
||||
|
||||
17
.github/workflows/expose_service_ports.sh
vendored
Executable file
17
.github/workflows/expose_service_ports.sh
vendored
Executable file
@ -0,0 +1,17 @@
|
||||
#!/bin/bash
|
||||
|
||||
yq eval '.services.weaviate.ports += ["8080:8080"]' -i docker/docker-compose.yaml
|
||||
yq eval '.services.weaviate.ports += ["50051:50051"]' -i docker/docker-compose.yaml
|
||||
yq eval '.services.qdrant.ports += ["6333:6333"]' -i docker/docker-compose.yaml
|
||||
yq eval '.services.chroma.ports += ["8000:8000"]' -i docker/docker-compose.yaml
|
||||
yq eval '.services["milvus-standalone"].ports += ["19530:19530"]' -i docker/docker-compose.yaml
|
||||
yq eval '.services.pgvector.ports += ["5433:5432"]' -i docker/docker-compose.yaml
|
||||
yq eval '.services["pgvecto-rs"].ports += ["5431:5432"]' -i docker/docker-compose.yaml
|
||||
yq eval '.services["elasticsearch"].ports += ["9200:9200"]' -i docker/docker-compose.yaml
|
||||
yq eval '.services.couchbase-server.ports += ["8091-8096:8091-8096"]' -i docker/docker-compose.yaml
|
||||
yq eval '.services.couchbase-server.ports += ["11210:11210"]' -i docker/docker-compose.yaml
|
||||
yq eval '.services.tidb.ports += ["4000:4000"]' -i docker/tidb/docker-compose.yaml
|
||||
yq eval '.services.oceanbase.ports += ["2881:2881"]' -i docker/docker-compose.yaml
|
||||
yq eval '.services.opengauss.ports += ["6600:6600"]' -i docker/docker-compose.yaml
|
||||
|
||||
echo "Ports exposed for sandbox, weaviate (HTTP 8080, gRPC 50051), tidb, qdrant, chroma, milvus, pgvector, pgvecto-rs, elasticsearch, couchbase, opengauss"
|
||||
49
.github/workflows/hotfix-cherry-pick.yml
vendored
49
.github/workflows/hotfix-cherry-pick.yml
vendored
@ -1,49 +0,0 @@
|
||||
name: Hotfix Cherry-Pick Provenance
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- 'hotfix/**'
|
||||
- 'lts/**'
|
||||
types:
|
||||
- opened
|
||||
- edited
|
||||
- reopened
|
||||
- ready_for_review
|
||||
- synchronize
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: hotfix-cherry-pick-${{ github.event.pull_request.number || github.run_id }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
check-cherry-pick-provenance:
|
||||
name: Require cherry-pick provenance
|
||||
runs-on: depot-ubuntu-24.04
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Fetch PR base, PR head, and main
|
||||
env:
|
||||
BASE_REF: ${{ github.base_ref }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
run: |
|
||||
git fetch --no-tags --prune origin \
|
||||
"+refs/heads/main:refs/remotes/origin/main" \
|
||||
"+refs/heads/${BASE_REF}:refs/remotes/origin/${BASE_REF}" \
|
||||
"+refs/pull/${PR_NUMBER}/head:refs/remotes/pull/${PR_NUMBER}/head"
|
||||
|
||||
- name: Load checker from main
|
||||
run: git show origin/main:.github/scripts/check-hotfix-cherry-picks.sh > "$RUNNER_TEMP/check-hotfix-cherry-picks.sh"
|
||||
|
||||
- name: Check PR commits
|
||||
env:
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
MAIN_REF: origin/main
|
||||
run: bash "$RUNNER_TEMP/check-hotfix-cherry-picks.sh"
|
||||
6
.github/workflows/main-ci.yml
vendored
6
.github/workflows/main-ci.yml
vendored
@ -55,6 +55,7 @@ jobs:
|
||||
api:
|
||||
- 'api/**'
|
||||
- '.github/workflows/api-tests.yml'
|
||||
- '.github/workflows/expose_service_ports.sh'
|
||||
- 'docker/.env.example'
|
||||
- 'docker/envs/middleware.env.example'
|
||||
- 'docker/docker-compose.middleware.yaml'
|
||||
@ -89,13 +90,11 @@ jobs:
|
||||
vdb:
|
||||
- 'api/core/rag/datasource/**'
|
||||
- 'api/tests/integration_tests/vdb/**'
|
||||
- 'api/conftest.py'
|
||||
- 'api/tests/pytest_dify.py'
|
||||
- 'api/providers/vdb/*/tests/**'
|
||||
- '.github/workflows/vdb-tests.yml'
|
||||
- '.github/workflows/expose_service_ports.sh'
|
||||
- 'docker/.env.example'
|
||||
- 'docker/envs/middleware.env.example'
|
||||
- 'docker/docker-compose.pytest.ports.yaml'
|
||||
- 'docker/docker-compose.yaml'
|
||||
- 'docker/docker-compose-template.yaml'
|
||||
- 'docker/generate_docker_compose'
|
||||
@ -115,6 +114,7 @@ jobs:
|
||||
- 'api/migrations/**'
|
||||
- 'api/.env.example'
|
||||
- '.github/workflows/db-migration-test.yml'
|
||||
- '.github/workflows/expose_service_ports.sh'
|
||||
- 'docker/.env.example'
|
||||
- 'docker/envs/middleware.env.example'
|
||||
- 'docker/docker-compose.middleware.yaml'
|
||||
|
||||
@ -63,8 +63,8 @@ jobs:
|
||||
id: render
|
||||
run: |
|
||||
comment_body="$(uv run --directory api python libs/pyrefly_type_coverage.py \
|
||||
--base "$GITHUB_WORKSPACE/base_report.json" \
|
||||
< "$GITHUB_WORKSPACE/pr_report.json")"
|
||||
--base base_report.json \
|
||||
< pr_report.json)"
|
||||
|
||||
{
|
||||
echo "### Pyrefly Type Coverage"
|
||||
|
||||
4
.github/workflows/pyrefly-type-coverage.yml
vendored
4
.github/workflows/pyrefly-type-coverage.yml
vendored
@ -65,9 +65,6 @@ jobs:
|
||||
# Save structured data for the fork-PR comment workflow
|
||||
cp /tmp/pyrefly_report_pr.json pr_report.json
|
||||
cp /tmp/pyrefly_report_base.json base_report.json
|
||||
# Keep fork-PR comments correct while the trusted workflow_run job is
|
||||
# still using the default-branch renderer, which resolves --base from api/.
|
||||
cp /tmp/pyrefly_report_base.json api/base_report.json
|
||||
|
||||
- name: Save PR number
|
||||
run: |
|
||||
@ -80,7 +77,6 @@ jobs:
|
||||
path: |
|
||||
pr_report.json
|
||||
base_report.json
|
||||
api/base_report.json
|
||||
pr_number.txt
|
||||
|
||||
- name: Comment PR with type coverage
|
||||
|
||||
4
.github/workflows/style.yml
vendored
4
.github/workflows/style.yml
vendored
@ -47,10 +47,6 @@ jobs:
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
run: uv run --directory api --dev lint-imports
|
||||
|
||||
- name: Run Response Contract Linter
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
run: uv run --project api --dev python api/dev/lint_response_contracts.py --fail-on-mismatch
|
||||
|
||||
- name: Run Type Checks
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
run: make type-check-core
|
||||
|
||||
2
.github/workflows/translate-i18n-claude.yml
vendored
2
.github/workflows/translate-i18n-claude.yml
vendored
@ -158,7 +158,7 @@ jobs:
|
||||
|
||||
- name: Run Claude Code for Translation Sync
|
||||
if: steps.context.outputs.CHANGED_FILES != ''
|
||||
uses: anthropics/claude-code-action@1dc994ee7a008f0ecc866d9ac23ef036b7229f84 # v1.0.127
|
||||
uses: anthropics/claude-code-action@476e359e6203e73dad705c8b322e333fabbd7416 # v1.0.119
|
||||
with:
|
||||
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
39
.github/workflows/vdb-tests-full.yml
vendored
39
.github/workflows/vdb-tests-full.yml
vendored
@ -48,6 +48,14 @@ jobs:
|
||||
- name: Install dependencies
|
||||
run: uv sync --project api --dev
|
||||
|
||||
- name: Set up dotenvs
|
||||
run: |
|
||||
cp docker/.env.example docker/.env
|
||||
cp docker/envs/middleware.env.example docker/middleware.env
|
||||
|
||||
- name: Expose Service Ports
|
||||
run: sh .github/workflows/expose_service_ports.sh
|
||||
|
||||
# - name: Set up Vector Store (TiDB)
|
||||
# uses: hoverkraft-tech/compose-action@v2.0.2
|
||||
# with:
|
||||
@ -56,13 +64,32 @@ jobs:
|
||||
# tidb
|
||||
# tiflash
|
||||
|
||||
- name: Set up Full Vector Store Matrix
|
||||
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0
|
||||
with:
|
||||
compose-file: |
|
||||
docker/docker-compose.yaml
|
||||
services: |
|
||||
weaviate
|
||||
qdrant
|
||||
couchbase-server
|
||||
etcd
|
||||
minio
|
||||
milvus-standalone
|
||||
pgvecto-rs
|
||||
pgvector
|
||||
chroma
|
||||
elasticsearch
|
||||
oceanbase
|
||||
|
||||
- name: setup test config
|
||||
run: |
|
||||
echo $(pwd)
|
||||
ls -lah .
|
||||
cp api/tests/integration_tests/.env.example api/tests/integration_tests/.env
|
||||
|
||||
# - name: Check VDB Ready (TiDB)
|
||||
# run: uv run --project api python api/providers/vdb/tidb-vector/tests/integration_tests/check_tiflash_ready.py
|
||||
|
||||
- name: Test Vector Stores
|
||||
run: |
|
||||
uv run --project api pytest \
|
||||
--start-vdb \
|
||||
--vdb-services "weaviate,qdrant,couchbase-server,etcd,minio,milvus-standalone,pgvecto-rs,pgvector,chroma,elasticsearch,oceanbase" \
|
||||
--timeout "${PYTEST_TIMEOUT:-180}" \
|
||||
api/providers/vdb/*/tests/integration_tests
|
||||
run: uv run --project api bash dev/pytest/pytest_vdb.sh
|
||||
|
||||
31
.github/workflows/vdb-tests.yml
vendored
31
.github/workflows/vdb-tests.yml
vendored
@ -45,6 +45,14 @@ jobs:
|
||||
- name: Install dependencies
|
||||
run: uv sync --project api --dev
|
||||
|
||||
- name: Set up dotenvs
|
||||
run: |
|
||||
cp docker/.env.example docker/.env
|
||||
cp docker/envs/middleware.env.example docker/middleware.env
|
||||
|
||||
- name: Expose Service Ports
|
||||
run: sh .github/workflows/expose_service_ports.sh
|
||||
|
||||
# - name: Set up Vector Store (TiDB)
|
||||
# uses: hoverkraft-tech/compose-action@v2.0.2
|
||||
# with:
|
||||
@ -53,14 +61,31 @@ jobs:
|
||||
# tidb
|
||||
# tiflash
|
||||
|
||||
- name: Set up Vector Stores for Smoke Coverage
|
||||
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0
|
||||
with:
|
||||
compose-file: |
|
||||
docker/docker-compose.yaml
|
||||
services: |
|
||||
db_postgres
|
||||
redis
|
||||
weaviate
|
||||
qdrant
|
||||
pgvector
|
||||
chroma
|
||||
|
||||
- name: setup test config
|
||||
run: |
|
||||
echo $(pwd)
|
||||
ls -lah .
|
||||
cp api/tests/integration_tests/.env.example api/tests/integration_tests/.env
|
||||
|
||||
# - name: Check VDB Ready (TiDB)
|
||||
# run: uv run --project api python api/providers/vdb/tidb-vector/tests/integration_tests/check_tiflash_ready.py
|
||||
|
||||
- name: Test Vector Stores
|
||||
run: |
|
||||
uv run --project api pytest \
|
||||
--start-vdb \
|
||||
--timeout "${PYTEST_TIMEOUT:-180}" \
|
||||
uv run --project api pytest --timeout "${PYTEST_TIMEOUT:-180}" \
|
||||
api/providers/vdb/vdb-chroma/tests/integration_tests \
|
||||
api/providers/vdb/vdb-pgvector/tests/integration_tests \
|
||||
api/providers/vdb/vdb-qdrant/tests/integration_tests \
|
||||
|
||||
6
.github/workflows/web-tests.yml
vendored
6
.github/workflows/web-tests.yml
vendored
@ -39,7 +39,7 @@ jobs:
|
||||
uses: ./.github/actions/setup-web
|
||||
|
||||
- name: Run tests
|
||||
run: vp test run --reporter=blob --reporter=minimal --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --coverage
|
||||
run: vp test run --reporter=blob --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --coverage
|
||||
|
||||
- name: Upload blob report
|
||||
if: ${{ !cancelled() }}
|
||||
@ -83,7 +83,7 @@ jobs:
|
||||
|
||||
- name: Report coverage
|
||||
if: ${{ env.CODECOV_TOKEN != '' }}
|
||||
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
|
||||
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
|
||||
with:
|
||||
directory: web/coverage
|
||||
flags: web
|
||||
@ -117,7 +117,7 @@ jobs:
|
||||
|
||||
- name: Report coverage
|
||||
if: ${{ env.CODECOV_TOKEN != '' }}
|
||||
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
|
||||
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
|
||||
with:
|
||||
directory: packages/dify-ui/coverage
|
||||
flags: dify-ui
|
||||
|
||||
57
Makefile
57
Makefile
@ -75,29 +75,23 @@ check:
|
||||
@echo "✅ Code check complete"
|
||||
|
||||
lint:
|
||||
@echo "🔧 Running ruff format, check with fixes, response contract lint, import linter, and dotenv-linter..."
|
||||
@echo "🔧 Running ruff format, check with fixes, import linter, and dotenv-linter..."
|
||||
@uv run --project api --dev ruff format ./api
|
||||
@uv run --project api --dev ruff check --fix ./api
|
||||
@$(MAKE) api-contract-lint
|
||||
@uv run --directory api --dev lint-imports
|
||||
@uv run --project api --dev dotenv-linter ./api/.env.example ./web/.env.example
|
||||
@echo "✅ Linting complete"
|
||||
|
||||
api-contract-lint:
|
||||
@echo "🔎 Linting Flask response contracts..."
|
||||
@uv run --project api --dev python api/dev/lint_response_contracts.py
|
||||
@echo "✅ Response contract lint complete"
|
||||
|
||||
type-check:
|
||||
@echo "📝 Running type checks (pyrefly + mypy)..."
|
||||
@./dev/pyrefly-check-local $(PATH_TO_CHECK)
|
||||
@uv --directory api run mypy --exclude-gitignore --exclude '(^|/)conftest\.py$$' --exclude 'tests/' --exclude 'migrations/' --exclude 'dev/generate_swagger_specs.py' --exclude 'dev/generate_fastopenapi_specs.py' --check-untyped-defs --disable-error-code=import-untyped .
|
||||
@uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --exclude 'dev/generate_swagger_specs.py' --exclude 'dev/generate_fastopenapi_specs.py' --check-untyped-defs --disable-error-code=import-untyped .
|
||||
@echo "✅ Type checks complete"
|
||||
|
||||
type-check-core:
|
||||
@echo "📝 Running core type checks (pyrefly + mypy)..."
|
||||
@./dev/pyrefly-check-local $(PATH_TO_CHECK)
|
||||
@uv --directory api run mypy --exclude-gitignore --exclude '(^|/)conftest\.py$$' --exclude 'tests/' --exclude 'migrations/' --exclude 'dev/generate_swagger_specs.py' --exclude 'dev/generate_fastopenapi_specs.py' --check-untyped-defs --disable-error-code=import-untyped .
|
||||
@uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --exclude 'dev/generate_swagger_specs.py' --exclude 'dev/generate_fastopenapi_specs.py' --check-untyped-defs --disable-error-code=import-untyped .
|
||||
@echo "✅ Core type checks complete"
|
||||
|
||||
test:
|
||||
@ -106,46 +100,7 @@ test:
|
||||
echo "Target: $(TARGET_TESTS)"; \
|
||||
uv run --project api --dev pytest $(TARGET_TESTS); \
|
||||
else \
|
||||
echo "Running backend unit tests"; \
|
||||
uv run --project api --dev pytest -p no:benchmark --timeout "$${PYTEST_TIMEOUT:-20}" -n auto \
|
||||
api/tests/unit_tests \
|
||||
api/providers/vdb/*/tests/unit_tests \
|
||||
api/providers/trace/*/tests/unit_tests \
|
||||
--ignore=api/tests/unit_tests/controllers; \
|
||||
uv run --project api --dev pytest --timeout "$${PYTEST_TIMEOUT:-20}" --cov-append \
|
||||
api/tests/unit_tests/controllers; \
|
||||
fi
|
||||
@echo "✅ Unit tests complete"
|
||||
|
||||
test-all:
|
||||
@echo "🧪 Running full backend test suite..."
|
||||
@if [ -n "$(TARGET_TESTS)" ]; then \
|
||||
echo "Target: $(TARGET_TESTS)"; \
|
||||
uv run --project api --dev pytest $(TARGET_TESTS); \
|
||||
else \
|
||||
echo "Running backend unit tests"; \
|
||||
uv run --project api --dev pytest -p no:benchmark --timeout "$${PYTEST_TIMEOUT:-20}" -n auto \
|
||||
api/tests/unit_tests \
|
||||
api/providers/vdb/*/tests/unit_tests \
|
||||
api/providers/trace/*/tests/unit_tests \
|
||||
--ignore=api/tests/unit_tests/controllers; \
|
||||
uv run --project api --dev pytest --timeout "$${PYTEST_TIMEOUT:-20}" --cov-append \
|
||||
api/tests/unit_tests/controllers; \
|
||||
echo "Running backend integration tests"; \
|
||||
uv run --project api --dev pytest -p no:benchmark --start-middleware -n auto \
|
||||
--timeout "$${PYTEST_TIMEOUT:-180}" \
|
||||
--cov-append \
|
||||
api/tests/integration_tests/workflow \
|
||||
api/tests/integration_tests/tools \
|
||||
api/tests/test_containers_integration_tests; \
|
||||
echo "Running VDB smoke tests"; \
|
||||
uv run --project api --dev pytest --start-vdb \
|
||||
--timeout "$${PYTEST_TIMEOUT:-180}" \
|
||||
--cov-append \
|
||||
api/providers/vdb/vdb-chroma/tests/integration_tests \
|
||||
api/providers/vdb/vdb-pgvector/tests/integration_tests \
|
||||
api/providers/vdb/vdb-qdrant/tests/integration_tests \
|
||||
api/providers/vdb/vdb-weaviate/tests/integration_tests; \
|
||||
PYTEST_XDIST_ARGS="-n auto" uv run --project api --dev dev/pytest/pytest_unit_tests.sh; \
|
||||
fi
|
||||
@echo "✅ Tests complete"
|
||||
|
||||
@ -197,11 +152,9 @@ help:
|
||||
@echo " make format - Format code with ruff"
|
||||
@echo " make check - Check code with ruff"
|
||||
@echo " make lint - Format, fix, and lint code (ruff, imports, dotenv)"
|
||||
@echo " make api-contract-lint - Check Flask response docs against returned schemas"
|
||||
@echo " make type-check - Run type checks (pyrefly, mypy)"
|
||||
@echo " make type-check-core - Run core type checks (pyrefly, mypy)"
|
||||
@echo " make test - Run backend unit tests (or TARGET_TESTS=./api/tests/<target_tests>)"
|
||||
@echo " make test-all - Run full backend tests, including Docker-backed suites"
|
||||
@echo ""
|
||||
@echo "Docker Build Targets:"
|
||||
@echo " make build-web - Build web Docker image"
|
||||
@ -211,4 +164,4 @@ help:
|
||||
@echo " make build-push-all - Build and push all Docker images"
|
||||
|
||||
# Phony targets
|
||||
.PHONY: build-web build-api push-web push-api build-all push-all build-push-all dev-setup prepare-docker prepare-web prepare-api dev-clean help format check lint api-contract-lint type-check test test-all
|
||||
.PHONY: build-web build-api push-web push-api build-all push-all build-push-all dev-setup prepare-docker prepare-web prepare-api dev-clean help format check lint type-check test
|
||||
|
||||
@ -657,7 +657,6 @@ PLUGIN_REMOTE_INSTALL_PORT=5003
|
||||
PLUGIN_REMOTE_INSTALL_HOST=localhost
|
||||
PLUGIN_MAX_PACKAGE_SIZE=15728640
|
||||
PLUGIN_MODEL_SCHEMA_CACHE_TTL=3600
|
||||
PLUGIN_MODEL_PROVIDERS_CACHE_TTL=86400
|
||||
INNER_API_KEY_FOR_PLUGIN=QaHbTe77CtuXmsfyhR7+vRjI/+XbV1AaFy691iy+kGDv2Jvy0/eAh8Y1
|
||||
|
||||
# Marketplace configuration
|
||||
@ -768,7 +767,6 @@ EVENT_BUS_REDIS_CHANNEL_TYPE=pubsub
|
||||
# Whether to use Redis cluster mode while use redis as event bus.
|
||||
# It's highly recommended to enable this for large deployments.
|
||||
EVENT_BUS_REDIS_USE_CLUSTERS=false
|
||||
EVENT_BUS_LISTENER_JOIN_TIMEOUT_MS=2000
|
||||
|
||||
# Whether to Enable human input timeout check task
|
||||
ENABLE_HUMAN_INPUT_TIMEOUT_TASK=true
|
||||
|
||||
@ -180,8 +180,6 @@ Quick checks while iterating:
|
||||
- Format: `make format`
|
||||
- Lint (includes auto-fix): `make lint`
|
||||
- Type check: `make type-check`
|
||||
- Unit tests: `make test`
|
||||
- Full backend tests, including Docker-backed suites: `make test-all`
|
||||
- Targeted tests: `make test TARGET_TESTS=./api/tests/<target_tests>`
|
||||
|
||||
Before opening a PR / submitting:
|
||||
@ -195,7 +193,6 @@ Before opening a PR / submitting:
|
||||
- Controllers: parse input via Pydantic, invoke services, return serialised responses; no business logic.
|
||||
- Services: coordinate repositories, providers, background tasks; keep side effects explicit.
|
||||
- Document non-obvious behaviour with concise docstrings and comments.
|
||||
- For `204 No Content` responses, return an empty body only; never return a dict, model, or other payload.
|
||||
- For Flask-RESTX controller request, query, and response schemas, follow `controllers/API_SCHEMA_GUIDE.md`.
|
||||
In short: use Pydantic models, document GET query params with `query_params_from_model(...)`, register response
|
||||
DTOs with `register_response_schema_models(...)`, serialize response DTOs with `dump_response(...)`,
|
||||
|
||||
@ -22,11 +22,9 @@ RUN apt-get update \
|
||||
libmpfr-dev libmpc-dev
|
||||
|
||||
# Install Python dependencies (workspace members under providers/vdb/)
|
||||
COPY api/pyproject.toml api/uv.lock ./
|
||||
COPY api/providers ./providers
|
||||
COPY dify-agent/pyproject.toml dify-agent/README.md /app/dify-agent/
|
||||
COPY dify-agent/src /app/dify-agent/src
|
||||
# Trust the checked-in lock during image builds; local path sources are copied from the repository context.
|
||||
COPY pyproject.toml uv.lock ./
|
||||
COPY providers ./providers
|
||||
# Trust the checked-in lock during image builds; dev-only path sources live outside the api/ context.
|
||||
RUN uv sync --frozen --no-dev
|
||||
|
||||
# production stage
|
||||
@ -110,10 +108,10 @@ RUN python -c "import tiktoken; tiktoken.encoding_for_model('gpt2')" \
|
||||
&& chown -R dify:dify ${TIKTOKEN_CACHE_DIR}
|
||||
|
||||
# Copy source code
|
||||
COPY --chown=dify:dify api /app/api/
|
||||
COPY --chown=dify:dify . /app/api/
|
||||
|
||||
# Prepare entrypoint script
|
||||
COPY --chown=dify:dify --chmod=755 api/docker/entrypoint.sh /entrypoint.sh
|
||||
COPY --chown=dify:dify --chmod=755 docker/entrypoint.sh /entrypoint.sh
|
||||
|
||||
|
||||
ARG COMMIT_SHA
|
||||
|
||||
@ -1,25 +0,0 @@
|
||||
*
|
||||
|
||||
!api/
|
||||
!api/**
|
||||
!dify-agent/
|
||||
!dify-agent/pyproject.toml
|
||||
!dify-agent/README.md
|
||||
!dify-agent/src/
|
||||
!dify-agent/src/**
|
||||
|
||||
api/.venv
|
||||
api/.venv/**
|
||||
api/.env
|
||||
api/*.env.*
|
||||
api/.idea
|
||||
api/.mypy_cache
|
||||
api/.ruff_cache
|
||||
api/storage/generate_files/*
|
||||
api/storage/privkeys/*
|
||||
api/storage/tools/*
|
||||
api/storage/upload_files/*
|
||||
api/logs
|
||||
api/*.log*
|
||||
**/__pycache__
|
||||
**/*.pyc
|
||||
@ -1 +0,0 @@
|
||||
"""External service client packages."""
|
||||
@ -1,74 +0,0 @@
|
||||
"""API-side integration boundary for the Dify Agent backend.
|
||||
|
||||
Public wire DTOs come from ``dify_agent.protocol``. This package only contains
|
||||
API adapters: request building from Dify product concepts, a thin client wrapper,
|
||||
event adaptation for future workflow integration, and deterministic fakes.
|
||||
"""
|
||||
|
||||
from clients.agent_backend.client import AgentBackendRunClient, DifyAgentBackendRunClient
|
||||
from clients.agent_backend.errors import (
|
||||
AgentBackendError,
|
||||
AgentBackendHTTPError,
|
||||
AgentBackendRequestBuildError,
|
||||
AgentBackendRunFailedError,
|
||||
AgentBackendStreamError,
|
||||
AgentBackendTransportError,
|
||||
AgentBackendValidationError,
|
||||
)
|
||||
from clients.agent_backend.event_adapter import (
|
||||
AgentBackendInternalEvent,
|
||||
AgentBackendInternalEventType,
|
||||
AgentBackendRunCancelledInternalEvent,
|
||||
AgentBackendRunEventAdapter,
|
||||
AgentBackendRunFailedInternalEvent,
|
||||
AgentBackendRunPausedInternalEvent,
|
||||
AgentBackendRunStartedInternalEvent,
|
||||
AgentBackendRunSucceededInternalEvent,
|
||||
AgentBackendStreamInternalEvent,
|
||||
)
|
||||
from clients.agent_backend.factory import create_agent_backend_run_client
|
||||
from clients.agent_backend.fake_client import FakeAgentBackendRunClient, FakeAgentBackendScenario
|
||||
from clients.agent_backend.request_builder import (
|
||||
AGENT_SOUL_PROMPT_LAYER_ID,
|
||||
DIFY_PLUGIN_CONTEXT_LAYER_ID,
|
||||
WORKFLOW_NODE_JOB_PROMPT_LAYER_ID,
|
||||
WORKFLOW_USER_PROMPT_LAYER_ID,
|
||||
AgentBackendModelConfig,
|
||||
AgentBackendOutputConfig,
|
||||
AgentBackendRunRequestBuilder,
|
||||
AgentBackendWorkflowNodeRunInput,
|
||||
redact_for_agent_backend_log,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"AGENT_SOUL_PROMPT_LAYER_ID",
|
||||
"DIFY_PLUGIN_CONTEXT_LAYER_ID",
|
||||
"WORKFLOW_NODE_JOB_PROMPT_LAYER_ID",
|
||||
"WORKFLOW_USER_PROMPT_LAYER_ID",
|
||||
"AgentBackendError",
|
||||
"AgentBackendHTTPError",
|
||||
"AgentBackendInternalEvent",
|
||||
"AgentBackendInternalEventType",
|
||||
"AgentBackendModelConfig",
|
||||
"AgentBackendOutputConfig",
|
||||
"AgentBackendRequestBuildError",
|
||||
"AgentBackendRunCancelledInternalEvent",
|
||||
"AgentBackendRunClient",
|
||||
"AgentBackendRunEventAdapter",
|
||||
"AgentBackendRunFailedError",
|
||||
"AgentBackendRunFailedInternalEvent",
|
||||
"AgentBackendRunPausedInternalEvent",
|
||||
"AgentBackendRunRequestBuilder",
|
||||
"AgentBackendRunStartedInternalEvent",
|
||||
"AgentBackendRunSucceededInternalEvent",
|
||||
"AgentBackendStreamError",
|
||||
"AgentBackendStreamInternalEvent",
|
||||
"AgentBackendTransportError",
|
||||
"AgentBackendValidationError",
|
||||
"AgentBackendWorkflowNodeRunInput",
|
||||
"DifyAgentBackendRunClient",
|
||||
"FakeAgentBackendRunClient",
|
||||
"FakeAgentBackendScenario",
|
||||
"create_agent_backend_run_client",
|
||||
"redact_for_agent_backend_log",
|
||||
]
|
||||
@ -1,130 +0,0 @@
|
||||
"""Synchronous API-side wrapper around the public ``dify-agent`` client.
|
||||
|
||||
``dify-agent`` owns the cross-service DTOs and HTTP/SSE implementation. The API
|
||||
backend keeps this thin wrapper so workflow code depends on a local protocol,
|
||||
gets API-native errors, and can use a deterministic fake in tests without
|
||||
creating another wire contract.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterator
|
||||
from typing import Protocol
|
||||
|
||||
from dify_agent.client import (
|
||||
DifyAgentClientError,
|
||||
DifyAgentHTTPError,
|
||||
DifyAgentStreamError,
|
||||
DifyAgentTimeoutError,
|
||||
DifyAgentValidationError,
|
||||
)
|
||||
from dify_agent.protocol import (
|
||||
CancelRunRequest,
|
||||
CancelRunResponse,
|
||||
CreateRunRequest,
|
||||
CreateRunResponse,
|
||||
RunEvent,
|
||||
RunStatusResponse,
|
||||
)
|
||||
|
||||
from clients.agent_backend.errors import (
|
||||
AgentBackendError,
|
||||
AgentBackendHTTPError,
|
||||
AgentBackendStreamError,
|
||||
AgentBackendTransportError,
|
||||
AgentBackendValidationError,
|
||||
)
|
||||
|
||||
|
||||
class AgentBackendRunClient(Protocol):
|
||||
"""Local boundary used by API workflow integrations to run Agent backend jobs."""
|
||||
|
||||
def create_run(self, request: CreateRunRequest) -> CreateRunResponse:
|
||||
"""Create one Agent backend run and return its accepted status."""
|
||||
|
||||
def cancel_run(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
|
||||
"""Request explicit cancellation for one Agent backend run."""
|
||||
|
||||
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
|
||||
"""Yield public ``dify-agent`` run events in stream order."""
|
||||
|
||||
def wait_run(self, run_id: str, *, timeout_seconds: float | None = None) -> RunStatusResponse:
|
||||
"""Wait for a run to reach a terminal status and return that status."""
|
||||
|
||||
|
||||
class _DifyAgentSyncClient(Protocol):
|
||||
"""Subset of ``dify_agent.client.Client`` used by the API wrapper."""
|
||||
|
||||
def create_run_sync(self, request: CreateRunRequest) -> CreateRunResponse:
|
||||
"""Create one run synchronously."""
|
||||
|
||||
def cancel_run_sync(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
|
||||
"""Cancel one run synchronously."""
|
||||
|
||||
def stream_events_sync(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
|
||||
"""Stream run events synchronously."""
|
||||
|
||||
def wait_run_sync(self, run_id: str, *, timeout_seconds: float | None = None) -> RunStatusResponse:
|
||||
"""Wait for terminal run status synchronously."""
|
||||
|
||||
|
||||
class DifyAgentBackendRunClient:
|
||||
"""Adapter from API sync call sites to ``dify_agent.client.Client`` sync methods."""
|
||||
|
||||
client: _DifyAgentSyncClient
|
||||
|
||||
def __init__(self, client: _DifyAgentSyncClient) -> None:
|
||||
self.client = client
|
||||
|
||||
def create_run(self, request: CreateRunRequest) -> CreateRunResponse:
|
||||
"""Create one run through ``POST /runs`` and normalize client exceptions."""
|
||||
try:
|
||||
return self.client.create_run_sync(request)
|
||||
except Exception as exc:
|
||||
raise _normalize_dify_agent_error(exc) from exc
|
||||
|
||||
def cancel_run(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
|
||||
"""Cancel one run through ``POST /runs/{run_id}/cancel`` and normalize exceptions."""
|
||||
try:
|
||||
return self.client.cancel_run_sync(run_id, request=request)
|
||||
except Exception as exc:
|
||||
raise _normalize_dify_agent_error(exc) from exc
|
||||
|
||||
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
|
||||
"""Stream run events from ``/events/sse`` with the wrapped client's reconnect policy."""
|
||||
try:
|
||||
yield from self.client.stream_events_sync(run_id, after=after)
|
||||
except Exception as exc:
|
||||
raise _normalize_dify_agent_error(exc) from exc
|
||||
|
||||
def wait_run(self, run_id: str, *, timeout_seconds: float | None = None) -> RunStatusResponse:
|
||||
"""Poll run status until terminal state and normalize client exceptions."""
|
||||
try:
|
||||
return self.client.wait_run_sync(run_id, timeout_seconds=timeout_seconds)
|
||||
except Exception as exc:
|
||||
raise _normalize_dify_agent_error(exc) from exc
|
||||
|
||||
|
||||
def _normalize_dify_agent_error(exc: Exception) -> AgentBackendError:
|
||||
"""Map public ``dify-agent`` client errors to API-side integration errors."""
|
||||
match exc:
|
||||
case DifyAgentValidationError() as error:
|
||||
return AgentBackendValidationError(
|
||||
"Agent backend request or response validation failed", detail=error.detail
|
||||
)
|
||||
case DifyAgentHTTPError() as error:
|
||||
return AgentBackendHTTPError(
|
||||
f"Agent backend HTTP {error.status_code}",
|
||||
status_code=error.status_code,
|
||||
detail=error.detail,
|
||||
)
|
||||
case DifyAgentTimeoutError() as error:
|
||||
return AgentBackendTransportError(str(error))
|
||||
case DifyAgentStreamError() as error:
|
||||
return AgentBackendStreamError(str(error))
|
||||
case DifyAgentClientError() as error:
|
||||
return AgentBackendTransportError(str(error))
|
||||
case AgentBackendError() as error:
|
||||
return error
|
||||
case _:
|
||||
return AgentBackendTransportError(str(exc) or type(exc).__name__)
|
||||
@ -1,61 +0,0 @@
|
||||
"""API-side errors for the Dify Agent backend integration.
|
||||
|
||||
The wire protocol and low-level HTTP behaviour are owned by ``dify-agent``.
|
||||
This module only normalizes those client errors into the API backend's boundary
|
||||
so workflow/node code does not depend directly on transport-specific exception
|
||||
classes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
|
||||
class AgentBackendError(Exception):
|
||||
"""Base error for API-side Agent backend integration failures."""
|
||||
|
||||
|
||||
class AgentBackendRequestBuildError(AgentBackendError):
|
||||
"""Raised when Dify product/workflow state cannot be mapped to a run request."""
|
||||
|
||||
|
||||
class AgentBackendTransportError(AgentBackendError):
|
||||
"""Raised for timeout or request-level failures talking to Agent backend."""
|
||||
|
||||
|
||||
class AgentBackendHTTPError(AgentBackendTransportError):
|
||||
"""Raised for Agent backend HTTP errors after status/detail normalization."""
|
||||
|
||||
status_code: int
|
||||
detail: object
|
||||
|
||||
def __init__(self, message: str, *, status_code: int, detail: object) -> None:
|
||||
self.status_code = status_code
|
||||
self.detail = detail
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class AgentBackendValidationError(AgentBackendError):
|
||||
"""Raised for local request validation or Agent backend 422 responses."""
|
||||
|
||||
detail: object
|
||||
|
||||
def __init__(self, message: str, *, detail: object) -> None:
|
||||
self.detail = detail
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class AgentBackendStreamError(AgentBackendError):
|
||||
"""Raised when an Agent backend event stream is malformed or exhausted."""
|
||||
|
||||
|
||||
class AgentBackendRunFailedError(AgentBackendError):
|
||||
"""Raised by callers that choose to translate a terminal failed run into an exception."""
|
||||
|
||||
run_id: str
|
||||
detail: Any
|
||||
|
||||
def __init__(self, run_id: str, detail: Any) -> None:
|
||||
self.run_id = run_id
|
||||
self.detail = detail
|
||||
super().__init__(f"Agent backend run failed: {run_id}")
|
||||
@ -1,167 +0,0 @@
|
||||
"""Adapt public ``dify-agent`` run events into API-internal event semantics.
|
||||
|
||||
The adapter does not define a new cross-service event contract. It consumes
|
||||
``dify_agent.protocol.RunEvent`` and produces small API-internal models that the
|
||||
future workflow Agent Node can map to Graphon/AppQueue events in phase 3.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import StrEnum
|
||||
from typing import Annotated, Literal, cast
|
||||
|
||||
from agenton.compositor import CompositorSessionSnapshot
|
||||
from dify_agent.protocol import (
|
||||
PydanticAIStreamRunEvent,
|
||||
RunCancelledEvent,
|
||||
RunEvent,
|
||||
RunFailedEvent,
|
||||
RunPausedEvent,
|
||||
RunStartedEvent,
|
||||
RunSucceededEvent,
|
||||
)
|
||||
from pydantic import BaseModel, ConfigDict, Field, JsonValue, TypeAdapter
|
||||
|
||||
_EVENT_DATA_ADAPTER = TypeAdapter(object)
|
||||
|
||||
|
||||
class AgentBackendInternalEventType(StrEnum):
|
||||
"""API-only event labels used before Graphon/AppQueue integration."""
|
||||
|
||||
RUN_STARTED = "run_started"
|
||||
STREAM_EVENT = "stream_event"
|
||||
RUN_PAUSED = "run_paused"
|
||||
RUN_SUCCEEDED = "run_succeeded"
|
||||
RUN_FAILED = "run_failed"
|
||||
RUN_CANCELLED = "run_cancelled"
|
||||
|
||||
|
||||
class AgentBackendInternalEventBase(BaseModel):
|
||||
"""Common fields preserved from public Dify Agent run events."""
|
||||
|
||||
run_id: str
|
||||
source_event_id: str | None = None
|
||||
|
||||
model_config = ConfigDict(extra="forbid", arbitrary_types_allowed=True)
|
||||
|
||||
|
||||
class AgentBackendRunStartedInternalEvent(AgentBackendInternalEventBase):
|
||||
"""API-internal marker for a started Agent backend run."""
|
||||
|
||||
type: Literal[AgentBackendInternalEventType.RUN_STARTED] = AgentBackendInternalEventType.RUN_STARTED
|
||||
|
||||
|
||||
class AgentBackendStreamInternalEvent(AgentBackendInternalEventBase):
|
||||
"""API-internal wrapper for one pydantic-ai stream event payload."""
|
||||
|
||||
type: Literal[AgentBackendInternalEventType.STREAM_EVENT] = AgentBackendInternalEventType.STREAM_EVENT
|
||||
event_kind: str | None = None
|
||||
data: JsonValue
|
||||
|
||||
|
||||
class AgentBackendRunSucceededInternalEvent(AgentBackendInternalEventBase):
|
||||
"""API-internal terminal success event carrying final output and session state."""
|
||||
|
||||
type: Literal[AgentBackendInternalEventType.RUN_SUCCEEDED] = AgentBackendInternalEventType.RUN_SUCCEEDED
|
||||
output: JsonValue
|
||||
session_snapshot: CompositorSessionSnapshot
|
||||
|
||||
|
||||
class AgentBackendRunPausedInternalEvent(AgentBackendInternalEventBase):
|
||||
"""API-internal resumable pause event for human handoff and Babysit flows."""
|
||||
|
||||
type: Literal[AgentBackendInternalEventType.RUN_PAUSED] = AgentBackendInternalEventType.RUN_PAUSED
|
||||
reason: str
|
||||
message: str | None = None
|
||||
session_snapshot: CompositorSessionSnapshot | None = None
|
||||
|
||||
|
||||
class AgentBackendRunFailedInternalEvent(AgentBackendInternalEventBase):
|
||||
"""API-internal terminal failure event carrying the backend-safe error text."""
|
||||
|
||||
type: Literal[AgentBackendInternalEventType.RUN_FAILED] = AgentBackendInternalEventType.RUN_FAILED
|
||||
error: str
|
||||
reason: str | None = None
|
||||
|
||||
|
||||
class AgentBackendRunCancelledInternalEvent(AgentBackendInternalEventBase):
|
||||
"""API-internal terminal cancellation event."""
|
||||
|
||||
type: Literal[AgentBackendInternalEventType.RUN_CANCELLED] = AgentBackendInternalEventType.RUN_CANCELLED
|
||||
reason: str | None = None
|
||||
message: str | None = None
|
||||
|
||||
|
||||
type AgentBackendInternalEvent = Annotated[
|
||||
AgentBackendRunStartedInternalEvent
|
||||
| AgentBackendStreamInternalEvent
|
||||
| AgentBackendRunPausedInternalEvent
|
||||
| AgentBackendRunSucceededInternalEvent
|
||||
| AgentBackendRunFailedInternalEvent
|
||||
| AgentBackendRunCancelledInternalEvent,
|
||||
Field(discriminator="type"),
|
||||
]
|
||||
|
||||
|
||||
class AgentBackendRunEventAdapter:
|
||||
"""Maps public ``dify-agent`` event variants to API-internal event variants."""
|
||||
|
||||
def adapt(self, event: RunEvent) -> list[AgentBackendInternalEvent]:
|
||||
"""Return zero or more API-internal events derived from one public run event."""
|
||||
match event:
|
||||
case RunStartedEvent():
|
||||
return [
|
||||
AgentBackendRunStartedInternalEvent(
|
||||
run_id=event.run_id,
|
||||
source_event_id=event.id,
|
||||
)
|
||||
]
|
||||
case PydanticAIStreamRunEvent():
|
||||
data = cast(JsonValue, _EVENT_DATA_ADAPTER.dump_python(event.data, mode="json"))
|
||||
event_kind = data.get("event_kind") if isinstance(data, dict) else None
|
||||
return [
|
||||
AgentBackendStreamInternalEvent(
|
||||
run_id=event.run_id,
|
||||
source_event_id=event.id,
|
||||
event_kind=event_kind if isinstance(event_kind, str) else None,
|
||||
data=data,
|
||||
)
|
||||
]
|
||||
case RunSucceededEvent():
|
||||
return [
|
||||
AgentBackendRunSucceededInternalEvent(
|
||||
run_id=event.run_id,
|
||||
source_event_id=event.id,
|
||||
output=event.data.output,
|
||||
session_snapshot=event.data.session_snapshot,
|
||||
)
|
||||
]
|
||||
case RunPausedEvent():
|
||||
return [
|
||||
AgentBackendRunPausedInternalEvent(
|
||||
run_id=event.run_id,
|
||||
source_event_id=event.id,
|
||||
reason=event.data.reason,
|
||||
message=event.data.message,
|
||||
session_snapshot=event.data.session_snapshot,
|
||||
)
|
||||
]
|
||||
case RunFailedEvent():
|
||||
return [
|
||||
AgentBackendRunFailedInternalEvent(
|
||||
run_id=event.run_id,
|
||||
source_event_id=event.id,
|
||||
error=event.data.error,
|
||||
reason=event.data.reason,
|
||||
)
|
||||
]
|
||||
case RunCancelledEvent():
|
||||
return [
|
||||
AgentBackendRunCancelledInternalEvent(
|
||||
run_id=event.run_id,
|
||||
source_event_id=event.id,
|
||||
reason=event.data.reason,
|
||||
message=event.data.message,
|
||||
)
|
||||
]
|
||||
raise TypeError(f"unsupported agent backend run event: {type(event).__name__}")
|
||||
@ -1,22 +0,0 @@
|
||||
"""Factories for API-side Agent backend clients."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dify_agent.client import Client
|
||||
|
||||
from clients.agent_backend.client import AgentBackendRunClient, DifyAgentBackendRunClient
|
||||
from clients.agent_backend.fake_client import FakeAgentBackendRunClient, FakeAgentBackendScenario
|
||||
|
||||
|
||||
def create_agent_backend_run_client(
|
||||
*,
|
||||
base_url: str | None = None,
|
||||
use_fake: bool = False,
|
||||
fake_scenario: str | FakeAgentBackendScenario = FakeAgentBackendScenario.SUCCESS,
|
||||
) -> AgentBackendRunClient:
|
||||
"""Create the API-side run client without hiding the ``dify-agent`` protocol."""
|
||||
if use_fake:
|
||||
return FakeAgentBackendRunClient(scenario=FakeAgentBackendScenario(fake_scenario))
|
||||
if base_url is None:
|
||||
raise ValueError("base_url is required when creating a real Agent backend client")
|
||||
return DifyAgentBackendRunClient(Client(base_url=base_url))
|
||||
@ -1,117 +0,0 @@
|
||||
"""Deterministic fake Agent backend client using public ``dify-agent`` events.
|
||||
|
||||
Tests should exercise the same ``RunEvent`` DTOs as the real HTTP client. This
|
||||
fake therefore replaces the previous custom mock protocol instead of emulating a
|
||||
separate ``agent-backend.v1`` event stream.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterator
|
||||
from datetime import UTC, datetime
|
||||
from enum import StrEnum
|
||||
|
||||
from agenton.compositor import CompositorSessionSnapshot
|
||||
from dify_agent.protocol import (
|
||||
CancelRunRequest,
|
||||
CancelRunResponse,
|
||||
CreateRunRequest,
|
||||
CreateRunResponse,
|
||||
RunEvent,
|
||||
RunFailedEvent,
|
||||
RunFailedEventData,
|
||||
RunStartedEvent,
|
||||
RunStatusResponse,
|
||||
RunSucceededEvent,
|
||||
RunSucceededEventData,
|
||||
)
|
||||
|
||||
_FIXED_TIME = datetime(2026, 1, 1, tzinfo=UTC)
|
||||
|
||||
|
||||
class FakeAgentBackendScenario(StrEnum):
|
||||
"""Deterministic fake scenarios for API-side integration tests."""
|
||||
|
||||
SUCCESS = "success"
|
||||
FAILED = "failed"
|
||||
|
||||
|
||||
class FakeAgentBackendRunClient:
|
||||
"""In-memory implementation of ``AgentBackendRunClient`` for unit tests."""
|
||||
|
||||
scenario: FakeAgentBackendScenario
|
||||
run_id: str
|
||||
request: CreateRunRequest | None
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
scenario: FakeAgentBackendScenario = FakeAgentBackendScenario.SUCCESS,
|
||||
run_id: str = "fake-run-1",
|
||||
) -> None:
|
||||
self.scenario = scenario
|
||||
self.run_id = run_id
|
||||
self.request = None
|
||||
|
||||
def create_run(self, request: CreateRunRequest) -> CreateRunResponse:
|
||||
"""Record the request and return a deterministic accepted response."""
|
||||
self.request = request
|
||||
return CreateRunResponse(run_id=self.run_id, status="running")
|
||||
|
||||
def cancel_run(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
|
||||
"""Return a deterministic cancellation response."""
|
||||
del request
|
||||
return CancelRunResponse(run_id=run_id, status="cancelled")
|
||||
|
||||
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
|
||||
"""Yield the deterministic public ``RunEvent`` sequence for ``run_id``."""
|
||||
for event in self._events(run_id):
|
||||
if after is not None and event.id is not None and event.id <= after:
|
||||
continue
|
||||
yield event
|
||||
|
||||
def wait_run(self, run_id: str, *, timeout_seconds: float | None = None) -> RunStatusResponse:
|
||||
"""Return a deterministic terminal status; timeout is accepted for protocol parity."""
|
||||
del timeout_seconds
|
||||
match self.scenario:
|
||||
case FakeAgentBackendScenario.SUCCESS:
|
||||
return RunStatusResponse(
|
||||
run_id=run_id,
|
||||
status="succeeded",
|
||||
created_at=_FIXED_TIME,
|
||||
updated_at=_FIXED_TIME,
|
||||
)
|
||||
case FakeAgentBackendScenario.FAILED:
|
||||
return RunStatusResponse(
|
||||
run_id=run_id,
|
||||
status="failed",
|
||||
created_at=_FIXED_TIME,
|
||||
updated_at=_FIXED_TIME,
|
||||
error="fake failure",
|
||||
)
|
||||
|
||||
def _events(self, run_id: str) -> tuple[RunEvent, ...]:
|
||||
match self.scenario:
|
||||
case FakeAgentBackendScenario.SUCCESS:
|
||||
return (
|
||||
RunStartedEvent(id="1-0", run_id=run_id, created_at=_FIXED_TIME),
|
||||
RunSucceededEvent(
|
||||
id="2-0",
|
||||
run_id=run_id,
|
||||
created_at=_FIXED_TIME,
|
||||
data=RunSucceededEventData(
|
||||
output={"text": "hello agent"},
|
||||
session_snapshot=CompositorSessionSnapshot(layers=[]),
|
||||
),
|
||||
),
|
||||
)
|
||||
case FakeAgentBackendScenario.FAILED:
|
||||
return (
|
||||
RunStartedEvent(id="1-0", run_id=run_id, created_at=_FIXED_TIME),
|
||||
RunFailedEvent(
|
||||
id="2-0",
|
||||
run_id=run_id,
|
||||
created_at=_FIXED_TIME,
|
||||
data=RunFailedEventData(error="fake failure", reason="unit_test"),
|
||||
),
|
||||
)
|
||||
@ -1,194 +0,0 @@
|
||||
"""Build ``dify-agent`` run requests from API-side product concepts.
|
||||
|
||||
This module is intentionally an adapter, not a wire DTO package. The emitted
|
||||
object is always ``dify_agent.protocol.CreateRunRequest`` so the Agent backend
|
||||
protocol has a single owner. API-only context such as Agent Soul vs workflow job
|
||||
prompt is preserved in layer names and metadata until the dedicated product
|
||||
schemas land in later phases.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import ClassVar
|
||||
|
||||
from agenton.compositor import CompositorSessionSnapshot
|
||||
from agenton.layers import ExitIntent
|
||||
from agenton_collections.layers.plain import PLAIN_PROMPT_LAYER_TYPE_ID, PromptLayerConfig
|
||||
from dify_agent.layers.dify_plugin import (
|
||||
DIFY_PLUGIN_LAYER_TYPE_ID,
|
||||
DIFY_PLUGIN_LLM_LAYER_TYPE_ID,
|
||||
DifyPluginCredentialValue,
|
||||
DifyPluginLayerConfig,
|
||||
DifyPluginLLMLayerConfig,
|
||||
)
|
||||
from dify_agent.layers.output import DIFY_OUTPUT_LAYER_TYPE_ID, DifyOutputLayerConfig
|
||||
from dify_agent.protocol import (
|
||||
DIFY_AGENT_MODEL_LAYER_ID,
|
||||
DIFY_AGENT_OUTPUT_LAYER_ID,
|
||||
CreateRunRequest,
|
||||
ExecutionContext,
|
||||
LayerExitSignals,
|
||||
RunComposition,
|
||||
RunLayerSpec,
|
||||
RunPurpose,
|
||||
)
|
||||
from pydantic import BaseModel, ConfigDict, Field, JsonValue, field_validator
|
||||
|
||||
AGENT_SOUL_PROMPT_LAYER_ID = "agent_soul_prompt"
|
||||
WORKFLOW_NODE_JOB_PROMPT_LAYER_ID = "workflow_node_job_prompt"
|
||||
WORKFLOW_USER_PROMPT_LAYER_ID = "workflow_user_prompt"
|
||||
DIFY_PLUGIN_CONTEXT_LAYER_ID = "plugin"
|
||||
|
||||
|
||||
class AgentBackendModelConfig(BaseModel):
|
||||
"""API-side model/plugin selection before it is converted to Dify Agent layers."""
|
||||
|
||||
tenant_id: str
|
||||
plugin_id: str
|
||||
model_provider: str
|
||||
model: str
|
||||
user_id: str | None = None
|
||||
credentials: dict[str, DifyPluginCredentialValue] = Field(default_factory=dict)
|
||||
model_settings: dict[str, JsonValue] = Field(default_factory=dict)
|
||||
|
||||
model_config: ClassVar[ConfigDict] = ConfigDict(extra="forbid")
|
||||
|
||||
|
||||
class AgentBackendOutputConfig(BaseModel):
|
||||
"""API-side structured output declaration for the conventional output layer."""
|
||||
|
||||
json_schema: dict[str, JsonValue]
|
||||
name: str = "final_result"
|
||||
description: str | None = None
|
||||
strict: bool | None = None
|
||||
|
||||
model_config: ClassVar[ConfigDict] = ConfigDict(extra="forbid")
|
||||
|
||||
|
||||
class AgentBackendWorkflowNodeRunInput(BaseModel):
|
||||
"""Inputs needed to build the first workflow-node-oriented Agent backend run request."""
|
||||
|
||||
model: AgentBackendModelConfig
|
||||
execution_context: ExecutionContext
|
||||
workflow_node_job_prompt: str
|
||||
user_prompt: str
|
||||
agent_soul_prompt: str | None = None
|
||||
purpose: RunPurpose = "workflow_node"
|
||||
idempotency_key: str | None = None
|
||||
output: AgentBackendOutputConfig | None = None
|
||||
session_snapshot: CompositorSessionSnapshot | None = None
|
||||
suspend_on_exit: bool = False
|
||||
metadata: dict[str, JsonValue] = Field(default_factory=dict)
|
||||
|
||||
model_config: ClassVar[ConfigDict] = ConfigDict(extra="forbid", arbitrary_types_allowed=True)
|
||||
|
||||
@field_validator("workflow_node_job_prompt", "user_prompt")
|
||||
@classmethod
|
||||
def _reject_blank_prompt(cls, value: str) -> str:
|
||||
if not value.strip():
|
||||
raise ValueError("prompt must not be blank")
|
||||
return value
|
||||
|
||||
|
||||
class AgentBackendRunRequestBuilder:
|
||||
"""Converts API product state into the public ``dify-agent`` run protocol."""
|
||||
|
||||
def build_for_workflow_node(self, run_input: AgentBackendWorkflowNodeRunInput) -> CreateRunRequest:
|
||||
"""Build a workflow Agent Node run request without defining another wire schema."""
|
||||
layers: list[RunLayerSpec] = []
|
||||
if run_input.agent_soul_prompt:
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=AGENT_SOUL_PROMPT_LAYER_ID,
|
||||
type=PLAIN_PROMPT_LAYER_TYPE_ID,
|
||||
metadata={**run_input.metadata, "origin": "agent_soul"},
|
||||
config=PromptLayerConfig(prefix=run_input.agent_soul_prompt),
|
||||
)
|
||||
)
|
||||
|
||||
layers.extend(
|
||||
[
|
||||
RunLayerSpec(
|
||||
name=WORKFLOW_NODE_JOB_PROMPT_LAYER_ID,
|
||||
type=PLAIN_PROMPT_LAYER_TYPE_ID,
|
||||
metadata={**run_input.metadata, "origin": "workflow_node_job"},
|
||||
config=PromptLayerConfig(prefix=run_input.workflow_node_job_prompt),
|
||||
),
|
||||
RunLayerSpec(
|
||||
name=WORKFLOW_USER_PROMPT_LAYER_ID,
|
||||
type=PLAIN_PROMPT_LAYER_TYPE_ID,
|
||||
metadata={**run_input.metadata, "origin": "workflow_user_prompt"},
|
||||
config=PromptLayerConfig(user=run_input.user_prompt),
|
||||
),
|
||||
RunLayerSpec(
|
||||
name=DIFY_PLUGIN_CONTEXT_LAYER_ID,
|
||||
type=DIFY_PLUGIN_LAYER_TYPE_ID,
|
||||
metadata=run_input.metadata,
|
||||
config=DifyPluginLayerConfig(
|
||||
tenant_id=run_input.model.tenant_id,
|
||||
plugin_id=run_input.model.plugin_id,
|
||||
user_id=run_input.model.user_id,
|
||||
),
|
||||
),
|
||||
RunLayerSpec(
|
||||
name=DIFY_AGENT_MODEL_LAYER_ID,
|
||||
type=DIFY_PLUGIN_LLM_LAYER_TYPE_ID,
|
||||
deps={"plugin": DIFY_PLUGIN_CONTEXT_LAYER_ID},
|
||||
metadata=run_input.metadata,
|
||||
config=DifyPluginLLMLayerConfig(
|
||||
model_provider=run_input.model.model_provider,
|
||||
model=run_input.model.model,
|
||||
credentials=run_input.model.credentials,
|
||||
model_settings=run_input.model.model_settings or None,
|
||||
),
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
if run_input.output is not None:
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_AGENT_OUTPUT_LAYER_ID,
|
||||
type=DIFY_OUTPUT_LAYER_TYPE_ID,
|
||||
metadata=run_input.metadata,
|
||||
config=DifyOutputLayerConfig(
|
||||
json_schema=run_input.output.json_schema,
|
||||
name=run_input.output.name,
|
||||
description=run_input.output.description,
|
||||
strict=run_input.output.strict,
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
return CreateRunRequest(
|
||||
composition=RunComposition(layers=layers),
|
||||
execution_context=run_input.execution_context,
|
||||
purpose=run_input.purpose,
|
||||
idempotency_key=run_input.idempotency_key,
|
||||
metadata=run_input.metadata,
|
||||
session_snapshot=run_input.session_snapshot,
|
||||
on_exit=LayerExitSignals(
|
||||
default=ExitIntent.SUSPEND if run_input.suspend_on_exit else ExitIntent.DELETE,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
_SENSITIVE_KEY_PARTS = ("secret", "credential", "token", "password", "api_key")
|
||||
|
||||
|
||||
def redact_for_agent_backend_log(value: object) -> object:
|
||||
"""Return a JSON-like copy with credential-bearing keys redacted for logs/tests."""
|
||||
if isinstance(value, BaseModel):
|
||||
return redact_for_agent_backend_log(value.model_dump(mode="json", warnings=False))
|
||||
if isinstance(value, dict):
|
||||
redacted: dict[object, object] = {}
|
||||
for key, item in value.items():
|
||||
key_text = str(key).lower()
|
||||
if any(part in key_text for part in _SENSITIVE_KEY_PARTS):
|
||||
redacted[key] = "[REDACTED]"
|
||||
else:
|
||||
redacted[key] = redact_for_agent_backend_log(item)
|
||||
return redacted
|
||||
if isinstance(value, list):
|
||||
return [redact_for_agent_backend_log(item) for item in value]
|
||||
return value
|
||||
@ -14,7 +14,6 @@ from .plugin import (
|
||||
setup_system_trigger_oauth_client,
|
||||
transform_datasource_credentials,
|
||||
)
|
||||
from .rbac import migrate_member_roles_to_rbac
|
||||
from .retention import (
|
||||
archive_workflow_runs,
|
||||
clean_expired_messages,
|
||||
@ -56,7 +55,6 @@ __all__ = [
|
||||
"migrate_annotation_vector_database",
|
||||
"migrate_data_for_plugin",
|
||||
"migrate_knowledge_vector_database",
|
||||
"migrate_member_roles_to_rbac",
|
||||
"migrate_oss",
|
||||
"old_metadata_migration",
|
||||
"remove_orphaned_files_on_storage",
|
||||
|
||||
@ -11,7 +11,6 @@ from configs import dify_config
|
||||
from core.helper import encrypter
|
||||
from core.plugin.entities.plugin_daemon import CredentialType
|
||||
from core.plugin.impl.plugin import PluginInstaller
|
||||
from core.plugin.plugin_service import PluginService
|
||||
from core.tools.utils.system_encryption import encrypt_system_params
|
||||
from extensions.ext_database import db
|
||||
from models import Tenant
|
||||
@ -21,6 +20,7 @@ from models.source import DataSourceApiKeyAuthBinding, DataSourceOauthBinding
|
||||
from models.tools import ToolOAuthSystemClient
|
||||
from services.plugin.data_migration import PluginDataMigration
|
||||
from services.plugin.plugin_migration import PluginMigration
|
||||
from services.plugin.plugin_service import PluginService
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@ -1,109 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import click
|
||||
from sqlalchemy import select
|
||||
|
||||
from core.db.session_factory import session_factory
|
||||
from models import TenantAccountJoin, TenantAccountRole
|
||||
from services.enterprise.rbac_service import ListOption, RBACService
|
||||
|
||||
|
||||
def _resolve_builtin_role_id(tenant_id: str, operator_account_id: str, legacy_role: str) -> str:
|
||||
"""Resolve a legacy workspace role to the current tenant's builtin RBAC role id.
|
||||
|
||||
The migration replays the old `TenantAccountJoin.role` values onto the
|
||||
RBAC member-role binding API. Builtin RBAC roles are tenant-scoped and
|
||||
identified by runtime ids, so the command must look them up per tenant.
|
||||
"""
|
||||
expected_builtin_name = {
|
||||
TenantAccountRole.OWNER.value: "所有者",
|
||||
TenantAccountRole.ADMIN.value: "管理者",
|
||||
TenantAccountRole.EDITOR.value: "编辑者",
|
||||
TenantAccountRole.NORMAL.value: "普通用户",
|
||||
TenantAccountRole.DATASET_OPERATOR.value: "知识库操作员",
|
||||
}.get(legacy_role)
|
||||
if not expected_builtin_name:
|
||||
raise ValueError(f"Unsupported legacy workspace role: {legacy_role}")
|
||||
|
||||
roles = RBACService.Roles.list(
|
||||
tenant_id=tenant_id,
|
||||
account_id=operator_account_id,
|
||||
options=ListOption(page_number=1, results_per_page=100),
|
||||
).data
|
||||
for role in roles:
|
||||
if role.is_builtin and role.category == "global_system_default" and role.name == expected_builtin_name:
|
||||
return str(role.id)
|
||||
|
||||
raise ValueError(f"Builtin RBAC role not found for tenant={tenant_id}, legacy_role={legacy_role}")
|
||||
|
||||
|
||||
@click.command("rbac-migrate-member-roles", help="Migrate legacy workspace member roles into RBAC member-role bindings.")
|
||||
@click.option("--tenant-id", help="Only migrate a single workspace.")
|
||||
@click.option("--dry-run", is_flag=True, default=False, help="Preview the migration without writing RBAC bindings.")
|
||||
def migrate_member_roles_to_rbac(tenant_id: str | None, dry_run: bool) -> None:
|
||||
"""Backfill RBAC member-role bindings from legacy `TenantAccountJoin.role` data.
|
||||
|
||||
This is an offline migration command for workspaces that already have
|
||||
members in the legacy role model but need matching records in the RBAC
|
||||
member-role binding store.
|
||||
"""
|
||||
click.echo(click.style("Starting RBAC member-role migration.", fg="green"))
|
||||
|
||||
with session_factory.create_session() as session:
|
||||
stmt = select(TenantAccountJoin).order_by(TenantAccountJoin.tenant_id.asc(), TenantAccountJoin.id.asc())
|
||||
if tenant_id:
|
||||
stmt = stmt.where(TenantAccountJoin.tenant_id == tenant_id)
|
||||
|
||||
joins = list(session.scalars(stmt).all())
|
||||
|
||||
if not joins:
|
||||
click.echo(click.style("No workspace members found for migration.", fg="yellow"))
|
||||
return
|
||||
|
||||
owner_account_by_tenant: dict[str, str] = {}
|
||||
resolved_role_ids: dict[tuple[str, str], str] = {}
|
||||
migrated_count = 0
|
||||
|
||||
for join in joins:
|
||||
workspace_id = str(join.tenant_id)
|
||||
member_account_id = str(join.account_id)
|
||||
legacy_role = str(join.role)
|
||||
|
||||
if workspace_id not in owner_account_by_tenant:
|
||||
owner_join = next(
|
||||
(
|
||||
item
|
||||
for item in joins
|
||||
if str(item.tenant_id) == workspace_id and str(item.role) == TenantAccountRole.OWNER.value
|
||||
),
|
||||
None,
|
||||
)
|
||||
if not owner_join:
|
||||
raise ValueError(f"Workspace owner not found for tenant={workspace_id}")
|
||||
owner_account_by_tenant[workspace_id] = str(owner_join.account_id)
|
||||
|
||||
operator_account_id = owner_account_by_tenant[workspace_id]
|
||||
cache_key = (workspace_id, legacy_role)
|
||||
if cache_key not in resolved_role_ids:
|
||||
resolved_role_ids[cache_key] = _resolve_builtin_role_id(workspace_id, operator_account_id, legacy_role)
|
||||
|
||||
resolved_role_id = resolved_role_ids[cache_key]
|
||||
click.echo(
|
||||
f"tenant={workspace_id} member={member_account_id} legacy_role={legacy_role} -> rbac_role_id={resolved_role_id}"
|
||||
)
|
||||
|
||||
if dry_run:
|
||||
continue
|
||||
|
||||
RBACService.MemberRoles.replace(
|
||||
tenant_id=workspace_id,
|
||||
account_id=operator_account_id,
|
||||
member_account_id=member_account_id,
|
||||
role_ids=[resolved_role_id],
|
||||
)
|
||||
migrated_count += 1
|
||||
|
||||
if dry_run:
|
||||
click.echo(click.style("Dry run completed. No RBAC bindings were written.", fg="yellow"))
|
||||
else:
|
||||
click.echo(click.style(f"RBAC member-role migration completed. Migrated {migrated_count} members.", fg="green"))
|
||||
@ -1,6 +1,6 @@
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Any, override
|
||||
from typing import Any
|
||||
|
||||
from pydantic.fields import FieldInfo
|
||||
from pydantic_settings import BaseSettings, PydanticBaseSettingsSource, SettingsConfigDict, TomlConfigSettingsSource
|
||||
@ -25,7 +25,6 @@ class RemoteSettingsSourceFactory(PydanticBaseSettingsSource):
|
||||
def __init__(self, settings_cls: type[BaseSettings]):
|
||||
super().__init__(settings_cls)
|
||||
|
||||
@override
|
||||
def get_field_value(self, field: FieldInfo, field_name: str) -> tuple[Any, str, bool]:
|
||||
raise NotImplementedError
|
||||
|
||||
@ -91,7 +90,6 @@ class DifyConfig(
|
||||
# Thanks for your concentration and consideration.
|
||||
|
||||
@classmethod
|
||||
@override
|
||||
def settings_customise_sources(
|
||||
cls,
|
||||
settings_cls: type[BaseSettings],
|
||||
|
||||
@ -29,11 +29,6 @@ class EnterpriseFeatureConfig(BaseSettings):
|
||||
"This helps gain runtime performance by trading off consistency.",
|
||||
)
|
||||
|
||||
RBAC_ENABLED: bool = Field(
|
||||
description="Enable enterprise RBAC APIs. When disabled, compatibility responses fall back to legacy roles.",
|
||||
default=False,
|
||||
)
|
||||
|
||||
|
||||
class EnterpriseTelemetryConfig(BaseSettings):
|
||||
"""
|
||||
|
||||
@ -1,4 +1,3 @@
|
||||
from configs.extra.agent_backend_config import AgentBackendConfig
|
||||
from configs.extra.archive_config import ArchiveStorageConfig
|
||||
from configs.extra.notion_config import NotionConfig
|
||||
from configs.extra.sentry_config import SentryConfig
|
||||
@ -6,7 +5,6 @@ from configs.extra.sentry_config import SentryConfig
|
||||
|
||||
class ExtraServiceConfig(
|
||||
# place the configs in alphabet order
|
||||
AgentBackendConfig,
|
||||
ArchiveStorageConfig,
|
||||
NotionConfig,
|
||||
SentryConfig,
|
||||
|
||||
@ -1,23 +0,0 @@
|
||||
from pydantic import Field
|
||||
from pydantic_settings import BaseSettings
|
||||
|
||||
|
||||
class AgentBackendConfig(BaseSettings):
|
||||
"""
|
||||
Configuration settings for the Agent backend runtime integration.
|
||||
"""
|
||||
|
||||
AGENT_BACKEND_BASE_URL: str | None = Field(
|
||||
description="Base URL for the Dify Agent backend service.",
|
||||
default=None,
|
||||
)
|
||||
|
||||
AGENT_BACKEND_USE_FAKE: bool = Field(
|
||||
description="Use the deterministic in-process fake Agent backend client.",
|
||||
default=False,
|
||||
)
|
||||
|
||||
AGENT_BACKEND_FAKE_SCENARIO: str = Field(
|
||||
description="Scenario used by the fake Agent backend client.",
|
||||
default="success",
|
||||
)
|
||||
@ -265,11 +265,6 @@ class PluginConfig(BaseSettings):
|
||||
default=60 * 60,
|
||||
)
|
||||
|
||||
PLUGIN_MODEL_PROVIDERS_CACHE_TTL: PositiveInt = Field(
|
||||
description="TTL in seconds for caching tenant plugin model providers in Redis",
|
||||
default=60 * 60 * 24,
|
||||
)
|
||||
|
||||
PLUGIN_MAX_FILE_SIZE: PositiveInt = Field(
|
||||
description="Maximum allowed size (bytes) for plugin-generated files",
|
||||
default=50 * 1024 * 1024,
|
||||
|
||||
@ -2,7 +2,6 @@ from typing import Literal, Protocol, cast
|
||||
from urllib.parse import quote_plus, urlunparse
|
||||
|
||||
from pydantic import AliasChoices, Field
|
||||
from pydantic.types import NonNegativeInt
|
||||
from pydantic_settings import BaseSettings
|
||||
|
||||
|
||||
@ -71,24 +70,6 @@ class RedisPubSubConfig(BaseSettings):
|
||||
default=600,
|
||||
)
|
||||
|
||||
PUBSUB_LISTENER_JOIN_TIMEOUT_MS: NonNegativeInt = Field(
|
||||
validation_alias=AliasChoices("EVENT_BUS_LISTENER_JOIN_TIMEOUT_MS", "PUBSUB_LISTENER_JOIN_TIMEOUT_MS"),
|
||||
description=(
|
||||
"Maximum time (milliseconds) that ``Subscription.close()`` waits for its listener thread to "
|
||||
"finish before returning. Bounds the tail latency between a terminal event being delivered to "
|
||||
"an SSE client and the response stream actually closing.\n\n"
|
||||
"The listener thread blocks on a polling read (XREAD BLOCK for streams, get_message timeout "
|
||||
"for pubsub/sharded) with a fixed 1s window, so close() naturally has to wait up to ~1s for "
|
||||
"the thread to notice the subscription was closed. Setting this lower (e.g. 100) lets close() "
|
||||
"return promptly while the daemon listener thread cleans itself up on the next poll "
|
||||
"boundary - safe because the listener holds no critical state and exits within one poll "
|
||||
"window. Setting it higher (e.g. 5000) gives the listener more grace before close() gives up "
|
||||
"and logs a warning. Default 2000ms preserves the pre-change behaviour.\n\n"
|
||||
"Also accepts ENV: EVENT_BUS_LISTENER_JOIN_TIMEOUT_MS."
|
||||
),
|
||||
default=2000,
|
||||
)
|
||||
|
||||
def _build_default_pubsub_url(self) -> str:
|
||||
defaults = _redis_defaults(self)
|
||||
if not defaults.REDIS_HOST or not defaults.REDIS_PORT:
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
from collections.abc import Mapping
|
||||
from typing import Any, override
|
||||
from typing import Any
|
||||
|
||||
from pydantic import Field
|
||||
from pydantic.fields import FieldInfo
|
||||
@ -48,7 +48,6 @@ class ApolloSettingsSource(RemoteSettingsSource):
|
||||
self.namespace = configs["APOLLO_NAMESPACE"]
|
||||
self.remote_configs = self.client.get_all_dicts(self.namespace)
|
||||
|
||||
@override
|
||||
def get_field_value(self, field: FieldInfo, field_name: str) -> tuple[Any, str, bool]:
|
||||
if not isinstance(self.remote_configs, dict):
|
||||
raise ValueError(f"remote configs is not dict, but {type(self.remote_configs)}")
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
import logging
|
||||
import os
|
||||
from collections.abc import Mapping
|
||||
from typing import Any, override
|
||||
from typing import Any
|
||||
|
||||
from pydantic.fields import FieldInfo
|
||||
|
||||
@ -41,7 +41,6 @@ class NacosSettingsSource(RemoteSettingsSource):
|
||||
except Exception as e:
|
||||
raise RuntimeError(f"Failed to parse config: {e}")
|
||||
|
||||
@override
|
||||
def get_field_value(self, field: FieldInfo, field_name: str) -> tuple[Any, str, bool]:
|
||||
field_value = self.remote_configs.get(field_name)
|
||||
if field_value is None:
|
||||
|
||||
@ -1,91 +0,0 @@
|
||||
"""Global pytest hooks for Dify backend tests.
|
||||
|
||||
This root conftest is loaded before package-specific conftests, which lets tests opt
|
||||
into Docker-backed middleware before application modules read environment config.
|
||||
It intentionally lives at the API root because pytest applies conftest.py files to
|
||||
tests below their directory, and this setup is shared by api/tests and api/providers.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from tests.pytest_dify import (
|
||||
DEFAULT_MIDDLEWARE_SERVICES,
|
||||
DEFAULT_VDB_SERVICES,
|
||||
DockerComposeStack,
|
||||
build_middleware_stack,
|
||||
build_vdb_stack,
|
||||
ensure_backend_test_environment,
|
||||
ensure_compose_env_files,
|
||||
parse_services,
|
||||
)
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
_DIFY_COMPOSE_STACKS_KEY = pytest.StashKey[list[DockerComposeStack]]()
|
||||
|
||||
# This must run at import time because package-specific conftests can import the
|
||||
# Flask app before pytest_configure hooks from this file are called.
|
||||
ensure_backend_test_environment(_REPO_ROOT)
|
||||
|
||||
|
||||
def pytest_addoption(parser: pytest.Parser) -> None:
|
||||
group = parser.getgroup("dify")
|
||||
group.addoption(
|
||||
"--start-middleware",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Start the Docker middleware services needed by API integration tests.",
|
||||
)
|
||||
group.addoption(
|
||||
"--middleware-services",
|
||||
default=",".join(DEFAULT_MIDDLEWARE_SERVICES),
|
||||
help="Comma-separated services from docker/docker-compose.middleware.yaml to start.",
|
||||
)
|
||||
group.addoption(
|
||||
"--start-vdb",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Start vector-store Docker services for VDB integration tests.",
|
||||
)
|
||||
group.addoption(
|
||||
"--vdb-services",
|
||||
default=",".join(DEFAULT_VDB_SERVICES),
|
||||
help="Comma-separated services from docker/docker-compose.yaml to start for VDB tests.",
|
||||
)
|
||||
|
||||
|
||||
def pytest_configure(config: pytest.Config) -> None:
|
||||
config.stash[_DIFY_COMPOSE_STACKS_KEY] = []
|
||||
|
||||
|
||||
def pytest_sessionstart(session: pytest.Session) -> None:
|
||||
config = session.config
|
||||
if hasattr(config, "workerinput"):
|
||||
return
|
||||
|
||||
stacks: list[DockerComposeStack] = []
|
||||
if config.getoption("start_middleware"):
|
||||
ensure_compose_env_files(_REPO_ROOT)
|
||||
stack = build_middleware_stack(_REPO_ROOT, parse_services(config.getoption("middleware_services")))
|
||||
stack.up()
|
||||
stacks.append(stack)
|
||||
|
||||
if config.getoption("start_vdb"):
|
||||
ensure_compose_env_files(_REPO_ROOT)
|
||||
stack = build_vdb_stack(_REPO_ROOT, parse_services(config.getoption("vdb_services")))
|
||||
stack.up()
|
||||
stacks.append(stack)
|
||||
|
||||
config.stash[_DIFY_COMPOSE_STACKS_KEY] = stacks
|
||||
|
||||
|
||||
def pytest_unconfigure(config: pytest.Config) -> None:
|
||||
if hasattr(config, "workerinput"):
|
||||
return
|
||||
|
||||
stacks = config.stash.get(_DIFY_COMPOSE_STACKS_KEY, [])
|
||||
for stack in reversed(stacks):
|
||||
stack.down()
|
||||
@ -10,7 +10,7 @@ import threading
|
||||
from abc import ABC, abstractmethod
|
||||
from collections.abc import Callable, Generator
|
||||
from contextlib import AbstractContextManager, contextmanager
|
||||
from typing import Any, Protocol, final, override, runtime_checkable
|
||||
from typing import Any, Protocol, final, runtime_checkable
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
@ -133,12 +133,10 @@ class NullAppContext(AppContext):
|
||||
self._config = config or {}
|
||||
self._extensions: dict[str, Any] = {}
|
||||
|
||||
@override
|
||||
def get_config(self, key: str, default: Any = None) -> Any:
|
||||
"""Get configuration value by key."""
|
||||
return self._config.get(key, default)
|
||||
|
||||
@override
|
||||
def get_extension(self, name: str) -> Any:
|
||||
"""Get extension by name."""
|
||||
return self._extensions.get(name)
|
||||
@ -148,7 +146,6 @@ class NullAppContext(AppContext):
|
||||
self._extensions[name] = extension
|
||||
|
||||
@contextmanager
|
||||
@override
|
||||
def enter(self) -> Generator[None, None, None]:
|
||||
"""Enter null context (no-op)."""
|
||||
yield
|
||||
|
||||
@ -6,7 +6,7 @@ import contextvars
|
||||
import threading
|
||||
from collections.abc import Generator
|
||||
from contextlib import contextmanager
|
||||
from typing import Any, final, override
|
||||
from typing import Any, final
|
||||
|
||||
from flask import Flask, current_app, g
|
||||
|
||||
@ -30,18 +30,15 @@ class FlaskAppContext(AppContext):
|
||||
"""
|
||||
self._flask_app = flask_app
|
||||
|
||||
@override
|
||||
def get_config(self, key: str, default: Any = None) -> Any:
|
||||
"""Get configuration value from Flask app config."""
|
||||
return self._flask_app.config.get(key, default)
|
||||
|
||||
@override
|
||||
def get_extension(self, name: str) -> Any:
|
||||
"""Get Flask extension by name."""
|
||||
return self._flask_app.extensions.get(name)
|
||||
|
||||
@contextmanager
|
||||
@override
|
||||
def enter(self) -> Generator[None, None, None]:
|
||||
"""Enter Flask app context."""
|
||||
with self._flask_app.app_context():
|
||||
|
||||
@ -2,9 +2,8 @@ from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, computed_field
|
||||
from pydantic import BaseModel, ConfigDict, computed_field
|
||||
|
||||
from fields.base import ResponseModel
|
||||
from graphon.file import helpers as file_helpers
|
||||
from models.model import IconType
|
||||
|
||||
@ -20,113 +19,6 @@ class SystemParameters(BaseModel):
|
||||
workflow_file_upload_limit: int
|
||||
|
||||
|
||||
class SimpleResultResponse(ResponseModel):
|
||||
result: str
|
||||
|
||||
|
||||
class SimpleResultMessageResponse(ResponseModel):
|
||||
result: str
|
||||
message: str
|
||||
|
||||
|
||||
class SimpleMessageResponse(ResponseModel):
|
||||
message: str
|
||||
|
||||
|
||||
class SimpleDataResponse(ResponseModel):
|
||||
data: str
|
||||
|
||||
|
||||
class SimpleResultDataResponse(ResponseModel):
|
||||
result: str
|
||||
data: str
|
||||
|
||||
|
||||
class SimpleResultStringListResponse(ResponseModel):
|
||||
result: str
|
||||
data: list[str]
|
||||
|
||||
|
||||
class SimpleResultOptionalDataResponse(ResponseModel):
|
||||
result: str
|
||||
data: str | None = None
|
||||
|
||||
|
||||
class AccessTokenData(ResponseModel):
|
||||
access_token: str
|
||||
|
||||
|
||||
class AccessTokenResultResponse(ResponseModel):
|
||||
result: str
|
||||
data: AccessTokenData
|
||||
|
||||
|
||||
class VerificationTokenResponse(ResponseModel):
|
||||
is_valid: bool
|
||||
email: str
|
||||
token: str
|
||||
|
||||
|
||||
class LoginStatusResponse(ResponseModel):
|
||||
logged_in: bool
|
||||
app_logged_in: bool
|
||||
|
||||
|
||||
class AccessModeResponse(ResponseModel):
|
||||
access_mode: str = Field(serialization_alias="accessMode", validation_alias="accessMode")
|
||||
|
||||
|
||||
class BooleanResultResponse(ResponseModel):
|
||||
result: bool
|
||||
|
||||
|
||||
class SuccessResponse(ResponseModel):
|
||||
success: bool
|
||||
|
||||
|
||||
class UsageCheckResponse(ResponseModel):
|
||||
is_using: bool
|
||||
|
||||
|
||||
class UsageCountResponse(ResponseModel):
|
||||
is_using: bool
|
||||
count: int
|
||||
|
||||
|
||||
class IndexInfoResponse(ResponseModel):
|
||||
welcome: str
|
||||
api_version: str
|
||||
server_version: str
|
||||
|
||||
|
||||
class AvatarUrlResponse(ResponseModel):
|
||||
avatar_url: str
|
||||
|
||||
|
||||
class TextContentResponse(ResponseModel):
|
||||
content: str
|
||||
|
||||
|
||||
class AllowedExtensionsResponse(ResponseModel):
|
||||
allowed_extensions: list[str]
|
||||
|
||||
|
||||
class UrlResponse(ResponseModel):
|
||||
url: str
|
||||
|
||||
|
||||
class RedirectUrlResponse(ResponseModel):
|
||||
redirect_url: str
|
||||
|
||||
|
||||
class ApiBaseUrlResponse(ResponseModel):
|
||||
api_base_url: str
|
||||
|
||||
|
||||
class NewAppResponse(ResponseModel):
|
||||
new_app_id: str
|
||||
|
||||
|
||||
class Parameters(BaseModel):
|
||||
opening_statement: str | None = None
|
||||
suggested_questions: list[str]
|
||||
|
||||
@ -36,24 +36,6 @@ class FileInfo(BaseModel):
|
||||
size: int
|
||||
|
||||
|
||||
def decode_remote_url(url: str, query_string: bytes | str = b"") -> str:
|
||||
decoded_url = urllib.parse.unquote(url)
|
||||
if isinstance(query_string, bytes):
|
||||
raw_query = query_string.decode()
|
||||
else:
|
||||
raw_query = query_string
|
||||
if not raw_query:
|
||||
return decoded_url
|
||||
|
||||
if decoded_url.endswith(("?", "&")):
|
||||
separator = ""
|
||||
elif urllib.parse.urlsplit(decoded_url).query:
|
||||
separator = "&"
|
||||
else:
|
||||
separator = "?"
|
||||
return f"{decoded_url}{separator}{raw_query}"
|
||||
|
||||
|
||||
def guess_file_info_from_response(response: httpx.Response):
|
||||
url = str(response.url)
|
||||
# Try to extract filename from URL
|
||||
|
||||
@ -44,8 +44,6 @@ from . import (
|
||||
spec,
|
||||
version,
|
||||
)
|
||||
from .agent import composer as agent_composer
|
||||
from .agent import roster as agent_roster
|
||||
|
||||
# Import app controllers
|
||||
from .app import (
|
||||
@ -133,7 +131,6 @@ from .workspace import (
|
||||
model_providers,
|
||||
models,
|
||||
plugin,
|
||||
rbac,
|
||||
tool_providers,
|
||||
trigger_providers,
|
||||
workspace,
|
||||
@ -146,9 +143,7 @@ __all__ = [
|
||||
"activate",
|
||||
"advanced_prompt_template",
|
||||
"agent",
|
||||
"agent_composer",
|
||||
"agent_providers",
|
||||
"agent_roster",
|
||||
"annotation",
|
||||
"api",
|
||||
"apikey",
|
||||
@ -202,7 +197,6 @@ __all__ = [
|
||||
"rag_pipeline_draft_variable",
|
||||
"rag_pipeline_import",
|
||||
"rag_pipeline_workflow",
|
||||
"rbac",
|
||||
"recommended_app",
|
||||
"saved_message",
|
||||
"setup",
|
||||
|
||||
@ -1,3 +0,0 @@
|
||||
from . import composer, roster
|
||||
|
||||
__all__ = ["composer", "roster"]
|
||||
@ -1,153 +0,0 @@
|
||||
from flask_restx import Resource
|
||||
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from models.model import AppMode
|
||||
from services.agent.composer_service import AgentComposerService
|
||||
from services.agent.composer_validator import ComposerConfigValidator
|
||||
from services.entities.agent_entities import ComposerSavePayload
|
||||
|
||||
register_schema_models(console_ns, ComposerSavePayload)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer")
|
||||
class WorkflowAgentComposerApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def get(self, app_model, node_id: str):
|
||||
_, tenant_id = current_account_with_tenant()
|
||||
return AgentComposerService.load_workflow_composer(
|
||||
tenant_id=tenant_id,
|
||||
app_id=app_model.id,
|
||||
node_id=node_id,
|
||||
)
|
||||
|
||||
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def put(self, app_model, node_id: str):
|
||||
account, tenant_id = current_account_with_tenant()
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
return AgentComposerService.save_workflow_composer(
|
||||
tenant_id=tenant_id,
|
||||
app_id=app_model.id,
|
||||
node_id=node_id,
|
||||
account_id=account.id,
|
||||
payload=payload,
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer/validate")
|
||||
class WorkflowAgentComposerValidateApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def post(self, app_model, node_id: str):
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
ComposerConfigValidator.validate_save_payload(payload)
|
||||
return {"result": "success", "errors": []}
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer/candidates")
|
||||
class WorkflowAgentComposerCandidatesApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def get(self, app_model, node_id: str):
|
||||
return AgentComposerService.get_workflow_candidates(app_id=app_model.id)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer/impact")
|
||||
class WorkflowAgentComposerImpactApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def post(self, app_model, node_id: str):
|
||||
_, tenant_id = current_account_with_tenant()
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
current_snapshot_id = payload.binding.current_snapshot_id if payload.binding else None
|
||||
if not current_snapshot_id:
|
||||
return {"current_snapshot_id": None, "workflow_node_count": 0, "bindings": []}
|
||||
return AgentComposerService.calculate_impact(tenant_id=tenant_id, current_snapshot_id=current_snapshot_id)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer/save-to-roster")
|
||||
class WorkflowAgentComposerSaveToRosterApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def post(self, app_model, node_id: str):
|
||||
account, tenant_id = current_account_with_tenant()
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
return AgentComposerService.save_workflow_composer(
|
||||
tenant_id=tenant_id,
|
||||
app_id=app_model.id,
|
||||
node_id=node_id,
|
||||
account_id=account.id,
|
||||
payload=payload,
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/agent-composer")
|
||||
class AgentAppComposerApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model()
|
||||
def get(self, app_model):
|
||||
_, tenant_id = current_account_with_tenant()
|
||||
return AgentComposerService.load_agent_app_composer(tenant_id=tenant_id, app_id=app_model.id)
|
||||
|
||||
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@get_app_model()
|
||||
def put(self, app_model):
|
||||
account, tenant_id = current_account_with_tenant()
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
return AgentComposerService.save_agent_app_composer(
|
||||
tenant_id=tenant_id,
|
||||
app_id=app_model.id,
|
||||
account_id=account.id,
|
||||
payload=payload,
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/agent-composer/validate")
|
||||
class AgentAppComposerValidateApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model()
|
||||
def post(self, app_model):
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
ComposerConfigValidator.validate_save_payload(payload)
|
||||
return {"result": "success", "errors": []}
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/agent-composer/candidates")
|
||||
class AgentAppComposerCandidatesApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model()
|
||||
def get(self, app_model):
|
||||
return AgentComposerService.get_agent_app_candidates(app_id=app_model.id)
|
||||
@ -1,132 +0,0 @@
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
|
||||
from extensions.ext_database import db
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from services.agent.roster_service import AgentRosterService
|
||||
from services.entities.agent_entities import RosterAgentCreatePayload, RosterAgentUpdatePayload, RosterListQuery
|
||||
|
||||
|
||||
class AgentInviteOptionsQuery(RosterListQuery):
|
||||
app_id: str | None = Field(default=None, description="Workflow app id for in-current-workflow markers")
|
||||
|
||||
|
||||
class AgentIdPath(BaseModel):
|
||||
agent_id: str
|
||||
|
||||
|
||||
register_schema_models(
|
||||
console_ns,
|
||||
AgentInviteOptionsQuery,
|
||||
AgentIdPath,
|
||||
RosterAgentCreatePayload,
|
||||
RosterAgentUpdatePayload,
|
||||
RosterListQuery,
|
||||
)
|
||||
|
||||
|
||||
def _agent_roster_service() -> AgentRosterService:
|
||||
return AgentRosterService(db.session)
|
||||
|
||||
|
||||
@console_ns.route("/agents")
|
||||
class AgentRosterListApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self):
|
||||
_, tenant_id = current_account_with_tenant()
|
||||
query = RosterListQuery.model_validate(request.args.to_dict(flat=True))
|
||||
return _agent_roster_service().list_roster_agents(
|
||||
tenant_id=tenant_id, page=query.page, limit=query.limit, keyword=query.keyword
|
||||
)
|
||||
|
||||
@console_ns.expect(console_ns.models[RosterAgentCreatePayload.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
def post(self):
|
||||
account, tenant_id = current_account_with_tenant()
|
||||
payload = RosterAgentCreatePayload.model_validate(console_ns.payload or {})
|
||||
service = _agent_roster_service()
|
||||
agent = service.create_roster_agent(tenant_id=tenant_id, account_id=account.id, payload=payload)
|
||||
return service.get_roster_agent_detail(tenant_id=tenant_id, agent_id=agent.id), 201
|
||||
|
||||
|
||||
@console_ns.route("/agents/invite-options")
|
||||
class AgentInviteOptionsApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self):
|
||||
_, tenant_id = current_account_with_tenant()
|
||||
query = AgentInviteOptionsQuery.model_validate(request.args.to_dict(flat=True))
|
||||
return _agent_roster_service().list_invite_options(
|
||||
tenant_id=tenant_id,
|
||||
page=query.page,
|
||||
limit=query.limit,
|
||||
keyword=query.keyword,
|
||||
app_id=query.app_id,
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/agents/<uuid:agent_id>")
|
||||
class AgentRosterDetailApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, agent_id: UUID):
|
||||
_, tenant_id = current_account_with_tenant()
|
||||
return _agent_roster_service().get_roster_agent_detail(tenant_id=tenant_id, agent_id=str(agent_id))
|
||||
|
||||
@console_ns.expect(console_ns.models[RosterAgentUpdatePayload.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
def patch(self, agent_id: UUID):
|
||||
account, tenant_id = current_account_with_tenant()
|
||||
payload = RosterAgentUpdatePayload.model_validate(console_ns.payload or {})
|
||||
return _agent_roster_service().update_roster_agent(
|
||||
tenant_id=tenant_id, agent_id=str(agent_id), account_id=account.id, payload=payload
|
||||
)
|
||||
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
def delete(self, agent_id: UUID):
|
||||
account, tenant_id = current_account_with_tenant()
|
||||
_agent_roster_service().archive_roster_agent(tenant_id=tenant_id, agent_id=str(agent_id), account_id=account.id)
|
||||
return "", 204
|
||||
|
||||
|
||||
@console_ns.route("/agents/<uuid:agent_id>/versions")
|
||||
class AgentRosterVersionsApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, agent_id: UUID):
|
||||
_, tenant_id = current_account_with_tenant()
|
||||
return {"data": _agent_roster_service().list_agent_versions(tenant_id=tenant_id, agent_id=str(agent_id))}
|
||||
|
||||
|
||||
@console_ns.route("/agents/<uuid:agent_id>/versions/<uuid:version_id>")
|
||||
class AgentRosterVersionDetailApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, agent_id: UUID, version_id: UUID):
|
||||
_, tenant_id = current_account_with_tenant()
|
||||
return _agent_roster_service().get_agent_version_detail(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
version_id=str(version_id),
|
||||
)
|
||||
@ -1,5 +1,4 @@
|
||||
from datetime import datetime
|
||||
from uuid import UUID
|
||||
|
||||
import flask_restx
|
||||
from flask_restx import Resource
|
||||
@ -147,7 +146,7 @@ class BaseApiKeyResource(Resource):
|
||||
db.session.execute(delete(ApiToken).where(ApiToken.id == api_key_id))
|
||||
db.session.commit()
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:resource_id>/api-keys")
|
||||
@ -156,7 +155,7 @@ class AppApiKeyListResource(BaseApiKeyListResource):
|
||||
@console_ns.doc(description="Get all API keys for an app")
|
||||
@console_ns.doc(params={"resource_id": "App ID"})
|
||||
@console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
|
||||
def get(self, resource_id: UUID):
|
||||
def get(self, resource_id): # type: ignore
|
||||
"""Get all API keys for an app"""
|
||||
return super().get(resource_id)
|
||||
|
||||
@ -165,7 +164,7 @@ class AppApiKeyListResource(BaseApiKeyListResource):
|
||||
@console_ns.doc(params={"resource_id": "App ID"})
|
||||
@console_ns.response(201, "API key created successfully", console_ns.models[ApiKeyItem.__name__])
|
||||
@console_ns.response(400, "Maximum keys exceeded")
|
||||
def post(self, resource_id: UUID):
|
||||
def post(self, resource_id): # type: ignore
|
||||
"""Create a new API key for an app"""
|
||||
return super().post(resource_id)
|
||||
|
||||
@ -181,9 +180,9 @@ class AppApiKeyResource(BaseApiKeyResource):
|
||||
@console_ns.doc(description="Delete an API key for an app")
|
||||
@console_ns.doc(params={"resource_id": "App ID", "api_key_id": "API key ID"})
|
||||
@console_ns.response(204, "API key deleted successfully")
|
||||
def delete(self, resource_id: UUID, api_key_id: UUID):
|
||||
def delete(self, resource_id, api_key_id):
|
||||
"""Delete an API key for an app"""
|
||||
return super().delete(str(resource_id), str(api_key_id))
|
||||
return super().delete(resource_id, api_key_id)
|
||||
|
||||
resource_type = ApiTokenType.APP
|
||||
resource_model = App
|
||||
@ -196,7 +195,7 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
|
||||
@console_ns.doc(description="Get all API keys for a dataset")
|
||||
@console_ns.doc(params={"resource_id": "Dataset ID"})
|
||||
@console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
|
||||
def get(self, resource_id: UUID):
|
||||
def get(self, resource_id): # type: ignore
|
||||
"""Get all API keys for a dataset"""
|
||||
return super().get(resource_id)
|
||||
|
||||
@ -205,7 +204,7 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
|
||||
@console_ns.doc(params={"resource_id": "Dataset ID"})
|
||||
@console_ns.response(201, "API key created successfully", console_ns.models[ApiKeyItem.__name__])
|
||||
@console_ns.response(400, "Maximum keys exceeded")
|
||||
def post(self, resource_id: UUID):
|
||||
def post(self, resource_id): # type: ignore
|
||||
"""Create a new API key for a dataset"""
|
||||
return super().post(resource_id)
|
||||
|
||||
@ -221,9 +220,9 @@ class DatasetApiKeyResource(BaseApiKeyResource):
|
||||
@console_ns.doc(description="Delete an API key for a dataset")
|
||||
@console_ns.doc(params={"resource_id": "Dataset ID", "api_key_id": "API key ID"})
|
||||
@console_ns.response(204, "API key deleted successfully")
|
||||
def delete(self, resource_id: UUID, api_key_id: UUID):
|
||||
def delete(self, resource_id, api_key_id):
|
||||
"""Delete an API key for a dataset"""
|
||||
return super().delete(str(resource_id), str(api_key_id))
|
||||
return super().delete(resource_id, api_key_id)
|
||||
|
||||
resource_type = ApiTokenType.DATASET
|
||||
resource_model = Dataset
|
||||
|
||||
@ -159,15 +159,13 @@ class AppAnnotationSettingUpdateApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
def post(self, app_id: UUID, annotation_setting_id: UUID):
|
||||
annotation_setting_id_str = str(annotation_setting_id)
|
||||
def post(self, app_id: UUID, annotation_setting_id):
|
||||
annotation_setting_id = str(annotation_setting_id)
|
||||
|
||||
args = AnnotationSettingUpdatePayload.model_validate(console_ns.payload)
|
||||
|
||||
setting_args: UpdateAnnotationSettingArgs = {"score_threshold": args.score_threshold}
|
||||
result = AppAnnotationService.update_app_annotation_setting(
|
||||
str(app_id), annotation_setting_id_str, setting_args
|
||||
)
|
||||
result = AppAnnotationService.update_app_annotation_setting(str(app_id), annotation_setting_id, setting_args)
|
||||
return result, 200
|
||||
|
||||
|
||||
@ -183,9 +181,9 @@ class AnnotationReplyActionStatusApi(Resource):
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_resource_check("annotation")
|
||||
@edit_permission_required
|
||||
def get(self, app_id: UUID, job_id: UUID, action: str):
|
||||
job_id_str = str(job_id)
|
||||
app_annotation_job_key = f"{action}_app_annotation_job_{job_id_str}"
|
||||
def get(self, app_id: UUID, job_id, action):
|
||||
job_id = str(job_id)
|
||||
app_annotation_job_key = f"{action}_app_annotation_job_{str(job_id)}"
|
||||
cache_result = redis_client.get(app_annotation_job_key)
|
||||
if cache_result is None:
|
||||
raise ValueError("The job does not exist.")
|
||||
@ -193,10 +191,10 @@ class AnnotationReplyActionStatusApi(Resource):
|
||||
job_status = cache_result.decode()
|
||||
error_msg = ""
|
||||
if job_status == "error":
|
||||
app_annotation_error_key = f"{action}_app_annotation_error_{job_id_str}"
|
||||
app_annotation_error_key = f"{action}_app_annotation_error_{str(job_id)}"
|
||||
error_msg = redis_client.get(app_annotation_error_key).decode()
|
||||
|
||||
return {"job_id": job_id_str, "job_status": job_status, "error_msg": error_msg}, 200
|
||||
return {"job_id": job_id, "job_status": job_status, "error_msg": error_msg}, 200
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/annotations")
|
||||
@ -271,12 +269,12 @@ class AnnotationApi(Resource):
|
||||
"message": "annotation_ids are required if the parameter is provided.",
|
||||
}, 400
|
||||
|
||||
AppAnnotationService.delete_app_annotations_in_batch(str(app_id), annotation_ids)
|
||||
return "", 204
|
||||
result = AppAnnotationService.delete_app_annotations_in_batch(str(app_id), annotation_ids)
|
||||
return result, 204
|
||||
# If no annotation_ids are provided, handle clearing all annotations
|
||||
else:
|
||||
AppAnnotationService.clear_all_annotations(str(app_id))
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/annotations/export")
|
||||
@ -337,7 +335,7 @@ class AnnotationUpdateDeleteApi(Resource):
|
||||
@edit_permission_required
|
||||
def delete(self, app_id: UUID, annotation_id: UUID):
|
||||
AppAnnotationService.delete_app_annotation(str(app_id), str(annotation_id))
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/annotations/batch-import")
|
||||
|
||||
@ -12,9 +12,8 @@ from sqlalchemy.orm import Session
|
||||
from werkzeug.datastructures import MultiDict
|
||||
from werkzeug.exceptions import BadRequest
|
||||
|
||||
from controllers.common.fields import RedirectUrlResponse, SimpleResultResponse
|
||||
from controllers.common.helpers import FileInfo
|
||||
from controllers.common.schema import register_enum_models, register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_enum_models, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
from controllers.console.workspace.models import LoadBalancingPayload
|
||||
@ -31,7 +30,6 @@ from core.ops.ops_trace_manager import OpsTraceManager
|
||||
from core.rag.entities import PreProcessingRule, Rule, Segmentation
|
||||
from core.rag.retrieval.retrieval_methods import RetrievalMethod
|
||||
from core.trigger.constants import TRIGGER_NODE_TYPES
|
||||
from configs import dify_config
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from graphon.enums import WorkflowExecutionStatus
|
||||
@ -42,7 +40,6 @@ from models.model import IconType
|
||||
from services.app_dsl_service import AppDslService
|
||||
from services.app_service import AppListParams, AppService, CreateAppParams
|
||||
from services.enterprise.enterprise_service import EnterpriseService
|
||||
from services.enterprise import rbac_service as enterprise_rbac_service
|
||||
from services.entities.dsl_entities import ImportMode, ImportStatus
|
||||
from services.entities.knowledge_entities.knowledge_entities import (
|
||||
DataSource,
|
||||
@ -348,7 +345,6 @@ class AppPartial(ResponseModel):
|
||||
create_user_name: str | None = None
|
||||
author_name: str | None = None
|
||||
has_draft_trigger: bool | None = None
|
||||
permission_keys: list[str] = Field(default_factory=list)
|
||||
|
||||
@computed_field(return_type=str | None) # type: ignore
|
||||
@property
|
||||
@ -384,7 +380,6 @@ class AppDetail(ResponseModel):
|
||||
updated_at: int | None = None
|
||||
access_mode: str | None = None
|
||||
tags: list[Tag] = Field(default_factory=list)
|
||||
permission_keys: list[str] = Field(default_factory=list)
|
||||
|
||||
@field_validator("created_at", "updated_at", mode="before")
|
||||
@classmethod
|
||||
@ -417,24 +412,7 @@ class AppExportResponse(ResponseModel):
|
||||
data: str
|
||||
|
||||
|
||||
def _collect_app_access_permission_keys(access_matrix: enterprise_rbac_service.AppAccessMatrix) -> list[str]:
|
||||
permission_keys: list[str] = []
|
||||
seen_permission_keys: set[str] = set()
|
||||
|
||||
for item in access_matrix.items:
|
||||
if not item.policy:
|
||||
continue
|
||||
for permission_key in item.policy.permission_keys:
|
||||
if permission_key in seen_permission_keys:
|
||||
continue
|
||||
seen_permission_keys.add(permission_key)
|
||||
permission_keys.append(permission_key)
|
||||
|
||||
return permission_keys
|
||||
|
||||
|
||||
register_enum_models(console_ns, RetrievalMethod, WorkflowExecutionStatus, DatasetPermissionEnum)
|
||||
register_response_schema_models(console_ns, RedirectUrlResponse, SimpleResultResponse)
|
||||
|
||||
register_schema_models(
|
||||
console_ns,
|
||||
@ -519,20 +497,6 @@ class AppListApi(Resource):
|
||||
if str(app.id) in res:
|
||||
app.access_mode = res[str(app.id)].access_mode
|
||||
|
||||
if app_pagination.items:
|
||||
if dify_config.RBAC_ENABLED:
|
||||
app_ids = [str(app.id) for app in app_pagination.items]
|
||||
permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
|
||||
str(current_tenant_id),
|
||||
current_user.id,
|
||||
app_ids,
|
||||
)
|
||||
for app in app_pagination.items:
|
||||
app.permission_keys = permission_keys_map.get(str(app.id), [])
|
||||
else:
|
||||
for app in app_pagination.items:
|
||||
app.permission_keys = []
|
||||
|
||||
workflow_capable_app_ids = [
|
||||
str(app.id) for app in app_pagination.items if app.mode in {"workflow", "advanced-chat"}
|
||||
]
|
||||
@ -610,7 +574,6 @@ class AppApi(Resource):
|
||||
@get_app_model(mode=None)
|
||||
def get(self, app_model):
|
||||
"""Get app detail"""
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
app_service = AppService()
|
||||
|
||||
app_model = app_service.get_app(app_model)
|
||||
@ -619,16 +582,6 @@ class AppApi(Resource):
|
||||
app_setting = EnterpriseService.WebAppAuth.get_app_access_mode_by_id(app_id=str(app_model.id))
|
||||
app_model.access_mode = app_setting.access_mode
|
||||
|
||||
if dify_config.RBAC_ENABLED:
|
||||
app_access_matrix = enterprise_rbac_service.RBACService.AppAccess.matrix(
|
||||
str(current_tenant_id),
|
||||
current_user.id,
|
||||
str(app_model.id),
|
||||
)
|
||||
app_model.permission_keys = _collect_app_access_permission_keys(app_access_matrix)
|
||||
else:
|
||||
app_model.permission_keys = []
|
||||
|
||||
response_model = AppDetailWithSite.model_validate(app_model, from_attributes=True)
|
||||
return response_model.model_dump(mode="json")
|
||||
|
||||
@ -678,7 +631,7 @@ class AppApi(Resource):
|
||||
app_service = AppService()
|
||||
app_service.delete_app(app_model)
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/copy")
|
||||
@ -771,7 +724,6 @@ class AppExportApi(Resource):
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/publish-to-creators-platform")
|
||||
class AppPublishToCreatorsPlatformApi(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[RedirectUrlResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -909,11 +861,7 @@ class AppTraceApi(Resource):
|
||||
@console_ns.doc(description="Update app tracing configuration")
|
||||
@console_ns.doc(params={"app_id": "Application ID"})
|
||||
@console_ns.expect(console_ns.models[AppTracePayload.__name__])
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Trace configuration updated successfully",
|
||||
console_ns.models[SimpleResultResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Trace configuration updated successfully")
|
||||
@console_ns.response(403, "Insufficient permissions")
|
||||
@setup_required
|
||||
@login_required
|
||||
|
||||
@ -97,7 +97,7 @@ class AppImportConfirmApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
def post(self, import_id: str):
|
||||
def post(self, import_id):
|
||||
# Check user role first
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
|
||||
@ -7,8 +7,7 @@ from pydantic import BaseModel, Field, field_validator
|
||||
from werkzeug.exceptions import InternalServerError, NotFound
|
||||
|
||||
import services
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.error import (
|
||||
AppUnavailableError,
|
||||
@ -67,7 +66,6 @@ class ChatMessagePayload(BaseMessagePayload):
|
||||
|
||||
|
||||
register_schema_models(console_ns, CompletionMessagePayload, ChatMessagePayload)
|
||||
register_response_schema_models(console_ns, SimpleResultResponse)
|
||||
|
||||
|
||||
# define completion message api for user
|
||||
@ -126,12 +124,12 @@ class CompletionMessageStopApi(Resource):
|
||||
@console_ns.doc("stop_completion_message")
|
||||
@console_ns.doc(description="Stop a running completion message generation")
|
||||
@console_ns.doc(params={"app_id": "Application ID", "task_id": "Task ID to stop"})
|
||||
@console_ns.response(200, "Task stopped successfully", console_ns.models[SimpleResultResponse.__name__])
|
||||
@console_ns.response(200, "Task stopped successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=AppMode.COMPLETION)
|
||||
def post(self, app_model, task_id: str):
|
||||
def post(self, app_model, task_id):
|
||||
if not isinstance(current_user, Account):
|
||||
raise ValueError("current_user must be an Account instance")
|
||||
|
||||
@ -207,12 +205,12 @@ class ChatMessageStopApi(Resource):
|
||||
@console_ns.doc("stop_chat_message")
|
||||
@console_ns.doc(description="Stop a running chat message generation")
|
||||
@console_ns.doc(params={"app_id": "Application ID", "task_id": "Task ID to stop"})
|
||||
@console_ns.response(200, "Task stopped successfully", console_ns.models[SimpleResultResponse.__name__])
|
||||
@console_ns.response(200, "Task stopped successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT])
|
||||
def post(self, app_model, task_id: str):
|
||||
def post(self, app_model, task_id):
|
||||
if not isinstance(current_user, Account):
|
||||
raise ValueError("current_user must be an Account instance")
|
||||
|
||||
|
||||
@ -1,5 +1,4 @@
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
import sqlalchemy as sa
|
||||
from flask import abort, request
|
||||
@ -30,6 +29,9 @@ from fields.conversation_fields import (
|
||||
from fields.conversation_fields import (
|
||||
ConversationWithSummaryPagination as ConversationWithSummaryPaginationResponse,
|
||||
)
|
||||
from fields.conversation_fields import (
|
||||
ResultResponse,
|
||||
)
|
||||
from libs.datetime_utils import naive_utc_now, parse_time_range
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from models import Conversation, EndUser, Message, MessageAnnotation
|
||||
@ -75,6 +77,7 @@ register_schema_models(
|
||||
ConversationMessageDetailResponse,
|
||||
ConversationWithSummaryPaginationResponse,
|
||||
ConversationDetailResponse,
|
||||
ResultResponse,
|
||||
CompletionConversationQuery,
|
||||
ChatConversationQuery,
|
||||
)
|
||||
@ -165,10 +168,10 @@ class CompletionConversationDetailApi(Resource):
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=AppMode.COMPLETION)
|
||||
@edit_permission_required
|
||||
def get(self, app_model, conversation_id: UUID):
|
||||
conversation_id_str = str(conversation_id)
|
||||
def get(self, app_model, conversation_id):
|
||||
conversation_id = str(conversation_id)
|
||||
return ConversationMessageDetailResponse.model_validate(
|
||||
_get_conversation(app_model, conversation_id_str), from_attributes=True
|
||||
_get_conversation(app_model, conversation_id), from_attributes=True
|
||||
).model_dump(mode="json")
|
||||
|
||||
@console_ns.doc("delete_completion_conversation")
|
||||
@ -182,16 +185,16 @@ class CompletionConversationDetailApi(Resource):
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=AppMode.COMPLETION)
|
||||
@edit_permission_required
|
||||
def delete(self, app_model, conversation_id: UUID):
|
||||
def delete(self, app_model, conversation_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
conversation_id_str = str(conversation_id)
|
||||
conversation_id = str(conversation_id)
|
||||
|
||||
try:
|
||||
ConversationService.delete(app_model, conversation_id_str, current_user)
|
||||
ConversationService.delete(app_model, conversation_id, current_user)
|
||||
except ConversationNotExistsError:
|
||||
raise NotFound("Conversation Not Exists.")
|
||||
|
||||
return "", 204
|
||||
return ResultResponse(result="success").model_dump(mode="json"), 204
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/chat-conversations")
|
||||
@ -318,10 +321,10 @@ class ChatConversationDetailApi(Resource):
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT])
|
||||
@edit_permission_required
|
||||
def get(self, app_model, conversation_id: UUID):
|
||||
conversation_id_str = str(conversation_id)
|
||||
def get(self, app_model, conversation_id):
|
||||
conversation_id = str(conversation_id)
|
||||
return ConversationDetailResponse.model_validate(
|
||||
_get_conversation(app_model, conversation_id_str), from_attributes=True
|
||||
_get_conversation(app_model, conversation_id), from_attributes=True
|
||||
).model_dump(mode="json")
|
||||
|
||||
@console_ns.doc("delete_chat_conversation")
|
||||
@ -335,16 +338,16 @@ class ChatConversationDetailApi(Resource):
|
||||
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT])
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
def delete(self, app_model, conversation_id: UUID):
|
||||
def delete(self, app_model, conversation_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
conversation_id_str = str(conversation_id)
|
||||
conversation_id = str(conversation_id)
|
||||
|
||||
try:
|
||||
ConversationService.delete(app_model, conversation_id_str, current_user)
|
||||
ConversationService.delete(app_model, conversation_id, current_user)
|
||||
except ConversationNotExistsError:
|
||||
raise NotFound("Conversation Not Exists.")
|
||||
|
||||
return "", 204
|
||||
return ResultResponse(result="success").model_dump(mode="json"), 204
|
||||
|
||||
|
||||
def _get_conversation(app_model, conversation_id):
|
||||
|
||||
@ -1,7 +1,6 @@
|
||||
import json
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
@ -163,7 +162,7 @@ class AppMCPServerRefreshController(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
def get(self, server_id: UUID):
|
||||
def get(self, server_id):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
server = db.session.scalar(
|
||||
select(AppMCPServer)
|
||||
|
||||
@ -1,7 +1,6 @@
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
@ -10,8 +9,7 @@ from sqlalchemy import exists, func, select
|
||||
from werkzeug.exceptions import InternalServerError, NotFound
|
||||
|
||||
from controllers.common.controller_schemas import MessageFeedbackPayload as _MessageFeedbackPayloadBase
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.error import (
|
||||
CompletionRequestError,
|
||||
@ -164,7 +162,6 @@ register_schema_models(
|
||||
MessageDetailResponse,
|
||||
MessageInfiniteScrollPaginationResponse,
|
||||
)
|
||||
register_response_schema_models(console_ns, SimpleResultResponse)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/chat-messages")
|
||||
@ -250,7 +247,7 @@ class MessageFeedbackApi(Resource):
|
||||
@console_ns.doc(description="Create or update message feedback (like/dislike)")
|
||||
@console_ns.doc(params={"app_id": "Application ID"})
|
||||
@console_ns.expect(console_ns.models[MessageFeedbackPayload.__name__])
|
||||
@console_ns.response(200, "Feedback updated successfully", console_ns.models[SimpleResultResponse.__name__])
|
||||
@console_ns.response(200, "Feedback updated successfully")
|
||||
@console_ns.response(404, "Message not found")
|
||||
@console_ns.response(403, "Insufficient permissions")
|
||||
@get_app_model
|
||||
@ -337,13 +334,13 @@ class MessageSuggestedQuestionApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT])
|
||||
def get(self, app_model, message_id: UUID):
|
||||
def get(self, app_model, message_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
message_id_str = str(message_id)
|
||||
message_id = str(message_id)
|
||||
|
||||
try:
|
||||
questions = MessageService.get_suggested_questions_after_answer(
|
||||
app_model=app_model, message_id=message_id_str, user=current_user, invoke_from=InvokeFrom.DEBUGGER
|
||||
app_model=app_model, message_id=message_id, user=current_user, invoke_from=InvokeFrom.DEBUGGER
|
||||
)
|
||||
except MessageNotExistsError:
|
||||
raise NotFound("Message not found")
|
||||
@ -418,10 +415,10 @@ class MessageApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, app_model, message_id: str):
|
||||
message_id_str = str(message_id)
|
||||
message_id = str(message_id)
|
||||
|
||||
message = db.session.scalar(
|
||||
select(Message).where(Message.id == message_id_str, Message.app_id == app_model.id).limit(1)
|
||||
select(Message).where(Message.id == message_id, Message.app_id == app_model.id).limit(1)
|
||||
)
|
||||
|
||||
if not message:
|
||||
|
||||
@ -128,6 +128,6 @@ class TraceAppConfigApi(Resource):
|
||||
result = OpsService.delete_tracing_app_config(app_id=app_model.id, tracing_provider=args.tracing_provider)
|
||||
if not result:
|
||||
raise TracingConfigNotExist()
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
except Exception as e:
|
||||
raise BadRequest(str(e))
|
||||
|
||||
@ -12,7 +12,6 @@ from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotF
|
||||
|
||||
import services
|
||||
from controllers.common.controller_schemas import DefaultBlockConfigQuery, WorkflowListQuery, WorkflowUpdatePayload
|
||||
from controllers.common.fields import NewAppResponse, SimpleResultResponse
|
||||
from controllers.common.schema import (
|
||||
register_response_schema_model,
|
||||
register_response_schema_models,
|
||||
@ -291,8 +290,6 @@ register_response_schema_models(
|
||||
WorkflowOnlineUser,
|
||||
WorkflowOnlineUsersByApp,
|
||||
WorkflowOnlineUsersResponse,
|
||||
NewAppResponse,
|
||||
SimpleResultResponse,
|
||||
)
|
||||
|
||||
|
||||
@ -872,7 +869,7 @@ class WorkflowTaskStopApi(Resource):
|
||||
@console_ns.doc("stop_workflow_task")
|
||||
@console_ns.doc(description="Stop running workflow task")
|
||||
@console_ns.doc(params={"app_id": "Application ID", "task_id": "Task ID"})
|
||||
@console_ns.response(200, "Task stopped successfully", console_ns.models[SimpleResultResponse.__name__])
|
||||
@console_ns.response(200, "Task stopped successfully")
|
||||
@console_ns.response(404, "Task not found")
|
||||
@console_ns.response(403, "Permission denied")
|
||||
@setup_required
|
||||
@ -1072,11 +1069,7 @@ class ConvertToWorkflowApi(Resource):
|
||||
@console_ns.doc("convert_to_workflow")
|
||||
@console_ns.doc(description="Convert application to workflow mode")
|
||||
@console_ns.doc(params={"app_id": "Application ID"})
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Application converted to workflow successfully",
|
||||
console_ns.models[NewAppResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Application converted to workflow successfully")
|
||||
@console_ns.response(400, "Application cannot be converted")
|
||||
@console_ns.response(403, "Permission denied")
|
||||
@setup_required
|
||||
@ -1113,11 +1106,7 @@ class WorkflowFeaturesApi(Resource):
|
||||
@console_ns.doc("update_workflow_features")
|
||||
@console_ns.doc(description="Update draft workflow features")
|
||||
@console_ns.doc(params={"app_id": "Application ID"})
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Workflow features updated successfully",
|
||||
console_ns.models[SimpleResultResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Workflow features updated successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
|
||||
@ -311,7 +311,7 @@ class WorkflowCommentDetailApi(Resource):
|
||||
user_id=current_user.id,
|
||||
)
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/workflow/comments/<string:comment_id>/resolve")
|
||||
@ -431,7 +431,7 @@ class WorkflowCommentReplyDetailApi(Resource):
|
||||
user_id=current_user.id,
|
||||
)
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/workflow/comments/mention-users")
|
||||
|
||||
@ -1,6 +1,5 @@
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import Literal, cast
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
@ -368,14 +367,14 @@ class WorkflowRunDetailApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
|
||||
def get(self, app_model: App, run_id: UUID):
|
||||
def get(self, app_model: App, run_id):
|
||||
"""
|
||||
Get workflow run detail
|
||||
"""
|
||||
run_id_str = str(run_id)
|
||||
run_id = str(run_id)
|
||||
|
||||
workflow_run_service = WorkflowRunService()
|
||||
workflow_run = workflow_run_service.get_workflow_run(app_model=app_model, run_id=run_id_str)
|
||||
workflow_run = workflow_run_service.get_workflow_run(app_model=app_model, run_id=run_id)
|
||||
if workflow_run is None:
|
||||
raise NotFoundError("Workflow run not found")
|
||||
|
||||
@ -397,17 +396,17 @@ class WorkflowRunNodeExecutionListApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
|
||||
def get(self, app_model: App, run_id: UUID):
|
||||
def get(self, app_model: App, run_id):
|
||||
"""
|
||||
Get workflow run node execution list
|
||||
"""
|
||||
run_id_str = str(run_id)
|
||||
run_id = str(run_id)
|
||||
|
||||
workflow_run_service = WorkflowRunService()
|
||||
user = cast("Account | EndUser", current_user)
|
||||
node_executions = workflow_run_service.get_workflow_run_node_executions(
|
||||
app_model=app_model,
|
||||
run_id=run_id_str,
|
||||
run_id=run_id,
|
||||
user=user,
|
||||
)
|
||||
|
||||
|
||||
@ -1,3 +1,5 @@
|
||||
from typing import Any
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
@ -38,29 +40,16 @@ class ActivatePayload(BaseModel):
|
||||
return timezone(value)
|
||||
|
||||
|
||||
class ActivationCheckResponse(BaseModel):
|
||||
is_valid: bool = Field(description="Whether token is valid")
|
||||
data: dict[str, Any] | None = Field(default=None, description="Activation data if valid")
|
||||
|
||||
|
||||
class ActivationResponse(BaseModel):
|
||||
result: str = Field(description="Operation result")
|
||||
|
||||
|
||||
class ActivationCheckData(BaseModel):
|
||||
workspace_name: str | None
|
||||
workspace_id: str | None
|
||||
email: str | None
|
||||
|
||||
|
||||
class ActivationCheckResponse(BaseModel):
|
||||
is_valid: bool = Field(description="Whether token is valid")
|
||||
data: ActivationCheckData | None = Field(default=None, description="Activation data if valid")
|
||||
|
||||
|
||||
register_schema_models(
|
||||
console_ns,
|
||||
ActivateCheckQuery,
|
||||
ActivatePayload,
|
||||
ActivationCheckData,
|
||||
ActivationCheckResponse,
|
||||
ActivationResponse,
|
||||
)
|
||||
register_schema_models(console_ns, ActivateCheckQuery, ActivatePayload, ActivationCheckResponse, ActivationResponse)
|
||||
|
||||
|
||||
@console_ns.route("/activate/check")
|
||||
|
||||
@ -1,10 +1,7 @@
|
||||
from uuid import UUID
|
||||
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from fields.base import ResponseModel
|
||||
from controllers.common.schema import register_schema_models
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from services.auth.api_key_auth_service import ApiKeyAuthService
|
||||
|
||||
@ -19,26 +16,11 @@ class ApiKeyAuthBindingPayload(BaseModel):
|
||||
credentials: dict = Field(...)
|
||||
|
||||
|
||||
class ApiKeyAuthDataSourceItem(ResponseModel):
|
||||
id: str
|
||||
category: str
|
||||
provider: str
|
||||
disabled: bool
|
||||
created_at: int
|
||||
updated_at: int
|
||||
|
||||
|
||||
class ApiKeyAuthDataSourceListResponse(ResponseModel):
|
||||
sources: list[ApiKeyAuthDataSourceItem]
|
||||
|
||||
|
||||
register_schema_models(console_ns, ApiKeyAuthBindingPayload)
|
||||
register_response_schema_models(console_ns, ApiKeyAuthDataSourceItem, ApiKeyAuthDataSourceListResponse)
|
||||
|
||||
|
||||
@console_ns.route("/api-key-auth/data-source")
|
||||
class ApiKeyAuthDataSource(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[ApiKeyAuthDataSourceListResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -88,11 +70,10 @@ class ApiKeyAuthDataSourceBindingDelete(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@is_admin_or_owner_required
|
||||
@console_ns.response(204, "Binding deleted successfully")
|
||||
def delete(self, binding_id: UUID):
|
||||
def delete(self, binding_id):
|
||||
# The role of the current user in the table must be admin or owner
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
ApiKeyAuthService.delete_provider_auth(current_tenant_id, str(binding_id))
|
||||
ApiKeyAuthService.delete_provider_auth(current_tenant_id, binding_id)
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
@ -1,5 +1,4 @@
|
||||
import logging
|
||||
from uuid import UUID
|
||||
|
||||
import httpx
|
||||
from flask import current_app, redirect, request
|
||||
@ -159,15 +158,16 @@ class OAuthDataSourceSync(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, provider: str, binding_id: UUID):
|
||||
binding_id_str = str(binding_id)
|
||||
def get(self, provider, binding_id):
|
||||
provider = str(provider)
|
||||
binding_id = str(binding_id)
|
||||
OAUTH_DATASOURCE_PROVIDERS = get_oauth_providers()
|
||||
with current_app.app_context():
|
||||
oauth_provider = OAUTH_DATASOURCE_PROVIDERS.get(provider)
|
||||
if not oauth_provider:
|
||||
return {"error": "Invalid provider"}, 400
|
||||
try:
|
||||
oauth_provider.sync_data_source(binding_id_str)
|
||||
oauth_provider.sync_data_source(binding_id)
|
||||
except httpx.HTTPStatusError as e:
|
||||
logger.exception(
|
||||
"An error occurred during the OAuthCallback process with %s: %s", provider, e.response.text
|
||||
|
||||
@ -4,8 +4,7 @@ from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from configs import dify_config
|
||||
from constants.languages import get_valid_language, languages
|
||||
from controllers.common.fields import SimpleResultDataResponse, VerificationTokenResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.auth.error import (
|
||||
EmailAlreadyInUseError,
|
||||
@ -59,7 +58,6 @@ class EmailRegisterResetPayload(BaseModel):
|
||||
|
||||
|
||||
register_schema_models(console_ns, EmailRegisterSendPayload, EmailRegisterValidityPayload, EmailRegisterResetPayload)
|
||||
register_response_schema_models(console_ns, SimpleResultDataResponse, VerificationTokenResponse)
|
||||
|
||||
|
||||
@console_ns.route("/email-register/send-email")
|
||||
@ -67,7 +65,6 @@ class EmailRegisterSendEmailApi(Resource):
|
||||
@setup_required
|
||||
@email_password_login_enabled
|
||||
@email_register_enabled
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultDataResponse.__name__])
|
||||
def post(self):
|
||||
args = EmailRegisterSendPayload.model_validate(console_ns.payload)
|
||||
normalized_email = args.email.lower()
|
||||
@ -92,7 +89,6 @@ class EmailRegisterCheckApi(Resource):
|
||||
@setup_required
|
||||
@email_password_login_enabled
|
||||
@email_register_enabled
|
||||
@console_ns.response(200, "Success", console_ns.models[VerificationTokenResponse.__name__])
|
||||
def post(self):
|
||||
args = EmailRegisterValidityPayload.model_validate(console_ns.payload)
|
||||
|
||||
|
||||
@ -9,8 +9,7 @@ from werkzeug.exceptions import Unauthorized
|
||||
import services
|
||||
from configs import dify_config
|
||||
from constants.languages import get_valid_language
|
||||
from controllers.common.fields import SimpleResultDataResponse, SimpleResultOptionalDataResponse, SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.auth.error import (
|
||||
AuthenticationFailedError,
|
||||
@ -82,12 +81,6 @@ class EmailCodeLoginPayload(BaseModel):
|
||||
|
||||
|
||||
register_schema_models(console_ns, LoginPayload, EmailPayload, EmailCodeLoginPayload)
|
||||
register_response_schema_models(
|
||||
console_ns,
|
||||
SimpleResultDataResponse,
|
||||
SimpleResultOptionalDataResponse,
|
||||
SimpleResultResponse,
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/login")
|
||||
@ -97,7 +90,6 @@ class LoginApi(Resource):
|
||||
@setup_required
|
||||
@email_password_login_enabled
|
||||
@console_ns.expect(console_ns.models[LoginPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultOptionalDataResponse.__name__])
|
||||
@decrypt_password_field
|
||||
def post(self):
|
||||
"""Authenticate user and login."""
|
||||
@ -171,7 +163,6 @@ class LoginApi(Resource):
|
||||
@console_ns.route("/logout")
|
||||
class LogoutApi(Resource):
|
||||
@setup_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def post(self):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
account = current_user
|
||||
@ -195,7 +186,6 @@ class ResetPasswordSendEmailApi(Resource):
|
||||
@setup_required
|
||||
@email_password_login_enabled
|
||||
@console_ns.expect(console_ns.models[EmailPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultDataResponse.__name__])
|
||||
def post(self):
|
||||
args = EmailPayload.model_validate(console_ns.payload)
|
||||
normalized_email = args.email.lower()
|
||||
@ -223,7 +213,6 @@ class ResetPasswordSendEmailApi(Resource):
|
||||
class EmailCodeLoginSendEmailApi(Resource):
|
||||
@setup_required
|
||||
@console_ns.expect(console_ns.models[EmailPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultDataResponse.__name__])
|
||||
def post(self):
|
||||
args = EmailPayload.model_validate(console_ns.payload)
|
||||
normalized_email = args.email.lower()
|
||||
@ -256,7 +245,6 @@ class EmailCodeLoginSendEmailApi(Resource):
|
||||
class EmailCodeLoginApi(Resource):
|
||||
@setup_required
|
||||
@console_ns.expect(console_ns.models[EmailCodeLoginPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@decrypt_code_field
|
||||
def post(self):
|
||||
args = EmailCodeLoginPayload.model_validate(console_ns.payload)
|
||||
@ -333,7 +321,6 @@ class EmailCodeLoginApi(Resource):
|
||||
|
||||
@console_ns.route("/refresh-token")
|
||||
class RefreshTokenApi(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def post(self):
|
||||
# Get refresh token from cookie instead of request body
|
||||
refresh_token = extract_refresh_token(request)
|
||||
|
||||
@ -1,7 +1,6 @@
|
||||
import json
|
||||
from collections.abc import Generator
|
||||
from typing import Any, Literal, cast
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource, fields, marshal_with
|
||||
@ -10,8 +9,7 @@ from sqlalchemy import select
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from controllers.common.fields import SimpleResultResponse, TextContentResponse
|
||||
from controllers.common.schema import get_or_create_model, register_response_schema_models, register_schema_model
|
||||
from controllers.common.schema import get_or_create_model, register_schema_model
|
||||
from core.datasource.entities.datasource_entities import DatasourceProviderType, OnlineDocumentPagesMessage
|
||||
from core.datasource.online_document.online_document_plugin import OnlineDocumentDatasourcePlugin
|
||||
from core.indexing_runner import IndexingRunner
|
||||
@ -56,7 +54,6 @@ class DataSourceNotionPreviewQuery(BaseModel):
|
||||
|
||||
|
||||
register_schema_model(console_ns, NotionEstimatePayload)
|
||||
register_response_schema_models(console_ns, SimpleResultResponse, TextContentResponse)
|
||||
|
||||
|
||||
integrate_icon_model = get_or_create_model("DataSourceIntegrateIcon", integrate_icon_fields)
|
||||
@ -160,7 +157,6 @@ class DataSourceApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def patch(self, binding_id, action: Literal["enable", "disable"]):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
binding_id = str(binding_id)
|
||||
@ -293,8 +289,7 @@ class DataSourceNotionApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[TextContentResponse.__name__])
|
||||
def get(self, page_id: UUID, page_type: str):
|
||||
def get(self, page_id, page_type):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
query = DataSourceNotionPreviewQuery.model_validate(request.args.to_dict())
|
||||
@ -307,11 +302,11 @@ class DataSourceNotionApi(Resource):
|
||||
plugin_id="langgenius/notion_datasource",
|
||||
)
|
||||
|
||||
page_id_str = str(page_id)
|
||||
page_id = str(page_id)
|
||||
|
||||
extractor = NotionExtractor(
|
||||
notion_workspace_id="",
|
||||
notion_obj_id=page_id_str,
|
||||
notion_obj_id=page_id,
|
||||
notion_page_type=page_type,
|
||||
notion_access_token=credential.get("integration_secret"),
|
||||
tenant_id=current_tenant_id,
|
||||
@ -367,8 +362,7 @@ class DataSourceNotionDatasetSyncApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def get(self, dataset_id: UUID):
|
||||
def get(self, dataset_id):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
if dataset is None:
|
||||
@ -385,8 +379,7 @@ class DataSourceNotionDocumentSyncApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def get(self, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, dataset_id, document_id):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
|
||||
@ -1,17 +1,14 @@
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
from typing import Any, cast
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator, model_validator
|
||||
from flask_restx import Resource, fields, marshal, marshal_with
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from sqlalchemy import func, select
|
||||
from werkzeug.exceptions import Forbidden, NotFound
|
||||
|
||||
import services
|
||||
from configs import dify_config
|
||||
from controllers.common.fields import ApiBaseUrlResponse, SimpleResultResponse, UsageCheckResponse
|
||||
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import get_or_create_model, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.apikey import ApiKeyItem, ApiKeyList
|
||||
from controllers.console.app.error import ProviderNotInitializeError
|
||||
@ -32,10 +29,26 @@ from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo
|
||||
from core.rag.index_processor.constant.index_type import IndexTechniqueType
|
||||
from core.rag.retrieval.retrieval_methods import RetrievalMethod
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from fields.dataset_fields import DatasetDetailResponse
|
||||
from fields.app_fields import app_detail_kernel_fields, related_app_list
|
||||
from fields.dataset_fields import (
|
||||
content_fields,
|
||||
dataset_detail_fields,
|
||||
dataset_fields,
|
||||
dataset_query_detail_fields,
|
||||
dataset_retrieval_model_fields,
|
||||
doc_metadata_fields,
|
||||
external_knowledge_info_fields,
|
||||
external_retrieval_model_fields,
|
||||
file_info_fields,
|
||||
icon_info_fields,
|
||||
keyword_setting_fields,
|
||||
reranking_model_fields,
|
||||
tag_fields,
|
||||
vector_setting_fields,
|
||||
weighted_score_fields,
|
||||
)
|
||||
from fields.document_fields import document_status_fields
|
||||
from graphon.model_runtime.entities.model_entities import ModelType
|
||||
from libs.helper import build_icon_url, dump_response, to_timestamp
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from libs.url_utils import normalize_api_base_url
|
||||
from models import ApiToken, Dataset, Document, DocumentSegment, UploadFile
|
||||
@ -44,9 +57,58 @@ from models.enums import ApiTokenType, SegmentStatus
|
||||
from models.provider_ids import ModelProviderID
|
||||
from services.api_token_service import ApiTokenCache
|
||||
from services.dataset_service import DatasetPermissionService, DatasetService, DocumentService
|
||||
from services.enterprise import rbac_service as enterprise_rbac_service
|
||||
|
||||
register_response_schema_models(console_ns, ApiBaseUrlResponse, SimpleResultResponse, UsageCheckResponse)
|
||||
# Register models for flask_restx to avoid dict type issues in Swagger
|
||||
dataset_base_model = get_or_create_model("DatasetBase", dataset_fields)
|
||||
|
||||
tag_model = get_or_create_model("Tag", tag_fields)
|
||||
|
||||
keyword_setting_model = get_or_create_model("DatasetKeywordSetting", keyword_setting_fields)
|
||||
vector_setting_model = get_or_create_model("DatasetVectorSetting", vector_setting_fields)
|
||||
|
||||
weighted_score_fields_copy = weighted_score_fields.copy()
|
||||
weighted_score_fields_copy["keyword_setting"] = fields.Nested(keyword_setting_model)
|
||||
weighted_score_fields_copy["vector_setting"] = fields.Nested(vector_setting_model)
|
||||
weighted_score_model = get_or_create_model("DatasetWeightedScore", weighted_score_fields_copy)
|
||||
|
||||
reranking_model = get_or_create_model("DatasetRerankingModel", reranking_model_fields)
|
||||
|
||||
dataset_retrieval_model_fields_copy = dataset_retrieval_model_fields.copy()
|
||||
dataset_retrieval_model_fields_copy["reranking_model"] = fields.Nested(reranking_model)
|
||||
dataset_retrieval_model_fields_copy["weights"] = fields.Nested(weighted_score_model, allow_null=True)
|
||||
dataset_retrieval_model = get_or_create_model("DatasetRetrievalModel", dataset_retrieval_model_fields_copy)
|
||||
|
||||
external_knowledge_info_model = get_or_create_model("ExternalKnowledgeInfo", external_knowledge_info_fields)
|
||||
|
||||
external_retrieval_model = get_or_create_model("ExternalRetrievalModel", external_retrieval_model_fields)
|
||||
|
||||
doc_metadata_model = get_or_create_model("DatasetDocMetadata", doc_metadata_fields)
|
||||
|
||||
icon_info_model = get_or_create_model("DatasetIconInfo", icon_info_fields)
|
||||
|
||||
dataset_detail_fields_copy = dataset_detail_fields.copy()
|
||||
dataset_detail_fields_copy["retrieval_model_dict"] = fields.Nested(dataset_retrieval_model)
|
||||
dataset_detail_fields_copy["tags"] = fields.List(fields.Nested(tag_model))
|
||||
dataset_detail_fields_copy["external_knowledge_info"] = fields.Nested(external_knowledge_info_model)
|
||||
dataset_detail_fields_copy["external_retrieval_model"] = fields.Nested(external_retrieval_model, allow_null=True)
|
||||
dataset_detail_fields_copy["doc_metadata"] = fields.List(fields.Nested(doc_metadata_model))
|
||||
dataset_detail_fields_copy["icon_info"] = fields.Nested(icon_info_model)
|
||||
dataset_detail_model = get_or_create_model("DatasetDetail", dataset_detail_fields_copy)
|
||||
|
||||
file_info_model = get_or_create_model("DatasetFileInfo", file_info_fields)
|
||||
|
||||
content_fields_copy = content_fields.copy()
|
||||
content_fields_copy["file_info"] = fields.Nested(file_info_model, allow_null=True)
|
||||
content_model = get_or_create_model("DatasetContent", content_fields_copy)
|
||||
|
||||
dataset_query_detail_fields_copy = dataset_query_detail_fields.copy()
|
||||
dataset_query_detail_fields_copy["queries"] = fields.Nested(content_model)
|
||||
dataset_query_detail_model = get_or_create_model("DatasetQueryDetail", dataset_query_detail_fields_copy)
|
||||
|
||||
app_detail_kernel_model = get_or_create_model("AppDetailKernel", app_detail_kernel_fields)
|
||||
related_app_list_copy = related_app_list.copy()
|
||||
related_app_list_copy["data"] = fields.List(fields.Nested(app_detail_kernel_model))
|
||||
related_app_list_model = get_or_create_model("RelatedAppList", related_app_list_copy)
|
||||
|
||||
|
||||
def _validate_indexing_technique(value: str | None) -> str | None:
|
||||
@ -65,14 +127,6 @@ def _validate_doc_form(value: str | None) -> str | None:
|
||||
return value
|
||||
|
||||
|
||||
def _ensure_permission_keys(dataset: Dataset, *, enabled: bool) -> None:
|
||||
if not enabled:
|
||||
setattr(dataset, "permission_keys", [])
|
||||
return
|
||||
if not isinstance(getattr(dataset, "permission_keys", None), list):
|
||||
setattr(dataset, "permission_keys", [])
|
||||
|
||||
|
||||
class DatasetCreatePayload(BaseModel):
|
||||
name: str = Field(..., min_length=1, max_length=40)
|
||||
description: str = Field("", max_length=400)
|
||||
@ -151,165 +205,9 @@ class ConsoleDatasetListQuery(BaseModel):
|
||||
tag_ids: list[str] = Field(default_factory=list, description="Filter by tag IDs")
|
||||
|
||||
|
||||
class DatasetListItemResponse(DatasetDetailResponse):
|
||||
partial_member_list: list[str]
|
||||
|
||||
|
||||
class DatasetListResponse(ResponseModel):
|
||||
data: list[DatasetListItemResponse]
|
||||
has_more: bool
|
||||
limit: int
|
||||
total: int
|
||||
page: int
|
||||
|
||||
|
||||
class DatasetDetailWithPartialMembersResponse(DatasetDetailResponse):
|
||||
partial_member_list: list[str] | None = None
|
||||
|
||||
|
||||
class DatasetQueryFileInfoResponse(ResponseModel):
|
||||
id: str
|
||||
name: str
|
||||
size: int
|
||||
extension: str
|
||||
mime_type: str
|
||||
source_url: str
|
||||
|
||||
|
||||
class DatasetQueryContentResponse(ResponseModel):
|
||||
content_type: str
|
||||
content: str
|
||||
file_info: DatasetQueryFileInfoResponse | None = None
|
||||
|
||||
|
||||
class DatasetQueryDetailResponse(ResponseModel):
|
||||
id: str
|
||||
queries: list[DatasetQueryContentResponse]
|
||||
source: str
|
||||
source_app_id: str | None
|
||||
created_by_role: str
|
||||
created_by: str
|
||||
created_at: int
|
||||
|
||||
@field_validator("created_at", mode="before")
|
||||
@classmethod
|
||||
def _normalize_created_at(cls, value: datetime | int | None) -> int | None:
|
||||
return to_timestamp(value)
|
||||
|
||||
|
||||
class DatasetQueryListResponse(ResponseModel):
|
||||
data: list[DatasetQueryDetailResponse]
|
||||
has_more: bool
|
||||
limit: int
|
||||
total: int
|
||||
page: int
|
||||
|
||||
|
||||
class RelatedAppResponse(ResponseModel):
|
||||
id: str
|
||||
name: str
|
||||
description: str
|
||||
mode: str = Field(validation_alias="mode_compatible_with_agent")
|
||||
icon_type: str | None
|
||||
icon: str | None
|
||||
icon_background: str | None
|
||||
icon_url: str | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _set_icon_url(self) -> "RelatedAppResponse":
|
||||
self.icon_url = self.icon_url or build_icon_url(self.icon_type, self.icon)
|
||||
return self
|
||||
|
||||
|
||||
class RelatedAppListResponse(ResponseModel):
|
||||
data: list[RelatedAppResponse]
|
||||
total: int
|
||||
|
||||
|
||||
class DocumentStatusResponse(ResponseModel):
|
||||
id: str
|
||||
indexing_status: str
|
||||
processing_started_at: int | None
|
||||
parsing_completed_at: int | None
|
||||
cleaning_completed_at: int | None
|
||||
splitting_completed_at: int | None
|
||||
completed_at: int | None
|
||||
paused_at: int | None
|
||||
error: str | None
|
||||
stopped_at: int | None
|
||||
completed_segments: int | None = None
|
||||
total_segments: int | None = None
|
||||
|
||||
@field_validator(
|
||||
"processing_started_at",
|
||||
"parsing_completed_at",
|
||||
"cleaning_completed_at",
|
||||
"splitting_completed_at",
|
||||
"completed_at",
|
||||
"paused_at",
|
||||
"stopped_at",
|
||||
mode="before",
|
||||
)
|
||||
@classmethod
|
||||
def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
|
||||
return to_timestamp(value)
|
||||
|
||||
|
||||
class DocumentStatusListResponse(ResponseModel):
|
||||
data: list[DocumentStatusResponse]
|
||||
|
||||
|
||||
class ErrorDocsResponse(DocumentStatusListResponse):
|
||||
total: int
|
||||
|
||||
|
||||
class IndexingEstimatePreviewItemResponse(ResponseModel):
|
||||
content: str
|
||||
child_chunks: list[str] | None = None
|
||||
summary: str | None = None
|
||||
|
||||
|
||||
class IndexingEstimateQaPreviewItemResponse(ResponseModel):
|
||||
question: str
|
||||
answer: str
|
||||
|
||||
|
||||
class IndexingEstimateResponse(ResponseModel):
|
||||
total_segments: int
|
||||
preview: list[IndexingEstimatePreviewItemResponse]
|
||||
qa_preview: list[IndexingEstimateQaPreviewItemResponse] | None = None
|
||||
|
||||
|
||||
class RetrievalSettingResponse(ResponseModel):
|
||||
retrieval_method: list[str]
|
||||
|
||||
|
||||
class PartialMemberListResponse(ResponseModel):
|
||||
data: list[str]
|
||||
|
||||
|
||||
class AutoDisableLogsResponse(ResponseModel):
|
||||
document_ids: list[str]
|
||||
count: int
|
||||
|
||||
|
||||
register_schema_models(
|
||||
console_ns, DatasetCreatePayload, DatasetUpdatePayload, IndexingEstimatePayload, ConsoleDatasetListQuery
|
||||
)
|
||||
register_response_schema_models(
|
||||
console_ns,
|
||||
DatasetDetailResponse,
|
||||
DatasetDetailWithPartialMembersResponse,
|
||||
DatasetListResponse,
|
||||
DatasetQueryListResponse,
|
||||
IndexingEstimateResponse,
|
||||
RelatedAppListResponse,
|
||||
DocumentStatusListResponse,
|
||||
ErrorDocsResponse,
|
||||
RetrievalSettingResponse,
|
||||
PartialMemberListResponse,
|
||||
AutoDisableLogsResponse,
|
||||
)
|
||||
|
||||
|
||||
def _get_retrieval_methods_by_vector_type(vector_type: str | None, is_mock: bool = False) -> dict[str, list[str]]:
|
||||
@ -392,8 +290,17 @@ def _get_retrieval_methods_by_vector_type(vector_type: str | None, is_mock: bool
|
||||
class DatasetListApi(Resource):
|
||||
@console_ns.doc("get_datasets")
|
||||
@console_ns.doc(description="Get list of datasets")
|
||||
@console_ns.doc(params=query_params_from_model(ConsoleDatasetListQuery))
|
||||
@console_ns.response(200, "Datasets retrieved successfully", console_ns.models[DatasetListResponse.__name__])
|
||||
@console_ns.doc(
|
||||
params={
|
||||
"page": "Page number (default: 1)",
|
||||
"limit": "Number of items per page (default: 20)",
|
||||
"ids": "Filter by dataset IDs (list)",
|
||||
"keyword": "Search keyword",
|
||||
"tag_ids": "Filter by tag IDs (list)",
|
||||
"include_all": "Include all datasets (default: false)",
|
||||
}
|
||||
)
|
||||
@console_ns.response(200, "Datasets retrieved successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -422,19 +329,6 @@ class DatasetListApi(Resource):
|
||||
query.include_all,
|
||||
)
|
||||
|
||||
for dataset in datasets:
|
||||
_ensure_permission_keys(dataset, enabled=dify_config.RBAC_ENABLED)
|
||||
|
||||
if dify_config.RBAC_ENABLED and datasets:
|
||||
dataset_ids = [str(dataset.id) for dataset in datasets]
|
||||
permission_keys_map = enterprise_rbac_service.RBACService.DatasetPermissions.batch_get(
|
||||
str(current_tenant_id),
|
||||
current_user.id,
|
||||
dataset_ids,
|
||||
)
|
||||
for dataset in datasets:
|
||||
setattr(dataset, "permission_keys", permission_keys_map.get(str(dataset.id), []))
|
||||
|
||||
# check embedding setting
|
||||
provider_manager = create_plugin_provider_manager(tenant_id=current_tenant_id)
|
||||
configurations = provider_manager.get_configurations(tenant_id=current_tenant_id)
|
||||
@ -445,7 +339,7 @@ class DatasetListApi(Resource):
|
||||
for embedding_model in embedding_models:
|
||||
model_names.append(f"{embedding_model.model}:{embedding_model.provider.provider}")
|
||||
|
||||
data = [dump_response(DatasetDetailResponse, dataset) for dataset in datasets]
|
||||
data = cast(list[dict[str, Any]], marshal(datasets, dataset_detail_fields))
|
||||
dataset_ids = [item["id"] for item in data if item.get("permission") == "partial_members"]
|
||||
partial_members_map: dict[str, list[str]] = {}
|
||||
if dataset_ids:
|
||||
@ -482,12 +376,12 @@ class DatasetListApi(Resource):
|
||||
"total": total,
|
||||
"page": query.page,
|
||||
}
|
||||
return dump_response(DatasetListResponse, response), 200
|
||||
return response, 200
|
||||
|
||||
@console_ns.doc("create_dataset")
|
||||
@console_ns.doc(description="Create a new dataset")
|
||||
@console_ns.expect(console_ns.models[DatasetCreatePayload.__name__])
|
||||
@console_ns.response(201, "Dataset created successfully", console_ns.models[DatasetDetailResponse.__name__])
|
||||
@console_ns.response(201, "Dataset created successfully")
|
||||
@console_ns.response(400, "Invalid request parameters")
|
||||
@setup_required
|
||||
@login_required
|
||||
@ -516,8 +410,7 @@ class DatasetListApi(Resource):
|
||||
except services.errors.dataset.DatasetNameDuplicateError:
|
||||
raise DatasetNameDuplicateError()
|
||||
|
||||
_ensure_permission_keys(dataset, enabled=dify_config.RBAC_ENABLED)
|
||||
return dump_response(DatasetDetailResponse, dataset), 201
|
||||
return marshal(dataset, dataset_detail_fields), 201
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>")
|
||||
@ -525,17 +418,13 @@ class DatasetApi(Resource):
|
||||
@console_ns.doc("get_dataset")
|
||||
@console_ns.doc(description="Get dataset details")
|
||||
@console_ns.doc(params={"dataset_id": "Dataset ID"})
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Dataset retrieved successfully",
|
||||
console_ns.models[DatasetDetailWithPartialMembersResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Dataset retrieved successfully", dataset_detail_model)
|
||||
@console_ns.response(404, "Dataset not found")
|
||||
@console_ns.response(403, "Permission denied")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID):
|
||||
def get(self, dataset_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
@ -545,8 +434,7 @@ class DatasetApi(Resource):
|
||||
DatasetService.check_dataset_permission(dataset, current_user)
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
_ensure_permission_keys(dataset, enabled=dify_config.RBAC_ENABLED)
|
||||
data = dump_response(DatasetDetailResponse, dataset)
|
||||
data = cast(dict[str, Any], marshal(dataset, dataset_detail_fields))
|
||||
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
|
||||
if dataset.embedding_model_provider:
|
||||
provider_id = ModelProviderID(dataset.embedding_model_provider)
|
||||
@ -579,18 +467,14 @@ class DatasetApi(Resource):
|
||||
@console_ns.doc("update_dataset")
|
||||
@console_ns.doc(description="Update dataset details")
|
||||
@console_ns.expect(console_ns.models[DatasetUpdatePayload.__name__])
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Dataset updated successfully",
|
||||
console_ns.models[DatasetDetailWithPartialMembersResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Dataset updated successfully", dataset_detail_model)
|
||||
@console_ns.response(404, "Dataset not found")
|
||||
@console_ns.response(403, "Permission denied")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
def patch(self, dataset_id: UUID):
|
||||
def patch(self, dataset_id):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
if dataset is None:
|
||||
@ -619,8 +503,7 @@ class DatasetApi(Resource):
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
_ensure_permission_keys(dataset, enabled=dify_config.RBAC_ENABLED)
|
||||
result_data = dump_response(DatasetDetailResponse, dataset)
|
||||
result_data = cast(dict[str, Any], marshal(dataset, dataset_detail_fields))
|
||||
tenant_id = current_tenant_id
|
||||
|
||||
if payload.partial_member_list is not None and payload.permission == DatasetPermissionEnum.PARTIAL_TEAM:
|
||||
@ -638,8 +521,7 @@ class DatasetApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(204, "Dataset deleted successfully")
|
||||
def delete(self, dataset_id: UUID):
|
||||
def delete(self, dataset_id):
|
||||
dataset_id_str = str(dataset_id)
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
@ -649,7 +531,7 @@ class DatasetApi(Resource):
|
||||
try:
|
||||
if DatasetService.delete_dataset(dataset_id_str, current_user):
|
||||
DatasetPermissionService.clear_partial_member_list(dataset_id_str)
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
else:
|
||||
raise NotFound("Dataset not found.")
|
||||
except services.errors.dataset.DatasetInUseError:
|
||||
@ -661,15 +543,11 @@ class DatasetUseCheckApi(Resource):
|
||||
@console_ns.doc("check_dataset_use")
|
||||
@console_ns.doc(description="Check if dataset is in use")
|
||||
@console_ns.doc(params={"dataset_id": "Dataset ID"})
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Dataset use status retrieved successfully",
|
||||
console_ns.models[UsageCheckResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Dataset use status retrieved successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID):
|
||||
def get(self, dataset_id):
|
||||
dataset_id_str = str(dataset_id)
|
||||
|
||||
dataset_is_using = DatasetService.dataset_use_check(dataset_id_str)
|
||||
@ -681,15 +559,11 @@ class DatasetQueryApi(Resource):
|
||||
@console_ns.doc("get_dataset_queries")
|
||||
@console_ns.doc(description="Get dataset query history")
|
||||
@console_ns.doc(params={"dataset_id": "Dataset ID"})
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Query history retrieved successfully",
|
||||
console_ns.models[DatasetQueryListResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Query history retrieved successfully", dataset_query_detail_model)
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID):
|
||||
def get(self, dataset_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
@ -707,24 +581,20 @@ class DatasetQueryApi(Resource):
|
||||
dataset_queries, total = DatasetService.get_dataset_queries(dataset_id=dataset.id, page=page, per_page=limit)
|
||||
|
||||
response = {
|
||||
"data": dataset_queries,
|
||||
"data": marshal(dataset_queries, dataset_query_detail_model),
|
||||
"has_more": len(dataset_queries) == limit,
|
||||
"limit": limit,
|
||||
"total": total,
|
||||
"page": page,
|
||||
}
|
||||
return dump_response(DatasetQueryListResponse, response), 200
|
||||
return response, 200
|
||||
|
||||
|
||||
@console_ns.route("/datasets/indexing-estimate")
|
||||
class DatasetIndexingEstimateApi(Resource):
|
||||
@console_ns.doc("estimate_dataset_indexing")
|
||||
@console_ns.doc(description="Estimate dataset indexing cost")
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Indexing estimate calculated successfully",
|
||||
console_ns.models[IndexingEstimateResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Indexing estimate calculated successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -821,15 +691,12 @@ class DatasetRelatedAppListApi(Resource):
|
||||
@console_ns.doc("get_dataset_related_apps")
|
||||
@console_ns.doc(description="Get applications related to dataset")
|
||||
@console_ns.doc(params={"dataset_id": "Dataset ID"})
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Related apps retrieved successfully",
|
||||
console_ns.models[RelatedAppListResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Related apps retrieved successfully", related_app_list_model)
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID):
|
||||
@marshal_with(related_app_list_model)
|
||||
def get(self, dataset_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
@ -849,7 +716,7 @@ class DatasetRelatedAppListApi(Resource):
|
||||
if app_model:
|
||||
related_apps.append(app_model)
|
||||
|
||||
return dump_response(RelatedAppListResponse, {"data": related_apps, "total": len(related_apps)}), 200
|
||||
return {"data": related_apps, "total": len(related_apps)}, 200
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/indexing-status")
|
||||
@ -857,19 +724,15 @@ class DatasetIndexingStatusApi(Resource):
|
||||
@console_ns.doc("get_dataset_indexing_status")
|
||||
@console_ns.doc(description="Get dataset indexing status")
|
||||
@console_ns.doc(params={"dataset_id": "Dataset ID"})
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Indexing status retrieved successfully",
|
||||
console_ns.models[DocumentStatusListResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Indexing status retrieved successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID):
|
||||
def get(self, dataset_id):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset_id = str(dataset_id)
|
||||
documents = db.session.scalars(
|
||||
select(Document).where(Document.dataset_id == dataset_id_str, Document.tenant_id == current_tenant_id)
|
||||
select(Document).where(Document.dataset_id == dataset_id, Document.tenant_id == current_tenant_id)
|
||||
).all()
|
||||
documents_status = []
|
||||
for document in documents:
|
||||
@ -907,8 +770,9 @@ class DatasetIndexingStatusApi(Resource):
|
||||
"completed_segments": completed_segments,
|
||||
"total_segments": total_segments,
|
||||
}
|
||||
documents_status.append(document_dict)
|
||||
return dump_response(DocumentStatusListResponse, {"data": documents_status}), 200
|
||||
documents_status.append(marshal(document_dict, document_status_fields))
|
||||
data = {"data": documents_status}
|
||||
return data, 200
|
||||
|
||||
|
||||
@console_ns.route("/datasets/api-keys")
|
||||
@ -977,15 +841,15 @@ class DatasetApiDeleteApi(Resource):
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@account_initialization_required
|
||||
def delete(self, api_key_id: UUID):
|
||||
def delete(self, api_key_id):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
api_key_id_str = str(api_key_id)
|
||||
api_key_id = str(api_key_id)
|
||||
key = db.session.scalar(
|
||||
select(ApiToken)
|
||||
.where(
|
||||
ApiToken.tenant_id == current_tenant_id,
|
||||
ApiToken.type == self.resource_type,
|
||||
ApiToken.id == api_key_id_str,
|
||||
ApiToken.id == api_key_id,
|
||||
)
|
||||
.limit(1)
|
||||
)
|
||||
@ -1001,7 +865,7 @@ class DatasetApiDeleteApi(Resource):
|
||||
db.session.delete(key)
|
||||
db.session.commit()
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/api-keys/<string:status>")
|
||||
@ -1009,8 +873,7 @@ class DatasetEnableApiApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def post(self, dataset_id: UUID, status: str):
|
||||
def post(self, dataset_id, status):
|
||||
dataset_id_str = str(dataset_id)
|
||||
|
||||
DatasetService.update_dataset_api_status(dataset_id_str, status == "enable")
|
||||
@ -1022,7 +885,7 @@ class DatasetEnableApiApi(Resource):
|
||||
class DatasetApiBaseUrlApi(Resource):
|
||||
@console_ns.doc("get_dataset_api_base_info")
|
||||
@console_ns.doc(description="Get dataset API base information")
|
||||
@console_ns.response(200, "API base info retrieved successfully", console_ns.models[ApiBaseUrlResponse.__name__])
|
||||
@console_ns.response(200, "API base info retrieved successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -1035,18 +898,13 @@ class DatasetApiBaseUrlApi(Resource):
|
||||
class DatasetRetrievalSettingApi(Resource):
|
||||
@console_ns.doc("get_dataset_retrieval_setting")
|
||||
@console_ns.doc(description="Get dataset retrieval settings")
|
||||
@console_ns.response(
|
||||
200, "Retrieval settings retrieved successfully", console_ns.models[RetrievalSettingResponse.__name__]
|
||||
)
|
||||
@console_ns.response(200, "Retrieval settings retrieved successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self):
|
||||
vector_type = dify_config.VECTOR_STORE
|
||||
return dump_response(
|
||||
RetrievalSettingResponse,
|
||||
_get_retrieval_methods_by_vector_type(vector_type, is_mock=False),
|
||||
)
|
||||
return _get_retrieval_methods_by_vector_type(vector_type, is_mock=False)
|
||||
|
||||
|
||||
@console_ns.route("/datasets/retrieval-setting/<string:vector_type>")
|
||||
@ -1054,19 +912,12 @@ class DatasetRetrievalSettingMockApi(Resource):
|
||||
@console_ns.doc("get_dataset_retrieval_setting_mock")
|
||||
@console_ns.doc(description="Get mock dataset retrieval settings by vector type")
|
||||
@console_ns.doc(params={"vector_type": "Vector store type"})
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Mock retrieval settings retrieved successfully",
|
||||
console_ns.models[RetrievalSettingResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Mock retrieval settings retrieved successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, vector_type: str):
|
||||
return dump_response(
|
||||
RetrievalSettingResponse,
|
||||
_get_retrieval_methods_by_vector_type(vector_type, is_mock=True),
|
||||
)
|
||||
def get(self, vector_type):
|
||||
return _get_retrieval_methods_by_vector_type(vector_type, is_mock=True)
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/error-docs")
|
||||
@ -1074,19 +925,19 @@ class DatasetErrorDocs(Resource):
|
||||
@console_ns.doc("get_dataset_error_docs")
|
||||
@console_ns.doc(description="Get dataset error documents")
|
||||
@console_ns.doc(params={"dataset_id": "Dataset ID"})
|
||||
@console_ns.response(200, "Error documents retrieved successfully", console_ns.models[ErrorDocsResponse.__name__])
|
||||
@console_ns.response(200, "Error documents retrieved successfully")
|
||||
@console_ns.response(404, "Dataset not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID):
|
||||
def get(self, dataset_id):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str)
|
||||
|
||||
return dump_response(ErrorDocsResponse, {"data": results, "total": len(results)}), 200
|
||||
return {"data": [marshal(item, document_status_fields) for item in results], "total": len(results)}, 200
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/permission-part-users")
|
||||
@ -1094,17 +945,13 @@ class DatasetPermissionUserListApi(Resource):
|
||||
@console_ns.doc("get_dataset_permission_users")
|
||||
@console_ns.doc(description="Get dataset permission user list")
|
||||
@console_ns.doc(params={"dataset_id": "Dataset ID"})
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Permission users retrieved successfully",
|
||||
console_ns.models[PartialMemberListResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Permission users retrieved successfully")
|
||||
@console_ns.response(404, "Dataset not found")
|
||||
@console_ns.response(403, "Permission denied")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID):
|
||||
def get(self, dataset_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
@ -1117,7 +964,9 @@ class DatasetPermissionUserListApi(Resource):
|
||||
|
||||
partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str)
|
||||
|
||||
return dump_response(PartialMemberListResponse, {"data": partial_members_list}), 200
|
||||
return {
|
||||
"data": partial_members_list,
|
||||
}, 200
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/auto-disable-logs")
|
||||
@ -1125,18 +974,14 @@ class DatasetAutoDisableLogApi(Resource):
|
||||
@console_ns.doc("get_dataset_auto_disable_logs")
|
||||
@console_ns.doc(description="Get dataset auto disable logs")
|
||||
@console_ns.doc(params={"dataset_id": "Dataset ID"})
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Auto disable logs retrieved successfully",
|
||||
console_ns.models[AutoDisableLogsResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Auto disable logs retrieved successfully")
|
||||
@console_ns.response(404, "Dataset not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID):
|
||||
def get(self, dataset_id):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
return dump_response(AutoDisableLogsResponse, DatasetService.get_dataset_auto_disable_logs(dataset_id_str)), 200
|
||||
return DatasetService.get_dataset_auto_disable_logs(dataset_id_str), 200
|
||||
|
||||
@ -5,7 +5,6 @@ from collections.abc import Sequence
|
||||
from contextlib import ExitStack
|
||||
from datetime import datetime
|
||||
from typing import Any, Literal, cast
|
||||
from uuid import UUID
|
||||
|
||||
import sqlalchemy as sa
|
||||
from flask import request, send_file
|
||||
@ -16,8 +15,7 @@ from werkzeug.exceptions import Forbidden, NotFound
|
||||
|
||||
import services
|
||||
from controllers.common.controller_schemas import DocumentBatchDownloadZipPayload
|
||||
from controllers.common.fields import SimpleResultMessageResponse, SimpleResultResponse, UrlResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from core.errors.error import (
|
||||
LLMBadRequestError,
|
||||
@ -206,7 +204,6 @@ register_schema_models(
|
||||
DocumentWithSegmentsResponse,
|
||||
DatasetAndDocumentResponse,
|
||||
)
|
||||
register_response_schema_models(console_ns, SimpleResultMessageResponse, SimpleResultResponse, UrlResponse)
|
||||
|
||||
|
||||
class DocumentResource(Resource):
|
||||
@ -316,9 +313,9 @@ class DatasetDocumentListApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID):
|
||||
def get(self, dataset_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset_id = str(dataset_id)
|
||||
raw_args = request.args.to_dict()
|
||||
param = DocumentDatasetListParam.model_validate(raw_args)
|
||||
page = param.page
|
||||
@ -343,7 +340,7 @@ class DatasetDocumentListApi(Resource):
|
||||
)
|
||||
except (ArgumentTypeError, ValueError, Exception):
|
||||
fetch = False
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
@ -352,7 +349,7 @@ class DatasetDocumentListApi(Resource):
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
query = select(Document).where(Document.dataset_id == dataset_id_str, Document.tenant_id == current_tenant_id)
|
||||
query = select(Document).where(Document.dataset_id == str(dataset_id), Document.tenant_id == current_tenant_id)
|
||||
|
||||
if status:
|
||||
query = DocumentService.apply_display_status_filter(query, status)
|
||||
@ -373,7 +370,7 @@ class DatasetDocumentListApi(Resource):
|
||||
sa.select(
|
||||
DocumentSegment.document_id, sa.func.sum(DocumentSegment.hit_count).label("total_hit_count")
|
||||
)
|
||||
.where(DocumentSegment.dataset_id == dataset_id_str)
|
||||
.where(DocumentSegment.dataset_id == str(dataset_id))
|
||||
.group_by(DocumentSegment.document_id)
|
||||
.subquery()
|
||||
)
|
||||
@ -445,11 +442,11 @@ class DatasetDocumentListApi(Resource):
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.expect(console_ns.models[KnowledgeConfig.__name__])
|
||||
@console_ns.response(200, "Documents created successfully", console_ns.models[DatasetAndDocumentResponse.__name__])
|
||||
def post(self, dataset_id: UUID):
|
||||
def post(self, dataset_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset_id = str(dataset_id)
|
||||
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
@ -473,7 +470,7 @@ class DatasetDocumentListApi(Resource):
|
||||
|
||||
try:
|
||||
documents, batch = DocumentService.save_document_with_dataset_id(dataset, knowledge_config, current_user)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
|
||||
except ProviderTokenNotInitError as ex:
|
||||
raise ProviderNotInitializeError(ex.description)
|
||||
@ -490,10 +487,9 @@ class DatasetDocumentListApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(204, "Documents deleted successfully")
|
||||
def delete(self, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
def delete(self, dataset_id):
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
@ -505,7 +501,7 @@ class DatasetDocumentListApi(Resource):
|
||||
except services.errors.document.DocumentIndexingError:
|
||||
raise DocumentIndexingError("Cannot delete document during indexing.")
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/datasets/init")
|
||||
@ -583,11 +579,11 @@ class DocumentIndexingEstimateApi(DocumentResource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, dataset_id, document_id):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
document = self.get_document(dataset_id_str, document_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
document_id = str(document_id)
|
||||
document = self.get_document(dataset_id, document_id)
|
||||
|
||||
if document.indexing_status in {IndexingStatus.COMPLETED, IndexingStatus.ERROR}:
|
||||
raise DocumentAlreadyFinishedError()
|
||||
@ -625,7 +621,7 @@ class DocumentIndexingEstimateApi(DocumentResource):
|
||||
data_process_rule_dict,
|
||||
document.doc_form,
|
||||
"English",
|
||||
dataset_id_str,
|
||||
dataset_id,
|
||||
)
|
||||
return estimate_response.model_dump(), 200
|
||||
except LLMBadRequestError:
|
||||
@ -648,10 +644,11 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID, batch: str):
|
||||
def get(self, dataset_id, batch):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
documents = self.get_batch_documents(dataset_id_str, batch)
|
||||
dataset_id = str(dataset_id)
|
||||
batch = str(batch)
|
||||
documents = self.get_batch_documents(dataset_id, batch)
|
||||
if not documents:
|
||||
return {"tokens": 0, "total_price": 0, "currency": "USD", "total_segments": 0, "preview": []}, 200
|
||||
data_process_rule = documents[0].dataset_process_rule
|
||||
@ -725,7 +722,7 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
|
||||
data_process_rule_dict,
|
||||
document.doc_form,
|
||||
"English",
|
||||
dataset_id_str,
|
||||
dataset_id,
|
||||
)
|
||||
return response.model_dump(), 200
|
||||
except LLMBadRequestError:
|
||||
@ -745,9 +742,10 @@ class DocumentBatchIndexingStatusApi(DocumentResource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID, batch: str):
|
||||
dataset_id_str = str(dataset_id)
|
||||
documents = self.get_batch_documents(dataset_id_str, batch)
|
||||
def get(self, dataset_id, batch):
|
||||
dataset_id = str(dataset_id)
|
||||
batch = str(batch)
|
||||
documents = self.get_batch_documents(dataset_id, batch)
|
||||
documents_status = []
|
||||
for document in documents:
|
||||
completed_segments = (
|
||||
@ -799,16 +797,16 @@ class DocumentIndexingStatusApi(DocumentResource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID, document_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
document = self.get_document(dataset_id_str, document_id_str)
|
||||
def get(self, dataset_id, document_id):
|
||||
dataset_id = str(dataset_id)
|
||||
document_id = str(document_id)
|
||||
document = self.get_document(dataset_id, document_id)
|
||||
|
||||
completed_segments = (
|
||||
db.session.scalar(
|
||||
select(func.count(DocumentSegment.id)).where(
|
||||
DocumentSegment.completed_at.isnot(None),
|
||||
DocumentSegment.document_id == str(document_id_str),
|
||||
DocumentSegment.document_id == str(document_id),
|
||||
DocumentSegment.status != SegmentStatus.RE_SEGMENT,
|
||||
)
|
||||
)
|
||||
@ -817,7 +815,7 @@ class DocumentIndexingStatusApi(DocumentResource):
|
||||
total_segments = (
|
||||
db.session.scalar(
|
||||
select(func.count(DocumentSegment.id)).where(
|
||||
DocumentSegment.document_id == str(document_id_str),
|
||||
DocumentSegment.document_id == str(document_id),
|
||||
DocumentSegment.status != SegmentStatus.RE_SEGMENT,
|
||||
)
|
||||
)
|
||||
@ -860,10 +858,10 @@ class DocumentApi(DocumentResource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID, document_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
document = self.get_document(dataset_id_str, document_id_str)
|
||||
def get(self, dataset_id, document_id):
|
||||
dataset_id = str(dataset_id)
|
||||
document_id = str(document_id)
|
||||
document = self.get_document(dataset_id, document_id)
|
||||
|
||||
metadata = request.args.get("metadata", "all")
|
||||
if metadata not in self.METADATA_CHOICES:
|
||||
@ -872,7 +870,7 @@ class DocumentApi(DocumentResource):
|
||||
if metadata == "only":
|
||||
response = {"id": document.id, "doc_type": document.doc_type, "doc_metadata": document.doc_metadata_details}
|
||||
elif metadata == "without":
|
||||
dataset_process_rules = DatasetService.get_process_rules(dataset_id_str)
|
||||
dataset_process_rules = DatasetService.get_process_rules(dataset_id)
|
||||
document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
|
||||
response = {
|
||||
"id": document.id,
|
||||
@ -906,7 +904,7 @@ class DocumentApi(DocumentResource):
|
||||
"need_summary": document.need_summary if document.need_summary is not None else False,
|
||||
}
|
||||
else:
|
||||
dataset_process_rules = DatasetService.get_process_rules(dataset_id_str)
|
||||
dataset_process_rules = DatasetService.get_process_rules(dataset_id)
|
||||
document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
|
||||
response = {
|
||||
"id": document.id,
|
||||
@ -948,24 +946,23 @@ class DocumentApi(DocumentResource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(204, "Document deleted successfully")
|
||||
def delete(self, dataset_id: UUID, document_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
def delete(self, dataset_id, document_id):
|
||||
dataset_id = str(dataset_id)
|
||||
document_id = str(document_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
|
||||
document = self.get_document(dataset_id_str, document_id_str)
|
||||
document = self.get_document(dataset_id, document_id)
|
||||
|
||||
try:
|
||||
DocumentService.delete_document(document)
|
||||
except services.errors.document.DocumentIndexingError:
|
||||
raise DocumentIndexingError("Cannot delete document during indexing.")
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/download")
|
||||
@ -974,7 +971,6 @@ class DocumentDownloadApi(DocumentResource):
|
||||
|
||||
@console_ns.doc("get_dataset_document_download_url")
|
||||
@console_ns.doc(description="Get a signed download URL for a dataset document's original uploaded file")
|
||||
@console_ns.response(200, "Download URL generated successfully", console_ns.models[UrlResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -1002,10 +998,10 @@ class DocumentBatchDownloadZipApi(DocumentResource):
|
||||
payload = DocumentBatchDownloadZipPayload.model_validate(console_ns.payload or {})
|
||||
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset_id = str(dataset_id)
|
||||
document_ids: list[str] = [str(document_id) for document_id in payload.document_ids]
|
||||
upload_files, download_name = DocumentService.prepare_document_batch_download_zip(
|
||||
dataset_id=dataset_id_str,
|
||||
dataset_id=dataset_id,
|
||||
document_ids=document_ids,
|
||||
tenant_id=current_tenant_id,
|
||||
current_user=current_user,
|
||||
@ -1032,22 +1028,18 @@ class DocumentProcessingApi(DocumentResource):
|
||||
@console_ns.doc(
|
||||
params={"dataset_id": "Dataset ID", "document_id": "Document ID", "action": "Action to perform (pause/resume)"}
|
||||
)
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Processing status updated successfully",
|
||||
console_ns.models[SimpleResultResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Processing status updated successfully")
|
||||
@console_ns.response(404, "Document not found")
|
||||
@console_ns.response(400, "Invalid action")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
def patch(self, dataset_id: UUID, document_id: UUID, action: Literal["pause", "resume"]):
|
||||
def patch(self, dataset_id, document_id, action: Literal["pause", "resume"]):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
document = self.get_document(dataset_id_str, document_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
document_id = str(document_id)
|
||||
document = self.get_document(dataset_id, document_id)
|
||||
|
||||
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
|
||||
if not current_user.is_dataset_editor:
|
||||
@ -1081,21 +1073,17 @@ class DocumentMetadataApi(DocumentResource):
|
||||
@console_ns.doc(description="Update document metadata")
|
||||
@console_ns.doc(params={"dataset_id": "Dataset ID", "document_id": "Document ID"})
|
||||
@console_ns.expect(console_ns.models[DocumentMetadataUpdatePayload.__name__])
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Document metadata updated successfully",
|
||||
console_ns.models[SimpleResultMessageResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Document metadata updated successfully")
|
||||
@console_ns.response(404, "Document not found")
|
||||
@console_ns.response(403, "Permission denied")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def put(self, dataset_id: UUID, document_id: UUID):
|
||||
def put(self, dataset_id, document_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
document = self.get_document(dataset_id_str, document_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
document_id = str(document_id)
|
||||
document = self.get_document(dataset_id, document_id)
|
||||
|
||||
req_data = DocumentMetadataUpdatePayload.model_validate(request.get_json() or {})
|
||||
|
||||
@ -1139,11 +1127,10 @@ class DocumentStatusApi(DocumentResource):
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_resource_check("vector_space")
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def patch(self, dataset_id: UUID, action: Literal["enable", "disable", "archive", "un_archive"]):
|
||||
def patch(self, dataset_id, action: Literal["enable", "disable", "archive", "un_archive"]):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
@ -1177,17 +1164,16 @@ class DocumentPauseApi(DocumentResource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(204, "Document paused successfully")
|
||||
def patch(self, dataset_id: UUID, document_id: UUID):
|
||||
def patch(self, dataset_id, document_id):
|
||||
"""pause document."""
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
dataset_id = str(dataset_id)
|
||||
document_id = str(document_id)
|
||||
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
document = DocumentService.get_document(dataset.id, document_id_str)
|
||||
document = DocumentService.get_document(dataset.id, document_id)
|
||||
|
||||
# 404 if document not found
|
||||
if document is None:
|
||||
@ -1203,7 +1189,7 @@ class DocumentPauseApi(DocumentResource):
|
||||
except services.errors.document.DocumentIndexingError:
|
||||
raise DocumentIndexingError("Cannot pause completed document.")
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/processing/resume")
|
||||
@ -1212,15 +1198,14 @@ class DocumentRecoverApi(DocumentResource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(204, "Document resumed successfully")
|
||||
def patch(self, dataset_id: UUID, document_id: UUID):
|
||||
def patch(self, dataset_id, document_id):
|
||||
"""recover document."""
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
document_id = str(document_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
document = DocumentService.get_document(dataset.id, document_id_str)
|
||||
document = DocumentService.get_document(dataset.id, document_id)
|
||||
|
||||
# 404 if document not found
|
||||
if document is None:
|
||||
@ -1235,7 +1220,7 @@ class DocumentRecoverApi(DocumentResource):
|
||||
except services.errors.document.DocumentIndexingError:
|
||||
raise DocumentIndexingError("Document is not in paused status.")
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/retry")
|
||||
@ -1245,12 +1230,11 @@ class DocumentRetryApi(DocumentResource):
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.expect(console_ns.models[DocumentRetryPayload.__name__])
|
||||
@console_ns.response(204, "Documents retry started successfully")
|
||||
def post(self, dataset_id: UUID):
|
||||
def post(self, dataset_id):
|
||||
"""retry document."""
|
||||
payload = DocumentRetryPayload.model_validate(console_ns.payload or {})
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
retry_documents = []
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
@ -1276,9 +1260,9 @@ class DocumentRetryApi(DocumentResource):
|
||||
logger.exception("Failed to retry document, document id: %s", document_id)
|
||||
continue
|
||||
# retry document
|
||||
DocumentService.retry_document(dataset_id_str, retry_documents)
|
||||
DocumentService.retry_document(dataset_id, retry_documents)
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/rename")
|
||||
@ -1288,7 +1272,7 @@ class DocumentRenameApi(DocumentResource):
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Document renamed successfully", console_ns.models[DocumentResponse.__name__])
|
||||
@console_ns.expect(console_ns.models[DocumentRenamePayload.__name__])
|
||||
def post(self, dataset_id: UUID, document_id: UUID):
|
||||
def post(self, dataset_id, document_id):
|
||||
# The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
|
||||
current_user, _ = current_account_with_tenant()
|
||||
if not current_user.is_dataset_editor:
|
||||
@ -1300,7 +1284,7 @@ class DocumentRenameApi(DocumentResource):
|
||||
payload = DocumentRenamePayload.model_validate(console_ns.payload or {})
|
||||
|
||||
try:
|
||||
document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name)
|
||||
document = DocumentService.rename_document(dataset_id, document_id, payload.name)
|
||||
except services.errors.document.DocumentIndexingError:
|
||||
raise DocumentIndexingError("Cannot delete document during indexing.")
|
||||
|
||||
@ -1312,16 +1296,15 @@ class WebsiteDocumentSyncApi(DocumentResource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def get(self, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, dataset_id, document_id):
|
||||
"""sync website document."""
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset.id, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset.id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
if document.tenant_id != current_tenant_id:
|
||||
@ -1332,7 +1315,7 @@ class WebsiteDocumentSyncApi(DocumentResource):
|
||||
if DocumentService.check_archived(document):
|
||||
raise ArchivedDocumentImmutableError()
|
||||
# sync document
|
||||
DocumentService.sync_website_document(dataset_id_str, document)
|
||||
DocumentService.sync_website_document(dataset_id, document)
|
||||
|
||||
return {"result": "success"}, 200
|
||||
|
||||
@ -1342,19 +1325,19 @@ class DocumentPipelineExecutionLogApi(DocumentResource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID, document_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
def get(self, dataset_id, document_id):
|
||||
dataset_id = str(dataset_id)
|
||||
document_id = str(document_id)
|
||||
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
document = DocumentService.get_document(dataset.id, document_id_str)
|
||||
document = DocumentService.get_document(dataset.id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
log = db.session.scalar(
|
||||
select(DocumentPipelineExecutionLog)
|
||||
.where(DocumentPipelineExecutionLog.document_id == document_id_str)
|
||||
.where(DocumentPipelineExecutionLog.document_id == document_id)
|
||||
.order_by(DocumentPipelineExecutionLog.created_at.desc())
|
||||
.limit(1)
|
||||
)
|
||||
@ -1379,11 +1362,7 @@ class DocumentGenerateSummaryApi(Resource):
|
||||
@console_ns.doc(description="Generate summary index for documents")
|
||||
@console_ns.doc(params={"dataset_id": "Dataset ID"})
|
||||
@console_ns.expect(console_ns.models[GenerateSummaryPayload.__name__])
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Summary generation started successfully",
|
||||
console_ns.models[SimpleResultResponse.__name__],
|
||||
)
|
||||
@console_ns.response(200, "Summary generation started successfully")
|
||||
@console_ns.response(400, "Invalid request or dataset configuration")
|
||||
@console_ns.response(403, "Permission denied")
|
||||
@console_ns.response(404, "Dataset not found")
|
||||
@ -1391,7 +1370,7 @@ class DocumentGenerateSummaryApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
def post(self, dataset_id: UUID):
|
||||
def post(self, dataset_id):
|
||||
"""
|
||||
Generate summary index for specified documents.
|
||||
|
||||
@ -1400,10 +1379,10 @@ class DocumentGenerateSummaryApi(Resource):
|
||||
then asynchronously generates summary indexes for the provided documents.
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset_id = str(dataset_id)
|
||||
|
||||
# Get dataset
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
@ -1437,7 +1416,7 @@ class DocumentGenerateSummaryApi(Resource):
|
||||
raise ValueError("Summary index is not enabled for this dataset. Please enable it in the dataset settings.")
|
||||
|
||||
# Verify all documents exist and belong to the dataset
|
||||
documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list)
|
||||
documents = DocumentService.get_documents_by_ids(dataset_id, document_list)
|
||||
|
||||
if len(documents) != len(document_list):
|
||||
found_ids = {doc.id for doc in documents}
|
||||
@ -1451,7 +1430,7 @@ class DocumentGenerateSummaryApi(Resource):
|
||||
if documents_to_update:
|
||||
document_ids_to_update = [str(doc.id) for doc in documents_to_update]
|
||||
DocumentService.update_documents_need_summary(
|
||||
dataset_id=dataset_id_str,
|
||||
dataset_id=dataset_id,
|
||||
document_ids=document_ids_to_update,
|
||||
need_summary=True,
|
||||
)
|
||||
@ -1464,11 +1443,11 @@ class DocumentGenerateSummaryApi(Resource):
|
||||
continue
|
||||
|
||||
# Dispatch async task
|
||||
generate_summary_index_task.delay(dataset_id_str, document.id)
|
||||
generate_summary_index_task.delay(dataset_id, document.id)
|
||||
logger.info(
|
||||
"Dispatched summary generation task for document %s in dataset %s",
|
||||
document.id,
|
||||
dataset_id_str,
|
||||
dataset_id,
|
||||
)
|
||||
|
||||
return {"result": "success"}, 200
|
||||
@ -1484,7 +1463,7 @@ class DocumentSummaryStatusApi(DocumentResource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, dataset_id, document_id):
|
||||
"""
|
||||
Get summary index generation status for a document.
|
||||
|
||||
@ -1498,11 +1477,11 @@ class DocumentSummaryStatusApi(DocumentResource):
|
||||
- summaries: List of summary records with status and content preview
|
||||
"""
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
dataset_id = str(dataset_id)
|
||||
document_id = str(document_id)
|
||||
|
||||
# Get dataset
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
@ -1516,8 +1495,8 @@ class DocumentSummaryStatusApi(DocumentResource):
|
||||
from services.summary_index_service import SummaryIndexService
|
||||
|
||||
result = SummaryIndexService.get_document_summary_status_detail(
|
||||
document_id=document_id_str,
|
||||
dataset_id=dataset_id_str,
|
||||
document_id=document_id,
|
||||
dataset_id=dataset_id,
|
||||
)
|
||||
|
||||
return result, 200
|
||||
|
||||
@ -1,6 +1,4 @@
|
||||
import uuid
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource, marshal
|
||||
@ -12,8 +10,7 @@ from werkzeug.exceptions import Forbidden, NotFound
|
||||
import services
|
||||
from configs import dify_config
|
||||
from controllers.common.controller_schemas import ChildChunkCreatePayload, ChildChunkUpdatePayload
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.error import ProviderNotInitializeError
|
||||
from controllers.console.datasets.error import (
|
||||
@ -33,7 +30,6 @@ from core.model_manager import ModelManager
|
||||
from core.rag.index_processor.constant.index_type import IndexTechniqueType
|
||||
from extensions.ext_database import db
|
||||
from extensions.ext_redis import redis_client
|
||||
from fields.base import ResponseModel
|
||||
from fields.segment_fields import child_chunk_fields, segment_fields
|
||||
from graphon.model_runtime.entities.model_entities import ModelType
|
||||
from libs.helper import escape_like_pattern
|
||||
@ -87,11 +83,6 @@ class BatchImportPayload(BaseModel):
|
||||
upload_file_id: str
|
||||
|
||||
|
||||
class SegmentBatchImportStatusResponse(ResponseModel):
|
||||
job_id: str
|
||||
job_status: str
|
||||
|
||||
|
||||
class ChildChunkBatchUpdatePayload(BaseModel):
|
||||
chunks: list[ChildChunkUpdateArgs]
|
||||
|
||||
@ -107,7 +98,6 @@ register_schema_models(
|
||||
ChildChunkBatchUpdatePayload,
|
||||
ChildChunkUpdateArgs,
|
||||
)
|
||||
register_response_schema_models(console_ns, SegmentBatchImportStatusResponse, SimpleResultResponse)
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/segments")
|
||||
@ -115,12 +105,12 @@ class DatasetDocumentSegmentListApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, dataset_id, document_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
document_id = str(document_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
@ -129,7 +119,7 @@ class DatasetDocumentSegmentListApi(Resource):
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
@ -150,7 +140,7 @@ class DatasetDocumentSegmentListApi(Resource):
|
||||
query = (
|
||||
select(DocumentSegment)
|
||||
.where(
|
||||
DocumentSegment.document_id == document_id_str,
|
||||
DocumentSegment.document_id == str(document_id),
|
||||
DocumentSegment.tenant_id == current_tenant_id,
|
||||
)
|
||||
.order_by(DocumentSegment.position.asc())
|
||||
@ -203,9 +193,7 @@ class DatasetDocumentSegmentListApi(Resource):
|
||||
if segment_ids:
|
||||
from services.summary_index_service import SummaryIndexService
|
||||
|
||||
summary_records = SummaryIndexService.get_segments_summaries(
|
||||
segment_ids=segment_ids, dataset_id=dataset_id_str
|
||||
)
|
||||
summary_records = SummaryIndexService.get_segments_summaries(segment_ids=segment_ids, dataset_id=dataset_id)
|
||||
# Only include enabled summaries (already filtered by service)
|
||||
summaries = {chunk_id: summary.summary_content for chunk_id, summary in summary_records.items()}
|
||||
|
||||
@ -229,20 +217,19 @@ class DatasetDocumentSegmentListApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(204, "Segments deleted successfully")
|
||||
def delete(self, dataset_id: UUID, document_id: UUID):
|
||||
def delete(self, dataset_id, document_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
segment_ids = request.args.getlist("segment_id")
|
||||
@ -255,7 +242,7 @@ class DatasetDocumentSegmentListApi(Resource):
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
SegmentService.delete_segments(segment_ids, document, dataset)
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/segment/<string:action>")
|
||||
@ -265,16 +252,15 @@ class DatasetDocumentSegmentApi(Resource):
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_resource_check("vector_space")
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def patch(self, dataset_id: UUID, document_id: UUID, action: Literal["enable", "disable"]):
|
||||
def patch(self, dataset_id, document_id, action):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# check user's model setting
|
||||
@ -325,17 +311,17 @@ class DatasetDocumentSegmentAddApi(Resource):
|
||||
@cloud_edition_billing_knowledge_limit_check("add_segment")
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.expect(console_ns.models[SegmentCreatePayload.__name__])
|
||||
def post(self, dataset_id: UUID, document_id: UUID):
|
||||
def post(self, dataset_id, document_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
if not current_user.is_dataset_editor:
|
||||
@ -365,7 +351,7 @@ class DatasetDocumentSegmentAddApi(Resource):
|
||||
payload_dict = payload.model_dump(exclude_none=True)
|
||||
SegmentService.segment_create_args_validate(payload_dict, document)
|
||||
segment = SegmentService.create_segment(payload_dict, document, dataset)
|
||||
return {"data": _get_segment_with_summary(segment, dataset_id_str), "doc_form": document.doc_form}, 200
|
||||
return {"data": _get_segment_with_summary(segment, dataset_id), "doc_form": document.doc_form}, 200
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/segments/<uuid:segment_id>")
|
||||
@ -376,19 +362,19 @@ class DatasetDocumentSegmentUpdateApi(Resource):
|
||||
@cloud_edition_billing_resource_check("vector_space")
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.expect(console_ns.models[SegmentUpdatePayload.__name__])
|
||||
def patch(self, dataset_id: UUID, document_id: UUID, segment_id: UUID):
|
||||
def patch(self, dataset_id, document_id, segment_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
|
||||
@ -408,10 +394,10 @@ class DatasetDocumentSegmentUpdateApi(Resource):
|
||||
except ProviderTokenNotInitError as ex:
|
||||
raise ProviderNotInitializeError(ex.description)
|
||||
# check segment
|
||||
segment_id_str = str(segment_id)
|
||||
segment_id = str(segment_id)
|
||||
segment = db.session.scalar(
|
||||
select(DocumentSegment)
|
||||
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
|
||||
.where(DocumentSegment.id == str(segment_id), DocumentSegment.tenant_id == current_tenant_id)
|
||||
.limit(1)
|
||||
)
|
||||
if not segment:
|
||||
@ -432,33 +418,32 @@ class DatasetDocumentSegmentUpdateApi(Resource):
|
||||
segment = SegmentService.update_segment(
|
||||
SegmentUpdateArgs.model_validate(payload.model_dump(exclude_none=True)), segment, document, dataset
|
||||
)
|
||||
return {"data": _get_segment_with_summary(segment, dataset_id_str), "doc_form": document.doc_form}, 200
|
||||
return {"data": _get_segment_with_summary(segment, dataset_id), "doc_form": document.doc_form}, 200
|
||||
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(204, "Segment deleted successfully")
|
||||
def delete(self, dataset_id: UUID, document_id: UUID, segment_id: UUID):
|
||||
def delete(self, dataset_id, document_id, segment_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# check segment
|
||||
segment_id_str = str(segment_id)
|
||||
segment_id = str(segment_id)
|
||||
segment = db.session.scalar(
|
||||
select(DocumentSegment)
|
||||
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
|
||||
.where(DocumentSegment.id == str(segment_id), DocumentSegment.tenant_id == current_tenant_id)
|
||||
.limit(1)
|
||||
)
|
||||
if not segment:
|
||||
@ -471,7 +456,7 @@ class DatasetDocumentSegmentUpdateApi(Resource):
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
SegmentService.delete_segment(segment, document, dataset)
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route(
|
||||
@ -479,7 +464,6 @@ class DatasetDocumentSegmentUpdateApi(Resource):
|
||||
"/datasets/batch_import_status/<uuid:job_id>",
|
||||
)
|
||||
class DatasetDocumentSegmentBatchImportApi(Resource):
|
||||
@console_ns.response(200, "Batch import started", console_ns.models[SegmentBatchImportStatusResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -487,17 +471,17 @@ class DatasetDocumentSegmentBatchImportApi(Resource):
|
||||
@cloud_edition_billing_knowledge_limit_check("add_segment")
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.expect(console_ns.models[BatchImportPayload.__name__])
|
||||
def post(self, dataset_id: UUID, document_id: UUID):
|
||||
def post(self, dataset_id, document_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
|
||||
@ -521,8 +505,8 @@ class DatasetDocumentSegmentBatchImportApi(Resource):
|
||||
batch_create_segment_to_index_task.delay(
|
||||
str(job_id),
|
||||
upload_file_id,
|
||||
dataset_id_str,
|
||||
document_id_str,
|
||||
dataset_id,
|
||||
document_id,
|
||||
current_tenant_id,
|
||||
current_user.id,
|
||||
)
|
||||
@ -530,11 +514,10 @@ class DatasetDocumentSegmentBatchImportApi(Resource):
|
||||
return {"error": str(e)}, 500
|
||||
return {"job_id": job_id, "job_status": "waiting"}, 200
|
||||
|
||||
@console_ns.response(200, "Batch import status", console_ns.models[SegmentBatchImportStatusResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, job_id=None, dataset_id: UUID | None = None, document_id: UUID | None = None):
|
||||
def get(self, job_id=None, dataset_id=None, document_id=None):
|
||||
if job_id is None:
|
||||
raise NotFound("The job does not exist.")
|
||||
job_id = str(job_id)
|
||||
@ -555,24 +538,24 @@ class ChildChunkAddApi(Resource):
|
||||
@cloud_edition_billing_knowledge_limit_check("add_segment")
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.expect(console_ns.models[ChildChunkCreatePayload.__name__])
|
||||
def post(self, dataset_id: UUID, document_id: UUID, segment_id: UUID):
|
||||
def post(self, dataset_id, document_id, segment_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# check segment
|
||||
segment_id_str = str(segment_id)
|
||||
segment_id = str(segment_id)
|
||||
segment = db.session.scalar(
|
||||
select(DocumentSegment)
|
||||
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
|
||||
.where(DocumentSegment.id == str(segment_id), DocumentSegment.tenant_id == current_tenant_id)
|
||||
.limit(1)
|
||||
)
|
||||
if not segment:
|
||||
@ -610,26 +593,26 @@ class ChildChunkAddApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, dataset_id: UUID, document_id: UUID, segment_id: UUID):
|
||||
def get(self, dataset_id, document_id, segment_id):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# check segment
|
||||
segment_id_str = str(segment_id)
|
||||
segment_id = str(segment_id)
|
||||
segment = db.session.scalar(
|
||||
select(DocumentSegment)
|
||||
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
|
||||
.where(DocumentSegment.id == str(segment_id), DocumentSegment.tenant_id == current_tenant_id)
|
||||
.limit(1)
|
||||
)
|
||||
if not segment:
|
||||
@ -646,9 +629,7 @@ class ChildChunkAddApi(Resource):
|
||||
limit = min(args.limit, 100)
|
||||
keyword = args.keyword
|
||||
|
||||
child_chunks = SegmentService.get_child_chunks(
|
||||
segment_id_str, document_id_str, dataset_id_str, page, limit, keyword
|
||||
)
|
||||
child_chunks = SegmentService.get_child_chunks(segment_id, document_id, dataset_id, page, limit, keyword)
|
||||
return {
|
||||
"data": marshal(child_chunks.items, child_chunk_fields),
|
||||
"total": child_chunks.total,
|
||||
@ -662,26 +643,26 @@ class ChildChunkAddApi(Resource):
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_resource_check("vector_space")
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
def patch(self, dataset_id: UUID, document_id: UUID, segment_id: UUID):
|
||||
def patch(self, dataset_id, document_id, segment_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# check segment
|
||||
segment_id_str = str(segment_id)
|
||||
segment_id = str(segment_id)
|
||||
segment = db.session.scalar(
|
||||
select(DocumentSegment)
|
||||
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
|
||||
.where(DocumentSegment.id == str(segment_id), DocumentSegment.tenant_id == current_tenant_id)
|
||||
.limit(1)
|
||||
)
|
||||
if not segment:
|
||||
@ -710,40 +691,39 @@ class ChildChunkUpdateApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(204, "Child chunk deleted successfully")
|
||||
def delete(self, dataset_id: UUID, document_id: UUID, segment_id: UUID, child_chunk_id: UUID):
|
||||
def delete(self, dataset_id, document_id, segment_id, child_chunk_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# check segment
|
||||
segment_id_str = str(segment_id)
|
||||
segment_id = str(segment_id)
|
||||
segment = db.session.scalar(
|
||||
select(DocumentSegment)
|
||||
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
|
||||
.where(DocumentSegment.id == str(segment_id), DocumentSegment.tenant_id == current_tenant_id)
|
||||
.limit(1)
|
||||
)
|
||||
if not segment:
|
||||
raise NotFound("Segment not found.")
|
||||
# check child chunk
|
||||
child_chunk_id_str = str(child_chunk_id)
|
||||
child_chunk_id = str(child_chunk_id)
|
||||
child_chunk = db.session.scalar(
|
||||
select(ChildChunk)
|
||||
.where(
|
||||
ChildChunk.id == str(child_chunk_id_str),
|
||||
ChildChunk.id == str(child_chunk_id),
|
||||
ChildChunk.tenant_id == current_tenant_id,
|
||||
ChildChunk.segment_id == segment.id,
|
||||
ChildChunk.document_id == document_id_str,
|
||||
ChildChunk.document_id == document_id,
|
||||
)
|
||||
.limit(1)
|
||||
)
|
||||
@ -760,7 +740,7 @@ class ChildChunkUpdateApi(Resource):
|
||||
SegmentService.delete_child_chunk(child_chunk, dataset)
|
||||
except ChildChunkDeleteIndexServiceError as e:
|
||||
raise ChildChunkDeleteIndexError(str(e))
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
@setup_required
|
||||
@login_required
|
||||
@ -768,39 +748,39 @@ class ChildChunkUpdateApi(Resource):
|
||||
@cloud_edition_billing_resource_check("vector_space")
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.expect(console_ns.models[ChildChunkUpdatePayload.__name__])
|
||||
def patch(self, dataset_id: UUID, document_id: UUID, segment_id: UUID, child_chunk_id: UUID):
|
||||
def patch(self, dataset_id, document_id, segment_id, child_chunk_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
dataset_id = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id)
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str)
|
||||
document_id = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id, document_id)
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# check segment
|
||||
segment_id_str = str(segment_id)
|
||||
segment_id = str(segment_id)
|
||||
segment = db.session.scalar(
|
||||
select(DocumentSegment)
|
||||
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
|
||||
.where(DocumentSegment.id == str(segment_id), DocumentSegment.tenant_id == current_tenant_id)
|
||||
.limit(1)
|
||||
)
|
||||
if not segment:
|
||||
raise NotFound("Segment not found.")
|
||||
# check child chunk
|
||||
child_chunk_id_str = str(child_chunk_id)
|
||||
child_chunk_id = str(child_chunk_id)
|
||||
child_chunk = db.session.scalar(
|
||||
select(ChildChunk)
|
||||
.where(
|
||||
ChildChunk.id == str(child_chunk_id_str),
|
||||
ChildChunk.id == str(child_chunk_id),
|
||||
ChildChunk.tenant_id == current_tenant_id,
|
||||
ChildChunk.segment_id == segment.id,
|
||||
ChildChunk.document_id == document_id_str,
|
||||
ChildChunk.document_id == document_id,
|
||||
)
|
||||
.limit(1)
|
||||
)
|
||||
|
||||
@ -1,13 +1,10 @@
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource, fields, marshal
|
||||
from pydantic import BaseModel, Field
|
||||
from werkzeug.exceptions import Forbidden, InternalServerError, NotFound
|
||||
|
||||
import services
|
||||
from controllers.common.fields import UsageCountResponse
|
||||
from controllers.common.schema import get_or_create_model, register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import get_or_create_model, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.datasets.error import DatasetNameDuplicateError
|
||||
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
|
||||
@ -30,8 +27,6 @@ from services.external_knowledge_service import ExternalDatasetService
|
||||
from services.hit_testing_service import HitTestingService
|
||||
from services.knowledge_service import BedrockRetrievalSetting, ExternalDatasetTestService
|
||||
|
||||
register_response_schema_models(console_ns, UsageCountResponse)
|
||||
|
||||
|
||||
def _build_dataset_detail_model():
|
||||
keyword_setting_model = get_or_create_model("DatasetKeywordSetting", keyword_setting_fields)
|
||||
@ -177,11 +172,11 @@ class ExternalApiTemplateApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, external_knowledge_api_id: UUID):
|
||||
def get(self, external_knowledge_api_id):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
external_knowledge_api_id_str = str(external_knowledge_api_id)
|
||||
external_knowledge_api_id = str(external_knowledge_api_id)
|
||||
external_knowledge_api = ExternalDatasetService.get_external_knowledge_api(
|
||||
external_knowledge_api_id_str, current_tenant_id
|
||||
external_knowledge_api_id, current_tenant_id
|
||||
)
|
||||
if external_knowledge_api is None:
|
||||
raise NotFound("API template not found.")
|
||||
@ -192,9 +187,9 @@ class ExternalApiTemplateApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.expect(console_ns.models[ExternalKnowledgeApiPayload.__name__])
|
||||
def patch(self, external_knowledge_api_id: UUID):
|
||||
def patch(self, external_knowledge_api_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
external_knowledge_api_id_str = str(external_knowledge_api_id)
|
||||
external_knowledge_api_id = str(external_knowledge_api_id)
|
||||
|
||||
payload = ExternalKnowledgeApiPayload.model_validate(console_ns.payload or {})
|
||||
ExternalDatasetService.validate_api_list(payload.settings)
|
||||
@ -202,7 +197,7 @@ class ExternalApiTemplateApi(Resource):
|
||||
external_knowledge_api = ExternalDatasetService.update_external_knowledge_api(
|
||||
tenant_id=current_tenant_id,
|
||||
user_id=current_user.id,
|
||||
external_knowledge_api_id=external_knowledge_api_id_str,
|
||||
external_knowledge_api_id=external_knowledge_api_id,
|
||||
args=payload.model_dump(),
|
||||
)
|
||||
|
||||
@ -211,16 +206,15 @@ class ExternalApiTemplateApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.response(204, "External knowledge API deleted successfully")
|
||||
def delete(self, external_knowledge_api_id: UUID):
|
||||
def delete(self, external_knowledge_api_id):
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
external_knowledge_api_id_str = str(external_knowledge_api_id)
|
||||
external_knowledge_api_id = str(external_knowledge_api_id)
|
||||
|
||||
if not (current_user.has_edit_permission or current_user.is_dataset_operator):
|
||||
raise Forbidden()
|
||||
|
||||
ExternalDatasetService.delete_external_knowledge_api(current_tenant_id, external_knowledge_api_id_str)
|
||||
return "", 204
|
||||
ExternalDatasetService.delete_external_knowledge_api(current_tenant_id, external_knowledge_api_id)
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/datasets/external-knowledge-api/<uuid:external_knowledge_api_id>/use-check")
|
||||
@ -228,16 +222,16 @@ class ExternalApiUseCheckApi(Resource):
|
||||
@console_ns.doc("check_external_api_usage")
|
||||
@console_ns.doc(description="Check if external knowledge API is being used")
|
||||
@console_ns.doc(params={"external_knowledge_api_id": "External knowledge API ID"})
|
||||
@console_ns.response(200, "Usage check completed successfully", console_ns.models[UsageCountResponse.__name__])
|
||||
@console_ns.response(200, "Usage check completed successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, external_knowledge_api_id: UUID):
|
||||
def get(self, external_knowledge_api_id):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
external_knowledge_api_id_str = str(external_knowledge_api_id)
|
||||
external_knowledge_api_id = str(external_knowledge_api_id)
|
||||
|
||||
external_knowledge_api_is_using, count = ExternalDatasetService.external_knowledge_api_use_check(
|
||||
external_knowledge_api_id_str, current_tenant_id
|
||||
external_knowledge_api_id, current_tenant_id
|
||||
)
|
||||
return {"is_using": external_knowledge_api_is_using, "count": count}, 200
|
||||
|
||||
@ -288,7 +282,7 @@ class ExternalKnowledgeHitTestingApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def post(self, dataset_id: UUID):
|
||||
def post(self, dataset_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
|
||||
@ -2,7 +2,6 @@ from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from flask_restx import Resource
|
||||
from pydantic import Field, field_validator
|
||||
@ -119,7 +118,7 @@ class HitTestingApi(Resource, DatasetsHitTestingBase):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
def post(self, dataset_id: UUID):
|
||||
def post(self, dataset_id):
|
||||
dataset_id_str = str(dataset_id)
|
||||
|
||||
dataset = self.get_and_validate_dataset(dataset_id_str)
|
||||
|
||||
@ -1,19 +1,13 @@
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from flask_restx import Resource
|
||||
from flask_restx import Resource, marshal_with
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from controllers.common.controller_schemas import MetadataUpdatePayload
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import account_initialization_required, enterprise_license_required, setup_required
|
||||
from fields.dataset_fields import (
|
||||
DatasetMetadataBuiltInFieldsResponse,
|
||||
DatasetMetadataListResponse,
|
||||
DatasetMetadataResponse,
|
||||
)
|
||||
from libs.helper import dump_response
|
||||
from fields.dataset_fields import dataset_metadata_fields
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from services.dataset_service import DatasetService
|
||||
from services.entities.knowledge_entities.knowledge_entities import (
|
||||
@ -27,12 +21,6 @@ from services.metadata_service import MetadataService
|
||||
register_schema_models(
|
||||
console_ns, MetadataArgs, MetadataOperationData, MetadataUpdatePayload, DocumentMetadataOperation, MetadataDetail
|
||||
)
|
||||
register_response_schema_models(
|
||||
console_ns,
|
||||
DatasetMetadataBuiltInFieldsResponse,
|
||||
DatasetMetadataListResponse,
|
||||
DatasetMetadataResponse,
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/metadata")
|
||||
@ -41,9 +29,9 @@ class DatasetMetadataCreateApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@enterprise_license_required
|
||||
@console_ns.response(201, "Metadata created successfully", console_ns.models[DatasetMetadataResponse.__name__])
|
||||
@marshal_with(dataset_metadata_fields)
|
||||
@console_ns.expect(console_ns.models[MetadataArgs.__name__])
|
||||
def post(self, dataset_id: UUID):
|
||||
def post(self, dataset_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
metadata_args = MetadataArgs.model_validate(console_ns.payload or {})
|
||||
|
||||
@ -54,22 +42,18 @@ class DatasetMetadataCreateApi(Resource):
|
||||
DatasetService.check_dataset_permission(dataset, current_user)
|
||||
|
||||
metadata = MetadataService.create_metadata(dataset_id_str, metadata_args)
|
||||
return dump_response(DatasetMetadataResponse, metadata), 201
|
||||
return metadata, 201
|
||||
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@enterprise_license_required
|
||||
@console_ns.response(
|
||||
200, "Metadata retrieved successfully", console_ns.models[DatasetMetadataListResponse.__name__]
|
||||
)
|
||||
def get(self, dataset_id: UUID):
|
||||
def get(self, dataset_id):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
metadata = MetadataService.get_dataset_metadatas(dataset)
|
||||
return dump_response(DatasetMetadataListResponse, metadata), 200
|
||||
return MetadataService.get_dataset_metadatas(dataset), 200
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/metadata/<uuid:metadata_id>")
|
||||
@ -78,9 +62,9 @@ class DatasetMetadataApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@enterprise_license_required
|
||||
@console_ns.response(200, "Metadata updated successfully", console_ns.models[DatasetMetadataResponse.__name__])
|
||||
@marshal_with(dataset_metadata_fields)
|
||||
@console_ns.expect(console_ns.models[MetadataUpdatePayload.__name__])
|
||||
def patch(self, dataset_id: UUID, metadata_id: UUID):
|
||||
def patch(self, dataset_id, metadata_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
payload = MetadataUpdatePayload.model_validate(console_ns.payload or {})
|
||||
name = payload.name
|
||||
@ -93,14 +77,13 @@ class DatasetMetadataApi(Resource):
|
||||
DatasetService.check_dataset_permission(dataset, current_user)
|
||||
|
||||
metadata = MetadataService.update_metadata_name(dataset_id_str, metadata_id_str, name)
|
||||
return dump_response(DatasetMetadataResponse, metadata), 200
|
||||
return metadata, 200
|
||||
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@enterprise_license_required
|
||||
@console_ns.response(204, "Metadata deleted successfully")
|
||||
def delete(self, dataset_id: UUID, metadata_id: UUID):
|
||||
def delete(self, dataset_id, metadata_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
metadata_id_str = str(metadata_id)
|
||||
@ -110,8 +93,7 @@ class DatasetMetadataApi(Resource):
|
||||
DatasetService.check_dataset_permission(dataset, current_user)
|
||||
|
||||
MetadataService.delete_metadata(dataset_id_str, metadata_id_str)
|
||||
# Frontend callers only await success and invalidate metadata caches; no response body is consumed.
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/datasets/metadata/built-in")
|
||||
@ -120,14 +102,9 @@ class DatasetMetadataBuiltInFieldApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@enterprise_license_required
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Built-in fields retrieved successfully",
|
||||
console_ns.models[DatasetMetadataBuiltInFieldsResponse.__name__],
|
||||
)
|
||||
def get(self):
|
||||
built_in_fields = MetadataService.get_built_in_fields()
|
||||
return dump_response(DatasetMetadataBuiltInFieldsResponse, {"fields": built_in_fields}), 200
|
||||
return {"fields": built_in_fields}, 200
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/metadata/built-in/<string:action>")
|
||||
@ -136,8 +113,7 @@ class DatasetMetadataBuiltInFieldActionApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@enterprise_license_required
|
||||
@console_ns.response(204, "Action completed successfully")
|
||||
def post(self, dataset_id: UUID, action: Literal["enable", "disable"]):
|
||||
def post(self, dataset_id, action: Literal["enable", "disable"]):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
@ -150,8 +126,7 @@ class DatasetMetadataBuiltInFieldActionApi(Resource):
|
||||
MetadataService.enable_built_in_field(dataset)
|
||||
case "disable":
|
||||
MetadataService.disable_built_in_field(dataset)
|
||||
# Frontend callers only await success and invalidate metadata caches; no response body is consumed.
|
||||
return "", 204
|
||||
return {"result": "success"}, 200
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/documents/metadata")
|
||||
@ -161,11 +136,7 @@ class DocumentMetadataEditApi(Resource):
|
||||
@account_initialization_required
|
||||
@enterprise_license_required
|
||||
@console_ns.expect(console_ns.models[MetadataOperationData.__name__])
|
||||
@console_ns.response(
|
||||
204,
|
||||
"Documents metadata updated successfully",
|
||||
)
|
||||
def post(self, dataset_id: UUID):
|
||||
def post(self, dataset_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str)
|
||||
@ -177,5 +148,4 @@ class DocumentMetadataEditApi(Resource):
|
||||
|
||||
MetadataService.update_documents_metadata(dataset, metadata_args)
|
||||
|
||||
# Frontend callers only await success and invalidate caches; no response body is consumed.
|
||||
return "", 204
|
||||
return {"result": "success"}, 200
|
||||
|
||||
@ -6,8 +6,7 @@ from pydantic import BaseModel, Field
|
||||
from werkzeug.exceptions import Forbidden, NotFound
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
|
||||
from core.plugin.impl.oauth import OAuthHandler
|
||||
@ -57,7 +56,6 @@ register_schema_models(
|
||||
DatasourceDefaultPayload,
|
||||
DatasourceUpdateNamePayload,
|
||||
)
|
||||
register_response_schema_models(console_ns, SimpleResultResponse)
|
||||
|
||||
|
||||
@console_ns.route("/oauth/plugin/<path:provider_id>/datasource/get-authorization-url")
|
||||
@ -211,7 +209,6 @@ class DatasourceAuth(Resource):
|
||||
@console_ns.route("/auth/plugin/datasource/<path:provider_id>/delete")
|
||||
class DatasourceAuthDeleteApi(Resource):
|
||||
@console_ns.expect(console_ns.models[DatasourceCredentialDeletePayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -309,7 +306,6 @@ class DatasourceAuthOauthCustomClient(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def delete(self, provider_id: str):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
@ -325,7 +321,6 @@ class DatasourceAuthOauthCustomClient(Resource):
|
||||
@console_ns.route("/auth/plugin/datasource/<path:provider_id>/default")
|
||||
class DatasourceAuthDefaultApi(Resource):
|
||||
@console_ns.expect(console_ns.models[DatasourceDefaultPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -347,7 +342,6 @@ class DatasourceAuthDefaultApi(Resource):
|
||||
@console_ns.route("/auth/plugin/datasource/<path:provider_id>/update-name")
|
||||
class DatasourceUpdateProviderNameApi(Resource):
|
||||
@console_ns.expect(console_ns.models[DatasourceUpdateNamePayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
|
||||
@ -6,8 +6,7 @@ from pydantic import BaseModel, Field
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from controllers.common.fields import SimpleDataResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import (
|
||||
account_initialization_required,
|
||||
@ -60,7 +59,6 @@ class Payload(BaseModel):
|
||||
|
||||
|
||||
register_schema_models(console_ns, Payload)
|
||||
register_response_schema_models(console_ns, SimpleDataResponse)
|
||||
|
||||
|
||||
@console_ns.route("/rag/pipeline/customized/templates/<string:template_id>")
|
||||
@ -87,7 +85,6 @@ class CustomizedPipelineTemplateApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@enterprise_license_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleDataResponse.__name__])
|
||||
def post(self, template_id: str):
|
||||
with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
|
||||
template = session.scalar(
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
from flask_restx import Resource, marshal
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from werkzeug.exceptions import Forbidden
|
||||
|
||||
import services
|
||||
@ -54,13 +54,12 @@ class CreateRagPipelineDatasetApi(Resource):
|
||||
yaml_content=payload.yaml_content,
|
||||
)
|
||||
try:
|
||||
with Session(db.engine, expire_on_commit=False) as session:
|
||||
with sessionmaker(db.engine).begin() as session:
|
||||
rag_pipeline_dsl_service = RagPipelineDslService(session)
|
||||
import_info = rag_pipeline_dsl_service.create_rag_pipeline_dataset(
|
||||
tenant_id=current_tenant_id,
|
||||
rag_pipeline_dataset_create_entity=rag_pipeline_dataset_create_entity,
|
||||
)
|
||||
session.commit()
|
||||
if rag_pipeline_dataset_create_entity.permission == "partial_members":
|
||||
DatasetPermissionService.update_partial_member_list(
|
||||
current_tenant_id,
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
from flask import request
|
||||
from flask_restx import Resource, fields, marshal_with # type: ignore
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from controllers.common.schema import get_or_create_model, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
@ -67,12 +67,10 @@ class RagPipelineImportApi(Resource):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
payload = RagPipelineImportPayload.model_validate(console_ns.payload or {})
|
||||
|
||||
# Use a plain Session so that caught exceptions inside the service
|
||||
# (which return FAILED status instead of re-raising) do not leave the
|
||||
# transaction in a closed state that a .begin() context manager cannot
|
||||
# handle. See app_import.py for the canonical pattern.
|
||||
with Session(db.engine, expire_on_commit=False) as session:
|
||||
# Create service with session
|
||||
with sessionmaker(db.engine).begin() as session:
|
||||
import_service = RagPipelineDslService(session)
|
||||
# Import app
|
||||
account = current_user
|
||||
result = import_service.import_rag_pipeline(
|
||||
account=account,
|
||||
@ -82,10 +80,6 @@ class RagPipelineImportApi(Resource):
|
||||
pipeline_id=payload.pipeline_id,
|
||||
dataset_name=payload.name,
|
||||
)
|
||||
if result.status == ImportStatus.FAILED:
|
||||
session.rollback()
|
||||
else:
|
||||
session.commit()
|
||||
|
||||
# Return appropriate status code based on result
|
||||
status = result.status
|
||||
@ -105,17 +99,15 @@ class RagPipelineImportConfirmApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@marshal_with(pipeline_import_model)
|
||||
def post(self, import_id: str):
|
||||
def post(self, import_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
with Session(db.engine, expire_on_commit=False) as session:
|
||||
# Create service with session
|
||||
with sessionmaker(db.engine).begin() as session:
|
||||
import_service = RagPipelineDslService(session)
|
||||
# Confirm import
|
||||
account = current_user
|
||||
result = import_service.confirm_import(import_id=import_id, account=account)
|
||||
if result.status == ImportStatus.FAILED:
|
||||
session.rollback()
|
||||
else:
|
||||
session.commit()
|
||||
|
||||
# Return appropriate status code based on result
|
||||
if result.status == ImportStatus.FAILED:
|
||||
@ -132,7 +124,7 @@ class RagPipelineImportCheckDependenciesApi(Resource):
|
||||
@edit_permission_required
|
||||
@marshal_with(pipeline_import_check_dependencies_model)
|
||||
def get(self, pipeline: Pipeline):
|
||||
with Session(db.engine, expire_on_commit=False) as session:
|
||||
with sessionmaker(db.engine).begin() as session:
|
||||
import_service = RagPipelineDslService(session)
|
||||
result = import_service.check_dependencies(pipeline=pipeline)
|
||||
|
||||
@ -150,7 +142,7 @@ class RagPipelineExportApi(Resource):
|
||||
# Add include_secret params
|
||||
query = IncludeSecretQuery.model_validate(request.args.to_dict())
|
||||
|
||||
with Session(db.engine, expire_on_commit=False) as session:
|
||||
with sessionmaker(db.engine).begin() as session:
|
||||
export_service = RagPipelineDslService(session)
|
||||
result = export_service.export_rag_pipeline_dsl(
|
||||
pipeline=pipeline, include_secret=query.include_secret == "true"
|
||||
|
||||
@ -1,7 +1,6 @@
|
||||
import json
|
||||
import logging
|
||||
from typing import Any, Literal, cast
|
||||
from uuid import UUID
|
||||
|
||||
from flask import abort, request
|
||||
from flask_restx import Resource
|
||||
@ -11,7 +10,6 @@ from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotF
|
||||
|
||||
import services
|
||||
from controllers.common.controller_schemas import DefaultBlockConfigQuery, WorkflowListQuery, WorkflowUpdatePayload
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.error import (
|
||||
@ -36,7 +34,6 @@ from core.app.apps.pipeline.pipeline_generator import PipelineGenerator
|
||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||
from extensions.ext_database import db
|
||||
from factories import variable_factory
|
||||
from fields.base import ResponseModel
|
||||
from fields.workflow_run_fields import (
|
||||
WorkflowRunDetailResponse,
|
||||
WorkflowRunNodeExecutionListResponse,
|
||||
@ -118,17 +115,6 @@ class RagPipelineRecommendedPluginQuery(BaseModel):
|
||||
type: str = "all"
|
||||
|
||||
|
||||
class RagPipelineWorkflowSyncResponse(ResponseModel):
|
||||
result: str
|
||||
hash: str
|
||||
updated_at: int
|
||||
|
||||
|
||||
class RagPipelineWorkflowPublishResponse(ResponseModel):
|
||||
result: str
|
||||
created_at: int
|
||||
|
||||
|
||||
register_schema_models(
|
||||
console_ns,
|
||||
DraftWorkflowSyncPayload,
|
||||
@ -147,9 +133,6 @@ register_schema_models(
|
||||
)
|
||||
register_response_schema_models(
|
||||
console_ns,
|
||||
RagPipelineWorkflowPublishResponse,
|
||||
RagPipelineWorkflowSyncResponse,
|
||||
SimpleResultResponse,
|
||||
WorkflowRunDetailResponse,
|
||||
WorkflowRunNodeExecutionListResponse,
|
||||
WorkflowRunNodeExecutionResponse,
|
||||
@ -189,7 +172,6 @@ class DraftRagPipelineApi(Resource):
|
||||
@account_initialization_required
|
||||
@get_rag_pipeline
|
||||
@edit_permission_required
|
||||
@console_ns.response(200, "Success", console_ns.models[RagPipelineWorkflowSyncResponse.__name__])
|
||||
def post(self, pipeline: Pipeline):
|
||||
"""
|
||||
Sync draft workflow
|
||||
@ -480,7 +462,6 @@ class RagPipelineDraftNodeRunApi(Resource):
|
||||
|
||||
@console_ns.route("/rag/pipelines/<uuid:pipeline_id>/workflow-runs/tasks/<string:task_id>/stop")
|
||||
class RagPipelineTaskStopApi(Resource):
|
||||
@console_ns.response(200, "Task stopped successfully", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@edit_permission_required
|
||||
@ -527,7 +508,6 @@ class PublishedRagPipelineApi(Resource):
|
||||
|
||||
return dump_response(WorkflowResponse, workflow)
|
||||
|
||||
@console_ns.response(200, "Success", console_ns.models[RagPipelineWorkflowPublishResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -650,7 +630,6 @@ class PublishedAllRagPipelineApi(Resource):
|
||||
|
||||
@console_ns.route("/rag/pipelines/<uuid:pipeline_id>/workflows/<string:workflow_id>/restore")
|
||||
class RagPipelineDraftWorkflowRestoreApi(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[RagPipelineWorkflowSyncResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -720,7 +699,6 @@ class RagPipelineByIdApi(Resource):
|
||||
|
||||
return dump_response(WorkflowResponse, workflow)
|
||||
|
||||
@console_ns.response(204, "Workflow deleted successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -876,14 +854,14 @@ class RagPipelineWorkflowRunDetailApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_rag_pipeline
|
||||
def get(self, pipeline: Pipeline, run_id: UUID):
|
||||
def get(self, pipeline: Pipeline, run_id):
|
||||
"""
|
||||
Get workflow run detail
|
||||
"""
|
||||
run_id_str = str(run_id)
|
||||
run_id = str(run_id)
|
||||
|
||||
rag_pipeline_service = RagPipelineService()
|
||||
workflow_run = rag_pipeline_service.get_rag_pipeline_workflow_run(pipeline=pipeline, run_id=run_id_str)
|
||||
workflow_run = rag_pipeline_service.get_rag_pipeline_workflow_run(pipeline=pipeline, run_id=run_id)
|
||||
if workflow_run is None:
|
||||
raise NotFound("Workflow run not found")
|
||||
|
||||
@ -905,13 +883,13 @@ class RagPipelineWorkflowRunNodeExecutionListApi(Resource):
|
||||
"""
|
||||
Get workflow run node execution list
|
||||
"""
|
||||
run_id_str = str(run_id)
|
||||
run_id = str(run_id)
|
||||
|
||||
rag_pipeline_service = RagPipelineService()
|
||||
user = cast("Account | EndUser", current_user)
|
||||
node_executions = rag_pipeline_service.get_rag_pipeline_workflow_run_node_executions(
|
||||
pipeline=pipeline,
|
||||
run_id=run_id_str,
|
||||
run_id=run_id,
|
||||
user=user,
|
||||
)
|
||||
|
||||
@ -961,15 +939,15 @@ class RagPipelineTransformApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def post(self, dataset_id: UUID):
|
||||
def post(self, dataset_id: str):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
|
||||
if not (current_user.has_edit_permission or current_user.is_dataset_operator):
|
||||
raise Forbidden()
|
||||
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset_id = str(dataset_id)
|
||||
rag_pipeline_transform_service = RagPipelineTransformService()
|
||||
result = rag_pipeline_transform_service.transform_dataset(dataset_id_str)
|
||||
result = rag_pipeline_transform_service.transform_dataset(dataset_id)
|
||||
return result
|
||||
|
||||
|
||||
|
||||
@ -6,8 +6,7 @@ from pydantic import BaseModel, Field, field_validator
|
||||
from werkzeug.exceptions import InternalServerError, NotFound
|
||||
|
||||
import services
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console.app.error import (
|
||||
AppUnavailableError,
|
||||
CompletionRequestError,
|
||||
@ -73,7 +72,6 @@ class ChatMessagePayload(BaseModel):
|
||||
|
||||
|
||||
register_schema_models(console_ns, CompletionMessageExplorePayload, ChatMessagePayload)
|
||||
register_response_schema_models(console_ns, SimpleResultResponse)
|
||||
|
||||
|
||||
# define completion api for user
|
||||
@ -132,8 +130,7 @@ class CompletionApi(InstalledAppResource):
|
||||
endpoint="installed_app_stop_completion",
|
||||
)
|
||||
class CompletionStopApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def post(self, installed_app, task_id: str):
|
||||
def post(self, installed_app, task_id):
|
||||
app_model = installed_app.app
|
||||
if app_model.mode != AppMode.COMPLETION:
|
||||
raise NotCompletionAppError()
|
||||
@ -208,8 +205,7 @@ class ChatApi(InstalledAppResource):
|
||||
endpoint="installed_app_stop_chat_completion",
|
||||
)
|
||||
class ChatStopApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def post(self, installed_app, task_id: str):
|
||||
def post(self, installed_app, task_id):
|
||||
app_model = installed_app.app
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
if app_mode not in {AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT}:
|
||||
|
||||
@ -1,5 +1,4 @@
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from pydantic import BaseModel, Field, TypeAdapter
|
||||
@ -7,7 +6,7 @@ from sqlalchemy.orm import sessionmaker
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from controllers.common.controller_schemas import ConversationRenamePayload
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console.explore.error import NotChatAppError
|
||||
from controllers.console.explore.wraps import InstalledAppResource
|
||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||
@ -35,7 +34,6 @@ class ConversationListQuery(BaseModel):
|
||||
|
||||
|
||||
register_schema_models(console_ns, ConversationListQuery, ConversationRenamePayload)
|
||||
register_response_schema_models(console_ns, ResultResponse)
|
||||
|
||||
|
||||
@console_ns.route(
|
||||
@ -91,8 +89,7 @@ class ConversationListApi(InstalledAppResource):
|
||||
endpoint="installed_app_conversation",
|
||||
)
|
||||
class ConversationApi(InstalledAppResource):
|
||||
@console_ns.response(204, "Conversation deleted successfully")
|
||||
def delete(self, installed_app, c_id: UUID):
|
||||
def delete(self, installed_app, c_id):
|
||||
app_model = installed_app.app
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
if app_mode not in {AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT}:
|
||||
@ -106,7 +103,7 @@ class ConversationApi(InstalledAppResource):
|
||||
except ConversationNotExistsError:
|
||||
raise NotFound("Conversation Not Exists.")
|
||||
|
||||
return "", 204
|
||||
return ResultResponse(result="success").model_dump(mode="json"), 204
|
||||
|
||||
|
||||
@console_ns.route(
|
||||
@ -115,7 +112,7 @@ class ConversationApi(InstalledAppResource):
|
||||
)
|
||||
class ConversationRenameApi(InstalledAppResource):
|
||||
@console_ns.expect(console_ns.models[ConversationRenamePayload.__name__])
|
||||
def post(self, installed_app, c_id: UUID):
|
||||
def post(self, installed_app, c_id):
|
||||
app_model = installed_app.app
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
if app_mode not in {AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT}:
|
||||
@ -145,8 +142,7 @@ class ConversationRenameApi(InstalledAppResource):
|
||||
endpoint="installed_app_conversation_pin",
|
||||
)
|
||||
class ConversationPinApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[ResultResponse.__name__])
|
||||
def patch(self, installed_app, c_id: UUID):
|
||||
def patch(self, installed_app, c_id):
|
||||
app_model = installed_app.app
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
if app_mode not in {AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT}:
|
||||
@ -169,8 +165,7 @@ class ConversationPinApi(InstalledAppResource):
|
||||
endpoint="installed_app_conversation_unpin",
|
||||
)
|
||||
class ConversationUnPinApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[ResultResponse.__name__])
|
||||
def patch(self, installed_app, c_id: UUID):
|
||||
def patch(self, installed_app, c_id):
|
||||
app_model = installed_app.app
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
if app_mode not in {AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT}:
|
||||
|
||||
@ -8,8 +8,7 @@ from pydantic import BaseModel, Field, computed_field, field_validator
|
||||
from sqlalchemy import and_, select
|
||||
from werkzeug.exceptions import BadRequest, Forbidden, NotFound
|
||||
|
||||
from controllers.common.fields import SimpleMessageResponse, SimpleResultMessageResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.explore.wraps import InstalledAppResource
|
||||
from controllers.console.wraps import account_initialization_required, cloud_edition_billing_resource_check
|
||||
@ -123,7 +122,6 @@ register_schema_models(
|
||||
InstalledAppResponse,
|
||||
InstalledAppListResponse,
|
||||
)
|
||||
register_response_schema_models(console_ns, SimpleMessageResponse, SimpleResultMessageResponse)
|
||||
|
||||
|
||||
@console_ns.route("/installed-apps")
|
||||
@ -211,7 +209,6 @@ class InstalledAppsListApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_resource_check("apps")
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleMessageResponse.__name__])
|
||||
def post(self):
|
||||
payload = InstalledAppCreatePayload.model_validate(console_ns.payload or {})
|
||||
|
||||
@ -261,7 +258,6 @@ class InstalledAppApi(InstalledAppResource):
|
||||
use InstalledAppResource to apply default decorators and get installed_app
|
||||
"""
|
||||
|
||||
@console_ns.response(204, "App uninstalled successfully")
|
||||
def delete(self, installed_app):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
if installed_app.app_owner_tenant_id == current_tenant_id:
|
||||
@ -270,9 +266,8 @@ class InstalledAppApi(InstalledAppResource):
|
||||
db.session.delete(installed_app)
|
||||
db.session.commit()
|
||||
|
||||
return "", 204
|
||||
return {"result": "success", "message": "App uninstalled successfully"}, 204
|
||||
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultMessageResponse.__name__])
|
||||
def patch(self, installed_app):
|
||||
payload = InstalledAppUpdatePayload.model_validate(console_ns.payload or {})
|
||||
|
||||
|
||||
@ -1,13 +1,12 @@
|
||||
import logging
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from pydantic import BaseModel, TypeAdapter
|
||||
from werkzeug.exceptions import InternalServerError, NotFound
|
||||
|
||||
from controllers.common.controller_schemas import MessageFeedbackPayload, MessageListQuery
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console.app.error import (
|
||||
AppMoreLikeThisDisabledError,
|
||||
CompletionRequestError,
|
||||
@ -50,7 +49,6 @@ class MoreLikeThisQuery(BaseModel):
|
||||
|
||||
|
||||
register_schema_models(console_ns, MessageListQuery, MessageFeedbackPayload, MoreLikeThisQuery)
|
||||
register_response_schema_models(console_ns, ResultResponse, SuggestedQuestionsResponse)
|
||||
|
||||
|
||||
@console_ns.route(
|
||||
@ -95,19 +93,18 @@ class MessageListApi(InstalledAppResource):
|
||||
)
|
||||
class MessageFeedbackApi(InstalledAppResource):
|
||||
@console_ns.expect(console_ns.models[MessageFeedbackPayload.__name__])
|
||||
@console_ns.response(200, "Feedback submitted successfully", console_ns.models[ResultResponse.__name__])
|
||||
def post(self, installed_app, message_id: UUID):
|
||||
def post(self, installed_app, message_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
app_model = installed_app.app
|
||||
|
||||
message_id_str = str(message_id)
|
||||
message_id = str(message_id)
|
||||
|
||||
payload = MessageFeedbackPayload.model_validate(console_ns.payload or {})
|
||||
|
||||
try:
|
||||
MessageService.create_feedback(
|
||||
app_model=app_model,
|
||||
message_id=message_id_str,
|
||||
message_id=message_id,
|
||||
user=current_user,
|
||||
rating=FeedbackRating(payload.rating) if payload.rating else None,
|
||||
content=payload.content,
|
||||
@ -124,13 +121,13 @@ class MessageFeedbackApi(InstalledAppResource):
|
||||
)
|
||||
class MessageMoreLikeThisApi(InstalledAppResource):
|
||||
@console_ns.expect(console_ns.models[MoreLikeThisQuery.__name__])
|
||||
def get(self, installed_app, message_id: UUID):
|
||||
def get(self, installed_app, message_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
app_model = installed_app.app
|
||||
if app_model.mode != "completion":
|
||||
raise NotCompletionAppError()
|
||||
|
||||
message_id_str = str(message_id)
|
||||
message_id = str(message_id)
|
||||
|
||||
args = MoreLikeThisQuery.model_validate(request.args.to_dict())
|
||||
|
||||
@ -140,7 +137,7 @@ class MessageMoreLikeThisApi(InstalledAppResource):
|
||||
response = AppGenerateService.generate_more_like_this(
|
||||
app_model=app_model,
|
||||
user=current_user,
|
||||
message_id=message_id_str,
|
||||
message_id=message_id,
|
||||
invoke_from=InvokeFrom.EXPLORE,
|
||||
streaming=streaming,
|
||||
)
|
||||
@ -169,19 +166,18 @@ class MessageMoreLikeThisApi(InstalledAppResource):
|
||||
endpoint="installed_app_suggested_question",
|
||||
)
|
||||
class MessageSuggestedQuestionApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SuggestedQuestionsResponse.__name__])
|
||||
def get(self, installed_app, message_id: UUID):
|
||||
def get(self, installed_app, message_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
app_model = installed_app.app
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
if app_mode not in {AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT}:
|
||||
raise NotChatAppError()
|
||||
|
||||
message_id_str = str(message_id)
|
||||
message_id = str(message_id)
|
||||
|
||||
try:
|
||||
questions = MessageService.get_suggested_questions_after_answer(
|
||||
app_model=app_model, user=current_user, message_id=message_id_str, invoke_from=InvokeFrom.EXPLORE
|
||||
app_model=app_model, user=current_user, message_id=message_id, invoke_from=InvokeFrom.EXPLORE
|
||||
)
|
||||
except MessageNotExistsError:
|
||||
raise NotFound("Message not found")
|
||||
|
||||
@ -1,11 +1,9 @@
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from pydantic import TypeAdapter
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from controllers.common.controller_schemas import SavedMessageCreatePayload, SavedMessageListQuery
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.explore.error import NotCompletionAppError
|
||||
from controllers.console.explore.wraps import InstalledAppResource
|
||||
@ -16,7 +14,6 @@ from services.errors.message import MessageNotExistsError
|
||||
from services.saved_message_service import SavedMessageService
|
||||
|
||||
register_schema_models(console_ns, SavedMessageListQuery, SavedMessageCreatePayload)
|
||||
register_response_schema_models(console_ns, ResultResponse)
|
||||
|
||||
|
||||
@console_ns.route("/installed-apps/<uuid:installed_app_id>/saved-messages", endpoint="installed_app_saved_messages")
|
||||
@ -45,7 +42,6 @@ class SavedMessageListApi(InstalledAppResource):
|
||||
).model_dump(mode="json")
|
||||
|
||||
@console_ns.expect(console_ns.models[SavedMessageCreatePayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[ResultResponse.__name__])
|
||||
def post(self, installed_app):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
app_model = installed_app.app
|
||||
@ -66,16 +62,15 @@ class SavedMessageListApi(InstalledAppResource):
|
||||
"/installed-apps/<uuid:installed_app_id>/saved-messages/<uuid:message_id>", endpoint="installed_app_saved_message"
|
||||
)
|
||||
class SavedMessageApi(InstalledAppResource):
|
||||
@console_ns.response(204, "Saved message deleted successfully")
|
||||
def delete(self, installed_app, message_id: UUID):
|
||||
def delete(self, installed_app, message_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
app_model = installed_app.app
|
||||
|
||||
message_id_str = str(message_id)
|
||||
message_id = str(message_id)
|
||||
|
||||
if app_model.mode != "completion":
|
||||
raise NotCompletionAppError()
|
||||
|
||||
SavedMessageService.delete(app_model, current_user, message_id_str)
|
||||
SavedMessageService.delete(app_model, current_user, message_id)
|
||||
|
||||
return "", 204
|
||||
return ResultResponse(result="success").model_dump(mode="json"), 204
|
||||
|
||||
@ -3,8 +3,7 @@ import logging
|
||||
from werkzeug.exceptions import InternalServerError
|
||||
|
||||
from controllers.common.controller_schemas import WorkflowRunPayload
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_model
|
||||
from controllers.common.schema import register_schema_model
|
||||
from controllers.console.app.error import (
|
||||
CompletionRequestError,
|
||||
ProviderModelCurrentlyNotSupportError,
|
||||
@ -35,7 +34,6 @@ from .. import console_ns
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
register_schema_model(console_ns, WorkflowRunPayload)
|
||||
register_response_schema_models(console_ns, SimpleResultResponse)
|
||||
|
||||
|
||||
@console_ns.route("/installed-apps/<uuid:installed_app_id>/workflows/run")
|
||||
@ -80,7 +78,6 @@ class InstalledAppWorkflowRunApi(InstalledAppResource):
|
||||
|
||||
@console_ns.route("/installed-apps/<uuid:installed_app_id>/workflows/tasks/<string:task_id>/stop")
|
||||
class InstalledAppWorkflowTaskStopApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def post(self, installed_app: InstalledApp, task_id: str):
|
||||
"""
|
||||
Stop workflow task
|
||||
|
||||
@ -1,6 +1,5 @@
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
@ -71,21 +70,6 @@ def _serialize_api_based_extension(extension: APIBasedExtension) -> dict[str, An
|
||||
return APIBasedExtensionResponse.model_validate(extension, from_attributes=True).model_dump(mode="json")
|
||||
|
||||
|
||||
def _serialize_saved_api_based_extension(extension: APIBasedExtension, api_key: str) -> dict[str, Any]:
|
||||
"""Serialize a saved extension with the plaintext key used for response masking only.
|
||||
|
||||
APIBasedExtensionService.save mutates the ORM object to hold the encrypted token before returning it. The response
|
||||
contract, however, should match list/detail responses, where api_key is masked from the decrypted token.
|
||||
"""
|
||||
return APIBasedExtensionResponse(
|
||||
id=extension.id,
|
||||
name=extension.name,
|
||||
api_endpoint=extension.api_endpoint,
|
||||
api_key=api_key,
|
||||
created_at=to_timestamp(extension.created_at),
|
||||
).model_dump(mode="json")
|
||||
|
||||
|
||||
@console_ns.route("/code-based-extension")
|
||||
class CodeBasedExtensionAPI(Resource):
|
||||
@console_ns.doc("get_code_based_extension")
|
||||
@ -141,7 +125,7 @@ class APIBasedExtensionAPI(Resource):
|
||||
api_key=payload.api_key,
|
||||
)
|
||||
|
||||
return _serialize_saved_api_based_extension(APIBasedExtensionService.save(extension_data), payload.api_key), 201
|
||||
return _serialize_api_based_extension(APIBasedExtensionService.save(extension_data))
|
||||
|
||||
|
||||
@console_ns.route("/api-based-extension/<uuid:id>")
|
||||
@ -153,7 +137,7 @@ class APIBasedExtensionDetailAPI(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, id: UUID):
|
||||
def get(self, id):
|
||||
api_based_extension_id = str(id)
|
||||
_, tenant_id = current_account_with_tenant()
|
||||
|
||||
@ -169,26 +153,21 @@ class APIBasedExtensionDetailAPI(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def post(self, id: UUID):
|
||||
def post(self, id):
|
||||
api_based_extension_id = str(id)
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
extension_data_from_db = APIBasedExtensionService.get_with_tenant_id(current_tenant_id, api_based_extension_id)
|
||||
|
||||
payload = APIBasedExtensionPayload.model_validate(console_ns.payload or {})
|
||||
api_key_for_response = extension_data_from_db.api_key
|
||||
|
||||
extension_data_from_db.name = payload.name
|
||||
extension_data_from_db.api_endpoint = payload.api_endpoint
|
||||
|
||||
if payload.api_key != HIDDEN_VALUE:
|
||||
extension_data_from_db.api_key = payload.api_key
|
||||
api_key_for_response = payload.api_key
|
||||
|
||||
return _serialize_saved_api_based_extension(
|
||||
APIBasedExtensionService.save(extension_data_from_db),
|
||||
api_key_for_response,
|
||||
)
|
||||
return _serialize_api_based_extension(APIBasedExtensionService.save(extension_data_from_db))
|
||||
|
||||
@console_ns.doc("delete_api_based_extension")
|
||||
@console_ns.doc(description="Delete API-based extension")
|
||||
@ -197,7 +176,7 @@ class APIBasedExtensionDetailAPI(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def delete(self, id: UUID):
|
||||
def delete(self, id):
|
||||
api_based_extension_id = str(id)
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
@ -205,4 +184,4 @@ class APIBasedExtensionDetailAPI(Resource):
|
||||
|
||||
APIBasedExtensionService.delete(extension_data_from_db)
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
@ -1,15 +1,12 @@
|
||||
from flask_restx import Resource
|
||||
from flask_restx import Resource, fields
|
||||
from werkzeug.exceptions import Unauthorized
|
||||
|
||||
from controllers.common.schema import register_response_schema_models
|
||||
from libs.login import current_account_with_tenant, current_user, login_required
|
||||
from services.feature_service import FeatureModel, FeatureService, LimitationModel, SystemFeatureModel
|
||||
from services.feature_service import FeatureService
|
||||
|
||||
from . import console_ns
|
||||
from .wraps import account_initialization_required, cloud_utm_record, setup_required
|
||||
|
||||
register_response_schema_models(console_ns, FeatureModel, LimitationModel, SystemFeatureModel)
|
||||
|
||||
|
||||
@console_ns.route("/features")
|
||||
class FeatureApi(Resource):
|
||||
@ -18,7 +15,7 @@ class FeatureApi(Resource):
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Success",
|
||||
console_ns.models[FeatureModel.__name__],
|
||||
console_ns.model("FeatureResponse", {"features": fields.Raw(description="Feature configuration object")}),
|
||||
)
|
||||
@setup_required
|
||||
@login_required
|
||||
@ -28,32 +25,7 @@ class FeatureApi(Resource):
|
||||
"""Get feature configuration for current tenant"""
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
payload = FeatureService.get_features(
|
||||
current_tenant_id,
|
||||
exclude_vector_space=True,
|
||||
).model_dump()
|
||||
payload.pop("vector_space", None)
|
||||
return payload
|
||||
|
||||
|
||||
@console_ns.route("/features/vector-space")
|
||||
class FeatureVectorSpaceApi(Resource):
|
||||
@console_ns.doc("get_tenant_feature_vector_space")
|
||||
@console_ns.doc(description="Get vector-space usage and limit for current tenant")
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Success",
|
||||
console_ns.models[LimitationModel.__name__],
|
||||
)
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_utm_record
|
||||
def get(self):
|
||||
"""Get vector-space usage and limit for current tenant"""
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
return FeatureService.get_vector_space(current_tenant_id).model_dump()
|
||||
return FeatureService.get_features(current_tenant_id).model_dump()
|
||||
|
||||
|
||||
@console_ns.route("/system-features")
|
||||
@ -63,7 +35,9 @@ class SystemFeatureApi(Resource):
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Success",
|
||||
console_ns.models[SystemFeatureModel.__name__],
|
||||
console_ns.model(
|
||||
"SystemFeatureResponse", {"features": fields.Raw(description="System feature configuration object")}
|
||||
),
|
||||
)
|
||||
def get(self):
|
||||
"""Get system-wide feature configuration
|
||||
|
||||
@ -1,5 +1,4 @@
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
@ -16,8 +15,7 @@ from controllers.common.errors import (
|
||||
TooManyFilesError,
|
||||
UnsupportedFileTypeError,
|
||||
)
|
||||
from controllers.common.fields import AllowedExtensionsResponse, TextContentResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console.wraps import (
|
||||
account_initialization_required,
|
||||
cloud_edition_billing_resource_check,
|
||||
@ -31,7 +29,6 @@ from services.file_service import FileService
|
||||
from . import console_ns
|
||||
|
||||
register_schema_models(console_ns, UploadConfig, FileResponse)
|
||||
register_response_schema_models(console_ns, AllowedExtensionsResponse, TextContentResponse)
|
||||
|
||||
PREVIEW_WORDS_LIMIT = 3000
|
||||
|
||||
@ -106,11 +103,10 @@ class FilePreviewApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[TextContentResponse.__name__])
|
||||
def get(self, file_id: UUID):
|
||||
file_id_str = str(file_id)
|
||||
def get(self, file_id):
|
||||
file_id = str(file_id)
|
||||
_, tenant_id = current_account_with_tenant()
|
||||
text = FileService(db.engine).get_file_preview(file_id_str, tenant_id)
|
||||
text = FileService(db.engine).get_file_preview(file_id, tenant_id)
|
||||
return {"content": text}
|
||||
|
||||
|
||||
@ -119,6 +115,5 @@ class FileSupportTypeApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[AllowedExtensionsResponse.__name__])
|
||||
def get(self):
|
||||
return {"allowed_extensions": list(DOCUMENT_EXTENSIONS)}
|
||||
|
||||
@ -5,8 +5,6 @@ from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import account_initialization_required, only_edition_cloud, setup_required
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
@ -50,9 +48,6 @@ class DismissNotificationPayload(BaseModel):
|
||||
notification_id: str = Field(...)
|
||||
|
||||
|
||||
register_response_schema_models(console_ns, SimpleResultResponse)
|
||||
|
||||
|
||||
@console_ns.route("/notification")
|
||||
class NotificationApi(Resource):
|
||||
@console_ns.doc("get_notification")
|
||||
@ -115,7 +110,6 @@ class NotificationDismissApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@only_edition_cloud
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
def post(self):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
payload = DismissNotificationPayload.model_validate(request.get_json())
|
||||
|
||||
@ -1,5 +1,6 @@
|
||||
import urllib.parse
|
||||
|
||||
import httpx
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
@ -10,7 +11,6 @@ from controllers.common.errors import (
|
||||
RemoteFileUploadError,
|
||||
UnsupportedFileTypeError,
|
||||
)
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from core.helper import ssrf_proxy
|
||||
from extensions.ext_database import db
|
||||
@ -24,16 +24,11 @@ class RemoteFileUploadPayload(BaseModel):
|
||||
url: str = Field(..., description="URL to fetch")
|
||||
|
||||
|
||||
register_schema_models(console_ns, RemoteFileUploadPayload)
|
||||
register_response_schema_models(console_ns, FileWithSignedUrl, RemoteFileInfo)
|
||||
|
||||
|
||||
@console_ns.route("/remote-files/<path:url>")
|
||||
class GetRemoteFileInfo(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[RemoteFileInfo.__name__])
|
||||
@login_required
|
||||
def get(self, url: str):
|
||||
decoded_url = helpers.decode_remote_url(url, request.query_string)
|
||||
decoded_url = urllib.parse.unquote(url)
|
||||
resp = ssrf_proxy.head(decoded_url)
|
||||
if resp.status_code != httpx.codes.OK:
|
||||
resp = ssrf_proxy.get(decoded_url, timeout=3)
|
||||
@ -46,8 +41,6 @@ class GetRemoteFileInfo(Resource):
|
||||
|
||||
@console_ns.route("/remote-files/upload")
|
||||
class RemoteFileUpload(Resource):
|
||||
@console_ns.expect(console_ns.models[RemoteFileUploadPayload.__name__])
|
||||
@console_ns.response(201, "File uploaded successfully", console_ns.models[FileWithSignedUrl.__name__])
|
||||
@login_required
|
||||
def post(self):
|
||||
payload = RemoteFileUploadPayload.model_validate(console_ns.payload)
|
||||
|
||||
@ -1,13 +1,11 @@
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from werkzeug.exceptions import Forbidden
|
||||
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
|
||||
from fields.base import ResponseModel
|
||||
@ -80,7 +78,6 @@ register_schema_models(
|
||||
TagListQueryParam,
|
||||
TagResponse,
|
||||
)
|
||||
register_response_schema_models(console_ns, SimpleResultResponse)
|
||||
|
||||
|
||||
@console_ns.route("/tags")
|
||||
@ -105,7 +102,6 @@ class TagListApi(Resource):
|
||||
return serialized_tags, 200
|
||||
|
||||
@console_ns.expect(console_ns.models[TagBasePayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[TagResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -128,21 +124,20 @@ class TagListApi(Resource):
|
||||
@console_ns.route("/tags/<uuid:tag_id>")
|
||||
class TagUpdateDeleteApi(Resource):
|
||||
@console_ns.expect(console_ns.models[TagUpdateRequestPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[TagResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def patch(self, tag_id: UUID):
|
||||
def patch(self, tag_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
tag_id_str = str(tag_id)
|
||||
tag_id = str(tag_id)
|
||||
# The role of the current user in the ta table must be admin, owner, or editor
|
||||
if not (current_user.has_edit_permission or current_user.is_dataset_editor):
|
||||
raise Forbidden()
|
||||
|
||||
payload = TagUpdateRequestPayload.model_validate(console_ns.payload or {})
|
||||
tag = TagService.update_tags(UpdateTagPayload(name=payload.name), tag_id_str)
|
||||
tag = TagService.update_tags(UpdateTagPayload(name=payload.name), tag_id)
|
||||
|
||||
binding_count = TagService.get_tag_binding_count(tag_id_str)
|
||||
binding_count = TagService.get_tag_binding_count(tag_id)
|
||||
|
||||
response = TagResponse.model_validate(
|
||||
{"id": tag.id, "name": tag.name, "type": tag.type, "binding_count": binding_count}
|
||||
@ -154,11 +149,10 @@ class TagUpdateDeleteApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@console_ns.response(204, "Tag deleted successfully")
|
||||
def delete(self, tag_id: UUID):
|
||||
tag_id_str = str(tag_id)
|
||||
def delete(self, tag_id):
|
||||
tag_id = str(tag_id)
|
||||
|
||||
TagService.delete_tag(tag_id_str)
|
||||
TagService.delete_tag(tag_id)
|
||||
|
||||
return "", 204
|
||||
|
||||
@ -209,7 +203,6 @@ class TagBindingCollectionApi(Resource):
|
||||
|
||||
@console_ns.doc("create_tag_binding")
|
||||
@console_ns.expect(console_ns.models[TagBindingPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -224,7 +217,6 @@ class TagBindingRemoveApi(Resource):
|
||||
@console_ns.doc("remove_tag_bindings")
|
||||
@console_ns.doc(description="Remove one or more tag bindings from a target.")
|
||||
@console_ns.expect(console_ns.models[TagBindingRemovePayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
|
||||
@ -12,13 +12,7 @@ from werkzeug.exceptions import NotFound
|
||||
|
||||
from configs import dify_config
|
||||
from constants.languages import supported_language
|
||||
from controllers.common.fields import (
|
||||
AvatarUrlResponse,
|
||||
SimpleResultDataResponse,
|
||||
SimpleResultResponse,
|
||||
VerificationTokenResponse,
|
||||
)
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.auth.error import (
|
||||
EmailAlreadyInUseError,
|
||||
@ -56,12 +50,6 @@ from models.enums import CreatorUserRole
|
||||
from models.model import UploadFile
|
||||
from services.account_service import AccountService
|
||||
from services.billing_service import BillingService
|
||||
from services.entities.auth_entities import (
|
||||
ChangeEmailNewEmailToken,
|
||||
ChangeEmailNewEmailVerifiedToken,
|
||||
ChangeEmailOldEmailToken,
|
||||
ChangeEmailOldEmailVerifiedToken,
|
||||
)
|
||||
from services.errors.account import CurrentPasswordIncorrectError as ServiceCurrentPasswordIncorrectError
|
||||
|
||||
|
||||
@ -243,19 +231,11 @@ register_schema_models(
|
||||
EducationStatusResponse,
|
||||
EducationAutocompleteResponse,
|
||||
)
|
||||
register_response_schema_models(
|
||||
console_ns,
|
||||
AvatarUrlResponse,
|
||||
SimpleResultDataResponse,
|
||||
SimpleResultResponse,
|
||||
VerificationTokenResponse,
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/account/init")
|
||||
class AccountInitApi(Resource):
|
||||
@console_ns.expect(console_ns.models[AccountInitPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
def post(self):
|
||||
@ -332,7 +312,6 @@ class AccountAvatarApi(Resource):
|
||||
@console_ns.expect(console_ns.models[AccountAvatarQuery.__name__])
|
||||
@console_ns.doc("get_account_avatar")
|
||||
@console_ns.doc(description="Get account avatar url")
|
||||
@console_ns.response(200, "Success", console_ns.models[AvatarUrlResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -494,7 +473,6 @@ class AccountDeleteVerifyApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultDataResponse.__name__])
|
||||
def get(self):
|
||||
account, _ = current_account_with_tenant()
|
||||
|
||||
@ -507,7 +485,6 @@ class AccountDeleteVerifyApi(Resource):
|
||||
@console_ns.route("/account/delete")
|
||||
class AccountDeleteApi(Resource):
|
||||
@console_ns.expect(console_ns.models[AccountDeletePayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -528,7 +505,6 @@ class AccountDeleteApi(Resource):
|
||||
@console_ns.route("/account/delete/feedback")
|
||||
class AccountDeleteUpdateFeedbackApi(Resource):
|
||||
@console_ns.expect(console_ns.models[AccountDeletionFeedbackPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
def post(self):
|
||||
payload = console_ns.payload or {}
|
||||
@ -608,7 +584,6 @@ class EducationAutoCompleteApi(Resource):
|
||||
@console_ns.route("/account/change-email")
|
||||
class ChangeEmailSendEmailApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ChangeEmailSendPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultDataResponse.__name__])
|
||||
@enable_change_email
|
||||
@setup_required
|
||||
@login_required
|
||||
@ -626,8 +601,8 @@ class ChangeEmailSendEmailApi(Resource):
|
||||
language = "zh-Hans"
|
||||
else:
|
||||
language = "en-US"
|
||||
account = current_user
|
||||
user_email = current_user.email
|
||||
account = None
|
||||
user_email = None
|
||||
email_for_sending = args.email.lower()
|
||||
# Default to the initial phase; any legacy/unexpected client input is
|
||||
# coerced back to `old_email` so we never trust the caller to declare
|
||||
@ -642,18 +617,24 @@ class ChangeEmailSendEmailApi(Resource):
|
||||
if reset_data is None:
|
||||
raise InvalidTokenError()
|
||||
|
||||
if not isinstance(reset_data, ChangeEmailOldEmailVerifiedToken):
|
||||
# The token used to request a new-email code must come from the
|
||||
# old-email verification step. This prevents the bypass described
|
||||
# in GHSA-4q3w-q5mc-45rq where the phase-1 token was reused here.
|
||||
token_phase = reset_data.get(AccountService.CHANGE_EMAIL_TOKEN_PHASE_KEY)
|
||||
if token_phase != AccountService.CHANGE_EMAIL_PHASE_OLD_VERIFIED:
|
||||
raise InvalidTokenError()
|
||||
if not reset_data.is_bound_to_account(current_user.id):
|
||||
raise InvalidTokenError()
|
||||
user_email = reset_data.email
|
||||
user_email = reset_data.get("email", "")
|
||||
|
||||
if user_email.lower() != current_user.email.lower():
|
||||
raise InvalidEmailError()
|
||||
|
||||
user_email = current_user.email
|
||||
else:
|
||||
if email_for_sending != current_user.email.lower():
|
||||
raise InvalidEmailError()
|
||||
email_for_sending = current_user.email
|
||||
account = AccountService.get_account_by_email_with_case_fallback(args.email)
|
||||
if account is None:
|
||||
raise AccountNotFound()
|
||||
email_for_sending = account.email
|
||||
user_email = account.email
|
||||
|
||||
token = AccountService.send_change_email_email(
|
||||
account=account,
|
||||
@ -668,13 +649,11 @@ class ChangeEmailSendEmailApi(Resource):
|
||||
@console_ns.route("/account/change-email/validity")
|
||||
class ChangeEmailCheckApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ChangeEmailValidityPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[VerificationTokenResponse.__name__])
|
||||
@enable_change_email
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def post(self):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
payload = console_ns.payload or {}
|
||||
args = ChangeEmailValidityPayload.model_validate(payload)
|
||||
|
||||
@ -687,26 +666,42 @@ class ChangeEmailCheckApi(Resource):
|
||||
token_data = AccountService.get_change_email_data(args.token)
|
||||
if token_data is None:
|
||||
raise InvalidTokenError()
|
||||
if not token_data.is_bound_to_account(current_user.id):
|
||||
raise InvalidTokenError()
|
||||
|
||||
normalized_token_email = token_data.email.lower()
|
||||
token_email = token_data.get("email")
|
||||
normalized_token_email = token_email.lower() if isinstance(token_email, str) else token_email
|
||||
if user_email != normalized_token_email:
|
||||
raise InvalidEmailError()
|
||||
|
||||
if args.code != token_data.code:
|
||||
if args.code != token_data.get("code"):
|
||||
AccountService.add_change_email_error_rate_limit(user_email)
|
||||
raise EmailCodeError()
|
||||
|
||||
if isinstance(token_data, ChangeEmailOldEmailToken | ChangeEmailNewEmailToken):
|
||||
refreshed_token_data = token_data.promote()
|
||||
else:
|
||||
# Only advance tokens that were minted by the matching send-code step;
|
||||
# refuse tokens that have already progressed or lack a phase marker so
|
||||
# the chain `old_email -> old_email_verified -> new_email -> new_email_verified`
|
||||
# is strictly enforced.
|
||||
phase_transitions = {
|
||||
AccountService.CHANGE_EMAIL_PHASE_OLD: AccountService.CHANGE_EMAIL_PHASE_OLD_VERIFIED,
|
||||
AccountService.CHANGE_EMAIL_PHASE_NEW: AccountService.CHANGE_EMAIL_PHASE_NEW_VERIFIED,
|
||||
}
|
||||
token_phase = token_data.get(AccountService.CHANGE_EMAIL_TOKEN_PHASE_KEY)
|
||||
if not isinstance(token_phase, str):
|
||||
raise InvalidTokenError()
|
||||
refreshed_phase = phase_transitions.get(token_phase)
|
||||
if refreshed_phase is None:
|
||||
raise InvalidTokenError()
|
||||
|
||||
# Verified, revoke the first token
|
||||
AccountService.revoke_change_email_token(args.token)
|
||||
|
||||
new_token = AccountService.generate_change_email_token(refreshed_token_data, current_user)
|
||||
# Refresh token data by generating a new token that carries the
|
||||
# upgraded phase so later steps can check it.
|
||||
_, new_token = AccountService.generate_change_email_token(
|
||||
user_email,
|
||||
code=args.code,
|
||||
old_email=token_data.get("old_email"),
|
||||
additional_data={AccountService.CHANGE_EMAIL_TOKEN_PHASE_KEY: refreshed_phase},
|
||||
)
|
||||
|
||||
AccountService.reset_change_email_error_rate_limit(user_email)
|
||||
return {"is_valid": True, "email": normalized_token_email, "token": new_token}
|
||||
@ -731,22 +726,27 @@ class ChangeEmailResetApi(Resource):
|
||||
if not AccountService.check_email_unique(normalized_new_email):
|
||||
raise EmailAlreadyInUseError()
|
||||
|
||||
current_user, _ = current_account_with_tenant()
|
||||
reset_data = AccountService.get_change_email_data(args.token)
|
||||
if not reset_data:
|
||||
raise InvalidTokenError()
|
||||
if not reset_data.is_bound_to_account(current_user.id):
|
||||
raise InvalidTokenError()
|
||||
|
||||
if not isinstance(reset_data, ChangeEmailNewEmailVerifiedToken):
|
||||
# Only tokens that completed both verification phases may be used to
|
||||
# change the email. This closes GHSA-4q3w-q5mc-45rq where a token from
|
||||
# the initial send-code step could be replayed directly here.
|
||||
token_phase = reset_data.get(AccountService.CHANGE_EMAIL_TOKEN_PHASE_KEY)
|
||||
if token_phase != AccountService.CHANGE_EMAIL_PHASE_NEW_VERIFIED:
|
||||
raise InvalidTokenError()
|
||||
|
||||
# Bind the new email to the token that was mailed and verified, so a
|
||||
# verified token cannot be reused with a different `new_email` value.
|
||||
if reset_data.email.lower() != normalized_new_email:
|
||||
token_email = reset_data.get("email")
|
||||
normalized_token_email = token_email.lower() if isinstance(token_email, str) else token_email
|
||||
if normalized_token_email != normalized_new_email:
|
||||
raise InvalidTokenError()
|
||||
|
||||
if current_user.email.lower() != reset_data.old_email.lower():
|
||||
old_email = reset_data.get("old_email", "")
|
||||
current_user, _ = current_account_with_tenant()
|
||||
if current_user.email.lower() != old_email.lower():
|
||||
raise AccountNotFound()
|
||||
|
||||
# Revoke only after all checks pass so failed attempts don't burn a
|
||||
@ -765,7 +765,6 @@ class ChangeEmailResetApi(Resource):
|
||||
@console_ns.route("/account/change-email/check-email-unique")
|
||||
class CheckEmailUnique(Resource):
|
||||
@console_ns.expect(console_ns.models[CheckEmailUniquePayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
def post(self):
|
||||
payload = console_ns.payload or {}
|
||||
|
||||
@ -1,5 +1,4 @@
|
||||
from urllib import parse
|
||||
from uuid import UUID
|
||||
|
||||
from flask import abort, request
|
||||
from flask_restx import Resource
|
||||
@ -7,8 +6,7 @@ from pydantic import BaseModel, Field, TypeAdapter
|
||||
|
||||
import services
|
||||
from configs import dify_config
|
||||
from controllers.common.fields import SimpleResultDataResponse, VerificationTokenResponse
|
||||
from controllers.common.schema import register_enum_models, register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_enum_models, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.auth.error import (
|
||||
CannotTransferOwnerToSelfError,
|
||||
@ -32,14 +30,13 @@ from libs.helper import extract_remote_ip
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from models.account import Account, TenantAccountRole
|
||||
from services.account_service import AccountService, RegisterService, TenantService
|
||||
from services.enterprise import rbac_service as enterprise_rbac_service
|
||||
from services.errors.account import AccountAlreadyInTenantError
|
||||
from services.feature_service import FeatureService
|
||||
|
||||
|
||||
class MemberInvitePayload(BaseModel):
|
||||
emails: list[str] = Field(default_factory=list)
|
||||
role: str
|
||||
role: TenantAccountRole
|
||||
language: str | None = None
|
||||
|
||||
|
||||
@ -71,7 +68,6 @@ register_schema_models(
|
||||
OwnerTransferCheckPayload,
|
||||
OwnerTransferPayload,
|
||||
)
|
||||
register_response_schema_models(console_ns, SimpleResultDataResponse, VerificationTokenResponse)
|
||||
|
||||
|
||||
def _is_role_enabled(role: TenantAccountRole | str, tenant_id: str) -> bool:
|
||||
@ -80,19 +76,6 @@ def _is_role_enabled(role: TenantAccountRole | str, tenant_id: str) -> bool:
|
||||
return FeatureService.get_features(tenant_id=tenant_id).dataset_operator_enabled
|
||||
|
||||
|
||||
def _serialize_member_roles(current_role: str | None, member_roles: list[enterprise_rbac_service.MemberRoleSummary]) -> list[dict[str, str]]:
|
||||
if member_roles:
|
||||
return [{"id": role.id, "name": role.name} for role in member_roles]
|
||||
if current_role:
|
||||
return [{"id": current_role, "name": current_role}]
|
||||
return []
|
||||
|
||||
|
||||
def _normalize_enum_value(value: object) -> str:
|
||||
normalized = getattr(value, "value", value)
|
||||
return str(normalized) if normalized is not None else ""
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/members")
|
||||
class MemberListApi(Resource):
|
||||
"""List all members of current tenant."""
|
||||
@ -106,36 +89,7 @@ class MemberListApi(Resource):
|
||||
if not current_user.current_tenant:
|
||||
raise ValueError("No current tenant")
|
||||
members = TenantService.get_tenant_members(current_user.current_tenant)
|
||||
if dify_config.RBAC_ENABLED:
|
||||
member_ids = [member.id for member in members]
|
||||
member_roles = enterprise_rbac_service.RBACService.MemberRoles.batch_get(
|
||||
str(current_user.current_tenant.id),
|
||||
current_user.id,
|
||||
member_ids,
|
||||
)
|
||||
roles_map = {item.account_id: item.roles for item in member_roles}
|
||||
else:
|
||||
roles_map = {}
|
||||
|
||||
serialized_members = []
|
||||
for member in members:
|
||||
current_role = _normalize_enum_value(member.current_role)
|
||||
serialized_members.append(
|
||||
{
|
||||
"id": member.id,
|
||||
"name": member.name,
|
||||
"email": member.email,
|
||||
"avatar": member.avatar,
|
||||
"last_login_at": member.last_login_at,
|
||||
"last_active_at": member.last_active_at,
|
||||
"created_at": member.created_at,
|
||||
"role": current_role,
|
||||
"roles": _serialize_member_roles(current_role, roles_map.get(member.id, [])),
|
||||
"status": _normalize_enum_value(member.status),
|
||||
}
|
||||
)
|
||||
|
||||
member_models = TypeAdapter(list[AccountWithRole]).validate_python(serialized_members)
|
||||
member_models = TypeAdapter(list[AccountWithRole]).validate_python(members, from_attributes=True)
|
||||
response = AccountWithRoleList(accounts=member_models)
|
||||
return response.model_dump(mode="json"), 200
|
||||
|
||||
@ -156,9 +110,8 @@ class MemberInviteEmailApi(Resource):
|
||||
invitee_emails = args.emails
|
||||
invitee_role = args.role
|
||||
interface_language = args.language
|
||||
if not dify_config.RBAC_ENABLED:
|
||||
if not TenantAccountRole.is_valid_role(invitee_role) or not TenantAccountRole.is_non_owner_role(invitee_role):
|
||||
return {"code": "invalid-role", "message": "Invalid role"}, 400
|
||||
if not TenantAccountRole.is_non_owner_role(invitee_role):
|
||||
return {"code": "invalid-role", "message": "Invalid role"}, 400
|
||||
current_user, _ = current_account_with_tenant()
|
||||
inviter = current_user
|
||||
if not inviter.current_tenant:
|
||||
@ -220,7 +173,7 @@ class MemberCancelInviteApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def delete(self, member_id: UUID):
|
||||
def delete(self, member_id):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
if not current_user.current_tenant:
|
||||
raise ValueError("No current tenant")
|
||||
@ -253,7 +206,7 @@ class MemberUpdateRoleApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def put(self, member_id: UUID):
|
||||
def put(self, member_id):
|
||||
payload = console_ns.payload or {}
|
||||
args = MemberRoleUpdatePayload.model_validate(payload)
|
||||
new_role = args.role
|
||||
@ -309,7 +262,6 @@ class SendOwnerTransferEmailApi(Resource):
|
||||
"""Send owner transfer email."""
|
||||
|
||||
@console_ns.expect(console_ns.models[OwnerTransferEmailPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultDataResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -347,7 +299,6 @@ class SendOwnerTransferEmailApi(Resource):
|
||||
@console_ns.route("/workspaces/current/members/owner-transfer-check")
|
||||
class OwnerTransferCheckApi(Resource):
|
||||
@console_ns.expect(console_ns.models[OwnerTransferCheckPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[VerificationTokenResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -396,7 +347,7 @@ class OwnerTransfer(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@is_allow_transfer_owner
|
||||
def post(self, member_id: UUID):
|
||||
def post(self, member_id):
|
||||
payload = console_ns.payload or {}
|
||||
args = OwnerTransferPayload.model_validate(payload)
|
||||
|
||||
|
||||
@ -5,8 +5,7 @@ from flask import request, send_file
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
|
||||
from graphon.model_runtime.entities.model_entities import ModelType
|
||||
@ -86,7 +85,6 @@ register_schema_models(
|
||||
ParserCredentialValidate,
|
||||
ParserPreferredProviderType,
|
||||
)
|
||||
register_response_schema_models(console_ns, SimpleResultResponse)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/model-providers")
|
||||
@ -179,7 +177,6 @@ class ModelProviderCredentialApi(Resource):
|
||||
return {"result": "success"}
|
||||
|
||||
@console_ns.expect(console_ns.models[ParserCredentialDelete.__name__])
|
||||
@console_ns.response(204, "Credential deleted successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@ -194,13 +191,12 @@ class ModelProviderCredentialApi(Resource):
|
||||
tenant_id=current_tenant_id, provider=provider, credential_id=args.credential_id
|
||||
)
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/model-providers/<path:provider>/credentials/switch")
|
||||
class ModelProviderCredentialSwitchApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ParserCredentialSwitch.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@ -275,7 +271,6 @@ class ModelProviderIconApi(Resource):
|
||||
@console_ns.route("/workspaces/current/model-providers/<path:provider>/preferred-provider-type")
|
||||
class PreferredProviderTypeUpdateApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ParserPreferredProviderType.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
|
||||
@ -5,8 +5,7 @@ from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_enum_models, register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_enum_models, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
|
||||
from graphon.model_runtime.entities.model_entities import ModelType
|
||||
@ -127,7 +126,6 @@ register_schema_models(
|
||||
Inner,
|
||||
ParserSwitch,
|
||||
)
|
||||
register_response_schema_models(console_ns, SimpleResultResponse)
|
||||
|
||||
register_enum_models(console_ns, ModelType)
|
||||
|
||||
@ -151,7 +149,6 @@ class DefaultModelApi(Resource):
|
||||
return jsonable_encoder({"data": default_model_entity})
|
||||
|
||||
@console_ns.expect(console_ns.models[ParserPostDefault.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@ -244,7 +241,6 @@ class ModelProviderModelApi(Resource):
|
||||
return {"result": "success"}, 200
|
||||
|
||||
@console_ns.expect(console_ns.models[ParserDeleteModels.__name__])
|
||||
@console_ns.response(204, "Model deleted successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@ -259,7 +255,7 @@ class ModelProviderModelApi(Resource):
|
||||
tenant_id=tenant_id, provider=provider, model=args.model, model_type=args.model_type
|
||||
)
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/model-providers/<path:provider>/models/credentials")
|
||||
@ -377,7 +373,6 @@ class ModelProviderModelCredentialApi(Resource):
|
||||
return {"result": "success"}
|
||||
|
||||
@console_ns.expect(console_ns.models[ParserDeleteCredential.__name__])
|
||||
@console_ns.response(204, "Credential deleted successfully")
|
||||
@setup_required
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@ -395,13 +390,12 @@ class ModelProviderModelCredentialApi(Resource):
|
||||
credential_id=args.credential_id,
|
||||
)
|
||||
|
||||
return "", 204
|
||||
return {"result": "success"}, 204
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/model-providers/<path:provider>/models/credentials/switch")
|
||||
class ModelProviderModelCredentialSwitchApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ParserSwitch.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@ -426,7 +420,6 @@ class ModelProviderModelCredentialSwitchApi(Resource):
|
||||
)
|
||||
class ModelProviderModelEnableApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ParserDeleteModels.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -448,7 +441,6 @@ class ModelProviderModelEnableApi(Resource):
|
||||
)
|
||||
class ModelProviderModelDisableApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ParserDeleteModels.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -532,7 +524,7 @@ class ModelProviderAvailableModelApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def get(self, model_type: str):
|
||||
def get(self, model_type):
|
||||
_, tenant_id = current_account_with_tenant()
|
||||
model_provider_service = ModelProviderService()
|
||||
models = model_provider_service.get_models_by_model_type(tenant_id=tenant_id, model_type=model_type)
|
||||
|
||||
@ -9,20 +9,18 @@ from werkzeug.datastructures import FileStorage
|
||||
from werkzeug.exceptions import Forbidden
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.fields import SuccessResponse
|
||||
from controllers.common.schema import register_enum_models, register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_enum_models, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.workspace import plugin_permission_required
|
||||
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
|
||||
from core.plugin.impl.exc import PluginDaemonClientSideError
|
||||
from core.plugin.plugin_service import PluginService
|
||||
from fields.base import ResponseModel
|
||||
from graphon.model_runtime.utils.encoders import jsonable_encoder
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from models.account import TenantPluginAutoUpgradeStrategy, TenantPluginPermission
|
||||
from services.plugin.plugin_auto_upgrade_service import PluginAutoUpgradeService
|
||||
from services.plugin.plugin_parameter_service import PluginParameterService
|
||||
from services.plugin.plugin_permission_service import PluginPermissionService
|
||||
from services.plugin.plugin_service import PluginService
|
||||
|
||||
|
||||
class ParserList(BaseModel):
|
||||
@ -139,12 +137,6 @@ class ParserReadme(BaseModel):
|
||||
language: str = Field(default="en-US")
|
||||
|
||||
|
||||
class PluginDebuggingKeyResponse(ResponseModel):
|
||||
key: str
|
||||
host: str
|
||||
port: int
|
||||
|
||||
|
||||
register_schema_models(
|
||||
console_ns,
|
||||
ParserList,
|
||||
@ -168,7 +160,6 @@ register_schema_models(
|
||||
ParserExcludePlugin,
|
||||
ParserReadme,
|
||||
)
|
||||
register_response_schema_models(console_ns, PluginDebuggingKeyResponse, SuccessResponse)
|
||||
|
||||
register_enum_models(
|
||||
console_ns,
|
||||
@ -195,7 +186,6 @@ def _read_upload_content(file: FileStorage, max_size: int) -> bytes:
|
||||
|
||||
@console_ns.route("/workspaces/current/plugin/debugging-key")
|
||||
class PluginDebuggingKeyApi(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[PluginDebuggingKeyResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -501,7 +491,6 @@ class PluginFetchInstallTaskApi(Resource):
|
||||
|
||||
@console_ns.route("/workspaces/current/plugin/tasks/<task_id>/delete")
|
||||
class PluginDeleteInstallTaskApi(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SuccessResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -517,7 +506,6 @@ class PluginDeleteInstallTaskApi(Resource):
|
||||
|
||||
@console_ns.route("/workspaces/current/plugin/tasks/delete_all")
|
||||
class PluginDeleteAllInstallTaskItemsApi(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SuccessResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -533,7 +521,6 @@ class PluginDeleteAllInstallTaskItemsApi(Resource):
|
||||
|
||||
@console_ns.route("/workspaces/current/plugin/tasks/<task_id>/delete/<path:identifier>")
|
||||
class PluginDeleteInstallTaskItemApi(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SuccessResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -599,7 +586,6 @@ class PluginUpgradeFromGithubApi(Resource):
|
||||
@console_ns.route("/workspaces/current/plugin/uninstall")
|
||||
class PluginUninstallApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ParserUninstall.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SuccessResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@ -618,7 +604,6 @@ class PluginUninstallApi(Resource):
|
||||
@console_ns.route("/workspaces/current/plugin/permission/change")
|
||||
class PluginChangePermissionApi(Resource):
|
||||
@console_ns.expect(console_ns.models[ParserPermissionChange.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SuccessResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
|
||||
@ -1,614 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from functools import wraps
|
||||
from typing import Any
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import AliasChoices, BaseModel, ConfigDict, Field, ValidationError, field_validator
|
||||
from werkzeug.exceptions import Forbidden, NotFound
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.console import console_ns
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from services.enterprise import rbac_service as svc
|
||||
|
||||
|
||||
_LEGACY_ROLE_PERMISSION_KEYS: dict[str, list[str]] = {
|
||||
# This is a compatibility projection from the pre-RBAC workspace roles into
|
||||
# the 2.0 permission matrix documented in "权限整理2.0". It intentionally
|
||||
# models the product-facing role surface for the new RBAC UI instead of the
|
||||
# legacy backend's exact hard-authorization checks.
|
||||
"owner": [
|
||||
*svc._LEGACY_WORKSPACE_OWNER_KEYS,
|
||||
*svc._LEGACY_APP_OWNER_KEYS,
|
||||
*svc._LEGACY_DATASET_OWNER_KEYS,
|
||||
],
|
||||
"admin": [
|
||||
*svc._LEGACY_WORKSPACE_ADMIN_KEYS,
|
||||
*svc._LEGACY_APP_ADMIN_KEYS,
|
||||
*svc._LEGACY_DATASET_ADMIN_KEYS,
|
||||
],
|
||||
"editor": [
|
||||
*svc._LEGACY_WORKSPACE_EDITOR_KEYS,
|
||||
*svc._LEGACY_APP_EDITOR_KEYS,
|
||||
*svc._LEGACY_DATASET_EDITOR_KEYS,
|
||||
],
|
||||
"normal": [
|
||||
*svc._LEGACY_WORKSPACE_NORMAL_KEYS,
|
||||
*svc._LEGACY_APP_NORMAL_KEYS,
|
||||
],
|
||||
"dataset_operator": [
|
||||
*svc._LEGACY_WORKSPACE_DATASET_OPERATOR_KEYS,
|
||||
*svc._LEGACY_DATASET_DATASET_OPERATOR_KEYS,
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _current_ids() -> tuple[str, str]:
|
||||
"""Return ``(tenant_id, account_id)`` for the authenticated user, or
|
||||
raise a 404 when no tenant is associated with the session.
|
||||
"""
|
||||
|
||||
user, tenant_id = current_account_with_tenant()
|
||||
if not tenant_id:
|
||||
raise NotFound("Current workspace not found")
|
||||
return tenant_id, user.id
|
||||
|
||||
|
||||
def _payload(model: type[BaseModel]) -> Any:
|
||||
"""Validate the JSON body against ``model`` or raise ``ValidationError``.
|
||||
|
||||
``ValidationError`` bubbles up as HTTP 400 thanks to
|
||||
``controllers/common/helpers.py`` error handling.
|
||||
"""
|
||||
try:
|
||||
return model.model_validate(console_ns.payload or {})
|
||||
except ValidationError as exc:
|
||||
# Re-raise as-is so the upstream error handler renders a 400.
|
||||
raise exc
|
||||
|
||||
|
||||
def _dump(model: BaseModel) -> dict[str, Any]:
|
||||
return model.model_dump(mode="json")
|
||||
|
||||
|
||||
class _PaginationQuery(BaseModel):
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
page_number: int | None = Field(default=None, ge=1, validation_alias=AliasChoices("page", "page_number"))
|
||||
results_per_page: int | None = Field(
|
||||
default=None, ge=1, le=100, validation_alias=AliasChoices("limit", "results_per_page")
|
||||
)
|
||||
reverse: bool | None = None
|
||||
|
||||
def to_inner_options(self) -> svc.ListOption:
|
||||
return svc.ListOption.model_validate(self.model_dump())
|
||||
|
||||
|
||||
class _RolesListQuery(_PaginationQuery):
|
||||
include_owner: int = Field(default=0, ge=0, le=1)
|
||||
|
||||
|
||||
def _pagination_options() -> svc.ListOption:
|
||||
return _PaginationQuery.model_validate(request.args.to_dict(flat=True)).to_inner_options()
|
||||
|
||||
|
||||
def _filter_out_owner(paginated: svc.Paginated[svc.RBACRole]) -> svc.Paginated[svc.RBACRole]:
|
||||
filtered = [r for r in paginated.data if r.name not in {"所有者", "owner"}]
|
||||
return svc.Paginated[svc.RBACRole](
|
||||
data=filtered,
|
||||
pagination=paginated.pagination,
|
||||
)
|
||||
|
||||
|
||||
def _legacy_workspace_roles(options: svc.ListOption | None = None) -> svc.Paginated[svc.RBACRole]:
|
||||
"""Return the built-in legacy workspace roles in the RBAC list shape.
|
||||
|
||||
This keeps the new `/rbac/roles` endpoint compatible with the original
|
||||
Dify role model when enterprise RBAC is disabled.
|
||||
"""
|
||||
|
||||
legacy_roles = [
|
||||
svc.RBACRole(
|
||||
id=role_name,
|
||||
tenant_id="",
|
||||
type=svc.RBACRoleType.WORKSPACE.value,
|
||||
category="global_system_default",
|
||||
name=role_name,
|
||||
description="",
|
||||
is_builtin=True,
|
||||
permission_keys=list(_LEGACY_ROLE_PERMISSION_KEYS[role_name]),
|
||||
role_tag="owner" if role_name == "owner" else "",
|
||||
)
|
||||
for role_name in ("owner", "admin", "editor", "normal", "dataset_operator")
|
||||
]
|
||||
|
||||
page_number = options.page_number if options and options.page_number is not None else 1
|
||||
results_per_page = options.results_per_page if options and options.results_per_page is not None else len(legacy_roles)
|
||||
reverse = options.reverse if options and options.reverse is not None else False
|
||||
|
||||
ordered_roles = list(reversed(legacy_roles)) if reverse else legacy_roles
|
||||
start = max(page_number - 1, 0) * results_per_page
|
||||
end = start + results_per_page
|
||||
paged_roles = ordered_roles[start:end]
|
||||
total_count = len(legacy_roles)
|
||||
total_pages = (total_count + results_per_page - 1) // results_per_page if results_per_page > 0 else 0
|
||||
|
||||
return svc.Paginated[svc.RBACRole](
|
||||
data=paged_roles,
|
||||
pagination=svc.Pagination(
|
||||
total_count=total_count,
|
||||
per_page=results_per_page,
|
||||
current_page=page_number,
|
||||
total_pages=total_pages,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Permission catalogs.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/role-permissions/catalog")
|
||||
class RBACWorkspaceCatalogApi(Resource):
|
||||
@login_required
|
||||
def get(self):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(svc.RBACService.Catalog.workspace(tenant_id, account_id))
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/role-permissions/catalog/app")
|
||||
class RBACAppCatalogApi(Resource):
|
||||
@login_required
|
||||
def get(self):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(svc.RBACService.Catalog.app(tenant_id, account_id))
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/role-permissions/catalog/dataset")
|
||||
class RBACDatasetCatalogApi(Resource):
|
||||
@login_required
|
||||
def get(self):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(svc.RBACService.Catalog.dataset(tenant_id, account_id))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Roles.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class _RoleUpsertRequest(BaseModel):
|
||||
"""Accepts the payload sent by the Create/Edit Role dialog."""
|
||||
|
||||
name: str
|
||||
description: str = ""
|
||||
permission_keys: list[str] = []
|
||||
|
||||
def to_mutation(self) -> svc.RoleMutation:
|
||||
return svc.RoleMutation(
|
||||
name=self.name,
|
||||
description=self.description,
|
||||
permission_keys=list(self.permission_keys),
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/roles")
|
||||
class RBACRolesApi(Resource):
|
||||
@login_required
|
||||
def get(self):
|
||||
tenant_id, account_id = _current_ids()
|
||||
query = _RolesListQuery.model_validate(request.args.to_dict(flat=True))
|
||||
options = query.to_inner_options()
|
||||
if not dify_config.RBAC_ENABLED:
|
||||
result = _legacy_workspace_roles(options)
|
||||
else:
|
||||
result = svc.RBACService.Roles.list(tenant_id, account_id, options=options)
|
||||
if query.include_owner == 0:
|
||||
result = _filter_out_owner(result)
|
||||
|
||||
data = []
|
||||
for role in result.data:
|
||||
if role.name in {"所有者", "owner"}:
|
||||
role.role_tag = "owner"
|
||||
else:
|
||||
role.role_tag = ""
|
||||
data.append(role)
|
||||
result.data = data
|
||||
return _dump(result)
|
||||
|
||||
@login_required
|
||||
def post(self):
|
||||
tenant_id, account_id = _current_ids()
|
||||
request = _payload(_RoleUpsertRequest)
|
||||
role = svc.RBACService.Roles.create(tenant_id, account_id, request.to_mutation())
|
||||
return _dump(role), 201
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/roles/<uuid:role_id>")
|
||||
class RBACRoleItemApi(Resource):
|
||||
@login_required
|
||||
def get(self, role_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(svc.RBACService.Roles.get(tenant_id, account_id, str(role_id)))
|
||||
|
||||
@login_required
|
||||
def put(self, role_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
request = _payload(_RoleUpsertRequest)
|
||||
role = svc.RBACService.Roles.update(tenant_id, account_id, str(role_id), request.to_mutation())
|
||||
return _dump(role)
|
||||
|
||||
@login_required
|
||||
def delete(self, role_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
svc.RBACService.Roles.delete(tenant_id, account_id, str(role_id))
|
||||
return {"result": "success"}
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/roles/<uuid:role_id>/copy")
|
||||
class RBACRoleCopyApi(Resource):
|
||||
@login_required
|
||||
def post(self, role_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
role = svc.RBACService.Roles.copy(tenant_id, account_id, str(role_id))
|
||||
return _dump(role), 201
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Access policies (tenant-level permission sets).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class _AccessPolicyCreateRequest(BaseModel):
|
||||
name: str
|
||||
resource_type: svc.RBACResourceType
|
||||
description: str = ""
|
||||
permission_keys: list[str] = []
|
||||
|
||||
|
||||
class _AccessPolicyUpdateRequest(BaseModel):
|
||||
name: str
|
||||
description: str = ""
|
||||
permission_keys: list[str] = []
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/access-policies")
|
||||
class RBACAccessPoliciesApi(Resource):
|
||||
@login_required
|
||||
def get(self):
|
||||
tenant_id, account_id = _current_ids()
|
||||
# `resource_type` is exposed as a query argument so the UI can show
|
||||
# only app-scoped or only dataset-scoped permission sets.
|
||||
resource_type = request.args.get("resource_type") or None
|
||||
return _dump(
|
||||
svc.RBACService.AccessPolicies.list(
|
||||
tenant_id,
|
||||
account_id,
|
||||
resource_type=resource_type,
|
||||
options=_pagination_options(),
|
||||
)
|
||||
)
|
||||
|
||||
@login_required
|
||||
def post(self):
|
||||
tenant_id, account_id = _current_ids()
|
||||
request = _payload(_AccessPolicyCreateRequest)
|
||||
policy = svc.RBACService.AccessPolicies.create(
|
||||
tenant_id,
|
||||
account_id,
|
||||
svc.AccessPolicyCreate(
|
||||
name=request.name,
|
||||
resource_type=request.resource_type,
|
||||
description=request.description,
|
||||
permission_keys=list(request.permission_keys),
|
||||
),
|
||||
)
|
||||
return _dump(policy), 201
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/access-policies/<uuid:policy_id>")
|
||||
class RBACAccessPolicyItemApi(Resource):
|
||||
@login_required
|
||||
def get(self, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(svc.RBACService.AccessPolicies.get(tenant_id, account_id, str(policy_id)))
|
||||
|
||||
@login_required
|
||||
def put(self, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
request = _payload(_AccessPolicyUpdateRequest)
|
||||
policy = svc.RBACService.AccessPolicies.update(
|
||||
tenant_id,
|
||||
account_id,
|
||||
str(policy_id),
|
||||
svc.AccessPolicyUpdate(
|
||||
name=request.name,
|
||||
description=request.description,
|
||||
permission_keys=list(request.permission_keys),
|
||||
),
|
||||
)
|
||||
return _dump(policy)
|
||||
|
||||
@login_required
|
||||
def delete(self, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
svc.RBACService.AccessPolicies.delete(tenant_id, account_id, str(policy_id))
|
||||
return {"result": "success"}
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/access-policies/<uuid:policy_id>/copy")
|
||||
class RBACAccessPolicyCopyApi(Resource):
|
||||
@login_required
|
||||
def post(self, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
policy = svc.RBACService.AccessPolicies.copy(tenant_id, account_id, str(policy_id))
|
||||
return _dump(policy), 201
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Per-app access (App Access Config).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class _ReplaceBindingsRequest(BaseModel):
|
||||
role_ids: list[str] = []
|
||||
account_ids: list[str] = []
|
||||
|
||||
@field_validator("role_ids", "account_ids", mode="before")
|
||||
@classmethod
|
||||
def _coerce_bindings(cls, value: Any) -> list[str]:
|
||||
if value is None:
|
||||
return []
|
||||
return value
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/my-permissions")
|
||||
class RBACMyPermissionsApi(Resource):
|
||||
@login_required
|
||||
def get(self):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(
|
||||
svc.RBACService.MyPermissions.get(
|
||||
tenant_id,
|
||||
account_id,
|
||||
app_id=request.args.get("app_id") or None,
|
||||
dataset_id=request.args.get("dataset_id") or None,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/apps/<uuid:app_id>/access-policy")
|
||||
class RBACAppMatrixApi(Resource):
|
||||
@login_required
|
||||
def get(self, app_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(svc.RBACService.AppAccess.matrix(tenant_id, account_id, str(app_id)))
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/apps/<uuid:app_id>/access-policies/<uuid:policy_id>/role-bindings")
|
||||
class RBACAppRoleBindingsApi(Resource):
|
||||
@login_required
|
||||
def get(self, app_id, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(
|
||||
svc.RBACService.AppAccess.list_role_bindings(tenant_id, account_id, str(app_id), str(policy_id))
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/apps/<uuid:app_id>/access-policies/<uuid:policy_id>/member-bindings")
|
||||
class RBACAppMemberBindingsApi(Resource):
|
||||
@login_required
|
||||
def get(self, app_id, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(
|
||||
svc.RBACService.AppAccess.list_member_bindings(tenant_id, account_id, str(app_id), str(policy_id))
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/apps/<uuid:app_id>/access-policies/<uuid:policy_id>/bindings")
|
||||
class RBACAppBindingsApi(Resource):
|
||||
@login_required
|
||||
def put(self, app_id, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
request = _payload(_ReplaceBindingsRequest)
|
||||
return _dump(
|
||||
svc.RBACService.AppAccess.replace_bindings(
|
||||
tenant_id,
|
||||
account_id,
|
||||
str(app_id),
|
||||
str(policy_id),
|
||||
svc.ReplaceBindings(role_ids=list(request.role_ids), account_ids=list(request.account_ids)),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Per-dataset access (Knowledge Base Access Config).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/datasets/<uuid:dataset_id>/access-policy")
|
||||
class RBACDatasetMatrixApi(Resource):
|
||||
@login_required
|
||||
def get(self, dataset_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(svc.RBACService.DatasetAccess.matrix(tenant_id, account_id, str(dataset_id)))
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/datasets/<uuid:dataset_id>/access-policies/<uuid:policy_id>/role-bindings")
|
||||
class RBACDatasetRoleBindingsApi(Resource):
|
||||
@login_required
|
||||
def get(self, dataset_id, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(
|
||||
svc.RBACService.DatasetAccess.list_role_bindings(
|
||||
tenant_id, account_id, str(dataset_id), str(policy_id)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/datasets/<uuid:dataset_id>/access-policies/<uuid:policy_id>/bindings")
|
||||
class RBACDatasetBindingsApi(Resource):
|
||||
@login_required
|
||||
def put(self, dataset_id, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
request = _payload(_ReplaceBindingsRequest)
|
||||
return _dump(
|
||||
svc.RBACService.DatasetAccess.replace_bindings(
|
||||
tenant_id,
|
||||
account_id,
|
||||
str(dataset_id),
|
||||
str(policy_id),
|
||||
svc.ReplaceBindings(role_ids=list(request.role_ids), account_ids=list(request.account_ids)),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route(
|
||||
"/workspaces/current/rbac/datasets/<uuid:dataset_id>/access-policies/<uuid:policy_id>/member-bindings"
|
||||
)
|
||||
class RBACDatasetMemberBindingsApi(Resource):
|
||||
@login_required
|
||||
def get(self, dataset_id, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(
|
||||
svc.RBACService.DatasetAccess.list_member_bindings(
|
||||
tenant_id, account_id, str(dataset_id), str(policy_id)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Workspace-level access (Settings > Access Rules).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/workspace/apps/access-policy")
|
||||
class RBACWorkspaceAppMatrixApi(Resource):
|
||||
@login_required
|
||||
def get(self):
|
||||
tenant_id, account_id = _current_ids()
|
||||
options = _pagination_options()
|
||||
return _dump(svc.RBACService.WorkspaceAccess.app_matrix(tenant_id, account_id, options=options))
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/workspace/apps/access-policies/<uuid:policy_id>/role-bindings")
|
||||
class RBACWorkspaceAppRoleBindingsApi(Resource):
|
||||
@login_required
|
||||
def get(self, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(
|
||||
svc.RBACService.WorkspaceAccess.list_app_role_bindings(tenant_id, account_id, str(policy_id))
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/workspace/apps/access-policies/<uuid:policy_id>/bindings")
|
||||
class RBACWorkspaceAppBindingsApi(Resource):
|
||||
@login_required
|
||||
def put(self, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
request = _payload(_ReplaceBindingsRequest)
|
||||
return _dump(
|
||||
svc.RBACService.WorkspaceAccess.replace_app_bindings(
|
||||
tenant_id,
|
||||
account_id,
|
||||
str(policy_id),
|
||||
svc.ReplaceBindings(role_ids=list(request.role_ids), account_ids=list(request.account_ids)),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/workspace/apps/access-policies/<uuid:policy_id>/member-bindings")
|
||||
class RBACWorkspaceAppMemberBindingsApi(Resource):
|
||||
@login_required
|
||||
def get(self, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(
|
||||
svc.RBACService.WorkspaceAccess.list_app_member_bindings(tenant_id, account_id, str(policy_id))
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/workspace/datasets/access-policy")
|
||||
class RBACWorkspaceDatasetMatrixApi(Resource):
|
||||
@login_required
|
||||
def get(self):
|
||||
tenant_id, account_id = _current_ids()
|
||||
options = _pagination_options()
|
||||
return _dump(svc.RBACService.WorkspaceAccess.dataset_matrix(tenant_id, account_id, options=options))
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/workspace/datasets/access-policies/<uuid:policy_id>/role-bindings")
|
||||
class RBACWorkspaceDatasetRoleBindingsApi(Resource):
|
||||
@login_required
|
||||
def get(self, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(
|
||||
svc.RBACService.WorkspaceAccess.list_dataset_role_bindings(tenant_id, account_id, str(policy_id))
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/workspace/datasets/access-policies/<uuid:policy_id>/bindings")
|
||||
class RBACWorkspaceDatasetBindingsApi(Resource):
|
||||
@login_required
|
||||
def put(self, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
request = _payload(_ReplaceBindingsRequest)
|
||||
return _dump(
|
||||
svc.RBACService.WorkspaceAccess.replace_dataset_bindings(
|
||||
tenant_id,
|
||||
account_id,
|
||||
str(policy_id),
|
||||
svc.ReplaceBindings(role_ids=list(request.role_ids), account_ids=list(request.account_ids)),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/workspace/datasets/access-policies/<uuid:policy_id>/member-bindings")
|
||||
class RBACWorkspaceDatasetMemberBindingsApi(Resource):
|
||||
@login_required
|
||||
def get(self, policy_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(
|
||||
svc.RBACService.WorkspaceAccess.list_dataset_member_bindings(tenant_id, account_id, str(policy_id))
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Member ↔ role bindings (Settings > Members > Assign roles).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class _ReplaceMemberRolesRequest(BaseModel):
|
||||
role_ids: list[str] = []
|
||||
|
||||
@field_validator("role_ids", mode="before")
|
||||
@classmethod
|
||||
def _coerce_role_ids(cls, value: Any) -> list[str]:
|
||||
if value is None:
|
||||
return []
|
||||
return value
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/rbac/members/<uuid:member_id>/rbac-roles")
|
||||
class RBACMemberRolesApi(Resource):
|
||||
@login_required
|
||||
def get(self, member_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
return _dump(svc.RBACService.MemberRoles.get(tenant_id, account_id, str(member_id)))
|
||||
|
||||
@login_required
|
||||
def put(self, member_id):
|
||||
tenant_id, account_id = _current_ids()
|
||||
request = _payload(_ReplaceMemberRolesRequest)
|
||||
return _dump(
|
||||
svc.RBACService.MemberRoles.replace(
|
||||
tenant_id,
|
||||
account_id,
|
||||
str(member_id),
|
||||
role_ids=list(request.role_ids),
|
||||
)
|
||||
)
|
||||
@ -10,8 +10,7 @@ from sqlalchemy.orm import sessionmaker
|
||||
from werkzeug.exceptions import Forbidden
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import (
|
||||
account_initialization_required,
|
||||
@ -253,7 +252,6 @@ register_schema_models(
|
||||
MCPProviderDeletePayload,
|
||||
MCPAuthPayload,
|
||||
)
|
||||
register_response_schema_models(console_ns, SimpleResultResponse)
|
||||
|
||||
|
||||
@console_ns.route("/workspaces/current/tool-providers")
|
||||
@ -876,7 +874,6 @@ class ToolBuiltinProviderSetDefaultApi(Resource):
|
||||
@console_ns.expect(console_ns.models[BuiltinProviderDefaultCredentialPayload.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@account_initialization_required
|
||||
def post(self, provider):
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
@ -1057,7 +1054,6 @@ class ToolProviderMCPApi(Resource):
|
||||
return {"result": "success"}
|
||||
|
||||
@console_ns.expect(console_ns.models[MCPProviderDeletePayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user