youngkingdom/ragflow
1
0
Fork 0
mirror of https://github.com/infiniflow/ragflow.git synced 2026-05-28 03:33:05 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,339 Commits 2 Branches 48 Tags
b2bf9155edf1bc5b4b42e1013401a7d828b2d6f7
Commit Graph

122 Commits

Author SHA1 Message Date
Jake Armstrong
b2bf9155ed Go: implement ASR in ZhipuAI driver (#15134) 
### What problem does this PR solve?

This PR implements ASR and TTS support for the ZhipuAI Go driver.

The ZhipuAI model config already advertises `glm-asr-2512` as an ASR
model, but the Go driver returned `zhipu, no such method` from
`TranscribeAudio`. This adds the documented audio transcription endpoint
suffix and sends multipart transcription requests with `model`,
`stream=false`, and `file` fields.

Per maintainer review, this also adds the ZhipuAI TTS endpoint suffix
and implements `AudioSpeech` / `AudioSpeechWithSender` for `glm-tts`.

Closes #15133

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
 2026-05-22 11:53:18 +08:00
ghost
b2053cc3c7 feat(go-models): add PPIO provider driver (#15099) 
### What problem does this PR solve?

Closes #15089.

Adds PPIO support to the Go model-provider layer so PPIO instances can
be routed through the Go API server with the same OpenAI-compatible
chat, streaming, model listing, and connection-check flow used by other
SaaS providers.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

## Summary

- Added a PPIO Go model driver.
- Added the PPIO provider catalog and default OpenAI-compatible API URL.
- Registered PPIO in the model factory.
- Added focused provider and provider-manager tests.

## What changed

- Implemented chat completions, SSE streaming, ListModels, and
CheckConnection for PPIO.
- Covered request shape, stream termination, reasoning fallback, model
listing, custom base URLs, safe transport setup, unsupported methods,
and provider config loading.
- Kept the provider catalog aligned with the existing RAGFlow PPIO
factory model set.
- Cleaned up pre-existing Go model package validation blockers so the
scoped provider tests can run normally with vet enabled.

## Why

The existing Python/provider catalog path includes PPIO, but the Go
model-provider layer did not have a PPIO driver, so the Go API server
could not instantiate or use PPIO as requested in #15089.
 2026-05-22 11:52:18 +08:00
Haruko386
1ece1c81da Go: implement rerank, asr, tts for TogetherAI (#15107) 
### What problem does this PR solve?

implement rerank, asr, tts for TogetherAI

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2026-05-21 20:57:04 +08:00
Haruko386
a725e114f9 Go: implement ASR and TTS for Xinference (#15096) 
### What problem does this PR solve?

implement ASR and TTS for Xinference

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
 2026-05-21 18:28:06 +08:00
tmimmanuel
38a8bc3dab fix(upstage): extract reasoning delta from streaming responses (#14817) 
### What problem does this PR solve?

`UpstageModel.ChatStreamlyWithSender` (in the driver merged via #14819)
only extracted `delta.content` from each SSE event. For the `solar-pro3`
reasoning family (and any future Upstage model that follows the same
wire shape), the chain-of-thought is streamed in a **separate
`delta.reasoning` field**, and the driver was silently dropping all of
it.

The non-streaming path already extracts `message.reasoning` into
`ChatResponse.ReasonContent` (added earlier in this PR's history), so
the same model produced **inconsistent behavior** between streaming and
non-streaming: a tenant calling `solar-pro3` with `reasoning_effort:
high` would see the reasoning trace if they used `ChatWithMessages` but
not if they used `ChatStreamlyWithSender`.

### Live evidence

Probed against `api.upstage.ai/v1/chat/completions` with `solar-pro3` +
`reasoning_effort: high` + `stream: true` (8000-token budget so the
reasoning has room to finish):

```
$ curl -sN -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
       -X POST https://api.upstage.ai/v1/chat/completions \
       -d '{"model":"solar-pro3","messages":[{"role":"user","content":"Compute 15% of 80."}],
            "max_tokens":8000,"stream":true,"reasoning_effort":"high"}'

# across 168 SSE events:
#   delta keys seen: [content reasoning role]
#   delta.content total len:   121 chars   (the visible answer)
#   delta.reasoning total len: 159 chars   (the chain-of-thought) <- driver dropped this
```

A representative event showing both fields side by side:

```json
data: {"choices":[{"index":0,"delta":{"reasoning":"15% = 0.15."}}]}
data: {"choices":[{"index":0,"delta":{"content":"15% of 80 is "}}]}
```

The 159 chars of reasoning were arriving on the wire and being thrown
away. `solar-pro2` was also probed (625 events); it does **not** emit
`delta.reasoning` — its reasoning is inlined into `delta.content` — so
this change is a no-op for it and for `solar-mini`.

### What this PR includes

- `internal/entity/models/upstage.go`: in the SSE scanner loop, extract
`delta.reasoning` before `delta.content` and forward each non-empty
chunk via the sender's second arg (the existing `reasonContent` channel
the non-stream path already populates).

The ordering contract is documented inline: reasoning chunks within a
single SSE event are emitted before content chunks, so a UI that pipes
both sees the chain-of-thought start before the answer for that token,
matching the wire order Upstage emits.

- `internal/entity/models/upstage_test.go`: three new tests pinning the
new behavior:
- `TestUpstageStreamExtractsReasoningDelta` — reasoning + content
forwarded to the right sender args; one-of invariant per call
- `TestUpstageStreamReasoningChunksArriveBeforeContent` — ordering
pinned within a single SSE event that carries both fields
- `TestUpstageStreamWithoutReasoningStillWorks` — regression net:
non-reasoning models (`solar-mini`, `solar-pro2`) continue to work; the
reason callback never fires

No interface change. No factory change. No config change.

### How was this tested?

```
$ go test -vet=off -run TestUpstage -count=1 -v ./internal/entity/models/...
... (existing tests 1..9 still pass) ...
=== RUN   TestUpstageStreamExtractsReasoningDelta
--- PASS: TestUpstageStreamExtractsReasoningDelta (0.01s)
=== RUN   TestUpstageStreamReasoningChunksArriveBeforeContent
--- PASS: TestUpstageStreamReasoningChunksArriveBeforeContent (0.01s)
=== RUN   TestUpstageStreamWithoutReasoningStillWorks
--- PASS: TestUpstageStreamWithoutReasoningStillWorks (0.00s)
PASS
ok      ragflow/internal/entity/models  0.034s
```

12/12 Upstage tests pass on go 1.25. `go build
./internal/entity/models/...` exits 0.

**Live integration test** (smoke test not committed) — the patched
driver was run directly against `api.upstage.ai/v1` with the same prompt
that produced the curl evidence above:

```
=== RUN   TestUpstageStreamReasoningLiveSmoke
    [OK] visible content: 50 chunks, 84 chars
    [OK] reasoning:       39 chunks, 90 chars
    content head 200:   "\\(15\\% = \\frac{15}{100}=0.15\\).\n\n\\[\n0.15 \\times 80 = 12.\n\\]\n\n**15 % of 80 is 12.**"
    reasoning head 200: "We need to compute 15% of 80. That's 0.15 * 80 = 12. So answer is 12. Provide explanation."

UPSTAGE STREAM REASONING SMOKE PASSED
--- PASS: TestUpstageStreamReasoningLiveSmoke (1.97s)
```

Before this fix, the same call would have produced **0 reasoning
chunks**. The 90 chars of reasoning that the patched driver now surfaces
are the chain-of-thought solar-pro3 emits when reasoning_effort is high.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2026-05-21 15:33:21 +08:00
tmimmanuel
85d0b46d8e fix(mistral): handle structured content from magistral reasoning models (#14805) 
### What problem does this PR solve?

`MistralModel.ChatWithMessages` (in the driver merged via #14807)
assumes that `choices[0].message.content` from `/v1/chat/completions` is
always a string and falls through to `return nil, fmt.Errorf("invalid
content format")` on anything else.

That assumption breaks for the **magistral reasoning family**
(`magistral-small-*`, `magistral-medium-*`). When the model needs a
chain-of-thought to answer, Mistral returns `content` as a **structured
array of typed parts**:

```json
"content": [
  {"type": "thinking",
   "thinking": [{"type": "text", "text": "Combined speed is 150 mph. 300 / 150 = 2 hours."}],
   "closed": true},
  {"type": "text", "text": "They will meet after **2 hours**."}
]
```

Concretely, this is what the live API returns today (probed against
`api.mistral.ai/v1`):

```
$ curl -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
       -X POST https://api.mistral.ai/v1/chat/completions \
       -d '{"model":"magistral-medium-latest",
            "messages":[{"role":"user","content":"two trains 60mph and 90mph, 300mi apart, when do they meet? step by step."}],
            "max_tokens":1024}'
HTTP 200
{ "choices":[{"message":{
    "role":"assistant",
    "content":[
      {"type":"thinking","thinking":[{"type":"text","text":"Okay, let's see..."}],"closed":true},
      {"type":"text","text":"To determine when the two trains meet..."}
    ]}}] }
```

With the current driver, every call like that returns the generic
`"invalid content format"` error. Trivial prompts that happen to fit in
a string answer still succeed, so the breakage is **non-deterministic
from the tenant's POV**: same model, same provider, sometimes works,
sometimes 500s with no useful error.

A secondary issue: `conf/models/mistral.json` does not include any
magistral model. The picker hid the broken path, which is why this
wasn't caught during #14807's review.

### What this PR includes

- New helper `extractMistralContent(raw interface{}) (answer,
reasonContent string, err error)` in
`internal/entity/models/mistral.go`, which normalizes both shapes
Mistral can return:
- `string` → historical path. `Answer = content`, `ReasonContent = ""`.
Preserves behavior for every non-reasoning model (`mistral-large-*`,
`mistral-small-*`, `ministral-*`, `codestral-*`, `pixtral-*`,
`open-mistral-nemo`).
- `[]interface{}` → walk the parts. Concatenate every `{"type":"text",
"text":...}` part into `Answer`; concatenate the inner text inside every
`{"type":"thinking", "thinking":[...]}` part into `ReasonContent`.
- `ChatWithMessages` now calls the helper instead of doing the raw
`.(string)` cast.
- Unknown part types are **skipped, not failed**. Mistral has been
adding new content variants quickly (audio chunks, citations, etc.);
this driver should not 500 every call when a new part type appears.
- `conf/models/mistral.json`: add `magistral-medium-latest` and
`magistral-small-latest`. Both are visible in `/v1/models` today.

No interface change. No factory change. No new dependencies.

### How was this tested?

**Unit tests** — 5 new tests in `internal/entity/models/mistral_test.go`
on top of the 27 already shipped via #14807:

- `TestMistralChatHandlesStringContent` — regression net for the
historical path
- `TestMistralChatExtractsReasoningFromStructuredContent` — the fixture
body is a trimmed copy of the actual `magistral-medium-latest` response
captured above; asserts both `Answer` and `ReasonContent` are populated
correctly
- `TestMistralChatHandlesStructuredContentWithoutThinking` —
`magistral-*` with a trivial answer returns a structured shape that has
only a `text` part; `ReasonContent` must stay empty
- `TestMistralChatIgnoresUnknownContentPartTypes` — `audio_url` and
`future_part_type` parts are skipped, `text` parts still flow through
- `TestExtractMistralContent` — table-driven unit coverage of the helper
for string, empty string, nil, empty array, text-only, thinking+text,
unsupported root type

```
$ go test -vet=off -run "TestMistral|TestExtractMistralContent" -count=1 -v ./internal/entity/models/...
=== RUN   TestMistralChatHandlesStringContent
--- PASS: TestMistralChatHandlesStringContent (0.00s)
=== RUN   TestMistralChatExtractsReasoningFromStructuredContent
--- PASS: TestMistralChatExtractsReasoningFromStructuredContent (0.00s)
=== RUN   TestMistralChatHandlesStructuredContentWithoutThinking
--- PASS: TestMistralChatHandlesStructuredContentWithoutThinking (0.00s)
=== RUN   TestMistralChatIgnoresUnknownContentPartTypes
--- PASS: TestMistralChatIgnoresUnknownContentPartTypes (0.00s)
=== RUN   TestExtractMistralContent
=== RUN   TestExtractMistralContent/plain_string
=== RUN   TestExtractMistralContent/empty_string
=== RUN   TestExtractMistralContent/nil
=== RUN   TestExtractMistralContent/empty_array
=== RUN   TestExtractMistralContent/text_only
=== RUN   TestExtractMistralContent/thinking_then_text
=== RUN   TestExtractMistralContent/unknown_root_type
--- PASS: TestExtractMistralContent (0.00s)
PASS
ok      ragflow/internal/entity/models  0.046s
```

All 32 Mistral tests pass on go 1.25. `go build
./internal/entity/models/...` exits 0.

**Live integration test** — driver exercised against `api.mistral.ai/v1`
with the patched code:

```
=== RUN   TestMistralMagistralSmoke
    [OK] "magistral-small-latest" present upstream
    [OK] "magistral-medium-latest" present upstream
    [OK trivial]    Answer="7"  ReasonContent=""
    [OK reasoning]  Answer len=797   head="To determine when the two trains meet, we can follow these steps:\n\n1. **Identify..."
                    ReasonContent len=1069 head="Okay, let's see. There are two trains, one going 60 mph and the other going 90 mph. They're moving towards each other, s..."

MAGISTRAL SMOKE PASSED
--- PASS: TestMistralMagistralSmoke (18.09s)
PASS
ok      ragflow/internal/entity/models  18.112s
```

What the live run proves on the wire:

- `magistral-small-latest` with a trivial prompt still uses the
string-content shape; the regression-net path is exercised against the
real server, not just the mock.
- `magistral-medium-latest` with a reasoning prompt uses the
structured-array shape; the new code path extracts a 1069-character
reasoning trace into `ChatResponse.ReasonContent` and a 797-character
visible answer into `ChatResponse.Answer`. Before this fix, the same
call returned `"invalid content format"` and the caller saw nothing.

The smoke-test file itself is not committed (live tests live outside the
PR diff, same convention used for prior provider PRs).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2026-05-21 15:33:14 +08:00
BitToby
bd4ce39038 Go: implement provider: Perplexity (#15008) 
## What
- Add Perplexity as a chat and embedding provider backed by its
OpenAI-compatible `/chat/completions` and `/v1/embeddings` APIs
- Register Perplexity in the Go model factory and provider config
- Support non-streaming chat, SSE streaming chat, embeddings, model
listing, and connection checks

Refs #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-21 15:33:02 +08:00
dripsmvcp
d5ba14a128 feat(go): implement provider Astraflow (#15062) (#15064) 
- Adds an `Astraflow` Go driver so the new API server can route
Astraflow (UCloud ModelVerse) chat instances, matching the existing
Python `AstraflowChat` (`rag/llm/chat_model.py:1237`). Follows the same
SaaS-driver shape used for Avian, Novita, TogetherAI, Replicate,
DeepInfra, Upstage, and LongCat.

Closes #15062

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-21 15:32:56 +08:00
dripsmvcp
5a18df0fd0 Go: implement provider: Avian (#15045) 
Closes #15044.

Avian was listed unchecked in the Go-rewrite tracker #14736 and already
had an llm_factories.json entry with 4 preconfigured chat models
(deepseek-v3.2, kimi-k2.5, glm-5, minimax-m2.5), but the Go API server
had no driver to route them. The Python side has supported Avian at
rag/llm/chat_model.py:1220 (AvianChat) via the LiteLLM openai/ provider
with default base https://api.avian.io/v1.

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-21 15:32:49 +08:00
sxxtony
7740ec6c95 Go: implement Embed (embeddings) in Replicate driver (#15073) 
### What problem does this PR solve?

`ReplicateModel.Embed` in `internal/entity/models/replicate.go` was a
`"replicate, no such method"` stub. Tracking issue #14736 lists
Replicate's embedding surface as not implemented. This PR wires it up
against Replicate's documented embedding schema.

Until this PR, a tenant who selected a Replicate embedding model got the
sentinel error on every embed call.

Co-authored-by: sxxtony <sxxtony@users.noreply.github.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-21 15:32:41 +08:00
web-dev0521
2d3a1a4483 feat(go-models): add Azure OpenAI model driver (#15022) 
## What problem does this PR solve?

Closes #15021.

The Go model-provider layer had no support for **Azure OpenAI**. Azure
OpenAI is *not* a drop-in base-URL swap of the OpenAI driver — it
differs in authentication, endpoint structure, and how models are listed
— so it needs its own `ModelDriver` implementation.

## Type of change

- [x] New feature (non-breaking change which adds functionality)

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-21 11:52:56 +08:00
Renzo
c7ac9b7171 Go: implement provider: GPUStack (chat) (#15024) 
### What problem does this PR solve?

Fixes #15023

GPUStack is listed as unchecked in the Go-rewrite tracker #14736, and
`internal/service/llm.go:171` already classifies it as a self-deployed
provider alongside Ollama, Xinference, LocalAI, and LM Studio — but
`internal/entity/models/` had no `gpustack.go` driver, so the new Go API
server could not route GPUStack instances. This PR adds the chat surface
for GPUStack so it lines up with the existing self-hosted Go drivers.

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-21 11:49:18 +08:00
Renzo
394cd5d116 Go: implement Embed in Xinference driver (#14932) 
## Summary

- Replaces the `"no such method"` stub on `XinferenceModel.Embed`
(`internal/entity/models/xinference.go`) with a real implementation
against Xinference's OpenAI-compatible `/v1/embeddings` endpoint.
- Adds the `"embedding": "v1/embeddings"` URL suffix to
`conf/models/xinference.json`.
- Mirrors the Python `XinferenceEmbed` class in
`rag/llm/embedding_model.py:407` for payload shape (OpenAI-compatible
`model + input` → `data[*].index + data[*].embedding`) and tolerates the
same no-auth default Xinference deployments use. Authorization is only
sent when a non-empty API key is configured, via the existing
`setXinferenceAuth` helper.
- Reuses the existing `normalizeXinferenceBaseURL` + `baseURLForRegion`
helpers so both `http://127.0.0.1:9997` and `http://127.0.0.1:9997/v1`
resolve to the same `/v1/embeddings` target without doubled `/v1`.
- Validates response indices — duplicate, missing, or out-of-range
`data[*].index` values fail with a clear error rather than silently
producing misaligned vectors.
- Returns `[]EmbeddingData` in original input order (placed by `Index`)
so downstream callers can index positionally without re-sorting.
- Forwards `EmbeddingConfig.Dimension` as `dimensions` when `> 0`,
matching the OpenAI cluster pattern.

Closes #14810

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-21 11:47:30 +08:00
Renzo
fec0b968e7 Go: implement Rerank in Novita driver (#15014) 
### What problem does this PR solve?

Fixes #15012

The Novita Go driver landed in #14850 and shipped a stub `Rerank` method
that returned `"novita, no such method"`, so Novita could not be used as
a rerank provider in RAGFlow. This PR fills that gap, in the same way
#14895 filled the Embed gap on the same driver.

Novita exposes a public rerank endpoint at `POST
https://api.novita.ai/openai/v1/rerank` that accepts the
Cohere-compatible request shape (`{model, query, documents, top_n}`)
with `Authorization: Bearer <api_key>`. `baai/bge-reranker-v2-m3` is
documented in Novita's model library with a 1024-token limit.
 2026-05-21 10:19:17 +08:00
Renzo
536ed07d27 Go: implement Rerank in Xinference driver (#15032) 
### What problem does this PR solve?

Fixes #14816

The Xinference Go driver landed chat in #14938 and Embed is in review in
#14932, but `Rerank` shipped as a stub that returns `"xinference, no
such method"`. Tenants who launch a rerank model with `--model-type
rerank` on their Xinference instance cannot route it through the Go API
server. This PR fills the gap.

Xinference exposes an OpenAI-compatible REST API. The rerank endpoint is
at `POST <base>/v1/rerank` and accepts the Cohere-shaped body `{model,
query, documents, top_n}`, returning `{results: [{index,
relevance_score}]}` — the same wire shape used by the merged NVIDIA
(#14778), Aliyun (#14676), Gitee (#14656), ZhipuAI (#14608), Novita
(#15014), and LocalAI (#14813) Rerank implementations. Documented in
[Xinference rerank
docs](https://inference.readthedocs.io/en/v1.6.1/models/model_abilities/rerank.html);
the [builtin rerank model
catalog](https://inference.readthedocs.io/en/stable/models/builtin/rerank/)
lists `bge-reranker-base`, `bge-reranker-large`, `bge-reranker-v2-m3`,
and others.
 2026-05-21 10:14:30 +08:00
sxxtony
63db30f0d9 Go: implement provider: n1n.ai (#15010) 
### What problem does this PR solve?

Add a Go driver for **n1n.ai** (https://docs.n1n.ai), one of the
unchecked providers on the umbrella tracking issue #14736. n1n.ai is an
OpenAI-compatible aggregator hosting a 450+ model catalog (GPT, Claude,
Gemini, DeepSeek, Kimi, Qwen, embedding + reranker families) under
`https://api.n1n.ai/v1`.

Until this PR, a tenant who configured `n1n` as a model provider in the
Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver.

---------

Co-authored-by: sxxtony <sxxtony@users.noreply.github.com>
 2026-05-21 10:13:15 +08:00
Jack Storment
dc01e0e51c Go: implement Embed (embeddings) in TogetherAI driver (#15017) 
### What problem does this PR solve?

Fixes #15015

The TogetherAI Go driver in `internal/entity/models/togetherai.go`
shipped a stub `Embed` method that returned `"TogetherAI, no such
method"`, so TogetherAI could not be used as an embedding provider in
RAGFlow. This PR fills that gap.

TogetherAI exposes a public OpenAI-compatible embeddings endpoint at
`POST https://api.together.ai/v1/embeddings` that accepts the standard
`{model, input}` shape with `Authorization: Bearer <api_key>` (confirmed
in TogetherAI's official docs:
https://docs.together.ai/docs/embeddings-overview). Documented embedding
models include `intfloat/multilingual-e5-large-instruct`,
`BAAI/bge-large-en-v1.5`, and `BAAI/bge-base-en-v1.5`.

### Changes

- `internal/entity/models/togetherai.go`: implement
`TogetherAIModel.Embed`.
- Validate inputs (api key, model name) and short-circuit on empty
texts.
  - Resolve region with the existing `baseURLForRegion` helper.
  - Build URL from `URLSuffix.Embedding`.
- Send `{model, input}` POST body, add `dimensions` when
`embeddingConfig.Dimension > 0` (matches the pattern in #14735).
  - Bearer auth + JSON content type, mirroring the chat path.
- Parse `{data: [{embedding, index}]}` and reorder by `index`, rejecting
out-of-range indices, duplicates, and missing entries so the output
always lines up with the input. Same shape as the merged Mistral,
Upstage, and Novita Embed implementations.
- `conf/models/togetherai.json`:
  - Add `"embedding": "embeddings"` to `url_suffix`.
- Add default embedding model entries for
`intfloat/multilingual-e5-large-instruct`, `BAAI/bge-large-en-v1.5`, and
`BAAI/bge-base-en-v1.5`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2026-05-20 20:48:44 +08:00
Haruko386
4a91ca5349 Go: implement provider: MinerU_Local (#15051) 
### What problem does this PR solve?
1. Add model types when add model
---
```
RAGFlow(user)> add model 'pipeline' to provider 'mineru_local' instance 'test' with tokens 131072 doc_parse;
SUCCESS
```

2. implement provider: MinerU_Local
---
**Verified from CLI**
```
RAGFlow(user)> parse with 'pipeline@test@mineru_local' file './internal/test.pdf'
+--------------------------------------+
| task_id                              |
+--------------------------------------+
| c7260e31-b6e2-4b36-955d-e9c60510c669 |
+--------------------------------------+
RAGFlow(user)> show 'test@mineru_local' task 'c7260e31-b6e2-4b36-955d-e9c60510c669'
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| content                                                                                                                                                                                                                                                          | index |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| # Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

Bingxin Ke Anton Obukhov Shengyu Huang Nando Metzger Rodrigo Caye Daudt Konrad Schindler Photogrammetry and Remote Sensing, ETH Zurich ¨

![](images/ae256101419715b544d13722...  | 1     |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2026-05-20 19:21:57 +08:00
Haruko386
2836a934b5 Go: implement provider: 302.AI and JieKou-AI (#15034) 
### What problem does this PR solve?

This PR implement implement provider 302.AI and JieKouAI

**The following functionalities are now supported:**

**302.ai**

- [x] chat / think chat / stream chat / stream think chat
- [x] Embedding
- [x] ASR
- [x] ListModels
- [x] Provider connection checking
- [x] Balance
- [x] Rerank
- [x] OCR
- [x] Doc Parse
- [x] Show task 
- [ ]  ~~List Tasks!~~
- [ ] TTS

**JieKouAI**

- [x] chat / think chat / stream chat / stream think chat
- [x] Embedding
- [x] Rerank
- [x] ListModels

**Verified examples from the CLI:**
```palintext
# jiekouAI

RAGFlow(user)> stream think chat with 'zai-org/glm-4.5@test@jiekouai' message 'Hi'
Thinking: Let me think about how to respond to this simple greeting. The user just said "Hi", which is a basic and friendly way to start a conversation. I should respond in a similarly warm and welcoming manner.First, I need to acknowledge their greeting and reciprocate with enthusiasm. Something like "Hello!" or "Hi there!" would work well to create a positive atmosphere right from the start.Next, I should make it clear that I'm ready to help. Since they haven't asked anything specific yet, I'll keep it open-ended and inviting. Perhaps offering assistance with a question or task would encourage them to engage further.I should also maintain a professional yet approachable tone. Being an AI assistant, I want to convey that I'm knowledgeable and capable, but also friendly and easy to talk to.Let me put this all together into a concise response. I'll start with a cheerful greeting, express my readiness to help, and finish with an open invitation for them to share what's on their mind. This should create a welcoming environment for whatever they want to discuss next.
Answer: ! I'm Claude, an AI assistant created by Anthropic. I'm here to help you with information, answer questions, or assist you with tasks. What can I help you with today?

RAGFlow(user)> think chat with 'zai-org/glm-4.5@test@jiekouai' message 'Hi'
Thinking: Let me consider how to respond to this greeting. The user initiated with a simple "Hi," so a friendly and open response would be most appropriate to encourage further conversation. I should maintain a welcoming tone while offering assistance.

The response should accomplish a few key things: return the greeting warmly, show openness to conversation, and offer specific ways I can help. This approach demonstrates both approachability and usefulness.

I'll start with a greeting in return, then express my availability to help, and finish by suggesting some areas where I can provide assistance. This creates a natural flow from acknowledgment to support.

It's important to keep the response concise but inviting. Since the user hasn't specified their needs yet, I'll present a few broad categories of assistance to spark their thinking about what they might want to discuss or ask about.

The response should end with an encouraging note that prompts them to share what's on their mind, keeping the conversational ball in their court while making it clear I'm ready to engage with whatever they need.
Answer: Hello! How can I help you today? Whether you have questions, need information, or just want to chat, I'm here to assist.

RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'text-embedding-3-large@test@jiekouai' dimension 16
+-----------+-------+
| dimension | index |
+-----------+-------+
| 3072      | 0     |
| 3072      | 1     |
+-----------+-------+

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'baai/bge-reranker-v2-m3@test@jiekouai' top 3
+-------+-----------------+
| index | relevance_score |
+-------+-----------------+
| 0     | 0.9830034       |
| 2     | 0.06399203      |
| 1     | 0.04665664      |
+-------+-----------------+


# 302.ai

RAGFlow(user)> think chat with 'kimi-k2.6@test@302.ai' message 'who r u'
Thinking: The user is asking "who r u" which is a casual way of asking "who are you." I need to identify myself as an AI assistant created by Moonshot AI. I should be friendly, concise, and helpful.

Key points to include:
- I am Kimi, an AI assistant made by Moonshot AI
- I can help with various tasks like answering questions, writing, analysis, coding, etc.
- Keep it casual but informative since the user used "r u" (text speak)

I should not:
- Pretend to be human
- Claim to have personal experiences or emotions
- Be overly formal or robotic

Simple, friendly response is best.
Answer: I'm Kimi, an AI assistant made by Moonshot AI. I can help you with answering questions, writing, coding, analysis, or just chatting. What can I do for you?
Time: 17.687750

RAGFlow(user)> stream think chat with 'kimi-k2.6@test@302.ai' message 'who r u'
Thinking:  user asked "who r u" which is a casual way of asking "who are you." I should introduce myself as Kimi, an AI assistant developed by Moonshot AI. I need to be friendly, concise, and accurate. I should mention my capabilities briefly and keep the tone helpful. Since the user used casual text speak ("r u"), I can match that energy with a friendly but still informative tone.Key points:- I'm Kimi, an AI assistant made by Moonshot AI- I can help with various tasks like answering questions, writing, coding, analysis, etc.- Keep it brief but warm- Don't claim to be human- Don't over-explainDraft:"I'm Kimi, an AI assistant created by Moonshot AI. I can help with answering questions, writing, coding, analysis, brainstorming, and lots of other tasks. What can I do for you?"This is good - direct, accurate, and inviting.
Answer:  Kimi, an AI assistant made by Moonshot AI. I can help with answering questions, writing, coding, analysis, brainstorming, and lots of other stuff. What can I do for you?
Time: 14.912576

RAGFlow(user)> asr with 'whisper-v3-turbo@test@302.ai' audio './internal/test.wav' param ''
+---------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                |
+---------------------------------------------------------------------------------------------------------------------+
| The examination and testimony of the experts enabled the Commission to conclude that five shots may have been fired |
+---------------------------------------------------------------------------------------------------------------------+

RAGFlow(user)> ocr with 'mistral-ocr-latest@test@302.ai' file './internal/test.pdf'
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                                                                                                                                                             |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| # Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

Bingxin Ke

Nando Metzger

Anton Obukhov

Rodrigo Caye Daudt

Shengyu Huang

Konrad Schindler

Photogrammetry and Remote Sensing, ETH Zürich

![img-0.jpeg](img-0.jpeg)
Figur...  |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+


RAGFlow(user)> parse with 'vlm@test@302.ai' file 'https://arxiv.org/pdf/2505.09358'
+--------------------------------------+
| task_id                              |
+--------------------------------------+
| 6de6eae6-c122-4b67-91e8-b061a0b8c087 |
+--------------------------------------+
RAGFlow(user)> show 'test@302.ai' task '6de6eae6-c122-4b67-91e8-b061a0b8c087'
+----------------------------------------------------------------------------+-------+
| content                                                                    | index |
+----------------------------------------------------------------------------+-------+
| https://file.302.ai/gpt/imgs/20260519/b340fdff4774699c287fe4ee4658b317.zip | 0     |
+----------------------------------------------------------------------------+-------+

RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'jina-embeddings-v3@test@302.ai' dimension 16
+-----------+-------+
| dimension | index |
+-----------+-------+
| 1024      | 0     |
| 1024      | 1     |
+-----------+-------+
RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'jina-reranker-v2-base-multilingual@test@302.ai' top 3;
+-------+-----------------+
| index | relevance_score |
+-------+-----------------+
| 0     | 0.74167407      |
| 2     | 0.18832397      |
| 1     | 0.15713684      |
+-------+-----------------+
```


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
 2026-05-20 14:10:15 +08:00
tmimmanuel
243d9ed281 Add TogetherAI chat provider (#14957) 
## What
- Add TogetherAI as a chat provider backed by its OpenAI-compatible
`/v1/chat/completions` API
- Register TogetherAI in the Go model factory and provider config
- Support non-streaming chat, SSE streaming chat, model listing, and
connection checks

## Notes
- Uses the current TogetherAI OpenAI-compatible base URL
`https://api.together.ai/v1`
- Forwards documented chat parameters from `ChatConfig`: `max_tokens`,
`temperature`, `top_p`, `stop`, and GPT-OSS `reasoning_effort`
- Routes Together reasoning traces from `reasoning` /
`reasoning_content` into `ReasonContent`

## Tests
- `go test -vet=off -run TestTogetherAI -count=1
./internal/entity/models`
- `go test -vet=off -count=1 ./internal/entity/models`

Refs #14736
 2026-05-19 15:10:42 +08:00
tmimmanuel
09a06f1b00 Go: implement provider: Xinference (#14938) 
### What problem does this PR solve?

Closes #14808.

Adds a Go model driver for Xinference so self-hosted Xinference chat
models can be used through the Go provider layer instead of falling
through to the dummy driver. Xinference exposes an OpenAI-compatible API
under `/v1`; the driver accepts either a root endpoint such as
`http://127.0.0.1:9997` or an OpenAI-compatible endpoint such as
`http://127.0.0.1:9997/v1` and normalizes it before calling chat or
model-listing routes.

### What is changed?

- Add `internal/entity/models/xinference.go` implementing `ModelDriver`
for Xinference chat.
- Route provider name `xinference` in
`internal/entity/models/factory.go`.
- Add `conf/models/xinference.json` as a local provider config.
- Add focused unit tests in `internal/entity/models/xinference_test.go`.

Initial method coverage:

- `ChatWithMessages`: POST `/v1/chat/completions`.
- `ChatStreamlyWithSender`: SSE streaming from `/v1/chat/completions`.
- `ListModels`: GET `/v1/models`.
- `CheckConnection`: lightweight `ListModels` probe.
- Optional auth: send `Authorization: Bearer <api_key>` only when a
non-empty key is configured, matching Xinference no-auth and
auth-enabled deployments.
- `Balance`, `Embed`, `Rerank`, ASR, TTS, and OCR return `no such
method` for this initial chat-provider PR.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Bug Fix (non-breaking change which fixes an issue)

### Tests

- `go test -vet=off -run TestXinference -count=1
./internal/entity/models/...`
- `go test -vet=off -count=1 ./internal/entity/models/...`

### References

- Xinference docs:
https://inference.readthedocs.io/zh-cn/latest/index.html
- OpenAI-compatible chat usage:
https://inference.readthedocs.io/zh-cn/latest/getting_started/using_xinference.html
- API key auth:
https://inference.readthedocs.io/zh-cn/latest/user_guide/auth_system.html

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-19 15:10:13 +08:00
tmimmanuel
4c9529ef36 Add Replicate chat provider (#14958) 
## What
- Add Replicate as a chat provider backed by the documented predictions
API
- Register Replicate in the Go model factory and provider config
- Support non-streaming chat through sync predictions, polling fallback,
streaming through `urls.stream`, model listing, and connection checks

## Notes
- Uses `POST /v1/predictions` with Replicate model identifiers in
`version`, which supports official and community model identifiers
- Maps RAGFlow messages into Replicate prompt-shaped inputs (`prompt`,
optional `system_prompt`) and forwards common documented LLM inputs:
`max_new_tokens`, `temperature`, `top_p`
- Preserves whitespace in SSE output chunks and emits RAGFlow `[DONE]`
at stream completion

## Tests
- `go test -vet=off -run TestReplicate -count=1
./internal/entity/models`
- `go test -vet=off -count=1 ./internal/entity/models`

Refs #14736
 2026-05-19 11:10:36 +08:00
Haruko386
db9e782747 Go: implement provider: MinerU (#14990) 
### What problem does this PR solve?

Implement MinerU Provider

**The following functionalities are now supported:**

**MinerU**
----
- [x] Parse file
- [x] Show task
- [ ] ~~List tasks~~

**Verified examples from the CLI:**
```plaintext
RAGFlow(user)> parse with 'vlm@test@mineru' file 'https://arxiv.org/pdf/2505.09358'
+--------------------------------------+
| task_id                              |
+--------------------------------------+
| 142ac8ea-d9d0-4a68-a2d1-d3af67635dc9 |
+--------------------------------------+

RAGFlow(user)> show 'test@mineru' task '142ac8ea-d9d0-4a68-a2d1-d3af67635dc9'
+--------------------------------------------+-------+
| content                                    | index |
+--------------------------------------------+-------+
| Task is running... Progress: 17 / 18 pages | 0     |
+--------------------------------------------+-------+

RAGFlow(user)> show 'test@mineru' task '142ac8ea-d9d0-4a68-a2d1-d3af67635dc9'
+--------------------------------------------------------------------------------------------+-------+
| content                                                                                    | index |
+--------------------------------------------------------------------------------------------+-------+
| https://cdn-mineru.openxlab.org.cn/pdf/2026-05-18/142ac8ea-d9d0-4a68-a2d1-d3af67635dc9.zip | 0     |
+--------------------------------------------------------------------------------------------+-------+

```


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
 2026-05-19 10:49:33 +08:00
buua436
d7fb4bdb4e Go: align document list response (#14982) 
### What problem does this PR solve?

align document list response

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2026-05-18 20:00:11 +08:00
Haruko386
92145dc764 Go: implement provider: DeepInfra, XunFei (#14978) 
### What problem does this PR solve?

This PR implement implement provider  and Mistral, DeepInfra, XunFei

**The following functionalities are now supported:**

**DeepInfra**

- [x] chat / think chat / stream chat / stream think chat
- [x] Embedding
- [x] ASR
- [x] TTS
- [x] ListModels
- [x] Provider connection checking
- [x] Balance
- [ ] ~~Rerank~~

**XunFei**

- [x] chat / think chat / stream chat / stream think chat

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
 2026-05-18 16:57:42 +08:00
buua436
b40b0bf996 Go: fix siliconflow embedding response (#14975) 
### What problem does this PR solve?

fix siliconflow embedding response

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2026-05-18 15:07:07 +08:00
tmimmanuel
b09da6e347 Go: implement provider: CometAPI (#14930) 
### What problem does this PR solve?

Adds the Go model provider driver for CometAPI, which is listed as
unchecked in the Go provider tracking issue #14736 and requested in
#14804. Without this, the Go layer falls back to the dummy driver for
the `cometapi` provider.

Fixes #14804

### What this PR includes

- New `internal/entity/models/cometapi.go` implementing `ModelDriver`
for CometAPI.
- New `conf/models/cometapi.json` with CometAPI base URLs and
representative chat / embedding models from the public catalog.
- `factory.go`: route `"cometapi"` to `NewCometAPIModel`.
- Unit tests in `internal/entity/models/cometapi_test.go`.

### Method coverage

- `ChatWithMessages`: `POST /v1/chat/completions`.
- `ChatStreamlyWithSender`: SSE streaming on the same endpoint.
- `Embed`: `POST /v1/embeddings`, including optional `dimensions`.
- `ListModels`: `GET /api/models` public catalog.
- `Balance`: `GET https://query.cometapi.com/user/quota?key=...`.
- `CheckConnection`: delegates to the quota query to verify the key.
- `Rerank`, ASR, TTS, OCR: return `no such method` for now.

No ModelDriver interface change. No new dependencies.

### How was this tested?

```bash
go test -vet=off -run TestCometAPI -count=1 ./internal/entity/models/...
go test -vet=off -count=1 ./internal/entity/models/...
```

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Signed-off-by: majiayu000 <1835304752@qq.com>
Co-authored-by: 加帆 <Jiafan@users.noreply.github.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: bulexu <baiheng527@gmail.com>
Co-authored-by: xubh <xubh@wikiflyer.cn>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Carve_ <75568342+Rynzie02@users.noreply.github.com>
Co-authored-by: Paul Y Hui <paulhui@seismic.com>
Co-authored-by: LIRUI YU <128563231+LiruiYu33@users.noreply.github.com>
Co-authored-by: yun.kou <koopking@gmail.com>
Co-authored-by: Yun.kou <yunkou@deepglint.com>
Co-authored-by: Ahmad Intisar <168020872+ahmadintisar@users.noreply.github.com>
Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
Co-authored-by: chanx <1243304602@qq.com>
Co-authored-by: Syed Shahmeer Ali <syedshahmeerali196@gmail.com>
Co-authored-by: Octopus <liyuan851277048@icloud.com>
Co-authored-by: lif <1835304752@qq.com>
 2026-05-18 14:31:16 +08:00
carlos4s
2eba2c4d75 Add Anthropic Go model provider (#14940) 
### What problem does this PR solve?

Adds the missing Anthropic provider implementation for the Go model
provider layer.

Closes #14939

### What changed

- Add `conf/models/anthropic.json` with Anthropic Claude chat/vision
models and API endpoints.
- Add `internal/entity/models/anthropic.go` implementing non-streaming
Messages API chat, model listing, and connection checking.
- Register `anthropic` in the Go model factory.
- Add httptest coverage for headers, payload mapping, response parsing,
validation errors, provider errors, model listing, connection checking,
factory registration, and unsupported methods.

### Notes

Streaming chat is left as an explicit `no such method` follow-up because
this initial provider focuses on non-streaming chat and connection
checking.

### Tests

- `docker run --rm -v
/home/ubuntu/Documents/gitTensor_repos/carlos/ragflow:/work -v
/tmp/ragflow-go-cache:/go/pkg/mod -v
/tmp/ragflow-go-build:/root/.cache/go-build -w /work golang:1.25 go test
-vet=off ./internal/entity/models -run Anthropic -count=1 -v`
- `docker run --rm -v
/home/ubuntu/Documents/gitTensor_repos/carlos/ragflow:/work -v
/tmp/ragflow-go-cache:/go/pkg/mod -v
/tmp/ragflow-go-build:/root/.cache/go-build -w /work golang:1.25 go test
-vet=off ./internal/entity -count=1`
- `git diff --check`
- `jq . conf/models/anthropic.json >/dev/null`

Plain `go test ./internal/entity/models` currently hits pre-existing
unrelated vet findings in other provider files (`baidu.go`, `cohere.go`,
`fishaudio.go`, `openrouter.go`).

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-18 12:03:33 +08:00
Jake Armstrong
fe1433d1ff Go: add Jina chat completions support (#14935) 
### What problem does this PR solve?

This PR adds non-streaming chat support for the Jina Go model provider.

The Jina provider was added with embedding, rerank, model listing, and
connection checking, but `ChatWithMessages` still returned a
not-implemented error even though Jina exposes an OpenAI-compatible
`/v1/chat/completions` endpoint.

Closes #14933

**The following functionalities are now supported:**

### **Jina:**
- [x] Chat
- [ ] Stream Chat
- [x] Embedding
- [x] Rerank
- [x] Model listing
- [x] Provider connection checking
- [ ] Balance

### **Implementation details:**
- Implements `JinaModel.ChatWithMessages`
- Sends `Authorization: Bearer <api-key>` and JSON chat completion
requests
- Validates API key, model name, messages, and configured region before
making requests
- Forwards supported chat config fields: `max_tokens`, `temperature`,
`top_p`, and `stop`
- Parses the first chat completion choice into `ChatResponse.Answer`
- Adds `jina-ai/jina-vlm` as a chat-capable model in
`conf/models/jina.json`
- Adds focused unit tests for request construction, auth, response
parsing, validation errors, provider errors, and region handling

**Verification:**

```plaintext
docker run --rm -v $PWD:/repo -w /repo golang:1.25 sh -c '/usr/local/go/bin/gofmt -w internal/entity/models/jina.go internal/entity/models/jina_test.go && /usr/local/go/bin/go test -vet=off ./internal/entity/models -run TestJina -count=1'
ok  	ragflow/internal/entity/models	0.037s
```

Note: `go test ./internal/entity/models -run TestJina -count=1`
currently hits unrelated existing vet findings in other provider files,
so the focused Jina tests were run with `-vet=off`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-18 12:03:12 +08:00
Panda Dev
6794ad2f70 Go: implement Embed (embeddings) in Novita driver (#14895) 
### What problem does this PR solve?

Fixes #14893

The Novita Go driver landed in #14850 and shipped a stub `Embed` method
that returned `"novita, no such method"`, so Novita could not be used as
an embedding provider in RAGFlow. This PR fills that gap.

Novita exposes a public embeddings endpoint at `POST
https://api.novita.ai/v3/embeddings` that accepts the standard
OpenAI-compatible request shape (`{model, input}`) with `Authorization:
Bearer <api_key>`. Two embedding models are documented in Novita's model
library: `baai/bge-m3` (multilingual, 8192 tokens) and
`baai/bge-large-en-v1.5`.

### Changes

- `internal/entity/models/novita.go`: implement `NovitaModel.Embed`.
- Validate inputs (api key, model name) and short-circuit on empty
texts.
  - Resolve region with the existing `baseURLForRegion` helper.
- Build URL from `URLSuffix.Embedding` (the embeddings path lives under
`/v3/`, separate from the chat path under `/openai/v1/`).
- Send `{model, input}` POST body, add `dimensions` when
`embeddingConfig.Dimension > 0` (matches the pattern in #14735).
  - Bearer auth + JSON content type, mirroring the chat path.
- Parse `{data: [{embedding, index}]}` and reorder by `index`, rejecting
out-of-range indices, duplicates, and missing entries so the output
always lines up with the input. Same shape as the merged Mistral and
Upstage Embed implementations.
- `conf/models/novita.json`:
  - Add `"embedding": "v3/embeddings"` to `url_suffix`.
- Add default embedding model entries for `baai/bge-m3` and
`baai/bge-large-en-v1.5` so they appear in the model picker.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-18 12:02:28 +08:00
Haruko386
bf41d35729 Go: implement PaddleOCR provider and implement ASR for CoHere (#14954) 
### What problem does this PR solve?

This PR implement implement OCR for Baidu and Mistral, implement
PaddleOCR provider and implement ASR for CoHere

**Verified examples from the CLI:**

```
RAGFlow(user)> ocr with 'mistral-ocr-2512@test@mistral' file './internal/text.jpg'
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                                                                                                                                                             |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parallel to these organizational innovations there were significant complementary technical innovations (e.g., improved methods of manufacturing cast-iron pipe and of coating interiors for pressure maintenance, and newer paving and construction material... |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+


RAGFlow(user)> ocr with 'paddleocr-vl-0.9b@test@baidu' file './internal/text.jpg'
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                                                                                                                                                             |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parallel to these organizational innovations there were significant complementary technical innovations (e.g., improved methods of manufacturing cast-iron pipe and of coating interiors for pressure maintenance, and newer paving and construction material... |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

# PaddleOCR
RAGFlow(user)> ocr with 'PaddleOCR-VL-1.5@test@paddleocr' file './internal/test.pdf'
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                                                                                                                                                             |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| # Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

Bingxin Ke

Nando Metzger

Photogra

Anton Obukhov

Rodrigo Caye Daudt

netry and Remote Sensing,

Shengyu Huang

Konrad Schindler

ETH Zürich





<div style="text-align: c...  |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

# Cohere

RAGFlow(user)> asr with 'cohere-transcribe-03-2026@test@cohere' audio './internal/test.wav' param '{"language": "en"}'
+-----------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                  |
+-----------------------------------------------------------------------------------------------------------------------+
|  The examination and testimony of the experts enabled the Commission to conclude that five shots may have been fired. |
+-----------------------------------------------------------------------------------------------------------------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
 2026-05-15 18:41:43 +08:00
Haruko386
c2863173b0 Go: implement TTS, ASR for Siliconflow and TTs for StepFun (#14944) 
### What problem does this PR solve?

This PRimplement TTS, ASR for Siliconflow and TTs for StepFun

**The following functionalities are now supported:**

**SiliConFlow:**
- [x] Text To Speech
- [x] Audio To Text
- [x] Stream Audio To Text

**StrepFun:**

- [x] Audio To Text
- [x] Stream Audio To Text

**Verified examples from the CLI:**
```plaintext
# SiliconFlow

RAGFlow(user)> tts with 'FunAudioLLM/CosyVoice2-0.5B@test@Siliconflow' text 'hello? show yourself' play format 'wav' param '{"voice": "fnlp/MOSS-TTSD-v0.5:alex"}'
SUCCESS

RAGFlow(user)> asr with 'FunAudioLLM/SenseVoiceSmall@test@siliconflow' audio './internal/test.wav' param ''
+----------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                 |
+----------------------------------------------------------------------------------------------------------------------+
| The examination and testimony of the experts enabled the commission to conclude that five shots may have been fired. |
+----------------------------------------------------------------------------------------------------------------------+

RAGFlow(user)> stream asr with 'FunAudioLLM/SenseVoiceSmall@test@siliconflow' audio './internal/test.wav' param ''
+----------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                 |
+----------------------------------------------------------------------------------------------------------------------+
| The examination and testimony of the experts enabled the commission to conclude that five shots may have been fired. |
+----------------------------------------------------------------------------------------------------------------------+
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
 2026-05-15 14:03:33 +08:00
Hunnyboy1217
86bcf9767d Go: implement Rerank in vLLM driver (#14878) (#14880) 
### What problem does this PR solve?

Closes #14878.

`VllmModel.Rerank()` in
[internal/entity/models/vllm.go:551](internal/entity/models/vllm.go#L551)
is currently a stub returning `nil, fmt.Errorf("%s, Rerank not
implemented", z.Name())`, and
[conf/models/vllm.json](conf/models/vllm.json) is missing a `rerank`
entry in `url_suffix`. Chat (long-standing) and embeddings (#14688)
already work, so rerank is the last missing leg of the retrieval
pipeline for operators running everything on a single self-hosted vLLM
server — today they have to point rerank at a different provider, which
defeats the point of a fully local deployment.

Upstream vLLM has supported a Jina/Cohere-compatible `POST /v1/rerank`
endpoint since v0.7
([vllm-project/vllm#12376](https://github.com/vllm-project/vllm/pull/12376)).
The request/response shape is essentially identical to the NVIDIA driver
landed in #14778, so this PR mirrors that structure with two
vLLM-specific adjustments.

This PR replaces the stub with a real implementation against vLLM's
`/v1/rerank`:

- `POST {baseURL}/rerank`
- Request body: `{"model": "<modelName>", "query": "<query>",
"documents": [...], "top_n": <int>}` — documents are a flat `[]string`,
**not** wrapped as `{text: "..."}` like NVIDIA's `/ranking`.
- Response body: `{"results": [{"index": int, "relevance_score": float},
...]}` (Jina-compatible; the optional `document` field is ignored since
callers reconstruct text via `Index`).
- `Authorization: Bearer <ApiKey>` is set **only when `APIConfig.ApiKey`
is non-empty**, matching the existing `Embed`/`ListModels` behaviour in
this file. vLLM is a local driver and can be deployed without an API
key.

The return shape matches the existing `*RerankResponse` contract used by
the NVIDIA ([nvidia.go:461](internal/entity/models/nvidia.go#L461)),
Aliyun ([aliyun.go:507](internal/entity/models/aliyun.go#L507)), and
ZhipuAI ([zhipu-ai.go:554](internal/entity/models/zhipu-ai.go#L554))
drivers, i.e. `Data []RerankResult` carrying `{Index, RelevanceScore}`
in the API's ranking order. Callers that need original-input order sort
by `Index`.

Behaviour requirements from the issue, all covered:

1. Empty `documents` → returns `&RerankResponse{}` without an HTTP call.
2. Missing `modelName` → `"model name is required"` validation error.
3. `rerankConfig.TopN` honored when `0 < TopN < len(documents)`;
otherwise `top_n` defaults to `len(documents)` so callers get a score
per input.
4. Non-200 responses return an error including upstream status and body
(`"vLLM rerank API error: <status>, body: <body>"`).
5. Response `index` values are bounds-checked against `len(documents)`.

**Scope:**

- [internal/entity/models/vllm.go](internal/entity/models/vllm.go) —
replaces the `Rerank` stub at line 551 with a real implementation; adds
`vllmRerankRequest`/`vllmRerankResponse` types for the slim subset of
the payload we need. Region/baseURL resolution, 30s context timeout,
conditional bearer header, and error wrapping all follow the existing
patterns in this file.
- [conf/models/vllm.json](conf/models/vllm.json) — adds `"rerank":
"rerank"` to `url_suffix`, joined to the operator-configured vLLM base
URL the same way the NVIDIA driver joins at
[nvidia.go:485](internal/entity/models/nvidia.go#L485).
-
[internal/entity/models/vllm_rerank_test.go](internal/entity/models/vllm_rerank_test.go)
— adds 7 `httptest`-backed tests mirroring `nvidia_rerank_test.go`:
happy path (out-of-order ranking → Index preservation), `top_n` clamp to
`RerankConfig.TopN`, empty-documents short-circuit, missing-model-name
validation, HTTP error propagation, out-of-range index rejection, and a
vLLM-specific `TestVllmRerankWithoutAPIKey` locking in the optional-auth
behaviour that distinguishes this driver from NVIDIA.

**Out of scope:** no interface change, no DDL, no frontend change. Chat,
embeddings, and balance paths are untouched. No new user-facing docs
required beyond the existing rerank model setup page — vLLM joins the
list of providers whose rerank model can be selected once `/v1/rerank`
is exposed by the server.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2026-05-15 13:27:22 +08:00
Jin Hai
3a5df08c76 Go: add file parse command (#14892) 
### What problem does this PR solve?

```
RAGFlow(user)> ocr with 'hunyuanocr@test@gitee' file './picture.png'
+----------------------------------------------------------+
| text                                                     |
+----------------------------------------------------------+
| 生活不是等待风暴过去，而是学会在雨中翩翩起舞。
——佚名                                                       |
+----------------------------------------------------------+

RAGFlow(user)> list 'test@gitee' tasks;
+---------+----------------------------------+
| status  | task_id                          |
+---------+----------------------------------+
| success | C3FX4MQNKY5MGC6ZFMIXIAMJKHCEBQB5 |
+---------+----------------------------------+
RAGFlow(user)> show 'test@gitee' task 'C3FX4MQNKY5MGC6ZFMIXIAMJKHCEBQB5';
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| content                                                                                                                                                                                                                                                          | index |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| # PDF 1: Purpose of RAGFlow  

RAGFlow is an open source Retrieval-Augmented Generation (RAG) engine designed to turn raw documents into reliable context for large language models.Its purpose is to make it practical to build an Al assistant that can ans... | 1     |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+

```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-15 12:29:52 +08:00
Haruko386
106f4b777e Go: implement TTS for fishaudio, openrouter and asr for fishaudio (#14926) 
### What problem does this PR solve?

This PR implement TTS for FishAudio and MiniMax provider and ASR for
FishAudio

**The following functionalities are now supported:**

**FishAudio:**
- [x] Text To Speech
- [x] Stream Text To Speech
- [x] Audio To Text

**OpenRouter:**

- [x] Text To Speech

**Verified examples from the CLI:**
```plaintext

**FishAudio**

RAGFlow(user)> tts with 's1@test@fishaudio' text 'He who desires but acts not, breeds pestilence.' play format 'wav' save './internal' param '{"reference_id": "90e65eaaf50e4470b8e6d43ee6afd7d5", "temperature": 0.7, "top_p": 0.7, "prosody": {"speed": 1, "volume": 0, "normalize_loudness": true}, "chunk_length": 300, "normalize": true, "sample_rate": 44100, "mp3_bitrate": 128, "latency": "normal", "max_new_tokens": 1024, "repetition_penalty": 1.2, "min_chunk_length": 50, "condition_on_previous_chunks": true, "early_stop_threshold": 1}'
Saved to directory: /home/infiniflow/Documents/development/ragflow/internal/s1_output.wav
SUCCESS

RAGFlow(user)> stream tts with 's1@test@fishaudio' text 'He who desires but acts not, breeds pestilence.' play format 'wav' save './internal' param '{"reference_id": "90e65eaaf50e4470b8e6d43ee6afd7d5", "temperature": 0.7, "top_p": 0.7, "prosody": {"speed": 1, "volume": 0, "normalize_loudness": true}, "chunk_length": 300, "normalize": true, "sample_rate": 44100, "mp3_bitrate": 128, "latency": "normal", "max_new_tokens": 1024, "repetition_penalty": 1.2, "min_chunk_length": 50, "condition_on_previous_chunks": true, "early_stop_threshold": 1}'
Saved to directory: /home/infiniflow/Documents/development/ragflow/internal/s1_output.wav
SUCCESS

RAGFlow(user)> asr with 'transcribe-1@test@fishaudio' audio './internal/test.wav' param '{"language": "en", "ignore_timestamps": true}'
+----------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                 |
+----------------------------------------------------------------------------------------------------------------------+
| The examination and testimony of the experts enabled the commission to conclude that five shots may have been fired. |
+----------------------------------------------------------------------------------------------------------------------+

```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
 2026-05-14 18:58:00 +08:00
buua436
f0122179dd GO: align time units with Python and centralize timestamp injection in BaseModel (#14875) 
### What problem does this PR solve?

align time units with Python and centralize timestamp injection in
BaseModel

### Type of change

- [x] Refactoring
 2026-05-14 13:46:46 +08:00
Haruko386
ef46005ef1 Go: implement TTS for MiniMax provider and CLI testing for TTS (#14911) 
### What problem does this PR solve?

This PR implement TTS for MiniMax provider and CLI testing for TTS

**The following functionalities are now supported:**

**MiniMax:**
- [x] Chat / Stream Chat 
- [x] Embedding
- [x] Rerank
- [x] Model listing
- [x] Provider connection checking
- [x] Text To Speech
- [ ] OCRFile
- [ ] ~~Audio To Text~~
- [ ] ~~Balance~~

**Verified examples from the CLI:**

```plaintext
RAGFlow(user)> tts with 'speech-2.8-hd@test@minimax' text 'He who desires but acts not, breeds pestilence.' play format 'wav' save './internal' param '{"voice_setting": {"voice_id": "English_radiant_girl", "speed": 1, "vol": 1, "pitch": 0}, "audio_setting": {"sample_rate": 32000, "bitrate": 128000, "format": "wav", "channel": 1}, "output_format": "hex"}'
Saved to directory: /home/infiniflow/Documents/development/ragflow/internal/speech-2.8-hd_output.wav
SUCCESS

RAGFlow(user)> stream tts with 'speech-2.8-hd@test@minimax' text 'He who desires but acts not, breeds pestilence.' play format 'wav' save './internal' param '{"voice_setting": {"voice_id": "English_radiant_girl", "speed": 1, "vol": 1, "pitch": 0}, "audio_setting": {"sample_rate": 32000, "bitrate": 128000, "format": "wav", "channel": 1}, "output_format": "hex"}'
Saved to directory: /home/infiniflow/Documents/development/ragflow/internal/speech-2.8-hd_output.wav
SUCCESS
```
Set `Play` to play audio in CLI
Set `Save` `PATH_TO_SAVE` to save file
Set `format` to save file in wav or mp3
Set `Param` align with official request body

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2026-05-14 13:19:31 +08:00
tmimmanuel
cb01529d8b Go: implement provider: Voyage AI (#14811) 
### What problem does this PR solve?

Add a Go driver for Voyage AI (https://voyageai.com), one of the
unchecked providers on the umbrella tracking issue #14736. Voyage AI is
**embed + rerank only** — no chat, no streaming, no `/v1/models`
endpoint. It's the first provider in the Go layer of this shape.

Until this PR, a tenant who configured `voyage` as a model provider in
the Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver.

### What this PR includes

- New `internal/entity/models/voyage.go` with a `VoyageModel`
implementing the `ModelDriver` interface.
- New `conf/models/voyage.json` with 6 embedding models (`voyage-3.5`,
`voyage-3.5-lite`, `voyage-3-large`, `voyage-code-3`, `voyage-law-2`,
`voyage-finance-2`) and 2 rerank models (`rerank-2`, `rerank-2-lite`).
- `factory.go`: route `"voyage"` to `NewVoyageModel`.
- `internal/entity/models/voyage_test.go`: 19 unit tests.

### How the driver works

- **Embed**: `POST /v1/embeddings`. Response is OpenAI-shaped (`{data:
[{embedding, index, object, text}], model, usage}`). Driver reorders by
`index`, rejects duplicate / out-of-range / missing slots, and
short-circuits empty input without an HTTP call.
- **Rerank**: `POST /v1/rerank`. Voyage uses **`top_k`** as the request
param name (not `top_n` like Aliyun/SiliconFlow); the driver translates
`RerankConfig.TopN` → `top_k`. Response is Cohere-shaped (`{data:
[{relevance_score, index}], model}`), so the existing
`RerankResponse{Data: []RerankResult{Index, RelevanceScore}}` shape fits
cleanly.
- **`ListModels`**: returns a hardcoded list of `voyageKnownModels`.
Voyage does **not** expose `/v1/models` (probed live, returns 404), so
the driver synthesizes the list from the same set the config ships. New
upstream models are added by extending one slice.
- **`CheckConnection`**: pings a 1-input embed call against
`voyage-3.5`. Without `/v1/models`, this is the cheapest way to verify
the API key + network path before a tenant tries a real workload.
- **`ChatWithMessages` / `ChatStreamlyWithSender` / `Balance` /
`TranscribeAudio` / `AudioSpeech` / `OCRFile`**: all return `"no such
method"`. Voyage does not host any of these surfaces.

No interface change. No new dependencies.

### How was this tested?

**19 unit tests** in `internal/entity/models/voyage_test.go` — all pass
on go 1.25:

```
$ go test -vet=off -run TestVoyage -count=1 ./internal/entity/models/...
ok      ragflow/internal/entity/models  0.036s
```

Coverage: Name; Embed (happy path, reorder, empty-input, missing
key/model, duplicate index, out-of-range index, missing slot); Rerank
(happy path with `top_k` assertion, default-to-len-documents, empty
documents, out-of-range index); ListModels (static list, missing key);
CheckConnection (happy, 401); chat methods sentinels; Balance sentinel;
audio/OCR sentinels.

`go build ./internal/entity/models/...` exits 0.

**Live integration test** against `api.voyageai.com`:

```
=== RUN   TestVoyageLiveSmoke
    [OK] Name() = "voyage"
    [OK] ListModels (static): 8 models -> [voyage-3.5 voyage-3.5-lite voyage-3-large voyage-code-3 voyage-law-2 voyage-finance-2 rerank-2 rerank-2-lite]
    [OK] CheckConnection
    [OK] Embed vectors=3 dim=1024 indices=[0 1 2]
    [OK] Embed(empty) -> 0 vectors
    [OK] Rerank results=3 scores=[0.8125 0.59765625 0.39453125]
    [OK] ChatWithMessages returns voyage, no such method
    [OK] Balance returns voyage, no such method

VOYAGE LIVE SMOKE PASSED
--- PASS: TestVoyageLiveSmoke (0.81s)
```

What the live run proves on the wire:

- Auth (`Bearer <key>`) accepted by `api.voyageai.com`.
- Embed `voyage-3.5` on 3 inputs returns 3 vectors at dim 1024 with
`index` field preserved as `[0, 1, 2]` — the reorder-by-index code is
exercised on real data.
- Empty input short-circuits without an HTTP call (mock server would
have been hit if it did).
- Rerank `rerank-2` on 3 docs returns 3 real `relevance_score` floats
`[0.8125, 0.598, 0.395]`. The `top_k` translation works on the live
wire.
- All sentinel methods return the documented `"no such method"` strings.

### Note on PR history

This branch was previously named for LocalAI Embed work which is now
consolidated into PR #14813. The branch was reset to `upstream/main` and
rebuilt for Voyage. Diff against `main` is a clean +838 lines across 4
files.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Tracking: #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-14 09:46:54 +08:00
Jin Hai
30d1c1dc28 Fix go compilation (#14900) 
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-13 20:05:56 +08:00
tmimmanuel
0a4b733b2a Go: implement Rerank in LocalAI driver (#14813) 
### What problem does this PR solve?

The LocalAI Go driver landed in #14809 and Embed landed in #14811.
`Rerank` was left as a stub that returns `"not implemented"`. This PR
fills the gap.

LocalAI exposes a public rerank endpoint at `<tenant-url>/v1/rerank`
with a Cohere-shaped request and response (`{model, query, documents,
top_n}` → `{results: [{index, relevance_score}]}`). The Python side has
had `LocalAIRerank` in `rag/llm/rerank_model.py` for a long time. Until
this PR, a tenant who wanted to use LocalAI for reranking in the Go
layer got `"not implemented"`.

### What this PR includes

- `conf/models/localai.json`: add `"rerank": "rerank"` under
`url_suffix` so the driver can build the URL from config. This matches
the `URLSuffix.Rerank` field already used by aliyun and siliconflow.
- `internal/entity/models/localai.go`: replace the `Rerank` stub with a
real implementation that POSTs to `/v1/rerank`. Adds local
request/response types `localAIRerankRequest` and
`localAIRerankResponse`.

No factory change. No interface change.

### How the implementation works

- Validate the model name and resolve the tenant-supplied base URL with
the existing `resolveBaseURL` helper.
- Wrap the request with `context.WithTimeout(nonStreamCallTimeout)` so
the call has a clear deadline. Same pattern `ChatWithMessages`,
`ListModels`, and `Embed` already use in this file.
- Only set the `Authorization` header when a non-empty API key was
supplied. LocalAI accepts an empty key by default, so this preserves the
optional-auth contract.
- Default `top_n` to `len(documents)` when `rerankConfig.TopN == 0`,
matching the existing Aliyun and SiliconFlow rerank implementations.
- Validate every `results[].index` against `len(documents)`. If the
upstream returns an out-of-range index, fail clearly instead of silently
writing past the slice.
- An empty `documents` slice returns `&RerankResponse{}` with no HTTP
call.
- Non-200 responses propagate the upstream status line and body.

### Note on stacking

This PR builds on #14809 (LocalAI driver) and #14811 (LocalAI Embed).
Until both merge, this PR's diff on GitHub will include all three
commits. After #14809 and #14811 land on `main`, GitHub will auto-reduce
this PR to only the `Rerank` changes (one commit, ~99 line diff in
`localai.go` plus 1 line in `localai.json`).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` returns exit 0 on go 1.25 (the
`go.mod` minimum).
- The full method set on `LocalAIModel` still matches the `ModelDriver`
interface.
- Pattern parity with the existing Aliyun Rerank
(`internal/entity/models/aliyun.go`) and SiliconFlow Rerank
(`internal/entity/models/siliconflow.go`) implementations.

Closes #14812
Depends on #14809, #14811
Tracking: #14736

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-13 19:35:19 +08:00
Jin Hai
b18640d228 Go: fix OCR command (#14891) 
### What problem does this PR solve?

RAGFlow(user)> ocr with 'hunyuanocr@test@gitee' file './picture.png'
+----------------------------------------------------------+
| text                                                     |
+----------------------------------------------------------+
| 生活不是等待风暴过去，而是学会在雨中翩翩起舞。

——佚名                                                       |
+----------------------------------------------------------+

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-13 17:29:53 +08:00
Panda Dev
bf90c8948a Go: implement ListModels in ZhipuAI driver (#14886) 
### What problem does this PR solve?

Fixes #14884

The ZhipuAI Go driver in `internal/entity/models/zhipu-ai.go` had a stub
`ListModels` method that always returned `"zhipu-ai, no such method"`.
The DeepSeek, Gitee, NVIDIA, OpenAI, SiliconFlow, and OpenRouter drivers
in the same package already implement `ListModels` against the
OpenAI-compatible `/models` endpoint, and the model picker UI relies on
it. This PR brings ZhipuAI in line with that pattern.

### Changes

- `internal/entity/models/zhipu-ai.go`: implement
`ZhipuAIModel.ListModels`.
  - Resolve region with default fallback.
- GET `${BaseURL[region]}/${URLSuffix.Models}` (resolves to
`https://open.bigmodel.cn/api/paas/v4/models` with the default region).
- Send `Authorization: Bearer <api_key>` when an API key is configured.
Omit the header when the key is empty, so an unauthenticated caller gets
a clear `401` from upstream.
- Surface non-200 responses with the upstream status line and body,
matching the other Go drivers.
- Parse the response via the package-level `DSModelList` / `DSModel`
types already used by DeepSeek, Gitee, and SiliconFlow.
- When the response includes `owned_by`, render the entry as
`id@owned_by`, matching the convention of Gitee and SiliconFlow.
- `conf/models/zhipu-ai.json`: add `"models": "models"` to `url_suffix`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2026-05-13 16:39:14 +08:00
tmimmanuel
8b53960819 Go: implement provider: LongCat (#14809) 
### What problem does this PR solve?

Add a Go driver for LongCat (Meituan, https://longcat.chat), one of the
unchecked providers on the umbrella tracking issue #14736. LongCat
exposes an OpenAI-compatible REST API at
`https://api.longcat.chat/openai/v1` with three public chat models
including `LongCat-Flash-Thinking`, a reasoning model that returns
chain-of-thought in `reasoning_content` (OpenAI o-series shape).

Until this PR, a tenant who configured `longcat` as a model provider in
the Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver.

### What this PR includes

- New `internal/entity/models/longcat.go` with a `LongCatModel`
implementing the `ModelDriver` interface.
- New `conf/models/longcat.json` with the 3 public chat models
(Flash-Chat, Flash-Lite, Flash-Thinking) and `url_suffix` for `chat` and
`models`.
- `factory.go`: route `"longcat"` to `NewLongCatModel`.

Method coverage:

- `ChatWithMessages`: `POST /openai/v1/chat/completions`, non-streaming
- `ChatStreamlyWithSender`: SSE stream against the same endpoint
- `ListModels` / `CheckConnection`: `GET /openai/v1/models`
- **Reasoning extraction**: `message.reasoning_content` (non-stream) and
`delta.reasoning_content` (stream) flow into
`ChatResponse.ReasonContent` / the sender's second arg. Matches the
OpenAI o-series convention also used by kimi-k2.6 and DeepSeek-R1.
- **`reasoning_effort` propagation**: `ChatConfig.Effort` → request body
`reasoning_effort` (LongCat-Flash-Thinking honors it; non-reasoning
models ignore it).
- `Embed` / `Rerank` / `Balance` / `TranscribeAudio` / `AudioSpeech` /
`OCRFile` return `"no such method"` (LongCat does not expose any of
these surfaces).

No interface change. No new dependencies.

### How was this tested?

**21 unit tests** in `internal/entity/models/longcat_test.go` — all
pass:

```
$ go test -vet=off -run TestLongCat -count=1 -v ./internal/entity/models/...
=== RUN   TestLongCatName
--- PASS: TestLongCatName (0.00s)
=== RUN   TestLongCatChatHappyPath
--- PASS: TestLongCatChatHappyPath (0.00s)
=== RUN   TestLongCatChatExtractsReasoningContent
--- PASS: TestLongCatChatExtractsReasoningContent (0.00s)
=== RUN   TestLongCatChatPropagatesReasoningEffort
--- PASS: TestLongCatChatPropagatesReasoningEffort (0.00s)
=== RUN   TestLongCatChatOmitsReasoningEffortWhenUnset
--- PASS: TestLongCatChatOmitsReasoningEffortWhenUnset (0.00s)
=== RUN   TestLongCatChatRequiresAPIKey
--- PASS: TestLongCatChatRequiresAPIKey (0.00s)
=== RUN   TestLongCatChatRequiresMessages
--- PASS: TestLongCatChatRequiresMessages (0.00s)
=== RUN   TestLongCatChatRejectsHTTPError
--- PASS: TestLongCatChatRejectsHTTPError (0.00s)
=== RUN   TestLongCatStreamHappyPath
--- PASS: TestLongCatStreamHappyPath (0.00s)
=== RUN   TestLongCatStreamExtractsReasoningContent
--- PASS: TestLongCatStreamExtractsReasoningContent (0.00s)
=== RUN   TestLongCatStreamRejectsExplicitFalse
--- PASS: TestLongCatStreamRejectsExplicitFalse (0.00s)
=== RUN   TestLongCatStreamRequiresSender
--- PASS: TestLongCatStreamRequiresSender (0.00s)
=== RUN   TestLongCatStreamFailsWithoutTerminal
--- PASS: TestLongCatStreamFailsWithoutTerminal (0.00s)
=== RUN   TestLongCatListModelsHappyPath
--- PASS: TestLongCatListModelsHappyPath (0.00s)
=== RUN   TestLongCatListModelsRequiresAPIKey
--- PASS: TestLongCatListModelsRequiresAPIKey (0.00s)
=== RUN   TestLongCatCheckConnectionDelegatesToListModels
--- PASS: TestLongCatCheckConnectionDelegatesToListModels (0.00s)
=== RUN   TestLongCatEmbedReturnsNoSuchMethod
--- PASS: TestLongCatEmbedReturnsNoSuchMethod (0.00s)
=== RUN   TestLongCatRerankReturnsNoSuchMethod
--- PASS: TestLongCatRerankReturnsNoSuchMethod (0.00s)
=== RUN   TestLongCatBalanceReturnsNoSuchMethod
--- PASS: TestLongCatBalanceReturnsNoSuchMethod (0.00s)
=== RUN   TestLongCatAudioOCRReturnNoSuchMethod
--- PASS: TestLongCatAudioOCRReturnNoSuchMethod (0.00s)
PASS
ok      ragflow/internal/entity/models  0.020s
```

`go build ./internal/entity/models/...` exits 0 on go 1.25.

**Live integration test** against `api.longcat.chat`:

```
=== RUN   TestLongCatLiveSmoke
    [OK] Name() = "longcat"
    [OK] CheckConnection
    [OK] ListModels: 5 models -> [LongCat-Flash-Lite LongCat-Flash-Chat LongCat-Flash-Thinking-2601 LongCat-Flash-Omni-2603 LongCat-2.0-Preview]
    [OK] Chat (Flash-Chat) answer="Got it! Let me know if you" reason=""
    [OK] Chat (Flash-Thinking) answer len=443 head="To find 15 % of 80, follow these steps:\n\n1. **Convert the percentage to a frac..."
                  ReasonContent len=557 head="The user asks: \"15% of 80?\" They want step by step reasoning and final answer in \\boxed{}. So we need to compute 15% of ..."
    [OK] Stream content: 78 chunks, 351 chars
    [OK] Stream reasoning: 107 chunks, 537 chars
    [OK] Balance returns longcat, no such method
    [OK] Embed returns longcat, no such method
    [OK] Rerank returns longcat, no such method

LONGCAT LIVE SMOKE PASSED
--- PASS: TestLongCatLiveSmoke (31.01s)
```

What the live run proves on the wire:

- Auth header (`Bearer <key>`) is accepted by `api.longcat.chat`.
- `/openai/v1/models` parser handles the real 5-model response (note:
live API returns versioned aliases `LongCat-Flash-Thinking-2601`,
`LongCat-Flash-Omni-2603`, `LongCat-2.0-Preview` plus the un-versioned
`LongCat-Flash-Chat` and `LongCat-Flash-Lite`).
- Non-stream chat against `LongCat-Flash-Chat`: visible answer parses
correctly, `ReasonContent` correctly empty.
- Non-stream chat against `LongCat-Flash-Thinking`: 443-char answer
flows into `Answer`, 557-char chain-of-thought flows into
`ReasonContent` via the new `message.reasoning_content` extraction.
- Streaming chat against `LongCat-Flash-Thinking`: 107 reasoning chunks
(537 chars) reach the sender's second arg via `delta.reasoning_content`;
78 content chunks (351 chars) reach the first arg. Before this code, the
reasoning chunks would have been silently dropped.
- All sentinel methods (Balance, Embed, Rerank, audio/OCR) return the
documented `"no such method"` strings.

### Note on PR history

This branch was previously named for LocalAI work which is now
consolidated into PR #14813. The branch was reset to `upstream/main` and
rebuilt for LongCat. The diff against `main` is a clean +969 lines
across 4 files.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Tracking: #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-13 16:27:56 +08:00
tmimmanuel
d63d3bb7d2 Go: implement provider: Novita.ai (#14850) 
### What problem does this PR solve?

Add a Go driver for Novita.ai (https://novita.ai), one of the unchecked
providers on the umbrella tracking issue #14736. Novita exposes an
OpenAI-compatible REST API at `https://api.novita.ai/v3/openai` and
proxies a large catalog of third-party models (DeepSeek, Llama, Qwen3,
Kimi, Gemma, Mistral, MiniMax, GLM, etc.) behind a single OpenAI-shaped
surface — 102 models live at the time of writing.

Until this PR, a tenant who configured `novita` as a model provider in
the Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver.

### What this PR includes

- New `internal/entity/models/novita.go` with a `NovitaModel`
implementing the `ModelDriver` interface (~520 lines).
- New `conf/models/novita.json` with 7 representative chat models
(DeepSeek-V4, Llama-3.3-70B, Qwen3-30B/235B reasoning, Kimi-K2,
Gemma-3-27B, Mistral-Nemo).
- `factory.go`: route `"novita"` to `NewNovitaModel`.
- `internal/entity/models/novita_test.go`: 23 unit tests.

### Notable design point: `<think>...</think>` reasoning extraction

Novita-routed reasoning models like `qwen3-*` and `deepseek-r1-*` embed
their chain-of-thought **inline inside content as `<think>...</think>`
tags**, rather than in a separate `reasoning_content` field. Verified
live by probing `api.novita.ai`:

```
content head 200: <think>
Okay, let's see. I need to find 15% of 80. Hmm, percentages can sometimes be tricky, but I think
content tail 100: h, that works.
Alternatively, 0.15 × 80. If I move the decimal two places to the left for </think>
```

Without handling, a tenant picking qwen3 via Novita would see raw
`<think>` tags in their UI answer — different from every other reasoning
provider in the Go layer.

The driver detects those tags and routes the inner text to
`ChatResponse.ReasonContent` (non-stream) or the sender's second arg
(stream), keeping the visible answer clean of tag clutter:

- **`splitNovitaThink`** — scans a complete content string. Used by the
non-streaming path. Handles multiple `<think>` blocks, unclosed tags
(the model got cut off mid-reasoning), pure-text content with no tags.
- **`novitaThinkExtractor`** — stateful streaming version. Buffers
trailing bytes that might be the start of a tag (e.g. `<thi` held back
when the next chunk completes `nk>`), then emits segments in routing
order so callers can pipe them to a UI. Tested with byte-level chunk
boundaries and tag-spanning scenarios.

### Method coverage

| Method | Behavior |
|---|---|
| `ChatWithMessages` | `POST /v3/openai/chat/completions`, `<think>`
extraction on response |
| `ChatStreamlyWithSender` | SSE stream, stateful `<think>` extraction
across deltas |
| `ListModels` / `CheckConnection` | `GET /v3/openai/models` (102 live)
|
| `Embed` / `Rerank` / `Balance` / `TranscribeAudio` / `AudioSpeech` /
`OCRFile` | `"no such method"` — Novita's OpenAI-compatible surface does
not expose any |

No interface change. No new dependencies.

### How was this tested?

**23 unit tests** in `internal/entity/models/novita_test.go` — all pass:

```
$ go test -vet=off -run "TestNovita|TestSplitNovita" -count=1 ./internal/entity/models/...
ok      ragflow/internal/entity/models  0.020s
```

Coverage:

- `splitNovitaThink` (5 cases: pure text, single block, leading text,
multiple blocks, unclosed tag)
- `novitaThinkExtractor` (6 cases: single-chunk, opening tag span,
closing tag span, byte-level chunking, no tags, lone `<` not as tag
start)
- `ChatWithMessages`: pure text, with `<think>` tags, missing API key,
empty messages, HTTP error
- `ChatStreamlyWithSender`: tag-stripping with spanning deltas, pure
content, sender-required, stream-true-required
- `ListModels` / `CheckConnection` (happy paths)
- All sentinel methods

`go build ./internal/entity/models/...` exits 0 on go 1.25.

**Live integration test** against `api.novita.ai/v3/openai`:

```
=== RUN   TestNovitaLiveSmoke
    [OK] Name() = "novita"
    [OK] CheckConnection
    [OK] ListModels: 102 models (showing first 6) [deepseek/deepseek-v4-pro deepseek/deepseek-v4-flash deepseek/deepseek-v3.2 xiaomimimo/mimo-v2.5-pro moonshotai/kimi-k2.6 zai-org/glm-5.1]
    [OK] Chat (llama-3.3) answer="ok" reason=""
    [OK] Chat (qwen3) answer len=0 head=""
                  ReasonContent len=1657 head="Okay, so I need to figure out what 15% of 80 is. Hmm, percentages can sometimes trip me up, but let ..."
    [OK] Stream content: 0 chunks, 0 chars; reasoning: 600 chunks, 1667 chars
    [OK] Embed/Rerank/Balance/TranscribeAudio/AudioSpeech/OCRFile all return "novita, no such method"

NOVITA LIVE SMOKE PASSED
--- PASS: TestNovitaLiveSmoke (26.18s)
```

What the live run proves on the wire:

- Auth (`Bearer <key>`) accepted by `api.novita.ai`.
- `/v3/openai/models` parser handles the real 102-model response.
- Non-stream chat against `meta-llama/llama-3.3-70b-instruct`: clean
string answer, empty ReasonContent (non-reasoning model, pure-text
path).
- Non-stream chat against `qwen/qwen3-30b-a3b-fp8`: 1657-char reasoning
extracted from `<think>...</think>` and routed to
`ChatResponse.ReasonContent`. Visible answer is 0 chars in this run
because qwen3 spent its 600-token budget entirely on reasoning before
reaching the answer phase — that's the model's behavior, not a driver
bug. The important thing: **no `<think>` tags leaked into the visible
Answer field**.
- Streaming against qwen3: 600 reasoning chunks (1667 chars) emitted via
the sender's 2nd arg across SSE deltas; **no `<think>` tag fragments
leaked into the content channel** despite tag boundaries crossing chunk
boundaries on the wire.
- All 6 sentinel methods return the documented `"no such method"`
strings.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Tracking: #14736
 2026-05-13 14:10:50 +08:00
Joseff
733d75d6a7 Fix(Go): make Baidu Encode fail loudly on malformed responses (#14721) 
### What problem does this PR solve?

The Baidu (Qianfan) `Encode` method silently swallowed malformed
responses. If a `data[]` item from the API was missing a field (`index`,
`embedding`, or unexpected shape), the loop did `continue` instead of
returning an error, leaving `nil` entries in the result slice. Callers
got back partial results with no indication anything went wrong, which
then crashes downstream consumers when they try to use a `nil` vector.

Concrete gaps fixed:

- No count-mismatch check between `data` length and input texts (only
checked for empty)
- No duplicate-index detection (a duplicate would silently overwrite)
- No missing-index final scan
- No empty-embedding rejection
- No per-call context timeout
- `EmbeddingConfig.Dimension` (added in #14735) was not propagated

This PR replaces `map[string]interface{}` parsing with a typed
`baiduEmbeddingResponse` struct, applies the standard four-layer
validation (count → out-of-range → duplicate → empty → final
missing-index scan), adds `context.WithTimeout(nonStreamCallTimeout)`,
and forwards `embeddingConfig.Dimension` as the `dimensions` parameter
(Baidu Qianfan v2 uses an OpenAI-compatible API).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2026-05-13 12:54:00 +08:00
Jin Hai
ad4717f40a Go: fix model type check when use the model (#14843) 
### What problem does this PR solve?
```
RAGFlow(user)> chat with 'glm-ocr@test@zhipu-ai' message 'what is this'
CLI error: expect  model glm-ocr@zhipu-ai is a chat or multimodal model
```
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-12 19:44:01 +08:00
Haruko386
45ee5ca9cd Go: implement provider: Jina (#14838) 
### What problem does this PR solve?

This PR completes the Jina provider

**The following functionalities are now supported:**

**Jina:**
- [ ] Chat / Stream Chat (Not available for now: [(Jina chat API
docs)](https://api.jina.ai/docs#/Search%20Foundation%20Models/chat_completions_v1_chat_completions_post))
- [x] Embedding
- [x] Rerank
- [x] Model listing
- [x] Provider connection checking
- [ ] ~~Balance~~

**Verified examples from the CLI:**

```plaintext
RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'jina-embeddings-v2-base-en@test@jina' dimension 16
+-----------+-------+
| dimension | index |
+-----------+-------+
| 768       | 0     |
| 768       | 1     |
+-----------+-------+

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'jina-reranker-v2-base-multilingual@test@jina' top 3;
+-------+-----------------+
| index | relevance_score |
+-------+-----------------+
| 0     | 0.74316794      |
| 2     | 0.18713269      |
| 1     | 0.15817434      |
+-------+-----------------+

RAGFlow(user)> list supported models from 'jina' 'test'
+---------------------------------------------+
| model_name                                  |
+---------------------------------------------+
| Jina AI: Jina VLM                           |
| Jina AI: Jina Reranker v3                   |
| Jina AI: Jina Code Embeddings 0.5b          |
| Jina AI: Jina Code Embeddings 1.5b          |
| Jina AI: Jina Embeddings v4                 |
| Jina AI: Jina Reranker M0                   |
| Jina AI: ReaderLM v2                        |
| Jina AI: Jina Clip v2                       |
| Jina AI: Jina Embeddings v3                 |
| Jina AI: Jina Colbert v2                    |
| Jina AI: Reader LM 0.5b                     |
| Jina AI: Reader LM 1.5b                     |
| Jina AI: Jina Reranker v2 Base Multilingual |
| Jina AI: Jina Clip v1                       |
| Jina AI: Jina Reranker v1 Tiny EN           |
| Jina AI: Jina Reranker v1 Turbo EN          |
| Jina AI: Jina Reranker v1 Base EN           |
| Jina AI: Jina Colbert v1 EN                 |
| Jina AI: Jina Embeddings v2 Base ES         |
| Jina AI: Jina Embeddings v2 Base Code       |
| Jina AI: Jina Embeddings v2 Base DE         |
| Jina AI: Jina Embeddings v2 Base ZH         |
| Jina AI: Jina Embeddings v2 Base EN         |
| Jina AI: Jina Embedding B EN v1             |
| Jina AI: Jina Embeddings v5 Text Small      |
| Jina AI: Jina Embeddings v5 Omni Small      |
| Jina AI: Jina Embeddings v5 Omni Nano       |
| Jina AI: Jina Embeddings v5 Text Nano       |
+---------------------------------------------+

RAGFlow(user)> check instance 'test' from 'jina'
SUCCESS
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2026-05-12 18:03:05 +08:00
tmimmanuel
7d3836907a Go: implement Embed (embeddings) in Mistral driver (#14807) 
### What problem does this PR solve?

The Mistral Go driver landed in #14805 with chat, list models, and check
connection. `Embed` was left as a stub that returns `"not implemented"`.
This PR fills the gap.

`conf/models/mistral.json` did not list any embedding model out of the
box, so a tenant who wanted to use Mistral end to end (chat +
embeddings) could not run an embedding call. This PR adds
`mistral-embed` to the config and a real `/v1/embeddings`
implementation.

### What this PR includes

- `conf/models/mistral.json`: add `"embedding": "embeddings"` under
`url_suffix` so the driver can build the URL from config (matches the
`URLSuffix.Embedding` field already used by openai, siliconflow,
zhipu-ai), and add a `mistral-embed` entry under `models`
(1024-dimensional vectors, 8192 max input tokens).
- `internal/entity/models/mistral.go`: replace the `Embed` stub with a
real implementation that POSTs to `/v1/embeddings`. Adds local response
types `mistralEmbeddingData` and `mistralEmbeddingResponse`.

No factory change. No interface change.

### How the implementation works

- Validate `apiConfig`, the API key, and the model name. Use the
existing `baseURLForRegion` helper so an unknown region fails fast with
a clear error.
- Wrap the request with `context.WithTimeout(nonStreamCallTimeout)` so
the call has a clear deadline. Same pattern as `ChatWithMessages` and
`ListModels` already use in this file.
- Send all input texts in one request. The Mistral API accepts the
`input` field as an array.
- Parse `data[*].embedding` and copy each slice into a `[]EmbeddingData`
indexed by `data[*].index` so the output order matches the input order
even if the API returns items in a different order.
- An empty input slice returns `[]EmbeddingData{}` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.
- A final pass checks that every input slot got a vector. If any slot is
still empty, return a clear error so the caller does not silently use a
zero vector.

### Note on stacking

This PR builds on #14805 (the Mistral driver). Until #14805 merges, this
PR's diff on GitHub will include both that PR's commits and this one.
After #14805 lands on `main`, GitHub will auto-reduce this PR to only
the `Embed` changes (one commit, ~111 line diff in `mistral.go` plus 8
lines in `mistral.json`).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` returns exit 0 on go 1.25 (the
`go.mod` minimum).
- The full method set on `MistralModel` still matches the `ModelDriver`
interface.
- Pattern parity with the existing OpenAI Embed implementation
(`internal/entity/models/openai.go`).

Closes #14806
Depends on #14805
Tracking: #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-12 17:45:48 +08:00
Jin Hai
d08bf02d9b Go: add ASR, TTS, OCR command (#14836) 
### What problem does this PR solve?

```
RAGFlow(user)> asr with 'glm-asr-2512@test@zhipu-ai' audio './speech.wav';
CLI error: zhipu, no such method
RAGFlow(user)> stream asr with 'glm-asr-2512@test@zhipu-ai' audio './speech.wav';
CLI error: zhipu, no such method

RAGFlow(user)> tts with 'glm-tts@test@zhipu-ai' text 'how are you';
CLI error: zhipu, no such method

RAGFlow(user)> stream tts with 'glm-tts@test@zhipu-ai' text 'how are you';
CLI error: zhipu, no such method

RAGFlow(user)> ocr with 'glm-ocr@test@zhipu-ai' file './test.log';
CLI error: zhipu, no such method
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-12 17:17:44 +08:00
tmimmanuel
eaa2e46b1e Go: implement Embed (embeddings) in Upstage driver (#14819) 
### What problem does this PR solve?

The Upstage Go driver landed in #14817 with chat, list models, and check
connection. `Embed` was left as a stub that returns `"not implemented"`.
This PR fills the gap.

Upstage exposes an OpenAI-compatible embeddings endpoint at
`https://api.upstage.ai/v1/solar/embeddings` via the
`solar-embedding-1-large` family (`solar-embedding-1-large-query` for
queries, `solar-embedding-1-large-passage` for passages), and the Python
side has had `UpstageEmbed(OpenAIEmbed)` in `rag/llm/embedding_model.py`
for a long time targeting this same path. The existing
`conf/models/upstage.json` did not list any embedding model out of the
box, so a tenant who wanted to use Upstage end to end could not run an
embedding call. This PR fills the gap.

### What this PR includes

- `conf/models/upstage.json`: add `"embedding": "embeddings"` under
`url_suffix` so the driver can build the URL from config (matches the
`URLSuffix.Embedding` field already used by openai, mistral,
siliconflow, zhipu-ai), and add `solar-embedding-1-large-query` and
`solar-embedding-1-large-passage` entries under `models`.
- `internal/entity/models/upstage.go`: replace the `Embed` stub with a
real implementation that POSTs to `/v1/solar/embeddings`. Adds local
response types `upstageEmbeddingData` and `upstageEmbeddingResponse`.

No factory change. No interface change.

### How the implementation works

- Validate `apiConfig`, the API key, and the model name. Use the
existing `baseURLForRegion` helper so an unknown region fails fast with
a clear error.
- Wrap the request with `context.WithTimeout(nonStreamCallTimeout)` so
the call has a clear deadline. Same pattern as `ChatWithMessages` and
`ListModels` already use in this file.
- Send all input texts in one request. The Upstage API accepts the
`input` field as an array.
- Parse `data[*].embedding` and copy each slice into a `[]EmbeddingData`
indexed by `data[*].index` so the output order matches the input order
even if the API returns items in a different order.
- An empty input slice returns `[]EmbeddingData{}` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.
- A final pass checks that every input slot got a vector. If any slot is
still empty, return a clear error so the caller does not silently use a
zero vector.

### Note on stacking

This PR builds on #14817 (the Upstage driver). Until #14817 merges, this
PR's diff on GitHub will include both that PR's commits and this one.
After #14817 lands on `main`, GitHub will auto-reduce this PR to only
the `Embed` changes (one commit, ~119 line diff in `upstage.go` plus ~15
lines in `upstage.json`).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` returns exit 0 on go 1.25 (the
`go.mod` minimum).
- The full method set on `UpstageModel` still matches the `ModelDriver`
interface.
- Pattern parity with the existing Mistral Embed
(`internal/entity/models/mistral.go`) and OpenAI Embed
(`internal/entity/models/openai.go`) implementations.

Closes #14818
Depends on #14817
Tracking: #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-05-12 16:11:06 +08:00