ragflow/internal/cli/README.md

# RAGFlow CLI (Go Version)

This is the Go implementation of the RAGFlow command-line interface, compatible with the Python version's syntax.

## Features

- Interactive mode and single command execution
- Full compatibility with Python CLI syntax
- Recursive descent parser for SQL-like commands
- Context Engine (Virtual Filesystem) for intuitive resource management
- Support for all major commands:
  - User management: LOGIN, REGISTER, CREATE USER, DROP USER, LIST USERS, etc.
  - Service management: LIST SERVICES, SHOW SERVICE, STARTUP/SHUTDOWN/RESTART SERVICE
  - Role management: CREATE ROLE, DROP ROLE, LIST ROLES, GRANT/REVOKE PERMISSION
  - Dataset management via Context Engine: `ls`, `search`, `mkdir`, `cat`, `rm`
  - Model management: SET/RESET DEFAULT LLM/VLM/EMBEDDING/etc.
  - And more...

## Usage

### Build and run

```bash
go build -o ragflow_cli ./cmd/ragflow_cli.go
./ragflow_cli
```

## Architecture

```
internal/cli/
├── cli.go              # Main CLI loop and interaction
├── client.go           # RAGFlowClient with Context Engine integration
├── http_client.go      # HTTP client for API communication
├── parser/             # Command parser package
│   ├── types.go        # Token and Command types
│   ├── lexer.go        # Lexical analyzer
│   └── parser.go       # Recursive descent parser
└── contextengine/      # Context Engine (Virtual Filesystem)
    ├── engine.go       # Core engine: path resolution, command routing
    ├── types.go        # Node, Command, Result types
    ├── provider.go     # Provider interface definition    
    ├── dataset_provider.go  # Dataset provider implementation
    ├── file_provider.go  # File manager provider implementation
    └── utils.go        # Helper functions
```

## Context Engine

The Context Engine provides a unified virtual filesystem interface over RAGFlow's RESTful APIs.

### Design Principles

1. **No Server-Side Changes**: All logic implemented client-side using existing APIs
2. **Provider Pattern**: Modular providers for different resource types (datasets, files, etc.)
3. **Unified Interface**: Common `ls`, `search`, `mkdir` commands across all providers
4. **Path-Based Navigation**: Virtual paths like `/datasets`, `/datasets/{name}/files`

### Supported Paths

| Path | Description |
|------|-------------|
| `/datasets` | List all datasets |
| `/datasets/{name}` | List documents in dataset (default behavior) |
| `/datasets/{name}/{doc}` | Get document info |

### Commands

#### `ls [path] [options]` - List nodes at path

List contents of a path in the context filesystem.

**Arguments:**
- `[path]` - Path to list (default: "datasets")

**Options:**
- `-n, --limit <number>` - Maximum number of items to display (default: 10)
- `-h, --help` - Show ls help message

**Examples:**
```bash
ls                              # List all datasets (default 10)
ls -n 20                        # List 20 datasets
ls datasets/kb1                 # List files in kb1 dataset
ls datasets/kb1 -n 50           # List 50 files in kb1 dataset
```

#### `search [options]` - Search for content

Semantic search in datasets.

**Options:**
- `-d, --dir <path>` - Directory to search in (can be specified multiple times)
- `-q, --query <query>` - Search query (required)
- `-k, --top-k <number>` - Number of top results to return (default: 10)
- `-t, --threshold <num>` - Similarity threshold, 0.0-1.0 (default: 0.2)
- `-h, --help` - Show search help message

**Output Formats:**
- Default: JSON format
- `--output plain` - Plain text format
- `--output table` - Table format with borders

**Examples:**
```bash
search -q "machine learning"                    # Search all datasets (JSON output)
search -d datasets/kb1 -q "neural networks"     # Search in kb1
search -d datasets/kb1 -q "AI" --output plain   # Plain text output
search -q "RAG" -k 20 -t 0.5                    # Return 20 results with threshold 0.5
```

#### `cat <path>` - Display content

Display document content (if available).

**Examples:**
```bash
cat myskills/doc.md   # Show content of doc.md file
cat datasets/kb1/document.pdf   # Error: cannot display binary file content
```

## Command Examples

```sql
-- Authentication
LOGIN USER 'admin@example.com';

-- User management
REGISTER USER 'john' AS 'John Doe' PASSWORD 'secret';
CREATE USER 'jane' 'password123';
DROP USER 'jane';
LIST USERS;
SHOW USER 'john';

-- Service management
LIST SERVICES;
SHOW SERVICE 1;
STARTUP SERVICE 1;
SHUTDOWN SERVICE 1;
RESTART SERVICE 1;
PING;

-- Role management
CREATE ROLE admin DESCRIPTION 'Administrator role';
LIST ROLES;
GRANT read,write ON datasets TO ROLE admin;

-- Dataset management
CREATE DATASET 'my_dataset' WITH EMBEDDING 'text-embedding-ada-002' PARSER 'naive';
LIST DATASETS;
DROP DATASET 'my_dataset';

-- Model configuration
SET DEFAULT LLM 'gpt-4';
SET DEFAULT EMBEDDING 'text-embedding-ada-002';
RESET DEFAULT LLM;

-- Context Engine (Virtual Filesystem)
ls;                                       -- List all datasets (default 10)
ls -n 20;                                 -- List 20 datasets
ls datasets/my_dataset;                   -- List documents in dataset
ls datasets/my_dataset -n 50;             -- List 50 documents
ls datasets/my_dataset/info;              -- Show dataset info
search -q "test";                         -- Search all datasets (JSON output)
search -d datasets/my_dataset -q "test";  -- Search in specific dataset

-- Meta commands
\?          -- Show help
\q          -- Quit
\c          -- Clear screen
```

## Parser Implementation

The parser uses a hand-written recursive descent approach instead of go-yacc for:
- Better control over error messages
- Easier to extend and maintain
- No code generation step required

The parser structure follows the grammar defined in the Python version, ensuring full syntax compatibility.