Files
ragflow/build.sh
sapienza yoan 9d37234953 build(go): make bash build.sh work on macOS arm64 (Homebrew) (#15009)
## Problem

The Go server build pipeline (`build.sh` + CMake + CGO bindings) was
tested on Ubuntu only. On macOS arm64 with Homebrew it fails in five
orthogonal places. None of these require platform-specific code paths —
the same source builds on both Linux and Darwin after these fixes.

## Reproduction (before)

```
$ uname -a
Darwin … 25.4.0 arm64
$ brew install cmake pcre2 simde
$ bash build.sh
…
error: 'simde/x86/sse4.1.h' file not found
error: implicit instantiation of undefined template 'std::basic_istringstream<char>'
error: no matching function for call to 'Join'
…
clang: error: no such file or directory: '/usr/local/lib/libpcre2-8.a'
```

## Fix (5 small, orthogonal changes)

### 1. `internal/cpp/CMakeLists.txt` — find Homebrew + libpcre2-8
portably

- Detect Apple platforms via `if(APPLE)`, call `brew --prefix` once, add
`${HOMEBREW_PREFIX}/include` and `${HOMEBREW_PREFIX}/lib`. No effect on
Linux.
- Replace the literal `libpcre2-8.a` link token (which only the Linux
linker finds in `/usr/local/lib` by default) with
`find_library(PCRE2_LIB NAMES pcre2-8 REQUIRED)`. Works on
`/usr/lib/x86_64-linux-gnu` (Debian/Ubuntu), `/usr/local/lib` (Intel Mac
& legacy Linux), `/opt/homebrew/lib` (Apple Silicon).

### 2. `internal/cpp/wordnet_lemmatizer.cpp` +
`internal/cpp/rag_analyzer.cpp` — explicit `#include <sstream>`

libstdc++ (Linux) pulls `<sstream>` in transitively via `<fstream>`;
libc++ (Apple Clang) doesn't, so the existing `std::istringstream` /
`std::ostringstream` uses fail to compile on macOS. One-line include in
each file.

### 3. `internal/cpp/rag_analyzer.cpp` — `Join` template overload fix

`Join(tokens, start, tokens.size(), delim)` at line 146 passes `size_t`
to an `int` parameter. C++23 strict mode in Apple Clang refuses the
implicit narrowing and reports the 4-arg overload as a substitution
failure, leaving the call ambiguous between the 3-arg and 4-arg
templates. Fix: explicit `static_cast<int>(tokens.size())`. Behaviour
identical on libstdc++ — the narrowing was always intentional.

### 4. `internal/binding/rag_analyzer.go` — split darwin CGO LDFLAGS

The existing `#cgo darwin LDFLAGS: ... /usr/local/lib/libpcre2-8.a` only
matches Intel Macs. Apple Silicon Homebrew installs to `/opt/homebrew`.
Split into `darwin,arm64` and `darwin,amd64` build constraints with the
right absolute path on each.

### 5. `build.sh` — accept Homebrew path in the pcre2 sanity check

The sanity check looked at two Linux paths only and then fell through to
`sudo apt -y install libpcre2-dev` on failure. Added
`/opt/homebrew/lib/libpcre2-8.a`, and on Darwin failure now exits
cleanly with the right `brew install pcre2` hint instead of trying
`apt`.

## Verified

- `bash build.sh` now completes on macOS arm64 (Apple Silicon, brew 4.x,
cmake 4.x, Apple Clang 17, Go 1.25, pcre2 10.x, simde 0.8.x).
- Produced binaries: `bin/server_main`, `bin/admin_server`,
`bin/ragflow_cli`.
- `bin/server_main` boots, connects MySQL, runs migrations, loads the 64
model provider configs cleanly.
- Still builds on Linux — the CMake additions are inside an `if(APPLE)`
guard, the `find_library` call matches Linux paths too, the build.sh
check still tries `apt` when not on Darwin.

## Out of scope

The Go server itself currently fails at runtime when not pointing at
Elasticsearch (`Failed to initialize doc engine: failed to ping
Elasticsearch`), but that's the placeholder Infinity engine documented
in `internal/engine/README.md` — unrelated to this build patchset.

---

Happy to split this into smaller PRs if you'd prefer (one per file). The
five changes are independent.
2026-05-21 15:33:09 +08:00

225 lines
6.4 KiB
Bash
Executable File

#!/bin/bash
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Get script directory
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$SCRIPT_DIR"
# Build directories
CPP_DIR="$PROJECT_ROOT/internal/cpp"
BUILD_DIR="$CPP_DIR/cmake-build-release"
RAGFLOW_SERVER_BINARY="$PROJECT_ROOT/bin/server_main"
ADMIN_SERVER_BINARY="$PROJECT_ROOT/bin/admin_server"
RAGFLOW_CLI_BINARY="$PROJECT_ROOT/bin/ragflow_cli"
echo -e "${GREEN}=== RAGFlow Go Server Build Script ===${NC}"
# Function to print section headers
print_section() {
echo -e "\n${YELLOW}>>> $1${NC}"
}
# Check dependencies
check_cpp_deps() {
print_section "Checking c++ dependencies"
command -v cmake >/dev/null 2>&1 || { echo -e "${RED}Error: cmake is required but not installed.${NC}"; exit 1; }
command -v g++ >/dev/null 2>&1 || { echo -e "${RED}Error: g++ is required but not installed.${NC}"; exit 1; }
# Check for pcre2 library
if [ -f "/usr/lib/x86_64-linux-gnu/libpcre2-8.a" ] || [ -f "/usr/local/lib/libpcre2-8.a" ]; then
echo "✓ pcre2 library found"
else
echo -e "${YELLOW}Warning: libpcre2-8.a not found. You may need to install libpcre2-dev:${NC}"
echo " sudo apt-get install libpcre2-dev"
fi
echo "✓ Required tools are available"
}
check_go_deps() {
print_section "Checking go dependencies"
command -v go >/dev/null 2>&1 || { echo -e "${RED}Error: go is required but not installed.${NC}"; exit 1; }
echo "✓ Required tools are available"
}
# Build C++ static library
build_cpp() {
print_section "Building C++ static library"
mkdir -p "$BUILD_DIR"
cd "$BUILD_DIR"
echo "Running cmake..."
cmake .. -DCMAKE_BUILD_TYPE=Release
echo "Building librag_tokenizer_c_api.a..."
make rag_tokenizer_c_api -j$(nproc)
if [ ! -f "$BUILD_DIR/librag_tokenizer_c_api.a" ]; then
echo -e "${RED}Error: Failed to build C++ static library${NC}"
exit 1
fi
echo -e "${GREEN}✓ C++ static library built successfully${NC}"
}
# Build Go server
build_go() {
print_section "Building RAGFlow go"
cd "$PROJECT_ROOT"
# Check if C++ library exists
if [ ! -f "$BUILD_DIR/librag_tokenizer_c_api.a" ]; then
echo -e "${RED}Error: C++ static library not found. Run with --cpp first.${NC}"
exit 1
fi
# Check for pcre2 library — known Linux paths + macOS Homebrew (Apple Silicon
# at /opt/homebrew, Intel Macs at /usr/local).
if [ -f "/usr/lib/x86_64-linux-gnu/libpcre2-8.a" ] \
|| [ -f "/usr/local/lib/libpcre2-8.a" ] \
|| [ -f "/opt/homebrew/lib/libpcre2-8.a" ]; then
echo "✓ pcre2 library found"
else
if [ "$(uname)" = "Darwin" ]; then
echo -e "${RED}Error: libpcre2-8.a not found. Install with: brew install pcre2${NC}"
exit 1
fi
echo -e "${YELLOW}Warning: libpcre2-8.a not found. You may need to install libpcre2-dev:${NC}"
sudo apt -y install libpcre2-dev
fi
echo "Building RAGFlow binary: $RAGFLOW_SERVER_BINARY, $ADMIN_SERVER_BINARY, and $RAGFLOW_CLI_BINARY"
GOPROXY=${GOPROXY:-https://goproxy.cn,https://proxy.golang.org,direct} CGO_ENABLED=1 go build -o "$RAGFLOW_SERVER_BINARY" cmd/server_main.go
GOPROXY=${GOPROXY:-https://goproxy.cn,https://proxy.golang.org,direct} CGO_ENABLED=1 go build -o "$ADMIN_SERVER_BINARY" cmd/admin_server.go
GOPROXY=${GOPROXY:-https://goproxy.cn,https://proxy.golang.org,direct} CGO_ENABLED=1 go build -o "$RAGFLOW_CLI_BINARY" cmd/ragflow_cli.go
if [ ! -f "$RAGFLOW_SERVER_BINARY" ]; then
echo -e "${RED}Error: Failed to build RAGFlow server binary${NC}"
exit 1
fi
if [ ! -f "$ADMIN_SERVER_BINARY" ]; then
echo -e "${RED}Error: Failed to build Admin server binary${NC}"
exit 1
fi
echo -e "${GREEN}✓ Go ragflow_server built successfully: $RAGFLOW_SERVER_BINARY${NC}"
echo -e "${GREEN}✓ Go admin_server built successfully: $ADMIN_SERVER_BINARY${NC}"
echo -e "${GREEN}✓ Go ragflow_cli built successfully: $RAGFLOW_CLI_BINARY${NC}"
}
# Clean build artifacts
clean() {
print_section "Cleaning build artifacts"
rm -rf "$BUILD_DIR"
rm -f "$RAGFLOW_SERVER_BINARY"
rm -f "$ADMIN_SERVER_BINARY"
echo -e "${GREEN}✓ Build artifacts cleaned${NC}"
}
# Run the server
run() {
if [ ! -f "$ADMIN_SERVER_BINARY" ]; then
echo -e "${RED}Error: Binary not found. Build first with --all or --go${NC}"
exit 1
fi
print_section "Starting ADMIN server"
cd "$PROJECT_ROOT"
./admin_server
if [ ! -f "$RAGFLOW_SERVER_BINARY" ]; then
echo -e "${RED}Error: Binary not found. Build first with --all or --go${NC}"
exit 1
fi
print_section "Starting server"
cd "$PROJECT_ROOT"
./server_main
}
# Show help
show_help() {
cat << EOF
Usage: $0 [OPTIONS]
Build script for RAGFlow Go server with C++ bindings.
OPTIONS:
--all, -a Build everything (C++ library + Go server) [default]
--cpp, -c Build only C++ static library
--go, -g Build only Go server (requires C++ library to be built)
--clean, -C Clean all build artifacts
--run, -r Build and run the server
--help, -h Show this help message
EXAMPLES:
$0 # Build everything
$0 --cpp # Build only C++ library
$0 --go # Build only Go server
$0 --run # Build and run
$0 --clean # Clean build artifacts
DEPENDENCIES:
- cmake >= 4.0
- go >= 1.24
- g++ with C++17/23 support
- libpcre2-dev
EOF
}
# Main function
main() {
case "${1:-}" in
--cpp|-c)
check_cpp_deps
build_cpp
;;
--go|-g)
check_go_deps
build_go
;;
--clean|-C)
clean
;;
--run|-r)
check_cpp_deps
check_go_deps
build_cpp
build_go
run
;;
--help|-h)
show_help
;;
--all|-a|"")
check_cpp_deps
check_go_deps
build_cpp
build_go
echo -e "\n${GREEN}=== Build completed successfully! ===${NC}"
echo "Binary: $RAGFLOW_SERVER_BINARY, $ADMIN_SERVER_BINARY"
;;
*)
echo -e "${RED}Unknown option: $1${NC}"
show_help
exit 1
;;
esac
}
main "$@"