GPU Acceleration

Nothing to install. XianScan uses DirectML (it ships its own DirectML.dll) when it finds a dedicated NVIDIA, AMD or Intel Arc GPU. Detection and cleaning run on the GPU; text recognition (OCR) runs on the CPU.

• Integrated graphics are skipped and run on the CPU.
• DirectML uses the GPU that Windows lists first. On a laptop with two GPUs, set xianscan.exe to High performance in Windows Settings -> System -> Display -> Graphics.
• On a fresh cloud Windows Server (AWS, Azure), the GPU shows up only after the vendor's display driver is installed. Until then, XianScan runs on the CPU.

Check what is active:

Invoke-RestMethod http://127.0.0.1:8124/api/system/hardware | ConvertTo-Json

"active_provider": "DmlExecutionProvider" means DirectML is in use.

The Linux release uses ONNX Runtime built for CUDA 13, so it needs:

• the NVIDIA driver 580 or newer,
• the CUDA 13 runtime libraries,
• cuDNN 9 for CUDA 13.

Older setups built for CUDA 12 (driver 550, nvidia-cudnn-cu12) are not enough; XianScan then runs on the CPU.

Step 1: Install the driver, CUDA 13 and cuDNN (Ubuntu 24.04)

Add NVIDIA's CUDA repository, following NVIDIA's instructions for your distribution, then:

sudo apt-get update
sudo apt-get install -y nvidia-driver-580-server cuda-toolkit-13-0 cudnn9-cuda-13
sudo reboot

After the reboot, nvidia-smi should list your GPU and show "CUDA Version: 13" or higher.

Step 2: Check that XianScan finds every library

The archive ships libonnxruntime_providers_cuda.so next to xianscan. Keep them in the same folder, then run:

cd ~/xianscan-app    # the folder with xianscan and the .so files
ldd ./libonnxruntime_providers_cuda.so | grep "not found"

No output means everything is found. If libraries are listed, add their folder to LD_LIBRARY_PATH (step 3).

Step 3: Start XianScan

export LD_LIBRARY_PATH="$PWD:/usr/local/cuda/lib64:$LD_LIBRARY_PATH"
./xianscan

The startup log and Settings -> Hardware & Compute should now show CUDA and your GPU.

Nothing to install. On Apple Silicon, XianScan runs detection, OCR and cleaning through Apple's CoreML.

Settings -> Hardware & Compute lets you pick the device (Auto, a GPU, or CPU), limit how much GPU memory XianScan may use (useful when a local LLM shares the GPU), and set how many pages and chapters are processed in parallel. Your choice is saved.

The same can be done without the UI, for scripts. A change made this way lasts until the next restart:

# Show the active device and GPUs (no token needed on the same machine)
curl http://localhost:8124/api/system/hardware

# Switch to CUDA and cap each model at 8 GB of GPU memory
curl -X POST http://localhost:8124/api/system/hardware \
  -H "Content-Type: application/json" \
  -d '{"device": "cuda", "vram_limit_mb": 8192}'

Valid devices are auto, cuda, dml, coreml and cpu. An unknown value is treated as auto. The memory limit only applies to CUDA. From another machine, add -H "Authorization: Bearer <token>".