Add GPU support and improve Docker deployment
- Add GPU deployment support with NVIDIA runtime - Update Dockerfile.allinone with GPU environment variables - Add comprehensive GPU_DEPLOYMENT.md guide - Make port 11434 (Ollama) optional for security - Update DEPLOYMENT.md with CPU and GPU deployment options - Simplify default docker run commands - Update healthcheck to only check web application - Add memory requirements documentation - Create MEMORY_REQUIREMENTS.md with model comparison - Add build-8b.sh script for lower memory usage - Document OOM troubleshooting steps - Improve Docker build process - Add BUILD_TROUBLESHOOTING.md for common issues - Add DISTRIBUTION.md for image distribution methods - Update .gitignore to exclude large binary files - Improve docker-entrypoint.sh with better diagnostics - Update .dockerignore to include ollama-linux-amd64.tgz - Add backup file exclusions to .gitignore
This commit is contained in:
113
DEPLOYMENT.md
113
DEPLOYMENT.md
@@ -117,19 +117,22 @@ rsync -avP --progress system-prompt-optimizer-allinone.tar user@server:/path/
|
||||
# 加载镜像(需要几分钟)
|
||||
docker load -i system-prompt-optimizer-allinone.tar
|
||||
|
||||
# 如果遇到权限错误,使用 sudo
|
||||
# sudo docker load -i system-prompt-optimizer-allinone.tar
|
||||
|
||||
# 验证镜像已加载
|
||||
docker images | grep system-prompt-optimizer
|
||||
```
|
||||
|
||||
#### 步骤 7: 启动服务
|
||||
|
||||
**CPU 模式(默认):**
|
||||
|
||||
```bash
|
||||
# 启动容器
|
||||
# 启动容器(推荐:仅暴露 Web 端口)
|
||||
docker run -d \
|
||||
--name system-prompt-optimizer \
|
||||
-p 8010:8010 \
|
||||
-p 11434:11434 \
|
||||
-v $(pwd)/outputs:/app/outputs \
|
||||
--restart unless-stopped \
|
||||
system-prompt-optimizer:allinone
|
||||
|
||||
@@ -137,7 +140,41 @@ docker run -d \
|
||||
docker logs -f system-prompt-optimizer
|
||||
```
|
||||
|
||||
**重要**:首次启动需要等待 30-60 秒,Ollama 服务需要初始化。
|
||||
**GPU 模式(推荐,如果有 NVIDIA GPU):**
|
||||
|
||||
```bash
|
||||
# 使用所有可用 GPU(推荐)
|
||||
docker run -d \
|
||||
--name system-prompt-optimizer \
|
||||
--gpus all \
|
||||
-p 8010:8010 \
|
||||
--restart unless-stopped \
|
||||
system-prompt-optimizer:allinone
|
||||
|
||||
# 或指定特定 GPU
|
||||
docker run -d \
|
||||
--name system-prompt-optimizer \
|
||||
--gpus '"device=0"' \
|
||||
-p 8010:8010 \
|
||||
--restart unless-stopped \
|
||||
system-prompt-optimizer:allinone
|
||||
|
||||
# 查看启动日志
|
||||
docker logs -f system-prompt-optimizer
|
||||
```
|
||||
|
||||
**GPU 部署前提条件**:
|
||||
- 已安装 NVIDIA 驱动 (`nvidia-smi` 可用)
|
||||
- 已安装 NVIDIA Container Toolkit
|
||||
- GPU 显存 ≥ 10GB (14b 模型) 或 ≥ 6GB (8b 模型)
|
||||
|
||||
**详细 GPU 部署指南**: 参见 [GPU_DEPLOYMENT.md](GPU_DEPLOYMENT.md)
|
||||
|
||||
**重要**:
|
||||
- 首次启动需要等待 30-60 秒(CPU)或 10-20 秒(GPU),Ollama 服务需要初始化
|
||||
- GPU 模式下推理速度提升 5-10 倍
|
||||
- 端口 11434 (Ollama) 是可选的,仅在需要外部访问 Ollama 时暴露
|
||||
- 不暴露 11434 更安全,因为 Ollama API 没有身份验证
|
||||
|
||||
#### 步骤 8: 验证部署
|
||||
|
||||
@@ -225,7 +262,8 @@ docker run -d \
|
||||
|
||||
### 端口映射
|
||||
|
||||
- **8010**: Web 界面和 API 端口
|
||||
- **8010**: Web 界面和 API 端口(必需)
|
||||
- **11434**: Ollama API 端口(可选,仅用于调试或外部访问 Ollama)
|
||||
|
||||
### 数据持久化
|
||||
|
||||
@@ -233,6 +271,71 @@ docker run -d \
|
||||
|
||||
## 故障排查
|
||||
|
||||
### 0. Docker 守护进程连接错误
|
||||
|
||||
**问题**: 运行 `docker` 命令时提示 "Cannot connect to the Docker daemon"
|
||||
|
||||
**症状**:
|
||||
```
|
||||
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
|
||||
```
|
||||
|
||||
**解决方案**:
|
||||
|
||||
**方法 1: 检查 Docker 服务状态**
|
||||
```bash
|
||||
# 检查 Docker 是否运行
|
||||
sudo systemctl status docker
|
||||
|
||||
# 如果未运行,启动它
|
||||
sudo systemctl start docker
|
||||
|
||||
# 设置开机自启
|
||||
sudo systemctl enable docker
|
||||
```
|
||||
|
||||
**方法 2: 添加用户到 docker 组(推荐)**
|
||||
```bash
|
||||
# 将当前用户添加到 docker 组
|
||||
sudo usermod -aG docker $USER
|
||||
|
||||
# 应用组变更(需要重新登录或使用 newgrp)
|
||||
newgrp docker
|
||||
|
||||
# 或者直接注销并重新登录
|
||||
|
||||
# 验证
|
||||
docker info
|
||||
```
|
||||
|
||||
**方法 3: 修复 Docker socket 权限**
|
||||
```bash
|
||||
# 检查 socket 权限
|
||||
ls -l /var/run/docker.sock
|
||||
|
||||
# 修复权限
|
||||
sudo chown root:docker /var/run/docker.sock
|
||||
sudo chmod 660 /var/run/docker.sock
|
||||
```
|
||||
|
||||
**方法 4: 临时使用 sudo**
|
||||
```bash
|
||||
# 如果上述方法不可行,使用 sudo 运行 Docker 命令
|
||||
sudo docker load -i system-prompt-optimizer-allinone.tar
|
||||
sudo docker run -d --name system-prompt-optimizer ...
|
||||
```
|
||||
|
||||
**验证修复**:
|
||||
```bash
|
||||
# 应该能正常显示 Docker 信息
|
||||
docker info
|
||||
|
||||
# 应该能看到当前用户在 docker 组中
|
||||
groups | grep docker
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 1. 无法连接 Ollama 服务
|
||||
|
||||
**问题**: 容器内无法访问宿主机的 Ollama 服务
|
||||
|
||||
Reference in New Issue
Block a user