Compare commits
2 Commits
602875b08c
...
26f8e0c648
| Author | SHA1 | Date | |
|---|---|---|---|
| 26f8e0c648 | |||
| 65cdcf29dc |
26
.dockerignore
Normal file
26
.dockerignore
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
__pycache__
|
||||||
|
*.pyc
|
||||||
|
*.pyo
|
||||||
|
*.pyd
|
||||||
|
.Python
|
||||||
|
*.so
|
||||||
|
*.egg
|
||||||
|
*.egg-info
|
||||||
|
dist
|
||||||
|
build
|
||||||
|
.git
|
||||||
|
.gitignore
|
||||||
|
.vscode
|
||||||
|
.idea
|
||||||
|
*.md
|
||||||
|
!README.md
|
||||||
|
local_docs
|
||||||
|
examples
|
||||||
|
outputs
|
||||||
|
.DS_Store
|
||||||
|
*.log
|
||||||
|
.env
|
||||||
|
.venv
|
||||||
|
venv
|
||||||
|
env
|
||||||
|
|
||||||
358
DEPLOYMENT.md
Normal file
358
DEPLOYMENT.md
Normal file
@@ -0,0 +1,358 @@
|
|||||||
|
# Docker 部署指南
|
||||||
|
|
||||||
|
本文档说明如何在无外网访问的服务器上部署系统提示词优化工具。
|
||||||
|
|
||||||
|
## 部署方案
|
||||||
|
|
||||||
|
本项目提供两种部署方案:
|
||||||
|
|
||||||
|
### 方案 A: All-in-One 镜像(推荐,适用于无外网服务器)
|
||||||
|
|
||||||
|
**优点**:
|
||||||
|
- 包含所有依赖:应用代码 + Ollama + LLM 模型
|
||||||
|
- 一个镜像文件,部署简单
|
||||||
|
- 无需在目标服务器上安装任何额外软件(除了 Docker)
|
||||||
|
|
||||||
|
**缺点**:
|
||||||
|
- 镜像文件很大(10-20GB)
|
||||||
|
- 传输时间较长
|
||||||
|
|
||||||
|
### 方案 B: 分离部署(适用于已有 Ollama 的服务器)
|
||||||
|
|
||||||
|
**优点**:
|
||||||
|
- 镜像文件较小(~500MB)
|
||||||
|
- 可以复用现有的 Ollama 服务
|
||||||
|
|
||||||
|
**缺点**:
|
||||||
|
- 需要在目标服务器上单独安装和配置 Ollama
|
||||||
|
- 需要手动下载模型
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 方案 A: All-in-One 部署(推荐)
|
||||||
|
|
||||||
|
### 前置要求
|
||||||
|
|
||||||
|
#### 在开发机器上(有外网访问)
|
||||||
|
|
||||||
|
1. **Docker** 已安装
|
||||||
|
2. **Ollama** 已安装并运行
|
||||||
|
3. **磁盘空间**:至少 30GB 可用空间
|
||||||
|
4. 已下载所需的 Ollama 模型:
|
||||||
|
- `qwen3:14b` (主模型,~8GB)
|
||||||
|
- `qwen3-embedding:4b` (嵌入模型,~2GB)
|
||||||
|
|
||||||
|
#### 在目标服务器上(无外网访问)
|
||||||
|
|
||||||
|
1. **Docker** 已安装
|
||||||
|
2. **磁盘空间**:至少 25GB 可用空间
|
||||||
|
|
||||||
|
### 部署步骤
|
||||||
|
|
||||||
|
#### 步骤 1: 下载所需的 Ollama 模型
|
||||||
|
|
||||||
|
在开发机器上,确保已下载所需模型:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 下载主模型(约 8GB)
|
||||||
|
ollama pull qwen3:14b
|
||||||
|
|
||||||
|
# 下载嵌入模型(约 2GB)
|
||||||
|
ollama pull qwen3-embedding:4b
|
||||||
|
|
||||||
|
# 验证模型已下载
|
||||||
|
ollama list
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 步骤 2: 导出 Ollama 模型
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 运行导出脚本
|
||||||
|
./export-ollama-models.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
这将创建 `ollama-models/` 目录,包含所有模型文件。
|
||||||
|
|
||||||
|
#### 步骤 3: 构建 All-in-One Docker 镜像
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 运行构建脚本(推荐)
|
||||||
|
./build-allinone.sh
|
||||||
|
|
||||||
|
# 或手动构建
|
||||||
|
docker build -f Dockerfile.allinone -t system-prompt-optimizer:allinone .
|
||||||
|
```
|
||||||
|
|
||||||
|
**注意**:构建过程可能需要 10-30 分钟,取决于机器性能。
|
||||||
|
|
||||||
|
#### 步骤 4: 导出 Docker 镜像
|
||||||
|
|
||||||
|
如果使用 `build-allinone.sh`,镜像已自动导出。否则手动导出:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 导出镜像(约 10-20GB)
|
||||||
|
docker save -o system-prompt-optimizer-allinone.tar system-prompt-optimizer:allinone
|
||||||
|
|
||||||
|
# 验证文件大小
|
||||||
|
ls -lh system-prompt-optimizer-allinone.tar
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 步骤 5: 传输到目标服务器
|
||||||
|
|
||||||
|
使用 scp、U盘或其他方式传输镜像文件:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 使用 scp(如果网络可达)
|
||||||
|
scp system-prompt-optimizer-allinone.tar user@server:/path/
|
||||||
|
|
||||||
|
# 或使用 rsync(支持断点续传)
|
||||||
|
rsync -avP --progress system-prompt-optimizer-allinone.tar user@server:/path/
|
||||||
|
|
||||||
|
# 或使用 U盘/移动硬盘物理传输
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 步骤 6: 在目标服务器上加载镜像
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 加载镜像(需要几分钟)
|
||||||
|
docker load -i system-prompt-optimizer-allinone.tar
|
||||||
|
|
||||||
|
# 验证镜像已加载
|
||||||
|
docker images | grep system-prompt-optimizer
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 步骤 7: 启动服务
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 启动容器
|
||||||
|
docker run -d \
|
||||||
|
--name system-prompt-optimizer \
|
||||||
|
-p 8010:8010 \
|
||||||
|
-p 11434:11434 \
|
||||||
|
-v $(pwd)/outputs:/app/outputs \
|
||||||
|
--restart unless-stopped \
|
||||||
|
system-prompt-optimizer:allinone
|
||||||
|
|
||||||
|
# 查看启动日志
|
||||||
|
docker logs -f system-prompt-optimizer
|
||||||
|
```
|
||||||
|
|
||||||
|
**重要**:首次启动需要等待 30-60 秒,Ollama 服务需要初始化。
|
||||||
|
|
||||||
|
#### 步骤 8: 验证部署
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 等待服务启动(约 30-60 秒)
|
||||||
|
sleep 60
|
||||||
|
|
||||||
|
# 健康检查
|
||||||
|
curl http://localhost:8010/health
|
||||||
|
|
||||||
|
# 应该返回:
|
||||||
|
# {"status":"ok","version":"0.1.0"}
|
||||||
|
|
||||||
|
# 检查 Ollama 服务
|
||||||
|
curl http://localhost:11434/api/tags
|
||||||
|
|
||||||
|
# 检查可用模型
|
||||||
|
curl http://localhost:8010/models
|
||||||
|
|
||||||
|
# 访问 Web 界面
|
||||||
|
# 浏览器打开: http://<服务器IP>:8010/ui/opro.html
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 方案 B: 分离部署
|
||||||
|
|
||||||
|
### 前置要求
|
||||||
|
|
||||||
|
#### 在目标服务器上
|
||||||
|
|
||||||
|
1. **Docker** 已安装
|
||||||
|
2. **Ollama** 服务已安装并运行
|
||||||
|
3. 已拉取所需的 Ollama 模型:
|
||||||
|
- `qwen3:14b` (主模型)
|
||||||
|
- `qwen3-embedding:4b` (嵌入模型)
|
||||||
|
|
||||||
|
### 部署步骤
|
||||||
|
|
||||||
|
#### 步骤 1: 构建应用镜像
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 在开发机器上构建
|
||||||
|
docker build -t system-prompt-optimizer:latest .
|
||||||
|
|
||||||
|
# 导出镜像
|
||||||
|
docker save -o system-prompt-optimizer.tar system-prompt-optimizer:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 步骤 2: 传输并加载
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 传输到目标服务器
|
||||||
|
scp system-prompt-optimizer.tar user@server:/path/
|
||||||
|
|
||||||
|
# 在目标服务器上加载
|
||||||
|
docker load -i system-prompt-optimizer.tar
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 步骤 3: 启动服务
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 使用 Docker Compose
|
||||||
|
docker-compose up -d
|
||||||
|
|
||||||
|
# 或使用 Docker 命令
|
||||||
|
docker run -d \
|
||||||
|
--name system-prompt-optimizer \
|
||||||
|
-p 8010:8010 \
|
||||||
|
-e OLLAMA_HOST=http://host.docker.internal:11434 \
|
||||||
|
-v $(pwd)/outputs:/app/outputs \
|
||||||
|
--add-host host.docker.internal:host-gateway \
|
||||||
|
--restart unless-stopped \
|
||||||
|
system-prompt-optimizer:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
## 配置说明
|
||||||
|
|
||||||
|
### 环境变量
|
||||||
|
|
||||||
|
在 `docker-compose.yml` 或 `docker run` 命令中可以配置以下环境变量:
|
||||||
|
|
||||||
|
- `OLLAMA_HOST`: Ollama 服务地址(默认: `http://host.docker.internal:11434`)
|
||||||
|
- `PYTHONUNBUFFERED`: Python 输出缓冲(默认: `1`)
|
||||||
|
|
||||||
|
### 端口映射
|
||||||
|
|
||||||
|
- **8010**: Web 界面和 API 端口
|
||||||
|
|
||||||
|
### 数据持久化
|
||||||
|
|
||||||
|
- `./outputs`: 用户反馈日志存储目录(映射到容器内 `/app/outputs`)
|
||||||
|
|
||||||
|
## 故障排查
|
||||||
|
|
||||||
|
### 1. 无法连接 Ollama 服务
|
||||||
|
|
||||||
|
**问题**: 容器内无法访问宿主机的 Ollama 服务
|
||||||
|
|
||||||
|
**解决方案**:
|
||||||
|
```bash
|
||||||
|
# 确保使用了 --add-host 参数
|
||||||
|
--add-host host.docker.internal:host-gateway
|
||||||
|
|
||||||
|
# 或者直接使用宿主机 IP
|
||||||
|
-e OLLAMA_HOST=http://192.168.1.100:11434
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. 模型不可用(All-in-One 部署)
|
||||||
|
|
||||||
|
**问题**: 容器内模型未正确加载
|
||||||
|
|
||||||
|
**解决方案**:
|
||||||
|
```bash
|
||||||
|
# 进入容器检查
|
||||||
|
docker exec -it system-prompt-optimizer bash
|
||||||
|
|
||||||
|
# 在容器内检查模型
|
||||||
|
ollama list
|
||||||
|
|
||||||
|
# 如果模型不存在,检查模型目录
|
||||||
|
ls -la /root/.ollama/models/
|
||||||
|
|
||||||
|
# 退出容器
|
||||||
|
exit
|
||||||
|
```
|
||||||
|
|
||||||
|
如果模型确实丢失,可能需要重新构建镜像。
|
||||||
|
|
||||||
|
### 3. 模型不可用(分离部署)
|
||||||
|
|
||||||
|
**问题**: Ollama 模型未安装
|
||||||
|
|
||||||
|
**解决方案**:
|
||||||
|
```bash
|
||||||
|
# 在宿主机上拉取模型
|
||||||
|
ollama pull qwen3:14b
|
||||||
|
ollama pull qwen3-embedding:4b
|
||||||
|
|
||||||
|
# 验证模型已安装
|
||||||
|
ollama list
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. 容器启动失败
|
||||||
|
|
||||||
|
**问题**: 端口被占用或权限问题
|
||||||
|
|
||||||
|
**解决方案**:
|
||||||
|
```bash
|
||||||
|
# 检查端口占用
|
||||||
|
netstat -tulpn | grep 8010
|
||||||
|
netstat -tulpn | grep 11434
|
||||||
|
|
||||||
|
# 更换端口(All-in-One 需要两个端口)
|
||||||
|
docker run -p 8011:8010 -p 11435:11434 ...
|
||||||
|
|
||||||
|
# 查看容器日志
|
||||||
|
docker logs system-prompt-optimizer
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. 性能问题
|
||||||
|
|
||||||
|
**问题**: 生成速度慢
|
||||||
|
|
||||||
|
**解决方案**:
|
||||||
|
- 确保 Ollama 使用 GPU 加速
|
||||||
|
- 使用更小的模型(如 `qwen3:4b`)
|
||||||
|
- 调整 `config.py` 中的 `GENERATION_POOL_SIZE`
|
||||||
|
|
||||||
|
## 更新部署
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. 在开发机器上重新构建镜像
|
||||||
|
docker build -t system-prompt-optimizer:latest .
|
||||||
|
|
||||||
|
# 2. 导出新镜像
|
||||||
|
docker save -o system-prompt-optimizer-new.tar system-prompt-optimizer:latest
|
||||||
|
|
||||||
|
# 3. 传输到服务器并加载
|
||||||
|
docker load -i system-prompt-optimizer-new.tar
|
||||||
|
|
||||||
|
# 4. 重启服务
|
||||||
|
docker-compose down
|
||||||
|
docker-compose up -d
|
||||||
|
|
||||||
|
# 或使用 docker 命令
|
||||||
|
docker stop system-prompt-optimizer
|
||||||
|
docker rm system-prompt-optimizer
|
||||||
|
docker run -d ... # 使用相同的启动命令
|
||||||
|
```
|
||||||
|
|
||||||
|
## 安全建议
|
||||||
|
|
||||||
|
1. **网络隔离**: 如果不需要外部访问,只绑定到 localhost
|
||||||
|
```bash
|
||||||
|
-p 127.0.0.1:8010:8010
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **防火墙**: 配置防火墙规则限制访问
|
||||||
|
```bash
|
||||||
|
# 只允许特定 IP 访问
|
||||||
|
iptables -A INPUT -p tcp --dport 8010 -s 192.168.1.0/24 -j ACCEPT
|
||||||
|
iptables -A INPUT -p tcp --dport 8010 -j DROP
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **日志管理**: 定期清理日志文件
|
||||||
|
```bash
|
||||||
|
# 限制 Docker 日志大小
|
||||||
|
docker run --log-opt max-size=10m --log-opt max-file=3 ...
|
||||||
|
```
|
||||||
|
|
||||||
|
## 联系支持
|
||||||
|
|
||||||
|
如有问题,请查看:
|
||||||
|
- 应用日志: `docker logs system-prompt-optimizer`
|
||||||
|
- Ollama 日志: `journalctl -u ollama -f`
|
||||||
|
- API 文档: http://localhost:8010/docs
|
||||||
|
|
||||||
38
Dockerfile
Normal file
38
Dockerfile
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
FROM python:3.10-slim
|
||||||
|
|
||||||
|
# Set working directory
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Install system dependencies
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
curl \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Copy requirements file
|
||||||
|
COPY requirements.txt .
|
||||||
|
|
||||||
|
# Install Python dependencies
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
|
|
||||||
|
# Copy application code
|
||||||
|
COPY _qwen_xinference_demo/ ./_qwen_xinference_demo/
|
||||||
|
COPY frontend/ ./frontend/
|
||||||
|
COPY config.py .
|
||||||
|
|
||||||
|
# Create outputs directory
|
||||||
|
RUN mkdir -p outputs
|
||||||
|
|
||||||
|
# Expose port
|
||||||
|
EXPOSE 8010
|
||||||
|
|
||||||
|
# Set environment variables
|
||||||
|
ENV PYTHONUNBUFFERED=1
|
||||||
|
ENV OLLAMA_HOST=http://host.docker.internal:11434
|
||||||
|
|
||||||
|
# Health check
|
||||||
|
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
||||||
|
CMD curl -f http://localhost:8010/health || exit 1
|
||||||
|
|
||||||
|
# Run the application
|
||||||
|
CMD ["uvicorn", "_qwen_xinference_demo.api:app", "--host", "0.0.0.0", "--port", "8010"]
|
||||||
|
|
||||||
50
Dockerfile.allinone
Normal file
50
Dockerfile.allinone
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
FROM python:3.10-slim
|
||||||
|
|
||||||
|
# Set working directory
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Install system dependencies including curl for Ollama
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
curl \
|
||||||
|
ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Install Ollama
|
||||||
|
RUN curl -fsSL https://ollama.com/install.sh | sh
|
||||||
|
|
||||||
|
# Copy requirements file
|
||||||
|
COPY requirements.txt .
|
||||||
|
|
||||||
|
# Install Python dependencies
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
|
|
||||||
|
# Copy application code
|
||||||
|
COPY _qwen_xinference_demo/ ./_qwen_xinference_demo/
|
||||||
|
COPY frontend/ ./frontend/
|
||||||
|
COPY config.py .
|
||||||
|
|
||||||
|
# Create necessary directories
|
||||||
|
RUN mkdir -p outputs /root/.ollama
|
||||||
|
|
||||||
|
# Copy pre-downloaded Ollama models
|
||||||
|
# This includes qwen3:14b and qwen3-embedding:4b
|
||||||
|
COPY ollama-models/ /root/.ollama/
|
||||||
|
|
||||||
|
# Expose ports
|
||||||
|
EXPOSE 8010 11434
|
||||||
|
|
||||||
|
# Set environment variables
|
||||||
|
ENV PYTHONUNBUFFERED=1
|
||||||
|
ENV OLLAMA_HOST=http://localhost:11434
|
||||||
|
|
||||||
|
# Copy startup script
|
||||||
|
COPY docker-entrypoint.sh /docker-entrypoint.sh
|
||||||
|
RUN chmod +x /docker-entrypoint.sh
|
||||||
|
|
||||||
|
# Health check
|
||||||
|
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
|
||||||
|
CMD curl -f http://localhost:8010/health && curl -f http://localhost:11434/api/tags || exit 1
|
||||||
|
|
||||||
|
# Run the startup script
|
||||||
|
ENTRYPOINT ["/docker-entrypoint.sh"]
|
||||||
|
|
||||||
117
QUICK_START.md
Normal file
117
QUICK_START.md
Normal file
@@ -0,0 +1,117 @@
|
|||||||
|
# 快速开始指南
|
||||||
|
|
||||||
|
## 离线部署(All-in-One 方案)
|
||||||
|
|
||||||
|
### 在开发机器上(有外网)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. 下载模型
|
||||||
|
ollama pull qwen3:14b
|
||||||
|
ollama pull qwen3-embedding:4b
|
||||||
|
|
||||||
|
# 2. 导出模型
|
||||||
|
./export-ollama-models.sh
|
||||||
|
|
||||||
|
# 3. 构建并导出 Docker 镜像
|
||||||
|
./build-allinone.sh
|
||||||
|
|
||||||
|
# 4. 传输到目标服务器
|
||||||
|
# 文件: system-prompt-optimizer-allinone.tar (约 10-20GB)
|
||||||
|
scp system-prompt-optimizer-allinone.tar user@server:/path/
|
||||||
|
```
|
||||||
|
|
||||||
|
### 在目标服务器上(无外网)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. 加载镜像
|
||||||
|
docker load -i system-prompt-optimizer-allinone.tar
|
||||||
|
|
||||||
|
# 2. 启动服务
|
||||||
|
docker run -d \
|
||||||
|
--name system-prompt-optimizer \
|
||||||
|
-p 8010:8010 \
|
||||||
|
-p 11434:11434 \
|
||||||
|
-v $(pwd)/outputs:/app/outputs \
|
||||||
|
--restart unless-stopped \
|
||||||
|
system-prompt-optimizer:allinone
|
||||||
|
|
||||||
|
# 3. 等待启动(约 60 秒)
|
||||||
|
sleep 60
|
||||||
|
|
||||||
|
# 4. 验证
|
||||||
|
curl http://localhost:8010/health
|
||||||
|
curl http://localhost:11434/api/tags
|
||||||
|
|
||||||
|
# 5. 访问界面
|
||||||
|
# http://<服务器IP>:8010/ui/opro.html
|
||||||
|
```
|
||||||
|
|
||||||
|
## 常用命令
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 查看日志
|
||||||
|
docker logs -f system-prompt-optimizer
|
||||||
|
|
||||||
|
# 重启服务
|
||||||
|
docker restart system-prompt-optimizer
|
||||||
|
|
||||||
|
# 停止服务
|
||||||
|
docker stop system-prompt-optimizer
|
||||||
|
|
||||||
|
# 删除容器
|
||||||
|
docker rm -f system-prompt-optimizer
|
||||||
|
|
||||||
|
# 进入容器
|
||||||
|
docker exec -it system-prompt-optimizer bash
|
||||||
|
|
||||||
|
# 检查模型
|
||||||
|
docker exec -it system-prompt-optimizer ollama list
|
||||||
|
```
|
||||||
|
|
||||||
|
## 端口说明
|
||||||
|
|
||||||
|
- **8010**: Web 界面和 API
|
||||||
|
- **11434**: Ollama 服务(仅 All-in-One 方案需要暴露)
|
||||||
|
|
||||||
|
## 文件说明
|
||||||
|
|
||||||
|
- `system-prompt-optimizer-allinone.tar`: 完整镜像(10-20GB)
|
||||||
|
- `outputs/`: 用户反馈日志目录
|
||||||
|
|
||||||
|
## 故障排查
|
||||||
|
|
||||||
|
### 服务无法启动
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 查看日志
|
||||||
|
docker logs system-prompt-optimizer
|
||||||
|
|
||||||
|
# 检查端口占用
|
||||||
|
netstat -tulpn | grep 8010
|
||||||
|
netstat -tulpn | grep 11434
|
||||||
|
```
|
||||||
|
|
||||||
|
### 模型不可用
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 进入容器检查
|
||||||
|
docker exec -it system-prompt-optimizer ollama list
|
||||||
|
|
||||||
|
# 应该看到:
|
||||||
|
# qwen3:14b
|
||||||
|
# qwen3-embedding:4b
|
||||||
|
```
|
||||||
|
|
||||||
|
### 性能慢
|
||||||
|
|
||||||
|
- 确保服务器有足够的 RAM(建议 16GB+)
|
||||||
|
- 如果有 GPU,使用支持 GPU 的 Docker 运行时
|
||||||
|
- 调整 `config.py` 中的 `GENERATION_POOL_SIZE`
|
||||||
|
|
||||||
|
## 更多信息
|
||||||
|
|
||||||
|
详细文档请参考:
|
||||||
|
- `DEPLOYMENT.md`: 完整部署指南
|
||||||
|
- `README.md`: 项目说明
|
||||||
|
- http://localhost:8010/docs: API 文档
|
||||||
|
|
||||||
146
README.md
146
README.md
@@ -1,58 +1,65 @@
|
|||||||
# OPRO Prompt Optimizer
|
# System Prompt Generator
|
||||||
|
|
||||||
## 功能概述
|
## 功能概述
|
||||||
|
|
||||||
OPRO (Optimization by PROmpting) 是一个基于大语言模型的提示词优化系统。本项目实现了真正的 OPRO 算法,通过迭代优化系统指令(System Instructions)来提升 LLM 在特定任务上的性能。
|
这是一个基于大语言模型的系统提示词(System Prompt)生成和迭代优化工具。通过简单的任务描述,自动生成高质量的系统指令,并支持基于用户选择的迭代改进。
|
||||||
|
|
||||||
### 核心功能
|
### 核心功能
|
||||||
|
|
||||||
- **系统指令优化**:使用 LLM 作为优化器,基于历史性能轨迹生成更优的系统指令
|
- **智能指令生成**:根据任务描述自动生成多个高质量的系统指令候选
|
||||||
- **多轮迭代优化**:支持多轮优化,每轮基于前一轮的性能反馈生成新的候选指令
|
- **迭代式改进**:基于用户选择的指令生成改进版本,避免被拒绝的方向
|
||||||
|
- **角色定义格式**:所有生成的指令都以角色定义开头(如"你是一个..."),符合最佳实践
|
||||||
- **智能候选选择**:通过语义聚类和多样性选择,从大量候选中筛选出最具代表性的指令
|
- **智能候选选择**:通过语义聚类和多样性选择,从大量候选中筛选出最具代表性的指令
|
||||||
- **性能评估**:支持自定义测试用例对系统指令进行自动评估
|
- **会话管理**:支持多个任务的并行管理和历史记录
|
||||||
- **会话管理**:支持多个优化任务的并行管理和历史记录
|
- **全面覆盖要求**:生成的指令全面覆盖任务的所有要求和细节,而非仅追求风格多样性
|
||||||
|
|
||||||
### 用户界面
|
### 用户界面
|
||||||
|
|
||||||
- **现代化聊天界面**:类似 Google Gemini 的简洁设计
|
- **现代化聊天界面**:类似 Google Gemini 的简洁设计
|
||||||
- **侧边栏会话管理**:可折叠的侧边栏,支持多会话切换
|
- **侧边栏会话管理**:可折叠的侧边栏,支持多会话切换
|
||||||
- **实时优化反馈**:每轮优化生成 3-5 个候选指令,用户可选择继续优化或执行
|
- **实时生成反馈**:每轮生成 5 个候选指令,用户可选择继续优化或复制使用
|
||||||
- **模型选择**:支持在界面中选择不同的 LLM 模型
|
- **模型选择**:支持在界面中选择不同的 LLM 模型
|
||||||
|
|
||||||
## 主要优化改进
|
## 核心特性
|
||||||
|
|
||||||
### 1. 真正的 OPRO 实现
|
### 1. 简单直观的工作流程
|
||||||
|
|
||||||
原始代码实现的是查询重写(Query Rewriting),而非真正的 OPRO。我们添加了完整的 OPRO 功能:
|
不同于复杂的 OPRO 算法(需要测试用例和自动评估),本工具采用简单直观的迭代改进方式:
|
||||||
|
|
||||||
- **系统指令生成**:`generate_system_instruction_candidates()` - 生成多样化的系统指令候选
|
- **初始生成**:输入任务描述 → 生成 5 个全面的系统指令候选
|
||||||
- **性能评估**:`evaluate_system_instruction()` - 基于测试用例评估指令性能
|
- **迭代改进**:选择喜欢的指令 → 生成基于该指令的改进版本,同时避免被拒绝的方向
|
||||||
- **轨迹优化**:基于历史 (instruction, score) 轨迹生成更优指令
|
- **无需评分**:不需要测试用例或性能评分,完全基于用户偏好进行改进
|
||||||
- **元提示工程**:专门设计的元提示用于指导 LLM 生成和优化系统指令
|
|
||||||
|
|
||||||
### 2. 性能优化
|
### 2. 高质量指令生成
|
||||||
|
|
||||||
- **候选池大小优化**:从 20 个候选减少到 10 个,速度提升约 2 倍
|
- **角色定义格式**:所有指令以"你是一个..."开头,符合系统提示词最佳实践
|
||||||
- **智能聚类选择**:使用 AgglomerativeClustering 从候选池中选择最具多样性的 Top-K
|
- **全面覆盖要求**:生成的指令全面覆盖任务的所有要求和细节
|
||||||
|
- **清晰可执行**:指令清晰、具体、可执行,包含必要的行为规范和输出格式
|
||||||
|
- **简体中文**:所有生成的指令使用简体中文
|
||||||
|
|
||||||
|
### 3. 性能优化
|
||||||
|
|
||||||
|
- **候选池大小优化**:生成 10 个候选,通过聚类选择 5 个最具多样性的
|
||||||
|
- **智能聚类选择**:使用 AgglomerativeClustering 从候选池中选择最具代表性的指令
|
||||||
- **嵌入服务回退**:Xinference → Ollama 自动回退机制,确保服务可用性
|
- **嵌入服务回退**:Xinference → Ollama 自动回退机制,确保服务可用性
|
||||||
|
|
||||||
### 3. API 架构改进
|
### 4. API 架构
|
||||||
|
|
||||||
- **新增 OPRO 端点**:
|
- **核心端点**:
|
||||||
- `POST /opro/create` - 创建 OPRO 优化任务
|
- `POST /opro/create` - 创建新任务
|
||||||
- `POST /opro/generate_and_evaluate` - 生成并自动评估候选
|
- `POST /opro/generate_and_evaluate` - 生成初始候选
|
||||||
- `POST /opro/execute` - 执行系统指令
|
- `POST /opro/refine` - 基于用户选择进行迭代改进
|
||||||
- `GET /opro/runs` - 获取所有优化任务
|
- `GET /opro/sessions` - 获取所有会话
|
||||||
- `GET /opro/run/{run_id}` - 获取特定任务详情
|
- `GET /opro/runs` - 获取所有任务
|
||||||
- **会话状态管理**:完整的 OPRO 运行状态跟踪(轨迹、测试用例、迭代次数)
|
- **会话管理**:支持多会话、多任务的并行管理
|
||||||
- **向后兼容**:保留原有查询重写功能,标记为 `opro-legacy`
|
- **向后兼容**:保留原有查询重写功能,标记为 `opro-legacy`
|
||||||
|
|
||||||
### 4. 前端界面重构
|
### 5. 前端界面
|
||||||
|
|
||||||
- **Gemini 风格设计**:简洁的白色/灰色配色,圆角设计,微妙的阴影效果
|
- **Gemini 风格设计**:简洁的白色/灰色配色,圆角设计,微妙的阴影效果
|
||||||
- **可折叠侧边栏**:默认折叠,支持会话列表管理
|
- **可折叠侧边栏**:默认折叠,支持会话列表管理
|
||||||
- **多行输入框**:支持多行文本输入,底部工具栏包含模型选择器
|
- **多行输入框**:支持多行文本输入,底部工具栏包含模型选择器
|
||||||
- **候选指令卡片**:每个候选显示编号、内容、分数,提供"继续优化"、"复制"、"执行"按钮
|
- **候选指令卡片**:每个候选显示编号和内容,提供"继续优化"和"复制"按钮
|
||||||
- **简体中文界面**:所有 UI 文本和生成的指令均使用简体中文
|
- **简体中文界面**:所有 UI 文本和生成的指令均使用简体中文
|
||||||
|
|
||||||
## 快速开始
|
## 快速开始
|
||||||
@@ -97,19 +104,18 @@ uvicorn _qwen_xinference_demo.api:app --host 0.0.0.0 --port 8010
|
|||||||
|
|
||||||
### 访问界面
|
### 访问界面
|
||||||
|
|
||||||
- **OPRO 优化界面**:http://127.0.0.1:8010/ui/opro.html
|
- **系统指令生成器**:http://127.0.0.1:8010/ui/opro.html
|
||||||
- **传统三栏界面**:http://127.0.0.1:8010/ui/
|
- **传统三栏界面**:http://127.0.0.1:8010/ui/
|
||||||
- **API 文档**:http://127.0.0.1:8010/docs
|
- **API 文档**:http://127.0.0.1:8010/docs
|
||||||
- **OpenAPI JSON**:http://127.0.0.1:8010/openapi.json
|
- **OpenAPI JSON**:http://127.0.0.1:8010/openapi.json
|
||||||
|
|
||||||
### 使用示例
|
### 使用示例
|
||||||
|
|
||||||
1. **创建新会话**:在 OPRO 界面点击"新建会话"或侧边栏的 + 按钮
|
1. **创建新会话**:在界面点击"新建会话"或侧边栏的 + 按钮
|
||||||
2. **输入任务描述**:例如"将中文翻译成英文"
|
2. **输入任务描述**:例如"帮我写一个专业的营销文案生成助手"
|
||||||
3. **查看候选指令**:系统生成 3-5 个优化的系统指令
|
3. **查看候选指令**:系统生成 5 个全面的系统指令,每个都以角色定义开头
|
||||||
4. **继续优化**:点击"继续优化"进行下一轮迭代
|
4. **选择并改进**:点击喜欢的指令上的"继续优化"按钮,生成基于该指令的改进版本
|
||||||
5. **执行指令**:点击"执行此指令"测试指令效果
|
5. **复制使用**:点击"复制"按钮将指令复制到剪贴板,用于你的应用中
|
||||||
6. **复制指令**:点击"复制"按钮将指令复制到剪贴板
|
|
||||||
|
|
||||||
## 配置说明
|
## 配置说明
|
||||||
|
|
||||||
@@ -123,8 +129,8 @@ OLLAMA_HOST = "http://127.0.0.1:11434"
|
|||||||
DEFAULT_CHAT_MODEL = "qwen3:8b"
|
DEFAULT_CHAT_MODEL = "qwen3:8b"
|
||||||
DEFAULT_EMBED_MODEL = "qwen3-embedding:4b"
|
DEFAULT_EMBED_MODEL = "qwen3-embedding:4b"
|
||||||
|
|
||||||
# OPRO 优化参数
|
# 生成参数
|
||||||
GENERATION_POOL_SIZE = 10 # 生成候选池大小
|
GENERATION_POOL_SIZE = 10 # 生成候选池大小(生成10个,聚类选择5个)
|
||||||
TOP_K = 5 # 返回给用户的候选数量
|
TOP_K = 5 # 返回给用户的候选数量
|
||||||
CLUSTER_DISTANCE_THRESHOLD = 0.15 # 聚类距离阈值
|
CLUSTER_DISTANCE_THRESHOLD = 0.15 # 聚类距离阈值
|
||||||
|
|
||||||
@@ -157,11 +163,30 @@ XINFERENCE_EMBED_URL = "http://127.0.0.1:9997/models/bge-base-zh/embed"
|
|||||||
|
|
||||||
## API 端点
|
## API 端点
|
||||||
|
|
||||||
### OPRO 相关(推荐使用)
|
### 会话管理
|
||||||
|
|
||||||
|
- `POST /opro/session/create` - 创建新会话
|
||||||
|
- `GET /opro/sessions` - 获取所有会话
|
||||||
|
- `GET /opro/session/{session_id}` - 获取会话详情
|
||||||
|
|
||||||
|
### 任务管理
|
||||||
|
|
||||||
|
- `POST /opro/create` - 在会话中创建新任务
|
||||||
|
- 请求体:`{"session_id": "xxx", "task_description": "任务描述", "model_name": "qwen3:8b"}`
|
||||||
|
- 返回:`{"run_id": "xxx", "task_description": "...", "iteration": 0}`
|
||||||
|
|
||||||
|
### 指令生成
|
||||||
|
|
||||||
|
- `POST /opro/generate_and_evaluate` - 生成初始候选指令
|
||||||
|
- 请求体:`{"run_id": "xxx", "top_k": 5, "pool_size": 10}`
|
||||||
|
- 返回:`{"candidates": [{"instruction": "...", "score": null}, ...]}`
|
||||||
|
|
||||||
|
- `POST /opro/refine` - 基于用户选择进行迭代改进
|
||||||
|
- 请求体:`{"run_id": "xxx", "selected_instruction": "用户选择的指令", "rejected_instructions": ["被拒绝的指令1", "被拒绝的指令2"]}`
|
||||||
|
- 返回:`{"candidates": [{"instruction": "...", "score": null}, ...], "iteration": 1}`
|
||||||
|
|
||||||
|
### 任务查询
|
||||||
|
|
||||||
- `POST /opro/create` - 创建优化任务
|
|
||||||
- `POST /opro/generate_and_evaluate` - 生成并评估候选
|
|
||||||
- `POST /opro/execute` - 执行系统指令
|
|
||||||
- `GET /opro/runs` - 获取所有任务
|
- `GET /opro/runs` - 获取所有任务
|
||||||
- `GET /opro/run/{run_id}` - 获取任务详情
|
- `GET /opro/run/{run_id}` - 获取任务详情
|
||||||
|
|
||||||
@@ -181,6 +206,37 @@ XINFERENCE_EMBED_URL = "http://127.0.0.1:9997/models/bge-base-zh/embed"
|
|||||||
|
|
||||||
详细 API 文档请访问:http://127.0.0.1:8010/docs
|
详细 API 文档请访问:http://127.0.0.1:8010/docs
|
||||||
|
|
||||||
|
## 工作原理
|
||||||
|
|
||||||
|
### 初始生成流程
|
||||||
|
|
||||||
|
1. 用户输入任务描述(如"帮我写一个专业的营销文案生成助手")
|
||||||
|
2. 系统使用 LLM 生成 10 个候选指令
|
||||||
|
3. 通过语义嵌入和聚类算法选择 5 个最具多样性的候选
|
||||||
|
4. 所有候选都以角色定义开头,全面覆盖任务要求
|
||||||
|
|
||||||
|
### 迭代改进流程
|
||||||
|
|
||||||
|
1. 用户选择喜欢的指令(如候选 #3)
|
||||||
|
2. 系统记录被拒绝的指令(候选 #1, #2, #4, #5)
|
||||||
|
3. 向 LLM 发送改进请求:"基于选中的指令生成改进版本,避免被拒绝指令的方向"
|
||||||
|
4. 生成新的 10 个候选,聚类选择 5 个返回
|
||||||
|
5. 用户可以继续迭代或复制使用
|
||||||
|
|
||||||
|
### 与 OPRO 的区别
|
||||||
|
|
||||||
|
**OPRO(原始算法)**:
|
||||||
|
- 需要测试用例(如数学题的正确答案)
|
||||||
|
- 自动评分(如准确率 0.73, 0.81)
|
||||||
|
- 基于性能轨迹优化
|
||||||
|
- 适用于有明确评估标准的任务
|
||||||
|
|
||||||
|
**本工具(简单迭代改进)**:
|
||||||
|
- 不需要测试用例
|
||||||
|
- 不需要自动评分
|
||||||
|
- 基于用户偏好改进
|
||||||
|
- 适用于任意通用任务
|
||||||
|
|
||||||
## 常见问题
|
## 常见问题
|
||||||
|
|
||||||
### 1. 无法连接 Ollama 服务
|
### 1. 无法连接 Ollama 服务
|
||||||
@@ -198,11 +254,17 @@ ollama serve
|
|||||||
|
|
||||||
### 3. 生成速度慢
|
### 3. 生成速度慢
|
||||||
|
|
||||||
- 调整 `GENERATION_POOL_SIZE` 减少候选数量
|
- 调整 `GENERATION_POOL_SIZE` 减少候选数量(如改为 6,返回 3 个)
|
||||||
- 使用更小的模型(如 `qwen3:4b`)
|
- 使用更小的模型(如 `qwen3:4b`)
|
||||||
- 确保 Ollama 使用 GPU 加速
|
- 确保 Ollama 使用 GPU 加速
|
||||||
|
|
||||||
### 4. 界面显示异常
|
### 4. 生成的指令质量不高
|
||||||
|
|
||||||
|
- 提供更详细的任务描述
|
||||||
|
- 多次迭代改进,选择最好的继续优化
|
||||||
|
- 尝试不同的模型
|
||||||
|
|
||||||
|
### 5. 界面显示异常
|
||||||
|
|
||||||
硬刷新浏览器缓存:
|
硬刷新浏览器缓存:
|
||||||
- **Mac**: `Cmd + Shift + R`
|
- **Mac**: `Cmd + Shift + R`
|
||||||
|
|||||||
@@ -24,7 +24,8 @@ from .opro.session_state import (
|
|||||||
from .opro.user_prompt_optimizer import generate_candidates
|
from .opro.user_prompt_optimizer import generate_candidates
|
||||||
from .opro.user_prompt_optimizer import (
|
from .opro.user_prompt_optimizer import (
|
||||||
generate_system_instruction_candidates,
|
generate_system_instruction_candidates,
|
||||||
evaluate_system_instruction
|
evaluate_system_instruction,
|
||||||
|
refine_instruction_candidates
|
||||||
)
|
)
|
||||||
|
|
||||||
from .opro.ollama_client import call_qwen
|
from .opro.ollama_client import call_qwen
|
||||||
@@ -159,6 +160,15 @@ class OPROExecuteReq(BaseModel):
|
|||||||
model_name: Optional[str] = None
|
model_name: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
class OPRORefineReq(BaseModel):
|
||||||
|
"""Request to refine based on selected instruction (simple iterative refinement, NOT OPRO)."""
|
||||||
|
run_id: str
|
||||||
|
selected_instruction: str
|
||||||
|
rejected_instructions: List[str]
|
||||||
|
top_k: Optional[int] = None
|
||||||
|
pool_size: Optional[int] = None
|
||||||
|
|
||||||
|
|
||||||
# ============================================================================
|
# ============================================================================
|
||||||
# LEGACY ENDPOINTS (Query Rewriting - NOT true OPRO)
|
# LEGACY ENDPOINTS (Query Rewriting - NOT true OPRO)
|
||||||
# ============================================================================
|
# ============================================================================
|
||||||
@@ -696,3 +706,44 @@ def opro_execute(req: OPROExecuteReq):
|
|||||||
})
|
})
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
raise AppException(500, f"Execution failed: {e}", "EXECUTION_ERROR")
|
raise AppException(500, f"Execution failed: {e}", "EXECUTION_ERROR")
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/opro/refine", tags=["opro-true"])
|
||||||
|
def opro_refine(req: OPRORefineReq):
|
||||||
|
"""
|
||||||
|
Simple iterative refinement based on user selection (NOT OPRO).
|
||||||
|
|
||||||
|
This generates new candidates based on the selected instruction while avoiding rejected ones.
|
||||||
|
No scoring, no trajectory - just straightforward refinement based on user preference.
|
||||||
|
"""
|
||||||
|
run = get_opro_run(req.run_id)
|
||||||
|
if not run:
|
||||||
|
raise AppException(404, "OPRO run not found", "RUN_NOT_FOUND")
|
||||||
|
|
||||||
|
top_k = req.top_k or config.TOP_K
|
||||||
|
pool_size = req.pool_size or config.GENERATION_POOL_SIZE
|
||||||
|
|
||||||
|
try:
|
||||||
|
candidates = refine_instruction_candidates(
|
||||||
|
task_description=run["task_description"],
|
||||||
|
selected_instruction=req.selected_instruction,
|
||||||
|
rejected_instructions=req.rejected_instructions,
|
||||||
|
top_k=top_k,
|
||||||
|
pool_size=pool_size,
|
||||||
|
model_name=run["model_name"]
|
||||||
|
)
|
||||||
|
|
||||||
|
# Update iteration counter
|
||||||
|
update_opro_iteration(req.run_id, candidates)
|
||||||
|
|
||||||
|
# Get updated run info
|
||||||
|
run = get_opro_run(req.run_id)
|
||||||
|
|
||||||
|
return ok({
|
||||||
|
"run_id": req.run_id,
|
||||||
|
"iteration": run["iteration"],
|
||||||
|
"candidates": [{"instruction": c, "score": None} for c in candidates],
|
||||||
|
"task_description": run["task_description"]
|
||||||
|
})
|
||||||
|
except Exception as e:
|
||||||
|
raise AppException(500, f"Refinement failed: {e}", "REFINEMENT_ERROR")
|
||||||
|
|||||||
@@ -56,14 +56,15 @@ def generate_initial_system_instruction_candidates(task_description: str, pool_s
|
|||||||
目标任务描述:
|
目标任务描述:
|
||||||
【{task_description}】
|
【{task_description}】
|
||||||
|
|
||||||
请根据以上任务,生成 {pool_size} 条高质量、风格各异的"System Instruction"候选指令。
|
请根据以上任务,生成 {pool_size} 条高质量、全面的"System Instruction"候选指令。
|
||||||
|
|
||||||
要求:
|
要求:
|
||||||
1. 每条指令必须有明显不同的风格和侧重点
|
1. 每条指令必须以角色定义开头(例如:"你是一个..."、"你是..."等)
|
||||||
2. 覆盖不同的实现策略(例如:简洁型、详细型、示例型、角色扮演型、步骤型等)
|
2. 每条指令必须全面覆盖任务的所有要求和细节
|
||||||
3. 这些指令应指导LLM的行为和输出格式,以最大化任务性能
|
3. 指令应清晰、具体、可执行,能够有效指导LLM完成任务
|
||||||
4. 每条指令单独成行,不包含编号或额外说明
|
4. 确保指令包含必要的行为规范、输出格式、注意事项等
|
||||||
5. 所有生成的指令必须使用简体中文
|
5. 每条指令单独成行,不包含编号或额外说明
|
||||||
|
6. 所有生成的指令必须使用简体中文
|
||||||
|
|
||||||
生成 {pool_size} 条指令:
|
生成 {pool_size} 条指令:
|
||||||
"""
|
"""
|
||||||
@@ -120,11 +121,68 @@ def generate_optimized_system_instruction(
|
|||||||
然后,生成 {pool_size} 条新的、有潜力超越 {highest_score:.4f} 分的System Instruction。
|
然后,生成 {pool_size} 条新的、有潜力超越 {highest_score:.4f} 分的System Instruction。
|
||||||
|
|
||||||
要求:
|
要求:
|
||||||
1. 每条指令必须有明显不同的改进策略
|
1. 每条指令必须以角色定义开头(例如:"你是一个..."、"你是..."等)
|
||||||
2. 结合高分指令的优点,避免低分指令的缺陷
|
2. 每条指令必须全面覆盖任务的所有要求和细节
|
||||||
3. 探索新的优化方向和表达方式
|
3. 结合高分指令的优点,避免低分指令的缺陷
|
||||||
4. 每条指令单独成行,不包含编号或额外说明
|
4. 指令应清晰、具体、可执行,能够有效指导LLM完成任务
|
||||||
5. 所有生成的指令必须使用简体中文
|
5. 每条指令单独成行,不包含编号或额外说明
|
||||||
|
6. 所有生成的指令必须使用简体中文
|
||||||
|
|
||||||
生成 {pool_size} 条优化后的指令:
|
生成 {pool_size} 条优化后的指令:
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def refine_based_on_selection(
|
||||||
|
task_description: str,
|
||||||
|
selected_instruction: str,
|
||||||
|
rejected_instructions: List[str],
|
||||||
|
pool_size: int = None
|
||||||
|
) -> str:
|
||||||
|
"""
|
||||||
|
Simple refinement: Generate variations based on selected instruction while avoiding rejected ones.
|
||||||
|
|
||||||
|
This is NOT OPRO - it's straightforward iterative refinement based on user preference.
|
||||||
|
No scoring, no trajectory, just: "I like this one, give me more like it (but not like those)."
|
||||||
|
|
||||||
|
Args:
|
||||||
|
task_description: Description of the task
|
||||||
|
selected_instruction: The instruction the user selected
|
||||||
|
rejected_instructions: The instructions the user didn't select
|
||||||
|
pool_size: Number of new candidates to generate
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Prompt for generating refined candidates
|
||||||
|
"""
|
||||||
|
import config
|
||||||
|
pool_size = pool_size or config.GENERATION_POOL_SIZE
|
||||||
|
|
||||||
|
rejected_text = ""
|
||||||
|
if rejected_instructions:
|
||||||
|
rejected_formatted = "\n".join(f"- {inst}" for inst in rejected_instructions)
|
||||||
|
rejected_text = f"""
|
||||||
|
**用户未选择的指令(避免这些方向):**
|
||||||
|
{rejected_formatted}
|
||||||
|
"""
|
||||||
|
|
||||||
|
return f"""
|
||||||
|
你是一个"System Prompt 改进助手"。
|
||||||
|
目标任务描述:
|
||||||
|
【{task_description}】
|
||||||
|
|
||||||
|
**用户选择的指令(基于此改进):**
|
||||||
|
{selected_instruction}
|
||||||
|
{rejected_text}
|
||||||
|
|
||||||
|
请基于用户选择的指令,生成 {pool_size} 条改进版本。
|
||||||
|
|
||||||
|
要求:
|
||||||
|
1. 每条指令必须以角色定义开头(例如:"你是一个..."、"你是..."等)
|
||||||
|
2. 保留用户选择指令的核心优点
|
||||||
|
3. 每条指令必须全面覆盖任务的所有要求和细节
|
||||||
|
4. 指令应清晰、具体、可执行,能够有效指导LLM完成任务
|
||||||
|
5. 避免与未选择指令相似的方向
|
||||||
|
6. 每条指令单独成行,不包含编号或额外说明
|
||||||
|
7. 所有生成的指令必须使用简体中文
|
||||||
|
|
||||||
|
生成 {pool_size} 条改进后的指令:
|
||||||
|
"""
|
||||||
|
|||||||
@@ -11,7 +11,8 @@ from .prompt_utils import (
|
|||||||
refine_instruction,
|
refine_instruction,
|
||||||
refine_instruction_with_history,
|
refine_instruction_with_history,
|
||||||
generate_initial_system_instruction_candidates,
|
generate_initial_system_instruction_candidates,
|
||||||
generate_optimized_system_instruction
|
generate_optimized_system_instruction,
|
||||||
|
refine_based_on_selection
|
||||||
)
|
)
|
||||||
|
|
||||||
def parse_candidates(raw: str) -> list:
|
def parse_candidates(raw: str) -> list:
|
||||||
@@ -147,3 +148,46 @@ def evaluate_system_instruction(
|
|||||||
correct += 1
|
correct += 1
|
||||||
|
|
||||||
return correct / total
|
return correct / total
|
||||||
|
|
||||||
|
|
||||||
|
def refine_instruction_candidates(
|
||||||
|
task_description: str,
|
||||||
|
selected_instruction: str,
|
||||||
|
rejected_instructions: List[str],
|
||||||
|
top_k: int = config.TOP_K,
|
||||||
|
pool_size: int = None,
|
||||||
|
model_name: str = None
|
||||||
|
) -> List[str]:
|
||||||
|
"""
|
||||||
|
Simple refinement: Generate new candidates based on user's selection.
|
||||||
|
|
||||||
|
This is NOT OPRO - just straightforward iterative refinement.
|
||||||
|
User picks a favorite, we generate variations of it while avoiding rejected ones.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
task_description: Description of the task
|
||||||
|
selected_instruction: The instruction the user selected
|
||||||
|
rejected_instructions: The instructions the user didn't select
|
||||||
|
top_k: Number of diverse candidates to return
|
||||||
|
pool_size: Number of candidates to generate before clustering
|
||||||
|
model_name: Optional model name to use
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of refined instruction candidates
|
||||||
|
"""
|
||||||
|
pool_size = pool_size or config.GENERATION_POOL_SIZE
|
||||||
|
|
||||||
|
# Generate the refinement prompt
|
||||||
|
meta_prompt = refine_based_on_selection(
|
||||||
|
task_description,
|
||||||
|
selected_instruction,
|
||||||
|
rejected_instructions,
|
||||||
|
pool_size
|
||||||
|
)
|
||||||
|
|
||||||
|
# Use LLM to generate refined candidates
|
||||||
|
raw = call_qwen(meta_prompt, temperature=0.9, max_tokens=1024, model_name=model_name)
|
||||||
|
|
||||||
|
# Parse and cluster
|
||||||
|
all_candidates = parse_candidates(raw)
|
||||||
|
return cluster_and_select(all_candidates, top_k=top_k)
|
||||||
|
|||||||
98
build-allinone.sh
Executable file
98
build-allinone.sh
Executable file
@@ -0,0 +1,98 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Build all-in-one Docker image with Ollama and models
|
||||||
|
# This creates a complete offline-deployable image
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
IMAGE_NAME="system-prompt-optimizer"
|
||||||
|
IMAGE_TAG="allinone"
|
||||||
|
EXPORT_FILE="${IMAGE_NAME}-${IMAGE_TAG}.tar"
|
||||||
|
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Building All-in-One Docker Image"
|
||||||
|
echo "=========================================="
|
||||||
|
echo ""
|
||||||
|
echo "This will create a Docker image containing:"
|
||||||
|
echo " - Python application"
|
||||||
|
echo " - Ollama service"
|
||||||
|
echo " - qwen3:14b model"
|
||||||
|
echo " - qwen3-embedding:4b model"
|
||||||
|
echo ""
|
||||||
|
echo "WARNING: The final image will be 10-20GB in size!"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Check if ollama-models directory exists
|
||||||
|
if [ ! -d "ollama-models" ]; then
|
||||||
|
echo "ERROR: ollama-models directory not found!"
|
||||||
|
echo ""
|
||||||
|
echo "Please run ./export-ollama-models.sh first to export the models."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "✓ Found ollama-models directory"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Check disk space
|
||||||
|
AVAILABLE_SPACE=$(df -h . | awk 'NR==2 {print $4}')
|
||||||
|
echo "Available disk space: $AVAILABLE_SPACE"
|
||||||
|
echo "Required: ~20GB for build process"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
read -p "Continue with build? (y/n) " -n 1 -r
|
||||||
|
echo
|
||||||
|
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
|
||||||
|
echo "Build cancelled."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Building Docker image..."
|
||||||
|
echo "=========================================="
|
||||||
|
docker build -f Dockerfile.allinone -t ${IMAGE_NAME}:${IMAGE_TAG} .
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Build complete!"
|
||||||
|
echo "=========================================="
|
||||||
|
docker images | grep ${IMAGE_NAME}
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Exporting image to ${EXPORT_FILE}..."
|
||||||
|
echo "=========================================="
|
||||||
|
echo "This will take several minutes..."
|
||||||
|
docker save -o ${EXPORT_FILE} ${IMAGE_NAME}:${IMAGE_TAG}
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Export complete!"
|
||||||
|
echo "=========================================="
|
||||||
|
ls -lh ${EXPORT_FILE}
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Deployment Instructions"
|
||||||
|
echo "=========================================="
|
||||||
|
echo ""
|
||||||
|
echo "1. Transfer ${EXPORT_FILE} to target server:"
|
||||||
|
echo " scp ${EXPORT_FILE} user@server:/path/"
|
||||||
|
echo ""
|
||||||
|
echo "2. On target server, load the image:"
|
||||||
|
echo " docker load -i ${EXPORT_FILE}"
|
||||||
|
echo ""
|
||||||
|
echo "3. Run the container:"
|
||||||
|
echo " docker run -d \\"
|
||||||
|
echo " --name system-prompt-optimizer \\"
|
||||||
|
echo " -p 8010:8010 \\"
|
||||||
|
echo " -p 11434:11434 \\"
|
||||||
|
echo " -v \$(pwd)/outputs:/app/outputs \\"
|
||||||
|
echo " --restart unless-stopped \\"
|
||||||
|
echo " ${IMAGE_NAME}:${IMAGE_TAG}"
|
||||||
|
echo ""
|
||||||
|
echo "4. Access the application:"
|
||||||
|
echo " http://<server-ip>:8010/ui/opro.html"
|
||||||
|
echo ""
|
||||||
|
echo "See DEPLOYMENT.md for more details."
|
||||||
|
|
||||||
37
build-and-export.sh
Executable file
37
build-and-export.sh
Executable file
@@ -0,0 +1,37 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Build and export Docker image for offline deployment
|
||||||
|
# Usage: ./build-and-export.sh
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
IMAGE_NAME="system-prompt-optimizer"
|
||||||
|
IMAGE_TAG="latest"
|
||||||
|
EXPORT_FILE="${IMAGE_NAME}.tar"
|
||||||
|
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Building Docker image..."
|
||||||
|
echo "=========================================="
|
||||||
|
docker build -t ${IMAGE_NAME}:${IMAGE_TAG} .
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Exporting Docker image to ${EXPORT_FILE}..."
|
||||||
|
echo "=========================================="
|
||||||
|
docker save -o ${EXPORT_FILE} ${IMAGE_NAME}:${IMAGE_TAG}
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Export complete!"
|
||||||
|
echo "=========================================="
|
||||||
|
ls -lh ${EXPORT_FILE}
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "Next steps:"
|
||||||
|
echo "1. Transfer ${EXPORT_FILE} to target server"
|
||||||
|
echo "2. Transfer docker-compose.yml to target server (optional)"
|
||||||
|
echo "3. On target server, run: docker load -i ${EXPORT_FILE}"
|
||||||
|
echo "4. On target server, run: docker-compose up -d"
|
||||||
|
echo ""
|
||||||
|
echo "See DEPLOYMENT.md for detailed instructions."
|
||||||
|
|
||||||
@@ -7,7 +7,7 @@ APP_CONTACT = {"name": "OPRO Team", "url": "http://127.0.0.1:8010/ui/"}
|
|||||||
OLLAMA_HOST = "http://127.0.0.1:11434"
|
OLLAMA_HOST = "http://127.0.0.1:11434"
|
||||||
OLLAMA_GENERATE_URL = f"{OLLAMA_HOST}/api/generate"
|
OLLAMA_GENERATE_URL = f"{OLLAMA_HOST}/api/generate"
|
||||||
OLLAMA_TAGS_URL = f"{OLLAMA_HOST}/api/tags"
|
OLLAMA_TAGS_URL = f"{OLLAMA_HOST}/api/tags"
|
||||||
DEFAULT_CHAT_MODEL = "qwen3:8b"
|
DEFAULT_CHAT_MODEL = "qwen3:14b"
|
||||||
DEFAULT_EMBED_MODEL = "qwen3-embedding:4b"
|
DEFAULT_EMBED_MODEL = "qwen3-embedding:4b"
|
||||||
|
|
||||||
# Xinference
|
# Xinference
|
||||||
|
|||||||
23
docker-compose.yml
Normal file
23
docker-compose.yml
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
version: '3.8'
|
||||||
|
|
||||||
|
services:
|
||||||
|
app:
|
||||||
|
build: .
|
||||||
|
container_name: system-prompt-optimizer
|
||||||
|
ports:
|
||||||
|
- "8010:8010"
|
||||||
|
environment:
|
||||||
|
- OLLAMA_HOST=http://host.docker.internal:11434
|
||||||
|
- PYTHONUNBUFFERED=1
|
||||||
|
volumes:
|
||||||
|
- ./outputs:/app/outputs
|
||||||
|
restart: unless-stopped
|
||||||
|
extra_hosts:
|
||||||
|
- "host.docker.internal:host-gateway"
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-f", "http://localhost:8010/health"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 3
|
||||||
|
start_period: 5s
|
||||||
|
|
||||||
35
docker-entrypoint.sh
Normal file
35
docker-entrypoint.sh
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
echo "Starting Ollama service..."
|
||||||
|
ollama serve &
|
||||||
|
|
||||||
|
# Wait for Ollama to be ready
|
||||||
|
echo "Waiting for Ollama to start..."
|
||||||
|
for i in {1..30}; do
|
||||||
|
if curl -s http://localhost:11434/api/tags > /dev/null 2>&1; then
|
||||||
|
echo "Ollama is ready!"
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
echo "Waiting for Ollama... ($i/30)"
|
||||||
|
sleep 2
|
||||||
|
done
|
||||||
|
|
||||||
|
# Check if models exist, if not, show warning
|
||||||
|
echo "Checking for models..."
|
||||||
|
ollama list
|
||||||
|
|
||||||
|
if ! ollama list | grep -q "qwen3:14b"; then
|
||||||
|
echo "WARNING: qwen3:14b model not found!"
|
||||||
|
echo "The application requires qwen3:14b to function properly."
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! ollama list | grep -q "qwen3-embedding"; then
|
||||||
|
echo "WARNING: qwen3-embedding model not found!"
|
||||||
|
echo "The application requires qwen3-embedding:4b for embeddings."
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Starting FastAPI application..."
|
||||||
|
exec uvicorn _qwen_xinference_demo.api:app --host 0.0.0.0 --port 8010
|
||||||
|
|
||||||
105
export-ollama-models.sh
Executable file
105
export-ollama-models.sh
Executable file
@@ -0,0 +1,105 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Export Ollama models for offline deployment
|
||||||
|
# This script copies Ollama models from your local machine
|
||||||
|
# so they can be bundled into the Docker image
|
||||||
|
#
|
||||||
|
# Required models:
|
||||||
|
# - qwen3:14b (main chat model)
|
||||||
|
# - qwen3-embedding:4b (embedding model)
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
MODELS_DIR="ollama-models"
|
||||||
|
OLLAMA_MODELS_PATH="$HOME/.ollama"
|
||||||
|
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Exporting Ollama models for offline deployment"
|
||||||
|
echo "=========================================="
|
||||||
|
|
||||||
|
# Check if Ollama is installed
|
||||||
|
if ! command -v ollama &> /dev/null; then
|
||||||
|
echo "ERROR: Ollama is not installed or not in PATH"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check if required models are available
|
||||||
|
echo ""
|
||||||
|
echo "Checking for required models..."
|
||||||
|
MISSING_MODELS=0
|
||||||
|
|
||||||
|
if ! ollama list | grep -q "qwen3:14b"; then
|
||||||
|
echo "ERROR: qwen3:14b model not found!"
|
||||||
|
echo "Please run: ollama pull qwen3:14b"
|
||||||
|
MISSING_MODELS=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! ollama list | grep -q "qwen3-embedding:4b"; then
|
||||||
|
echo "ERROR: qwen3-embedding:4b model not found!"
|
||||||
|
echo "Please run: ollama pull qwen3-embedding:4b"
|
||||||
|
MISSING_MODELS=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ $MISSING_MODELS -eq 1 ]; then
|
||||||
|
echo ""
|
||||||
|
echo "Please download the required models first:"
|
||||||
|
echo " ollama pull qwen3:14b"
|
||||||
|
echo " ollama pull qwen3-embedding:4b"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "✓ All required models found"
|
||||||
|
|
||||||
|
# Check if Ollama directory exists
|
||||||
|
if [ ! -d "$OLLAMA_MODELS_PATH" ]; then
|
||||||
|
echo "ERROR: Ollama directory not found at $OLLAMA_MODELS_PATH"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Create export directory
|
||||||
|
echo ""
|
||||||
|
echo "Creating export directory: $MODELS_DIR"
|
||||||
|
rm -rf "$MODELS_DIR"
|
||||||
|
mkdir -p "$MODELS_DIR"
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "Copying Ollama data from $OLLAMA_MODELS_PATH to $MODELS_DIR..."
|
||||||
|
echo "This may take several minutes (models are large)..."
|
||||||
|
|
||||||
|
# Copy the entire .ollama directory structure
|
||||||
|
cp -r "$OLLAMA_MODELS_PATH"/* "$MODELS_DIR/"
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Models exported successfully!"
|
||||||
|
echo "=========================================="
|
||||||
|
du -sh "$MODELS_DIR"
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "Directory structure:"
|
||||||
|
ls -lh "$MODELS_DIR/"
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "Models included:"
|
||||||
|
if [ -d "$MODELS_DIR/models/manifests/registry.ollama.ai/library" ]; then
|
||||||
|
ls -lh "$MODELS_DIR/models/manifests/registry.ollama.ai/library/"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Next steps:"
|
||||||
|
echo "=========================================="
|
||||||
|
echo "1. Build the all-in-one Docker image:"
|
||||||
|
echo " ./build-allinone.sh"
|
||||||
|
echo ""
|
||||||
|
echo "2. Or manually:"
|
||||||
|
echo " docker build -f Dockerfile.allinone -t system-prompt-optimizer:allinone ."
|
||||||
|
echo ""
|
||||||
|
echo "3. Export the image:"
|
||||||
|
echo " docker save -o system-prompt-optimizer-allinone.tar system-prompt-optimizer:allinone"
|
||||||
|
echo ""
|
||||||
|
echo "4. Transfer to target server:"
|
||||||
|
echo " scp system-prompt-optimizer-allinone.tar user@server:/path/"
|
||||||
|
echo ""
|
||||||
|
echo "Note: The final Docker image will be very large (10-20GB) due to the models."
|
||||||
|
|
||||||
@@ -6,7 +6,7 @@
|
|||||||
<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate">
|
<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate">
|
||||||
<meta http-equiv="Pragma" content="no-cache">
|
<meta http-equiv="Pragma" content="no-cache">
|
||||||
<meta http-equiv="Expires" content="0">
|
<meta http-equiv="Expires" content="0">
|
||||||
<title>OPRO - System Instruction Optimizer</title>
|
<title>系统提示词优化</title>
|
||||||
<script crossorigin src="https://unpkg.com/react@18/umd/react.production.min.js"></script>
|
<script crossorigin src="https://unpkg.com/react@18/umd/react.production.min.js"></script>
|
||||||
<script crossorigin src="https://unpkg.com/react-dom@18/umd/react-dom.production.min.js"></script>
|
<script crossorigin src="https://unpkg.com/react-dom@18/umd/react-dom.production.min.js"></script>
|
||||||
<script src="https://cdn.tailwindcss.com"></script>
|
<script src="https://cdn.tailwindcss.com"></script>
|
||||||
@@ -304,41 +304,55 @@
|
|||||||
createNewRun(msg);
|
createNewRun(msg);
|
||||||
}
|
}
|
||||||
|
|
||||||
async function handleContinueOptimize(selectedInstruction, selectedScore) {
|
async function handleContinueOptimize(selectedInstruction, allCandidates) {
|
||||||
if (!currentRunId || loading) return;
|
if (!currentRunId || loading || !selectedInstruction) return;
|
||||||
|
|
||||||
// First, evaluate the selected instruction to add it to trajectory
|
setLoading(true);
|
||||||
if (selectedInstruction) {
|
try {
|
||||||
setLoading(true);
|
// Get rejected instructions (all except the selected one)
|
||||||
try {
|
const rejectedInstructions = allCandidates
|
||||||
// Add the selected instruction to trajectory
|
.map(c => c.instruction)
|
||||||
const res = await fetch(`${API_BASE}/opro/evaluate`, {
|
.filter(inst => inst !== selectedInstruction);
|
||||||
method: 'POST',
|
|
||||||
headers: { 'Content-Type': 'application/json' },
|
|
||||||
body: JSON.stringify({
|
|
||||||
run_id: currentRunId,
|
|
||||||
instruction: selectedInstruction
|
|
||||||
})
|
|
||||||
});
|
|
||||||
const data = await res.json();
|
|
||||||
|
|
||||||
if (!data.success) {
|
// Call the refinement endpoint
|
||||||
throw new Error(data.error || 'Failed to evaluate instruction');
|
const res = await fetch(`${API_BASE}/opro/refine`, {
|
||||||
}
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({
|
||||||
|
run_id: currentRunId,
|
||||||
|
selected_instruction: selectedInstruction,
|
||||||
|
rejected_instructions: rejectedInstructions
|
||||||
|
})
|
||||||
|
});
|
||||||
|
const data = await res.json();
|
||||||
|
|
||||||
console.log('Evaluated instruction, score:', data.data.score);
|
if (!data.success) {
|
||||||
} catch (err) {
|
throw new Error(data.error || 'Failed to refine instruction');
|
||||||
alert('评估指令失败: ' + err.message);
|
|
||||||
console.error('Error evaluating instruction:', err);
|
|
||||||
setLoading(false);
|
|
||||||
return;
|
|
||||||
} finally {
|
|
||||||
setLoading(false);
|
|
||||||
}
|
}
|
||||||
}
|
|
||||||
|
|
||||||
// Then generate new candidates based on updated trajectory
|
// Add refined candidates to messages
|
||||||
await generateCandidates(currentRunId);
|
const newMessage = {
|
||||||
|
role: 'assistant',
|
||||||
|
type: 'candidates',
|
||||||
|
iteration: data.data.iteration,
|
||||||
|
candidates: data.data.candidates
|
||||||
|
};
|
||||||
|
|
||||||
|
setMessages(prev => {
|
||||||
|
const updated = [...prev, newMessage];
|
||||||
|
// Save to session messages
|
||||||
|
setSessionMessages(prevSessions => ({
|
||||||
|
...prevSessions,
|
||||||
|
[currentSessionId]: updated
|
||||||
|
}));
|
||||||
|
return updated;
|
||||||
|
});
|
||||||
|
} catch (err) {
|
||||||
|
alert('优化失败: ' + err.message);
|
||||||
|
console.error('Error refining instruction:', err);
|
||||||
|
} finally {
|
||||||
|
setLoading(false);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
function handleExecute(instruction) {
|
function handleExecute(instruction) {
|
||||||
@@ -495,7 +509,7 @@
|
|||||||
// Header
|
// Header
|
||||||
React.createElement('div', { className: 'px-4 py-3 border-b border-gray-200 bg-white flex items-center gap-3' },
|
React.createElement('div', { className: 'px-4 py-3 border-b border-gray-200 bg-white flex items-center gap-3' },
|
||||||
React.createElement('h1', { className: 'text-lg font-normal text-gray-800' },
|
React.createElement('h1', { className: 'text-lg font-normal text-gray-800' },
|
||||||
'OPRO'
|
'系统提示词优化'
|
||||||
),
|
),
|
||||||
currentSessionId && React.createElement('div', { className: 'text-sm text-gray-500' },
|
currentSessionId && React.createElement('div', { className: 'text-sm text-gray-500' },
|
||||||
sessions.find(s => s.session_id === currentSessionId)?.session_name || '当前会话'
|
sessions.find(s => s.session_id === currentSessionId)?.session_name || '当前会话'
|
||||||
@@ -537,7 +551,7 @@
|
|||||||
),
|
),
|
||||||
React.createElement('div', { className: 'flex gap-2' },
|
React.createElement('div', { className: 'flex gap-2' },
|
||||||
React.createElement('button', {
|
React.createElement('button', {
|
||||||
onClick: () => handleContinueOptimize(cand.instruction, cand.score),
|
onClick: () => handleContinueOptimize(cand.instruction, msg.candidates),
|
||||||
disabled: loading,
|
disabled: loading,
|
||||||
className: 'px-4 py-2 bg-white border border-gray-300 text-gray-700 rounded-lg hover:bg-gray-50 disabled:bg-gray-100 disabled:text-gray-400 disabled:cursor-not-allowed transition-colors text-sm font-medium'
|
className: 'px-4 py-2 bg-white border border-gray-300 text-gray-700 rounded-lg hover:bg-gray-50 disabled:bg-gray-100 disabled:text-gray-400 disabled:cursor-not-allowed transition-colors text-sm font-medium'
|
||||||
}, '继续优化'),
|
}, '继续优化'),
|
||||||
|
|||||||
7
requirements.txt
Normal file
7
requirements.txt
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
fastapi==0.109.0
|
||||||
|
uvicorn==0.27.0
|
||||||
|
requests==2.31.0
|
||||||
|
numpy==1.26.3
|
||||||
|
scikit-learn==1.4.0
|
||||||
|
pydantic==2.5.3
|
||||||
|
|
||||||
Reference in New Issue
Block a user