base_rag/QUICK_START.md

149 lines
3.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# 快速开始指南
## 安装依赖
1. 激活虚拟环境:
```bash
source venv/bin/activate
```
2. 安装依赖:
```bash
pip install -r requirements.txt
```
## 基本使用
### 1. 创建 RAG 类实例
```python
from base_rag.core import BaseRAG, FileStatus
class MyRAG(BaseRAG):
def ingest(self, file_path: str, **kwargs):
return self.process_file_to_vector_store(file_path, **kwargs)
def query(self, question: str) -> str:
docs = self.similarity_search_with_rerank(question)
if not docs:
return "没有找到相关信息"
return "\n".join([doc.page_content for doc in docs])
# 创建实例
rag = MyRAG(
vector_store_name="my_kb", # 知识库名称
storage_directory="./documents", # 文件存储目录
status_db_path="./file_status.db" # 状态数据库
)
```
### 2. 处理文件
```python
# 处理单个文件
result = rag.ingest("path/to/your/document.txt")
print(f"处理结果: {result['message']}")
# 批量处理文件
import os
for filename in os.listdir("./documents"):
if filename.endswith(('.txt', '.md', '.doc', '.docx')):
result = rag.ingest(f"./documents/{filename}")
print(f"{filename}: {result['message']}")
```
### 3. 查询知识库
```python
# 搜索相关文档
answer = rag.query("你的问题")
print(answer)
```
### 4. 查看文件状态
```python
# 查看所有文件状态
all_files = rag.get_file_processing_status()
for file_info in all_files:
print(f"{file_info['filename']}: {file_info['status']}")
# 查看已完成的文件
completed = rag.list_files_by_status(FileStatus.COMPLETED)
print(f"已处理完成: {len(completed)} 个文件")
# 查看处理失败的文件
failed = rag.list_files_by_status(FileStatus.ERROR)
for file_info in failed:
print(f"失败文件: {file_info['filename']}")
print(f"错误信息: {file_info['error_message']}")
```
## 支持的文件格式
- **.txt** - 纯文本文件
- **.md** - Markdown 文件
- **.doc/.docx** - Word 文档(需要安装 `unstructured``python-docx`
## 主要特性
1. **自动去重**:相同内容的文件不会重复处理
2. **状态跟踪**:实时跟踪文件处理状态
3. **错误处理**:处理失败的文件会记录错误信息
4. **简单API**:易于使用和扩展
5. **持久化存储**:使用 SQLite 数据库记录状态
## 运行示例
```bash
# 激活环境
source venv/bin/activate
# 运行完整示例
python examples/file_processing_example.py
# 运行简单测试
python examples/simple_test.py
```
## 配置选项
### 文档切分参数
```python
result = rag.ingest(
"document.txt",
chunk_size=500, # 切分块大小
chunk_overlap=50 # 重叠大小
)
```
### 嵌入模型配置
```python
rag = MyRAG(
embedding_config={
"type": "local",
"model_name": "BAAI/bge-small-zh-v1.5"
}
)
```
### 重排模型配置
```python
rag = MyRAG(
rerank_config={
"enabled": True,
"type": "local",
"model": "BAAI/bge-reranker-base",
"top_k": 3
}
)
```
## 数据存储
- **文件存储**`./documents/` 目录(可配置)
- **向量数据库**`./chroma_db/` 目录
- **状态数据库**`./file_status.db` 文件
文件名格式:`原文件名_哈希值前8位.扩展名`