149 lines
3.3 KiB
Markdown
149 lines
3.3 KiB
Markdown
# 快速开始指南
|
||
|
||
## 安装依赖
|
||
|
||
1. 激活虚拟环境:
|
||
```bash
|
||
source venv/bin/activate
|
||
```
|
||
|
||
2. 安装依赖:
|
||
```bash
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
## 基本使用
|
||
|
||
### 1. 创建 RAG 类实例
|
||
|
||
```python
|
||
from base_rag.core import BaseRAG, FileStatus
|
||
|
||
class MyRAG(BaseRAG):
|
||
def ingest(self, file_path: str, **kwargs):
|
||
return self.process_file_to_vector_store(file_path, **kwargs)
|
||
|
||
def query(self, question: str) -> str:
|
||
docs = self.similarity_search_with_rerank(question)
|
||
if not docs:
|
||
return "没有找到相关信息"
|
||
return "\n".join([doc.page_content for doc in docs])
|
||
|
||
# 创建实例
|
||
rag = MyRAG(
|
||
vector_store_name="my_kb", # 知识库名称
|
||
storage_directory="./documents", # 文件存储目录
|
||
status_db_path="./file_status.db" # 状态数据库
|
||
)
|
||
```
|
||
|
||
### 2. 处理文件
|
||
|
||
```python
|
||
# 处理单个文件
|
||
result = rag.ingest("path/to/your/document.txt")
|
||
print(f"处理结果: {result['message']}")
|
||
|
||
# 批量处理文件
|
||
import os
|
||
for filename in os.listdir("./documents"):
|
||
if filename.endswith(('.txt', '.md', '.doc', '.docx')):
|
||
result = rag.ingest(f"./documents/{filename}")
|
||
print(f"{filename}: {result['message']}")
|
||
```
|
||
|
||
### 3. 查询知识库
|
||
|
||
```python
|
||
# 搜索相关文档
|
||
answer = rag.query("你的问题")
|
||
print(answer)
|
||
```
|
||
|
||
### 4. 查看文件状态
|
||
|
||
```python
|
||
# 查看所有文件状态
|
||
all_files = rag.get_file_processing_status()
|
||
for file_info in all_files:
|
||
print(f"{file_info['filename']}: {file_info['status']}")
|
||
|
||
# 查看已完成的文件
|
||
completed = rag.list_files_by_status(FileStatus.COMPLETED)
|
||
print(f"已处理完成: {len(completed)} 个文件")
|
||
|
||
# 查看处理失败的文件
|
||
failed = rag.list_files_by_status(FileStatus.ERROR)
|
||
for file_info in failed:
|
||
print(f"失败文件: {file_info['filename']}")
|
||
print(f"错误信息: {file_info['error_message']}")
|
||
```
|
||
|
||
## 支持的文件格式
|
||
|
||
- **.txt** - 纯文本文件
|
||
- **.md** - Markdown 文件
|
||
- **.doc/.docx** - Word 文档(需要安装 `unstructured` 和 `python-docx`)
|
||
|
||
## 主要特性
|
||
|
||
1. **自动去重**:相同内容的文件不会重复处理
|
||
2. **状态跟踪**:实时跟踪文件处理状态
|
||
3. **错误处理**:处理失败的文件会记录错误信息
|
||
4. **简单API**:易于使用和扩展
|
||
5. **持久化存储**:使用 SQLite 数据库记录状态
|
||
|
||
## 运行示例
|
||
|
||
```bash
|
||
# 激活环境
|
||
source venv/bin/activate
|
||
|
||
# 运行完整示例
|
||
python examples/file_processing_example.py
|
||
|
||
# 运行简单测试
|
||
python examples/simple_test.py
|
||
```
|
||
|
||
## 配置选项
|
||
|
||
### 文档切分参数
|
||
```python
|
||
result = rag.ingest(
|
||
"document.txt",
|
||
chunk_size=500, # 切分块大小
|
||
chunk_overlap=50 # 重叠大小
|
||
)
|
||
```
|
||
|
||
### 嵌入模型配置
|
||
```python
|
||
rag = MyRAG(
|
||
embedding_config={
|
||
"type": "local",
|
||
"model_name": "BAAI/bge-small-zh-v1.5"
|
||
}
|
||
)
|
||
```
|
||
|
||
### 重排模型配置
|
||
```python
|
||
rag = MyRAG(
|
||
rerank_config={
|
||
"enabled": True,
|
||
"type": "local",
|
||
"model": "BAAI/bge-reranker-base",
|
||
"top_k": 3
|
||
}
|
||
)
|
||
```
|
||
|
||
## 数据存储
|
||
|
||
- **文件存储**:`./documents/` 目录(可配置)
|
||
- **向量数据库**:`./chroma_db/` 目录
|
||
- **状态数据库**:`./file_status.db` 文件
|
||
|
||
文件名格式:`原文件名_哈希值前8位.扩展名`
|