Compare commits

...
12 Commits
Author SHA1 Message Date
sansenhoshiandClaude Code 614e1d7e00 chore(bf-bot): 更新 BF 系列通知图
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-23 18:47:17 +08:00
sansenhoshiandClaude Code fc2e542418 chore(deps): 加载 nonebot_plugin_doroending
插件已装进 venv,补进 pyproject 的 plugins 列表才会被 nonebot 加载。
已用 bot.py 同款初始化流程(不含 nonebot.run)验证:全部插件加载成功,
doroending 注册 4 个 matcher。

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-23 18:47:17 +08:00
sansenhoshiandClaude Code 023dc1cb5d feat(web): 停止/重启 bot 的等待相位全局化 / 覆盖层收进 BotActionProvider
重启入口有两个 —— 顶栏账号菜单(任何页面可点)和设置页「进程控制」。以前两边
各写一套:设置页只有本页的 fixed 覆盖层,顶栏只弹个 toast。于是从插件页/日志页
点「重启 bot」时界面照常可点,请求全打在已经退出的旧进程上。

- 新增 src/botAction.tsx:Provider 持 phase(null / stop / restart)与 readyCount,
  覆盖层由 AppLayout 在 <Outlet/> 之外统一渲染,任何页面触发都盖得住;
  纯等待逻辑仍在 lib/botAction.ts,两边共用一份
- 设置页删掉本地的 restarting 态与两份覆盖层,只负责发起 + 用 phase 禁用按钮;
  改为消费 readyCount 在回连后重新拉数据(停机期间配置是旧的)
- 未挂载 Provider 时 run() 直接抛错,避免静默无覆盖层

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-23 18:47:15 +08:00
sansenhoshiandClaude Code 574216645c feat(video-analysis): 本地直发成功且群策略要下载链接时补传 S3
本地直发经 hexi/core/outbound_media 钩子改写后已真正可用,此前它因协议端不在本机
恒失败,public_url 因此总有值(每次实际都落到第 2 级 S3)。本地通道通了以后若不补,
public_url 会恒为 None,entry.py 的 `if pub and policy.sends_link` 会让「发送下载
链接」静默失效。

- _local_sent / _public_link_for 把「取链接」与「发送」解耦:只有策略真要链接
  (policy.sends_link)时才补一次 S3 上传,其余情况一概不碰 S3 —— 媒体不再进
  S3 归档是有意的,temp 保留交给 cleanup
- 补传失败只记日志返回 None:发送本身已经成功,不能反过来判失败
- tests/test_video_sender_link.py:覆盖 sends_link 开/关、多文件逐个补传、
  拿不到 public_url、上传抛异常、policy 为 None 五条路径

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-23 18:47:12 +08:00
sansenhoshiandClaude Code ed75a66982 feat(core): 出站媒体内联化全局钩子 / 本地文件改写为协议端可达 URL
协议端不在本机,而 NoneBot 的 f2s() 会把 Path 转成 file:///D:/...(file:// 的
语义是「协议端那台机器上的路径」),必然 ENOENT。受影响的不止本仓调用点,还有
装不了改不动的 pip 社区插件(本次报错来自 nonebot_plugin_doroending),只能全局拦截。

- 新增 hexi/core/outbound_media.py:挂 Bot.on_calling_api(公开钩子、适配器不覆盖),
  在真正调用适配器前原地改写发送类 API 的 data:默认改写成
  http://<本机可达IP>:<端口>/media/<token> 由协议端主动回拉(地址与 /hub 首页
  「协议端配对」同源,零新配置),拿不到可达地址时回退 base64://
- 文件上传类(upload_group_file / upload_private_file)单独处理:file 在顶层且
  只做 URL 不做 base64(几十上百 MB 不适合内联),覆盖 video_analysis 群文件
  「本地直传」兜底与 alconna 的 $onebot11:file
- 媒体路由 get_app().mount("/media", sub) 复用现有端口不新开服务;免鉴权是必须的
  (协议端登录不了 hub),安全模型改为「路径由 bot 改写那一刻自己登记、请求方只能
  出示不可猜 token」,配短 TTL;_mounted 为 False 时绝不发 URL
- 钩子内每个段独立 try/except,任何异常只告警不抛,绝不反过来把发送搞挂
- message_utils.common_proc_reply 占位图不再自己转 base64,改传 path= 交给钩子,
  避免两处各有一套策略
- tests/test_outbound_media.py:用 tmp_path 造真实文件而非 mock 文件系统,重点覆盖
  file:// 解析(空格/中文编码、手工未转义的字面 %XX)与 walker 各消息形态

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-23 18:47:09 +08:00
sansenhoshiandClaude Code 4b5e8b8b2e feat(web): 顶栏账号菜单接入 bot 启停 / 插件页分区 / 首页协议端配对信息
- 重启/关闭 bot 从设置页提到顶栏账号下拉(破坏性操作弹确认框),等待回连逻辑抽成
  lib/botAction.ts 供两处共用:先等旧进程让出端口,再以 dashboard SSE 真收到一帧为准,
  只看 HTTP 200 会在"端口起来但插件未挂接"时误判就绪
- 插件管理页按 带 Web 页面 / 带配置项 / 都没有 分区,卡片加类型与 Web/配置 徽章
- 首页「BOT 协议端信息」分两半:左半登录账号,右半 NoneBot 接入信息
  (IP / 运行端口 / WS 路径 / 访问令牌,均可一键复制,另给整条反向 WS 地址),
  监听地址非 0.0.0.0 时提示仅本机可连;「BOT 后端信息」移到自己一行
- dashboard.py 新增 endpoint 段:读 driver.config 的 host/port、驱动实际注册的
  反向 WS 路由(自动跳过根路径与尾斜杠别名)、适配器令牌与本机网卡 IP

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-22 14:23:52 +08:00
sansenhoshiandClaude Code 814c1994b6 feat(mc-server-status): 服务器列表 Web 管理页 / 数据层抽离 / mcstatus 13 适配
- services/servers.py:校验 + 读写 + 增删改收敛成一份数据层(顶层不 import nonebot,
  可脱离运行时单测),群命令与 Web API 都只经它改数据,校验不再各写一套
- web_hub.py + /hub/mc_server_status 页面:按群列出服务器,抽屉新增/编辑(改名)/
  测试连接,行内删除(删空一台连带删群);每条路由自带 require_admin
- handlers/server.py 改为薄封装,只在需要命令特有提示时自己判断
- services/mc.py 的图标属性名兼容 mcstatus ≥13 的 icon(旧版叫 favicon,
  读错会把整条查询打成"查询失败")
- tests/test_mc_servers.py:数据层归一/改名/删群/坏文件兜底 + 图标解码两版属性名

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-22 14:23:46 +08:00
sansenhoshiandClaude Code 4badcfcf32 feat(video-analysis): 群策略 v3 / 群文件投递通道 / Web 管理页
- policy.py:per-group 正交策略(自动解析 / 自动策略 / 禁用策略 / 存储 A·B·C /
  公网 / 链接 / 群文件 + 平台限定),list.json v1/v2 → v3 自动迁移,
  写入统一走 PolicyStore(加锁 + .tmp 原子替换 + 字段归一)
- 群文件并行通道 group_file.py:打包 zip(可选 pyzipper AES-256)后优先走 S3 预签名、
  本地直传兜底;设了密码但 pyzipper 不可用就放弃上传,不退化成明文
- list_proc.py 收敛到「视频策略」统一入口,权限判定改走 policy
- Web 管理页 /hub/video_analysis(群策略 + 链接解析面板)与 services/web_jobs.py
  (只复用纯函数层,Web 上下文不发消息;内存任务表 + 并发闸门 + 超时)
- 媒体命名统一到 utils.py({作者}_{作者id}/{作品名}[_短码]),cleanup 回收空目录
- 测试:policy / 命名 / 群文件 / web_jobs 四组

顺带 pyproject 的 pytest 加 testpaths=tests(避免收进 debug/ 下的调试脚本)。

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-22 14:23:32 +08:00
sansenhoshiandClaude Code 51b08ccb68 feat: 群分析本地消息库 / 处理中占位图 / Web 日志窗口重构
- 群分析: 新增 history_store 只读本地消息源(读 learning_chat 落库,含 uninfo
  昵称补齐/去重/截断判定),适配器优先读本地库、失败回退 OneBot 分页;分页
  锚点字段回退链修复 NapCat 传 message_seq 翻页断裂;新增「本地记录」开关
- core/message_utils: 新增 common_proc_reply 占位图通用回复(引用消息 + 处理中
  动图,支持后台任务显式指定 target),群分析/战况/倒放改用
- web_hub + web: 日志页改固定窗口滚动 + 翻页锚定 + 自动换行,SSE 日志轮转发
  reset 帧,入口 HTML no-cache,行数统计增量缓存,CPU 改非阻塞采样,登录信息缓存
- 插件内 CLAUDE.md / DESIGN.md 不入库(.gitignore),galgame_card 两份文档取消
  跟踪(文件保留在磁盘)

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-14 18:59:01 +08:00
sansenhoshiandClaude Code 662c28cb2f chore(deps): 补齐运行时依赖声明
代码里直接 import、但 requirements.txt / pyproject.toml 都没声明的包补齐:
beautifulsoup4、opencv-contrib-python、curl_cffi、yt-dlp、boto3、cloudscraper、
cjk-textwrap、lxml、jieba、ruamel.yaml、pytz、psutil、py-cpuinfo、numpy、
nonebot-plugin-htmlrender、taskgroup(仅 py<3.11)。
其中多数此前只是靠别的包传递依赖带上来的(如 curl_cffi 来自 yt-dlp 的 extra),
换环境或升级就会缺。

同时清理无任何引用的 chardet、gradio_client、aiohttp-retry、qrcode,
并把 playwright 下限与 pyproject 对齐(1.44 → 1.50)。

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-14 18:58:52 +08:00
sansenhoshi 131b92b319 结构调整
视频解析多图/多媒体结构 消息体适配
2026-09-08 14:25:32 +08:00
sansenhoshiandClaude Code 30899688a7 feat(video-analysis): 多文件媒体改走合并转发
抖音 note/小红书等多图、图+视频混合、多视频(动图)作品,下载产物
超过一个文件时直接拼成一条消息发送会失败(视频段不能与其他段混合),
统一改为合并转发: 一个图/视频一个节点,本地文件优先、S3 预签名链接
兜底; 单文件作品保持原有直发链路,多级发送链(本地→S3→回退)不变。

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-08 14:21:57 +08:00
128 changed files with 9459 additions and 1530 deletions
+4 -1
View File
@@ -78,7 +78,10 @@ bg.jpg
/.ai/
/.claude/
/hexi/config/
/CLAUDE.md
# ---- 开发文档(插件内 CLAUDE.md / DESIGN.md,含根目录,不入库) ----
**/CLAUDE.md
**/DESIGN.md
# helldivers 图标素材&生成脚本(不入库)
/dev/docs/HD2/
+6 -4
View File
@@ -34,13 +34,13 @@
| 插件 | 功能 | 主要命令 |
| --- | --- | --- |
| `nonebot_plugin_hexi_core` | 核心库:消息解析、权限分级、限流器 (token bucket)、命令冷却、热插件管理 | — |
| `hexi/core`(核心模块) | 核心库:消息解析、权限分级、限流器 (token bucket)、命令冷却、热插件管理(**非插件**) | — |
| `nonebot_plugin_video_analysis` | 视频链接解析 | — |
| `nonebot_plugin_regif` | GIF 注册与处理 | — |
| `nonebot_plugin_picfinder_take` | 识图(Playwright 驱动) | — |
| `nonebot_plugin_picstatus` | 头像状态图生成 | — |
| `nonebot_plugin_group_daily_analysis` | 群聊行为分析报告 | `/群分析` |
| `nonebot_plugin_web_hub` | 统一 Web 管理入口 | — |
| `hexi/web_hub`(核心模块) | 统一 Web 管理入口 `/hub`(**非插件**) | — |
## 快速开始
@@ -88,13 +88,15 @@ poetry run python bot.py
HeXi/
├── bot.py pyproject.toml README.md
├── hexi/
│ ├── plugins/ # 全部插件(nonebot.load_plugins("hexi") 加载)
│ ├── plugins/ # 全部插件(pyproject [tool.nonebot] plugin_dirs 加载)
│ ├── core/ # 机器人核心模块(非插件):工具库/插件控制面/热拔插管理
│ ├── web_hub/ # 机器人核心模块:统一 Web 管理台 + 鉴权/插件注册/配置标准
│ ├── data/ # 运行数据(不入库)
│ ├── config/ # 业务配置(不入库)
│ ├── resource/ # 全局静态资源(fonts/ 字体、menu_config 等)
│ ├── web/ # 统一 Web 前端(React + Vite,构建产物 dist/ 供 /hub 挂载)
│ ├── logs/ # 日志(不入库)
│ └── web_*.py # Web 管理相关(鉴权、插件注册)
│ └── web_hub/ # 见上:统一 Web 管理台/鉴权/插件注册/配置标准
├── docs/ # 插件开发标准文档
├── tests/ # pytest 测试
└── plugin_template/ # 新插件脚手架模板
+6 -2
View File
@@ -55,10 +55,14 @@ driver = nonebot.get_driver()
# 注册适配器
driver.register_adapter(ONEBOTV11Adapter)
# 加载插件
nonebot.load_plugins("hexi")
# 加载插件(hexi/plugins 下的自定义插件 + pyproject 声明的社区插件)
nonebot.load_from_toml("pyproject.toml")
# 核心模块(已从 hexi/plugins 移出,不再是插件):在插件管理器建立后再导入,
# 避免在插件管理器创建前 require 社区插件导致重复注册/“not loaded as a plugin”。
import hexi.core # noqa: F401,E402
import hexi.web_hub # noqa: F401,E402
if __name__ == "__main__":
logger.warning("hexi?启动!")
nonebot.run() # 直接运行,不需要传参
+2
View File
@@ -3,6 +3,8 @@
> 依据 《插件结构标准》(docs/plugin-structure-standard.md) 与 《插件配置文件标准》(docs/plugin-config-standard.md),
> 并参照脚手架模板 `../plugin_template/nonebot_plugin_template`。
> 说明:初版为**审计报告**;后续按用户确认已开始落地改造(见下方「本轮已落地改动」)。
>
> **后续变动**:其中 `hexi_core` 与 `web_hub` 已从 `hexi/plugins/` 移出,改为机器人核心模块 `hexi/core`、`hexi/web_hub`(非插件),本报告中这两项的“插件”表述仅供参考。
---
+16 -12
View File
@@ -20,8 +20,12 @@
```text
hexi/
├── web_config.py # 配置标准核心:schema 注册 / 值库 / 保存热刷新
├── config_standard.py # pydantic Config 一键接入的辅助封装
├── web_hub/ # 统一 Web 管理台 / 鉴权 / 配置标准
│ ├── web_config.py # 配置标准核心:schema 注册 / 值库 / 保存热刷新
│ ├── config_standard.py # pydantic Config 一键接入的辅助封装
│ ├── web_auth.py # 统一 Web 鉴权(OAuth2 + SQLite)
│ ├── web_plugin_registry.py # 统一 Web 插件注册中心
│ └── web_hub_auth.py # 兼容 shim
├── config/
│ └── plugin_config.json # 统一值库:{ "<plugin_id>": { key: value, ... } }
└── web/ # /hub 前端(通用 schema 表单渲染器)
@@ -143,7 +147,7 @@ hexi/
```python
# 在插件 __init__.py 里
from hexi.config_standard import register_model_config
from hexi.web_hub.config_standard import register_model_config
from .config import config
register_model_config(
@@ -168,7 +172,7 @@ register_model_config(
### 方式 B:非 pydantic 插件手动注册
```python
from hexi.web_config import register_plugin_config
from hexi.web_hub.web_config import register_plugin_config
def _get(): # 返回当前生效值 dict
return {"field": get_my_cur_value("field")}
@@ -209,7 +213,7 @@ register_plugin_config(
2. **写值库** `plugin_config.json`(原子写)。
3. **写 .env**:非 list 字段写入 `os.environ` + `../../.env`(保证重启仍生效);list/path/object 跳过,避免 `str(list)` 破坏重启解析。
4. **调 `apply(values)`**:把值热应用到插件运行态对象(list/path/object 也在此生效)。
5. 若插件未提供 apply,则 `hot_reload(plugin_id)` 让插件重载。
5. 若插件未提供 apply,则 `hot_reload(plugin_id)` 让插件重载;**library/非 application 插件不受热重载**(见《插件类型分类》,热拔插会破坏依赖它的插件)。
**为什么 apply 与 env 都做**:NoneBot 的 `get_driver().config` 在启动时即固定,重载插件也读不到新 env;所以运行期必须 apply,重启靠 env。
@@ -220,7 +224,7 @@ register_plugin_config(
| 方法 | 路径 | 说明 |
|---|---|---|
| GET | `/api/plugins` | 所有已注册 Web 插件 |
| GET | `/api/plugins/catalog` | 所有应用插件 + `has_config/has_web/web_path` |
| GET | `/api/plugins/catalog` | 所有 application/library 插件(含带配置项的 library)+ `type/has_config/has_web/web_path` |
| GET | `/api/plugins/{id}/config` | 读 `{schema, values, revision}`(登录);secret 值只返回 `****` |
| POST | `/api/plugins/{id}/config` | 保存 `{ "revision": n, "values": { key: value } }`;冲突 409,校验失败 422 |
| DELETE | `/api/plugins/{id}/config` | 清空该插件覆盖,恢复默认(可选) |
@@ -242,7 +246,7 @@ register_plugin_config(
## 9. 权限与安全
- 全部配置接口走 `hexi.web_auth.require_admin`(OAuth2 + SQLite)。
- 全部配置接口走 `hexi.web_hub.web_auth.require_admin`(OAuth2 + SQLite)。
- 配置 POST 必须携带 GET 返回的 `revision`;缺失返回 428,冲突返回 409。
- 只允许 `plugin_id` 存在于注册表,未注册返回 `ok:false`(防任意写入)。
- 部署时必须显式设置 Web 管理员凭据;禁止生产环境使用默认的 `admin/admin`。
@@ -277,7 +281,7 @@ register_plugin_config(
```python
# hexi/plugins/nonebot_plugin_helldivers_tools/__init__.py
from hexi.config_standard import register_model_config
from hexi.web_hub.config_standard import register_model_config
from .config import config as _hd2_config
register_model_config(
@@ -295,7 +299,7 @@ register_model_config(
## 12. 已接入与待接入
- **已接入**:`helldivers_tools`、`mc_server_status`、`video_analysis`、`steam_info`、`picstatus`、`galgame_card`。
- **未接入(建议后续)**:`learning_chat`、`group_daily_analysis`(已有独自 Web 配置,避免冲突)、`picfinder_take`、`bf_bot`、`group_tools`、`hexi_core`(模块级常量/SUPERUSERS,运行期热更复杂)。
- **未接入(建议后续)**:`learning_chat`、`group_daily_analysis`(已有独自 Web 配置,避免冲突)、`picfinder_take`、`bf_bot`、`group_tools`、`hexi/core`(模块级常量/SUPERUSERS,运行期热更复杂)。
---
@@ -316,7 +320,7 @@ register_model_config(
### 13.2 通用注册 API(任意来源)
```python
from hexi.config_standard import register_config_items
from hexi.web_hub.config_standard import register_config_items
register_config_items(
__name__, # = NoneBot 插件模块名
@@ -339,7 +343,7 @@ register_config_items(
Web 保存后 apply 会把新值写回来源(模块属性/dict/自定义),但若插件在别处是用 `from .config import X` **值拷贝**进来的量,不受影响。要真正运行期生效,插件在读配置处改用统一 API:
```python
from hexi.web_config import get_effective_value
from hexi.web_hub.web_config import get_effective_value
limit = get_effective_value("hexi.plugins.nonebot_plugin_picfinder_take", "DAILY_LIMIT", 50)
if not check_quota(limit): ...
@@ -395,7 +399,7 @@ if not check_quota(limit): ...
### 14.3 一键注册 API
```python
from hexi.config_standard import register_object_set
from hexi.web_hub.config_standard import register_object_set
register_object_set(
__name__, # NoneBot 插件模块名
+5 -4
View File
@@ -84,6 +84,7 @@ nonebot_plugin_xxx/
## 4. `__init__.py` 标准职责(固定顺序)
1. `__plugin_meta__ = PluginMetadata(name=..., description=..., usage=..., type="application")`。
> **type 取值(NoneBot 发布规范)**:`application`(向机器人用户提供功能,支持热插拔/热重载)或 `library`(为其他插件提供能力,不可热拔插)。新插件默认 `application`。本仓库的 `hexi/core`、`hexi/web_hub` 已从插件目录移出,作为机器人核心模块(**非插件**),直接不参与热拔插。
2. 声明依赖 `require("nonebot_plugin_alconna")` 等。
3. 显式导入子模块(`from . import handlers, services, utils`),注册 matcher/handler。
4. **配置注册**(在 config 就绪后):`register_model_config`(pydantic Config)或 `register_config_items` / `register_object_set`(来源无关)。
@@ -108,7 +109,7 @@ config = get_plugin_config(Config)
### 4.2 来源无关型(模块常量 / 配置文件 / DB)
```python
from hexi.config_standard import register_config_items, register_object_set
from hexi.web_hub.config_standard import register_config_items, register_object_set
register_object_set(__name__, "groups", [ {item_schema...} ], key_field="group_id",
getter=get_groups, setter=set_groups, nosave=True)
```
@@ -133,20 +134,20 @@ register_object_set(__name__, "groups", [ {item_schema...} ], key_field="group_i
- `services/`:业务编排,可依赖 repository / 外部 API;把「一个业务动作」收敛到一个方法。
- **禁止** `from x import *`、`except: pass`、`print()`;用 `logger`。
- async handler 内避免同步阻塞(`requests` / `time.sleep` / 同步爬虫);需要就 `asyncio.to_thread`。
- 命令触发:需要 @ 用 `rule=to_me()`;全局命令注意与其它插件冲突;高开销命令加冷却/限频(`hexi_core` 的 `cooldown/rate_limit`)。
- 命令触发:需要 @ 用 `rule=to_me()`;全局命令注意与其它插件冲突;高开销命令加冷却/限频(`hexi/core` 的 `cooldown/rate_limit`)。
- 外部 API:统一超时 + 重试 + 失败降级/用户提示。
## 8. Web 接入标准
- 需要独立 Web 页:`register_web_plugin(id, name, icon, lambda: build_app(), module_name=__name__)`,hub 启动自动挂载 `/api/<id>`。
- 需要 Web 配置:`register_plugin_config` / `register_model_config` / `register_config_items` / `register_object_set`(schema 驱动表单)。
- 鉴权统一用 `hexi.web_auth.require_admin`(OAuth2 + SQLite),不要自造一套。
- 鉴权统一用 `hexi.web_hub.web_auth.require_admin`(OAuth2 + SQLite),不要自造一套。
- 敏感字段 `secret=True`,前端掩码;写回允许明文。
## 9. 插件迁移检查清单
- [ ] 包名改为 `nonebot_plugin_*`,位置在 `../../hexi/plugins`。
- [ ] 有 `__plugin_meta__`,`type="application"`。
- [ ] 有 `__plugin_meta__`,`type="application"`(核心/框架插件标 `library`,不参与热拔插)。
- [ ] 配置已接入统一标准(Web 能读能改,运行期/重启生效)。
- [ ] 命令在 `handlers/`,业务在 `services/`,工具在 `utils/`。
- [ ] 数据访问集中(repository / services),原子写 / 事务。
+75
View File
@@ -0,0 +1,75 @@
# HeXi 插件类型分类(library / application)
> 依据 NoneBot 发布规范:`type` 是插件类别,发布必填。当前有效类别:
> - `library`:为其他插件编写提供功能(核心库/框架),**不参与热拔插/热重载**。
> - `application`:向机器人用户提供功能,**可热拔插/热重载**。
>
> 本仓库自定义插件统一在 `hexi/plugins/`(经 `nonebot.load_plugins("hexi")` 加载);
> pip 社区插件由其自带元数据决定类别,本仓库不改动,但热拔插管理器会按类别保护。
>
> **变动**:原 `nonebot_plugin_hexi_core` / `nonebot_plugin_web_hub` 已从 `hexi/plugins/` 移出,现为机器人核心模块 `hexi/core`、`hexi/web_hub`(**非插件**),不再参与插件类型分类与热拔插,故下表不列出。
---
## 自定义插件(`hexi/plugins/`)
| 插件(模块名) | type | 说明 |
|---|---|---|
| `hexi.plugins.nonebot_plugin_bf_bot` | `application` | 战地系列战绩查询(BF3/4/1/5/2042/6)。 |
| `hexi.plugins.nonebot_plugin_steam_info` | `application` | Steam 信息播报/查询。 |
| `hexi.plugins.nonebot_plugin_mc_server_status` | `application` | Minecraft 服务器状态查询。 |
| `hexi.plugins.nonebot_plugin_ncm_saying` | `application` | 网易云热评。 |
| `hexi.plugins.nonebot_plugin_helldivers_tools` | `application` | 绝地潜兵 2 前线战况/战备。 |
| `hexi.plugins.nonebot_plugin_learning_chat` | `application` | 马尔可夫链群聊学习/复读。 |
| `hexi.plugins.nonebot_plugin_galgame_card` | `application` | 群聊人设卡生成/展示。 |
| `hexi.plugins.nonebot_plugin_group_tools` | `application` | 群管理工具集。 |
| `hexi.plugins.nonebot_plugin_group_daily_analysis` | `application` | 群聊行为分析报告。 |
| `hexi.plugins.nonebot_plugin_dailywife` | `application` | 每日随机抽取群友。 |
| `hexi.plugins.nonebot_plugin_deer_pipe` | `application` | 每日打卡(鹿)。 |
| `hexi.plugins.nonebot_plugin_dice` | `application` | 掷骰子/结婚判定。 |
| `hexi.plugins.nonebot_plugin_makeaquote` | `application` | 名人/语录图生成。 |
| `hexi.plugins.nonebot_plugin_random_jm_code` | `application` | 随机 JM 码。 |
| `hexi.plugins.nonebot_plugin_video_analysis` | `application` | 视频链接解析。 |
| `hexi.plugins.nonebot_plugin_regif` | `application` | GIF 倒放/处理。 |
| `hexi.plugins.nonebot_plugin_picfinder_take` | `application` | 二次元搜图。 |
| `hexi.plugins.nonebot_plugin_picstatus` | `application` | 设备状态图。 |
| `hexi.plugins.nonebot_plugin_huoziyinshua` | `application` | otto 活字印刷(语音合成)。 |
| `hexi.plugins.memes_ops` | `application` | 给 `nonebot_plugin_memes` 追加裸词选项(用户侧语法)。 |
| `hexi.plugins.nonebot_plugin_deadlock` | - | **停用**:整文件被注释,不注册任何功能(历史遗留)。 |
| `hexi.plugins.nonebot_plugin_brash_general_supercredits_tools` | - | **空壳**:未完成占位,`__init__.py` 仅注释,无功能。 |
---
## pip 社区插件(`pyproject.toml` 声明)
这些插件由安装包自带的 `__plugin_meta__` 决定类别,本仓库不改动;其中的 library 型插件同样受热拔插保护。
| 插件 | 常见类别 | 说明 |
|---|---|---|
| `nonebot_plugin_apscheduler` | library | 定时任务框架。 |
| `nonebot_plugin_alconna` | library | 命令解析/消息框架。 |
| `nonebot_plugin_saa` | library | 跨平台发送辅助。 |
| `nonebot_plugin_session` | library | 会话/状态管理。 |
| `nonebot_plugin_userinfo` | library | 用户信息接口。 |
| `nonebot_plugin_user` | application | 用户数据接口(含绑定/查看命令)。 |
| `nonebot_plugin_datastore` | library | 数据存储底座。 |
| `nonebot_plugin_orm` | library | ORM 底座。 |
| `nonebot_plugin_fix_qq_img_ssl` | library | 图片 SSL 修复补丁。 |
| `nonebot_plugin_wordcloud` | application | 词云生成。 |
| `nonebot_plugin_memes` | application | 表情包。 |
| `nonebot_plugin_multincm` | application | 多源网抑云点歌。 |
| `nonebot_plugin_random_stereotypes` | application | 随机刻板印象。 |
| `nonebot_plugin_rollpig` | application | 滚猪。 |
> 说明:社区插件类别以其安装包 `PluginMetadata.type` 为准,若与本表不符请以包内元数据为准。
---
## 热拔插保护
`../../hexi/core/plugin_manager.py` 中的 `hot_load / hot_unload / hot_reload` 会先判断目标插件类别:
- 目标为 `application` → 正常热拔插。
- 其余(`library` / 未声明类别,含根包无 meta 但被依赖的核心/框架,如 alconna)→ **拒绝**并记录 warning(热拔插会使依赖它的插件运行混乱)。
`hexi/web_hub/web_config.py` 保存配置触发的 `hot_reload` 同样受此保护:非 application 插件若未提供 `apply` 回调,将不会执行热重载。
@@ -31,8 +31,8 @@ nonebot_plugin_template/
## Web 子应用
- `web/admin.py::build_admin_app()` 返回一个 FastAPI 实例,经 `register_web_plugin` 由 hub 挂载到 `/api/template`。
- 鉴权统一 `hexi.web_auth.require_admin`;示例端点:`GET /ping`、`GET /config`(读插件配置)。
- 想要更丰富的管理页:前端放 `web/dist/`,在 admin.py 里挂 `StaticFiles` + SPA 兜底即可(参考 `nonebot_plugin_web_hub`)。
- 鉴权统一 `hexi.web_hub.web_auth.require_admin`;示例端点:`GET /ping`、`GET /config`(读插件配置)。
- 想要更丰富的管理页:前端放 `web/dist/`,在 admin.py 里挂 `StaticFiles` + SPA 兜底即可(参考 `hexi/web_hub`)。
## 约定
- `plugin_id` = 模块名(自动取 `__package__`;web 子应用用 `__package__.rsplit(".", 1)[0]`)。
@@ -24,7 +24,7 @@ from . import config as _config # noqa: E402,F401
from . import handlers as _handlers # noqa: E402,F401
from . import web as _web # noqa: E402,F401
from hexi.web_plugin_registry import register_web_plugin # noqa: E402
from hexi.web_hub.web_plugin_registry import register_web_plugin # noqa: E402
_config.register_config()
@@ -11,7 +11,7 @@ from __future__ import annotations
from nonebot import get_plugin_config
from pydantic import BaseModel
from hexi.config_standard import register_config_items, register_model_config, register_object_set
from hexi.web_hub.config_standard import register_config_items, register_model_config, register_object_set
# plugin_id = NoneBot 插件模块名
_PLUGIN_ID = __package__
@@ -8,7 +8,7 @@ from nonebot.rule import to_me
from nonebot.params import CommandArg
from nonebot_plugin_alconna import UniMessage
from hexi.web_config import get_effective_value
from hexi.web_hub.web_config import get_effective_value
from ..services.main import do_something
@@ -2,7 +2,7 @@
from __future__ import annotations
from hexi.web_config import get_effective_value
from hexi.web_hub.web_config import get_effective_value
from ..repository import get_record, save_record
@@ -2,9 +2,9 @@
- 由 `__init__.py` 里 `register_web_plugin("template", ..., module_name=__name__)` 注册,
hub 启动时自动挂载到 `/api/template`。
- 鉴权统一走 `hexi.web_auth.require_admin`(OAuth2 + SQLite);注意它本身是 FastAPI 的
- 鉴权统一走 `hexi.web_hub.web_auth.require_admin`(OAuth2 + SQLite);注意它本身是 FastAPI 的
`Depends` 实例,直接用 `_: dict = API` 注入,不要再包一层 `Depends(...)`。
- 可直接读写 `hexi.web_config` 的插件配置。
- 可直接读写 `hexi.web_hub.web_config` 的插件配置。
"""
from __future__ import annotations
@@ -13,7 +13,7 @@ from fastapi import FastAPI
from fastapi.responses import HTMLResponse
from hexi import web_config
from hexi.web_auth import require_admin as API
from hexi.web_hub.web_auth import require_admin as API
# 插件模块名(去掉 .web):= NoneBot 模块名 = plugin_id
_PLUGIN_ID = __package__.rsplit(".", 1)[0]
+1
View File
@@ -0,0 +1 @@
# 机器人核心目录
@@ -1,18 +1,11 @@
from nonebot import get_driver, logger
from nonebot.plugin import PluginMetadata
__plugin_meta__ = PluginMetadata(
name="HeXi 核心工具",
description="HeXi 公共工具库:权限分级、消息解析、限频器、命令冷却、插件热插拔等",
usage="工具库 + 帮助/插件管理命令(帮助、插件列表、加载/卸载/重载插件)",
type="application",
)
# 显式导入子模块注册 matcher(load_plugins 只加载到插件包层,不会自动导入包内模块)
# 显式导入子模块注册 matcher/工具(hexi/core 为机器人核心模块,非 NoneBot 插件)
from . import ( # noqa: E402,F401
cooldown,
custom_utils,
message_utils,
outbound_media,
plugin_control,
plugin_manager,
rate_limit,
@@ -24,3 +17,9 @@ async def _startup_plugin_gate() -> None:
"""等所有插件 matcher 注册完成后,把统一 filter 规则注入到 application 插件。"""
n = plugin_control.instrument_plugin_gate()
logger.info(f"插件控制面: 已注入 {n} 条 gateway 规则")
@get_driver().on_startup
async def _startup_outbound_media() -> None:
"""挂载本机媒体服务:协议端据此回拉 bot 生成的本地文件(内部已处理降级)。"""
outbound_media.mount_media_endpoint()
@@ -7,7 +7,7 @@
- "global":所有人共享一个 CD
用法一(装饰器,包在 on_command 外面):
from hexi.plugins.nonebot_plugin_hexi_core.cooldown import cooldown
from hexi.core.cooldown import cooldown
# 单用户 10s CD,群聊/私聊共用
bf_cmd = cooldown(10, hint="查询太频繁了,{secs} 秒后再试")(on_command("bf3"))
@@ -19,7 +19,7 @@
...
用法二(手动查询/触发,自定义提示文本):
from hexi.plugins.nonebot_plugin_hexi_core.cooldown import Cooldown
from hexi.core.cooldown import Cooldown
cd = Cooldown(10)
if cd.in_cd(ev.get_user_id()):
@@ -36,7 +36,7 @@ from nonebot.adapters import Event
from nonebot.dependencies import Dependent
from nonebot.matcher import Matcher
from hexi.plugins.nonebot_plugin_hexi_core.rate_limit import RateLimiter, get_limiter
from hexi.core.rate_limit import RateLimiter, get_limiter
# 无 name 时的实例序号,保证同一进程内各实例的注册表 key 互不冲突
_cd_seq = itertools.count(1)
@@ -134,7 +134,7 @@ if hexi.is_superuser(event): # 超级用户
# 架构设计
```
hexi/plugins/nonebot_plugin_hexi_core/
hexi/core/
├── __init__.py # 插件入口,导出 hexi 对象
├── poke.py # 戳一戳
├── forward.py # 合并转发消息构建器
@@ -1,11 +1,19 @@
from pathlib import Path
from nonebot import get_bot
from nonebot import require
from nonebot.adapters import Event
from nonebot.adapters.onebot.v11 import Bot, Message, MessageEvent, MessageSegment
from nonebot.log import logger
from nonebot_plugin_alconna import UniMessage
from nonebot_plugin_alconna.uniseg import Receipt, Target
require("nonebot_plugin_htmlrender")
from nonebot_plugin_htmlrender import md_to_pic
# 「处理中」占位动图(任务驱动通用回复用)
THINKING_GIF = Path(__file__).parents[1] / "resource" / "imgs" / "thinking.gif"
async def send_markdown(
bot: Bot,
@@ -108,3 +116,64 @@ async def send_forward_msg(
except Exception as e:
logger.warning(f"合并转发发送失败,回退文本: {e}")
await bot.send(event, fallback)
async def common_proc_reply(
message_id: str | None = None,
*,
target: Event | Target | None = None,
image: str | Path = THINKING_GIF,
text: str | None = None,
) -> Receipt | None:
"""任务驱动通用回复:引用指定消息并回一张「处理中」占位图(UniMessage 版)
引用哪条消息 / 发到哪:
- 事件处理中(有上下文):不传参数即引用当前触发消息,发回当前会话;
- 传 message_id 则引用指定消息(仍需事件上下文或用 target 指定会话);
- 后台任务(无事件上下文):必须显式传 target(Event 或 alconna Target),
如 `Target("872490448")`(群)/`Target("2931589710", private=True)`(私聊);
此时不传 message_id 即为不带引用的普通发送。
Args:
message_id: 要引用的消息 id,默认当前事件消息
target: 发送目标(Event/Target);仅后台任务需要
image: 占位图路径(或 http(s) 链接),默认 hexi/resource/imgs/thinking.gif
text: 附加文本,为空只发图
Returns:
alconna Receipt: 可 `.recall()` 撤回占位图、`.msg_ids` 取消息 id
(是否支持编辑见 `.editable`);发送失败返回 None
"""
def build() -> UniMessage:
# 每次重建:alconna 的 reply_to 会把引用段插入消息自身,降级重发需干净副本
# 本地文件统一交给 hexi.core.outbound_media 的 call_api 钩子改写
# (URL 优先/base64 兜底),这里不再自己转 base64——占位图本就是发图,
# 走同一套策略才不会出现「一处能发一处不能发」。关掉 HEXI_INLINE_MEDIA
# 的语义是「协议端就在本机」,此时 file:// 同样是有效的。
if isinstance(image, str) and image.startswith(("http://", "https://")):
img = UniMessage.image(url=image)
else:
img = UniMessage.image(path=Path(image))
return UniMessage.text(f"{text} ") + img if text else img
if message_id:
reply_to: str | bool = message_id
elif target is None:
reply_to = True # 从当前事件上下文取被引用消息
else:
reply_to = False # 后台任务且未指定消息 id → 普通发送
try:
return await build().send(target=target, reply_to=reply_to)
except Exception as e:
if not reply_to:
logger.error(f"占位图发送失败: {e}")
return None
logger.warning(f"占位回复引用发送失败,降级为不带引用: {e}")
try:
return await build().send(target=target)
except Exception as e:
logger.error(f"占位图发送失败: {e}")
return None
+487
View File
@@ -0,0 +1,487 @@
"""出站媒体内联化:把本地文件改写为协议端可达的 URL(或 base64 兜底)。
生产环境的 OneBot 协议端不在本机:给 API 传本地路径时,NoneBot 的 `f2s()`
会把 `Path` 转成 `file:///D:/...`(`str` 则原样透传)——而 `file://` 的语义是
「**协议端那台机器上的路径**」,协议端 stat 必然 ENOENT。受影响的不止本仓插件,
还有 pip 装的社区插件(如 nonebot_plugin_doroending),它们的代码改不了,
只能全局拦截。
本模块挂在 `Bot.on_calling_api`(NoneBot 公开钩子,`call_api` 的必经之路,
`bot.send` / alconna `finish` / 直接 `send_group_msg` 全从这里过)上,
在真正调用适配器前改写发送类 API 里的本地文件:
1. **默认改写为 `http://<本机可达IP>:<端口>/media/<token>`**,由协议端主动来拉。
地址取自 `driver.config` 的 host/port + 本机网卡,与 /hub 首页「协议端配对」
展示的是同一来源;反向 WS 部署下协议端本就连得到这个地址(它连的就是它),
所以不需要任何新配置。体积不受 base64 膨胀影响,视频/多媒体转发尤其受益。
2. **拿不到可达地址时回退 `base64://`**:字节由 bot 直接携带,不依赖协议端
反过来拨通本机的任何假设,跨 NAT/隧道也成立。
不改写 `http(s)://` / `base64://` / 不存在的路径——收到的图片 id(如 `{abc}.jpg`)
和协议端自己的缓存路径都靠这条自然放行。任何异常只告警不抛:
拦截层绝不能反过来把发送搞挂。
两类 API 的处理方式不同:
- **消息发送**(`send_*_msg` / `send_*_forward_msg`):媒体在消息段里,递归遍历
后按上面两级改写(URL 优先,base64 兜底)。
- **文件上传**(`upload_group_file` / `upload_private_file`):`file` 在顶层,
且**只做 URL 不做 base64**——文件体积不适合内联。覆盖这条让
`video_analysis` 的群文件「本地直传」兜底和 alconna 的 `$onebot11:file`
一起受益(后者传的是 `Path.as_posix()` 裸路径)。
"""
from __future__ import annotations
import asyncio
import os
import secrets
import socket
import time
from base64 import b64encode
from pathlib import Path
from typing import Any
from urllib.parse import urlparse
from urllib.request import url2pathname
from nonebot.adapters import Bot as BaseBot
# 用 nonebot.log 而非 `from nonebot import logger`:与 rate_limit 保持一致,
# 模块级 import 保持干净,测试才能 importlib 按路径裸加载(不经包 __init__)。
from nonebot.log import logger
try: # NoneBot 内部路径:V11/V12 的 MessageSegment 都继承自它
from nonebot.internal.adapter.message import MessageSegment as BaseMessageSegment
except Exception: # pragma: no cover - 内部路径变更时的兜底,见 _as_segment
BaseMessageSegment = None # type: ignore[assignment,misc]
# 消息段里代表「本地/远端媒体文件」的 type(file 字段语义)
MEDIA_SEG_TYPES = frozenset({"image", "record", "video"})
# OneBot V11 发送类 API(与 picstatus misc_statistics 的清单一致,另加合并转发)。
_SEND_APIS = frozenset(
{
"send_private_msg",
"send_group_msg",
"send_msg",
"send_private_forward_msg",
"send_group_forward_msg",
"send_forward_msg",
}
)
# 文件上传类 API:file 字段在**顶层**而不是消息段里,所以走单独一小段逻辑,
# 且**只做 URL、不做 base64 兜底**——文件动辄几十上百 MB,内联会撑爆内存和
# WS 帧,OneBot 的文件上传语义本来也不是为内联设计的。
_FILE_UPLOAD_APIS = frozenset({"upload_group_file", "upload_private_file"})
# 只有 OneBot 的 file 字段认 file:// / base64:// 语义;bot.py 当前只注册 V11
# (pyproject.toml 虽列了 V12,但代码里没装适配器)。
_ONEBOT_NAMES = frozenset({"OneBot V11"})
# 通配绑定:值本身不可拨号,需要换成具体网卡地址
_WILDCARD_HOSTS = frozenset({"", "0.0.0.0", "::", "*"})
_MAX_DEPTH = 8
_MAX_TOKENS = 256
_DEFAULT_TTL = 300.0
_DEFAULT_MAX_MB = 4.0
# token -> (文件路径, 过期时刻 monotonic)
_tokens: dict[str, tuple[Path, float]] = {}
# 媒体服务是否真的挂上了:没挂上就不能发 URL(否则协议端必然 404)
_mounted = False
# ---------------------------------------------------------------- 环境开关
def _env_flag(name: str, default: bool) -> bool:
raw = os.environ.get(name)
if raw is None:
return default
return raw.strip().lower() not in {"", "0", "false", "no", "off"}
def _enabled() -> bool:
"""总开关(默认开)。关闭的语义是「协议端就在本机」,此时 file:// 是有效的。"""
return _env_flag("HEXI_INLINE_MEDIA", True)
def _url_enabled() -> bool:
"""URL 优先模式(默认开);关掉则一律走 base64。"""
return _env_flag("HEXI_INLINE_MEDIA_URL", True)
def _ttl() -> float:
try:
return float(os.environ.get("HEXI_MEDIA_URL_TTL", "") or _DEFAULT_TTL)
except ValueError:
return _DEFAULT_TTL
def _max_inline_mb() -> float:
try:
return float(os.environ.get("HEXI_INLINE_MEDIA_MAX_MB", "") or _DEFAULT_MAX_MB)
except ValueError:
return _DEFAULT_MAX_MB
# ---------------------------------------------------------------- 路径解析
def local_file_of(value: Any) -> Path | None:
"""把 file 字段值解析成本机真实存在的文件路径;不是本地文件则 None。
- `http(s)://` / `base64://` → None(原样放行)
- `file://` → `url2pathname` 还原
- 其余按原始路径候选(`f2s()` 对 `str` 是原样透传的)
- 不存在 / 非法 → None(收到的图片 id、协议端缓存路径靠这条放行)
"""
if not isinstance(value, str) or not value:
return None
if value.startswith(("http://", "https://", "base64://")):
return None
candidates: list[str] = []
if value.startswith("file://"):
# 关键陷阱:不能直接 unquote(urlparse(uri).path)——Windows 上会得到
# 前导斜杠的 `/D:/...`,is_file() 恒为 False 而**静默不转换**,
# 只在跨机时才暴露。url2pathname 才能正确还原成 `D:\\...`。
try:
uri_path = urlparse(value).path
candidates.append(url2pathname(uri_path))
except Exception:
return None
# 有人手工拼 file:// 且不做百分号转义(video_analysis._file_uri 即如此:
# `"file:///" + path.replace("\\", "/")`)。对已经未转义的串再 unquote,
# 文件名含字面 %XX 时会认错,所以把原始路径也列为候选兜底。
if len(uri_path) > 2 and uri_path[0] == "/" and uri_path[2] == ":":
candidates.append(uri_path[1:]) # /D:/x → D:/x
else:
# 裸路径:f2s() 对 str 原样透传;alconna 的 $onebot11:file 走 as_posix()
candidates.append(value)
for raw in candidates:
try:
path = Path(raw)
if path.is_file():
return path
except (OSError, ValueError):
continue
return None
# ---------------------------------------------------------------- 媒体服务
def _is_private(ip: str) -> bool:
parts = ip.split(".")
if len(parts) != 4:
return False
try:
a, b = int(parts[0]), int(parts[1])
except ValueError:
return False
return a == 10 or (a == 172 and 16 <= b <= 31) or (a == 192 and b == 168)
def _local_ips() -> list[str]:
"""本机可被协议端访问的 IPv4 列表(私有网段优先,排除环回/链路本地)。
与 hexi/web_hub/dashboard.py 的 `_local_ips` 同源逻辑(那边供 /hub 首页
「协议端配对」展示,即用户已经验证过能用的地址);此处刻意不 import,
避免 hexi.core → hexi.web_hub 的反向依赖。
"""
ips: list[str] = []
try:
import psutil
for addrs in psutil.net_if_addrs().values():
for addr in addrs:
if addr.family == socket.AF_INET:
ips.append(str(addr.address or ""))
except Exception: # noqa: BLE001 - psutil 缺失/异常时退回 getaddrinfo
pass
if not ips:
try:
infos = socket.getaddrinfo(socket.gethostname(), None, socket.AF_INET)
ips.extend(info[4][0] for info in infos)
except OSError:
pass
out: list[str] = []
for ip in ips:
ip = ip.strip()
if not ip or ":" in ip or ip.count(".") < 3:
continue
# 环回与链路本地(169.254.x)协议端访问不到,排除
if ip.startswith(("127.", "169.254.")):
continue
if ip not in out:
out.append(ip)
out.sort(key=lambda ip: (not _is_private(ip), ip))
return out
def _listen_addr() -> tuple[str, int] | None:
"""本机媒体服务的对外可达地址(host, port)。"""
try:
from nonebot import get_driver
config = get_driver().config
port = int(getattr(config, "port", 0) or 0)
host = str(getattr(config, "host", "") or "").strip()
except Exception: # noqa: BLE001
return None
if not port:
return None
if host not in _WILDCARD_HOSTS:
# 显式绑定(如 dev 的 127.0.0.1)直接用
return host, port
for ip in _local_ips():
return ip, port
return None
def _base_url() -> str | None:
"""媒体服务基地址;未挂载/无可达地址时 None(调用方据此回退 base64)。"""
if not _mounted:
return None
override = os.environ.get("HEXI_MEDIA_BASE_URL", "").strip()
if override:
return override.rstrip("/")
addr = _listen_addr()
if addr is None:
return None
host, port = addr
return f"http://{host}:{port}"
def _gc() -> None:
now = time.monotonic()
for token, (_, expires) in list(_tokens.items()):
if expires <= now:
del _tokens[token]
while len(_tokens) >= _MAX_TOKENS:
_tokens.pop(next(iter(_tokens)))
def _publish(path: Path) -> str:
"""登记一个一次性 token(值随机不可猜),返回它。"""
_gc()
token = secrets.token_urlsafe(16)
_tokens[token] = (path, time.monotonic() + _ttl())
return token
def mount_media_endpoint() -> None:
"""把媒体路由挂到现有 ASGI app 上(同一个端口,不新开服务)。
免鉴权是**必须**的——协议端登录不了 hub——所以安全模型改为
「路径由 bot 在改写那一刻自己登记,请求方只能出示不可猜 token、
根本无法表达路径」,天然没有目录穿越面,配合短 TTL。
"""
global _mounted
if not _enabled():
logger.info("出站媒体内联化已禁用 (HEXI_INLINE_MEDIA=false),按原值发送")
return
if _mounted:
return
try:
from fastapi import FastAPI
from nonebot import get_app
sub = FastAPI(
title="HeXi Outbound Media",
docs_url=None,
redoc_url=None,
openapi_url=None,
)
@sub.get("/{token}")
async def media(token: str):
from fastapi import HTTPException
from fastapi.responses import FileResponse
entry = _tokens.get(token)
if entry is None:
raise HTTPException(status_code=404, detail="媒体不存在或已过期")
path, expires = entry
if expires <= time.monotonic():
_tokens.pop(token, None)
raise HTTPException(status_code=404, detail="媒体链接已过期")
if not path.is_file():
raise HTTPException(status_code=404, detail="文件已不存在")
# no-store:过期媒体不该被中间层缓存住
return FileResponse(path, headers={"Cache-Control": "no-store"})
get_app().mount("/media", sub)
_mounted = True
logger.info("出站媒体服务已挂载: /media/<token>")
except Exception as e: # noqa: BLE001 - 挂载失败只降级,不影响启动
logger.warning(
f"出站媒体服务挂载失败,将回退 base64: {type(e).__name__}: {e}"
)
# ---------------------------------------------------------------- 改写
async def _to_base64(path: Path) -> str:
# 同步 IO 挪到线程,别在事件循环里读文件
data = await asyncio.to_thread(path.read_bytes)
size_mb = len(data) / 1048576
limit = _max_inline_mb()
if size_mb > limit:
# 超阈值仍要发:不转是 100% 必失败,转了才可能成功
logger.warning(
f"出站媒体内联: {path.name} 体积 {size_mb:.1f}MB 超过 "
f"HEXI_INLINE_MEDIA_MAX_MB={limit:g},仍以 base64 发送"
)
return f"base64://{b64encode(data).decode()}"
async def rewrite_file(value: Any) -> str | None:
"""改写单个 file 字段值;无需改写(或改写失败)返回 None。"""
path = local_file_of(value)
if path is None:
return None
try:
if _url_enabled():
base = _base_url()
if base:
return f"{base}/media/{_publish(path)}"
return await _to_base64(path)
except Exception as e: # noqa: BLE001 - 单个段失败只告警,按原值发送
logger.warning(f"出站媒体改写失败,按原值发送 {path}: {type(e).__name__}: {e}")
return None
async def rewrite_upload_file(value: Any) -> str | None:
"""文件上传类 API 的 file 字段改写;无需改写返回 None。
只做 URL,**不做 base64 兜底**:拿不到可达地址就原样放行,行为与改动前
一致(不保证成功,但绝不会比原来更糟)。`HEXI_INLINE_MEDIA_URL=false`
同样会让这里放行——那个开关的语义是「媒体服务这条路不可用」,文件上传
依赖同一条路,理应一起关掉。
"""
if not _url_enabled():
return None
path = local_file_of(value)
if path is None:
return None
try:
base = _base_url()
if not base:
return None
return f"{base}/media/{_publish(path)}"
except Exception as e: # noqa: BLE001
logger.warning(f"群/私聊文件改写失败,按原值发送 {path}: {type(e).__name__}: {e}")
return None
def _as_segment(node: Any) -> tuple[str, dict] | None:
"""识别消息段对象 → (type, data);不是消息段返回 None。
MessageSegment 不是 dict 子类(但实现了 keys/get),所以单靠 isinstance(dict)
认不出来;这里以 NoneBot 的公共基类为准,并留一条结构兜底以防内部路径变更。
"""
if BaseMessageSegment is not None and isinstance(node, BaseMessageSegment):
seg_type, seg_data = node.type, node.data
else:
seg_type = getattr(node, "type", None)
seg_data = getattr(node, "data", None)
if isinstance(seg_type, str) and isinstance(seg_data, dict):
return seg_type, seg_data
return None
async def _walk_segment(seg_type: str, seg_data: dict, depth: int) -> None:
if seg_type in MEDIA_SEG_TYPES:
rewritten = await rewrite_file(seg_data.get("file"))
if rewritten is not None:
seg_data["file"] = rewritten
# 合并转发节点:content 可能是 str / list[dict] / Message,里面还可能嵌媒体
# (MessageSegment.node_custom 把 Message 原样塞进 data["content"])
for key in ("content", "messages"):
if key in seg_data:
await _walk(seg_data[key], depth + 1)
async def _walk(node: Any, depth: int = 0) -> None:
"""深度受限的通用遍历,原地改写。
形态是散的(Message / 纯 dict 段 / list[dict] / 转发节点的三种 content),
写死形状必漏,所以按结构特征递归而不是枚举。
"""
if node is None or depth > _MAX_DEPTH:
return
# str/bytes 是叶子;数字等标量也直接跳过
if isinstance(node, (str, bytes, int, float, bool)):
return
seg = _as_segment(node)
if seg is not None:
await _walk_segment(seg[0], seg[1], depth)
return
if isinstance(node, (list, tuple)): # 含 Message(list 子类)
for item in node:
await _walk(item, depth + 1)
return
if isinstance(node, dict):
seg_type, seg_data = node.get("type"), node.get("data")
if isinstance(seg_type, str) and isinstance(seg_data, dict):
# 段字典({"type": "image", "data": {...}} / node 字典)
await _walk_segment(seg_type, seg_data, depth)
return
for value in node.values():
await _walk(value, depth + 1)
# ---------------------------------------------------------------- 钩子
@BaseBot.on_calling_api
async def _inline_outbound_media(bot: BaseBot, api: str, data: dict[str, Any]) -> None:
"""call_api 前置钩子:把出站 API 里的本地文件改写成可跨机访问的形式。
钩子拿到的是 `_call_api` 之前**同一个可变 data dict**,且 hook 的 task group
在 `_call_api` 之前 await 完成,所以原地改生效。
"""
if not _enabled():
return
is_send = api in _SEND_APIS
is_upload = api in _FILE_UPLOAD_APIS
if not (is_send or is_upload) or not isinstance(data, dict):
return
try:
if bot.adapter.get_name() not in _ONEBOT_NAMES:
return
except Exception: # noqa: BLE001
return
if is_upload:
# 文件上传的 file 在顶层(不是消息段);name/folder 等参数一律不碰
try:
rewritten = await rewrite_upload_file(data.get("file"))
except Exception as e: # noqa: BLE001 - 绝不反过来搞挂发送
logger.warning(f"出站媒体改写异常(upload.file),按原值发送: {e}")
return
if rewritten is not None:
data["file"] = rewritten
return
for key in ("message", "messages"):
if key in data:
try:
await _walk(data[key])
except Exception as e: # noqa: BLE001 - 绝不反过来搞挂发送
logger.warning(f"出站媒体改写异常({key}),按原值发送: {e}")
@@ -27,7 +27,7 @@ from nonebot import get_loaded_plugins, logger
from nonebot.internal.matcher import matchers as matchers_registry
from nonebot.rule import Rule
_PLUGIN_ROOT = Path(__file__).resolve().parents[2] # hexi/
_PLUGIN_ROOT = Path(__file__).resolve().parents[1] # hexi/
DATA_DIR = _PLUGIN_ROOT / "data"
STORE_PATH = DATA_DIR / "plugin_control.json"
@@ -153,7 +153,7 @@ def remove_plugin(plugin_id: str) -> None:
def list_plugins() -> list[dict[str, Any]]:
"""枚举所有 application 插件(或注册了配置 schema 的插件)及其控制面状态。"""
from nonebot.plugin import get_loaded_plugins
from hexi.web_config import has_schema
from hexi.web_hub.web_config import has_schema
result: list[dict[str, Any]] = []
seen: set[str] = set()
@@ -174,6 +174,7 @@ def list_plugins() -> list[dict[str, Any]]:
"name": meta.name,
"description": meta.description or "",
"usage": meta.usage or "",
"type": meta.type,
"enabled": ctl["global"]["enabled"],
"chat": ctl["global"]["chat"],
"groups": {gid: lvl for gid, lvl in ctl["groups"].items()},
@@ -16,7 +16,6 @@ from nonebot.params import CommandArg
from nonebot.permission import SUPERUSER
from nonebot.plugin import (
Plugin,
PluginMetadata,
_plugins,
get_loaded_plugins,
get_plugin_by_module_name,
@@ -26,16 +25,10 @@ from nonebot.rule import Rule, to_me
require("nonebot_plugin_alconna")
from nonebot_plugin_alconna import UniMessage
from hexi.plugins.nonebot_plugin_hexi_core.message_utils import send_forward_msg
from hexi.core.message_utils import send_forward_msg
from . import plugin_control
__plugin_meta__ = PluginMetadata(
name="插件管理",
description="热插拔插件:运行时加载/卸载/重载插件(仅超管)",
usage="插件列表 / 加载插件 <模块名> / 卸载插件 <模块名> / 重载插件 <模块名>",
type="application",
)
# 插件热拔插管理(hexi.core 核心模块,非 NoneBot 插件)
def _forward_sender_name() -> str:
"""合并转发节点的发送者昵称(用机器人配置昵称,插件名写进 content)"""
nickname = get_driver().config.nickname
@@ -94,8 +87,33 @@ def _unregister_plugin(plugin: Plugin) -> None:
parent.sub_plugins.discard(plugin)
def _is_library(name: str) -> bool:
"""判断模块是否归属「非 application」插件(library / 未声明类别)。
只有 application 插件可热拔插;library(含根包未声明类别但被依赖的核心/框架)
都应保护,热拔插会破坏依赖它的插件运行态。
优先级:NoneBot 插件注册表 > sys.modules 回溯(含父包 __plugin_meta__)。
"""
plugin = get_plugin_by_module_name(name)
if plugin is not None:
meta = plugin.metadata
return not (meta and meta.type == "application")
parts = name.split(".")
for i in range(len(parts), 0, -1):
module = sys.modules.get(".".join(parts[:i]))
if module is not None:
meta = getattr(module, "__plugin_meta__", None)
if meta is not None:
return meta.type != "application"
return False
def hot_load(name: str) -> bool:
"""热加载插件模块(直接 import,matcher 随之注册)"""
if _is_library(name):
logger.warning(f"插件 {name} 是非 application(library/未声明)插件,不支持热加载")
return False
if name in _hot_plugins:
return False
if any(p.module_name == name for p in get_loaded_plugins()):
@@ -115,7 +133,14 @@ def hot_unload(name: str) -> bool:
不限于热加载的插件:启动时加载的插件(hexi/plugins 与 pyproject.toml
声明)同样支持卸载,以 NoneBot 插件注册表为准查找。
安全限制:非 application 插件(library/未声明类别,常为核心库/框架)被其他
插件依赖,热卸载会让依赖方引用悬空并导致运行混乱,故拒绝卸载。
"""
if _is_library(name):
logger.warning(f"插件 {name} 是非 application(library/未声明)插件,热卸载会使依赖它的插件运行混乱,已拒绝")
return False
plugin = get_plugin_by_module_name(name)
if plugin is None and name not in _hot_plugins:
return False
@@ -148,6 +173,9 @@ def hot_unload(name: str) -> bool:
def hot_reload(name: str) -> bool:
"""热重载插件:先卸载再加载"""
if _is_library(name):
logger.warning(f"插件 {name} 是非 application(library/未声明)插件,热重载会使依赖它的插件运行混乱,已拒绝")
return False
hot_unload(name)
return hot_load(name)
@@ -3,7 +3,7 @@
纯逻辑实现,不依赖 NoneBot 运行时;同步(requests)与异步(httpx/aiohttp)均可用。
用法:
from hexi.plugins.nonebot_plugin_hexi_core import rate_limit
from hexi.core import rate_limit
# 异步请求前取令牌(无令牌时等待,最多等 timeout 秒,超时返回 False)
if not await rate_limit.acquire("steam", rate=1, capacity=2):
+1
View File
@@ -0,0 +1 @@
# 机器人插件目录
@@ -29,6 +29,7 @@ __plugin_meta__ = PluginMetadata(
解绑: /解绑 (解除你的QQ与当前绑定的EAID)
修改绑定: /修改绑定 EAID (修改你的QQ与当前绑定的EAID)
""".strip(),
type="application",
extra={
},
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 326 KiB

After

Width:  |  Height:  |  Size: 199 KiB

@@ -1,81 +0,0 @@
# CLAUDE.md — 群聊人设卡插件开发文档
本文件是**本插件内开发**的唯一入口文档;需要项目全局信息(Poetry 命令、启动方式、测试约定)时再查项目根目录的 CLAUDE.md。详细设计定稿见同目录 `DESIGN.md`。
## 插件概述
基于群聊语料蒸馏群成员的形象风格,生成 galgame 风格人物卡(九段画像)。
- **目标**:生成"人的画像",不是关系网分析;样貌参考为虚构,永远带标注
- **隐私硬约束**:只采集 opt-in 成员;敏感数据本地正则替换为占位符,**绝不经过 LLM**
- **维度隔离**:每个 `(group_id, user_id)` 是独立人设,跨群不混
## 目录结构
```
nonebot_plugin_galgame_card/
├── __init__.py 入口层:on_message 采集器(鉴权/采样/调治理落库)
├── models.py 数据层:五张表(persona_group/user/chat_log/impression/summary)
├── repository.py 数据层:仓储(来源无关,所有读写唯一入口)
├── processor.py 治理层:纯函数(五维提取/噪声/脱敏),可单测
├── config.py 插件配置(仅图片识别开关等;Web 鉴权统一走 /hub)
├── web_hub.py Web API 子应用(挂载到 /api/galgame_card,auth=hexi.web_auth)
├── DESIGN.md 设计定稿(数据模型/流水线/九段协议/证据纪律)
└── CLAUDE.md 本文件
```
**Web 管理后台**:已嵌入统一管理台 `/hub/`,前端由 `hexi/web` 渲染,API 挂载到 `/api/galgame_card`,鉴权与 /hub 共用 `hexi.web_auth`(OAuth2 + SQLite)。功能:群开关、参与者增删、语料/印象/画像浏览与删除、清空群数据。改后端需重启 bot。
## 核心设计(速览,细节见 DESIGN.md)
1. **两级闸门**:群开关 `persona_group.enabled`(默认关)+ 个人 opt-in `persona_user`,都过才采集
2. **两级流水线**:语料 →(攒够 N 条)→ 印象(LLM 自然语言,增量中间层)→(攒够 M 条)→ 画像(九段 markdown,版本化)
3. **消息五维**:内容 / 谁发的 / 发给谁(回复/@)/ 几点发的 / 被回复内容快照——只存治理后纯文本
4. **脱敏四层**(`desensitize()`):明确模式(最长优先排序防截胡)→ 定位式(同条关键词+值)→ 跨条语境(关键词在附近消息)→ 兜底(保守替换)
5. **九段画像协议**:身份印象/性格特征/说话风格/口头禅语录/兴趣话题/相处模式/时间画像/样貌参考(虚构)/不确定信息
6. **总结路径不进消息 handler**:LLM 调用只在调度器触发,防延迟/限流
## 开发命令
```bash
# 跑本插件测试(治理层纯函数,9+ 个用例)
poetry run pytest tests/test_persona_processor.py
# 全量测试
poetry run pytest
# 启动/重启验证:PyCharm 的 "start bot" 运行配置(勿用 bat 脚本)
```
## 数据库
- orm 默认库:`data/nonebot_plugin_orm/db.sqlite3`(不是 `hexi/data/data.db`)
- 建表:bot 启动时 orm 自动 create_all(新表加在 `models.py` 里即可,重启生效)
- 配置键是 `SQLALCHEMY_DATABASE_URL`(本插件未设置,走默认库)
## 开发注意事项(踩过的坑)
0. **⚠️ orm 启动自动同步会清空表数据**:`.env` 里 `ALEMBIC_STARTUP_CHECK=false` 时,nonebot_plugin_orm 每次启动都 autogenerate 同步数据库模式,**模型一有变更(改 models.py)就会重建表、清空全部数据**(2026-08-11 实测踩坑,全表被清)。已在本插件 `__init__.py` 导入期把 `migrate.sync` 替换为安全空操作。**今后 schema 演进只准通过 `repository.ensure_schema()` 显式 ALTER**,改完 models.py 后要在重启前手动执行对应 ALTER(或加进 ensure_schema)。
1. **`on_message` 必须 `block=False`**:否则事件流被拦截,群里其他插件全废
2. **只收群消息**:handler 参数注解 `GroupMessageEvent`(类型注解即过滤器)
3. **脱敏正则排序**:身份证/银行卡必须在手机号之前,否则手机号截胡身份证数字段
4. **定位式替换只替换值、保留关键词**:`密码是 xyz789` → `密码是 [密码]`,关键词不能丢
5. **测试不能裸 import 包**:`__init__.py` 触发 NoneBot 初始化,用 importlib 按路径加载 processor(见 `tests/test_persona_processor.py`,与 test_rate_limit 同款)
6. **总结/印象生成**:绝不在消息 handler 里调 LLM,走调度层(apscheduler)
7. **仓储并发**:`add_summary` 版本自增有并发撞 UNIQUE 风险,调度层加锁保护
## 当前进度
- ✅ 数据层:五表 + 仓储(含群开关、滚动淘汰、版本自增)
- ✅ 采集层:消息路径(监控→鉴权→治理→脱敏→采样→落库)
- ✅ Web 管理后台:嵌入 /hub(/api/galgame_card + hexi/web 页面;群开关、参与者、数据浏览/清理)
- ⏳ QQ 命令集:`开启人设采集` / `加入人设` / `退出人设` / `查看人设`(Web 已覆盖同等功能,QQ 命令可选做)
- ⏳ 总结路径:LLM 客户端、印象生成、九段画像生成、调度触发
- ⏳ 呈现层:人物卡展示/图片渲染
## 待决策点
- 触发阈值(印象 ≥50 条新语料 / 画像 ≥5 条新印象,⏳ 待调)
- 脱敏兜底位数(裸数字 ≥6 位默认替换 `[账号]`,保守优先;误杀多可提到 8 位,动 `BARE_DIGITS_RE`)
- 密保答案场景(中文值正则误杀率高,方案待定)
- 命令名与权限(超管/群主)
@@ -1,194 +0,0 @@
# 群聊人设卡(Galgame 风格人物卡构建器)设计文档
> 本文档固化设计决策,作为各层实现的唯一依据。标注 ⏳ 的为草案/待定项。
## 1. 定位
**目标**:基于群聊发言语料,蒸馏群成员的形象风格,生成 galgame 风格的人物卡(人设)。
**非目标**:
- 不做关系网分析("相处模式"只是画像的一个段落,不是独立产品)
- 不做真实身份推断(年龄/职业/住址等现实信息只能进"不确定信息"段)
- 不生成真实样貌(样貌参考为虚构,永远带标注)
**消费方**:① 展示给人看(人物卡);② 可选:作为扮演 prompt 注入(MaiBot 兼容方向)。
## 2. 整体架构
功能分层:采集 → 治理 → 存储 → 调度 → 分析 → 呈现。
两级流水线(借鉴 MaiBot 印象机制):
```
群聊消息 ──五维落库──> 语料 ──(攒够 N 条)──> 印象(LLM 自然语言) ──(攒够 M 条)──> 画像(九段协议) ──> 版本化快照
```
- **语料 → 印象**:每次对"上次印象之后的新语料"生成一段自然语言印象(话题/氛围/互动),存 `persona_impression`。印象是增量中间产物,画像不重读全部原文。
- **印象 → 画像**:从印象集 + 规则统计(@/回复 互动、活跃时段)生成九段人物卡。
- 画像每次生成都是新版本(version +1),永久留档可对比。
## 3. 数据层(已实现 ✅)
五张表,前缀 `persona_`:
### persona_group —— 群采集开关(数据来源总闸门)
| 字段 | 类型 | 说明 |
|---|---|---|
| group_id | BigInteger PK | 群号 |
| enabled | Boolean 默认 False | 该群是否开启采集(默认关,需显式开启) |
| updated_at | DateTime | 最后变更时间 |
群关闭 → 该群所有人一律不采集;群开启后,个人还需 opt-in(两级闸门)。
### persona_user —— 参与者名单(按群 opt-in)
| 字段 | 类型 | 说明 |
|---|---|---|
| user_id | BigInteger PK | 参与人 QQ |
| group_id | BigInteger PK | 所在群 |
| joined_at | DateTime | 加入时间 |
### persona_chat_log —— 采集语料(五维 + 发言段链条)
| 字段 | 类型 | 说明 |
|---|---|---|
| id | Integer PK 自增 | 全局有序,印象覆盖区间用它表示 |
| user_id | BigInteger | 谁发的 |
| group_id | BigInteger | |
| nickname | String(64) | 群昵称快照 |
| content | Text | 内容(纯文本,已脱敏截断) |
| target_user_id | BigInteger NULL | 发给谁(ev.reply 的 sender / @ 对象;无则 NULL=群聊漫谈) |
| target_inherited | Boolean | target 是否从发言段链条继承(对上一句的解释/补充仍算发给同一对象) |
| follows_id | Integer NULL | 发言段链条:同一说话人的上一条语料 id(间隔 ≤ 5 分钟) |
| reply_to_content | Text NULL | 被回复内容快照(对方不在语料里也能知道他在回应什么) |
| created_at | DateTime | 几点发的 |
索引:`(user_id, group_id, created_at)`。滚动保留:单用户单群上限 3000 条(⏳ 常量待定)。
**发言段链条**(解决"不带 @/回复 的后续补充丢目标"):`@B 借我玩玩` → 下一条 `我的意思是借号不是借人`(无显式目标)继承 target=B 并打 `target_inherited` 标记;分析层窗口组装可沿 `follows_id` 回溯整段发言。
### persona_impression —— 印象(两级流水线中间产物)
| 字段 | 类型 | 说明 |
|---|---|---|
| id | Integer PK 自增 | |
| user_id / group_id | BigInteger | |
| content | Text | LLM 生成的自然语言印象 |
| cover_from_id / cover_to_id | Integer | 覆盖的语料 id 区间(增量依据) |
| model | String(64) | 生成模型 |
| created_at | DateTime | |
### persona_image —— 图片识别结果缓存(⏳ 多模态预留,未启用)
| 字段 | 类型 | 说明 |
|---|---|---|
| hash | String(64) PK | 图片 hash(对应 chat_log.image_hashes) |
| description | Text | 多模态识别结果(表情包梗/截图内容) |
| model | String(64) | 识别模型 |
| recognized_at | DateTime | |
**预留接口**:`vision.py`(BaseImageRecognizer,当前为 Noop 占位)。将来接入多模态 LLM 后:异步后台识别(绝不在消息路径同步调)、同一 hash 只识别一次(缓存复用)、失败不影响采集。识别描述供分析层窗口组装喂给总结 LLM。
### persona_summary —— 画像快照(版本化)
| 字段 | 类型 | 说明 |
|---|---|---|
| id | Integer PK 自增 | |
| user_id / group_id | BigInteger | |
| version | Integer | 每次生成 +1,同人同群唯一 |
| card_text | Text | 九段 markdown 画像原文 |
| corpus_count | Integer | 语料覆盖条数(元信息) |
| impression_count | Integer | 使用的印象条数(元信息) |
| model | String(64) | 生成模型 |
| created_at | DateTime | |
约束:`UNIQUE(user_id, group_id, version)`(并发写入由调度层锁保护)。
## 4. 九段画像协议(已定稿 ✅)
格式:markdown 固定标题 + 有界 bullet,代码可解析、人可编辑、LLM 可生成、可注入 prompt。
```
# 人物卡 · {主称呼}
语料 {N} 条 · 时间跨度 {start}-{end} · 版本 v{n} · 生成于 {date}
## 身份印象 ≤4 条 群内可见的:自称方式、群角色(吐槽役)、常用昵称
## 性格特征 ≤6 条 毒舌但心软 / 重度拖延 / 嘴硬
## 说话风格 ≤5 条 爱用"草"开头、句尾 wwww、先吐槽再给结论
## 口头禅语录 ≤6 条 带原文引用:"有一说一,这个图确实带"
## 兴趣话题 ≤5 条 明日方舟(资深);聊工作→抱怨、聊感情→回避
## 相处模式 ≤4 条 对小B互怼最多,对新人客气(一句话式,非关系网)
## 时间画像 ≤3 条 深夜 22-02 点活跃,白天潜水
## 样貌参考 ≤3 条 🎨 虚构标注:基于气质的参考描述
## 不确定信息 ≤3 条 疑是学生(语料出现"上课"),未证实
```
- 段内 bullet 为纯文本,可带原文引用(口头禅语录段必须带原文)
- 样貌参考段**必须**带 🎨 虚构标注与设计依据("基于 XX 气质")
- 现实身份信息只能出现在"不确定信息"段
- ⏳ 每段具体生成约束(prompt 细则)属分析层,待细化
## 5. 证据纪律(已定稿 ✅)
三层防编造(借鉴 MaiBot):
1. 印象 prompt 明文约束:"不要添加语料中没有依据的新事实"
2. 规则统计(互动对象、活跃时段)优先于 LLM 分类结果
3. LLM 分类结果默认降级进"不确定信息"段——**模型说的不算稳定真相**
指纹缓存(⏳):证据(印象集 + 统计)hash 未变则不重新生成画像。
## 6. 采集与治理(✅ 消息路径已实现,⏳ 阈值待调)
实现位置:`__init__.py`(on_message 入口 + 鉴权 + 采样)+ `processor.py`(纯函数治理,可单测)。
**两级闸门**:群开关 `persona_group.enabled`(群级,默认关)+ 个人 opt-in `persona_user`(个人,群内开启才生效)。两条同时满足才采集。
- 群开关控制:`开启人设采集 @群` / `关闭人设采集`(⏳ 命令名待定,权限:超管/群主)
- 只采集 `persona_user` 名单内成员(opt-in),退出即停
**脱敏(硬约束:本地正则完成,绝不经过 LLM——LLM 只接触脱敏后文本)**:
敏感值**替换为占位符**而非丢弃整条,保留对话语境(如"借号"互动是人格素材,凭证不是)。
实现:`processor.desensitize()`,四层,覆盖场景:借号/验证码代收/密码口令/兑换码卡密/密保/收入/联系方式变体/位置。
| 规则层 | 模式 | 占位符 |
|---|---|---|
| 明确模式(按最长优先排序,防截胡) | 身份证 → 银行卡 → 邮箱 → 手机号(含分隔变体) → IP → wxid → 坐标 → 车牌 | `[身份证]` `[银行卡]` `[邮箱]` `[手机号]` `[IP]` `[微信号]` `[坐标]` `[车牌]` |
| 定位式(同条"关键词+值") | `密码是 xyz789` `账号 abc123` `激活码 ABCDE-1` `验证码是 123456` `VX: xxx` `月薪 25000`(连接词支持 是/为/冒号/空格) | `[密码]` `[账号]` `[兑换码]` `[验证码]` `[微信号]` `[收入]` |
| 跨条语境 | 关键词在附近消息(如"收下验证码"→ 下一条 `123456`)→ 本条值按语境类型替换 | `[验证码]` `[密码]` `[兑换码]` |
| 兜底 | 裸长数字串 ≥6 位 → `[账号]`;字母+数字混合 ≥6 位 → `[密码]`(保守替换,宁误杀不放过) | `[账号]` `[密码]` |
语境来源:本群最近 3 条已治理文本(复用复读检测的内存窗口)。
待扩展场景(⏳):密保答案("你妈妈的名字")、QQ 号文本、代充代练语境。
- 噪声过滤:纯表情图(无文字)、复读、命令/签到、长链接轰炸
- 长文截断(约 200 字/条)
- 采样:连续刷屏 5 秒内只记 1 条
## 7. 调度与触发(⏳ 草案)
| 方式 | 条件 |
|---|---|
| 手动 | 管理员 `生成人物卡 @xxx`(强制,无视阈值) |
| 印象 | 新语料 ≥ 50 条(⏳)且距上次印象 ≥ 24h |
| 画像 | 新印象 ≥ 5 条(⏳)或语料显著增长;首次需语料 ≥ 200 条 |
| 防重入 | 同人同群生成中加锁 |
## 8. 入口层命令集(⏳ 草案)
`加入人设` `退出人设`(opt-in 控制)、`查看人设 @xxx`(展示九段卡)、`生成人设 @xxx`(管理员强制)。
## 9. 借鉴与不借鉴 MaiBot(已定稿 ✅)
**借鉴**:两级流水线(印象机制)、九段协议格式(段落文本协议)、证据纪律三层、指纹缓存、防串人(证据绑定 user_id)。
**不借鉴**:向量库 + BM25 双路召回 + PPR(语料量级 SQL 直查即可)、完整 A_memorix 记忆系统、md5 person_id(QQ 号即 id)。
## 10. 开发阶段
- **Phase 1**:数据层(四表 + 仓储)✅ 本文档落盘时完成
- **Phase 2**:治理层(采集过滤 + opt-in 命令)
- **Phase 3**:分析层(印象/画像生成,LLM 客户端)
- **Phase 4**:调度层(触发/锁)+ 呈现层(人物卡展示)
- ⏳ 后续可选:galgame 风格卡面图片渲染(协议文本为渲染源)
@@ -30,8 +30,8 @@ from . import models # noqa: F401 # 确保表结构注册
from . import repository, processor
from . import handlers as _handlers # noqa: E402
from hexi.web_config import register_plugin_config # noqa: E402
from hexi.web_plugin_registry import register_web_plugin # noqa: E402
from hexi.web_hub.web_config import register_plugin_config # noqa: E402
from hexi.web_hub.web_plugin_registry import register_web_plugin # noqa: E402
from . import web_hub # noqa: E402
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""群聊人设卡 Web API 子应用(挂载到 /api/galgame_card)。
统一鉴权走 hexi.web_auth(OAuth2 + SQLite),与统一管理台 /hub 共用登录态。
统一鉴权走 hexi.web_hub.web_auth(OAuth2 + SQLite),与统一管理台 /hub 共用登录态。
前端由统一管理台 hexi/web 渲染。
"""
@@ -12,7 +12,7 @@ from typing import Optional
from fastapi import FastAPI
from fastapi.responses import JSONResponse
from hexi.web_auth import require_admin
from hexi.web_hub.web_auth import require_admin
from . import repository
@@ -41,7 +41,7 @@ async def _first_bot():
def build_admin_app() -> FastAPI | None:
"""构建群聊人设卡管理 API 子应用(挂载到 /api/galgame_card)。
鉴权由 hub 统一登录态提供(hexi.web_auth)。
鉴权由 hub 统一登录态提供(hexi.web_hub.web_auth)。
"""
app = FastAPI(title="Galgame Card API")
@@ -17,6 +17,8 @@ from nonebot.params import CommandArg
from nonebot.plugin import PluginMetadata
from nonebot.rule import to_me
from ...core import message_utils
require("nonebot_plugin_alconna")
from nonebot_plugin_alconna import UniMessage # noqa: E402
@@ -31,7 +33,6 @@ from .service import get_services, register_bot_adapter # noqa: E402
from .renderer import html_render # noqa: E402
from .templates import list_templates, template_exists # noqa: E402
# ---------- 指令定义(全部要求 @ 机器人) ----------
analysis_cmd = on_command(
@@ -192,7 +193,7 @@ async def _(bot: Bot, event: GroupMessageEvent):
group_id = str(event.group_id)
days = _parse_days(event)
await UniMessage.text("正在启动分析引擎,正在拉取最近消息...").send()
await message_utils.common_proc_reply(event.message_id)
try:
svc = get_services()
@@ -347,6 +348,7 @@ async def _(bot: Bot, event: GroupMessageEvent):
f"用户称号: {'开' if cm.get_user_title_analysis_enabled() else '关'}",
f"金句分析: {'开' if cm.get_golden_quote_analysis_enabled() else '关'}",
f"聊天质量: {'开' if cm.get_chat_quality_analysis_enabled() else '关'}",
f"本地记录: {'开' if cm.get_use_local_history() else '关'}",
"用法:设置分析 [参数] [值],如:设置分析 天数 3",
]
await UniMessage.text(chr(10).join(lines)).send()
@@ -362,8 +364,12 @@ _SETTING_MAP = {
"称号": ("set_user_title_analysis_enabled", str, "用户称号已{}"),
"金句": ("set_golden_quote_analysis_enabled", str, "金句分析已{}"),
"聊天质量": ("set_chat_quality_analysis_enabled", str, "聊天质量已{}"),
"本地记录": ("set_use_local_history", str, "本地记录已{}"),
}
# 走开/关布尔语义的键(值为 _BOOL_WORDS 里的词)
_BOOL_KEYS = {"话题", "称号", "金句", "聊天质量", "本地记录"}
_BOOL_WORDS = {"开": True, "on": True, "true": True, "关": False, "off": False, "false": False}
@@ -375,7 +381,9 @@ async def _(bot: Bot, event: GroupMessageEvent, args: tuple = CommandArg()):
tokens = _cmd_tokens(args)
if len(tokens) < 2:
await UniMessage.text(
"用法:设置分析 [参数] [值]" + chr(10) + "参数:天数/窗口/最大消息/最小消息/输出格式/话题/称号/金句/聊天质量"
"用法:设置分析 [参数] [值]"
+ chr(10)
+ "参数:天数/窗口/最大消息/最小消息/输出格式/话题/称号/金句/聊天质量/本地记录"
).send()
return
key = tokens[0]
@@ -396,7 +404,7 @@ async def _(bot: Bot, event: GroupMessageEvent, args: tuple = CommandArg()):
await UniMessage.text(f"{key} 已设为 {v}").send()
elif conv is str:
v = val.lower()
if key in ("话题", "称号", "金句", "聊天质量"):
if key in _BOOL_KEYS:
if v not in _BOOL_WORDS:
await UniMessage.text("布尔值请填:开/关 或 on/off").send()
return
@@ -3,12 +3,10 @@
from __future__ import annotations
import asyncio
import base64
import time
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any
from . import history_store
from .core.domain.value_objects.unified_group import UnifiedGroup, UnifiedMember
from .core.domain.value_objects.unified_message import (
MessageContent,
@@ -17,6 +15,29 @@ from .core.domain.value_objects.unified_message import (
)
from .core.utils.logger import logger
# 分页锚点字段链(后端差异实测):NapCat 的 message_seq 参数按 OB11
# message_id(shortId 短 ID 映射) 解析,传消息内的 message_seq(内核 msgSeq)
# 命中不了映射 → NapCat 抛"消息不存在",翻页断裂;go-cqhttp/LLOneBot 等
# 则识别 message_seq。优先 message_id,翻页无进度时依次回退。
ANCHOR_FIELDS = ("message_id", "message_seq", "seq", "real_id")
def _fmt_ts(ts: int) -> str:
"""时间戳转日志用的可读时间。"""
try:
return datetime.fromtimestamp(ts).strftime("%Y-%m-%d %H:%M")
except (OSError, OverflowError, ValueError):
return str(ts)
def _pick_anchor(raw: dict[str, Any], fields: tuple[str, ...], start: int):
"""从消息中按字段优先级提取分页锚点值"""
for field in fields[start:]:
val = raw.get(field)
if val is not None:
return val
return None
class OneBotAdapter:
"""面向 NoneBot OneBot V11 的最小适配器。"""
@@ -31,6 +52,8 @@ class OneBotAdapter:
self.platform_id = str(self.config.get("platform_id") or "onebot")
self.bot_self_ids = [str(x) for x in self.config.get("bot_self_ids", [])]
self.filter_bot_messages = bool(self.config.get("filter_bot_messages", True))
# 优先读本地消息库(learning_chat 落库),取不到再回退接口分页
self.use_local_history = bool(self.config.get("use_local_history", True))
# —— 消息拉取 ——
async def fetch_messages(
@@ -52,8 +75,21 @@ class OneBotAdapter:
start_ts = int(
(datetime.now() - timedelta(days=days)).timestamp()
)
end_ts = int(datetime.now().timestamp())
# 本地消息库优先(完整且毫秒级);before_id 无法映射到时间戳,故跳过
if self.use_local_history and not before_id:
local = await self._fetch_from_local_store(
group_id, start_ts, end_ts, max_count
)
if local:
return local
current_anchor = before_id
anchor_idx = 0
no_progress_pages = 0
last_earliest: dict[str, Any] | None = None
while len(all_raw) < max_count:
fetch_count = min(chunk_size, max_count - len(all_raw))
params: dict[str, Any] = {
@@ -85,13 +121,34 @@ class OneBotAdapter:
break
messages = result.get("messages", [])
if not messages:
# 空页可能意味着当前锚点字段不被后端识别(如 NapCat 外的
# 实现收到 message_id),换下一字段重试一次,仍空则结束。
if current_anchor is None or last_earliest is None:
break
if no_progress_pages >= 1 or anchor_idx >= len(ANCHOR_FIELDS) - 1:
logger.warning(
"OneBot 分页拉取: 返回空页且锚点字段已耗尽,停止回溯"
)
break
no_progress_pages += 1
anchor_idx += 1
current_anchor = _pick_anchor(
last_earliest, ANCHOR_FIELDS, anchor_idx
)
if current_anchor is None:
break
logger.warning(
f"OneBot 分页拉取: 空页,锚点字段切换为 {ANCHOR_FIELDS[anchor_idx]}"
)
continue
first = messages[0]
last = messages[-1]
earliest = first if first.get("time", 0) <= last.get("time", 0) else last
last_earliest = earliest
chunk_earliest_ts = earliest.get("time", 0)
prev_len = len(all_raw)
for raw in messages:
msg_time = raw.get("time", 0)
msg_id = str(raw.get("message_id", ""))
@@ -100,19 +157,42 @@ class OneBotAdapter:
if start_ts <= msg_time <= int(datetime.now().timestamp()):
all_raw.append(raw)
seen_raw_ids.add(msg_id)
added = len(all_raw) - prev_len
seq_val = (
earliest.get("message_seq")
or earliest.get("real_id")
or earliest.get("seq")
)
mid_val = earliest.get("message_id")
new_anchor = seq_val if seq_val is not None else mid_val
if chunk_earliest_ts <= start_ts:
logger.info(
f"OneBot 分页拉取: 已到达起始时间,共 {len(all_raw)} 条"
)
break
if added == 0:
# 本页没有新增消息:锚点不生效(后端不支持该字段)或数据已取尽。
# 先切换锚点字段再试一次,仍无新增则结束。
no_progress_pages += 1
if no_progress_pages >= 2:
logger.warning(
"OneBot 分页拉取: 连续 2 页无新增,停止回溯"
)
break
if anchor_idx < len(ANCHOR_FIELDS) - 1:
anchor_idx += 1
logger.warning(
f"OneBot 分页拉取: 锚点字段切换为 {ANCHOR_FIELDS[anchor_idx]}"
)
else:
no_progress_pages = 0
new_anchor = _pick_anchor(earliest, ANCHOR_FIELDS, anchor_idx)
if new_anchor is None:
break
if current_anchor and str(new_anchor) == str(current_anchor):
logger.info("OneBot 分页拉取: 锚点未位移,历史已取尽")
break
current_anchor = new_anchor
logger.info(
f"OneBot 分页拉取进度: {len(all_raw)} 条,"
f"锚点({ANCHOR_FIELDS[anchor_idx]}): {new_anchor}"
)
await asyncio.sleep(0.05)
unified: list[UnifiedMessage] = []
@@ -131,6 +211,54 @@ class OneBotAdapter:
logger.warning(f"OneBot 分页获取消息失败: {e}")
return []
async def _fetch_from_local_store(
self, group_id: str, start_ts: int, end_ts: int, max_count: int
) -> list[UnifiedMessage]:
"""从本地消息库(learning_chat 落库)取群历史;不可用时返回空以回退分页。"""
try:
result = await asyncio.to_thread(
history_store.fetch_group_messages,
group_id,
start_ts,
end_ts,
max_count,
)
except Exception as e:
logger.warning(f"本地历史读取失败,回退接口分页: {e}")
return []
if not result:
logger.info(
f"本地历史无数据({result.error or '窗口内无消息'}),"
"回退 OneBot 分页拉取"
)
return []
logger.info(
f"本地历史记录拉取: group={group_id}, source={history_store.MESSAGE_TABLE}, "
f"count={len(result.messages)}, "
f"窗口=[{_fmt_ts(start_ts)}..{_fmt_ts(end_ts)}], "
f"名称覆盖={result.names_resolved}/{len(result.messages)}, "
f"去重={result.duplicates}"
)
if result.truncated:
window_total = (
result.window_total if result.window_total is not None else "?"
)
earliest = result.messages[0]["time"] if result.messages else end_ts
logger.warning(
f"本地历史截断: group={group_id}, max_messages={max_count}, "
f"窗口内共 {window_total} 条, 实际取最近 {len(result.messages)} 条, "
f"最早={_fmt_ts(earliest)}, 窗口起点={_fmt_ts(start_ts)}"
)
unified: list[UnifiedMessage] = []
for raw in result.messages:
converted = self._convert_message(raw, group_id)
if converted:
unified.append(converted)
return unified
def _convert_message(self, raw: dict, group_id: str) -> UnifiedMessage | None:
try:
sender = raw.get("sender", {})
@@ -98,6 +98,8 @@ def _default_config() -> dict:
cfg.setdefault("basic", {}).setdefault("max_messages", 1000)
cfg.setdefault("basic", {}).setdefault("min_messages_threshold", 50)
cfg.setdefault("basic", {}).setdefault("filter_bot_messages", True)
# 优先读本地消息库(learning_chat 落库),取不到再回退接口分页
cfg.setdefault("basic", {}).setdefault("use_local_history", True)
cfg.setdefault("analysis_features", {}).setdefault("chat_quality_analysis_enabled", False)
cfg.setdefault("incremental", {}).setdefault("incremental_enabled", False)
return cfg
@@ -770,6 +770,15 @@ class ConfigManager:
self._ensure_group("basic")["filter_bot_messages"] = enabled
self.config.save_config()
def get_use_local_history(self) -> bool:
"""获取是否优先从本地消息库读取群历史。"""
return self._get_group("basic").get("use_local_history", True)
def set_use_local_history(self, enabled: bool):
"""设置是否优先从本地消息库读取群历史(关闭则始终走接口分页)。"""
self._ensure_group("basic")["use_local_history"] = enabled
self.config.save_config()
def get_html_output_dir(self) -> str:
"""获取HTML输出目录"""
@@ -0,0 +1,281 @@
"""本地群历史记录源:只读读取 learning_chat 落库的群消息。
`nonebot_plugin_learning_chat` 会把机器人收到的每条群消息写入
`learning_chat_message`(与该群是否开启学习无关),比 OneBot
`get_group_msg_history` 分页更完整——部分后端只能取回一页就被截断。
昵称/群名片从同库的 `nonebot_plugin_uninfo_*` 表离线补齐,不额外调接口。
模块只依赖标准库(logger 做守卫导入),便于单测用 importlib 裸加载;
读连接一律 `mode=ro`(绝不建库),任何失败都返回带 error 的空结果。
"""
from __future__ import annotations
import json
import os
import sqlite3
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
try: # 插件包内导入:走插件 logger
from .core.utils.logger import logger
except Exception: # 单测裸加载(无包上下文)时降级为标准库 logger
import logging
logger = logging.getLogger(__name__)
MESSAGE_TABLE = "learning_chat_message"
# nonebot_plugin_uninfo 的 SceneType.GROUP
SCENE_TYPE_GROUP = 1
# 显式指定本地消息库位置(测试或非默认部署用)
DB_PATH_ENV = "HEXI_GROUP_DAILY_HISTORY_DB"
_MESSAGE_SQL = (
"SELECT id, user_id, message_id, raw_message, message, plain_text, time "
f"FROM {MESSAGE_TABLE} "
"WHERE group_id = ? AND time >= ? AND time <= ? "
"ORDER BY time DESC, id DESC LIMIT ?"
)
_COUNT_SQL = (
f"SELECT COUNT(*) FROM {MESSAGE_TABLE} "
"WHERE group_id = ? AND time >= ? AND time <= ?"
)
# 同库 uninfo 三表:拿本群成员的 QQ 昵称与群名片
_NAMES_SQL = (
"SELECT u.user_id AS user_id, "
"u.user_data AS user_data, "
"s.member_data AS member_data "
"FROM nonebot_plugin_uninfo_scenemodel AS sc "
"JOIN nonebot_plugin_uninfo_sessionmodel AS s "
"ON s.scene_persist_id = sc.id "
"JOIN nonebot_plugin_uninfo_usermodel AS u "
"ON u.id = s.user_persist_id "
"WHERE sc.scene_id = ? AND sc.scene_type = ?"
)
@dataclass
class LocalHistoryResult:
"""本地历史读取结果;空 messages 表示需要回退到接口分页。"""
messages: list[dict[str, Any]] = field(default_factory=list)
truncated: bool = False
window_total: int | None = None
names_resolved: int = 0
duplicates: int = 0
error: str | None = None
def __bool__(self) -> bool:
return bool(self.messages)
def _parse_sqlite_url(raw: Any) -> Path | None:
"""从 SQLAlchemy 数据库配置里解析出 sqlite 文件路径。"""
if raw is None:
return None
text = str(raw).strip()
if not text.startswith("sqlite") or "memory" in text:
return None
# sqlite+aiosqlite:///D:/path/db.sqlite3 或 sqlite:///./rel/db.sqlite3
_, _, tail = text.partition("///")
tail = tail.strip()
if not tail:
return None
try:
return Path(tail).expanduser()
except (OSError, ValueError):
return None
def _driver_db_path() -> Path | None:
"""读取 `SQLALCHEMY_DATABASE_URL`(环境变量优先,其次 .env 配置)。"""
for key in ("SQLALCHEMY_DATABASE_URL", "sqlalchemy_database_url"):
raw = os.environ.get(key)
if raw:
return _parse_sqlite_url(raw)
try:
from nonebot import get_driver
raw = getattr(get_driver().config, "sqlalchemy_database_url", None)
except Exception:
return None
return _parse_sqlite_url(raw)
def _localstore_db_path() -> Path | None:
"""nonebot_plugin_orm 默认库:localstore 数据目录下的 db.sqlite3。"""
try:
from nonebot_plugin_localstore import get_data_dir
return get_data_dir("nonebot_plugin_orm") / "db.sqlite3"
except Exception:
return None
def resolve_db_path() -> Path:
"""解析本地消息库路径:环境变量 → ORM 配置 → localstore → 仓库默认位置。"""
override = os.environ.get(DB_PATH_ENV)
if override:
return Path(override).expanduser()
for candidate in (_driver_db_path(), _localstore_db_path()):
if candidate is not None:
return candidate
# hexi/plugins/<plugin>/history_store.py -> hexi/
return (
Path(__file__).resolve().parents[2]
/ "data"
/ "nonebot_plugin_orm"
/ "db.sqlite3"
)
def _open_readonly(path: Path) -> sqlite3.Connection:
"""只读打开。路径可能含空格,必须用 as_uri 转义后的 file URI。"""
con = sqlite3.connect(f"{path.resolve().as_uri()}?mode=ro", uri=True, timeout=3.0)
con.row_factory = sqlite3.Row
return con
def _loads(raw: Any) -> dict[str, Any]:
if not raw:
return {}
if isinstance(raw, dict):
return raw
try:
data = json.loads(raw)
except (TypeError, ValueError):
return {}
return data if isinstance(data, dict) else {}
def _load_display_names(con: sqlite3.Connection, group_id: int) -> dict[str, tuple[str, str | None]]:
"""取本群成员的 (昵称, 群名片);uninfo 表缺失时返回空表。"""
names: dict[str, tuple[str, str | None]] = {}
for row in con.execute(_NAMES_SQL, (str(group_id), SCENE_TYPE_GROUP)):
user_data = _loads(row["user_data"])
member_data = _loads(row["member_data"])
nickname = str(user_data.get("name") or "")
card = str(member_data.get("nick") or "") or None
if nickname or card:
names[str(row["user_id"])] = (nickname, card)
return names
def _parse_cq(cq: str) -> list[dict[str, Any]]:
"""CQ 串转 OneBot 消息段(复用适配器自带解析,含反转义)。"""
try:
from nonebot.adapters.onebot.v11 import Message
return [{"type": seg.type, "data": dict(seg.data)} for seg in Message(cq)]
except Exception as exc:
logger.debug(f"本地历史 CQ 解析失败,降级为纯文本: {exc}")
return [{"type": "text", "data": {"text": cq}}]
def _build_raw_message(
row: sqlite3.Row, names: dict[str, tuple[str, str | None]]
) -> dict[str, Any] | None:
"""组装成 adapter._convert_message 能直接消费的 OneBot 原始结构。"""
message_id = row["message_id"]
if message_id is None:
return None
user_id = str(row["user_id"])
nickname, card = names.get(user_id, ("", None))
cq = row["raw_message"] or row["message"] or row["plain_text"] or ""
return {
"message_id": message_id,
"time": int(row["time"] or 0),
"sender": {"user_id": user_id, "nickname": nickname, "card": card or ""},
"message": _parse_cq(cq),
}
def _fetch_group_messages(
group_id: str | int,
start_ts: int,
end_ts: int,
limit: int,
db_path: Path | str | None,
) -> LocalHistoryResult:
try:
gid = int(group_id)
except (TypeError, ValueError):
return LocalHistoryResult(error=f"群号非法: {group_id!r}")
try:
row_limit = max(1, int(limit))
except (TypeError, ValueError):
row_limit = 1000
path = Path(db_path) if db_path is not None else resolve_db_path()
if not path.exists():
return LocalHistoryResult(error=f"本地消息库不存在: {path}")
start, end = int(start_ts), int(end_ts)
con = _open_readonly(path)
try:
# 多取一条用于判定截断(多出来的那条正是窗口内最旧的消息)
rows = con.execute(_MESSAGE_SQL, (gid, start, end, row_limit + 1)).fetchall()
truncated = len(rows) > row_limit
if truncated:
rows = rows[:row_limit]
window_total = None
if truncated:
try:
window_total = int(con.execute(_COUNT_SQL, (gid, start, end)).fetchone()[0])
except (sqlite3.Error, TypeError, ValueError):
window_total = None
try:
names = _load_display_names(con, gid)
except sqlite3.Error as exc:
logger.debug(f"本地历史昵称补齐失败(忽略): {exc}")
names = {}
finally:
con.close()
messages: list[dict[str, Any]] = []
seen: set[str] = set()
duplicates = 0
names_resolved = 0
for row in rows:
raw = _build_raw_message(row, names)
if raw is None:
continue
message_id = str(raw["message_id"])
if not message_id:
continue
if message_id in seen:
duplicates += 1
continue
seen.add(message_id)
sender = raw["sender"]
if sender["nickname"] or sender["card"]:
names_resolved += 1
messages.append(raw)
# 查询按 (time, id) 倒序取最近 N 条,翻回时间升序
messages.reverse()
return LocalHistoryResult(
messages=messages,
truncated=truncated,
window_total=window_total,
names_resolved=names_resolved,
duplicates=duplicates,
)
def fetch_group_messages(
group_id: str | int,
start_ts: int,
end_ts: int,
limit: int = 1000,
db_path: Path | str | None = None,
) -> LocalHistoryResult:
"""读取 [start_ts, end_ts] 闭区间内的群消息(时间升序,最多 limit 条)。
同步函数,调用方用 `asyncio.to_thread` 包;失败只返回带 error 的空结果。
"""
try:
return _fetch_group_messages(group_id, start_ts, end_ts, limit, db_path)
except Exception as exc: # 兜底:本地库问题不该影响分析主链路
logger.warning(f"本地历史读取异常: {exc}", exc_info=True)
return LocalHistoryResult(error=str(exc))
@@ -119,6 +119,7 @@ def register_bot_adapter(bot: Any, platform_id: str | None = None) -> OneBotAdap
"platform_id": platform_id or "onebot",
"bot_self_ids": bot_self_ids,
"filter_bot_messages": config_manager.get_filter_bot_messages(),
"use_local_history": config_manager.get_use_local_history(),
},
)
bot_manager.register_adapter(adapter, platform_id or "onebot")
@@ -8,7 +8,7 @@ from __future__ import annotations
import sys as _sys
from hexi.config_standard import register_config_items
from hexi.web_hub.config_standard import register_config_items
from . import constants
@@ -10,7 +10,7 @@ from nonebot.adapters.onebot.v11 import Bot, GroupMessageEvent, MessageEvent
from nonebot.rule import Rule, is_type
from nonebot_plugin_alconna import UniMessage
from hexi.plugins.nonebot_plugin_hexi_core.custom_utils import check_manage
from hexi.core.custom_utils import check_manage
from ...services.moderation import group_silence, member_silence, silence
from ...utils.message import get_at_id, has_at_all
@@ -6,7 +6,7 @@ from nonebot import on_fullmatch, on_startswith, require
from nonebot.adapters.onebot.v11 import Bot, GroupMessageEvent
from nonebot_plugin_alconna import UniMessage
from hexi.plugins.nonebot_plugin_hexi_core.custom_utils import check_manage
from hexi.core.custom_utils import check_manage
from ...services.profile import card_edit, group_name, title_get
from ...utils.message import get_at_id
@@ -2,7 +2,7 @@ from nonebot.adapters.onebot.v11 import Bot, GroupMessageEvent, MessageEvent
from nonebot_plugin_alconna import UniMessage
from hexi.plugins.nonebot_plugin_hexi_core.custom_utils import check_manage
from hexi.core.custom_utils import check_manage
from ..utils.permissions import is_bot_manage, self_member_info
from ..utils.text import parse_seconds
@@ -2,7 +2,7 @@ from nonebot.adapters.onebot.v11 import Bot, GroupMessageEvent
from nonebot_plugin_alconna import UniMessage
from hexi.plugins.nonebot_plugin_hexi_core.custom_utils import check_manage
from hexi.core.custom_utils import check_manage
from ..utils.permissions import is_bot_manage, self_member_info
@@ -3,7 +3,7 @@ from nonebot.adapters.onebot.v11 import Bot, GroupMessageEvent
from nonebot_plugin_alconna import UniMessage
from hexi.plugins.nonebot_plugin_hexi_core.custom_utils import (
from hexi.core.custom_utils import (
check_bot_admin,
check_manage,
)
@@ -16,7 +16,7 @@ from .utils.war_renderer import render_war_briefing # noqa: F401
from .utils import gen_ms_img, pic2b64 # noqa: F401
from .services.equipment_store import init_db as _init_equipment_db
from .web import stratagem_admin as _stratagem_admin
from hexi.web_plugin_registry import register_web_plugin
from hexi.web_hub.web_plugin_registry import register_web_plugin
__plugin_meta__ = PluginMetadata(
name="绝地潜兵小助手",
@@ -47,7 +47,7 @@ register_web_plugin(
logger.info("战备管理 Web 插件已注册(/api/helldivers)")
# 统一配置标准:把 API 相关配置暴露给 Web 管理台(/hub 插件配置)
from hexi.config_standard import register_model_config # noqa: E402
from hexi.web_hub.config_standard import register_model_config # noqa: E402
from .config import config as _hd2_config # noqa: E402
register_model_config(
@@ -14,6 +14,7 @@ from nonebot.internal.params import ArgPlainText
from nonebot.matcher import Matcher
from PIL import Image
from hexi.core import message_utils
from ..services.equipment import get_equipment_by_combination, get_random_equipment
from ..services.hd2_api import get_briefing_data
from ..utils import gen_ms_img, pic2b64
@@ -24,7 +25,7 @@ _IMG_DIR = _BASE_DIR / "res" / "img"
async def _send_war_card(matcher: Matcher, ev: MessageEvent, top_per_race: Optional[int]) -> None:
await matcher.send("正在获取前线战况,请民主的等待!")
await message_utils.common_proc_reply(ev.message_id)
try:
data = await get_briefing_data()
png = await render_war_briefing(data, top_per_race)
Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

@@ -2,7 +2,7 @@
"""战备管理 API:作为统一 Web 的一个模块,挂到 /api/helldivers。
- 不再自己出 HTML,页面由统一前端 (hexi/web) 渲染。
- 鉴权统一走 hexi.web_auth(OAuth2 + SQLite)。
- 鉴权统一走 hexi.web_hub.web_auth(OAuth2 + SQLite)。
"""
from __future__ import annotations
@@ -12,7 +12,7 @@ from pathlib import Path
from fastapi import Depends, FastAPI, HTTPException, Request
from fastapi.responses import FileResponse, JSONResponse
from hexi.web_auth import require_admin
from hexi.web_hub.web_auth import require_admin
from ..services.equipment_store import (
create_stratagem,
@@ -7,7 +7,7 @@ require("nonebot_plugin_orm")
from .config import NICKNAME, config_manager, register_config as _register_learning_config # noqa: E402
from . import web_hub # noqa: E402
from hexi.web_plugin_registry import register_web_plugin # noqa: E402
from hexi.web_hub.web_plugin_registry import register_web_plugin # noqa: E402
# 注册到统一 Web 管理台(/hub):hub 启动时挂载 /api/learning_chat
register_web_plugin(
@@ -113,7 +113,7 @@ def register_config() -> None:
getter/setter 直接读写 config_manager.config,并保存到 learning_chat.yml。
权威源是插件自身的 yml,故 nosave 不写入 plugin_config.json。
"""
from hexi.config_standard import register_config_items
from hexi.web_hub.config_standard import register_config_items
def _get(key):
return getattr(config_manager.config, key)
@@ -6,7 +6,7 @@ import time
from dataclasses import dataclass
from functools import cached_property, cmp_to_key
# from ..nonebot_plugin_hexi_core.MessageUtils import send_poke
# from ..core.MessageUtils import send_poke
try:
import jieba_fast.analyse as jieba_analyse
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""群聊学习 Web API 子应用(挂载到 /api/learning_chat)。
统一鉴权走 hexi.web_auth(OAuth2 + SQLite),与统一管理台 /hub 共用登录态。
统一鉴权走 hexi.web_hub.web_auth(OAuth2 + SQLite),与统一管理台 /hub 共用登录态。
前端由统一管理台 hexi/web 渲染。
"""
@@ -22,7 +22,7 @@ from nonebot import get_adapter
from nonebot.adapters.onebot.v11 import Adapter
from nonebot_plugin_orm import get_session
from hexi.web_auth import require_admin
from hexi.web_hub.web_auth import require_admin
from .services.learn import LearningChat
from .models import ChatMessage, ChatContext, ChatAnswer, ChatBlackList
@@ -5,7 +5,7 @@ from nonebot.plugin import PluginMetadata
from .config import Config, pc, save_file # noqa: E402,F401
# 统一配置标准:把管理 QQ / 机器人 QQ / 数据文件名暴露给 Web 管理台
from hexi.config_standard import register_model_config # noqa: E402
from hexi.web_hub.config_standard import register_model_config # noqa: E402
register_model_config(
__name__,
@@ -38,4 +38,17 @@ __plugin_meta__ = PluginMetadata(
""",
)
from . import web_hub as _web_hub # noqa: E402
from .handlers import add_server, del_server, list_all, test_server, xinxi # noqa: E402,F401
# 注册到统一 Web 管理台(/hub):hub 启动时挂载 /api/mc_server_status,
# 前端页面 hexi/web/src/plugins/mc_server_status/(服务器数据的增删改)
from hexi.web_hub.web_plugin_registry import register_web_plugin # noqa: E402
register_web_plugin(
"mc_server_status",
"MC服务器",
"server",
lambda: _web_hub.build_admin_app(),
module_name=__name__,
)
@@ -1,4 +1,3 @@
from json import dump, load
from os import makedirs
from pathlib import Path
from typing import List, Optional
@@ -7,6 +6,8 @@ from nonebot import get_bot, get_bots, get_driver, get_plugin_config
from nonebot.adapters import Bot
from pydantic import BaseModel
from .services import servers as store
# 数据目录:项目 hexi/data/ 下(与其它插件数据统一)
DATA_DIR = Path(__file__).resolve().parents[2] / "data"
@@ -40,18 +41,16 @@ async def on_startup():
save_file()
else:
load_file()
# 群命令与 Web 管理台都改 var.group_list,保存时整表落盘(见 services/servers.py)
def load_file():
with open(DATA_DIR / pc.mc_status_data_filename, "r", encoding="utf-8") as r:
tmp_data = load(r)
for i in tmp_data:
var.group_list[int(i)] = tmp_data[i]
"""数据读写都在 services/servers.py(群命令与 Web 管理台共用同一份校验)。"""
var.group_list.update(store.load_servers(pc.mc_status_data_filename))
def save_file():
with open(DATA_DIR / pc.mc_status_data_filename, "w", encoding="utf-8") as w:
dump(var.group_list, w, indent=4, ensure_ascii=False)
store.save_servers(pc.mc_status_data_filename, var.group_list)
# qq机器人连接时执行
@@ -14,7 +14,8 @@ from nonebot.log import logger
from nonebot.params import RegexGroup
from nonebot.plugin import PluginMetadata
from ..config import pc, save_file, var
from ..config import pc, var
from ..services import servers as store
from ..services.mc import check_mc_status
__plugin_meta__ = PluginMetadata(
@@ -76,18 +77,13 @@ async def _(mp=RegexGroup()):
return
group = int(mp[1])
new_server_name = mp[2]
server_host = mp[3]
server_type = mp[4].lower()
if server_type not in ["js", "bds"]:
await add_server.finish("类型请填js或bds")
if group not in var.group_list:
var.group_list[group] = {new_server_name: [server_host, server_type]}
else:
for server_name in var.group_list[group]:
if new_server_name == server_name:
if new_server_name in var.group_list.get(group, {}):
await add_server.finish("有同名服务器啦!")
var.group_list[group][new_server_name] = [server_host, server_type]
save_file()
# 校验 + 落盘都在 services/servers.py(与 Web 管理台同一套)
try:
store.set_server(group, new_server_name, mp[3], mp[4])
except store.ServerError as e:
await add_server.finish("类型请填js或bds" if "类型请填" in str(e) else str(e))
await add_server.finish("添加成功")
@@ -96,18 +92,11 @@ async def _(mp=RegexGroup()):
if not mp[0]:
await del_server.finish(f"删除服务器 [群号] [名称]")
return
group = int(mp[1])
name = mp[2]
if group not in var.group_list:
await del_server.finish("这个群没有添加服务器")
if name in var.group_list[group]:
var.group_list[group].pop(name)
if not var.group_list[group]:
var.group_list.pop(group)
save_file()
try:
store.delete_server(int(mp[1]), mp[2])
except store.ServerError as e:
await del_server.finish(str(e))
await del_server.finish("删除成功")
else:
await del_server.finish("没找到该名称的服务器")
@list_all.handle()
@@ -129,9 +118,9 @@ async def _(mp=RegexGroup()):
if not mp[0]:
await test_server.finish("测试服务器 [服务器地址] [类型]\n类型写js或bds,js是Java服务器,bds是基岩服务器")
return
server_host = mp[1]
server_type = mp[2].lower()
if server_type not in ["js", "bds"]:
await add_server.finish("类型请填js或bds")
msg = await check_mc_status("测试", server_host, server_type)
await list_all.finish(msg)
try:
server_type = store.normalize_type(mp[2])
except store.ServerError as e:
await test_server.finish(str(e))
msg = await check_mc_status("测试", mp[1], server_type)
await test_server.finish(msg)
@@ -5,15 +5,31 @@ from __future__ import annotations
from base64 import b64decode
from io import BytesIO
from re import findall
from typing import Union
from mcstatus import BedrockServer, JavaServer
from nonebot.adapters.onebot.v11 import Message, MessageSegment as MS
def _server_icon(status) -> bytes | None:
"""取服务器图标字节;没有图标或格式不认识就返回 None。
属性名跨版本换过:mcstatus ≥13 叫 `icon`,11.x/12.x 叫 `favicon`
(13.0.0 的 changelog:`JavaStatusResponse.favicon` → `icon`,值没变)。
值可能是完整的 data URI,也可能是不带前缀的裸 base64,两种都认。
"""
raw = getattr(status, "icon", None) or getattr(status, "favicon", None)
if not raw:
return None
data = raw.split("base64,", 1)[-1] if "base64," in raw else raw
try:
return b64decode(data)
except Exception: # noqa: BLE001 — 图标坏了不影响其它字段
return None
async def check_mc_status(
name: str, host: str, server_type: str
) -> Union[str, Message]:
) -> str | Message:
try:
if server_type == "js":
js = await JavaServer.async_lookup(host, timeout=2)
@@ -38,11 +54,8 @@ async def check_mc_status(
else:
player_list = "没人在线"
latency = round(status.latency)
if status.favicon:
aa, bb = status.favicon.split("base64,")
icon = MS.image(BytesIO(b64decode(bb))) + "\n"
else:
icon = ""
icon_data = _server_icon(status)
icon = MS.image(BytesIO(icon_data)) + "\n" if icon_data else ""
msg = (
icon
+ f"名称:{name}\n版本:{version}\n在线:{online}\n延迟:{latency}ms\n在线列表:\n{player_list}"
@@ -0,0 +1,252 @@
"""MC 服务器状态 —— 服务器列表数据层(校验 + 读写 + 增删改)。
内存态就是 `config.var.group_list`,形态是 `{int 群号: {服务器名: [地址, 类型]}}`
(类型 `js`=Java / `bds`=基岩)。群命令(handlers/server.py)与 Web 管理台
(web_hub.py)都只经这里改数据,校验只有这一份,两边不会各自跑偏。
本模块顶层**不 import nonebot**,运行态 config 到函数里再延迟 import
(见 `_runtime()`),这样数据层能脱离 NoneBot 运行时被单测加载。
"""
from __future__ import annotations
import json
import logging
from pathlib import Path
from typing import Any
try: # 单测里可能拿不到 nonebot.logger,退化成标准库
from nonebot.log import logger
except Exception: # noqa: BLE001 — 拿不到日志不影响数据层可用
logger = logging.getLogger(__name__)
#: 服务器类型 → 中文说明(排序即前端下拉顺序)
SERVER_TYPES: dict[str, str] = {"js": "Java 版", "bds": "基岩版"}
#: 地址里写了端口才连得快,否则 Java 版要多探几轮(上游加的注释,保留)
ADDRESS_HINT = "地址带端口查得更快,如 mc.example.com:25565、mc.example.com:19132"
MAX_NAME_LEN = 24
MAX_GROUP_LEN = 12
MAX_ADDRESS_LEN = 64
class ServerError(ValueError):
"""校验失败 / 目标不存在,`str(e)` 就是给用户看的话术。"""
def _runtime() -> Any:
"""取运行态 `config` 模块(数据目录、配置、内存态都在那边)。
到调用时才 import:本模块要能在"插件包还没初始化"的场景下被加载,
只有真正读写数据时才需要 config(单测按包结构加载本模块,见 tests/test_mc_servers.py)。
"""
from .. import config
return config
# ───────────────────────── 校验 / 归一化 ─────────────────────────
def normalize_type(value: Any, default: str = "js") -> str:
"""类型归一:`js`/`java` → js,`bds`/`be`/`bedrock` → bds;空值取 default。"""
if value is None or str(value).strip() == "":
return default
text = str(value).strip().lower()
if text in ("js", "java"):
return "js"
if text in ("bds", "be", "bedrock"):
return "bds"
raise ServerError("类型请填 js 或 bds(js = Java 版,bds = 基岩版)")
def normalize_address(value: Any) -> str:
"""地址归一:去空白、去协议前缀与尾部斜杠;`bds` 默认端口 19132 不写进数据里。"""
text = str(value or "").strip()
if not text:
raise ServerError("服务器地址不能为空")
for prefix in ("minecraft://", "mc://", "http://", "https://"):
if text.lower().startswith(prefix):
text = text[len(prefix) :]
break
text = text.strip().strip("/").strip()
if not text:
raise ServerError("服务器地址不能为空")
if any(ch.isspace() for ch in text) or "/" in text:
raise ServerError("服务器地址里不能有空格或斜杠")
if len(text) > MAX_ADDRESS_LEN:
raise ServerError(f"服务器地址太长了(最多 {MAX_ADDRESS_LEN} 个字符)")
if text.startswith(":") or text.endswith(":"):
raise ServerError("服务器地址的端口不能为空")
return text
def normalize_name(value: Any) -> str:
"""服务器名归一:去首尾空白、单双引号;名字里有空格是允许的。"""
text = str(value or "").strip()
if len(text) >= 2 and text[0] == text[-1] and text[0] in "\"'“”":
text = text[1:-1].strip()
if not text:
raise ServerError("服务器名称不能为空")
if len(text) > MAX_NAME_LEN:
raise ServerError(f"服务器名称太长了(最多 {MAX_NAME_LEN} 个字符)")
return text
def parse_group_id(value: Any) -> int:
"""群号:必须是纯数字(DataStore/命令层都按 int 存)。"""
text = str(value if value is not None else "").strip()
if not text:
raise ServerError("群号不能为空")
if len(text) > MAX_GROUP_LEN:
raise ServerError("群号看上去不对(最多 12 位数字)")
if not text.isdigit():
raise ServerError("群号必须是纯数字")
if not int(text):
raise ServerError("群号看上去不对")
return int(text)
# ───────────────────────── 数据读写 ─────────────────────────
def data_path(filename: str) -> Path:
return _runtime().DATA_DIR / filename
def load_servers(filename: str) -> dict[int, dict[str, list[str]]]:
"""读数据文件;文件不存在/内容坏了按"空数据 + 警告"处理,绝不让插件起不来。"""
path = data_path(filename)
if not path.exists():
return {}
try:
with open(path, encoding="utf-8") as f:
raw = json.load(f)
except Exception as e: # noqa: BLE001
logger.warning(f"MC 服务器数据读取失败(按空数据处理):{path} {e!r}")
return {}
data: dict[int, dict[str, list[str]]] = {}
if not isinstance(raw, dict):
logger.warning(f"MC 服务器数据格式不是对象,已忽略:{path}")
return data
for group_key, servers in raw.items():
try:
group_id = int(group_key)
except (TypeError, ValueError):
logger.warning(f"MC 服务器数据里有个非数字群号,已忽略:{group_key!r}")
continue
if not isinstance(servers, dict):
continue
clean: dict[str, list[str]] = {}
for name, value in servers.items():
if isinstance(value, (list, tuple)) and len(value) >= 2:
clean[str(name)] = [str(value[0]), str(value[1])]
data[group_id] = clean
return data
def save_servers(filename: str, data: dict[int, dict[str, list[str]]]) -> None:
"""整表落盘(先写 .tmp 再替换,避免写一半断电留下半截 JSON)。"""
path = data_path(filename)
path.parent.mkdir(parents=True, exist_ok=True)
tmp = path.with_suffix(path.suffix + ".tmp")
with open(tmp, "w", encoding="utf-8") as f:
json.dump(data, f, indent=4, ensure_ascii=False)
tmp.replace(path)
def _state() -> tuple[Any, dict[int, dict[str, list[str]]]]:
config = _runtime()
return config, config.var.group_list
def snapshot() -> dict[int, dict[str, list[str]]]:
"""整表深拷贝(给 Web 只读接口用,避免调用方误改内存态)。"""
_, groups = _state()
return {gid: {name: list(value) for name, value in servers.items()} for gid, servers in groups.items()}
def mutate(func, *args, **kwargs):
"""校验通过后才落盘;冲突类错误不要走到这里(改内存前就该抛)。"""
config, groups = _state()
result = func(groups, *args, **kwargs)
save_servers(config.pc.mc_status_data_filename, groups)
return result
# ───────────────────────── 增删改 ─────────────────────────
def _upsert(
groups: dict[int, dict[str, list[str]]],
group_id: int,
old_name: str | None,
name: str,
address: str,
server_type: str,
) -> bool:
servers = groups.setdefault(group_id, {})
if old_name and old_name != name:
# 改名:先查旧名在不在、新名会不会撞,确认没问题再动数据
if old_name not in servers:
raise ServerError(f"找不到要改名的服务器「{old_name}」")
if name in servers:
raise ServerError(f"群里已经有一个叫「{name}」的服务器了")
servers.pop(old_name)
servers[name] = [address, server_type]
return True
created = name not in servers
servers[name] = [address, server_type]
return created
def set_server(
group_id: int,
name: str,
address: str,
server_type: str,
old_name: str | None = None,
) -> bool:
"""新增/更新一台服务器,返回是否新建。`old_name` 传了且与 name 不同即为改名。"""
group_id = parse_group_id(group_id)
name = normalize_name(name)
address = normalize_address(address)
server_type = normalize_type(server_type)
old = normalize_name(old_name) if old_name else None
return mutate(_upsert, group_id, old, name, address, server_type)
def _delete(
groups: dict[int, dict[str, list[str]]], group_id: int, name: str | None
) -> None:
if group_id not in groups:
raise ServerError(f"群 {group_id} 还没有配置服务器")
if name is None: # 整个群一起删
groups.pop(group_id)
return
if name not in groups[group_id]:
raise ServerError(f"群里没有叫「{name}」的服务器")
groups[group_id].pop(name)
if not groups[group_id]: # 删空了就把群一起收掉,别留空壳
groups.pop(group_id)
def delete_server(group_id: int, name: str) -> None:
"""删掉某群里的一台服务器(删空了连带删群)。"""
group_id = parse_group_id(group_id)
mutate(_delete, group_id, normalize_name(name))
def delete_group(group_id: int) -> None:
"""删掉整个群的服务器配置。"""
group_id = parse_group_id(group_id)
mutate(_delete, group_id, None)
def lookup(group_id: int, name: str) -> list[str] | None:
"""按群号 + 服务器名取 `[地址, 类型]`(给 Web 的「测试连接」用)。"""
_, groups = _state()
return groups.get(int(group_id), {}).get(name)
@@ -0,0 +1,159 @@
"""MC 服务器信息查询 —— Web API 子应用(挂载到 /api/mc_server_status)。
服务器数据(`hexi/data/mc_status_data.json`:`{群号: {名字: [地址, 类型]}}`)
的唯一 Web 读写入口:列出 / 新增 / 编辑(含改名)/ 删除 / 测试连接。
写操作全部落在 services/servers.py(校验只有那一份),和群里的
`添加服务器`/`删除服务器` 命令改的是同一份内存态 + 同一个文件。
鉴权走 hexi.web_hub.web_auth(OAuth2 + SQLite),与统一管理台 /hub 共用登录态。
**每条路由都要自带 `dependencies=[auth]`** —— mount 层没有兜底,漏一条就是匿名可访问。
前端页面:hexi/web/src/plugins/mc_server_status/index.tsx。
"""
from __future__ import annotations
from fastapi import FastAPI
from fastapi.responses import JSONResponse
from hexi.web_hub.web_auth import require_admin
from .services import mc as mc_service
from .services import servers as store
API = require_admin
def _ok(data=None, msg: str = "ok") -> JSONResponse:
return JSONResponse({"status": 0, "msg": msg, "data": data})
def _fail(msg: str, status: int = 400) -> JSONResponse:
return JSONResponse({"status": status, "msg": msg})
async def _group_names() -> dict[str, str]:
"""群号 → 群名;拿不到 bot(未连接)时返回空表,前端只显示群号。"""
try:
from nonebot import get_adapter
from nonebot.adapters.onebot.v11 import Adapter
bots = get_adapter(Adapter).bots
bot = next(iter(bots.values()), None)
if bot is None:
return {}
return {
str(g["group_id"]): g.get("group_name") or ""
for g in await bot.get_group_list()
}
except Exception: # noqa: BLE001 — 未连接/适配器未加载都按拿不到处理
return {}
def _payload() -> dict:
"""整表视图:群列表(含群名与在线标记)+ 类型选项。"""
groups = store.snapshot()
return {
"groups": [
{
"group_id": str(gid),
"servers": [
{"name": name, "address": value[0], "type": value[1]}
for name, value in sorted(servers.items())
],
}
for gid, servers in sorted(groups.items())
],
"types": store.SERVER_TYPES,
"address_hint": store.ADDRESS_HINT,
}
async def _payload_with_names() -> dict:
"""跑一次 payload 再把群名补上(两条路由共用的收尾)。"""
data = _payload()
names = await _group_names()
for row in data["groups"]:
row["group_name"] = names.get(row["group_id"], "")
row["online"] = row["group_id"] in names
data["bot_online"] = bool(names)
return data
def build_admin_app() -> FastAPI | None:
"""构建 MC 服务器数据管理 API 子应用(挂载到 /api/mc_server_status)。"""
app = FastAPI(title="MC Server Status API")
auth = require_admin
@app.get("/overview", response_class=JSONResponse, dependencies=[auth])
async def overview():
"""一次拿全:所有群 + 服务器列表 + 类型选项。"""
return _ok(await _payload_with_names())
@app.post("/server", response_class=JSONResponse, dependencies=[auth])
async def save_server(data: dict):
"""新增/更新一台服务器;带 old_name 且与 name 不同即为改名。"""
try:
group_id = store.parse_group_id(data.get("group_id"))
name = store.normalize_name(data.get("name"))
created = store.set_server(
group_id,
name,
str(data.get("address") or ""),
str(data.get("type") or ""),
old_name=str(data.get("old_name") or "").strip() or None,
)
except store.ServerError as e:
return _fail(str(e))
return _ok(
await _payload_with_names(),
f"服务器「{name}」已{'添加' if created else '保存'}",
)
@app.delete("/server/{group_id}/{name}", response_class=JSONResponse, dependencies=[auth])
async def remove_server(group_id: str, name: str):
"""删掉某群里的一台服务器(删空了连带删群)。"""
try:
store.delete_server(group_id, name)
except store.ServerError as e:
return _fail(str(e))
return _ok(await _payload_with_names(), f"服务器「{name}」已删除")
@app.delete("/group/{group_id}", response_class=JSONResponse, dependencies=[auth])
async def remove_group(group_id: str):
"""删掉整个群的服务器配置。"""
try:
store.delete_group(group_id)
except store.ServerError as e:
return _fail(str(e))
return _ok(await _payload_with_names(), f"群 {group_id} 的服务器配置已删除")
@app.post("/test", response_class=JSONResponse, dependencies=[auth])
async def test_server(data: dict):
"""测试连接:给了群号+名字就测存量那台,否则测临时填的地址(不落库)。
对不上的地址走 mcstatus 自己的 2 秒超时,返回的是查询失败文案而不是报错。
"""
group_id = str(data.get("group_id") or "").strip()
name = str(data.get("name") or "").strip()
address = str(data.get("address") or "").strip()
server_type = str(data.get("type") or "").strip()
if group_id.isdigit() and name:
found = store.lookup(int(group_id), name)
if found:
address, server_type = found[0], found[1]
if not address:
return _fail("先填服务器地址")
try:
address = store.normalize_address(address)
server_type = store.normalize_type(server_type)
except store.ServerError as e:
return _fail(str(e))
message = await mc_service.check_mc_status(name or "测试", address, server_type)
# check_mc_status 正常返回 str;带图标的 Java 服返回 Message(Web 侧只要纯文本)
text = str(message) if not isinstance(message, str) else message
return _ok({"ok": "查询失败" not in text, "message": text}, "测试完成")
return app
@@ -7,7 +7,7 @@ from __future__ import annotations
import sys as _sys
from hexi.config_standard import register_config_items
from hexi.web_hub.config_standard import register_config_items
_PLUGIN_ID = __package__
@@ -18,7 +18,7 @@ __plugin_meta__ = PluginMetadata(
)
# 统一配置标准:把「插件内部常量」也暴露给 Web 管理台(来源无关)。
from hexi.config_standard import register_config_items # noqa: E402
from hexi.web_hub.config_standard import register_config_items # noqa: E402
from . import config as _cfgmod # noqa: E402
register_config_items(
@@ -20,7 +20,7 @@ from .templates import load_builtin_templates, loaded_templates
# 统一配置标准:把常用、可安全热更的配置暴露给 Web 管理台
# (路径/嵌套模型/复杂 list 字段不改,避免格式互转出错)
from hexi.config_standard import register_model_config # noqa: E402
from hexi.web_hub.config_standard import register_model_config # noqa: E402
register_model_config(
__name__,
@@ -9,7 +9,7 @@ from __future__ import annotations
import sys as _sys
from hexi.config_standard import register_config_items
from hexi.web_hub.config_standard import register_config_items
# plugin_id = NoneBot 插件模块名
_PLUGIN_ID = __package__
@@ -13,7 +13,7 @@ from nonebot.plugin.on import on_notice, on_command
from nonebot.rule import to_me
from nonebot_plugin_alconna import UniMessage, get_target
from hexi.plugins.nonebot_plugin_hexi_core.message_utils import send_poke
from hexi.core.message_utils import send_poke
from .. import config as _cfg
from ..services.store import add_new_jm_code, get_random_jm_code
@@ -8,7 +8,8 @@ from nonebot.log import logger
from nonebot.plugin.on import on_keyword
from nonebot_plugin_alconna import UniMessage
from hexi.plugins.nonebot_plugin_hexi_core.message_utils import get_reply_message
from hexi.core import message_utils
from hexi.core.message_utils import get_reply_message
from ..services.gif import reverse_gif_bytes
@@ -21,10 +22,11 @@ re_gif = on_keyword({"倒放"})
async def rev_gif(event: MessageEvent, bot: Bot):
# 情况1,用户对需要倒放的gif进行回复
messages = get_reply_message(event)
await match_revgif(messages)
message_id = event.message_id
await match_revgif(messages,message_id)
async def match_revgif(messages):
async def match_revgif(messages,message_id):
img_urls = []
for message in messages:
logger.info(f"遍历消息: {message}")
@@ -38,7 +40,7 @@ async def match_revgif(messages):
return
for url in img_urls:
logger.info(f"获取到的图片链接:{url}")
await UniMessage.text("ℹ正在翻转图片序列,请稍候").send()
await message_utils.common_proc_reply(message_id)
data, err = await reverse_gif_bytes(img_urls)
if err:
await UniMessage.text(err).send()
@@ -6,12 +6,9 @@ from io import BytesIO
from pathlib import Path
from nonebot.log import logger
from PIL import Image as PILImage
from nonebot.params import Depends
from nonebot.params import CommandArg
from nonebot import on_command, require
from nonebot.permission import SUPERUSER
from nonebot import require
from typing import Optional, List, Dict
from nonebot.adapters import Message, Event
from nonebot.adapters import Event
from nonebot.plugin import PluginMetadata, inherit_supported_adapters
require("nonebot_plugin_alconna")
@@ -20,9 +17,8 @@ require("nonebot_plugin_apscheduler")
from nonebot_plugin_alconna.uniseg import get_target as uni_get_target
import nonebot_plugin_localstore as store
from nonebot_plugin_apscheduler import scheduler
from nonebot_plugin_alconna import Text, Image, UniMessage, Target, At
from nonebot.adapters.onebot.v11 import Bot, MessageEvent, MessageSegment
from hexi.plugins.nonebot_plugin_hexi_core.cooldown import cooldown
from nonebot_plugin_alconna import Text, Image, UniMessage, Target
from nonebot.adapters.onebot.v11 import Bot
from .config import Config
from .models import ProcessedPlayer
from .data_source import BindData, SteamInfoData, ParentData, DisableParentData
@@ -122,7 +118,7 @@ except Exception as e:
logger.error(f"初始化失败: {e}")
# 统一配置标准:把常见可调配置暴露给 Web 管理台
from hexi.config_standard import register_model_config # noqa: E402
from hexi.web_hub.config_standard import register_model_config # noqa: E402
register_model_config(
__name__,
@@ -4,14 +4,14 @@ from __future__ import annotations
import asyncio
from nonebot import logger, on_command, require
from nonebot import logger, on_command
from nonebot.adapters import Message, Event
from nonebot.adapters.onebot.v11 import Bot, MessageEvent, MessageSegment
from nonebot.params import CommandArg, Depends
from nonebot.permission import SUPERUSER
from nonebot_plugin_alconna import At, Image, Target, Text, UniMessage
from nonebot_plugin_alconna import At, Image, Target, UniMessage
from hexi.plugins.nonebot_plugin_hexi_core.cooldown import cooldown
from hexi.core.cooldown import cooldown
# 业务/状态在插件根 __init__ 中(本插件为社区插件,服务层保留在根模块)
from .. import (
@@ -36,11 +36,8 @@ from .. import (
get_user_data,
image_to_bytes,
key_select,
key_select_2,
localize_game_name,
parent_data,
simplize_steam_player_data,
steam_info_data,
_fill_missing_icon_hashes,
_query_game_names,
_upsert_game_icon_hashes,
@@ -9,7 +9,7 @@ from typing import List, Optional, Dict
from datetime import datetime, timezone
import asyncio
import json
from hexi.plugins.nonebot_plugin_hexi_core.rate_limit import acquire
from hexi.core.rate_limit import acquire
from .config import Config
@@ -203,7 +203,7 @@ def format_api_key_stats(stats: Dict) -> str:
stats 结构: {key: {"call_count": int, "last_called": str | None}}
"""
lines = ["Steam API key 使用统计:"]
lines = ["\nSteam API key 使用统计:"]
for key, info in stats.items():
call_count = info.get("call_count", 0)
last_called = info.get("last_called")
@@ -18,5 +18,22 @@ __plugin_meta__ = PluginMetadata(
# 显式导入子模块:注册配置 schema + 消息 matcher(配合 load_plugins 只加载到包层)
from . import config as _config # noqa: E402
from . import handlers as _handlers # noqa: E402
from . import list_proc as _list_proc # noqa: E402
from . import web_hub as _web_hub # noqa: E402
from hexi.web_hub.web_plugin_registry import register_web_plugin # noqa: E402
# 注册到统一 Web 管理台(/hub):hub 启动时挂载 /api/video_analysis,
# 前端页面 hexi/web/src/plugins/video_analysis/
register_web_plugin(
"video_analysis",
"视频解析",
"video",
lambda: _web_hub.build_admin_app(),
module_name=__name__,
)
_config.register_config()
# 导入期把群策略读进内存(含 list.json v1/v2 → v3 迁移),
# 之后 verify_user 走内存,不再碰磁盘。
_list_proc.warmup()
@@ -10,7 +10,9 @@ hexi/data/temp 是下载媒体中转区(见 utils.get_temp_root),发送成功后
- /清理temp [天数] 清理 temp 下超过 N 天(默认取配置)未修改的文件
- /temp统计 查看 temp 目录占用情况
目录结构始终保留,占用中的文件自动跳过。
媒体按「作者目录/作品目录」分层落盘,文件删完后这些空目录会一并收掉
(只删同为过期、且确实为空的目录 —— 正在落盘的目录 mtime 很新,不会误删)。
占用中的文件自动跳过。
"""
import asyncio
@@ -22,7 +24,7 @@ from nonebot import logger, require, get_plugin_config, on_command
from nonebot.adapters.onebot.v11 import Bot, MessageEvent
from pydantic import BaseModel
from hexi.plugins.nonebot_plugin_hexi_core.custom_utils import check_manage
from hexi.core.custom_utils import check_manage
require("nonebot_plugin_apscheduler")
from nonebot_plugin_apscheduler import scheduler
@@ -71,11 +73,39 @@ def _walk_files(root: Path) -> list[Path]:
return files
def _prune_empty_dirs(root: Path, max_age: float) -> int:
"""自底向上收掉空目录(作者的层与作品的层都算),返回删除的目录数。
只删「本身就是空」且 mtime 已过期的目录:正在落盘的目录刚建出来、
mtime 很新,不会被误删;刚删完文件的目录 mtime 会被刷新,留到下一轮。
root 自身不在 rglob 结果里,不会被删。
"""
removed = 0
# 目录另有 60s 下限:清理temp 0 时不能把"刚建出来、还没写第一个文件"的
# 目录(下载落盘点先 mkdir 再 open)删掉
dir_age = max(max_age, 60)
dirs = [p for p in root.rglob("*") if p.is_dir()]
# 深的先处理:子目录删掉后父目录才可能变空,同一轮里能被顺带收掉
for path in sorted(dirs, key=lambda p: len(p.parts), reverse=True):
try:
if any(path.iterdir()):
continue
if not _file_is_stale(path, dir_age):
continue
path.rmdir()
removed += 1
logger.info(f"temp 清理: 删除空目录 {path}")
except OSError:
# 被占用 / 刚被别的进程删掉 → 留待下轮
continue
return removed
def clean_temp_files(sub: str = "", days: int | None = None) -> tuple[int, int]:
"""清理 temp[/sub] 下超过期限的文件。
"""清理 temp[/sub] 下超过期限的文件与随之空掉的目录。
Returns:
(removed, total) — 删除数、统计到的文件总数
(removed, total) — 删除的文件数、统计到的文件总数
"""
root = get_temp_root(sub)
if not root.is_dir():
@@ -94,17 +124,22 @@ def clean_temp_files(sub: str = "", days: int | None = None) -> tuple[int, int]:
except OSError as e:
# 文件被占用(如发送中)等场景,留待下轮
logger.warning(f"temp 清理: 跳过 {path} ({e})")
dirs = _prune_empty_dirs(root, max_age)
if dirs:
logger.info(f"temp 清理: 同时收掉 {dirs} 个空目录")
return removed, len(files)
def temp_stats(sub: str = "") -> dict:
"""统计 temp[/sub] 目录:文件数、总大小(字节)"""
"""统计 temp[/sub] 目录:文件数、总大小(字节)、目录数(含作者/作品层)"""
root = get_temp_root(sub)
if not root.is_dir():
return {"files": 0, "bytes": 0}
return {"files": 0, "bytes": 0, "dirs": 0}
files = _walk_files(root)
total_bytes = sum(p.stat().st_size for p in files if p.exists())
return {"files": len(files), "bytes": total_bytes}
dirs = sum(1 for p in root.rglob("*") if p.is_dir())
return {"files": len(files), "bytes": total_bytes, "dirs": dirs}
# ── 手动清理命令(manual 模式,auto 模式下也可用) ──────────────
@@ -141,11 +176,18 @@ async def _handle_clean(bot: Bot, event: MessageEvent):
async def _handle_stats(event: MessageEvent):
st = await _run_stats()
if st["files"] == 0:
if st.get("dirs"):
await UniMessage.text(
f"temp 目录下没有文件了,还剩 {st['dirs']} 个空目录"
f"(下次清理/清理temp 会一并收掉)。"
).send()
else:
await UniMessage.text("temp 目录目前是空的。").send()
else:
size_mb = st["bytes"] / 1024 / 1024
await UniMessage.text(
f"temp 目录:共 {st['files']} 个文件,占用 {size_mb:.1f} MB。"
f"temp 目录:共 {st['files']} 个文件,占用 {size_mb:.1f} MB,"
f"{st.get('dirs', 0)} 个目录。"
).send()
@@ -1,21 +1,48 @@
"""统一配置注册:把插件配置按文档标准接入 hexi.web_config(Web 可读可改)。
"""统一配置注册:把插件配置按文档标准接入 hexi.web_hub.web_config(Web 可读可改)。
- temp 清理配置(env pydantic) → register_model_config
- 群文件投递配置(打包/解压密码, env pydantic) → register_model_config
- S3 存储配置(原硬编码在 services/storage/s3.py) → register_config_items(store=s3 模块)
- 群分组配置(data/list.json) → register_config_items(type=json, nosave)
群策略(data/list.json v3)不在这里注册:它有独立的 Web 页面与 API
(见 web_hub.py),不再走配置抽屉里的裸 JSON 编辑。
"""
from __future__ import annotations
from hexi.config_standard import register_config_items, register_model_config
from nonebot import get_plugin_config
from pydantic import BaseModel
from . import cleanup, list_proc # noqa: F401
from hexi.web_hub.config_standard import register_config_items, register_model_config
from . import cleanup # noqa: F401
from .services.storage import s3 as _s3mod
# 插件模块名 = plugin_id(与 NoneBot 模块名一致)
_PLUGIN_ID = __package__
class GroupFileConfig(BaseModel):
# 上传群文件时打包成一个 zip(关掉则逐个上传原文件)
video_analysis_group_file_zip: bool = True
# 压缩包解压密码(留空 = 不加密; 设置后要求 pyzipper 可用, 否则放弃群文件上传)
video_analysis_group_file_password: str = ""
group_file_config = get_plugin_config(GroupFileConfig)
def group_file_settings() -> tuple[bool, str]:
"""群文件投递设置:(是否打包成 zip, 解压密码)。
运行期读取实例属性,所以 Web 保存后立即生效(apply 是 setattr)。
"""
return (
bool(group_file_config.video_analysis_group_file_zip),
str(group_file_config.video_analysis_group_file_password or ""),
)
def _reset_s3_caches(_values=None, store=None):
"""保存 S3 配置后清空懒加载客户端缓存,让下次上传用新配置重建。"""
if store is None:
@@ -59,7 +86,29 @@ def register_config() -> None:
apply_extra=lambda _values, _conf: cleanup.reload_cleanup_config(),
)
# 2) S3 存储配置(来源无关)
# 2) 群文件投递:打包 / 解压密码
register_model_config(
_PLUGIN_ID,
group_file_config,
fields=[
"video_analysis_group_file_zip",
"video_analysis_group_file_password",
],
labels={
"video_analysis_group_file_zip": "群文件打包成 zip",
"video_analysis_group_file_password": "压缩包解压密码",
},
descriptions={
"video_analysis_group_file_zip": "开启后群文件收到的是一个压缩包;关闭则逐个上传原文件",
"video_analysis_group_file_password": "留空 = 不加密;设置后用 AES-256 加密(需要 7-Zip/WinRAR 等工具解压),缺 pyzipper 时会放弃群文件上传而不是传明文",
},
types={
"video_analysis_group_file_zip": "bool",
"video_analysis_group_file_password": "password",
},
)
# 3) S3 存储配置(来源无关)
register_config_items(
_PLUGIN_ID,
[
@@ -86,19 +135,3 @@ def register_config() -> None:
store=_s3mod,
apply_extra=_reset_s3_caches,
)
# 3) 群分组配置(data/list.json, 权威源在插件自身)
register_config_items(
_PLUGIN_ID,
[
{
"key": "group_config",
"label": "群分组配置",
"type": "json",
"description": "data/list.json 内容。groups 为 {群号: {auto, plan(A/B), auto_link[]}},blacklist 为禁用用户 QQ 列表。白名单即 groups 的键。",
"getter": list_proc.get_group_config_sync,
"setter": list_proc.set_group_config_sync,
"nosave": True,
}
],
)
@@ -12,6 +12,7 @@ from ..services.fetchers.douyin_api import fetch_douyin_content
from ..services.fetchers.douyin_ssr import MOBILE_UA, fetch_douyin_note_ssr
from ..models import DouyinFetchError
from ..utils import get_data_dir, parse_netscape_cookies
from ..policy import Policy
from .sender import PendingMedia, _as_paths
SHORT_LINK_PATTERN = re.compile(r"(v\.douyin\.com/[A-Za-z0-9_\-]+)")
@@ -68,13 +69,18 @@ async def parse_douyin(
logger.info(f"短链重定向: {target_url} -> {resolved}")
target_url = resolved
else:
# 全链接: www.douyin.com/note/xxx 或 www.douyin.com/video/xxx
# 全链接: www.douyin.com/note/xxx、www.douyin.com/video/xxx,
# 以及抖音复制出的 iesdouyin.com/share/video|note/xxx(弱网/无网下 App
# 生成的恢复连接同样会是这个形态)。统一提取出作品 id 后再交给 SSR / playwright。
m = re.search(
r"(https?://(?:www\.)?douyin\.com/(?:note|video)/\d+)", url
r"(https?://(?:www\.)?(?:m\.)?(?:ies)?douyin\.com/(?:share/)?(note|video)/(\d+))",
url,
)
if not m:
return None, None, False
target_url = m.group(1)
# 统一规范化为 www.douyin.com/{note|video}/{id},避免 iesdouyin 分享页
# 在无登录态/弱网下只返回壳页,直接落到浏览器可正常打开的作品页。
target_url = f"https://www.douyin.com/{m.group(2)}/{m.group(3)}"
try:
is_img_post = False
@@ -106,11 +112,13 @@ async def process_douyin_res(
file_paths: Union[Path, list[Path]],
is_private: bool,
image_post: bool,
plan: str | None = None,
policy: Policy | None = None,
platform: str | None = None,
) -> tuple[Optional[PendingMedia], Optional[str]]:
"""下载已完成 → 打包为待发送媒体(不上传、不发送、不清理)
多级发送(temp 本地 → S3 链接 → 回退本地)由 send_pending_media 统一处理。
多级发送(temp 本地 → S3 链接 → 回退本地)与群文件上传由
send_pending_media 统一按 policy 处理;platform 用于群文件限定平台。
"""
if not file_paths:
return None, None
@@ -119,7 +127,8 @@ async def process_douyin_res(
files=_as_paths(file_paths),
image_post=image_post,
is_private=is_private,
plan=plan,
policy=policy,
platform=platform,
title=title,
),
None,
@@ -4,7 +4,6 @@ from __future__ import annotations
import html
import re
from typing import Optional
from nonebot import on_message, logger
from nonebot.adapters import Event
@@ -17,7 +16,8 @@ from ..services.fetchers.rednote_content import fetch_rednote_content
from .douyin import parse_douyin, process_douyin_res
from .sender import PendingMedia, send_pending_media
from .universal import handle_universal
from ..list_proc import AUTO_LINK_KEYWORDS, get_group_auto_link, verify_user
from ..list_proc import get_policy, is_group_whitelisted, is_user_blacklisted
from ..policy import Policy, match_platform
URL_PATTERN = re.compile(r"(https?://\S+)")
XCX_PATTERN = r"QQ小程序(?:&amp;#93;|&#93;|\])"
@@ -45,64 +45,71 @@ active_video_handler = on_message(priority=10, block=False, rule=to_me())
async def _check_access(
event: Event, *, auto_only: bool = False, msg: str | None = None
) -> tuple[bool, str | None]:
) -> tuple[bool, Policy | None]:
"""统一权限检查。
auto_only=True → 自动解析:需白名单 + 开启自动解析,或 auto_link 关键词命中。
auto_only=False → 主动触发:需非黑名单,群聊还需白名单。
auto_only=True → 自动解析:需白名单群 + (开启自动解析或消息命中自动策略)。
auto_only=False → 主动触发:需白名单群 + 非黑名单用户。
私聊不做自动解析,读 default 节策略后直接解析。
禁用策略(ban_link)是消息级的(按链接判定),见 match_message。
Returns:
(allowed, plan) — plan 用于 S3 路由,不允许时为 None
(allowed, policy) — policy 决定存储/投递,不允许时为 None
"""
white, black, auto, plan = await verify_user(event)
target = get_target(event)
if target.private:
if auto_only:
logger.info("权限分析:自动解析不处理私聊")
return False, None
if black:
if is_user_blacklisted(event):
logger.info(f"权限分析:黑名单用户私聊,不回复: {event.get_user_id()}")
return False, None
logger.info("权限分析:私聊,直接解析")
return True, None
return True, get_policy(event)
group_id = str(event.group_id)
if not white:
if not is_group_whitelisted(event):
logger.info(f"权限分析:群 {group_id} 不在白名单,不做处理")
return False, None
if auto_only and not auto:
if msg is not None and await _match_auto_link(event, msg):
logger.info(f"权限分析:群 {group_id} 未开启自动解析,但自动链接关键词命中")
else:
logger.info(f"权限分析:群 {group_id} 未开启自动解析")
return False, None
if not auto_only and black:
if is_user_blacklisted(event):
logger.info(f"权限分析:黑名单用户,不回复: {event.get_user_id()}")
return False, None
policy = get_policy(event)
if auto_only and not policy.auto and not _match_auto_link(msg or "", policy):
logger.info(f"权限分析:群 {group_id} 未开启自动解析且未命中自动策略")
return False, None
logger.info(
f"权限分析:群 {group_id} 权限通过 — "
f"自动解析: {auto}, 方案: {plan or '默认(PLANC)'}"
f"权限分析:群 {group_id} 权限通过 — 自动解析: {policy.auto}, "
f"存储: {policy.plan}, 公网: {policy.upload_public}, "
f"群文件: {policy.upload_group_file}"
)
return True, plan
return True, policy
async def _match_auto_link(event: Event, msg: str) -> bool:
"""消息中的 URL 是否命中群配置的 auto_link 关键词。"""
keywords = await get_group_auto_link(event)
if not keywords:
def _match_auto_link(msg: str, policy: Policy) -> bool:
"""消息中的 URL 是否命中群策略的自动策略(auto_link)。"""
if not policy.auto_link:
return False
urls = URL_PATTERN.findall(msg)
if not urls:
for url in URL_PATTERN.findall(msg):
platform = policy.auto_matched(url)
if platform:
logger.info(f"自动策略:{platform} 命中消息 {url}")
return True
return False
for kw in keywords:
domains = AUTO_LINK_KEYWORDS.get(kw, (kw,))
if any(any(domain in url for domain in domains) for url in urls):
logger.info(f"自动链接:关键词 {kw} 命中消息 {urls}")
def _skip_banned(url: str, policy: Policy) -> bool:
"""链接是否命中禁用策略(命中即静默丢弃,只记日志)。"""
platform = policy.banned(url)
if platform:
logger.info(f"禁用策略:{platform} 已禁用,忽略链接 {url}")
return True
return False
@@ -110,54 +117,60 @@ async def _match_auto_link(event: Event, msg: str) -> bool:
@auto_video_handler.handle()
async def handle_auto_video(event: Event):
msg = str(event.get_message()).strip()
allowed, plan = await _check_access(event, auto_only=True, msg=msg)
if allowed:
await match_message(event, plan=plan)
allowed, policy = await _check_access(event, auto_only=True, msg=msg)
if allowed and policy is not None:
await match_message(event, policy)
@active_video_handler.handle()
async def handle_active_video(event: Event):
allowed, plan = await _check_access(event, auto_only=False)
if allowed:
await match_message(event, plan=plan)
allowed, policy = await _check_access(event, auto_only=False)
if allowed and policy is not None:
await match_message(event, policy)
async def match_message(event: Event, plan: str | None = None):
"""消息匹配与分派:文本链接 / QQ小程序卡片统一走 dispatch_url。"""
async def match_message(event: Event, policy: Policy):
"""消息匹配与分派:文本链接 / QQ小程序卡片统一走 dispatch_url。
命中禁用策略的链接在这里丢弃;消息里还有其它可用链接则继续解析。
"""
msg = str(event.get_message()).strip()
logger.info(f"消息解析:获取到的消息:{msg}")
is_private = get_target(event).private
message = None
public_url = None
url = ""
if re.search(XCX_PATTERN, msg) or "CQ:json" in msg or "CQ:share" in msg:
logger.info("消息解析:检测到 CQ 卡片")
url = await _extract_xcx_url(msg)
url = await _extract_xcx_url(msg) or ""
logger.info(f"消息解析:卡片链接:{url}")
if not url or not any(domain in url for domain in VALID_HOSTS):
return
message, public_url = await dispatch_url(url, is_private, plan=plan)
if _skip_banned(url, policy):
return
message, public_url = await dispatch_url(url, is_private, policy)
if not message:
return
else:
urls = URL_PATTERN.findall(msg)
urls = [u for u in URL_PATTERN.findall(msg) if not _skip_banned(u, policy)]
for url in urls:
logger.info(f"消息解析:作品链接:{url}")
message, public_url = await dispatch_url(url, is_private, plan=plan)
message, public_url = await dispatch_url(url, is_private, policy)
if message:
break
if not message:
return
if isinstance(message, PendingMedia):
ok, pub = await send_pending_media(message)
if pub:
ok, pub = await send_pending_media(message, event)
if pub and policy.sends_link:
await UniMessage.text(f"{pub}").send()
if not ok:
await UniMessage.text(f"媒体发送失败:{url}").send()
else:
if public_url:
if public_url and policy.sends_link:
await UniMessage.text(f"{public_url}").send()
await message.send()
@@ -165,8 +178,8 @@ async def match_message(event: Event, plan: str | None = None):
async def dispatch_url(
url: str,
is_private: bool,
plan: str | None = None,
) -> tuple[Optional[UniMessage], Optional[str]]:
policy: Policy,
) -> tuple[UniMessage | None, str | None]:
"""按平台分派解析(文本链接与小程序卡片共用)。"""
url = url.rstrip(",。!?、;:)】》\"')")
@@ -176,12 +189,22 @@ async def dispatch_url(
logger.info(f"b23 短链重定向: {url} -> {resolved}")
url = resolved
# 平台标签:短链重定向之后再判定(群文件限定平台用)
platform = match_platform(url)
if "douyin.com" in url or "v.douyin.com" in url or "iesdouyin.com" in url:
await UniMessage.text("检测到链接,正在处理,请稍候...").send()
try:
title, parsed_path, image_post = await parse_douyin(url)
if not title and not parsed_path:
# 链接识别成功但未能获取到作品(网络/风控/链接失效等),
# 给出明确失败提示,避免只发"请稍候"后无下文。
await UniMessage.text(f"无法解析到媒体:{url}").send()
logger.warning(f"媒体解析:未能获取到作品:{url}")
return None, None
return await process_douyin_res(
title, parsed_path, is_private, image_post, plan=plan
title, parsed_path, is_private, image_post,
policy=policy, platform=platform,
)
except Exception:
await UniMessage.text(f"无法解析到媒体:{url}").send()
@@ -208,7 +231,7 @@ async def dispatch_url(
return None, None
return await process_douyin_res(
title, parsed_path, is_private,
isinstance(parsed_path, list), plan=plan,
isinstance(parsed_path, list), policy=policy, platform=platform,
)
except Exception:
await UniMessage.text(f"无法解析到内容:{url}").send()
@@ -227,7 +250,7 @@ async def dispatch_url(
return None, None
return await process_douyin_res(
title, parsed_path, is_private,
isinstance(parsed_path, list), plan=plan,
isinstance(parsed_path, list), policy=policy, platform=platform,
)
except Exception:
await UniMessage.text(f"无法解析到内容:{url}").send()
@@ -237,7 +260,7 @@ async def dispatch_url(
if any(domain in url for domain in VALID_HOSTS):
await UniMessage.text("检测到链接,正在处理,请稍候...").send()
try:
return await handle_universal(url, is_private, plan=plan)
return await handle_universal(url, is_private, policy, platform)
except Exception as e:
logger.exception(e)
await UniMessage.text("下载过程中出现错误。").send()
@@ -246,7 +269,7 @@ async def dispatch_url(
return None, None
async def _extract_xcx_url(msg: str) -> Optional[str]:
async def _extract_xcx_url(msg: str) -> str | None:
"""从 CQ 卡片消息中提取跳转 URL(保留 query 参数)。"""
match = re.search(r'"qqdocurl":"(.*?)"', msg)
if not match:
@@ -5,17 +5,37 @@
2. 失败 → 上传本地 S3,用预签名链接发送
3. 再失败 → 回退 temp 本地文件再发一次
本地直发要真的通,得让协议端能拿到文件:2026-09-22 起由 `hexi/core/outbound_media`
的全局钩子把本地路径改写成 `/media/<token>` 链接(此前本地那级因协议端不在本机、
`file://` 必然 ENOENT 而恒失败,实际每次都落到第 2 级 S3)。因此**媒体不再进 S3
归档是有意的**——只有群策略真要发下载链接(`policy.sends_link`)时才补传一次拿
public_url,见 `_local_sent`。
多媒体(2026-09-08 用户需求): 多图(图集)、图+视频混合、多视频(动图)
等一个作品落地成多个文件时,直接 UniMessage 拼成一个消息发送会报错
(视频段不能与其他段混合,一条消息也放不下多段视频/图集体验差)
→ 统一走合并转发,一个图/视频一个节点。
群文件(并行通道): `policy.upload_group_file` 开着时,消息链跑完后额外把文件
传到群文件(默认打包成一个 zip,可配解压密码,见 config.group_file_settings),
失败只记日志、不影响发送结果(打包/上传实现在 services/storage/group_file.py)。
temp 下的文件发送成功后也不清理(用户手动处理 data/temp)。
"""
import asyncio
from dataclasses import dataclass
from pathlib import Path
from typing import Optional, Union
from nonebot import logger
from nonebot import get_bot, get_driver, logger
from nonebot.adapters import Event
from nonebot_plugin_alconna import UniMessage
from ..config import group_file_settings
from ..policy import Policy
from ..services.storage.group_file import upload_group_files
from ..services.storage.s3 import upload_with_plan
from ..utils import media_rel_dir_of
@dataclass
@@ -25,21 +45,116 @@ class PendingMedia:
files: list[Path]
image_post: bool = False
is_private: bool = False
plan: Optional[str] = None
policy: Policy | None = None
#: 平台规范标签(见 policy.match_platform),群文件限定平台用
platform: str | None = None
title: str = ""
def _as_paths(file_paths: Union[Path, list[Path]]) -> list[Path]:
def _as_paths(file_paths: Path | list[Path]) -> list[Path]:
if isinstance(file_paths, list):
return [Path(p) for p in file_paths]
return [Path(file_paths)]
# ─────────────────────── 合并转发(多媒体专用) ───────────────────────
#: 按视频段发送的扩展名(其余按图片发;直链下载可能落 webm/mov 等)
_VIDEO_SUFFIXES = {".mp4", ".webm", ".mov", ".flv", ".mkv", ".ts"}
def _is_video(fp: Path) -> bool:
return fp.suffix.lower() in _VIDEO_SUFFIXES
def _needs_forward(files: list[Path]) -> bool:
"""是否需要走合并转发:多图/图+视频混合/多视频(动图)等多媒体作品
落地成多个文件(>1 个)统一一个媒体一个节点转发;
单图/单视频保持原有直发链路。
"""
return len(files) > 1
def _forward_sender_name() -> str:
"""合并转发节点的发送者昵称(用机器人配置昵称)"""
nickname = get_driver().config.nickname
return next(iter(nickname), "Bot") if nickname else "Bot"
def _media_segment(fp: Path, url: str | None = None) -> dict:
"""onebot v11 消息段字典:image / video,file 为本地路径或链接"""
media_type = "video" if _is_video(fp) else "image"
return {"type": media_type, "data": {"file": url or str(fp)}}
def _build_forward_nodes(
items: list[tuple[Path, str | None]],
sender_name: str,
self_id: int,
) -> list[dict]:
"""合并转发节点列表:一个媒体一个节点,content 为该媒体的消息段数组"""
return [
{
"type": "node",
"data": {
"name": sender_name,
"uin": str(self_id),
"content": [_media_segment(fp, url)],
},
}
for fp, url in items
]
async def _send_forward_media(
items: list[tuple[Path, str | None]],
sender_name: str,
*,
group_id: int | None = None,
user_id: int | None = None,
) -> bool:
"""发送合并转发消息(群聊 / 私聊),失败返回 False"""
try:
bot = get_bot()
nodes = _build_forward_nodes(items, sender_name, bot.self_id)
if group_id is not None:
await bot.call_api(
"send_group_forward_msg", group_id=group_id, messages=nodes
)
else:
await bot.call_api("send_forward_msg", user_id=user_id, messages=nodes)
return True
except Exception as e:
logger.warning(f"合并转发发送失败: {e}")
return False
async def _build_s3_forward_items(
media: PendingMedia,
) -> tuple[list[tuple[Path, str | None]], str | None]:
"""上传本地 S3,返回 ([(文件, 预签名链接)...], public_url)"""
items: list[tuple[Path, str | None]] = []
public_url = None
for fp in media.files:
local_url, pub = upload_with_plan(fp, policy=media.policy)
if not local_url:
raise RuntimeError(f"上传本地 S3 失败: {fp}")
if pub:
public_url = pub
items.append((fp, local_url))
return items, public_url
# ───────────────────────────── 多级发送 ─────────────────────────────
def _build_local_msg(files: list[Path], image_post: bool) -> UniMessage:
"""本地文件版消息(mp4 → 视频,其余 → 图片)"""
"""本地文件版消息(视频扩展名 → 视频段,其余 → 图片段)"""
msg = UniMessage()
for fp in files:
if fp.suffix.lower() == ".mp4":
if _is_video(fp):
msg.video(path=fp)
else:
msg.image(path=fp)
@@ -48,43 +163,98 @@ def _build_local_msg(files: list[Path], image_post: bool) -> UniMessage:
def _build_s3_msg(
media: PendingMedia,
) -> tuple[UniMessage, Optional[str]]:
) -> tuple[UniMessage, str | None]:
"""上传本地 S3 并构建链接版消息,返回 (message, public_url)"""
msg = UniMessage()
public_url = None
for fp in media.files:
local_url, pub = upload_with_plan(
fp,
plan=media.plan,
is_private=media.is_private,
title=media.title,
image_post=media.image_post,
)
local_url, pub = upload_with_plan(fp, policy=media.policy)
if not local_url:
raise RuntimeError(f"上传本地 S3 失败: {fp}")
if pub:
public_url = pub
if fp.suffix.lower() == ".mp4":
if _is_video(fp):
msg.video(url=local_url)
else:
msg.image(url=local_url)
return msg, public_url
async def send_pending_media(media: PendingMedia) -> tuple[bool, Optional[str]]:
"""多级发送,返回 (是否成功, public_url)
async def _public_link_for(media: PendingMedia) -> str | None:
"""发送已由本地通道完成后,为「发送下载链接」策略补一次 S3 上传。
public_url 仅在走 S3 链接发送成功时返回(调用方决定是否发文字)。
本地直发不经 S3(最快),但群策略开了链接就需要公网 URL——这里**只取链接**,
不重建消息。失败只记日志并返回 None:发送本身已经成功了,不能反过来判失败。
"""
public_url = None
for fp in media.files:
try:
_, pub = await asyncio.to_thread(upload_with_plan, fp, policy=media.policy)
except Exception as e: # noqa: BLE001
logger.warning(f"补传 S3 取下载链接失败 {fp}: {e}")
return None
if pub:
public_url = pub
return public_url
async def _local_sent(media: PendingMedia) -> tuple[bool, str | None]:
"""本地通道发送成功后的返回。
只有策略真的要发下载链接(`sends_link` = send_link 且已开公网)时才补 S3;
其余情况一概不碰 S3 —— 媒体不走 S3 归档是有意为之(temp 保留由 cleanup 管)。
"""
if media.policy is not None and media.policy.sends_link:
return True, await _public_link_for(media)
return True, None
async def _send_media_core(
media: PendingMedia, event: Event | None = None
) -> tuple[bool, str | None]:
"""多级发送主体,返回 (是否成功, public_url)
public_url 有两种来源:走 S3 链接发送成功,或本地发送成功但策略要链接时
补传 S3(见 `_local_sent`);其余情况为 None(调用方据 `sends_link` 决定是否发文字)。
temp 文件发送成功后保留(用户手动清理 data/temp)。
多媒体作品(多图 / 图+视频 / 多视频)先走合并转发(本地文件 → S3 链接),
失败后仍落回下方普通消息链路兜底。
"""
if not media.files:
return False, None
# ── 0. 多媒体 → 合并转发(一个图/视频一个节点)─────────────────
if event is not None and _needs_forward(media.files):
group_id = getattr(event, "group_id", None)
user_id = None if group_id is not None else getattr(event, "user_id", None)
if group_id is not None or user_id is not None:
sender_name = _forward_sender_name()
if await _send_forward_media(
[(fp, None) for fp in media.files],
sender_name,
group_id=group_id,
user_id=user_id,
):
logger.info("媒体发送成功(合并转发,本地文件)")
return await _local_sent(media)
try:
s3_items, public_url = await _build_s3_forward_items(media)
if await _send_forward_media(
s3_items, sender_name, group_id=group_id, user_id=user_id
):
logger.info("媒体发送成功(合并转发,S3 链接)")
return True, public_url
except Exception as e:
logger.warning(f"S3 链接合并转发失败,走普通消息链路: {e}")
else:
logger.warning("无法确定群号/用户号,跳过合并转发,走普通消息链路")
# ── 1. temp 本地文件直接发送 ──────────────────────────────
try:
await _build_local_msg(media.files, media.image_post).send()
logger.info("媒体发送成功(temp 本地文件直达)")
return True, None
return await _local_sent(media)
except Exception as e:
logger.warning(f"temp 本地文件发送失败,切换本地 S3 链接: {e}")
@@ -101,7 +271,51 @@ async def send_pending_media(media: PendingMedia) -> tuple[bool, Optional[str]]:
try:
await _build_local_msg(media.files, media.image_post).send()
logger.info("媒体发送成功(回退 temp 本地文件)")
return True, None
return await _local_sent(media)
except Exception as e:
logger.exception(f"回退发送失败: {e}")
return False, None
async def _upload_group_files(media: PendingMedia, event: Event | None) -> None:
"""群文件并行通道:消息链跑完后按策略额外传一份(失败只记日志)。
是否打包成 zip / 是否加密由全局配置决定(config.group_file_settings);
平台清单(policy.group_file_platforms)非空时只传清单里的平台,其余平台
照常走消息、不传群文件。
"""
policy = media.policy
if policy is None or event is None:
return
if not policy.allows_group_file(media.platform):
if policy.upload_group_file:
logger.info(
f"群文件限定平台 {policy.group_file_platforms},"
f"本次为 {media.platform or '未知平台'},跳过群文件上传"
)
return
group_id = getattr(event, "group_id", None)
if group_id is None:
return
zip_files, password = group_file_settings()
await upload_group_files(
media.files,
int(group_id),
title=media.title,
zip_files=zip_files,
password=password,
policy=policy,
rel_dir=media_rel_dir_of(media.files[0]),
)
async def send_pending_media(
media: PendingMedia, event: Event | None = None
) -> tuple[bool, str | None]:
"""多级发送 + 群文件并行通道,返回 (是否成功, public_url)。"""
if not media.files:
return False, None
ok, public_url = await _send_media_core(media, event)
await _upload_group_files(media, event)
return ok, public_url
@@ -1,11 +1,11 @@
"""通用平台视频解析编排层 — B站 / YouTube / Twitter 等"""
from pathlib import Path
from typing import Optional
from nonebot import logger
from nonebot_plugin_alconna import UniMessage
from ..policy import Policy
from ..services.fetchers.video_downloader import download_video
from .sender import PendingMedia, _as_paths
@@ -13,15 +13,16 @@ from .sender import PendingMedia, _as_paths
async def handle_universal(
url: str,
is_private: bool,
plan: str | None = None,
policy: Policy | None = None,
platform: str | None = None,
) -> tuple[Optional[PendingMedia], Optional[str]]:
"""
下载通用平台视频 → 打包待发送媒体(上传/发送由 sender 多级处理)
下载通用平台视频 → 打包待发送媒体(上传/发送由 sender 按 policy 多级处理)
Returns:
(PendingMedia, public_url) — None 表示下载失败
"""
video_file = await download_video(url)
video_file, title = await download_video(url)
if not video_file:
await UniMessage.text("视频下载失败。").send()
return None, None
@@ -33,8 +34,9 @@ async def handle_universal(
files=_as_paths(video_file),
image_post=False,
is_private=is_private,
plan=plan,
title="title",
policy=policy,
platform=platform,
title=title,
),
None,
)
@@ -1,364 +1,434 @@
import json
import random
import asyncio
import os
from typing import List
"""群策略管理 —— 命令入口与访问层。
策略模型、平台定义与存储(list.json v3)见 `policy.py`。
本模块只做两件事:给 handlers 提供同步访问器(走内存),以及把命令解析成
`STORE.update_group(...)` 调用。
命令统一入口 `视频策略`:
视频策略 查看当前群策略面板
视频策略 <群号> ... 管理员操作指定群
视频策略 自动 on|off
视频策略 自动策略 +小红书 -抖音 命中即解析(即使关了自动解析)
视频策略 禁用策略 +X 命中即不解析(自动/手动都不解析)
视频策略 存储 A|B|C
视频策略 公网 on|off
视频策略 链接 on|off 发送下载链接(需先开公网)
视频策略 群文件 on|off
视频策略 黑名单 +QQ -QQ 全局用户黑名单
视频策略 白名单 +群号 -群号
所有设置项都要求超管(`check_admin`);设置时群会自动加入白名单。
"""
from __future__ import annotations
from typing import Any
from nonebot.adapters.onebot.v11 import MessageSegment, PokeNotifyEvent, Message
from nonebot.plugin.on import on_notice, on_command
from nonebot import on_type, on_message, logger
from nonebot.adapters import Event
from nonebot.matcher import Matcher
from nonebot.params import CommandArg
from nonebot.rule import to_me
from nonebot_plugin_alconna import UniMessage, get_target
from nonebot.plugin.on import on_command
from nonebot_plugin_alconna import UniMessage
from nonebot_plugin_alconna.uniseg import get_target
from hexi.plugins.nonebot_plugin_hexi_core.custom_utils import check_admin
# from hexi.plugins.nonebot_plugin_hexi_core.MessageUtils import send_poke
from hexi.core.custom_utils import check_admin
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
FILE_PATH = os.path.join(BASE_DIR, "data", "list.json")
from .policy import (
PLANS,
PLATFORMS,
STORE,
Policy,
apply_platform_diff,
normalize_platforms,
)
USER_DATA: dict[str, str] = {}
_LOADED = False
LOCK = asyncio.Lock()
# ─────────────────────────── 访问层 ───────────────────────────
add_black_list = on_command("添加黑名单",rule=check_admin)
def warmup() -> None:
"""导入期调用一次:把 list.json 读进内存(含 v1/v2 → v3 迁移)。"""
STORE.load()
def get_policy(event: Event) -> Policy:
"""事件对应的策略:私聊读 default 节,群聊读该群策略。"""
if get_target(event).private:
return STORE.default_policy()
return STORE.get(str(event.group_id))
def is_group_whitelisted(event: Event) -> bool:
"""私聊不适用(返回 False,调用方应先判断私聊)。"""
return STORE.is_whitelisted(str(getattr(event, "group_id", "")))
def is_user_blacklisted(event: Event) -> bool:
return STORE.is_blacklisted(event.get_user_id())
# ─────────────────────────── 命令解析 ───────────────────────────
_TRUE_WORDS = {"on", "开", "true", "1", "yes", "启用"}
_FALSE_WORDS = {"off", "关", "false", "0", "no", "禁用"}
TOGGLE_USAGE = "用法:视频策略 自动|公网|链接|群文件 on|off"
def _args(event: Event, matcher: Matcher) -> list[str]:
"""去掉命令前缀后的参数列表。"""
text = event.get_message().extract_plain_text().strip()
cmd = matcher.state["_prefix"]["command"][0]
return text.replace(cmd, "", 1).strip().split()
def _parse_onoff(raw: str) -> bool | None:
word = raw.strip().lower()
if word in _TRUE_WORDS:
return True
if word in _FALSE_WORDS:
return False
return None
def _split_target(args: list[str], event: Event) -> tuple[str | None, list[str]]:
"""首个参数是纯数字群号时视为操作目标群,返回 (群号, 剩余参数)。"""
if args and args[0].isdigit() and len(args[0]) >= 5:
return args[0], args[1:]
return None, args
def _fmt_platforms(values: list[str]) -> str:
return "、".join(values) if values else "—"
def _panel(group_id: str | None, policy: Policy, whitelisted: bool) -> str:
head = f"群 {group_id} 策略" if group_id else "私聊/默认策略"
if group_id and not whitelisted:
head += "(未加入白名单,本群不会自动解析)"
link_state = "开" if policy.sends_link else "关"
if policy.send_link and not policy.upload_public:
link_state += "(未上传公网,实际不发)"
file_state = "开" if policy.upload_group_file else "关"
if policy.upload_group_file and policy.group_file_platforms:
file_state += f"(仅 {'、'.join(policy.group_file_platforms)})"
return "\n".join(
[
head,
f"自动解析:{'开' if policy.auto else '关'}",
f"自动策略:{_fmt_platforms(policy.auto_link)}",
f"禁用策略:{_fmt_platforms(policy.ban_link)}",
f"存储策略:{policy.plan}",
f"上传公网:{'开' if policy.upload_public else '关'}",
f"发送链接:{link_state}",
f"上传群文件:{file_state}",
f"平台标签:{'、'.join(PLATFORMS)}",
]
)
# ─────────────────────────── 命令注册 ───────────────────────────
policy_cmd = on_command("视频策略", rule=check_admin)
add_white_list = on_command("添加白名单", rule=check_admin)
remove_white_list = on_command("移除白名单", rule=check_admin)
add_black_list = on_command("添加黑名单", rule=check_admin)
remove_black_list = on_command("移除黑名单", rule=check_admin)
# 旧命令保留兼容
add_auto_list = on_command("添加自动名单", rule=check_admin)
set_plan_cmd = on_command("设置方案", rule=check_admin)
set_auto_link_cmd = on_command("设置自动链接", rule=check_admin)
# 自动链接关键词 → 域名匹配表(auto_link 配置项使用)
AUTO_LINK_KEYWORDS = {
"xhs": ("xiaohongshu.com", "xhslink.com", "xhslink.cn"),
"bilibili": ("bilibili.com", "b23.tv", "bili2233.cn"),
"b23": ("bilibili.com", "b23.tv", "bili2233.cn"),
"douyin": ("douyin.com", "v.douyin.com", "iesdouyin.com",
"m.douyin.com", "jingxuan.douyin.com"),
"yt": ("youtube.com", "youtu.be"),
"youtube": ("youtube.com", "youtu.be"),
"x": ("x.com", "twitter.com"),
"twitter": ("x.com", "twitter.com"),
}
@add_black_list.handle()
async def handle_add_black(event: Event, matcher: Matcher):
input_id = event.get_message().extract_plain_text().strip()
async def _target_group(
args: list[str], event: Event, matcher: Matcher
) -> tuple[str | None, list[str]]:
"""解析目标群号与剩余参数;未指定时用当前群(私聊则报错返回 None)。"""
group_id, rest = _split_target(args, event)
if group_id is not None:
return group_id, rest
if get_target(event).private:
cmd = matcher.state["_prefix"]["command"][0]
input_id = input_id.replace(cmd, "").strip()
ok = await _add_to_blacklist(input_id)
if ok:
await UniMessage.text(f"{input_id} 已加入黑名单").send()
await UniMessage.text(f"私聊下请带上群号,如:{cmd} 123456789 自动 on").send()
return None, rest
return str(event.group_id), rest
@policy_cmd.handle()
async def handle_policy(event: Event, matcher: Matcher):
args = _args(event, matcher)
group_id, rest = await _target_group(args, event, matcher)
if group_id is None:
return
# 无子命令 → 面板
if not rest:
policy = STORE.get(group_id)
await UniMessage.text(
_panel(group_id, policy, STORE.is_whitelisted(group_id))
).send()
return
what, options = rest[0], rest[1:]
changes: dict[str, Any] = {}
note = ""
if what in ("自动", "自动解析"):
if len(options) != 1 or (value := _parse_onoff(options[0])) is None:
await UniMessage.text(TOGGLE_USAGE).send()
return
changes["auto"] = value
note = f"自动解析已{'开启' if value else '关闭'}"
elif what in ("自动策略", "自动链接"):
platforms = apply_platform_diff(STORE.get(group_id).auto_link, options)
if platforms is None:
await UniMessage.text(
f"用法:视频策略 自动策略 +平台/-平台(平台:{'、'.join(PLATFORMS)})"
).send()
return
changes["auto_link"] = platforms
note = f"自动策略已设为:{_fmt_platforms(platforms)}"
elif what in ("禁用策略", "禁用链接"):
platforms = apply_platform_diff(STORE.get(group_id).ban_link, options)
if platforms is None:
await UniMessage.text(
f"用法:视频策略 禁用策略 +平台/-平台(平台:{'、'.join(PLATFORMS)})"
).send()
return
changes["ban_link"] = platforms
note = f"禁用策略已设为:{_fmt_platforms(platforms)}"
elif what in ("存储", "方案", "存储策略"):
plan = options[0].strip().upper() if options else ""
if plan not in PLANS:
await UniMessage.text("用法:视频策略 存储 A|B|C").send()
return
changes["plan"] = plan
note = f"存储策略已设为 {plan}"
elif what in ("公网", "上传公网"):
if len(options) != 1 or (value := _parse_onoff(options[0])) is None:
await UniMessage.text(f"用法:视频策略 {what} on|off").send()
return
changes["upload_public"] = value
note = f"上传公网已{'开启' if value else '关闭'}"
if not value and STORE.get(group_id).send_link:
note += "(发送链接已开但无公网链接,实际不会发)"
elif what in ("链接", "发送链接", "下载链接"):
if len(options) != 1 or (value := _parse_onoff(options[0])) is None:
await UniMessage.text(f"用法:视频策略 {what} on|off").send()
return
changes["send_link"] = value
note = f"发送下载链接已{'开启' if value else '关闭'}"
if value and not STORE.get(group_id).upload_public:
note += ";当前未开启上传公网,需先:视频策略 公网 on"
elif what in ("群文件平台", "群文件限定平台"):
platforms = apply_platform_diff(
STORE.get(group_id).group_file_platforms, options
)
if platforms is None:
await UniMessage.text(
"用法:视频策略 群文件平台 +平台 -平台"
f"(平台:{'、'.join(PLATFORMS)};留空 = 全部平台)"
).send()
return
changes["group_file_platforms"] = platforms
note = (
f"群文件限定平台已设为:{_fmt_platforms(platforms)}"
if platforms
else "群文件限定平台已清空(所有平台都传群文件)"
)
elif what in ("群文件", "上传群文件"):
if len(options) != 1 or (value := _parse_onoff(options[0])) is None:
await UniMessage.text(f"用法:视频策略 {what} on|off").send()
return
changes["upload_group_file"] = value
note = f"上传群文件已{'开启' if value else '关闭'}"
elif what == "黑名单":
note = await _handle_blacklist(options)
if note is None:
return
elif what == "白名单":
note = await _handle_whitelist(options)
if note is None:
return
else:
await UniMessage.text(f"{input_id} 已在黑名单中").send()
await UniMessage.text(
"用法:视频策略 [群号] 自动|自动策略|禁用策略|存储|公网|链接|"
"群文件|群文件平台|黑名单|白名单 ..."
).send()
return
if changes:
policy = await STORE.update_group(group_id, create=True, **changes)
if policy is None:
await UniMessage.text(f"群 {group_id} 不在白名单中,请先添加白名单").send()
return
note += "\n\n" + _panel(group_id, policy, STORE.is_whitelisted(group_id))
await UniMessage.text(note).send()
async def _handle_blacklist(options: list[str]) -> str | None:
"""全局黑名单增删,返回提示语;参数非法返回 None(已回复用法)。"""
if not options:
current = "、".join(STORE.blacklist()) or "—"
return f"全局黑名单:{current}\n用法:视频策略 黑名单 +QQ -QQ"
for token in options:
qq = token[1:].strip()
if token[:1] not in "+-" or not qq.isdigit():
return "用法:视频策略 黑名单 +QQ -QQ"
added, removed = [], []
for token in options:
qq = token[1:].strip()
if token[0] == "+" and await STORE.add_blacklist(qq):
added.append(qq)
elif token[0] == "-" and await STORE.remove_blacklist(qq):
removed.append(qq)
parts = []
if added:
parts.append(f"已加入黑名单:{'、'.join(added)}")
if removed:
parts.append(f"已移出黑名单:{'、'.join(removed)}")
return "\n".join(parts) if parts else "黑名单无变化"
async def _handle_whitelist(options: list[str]) -> str | None:
"""白名单增删,返回提示语;参数非法返回 None(已回复用法)。"""
if not options:
current = "、".join(sorted(STORE.all_groups())) or "—"
return f"白名单:{current}\n用法:视频策略 白名单 +群号 -群号"
for token in options:
gid = token[1:].strip()
if token[:1] not in "+-" or not gid.isdigit():
return "用法:视频策略 白名单 +群号 -群号"
added, removed = [], []
for token in options:
gid = token[1:].strip()
if token[0] == "+" and await STORE.update_group(gid, create=True) is not None:
added.append(gid)
elif token[0] == "-" and await STORE.remove_group(gid):
removed.append(gid)
parts = []
if added:
parts.append(f"已加入白名单:{'、'.join(added)}")
if removed:
parts.append(f"已移除白名单:{'、'.join(removed)}")
return "\n".join(parts) if parts else "白名单无变化"
# ───────────────────── 旧命令(兼容保留) ─────────────────────
@add_white_list.handle()
async def handle_add_white(event: Event, matcher: Matcher):
input_id = event.get_message().extract_plain_text().strip()
cmd = matcher.state["_prefix"]["command"][0]
input_id = input_id.replace(cmd, "").strip()
ok = await add_white_user(input_id)
if ok:
await UniMessage.text(f"群 {input_id} 已加入白名单").send()
else:
await UniMessage.text(f"群 {input_id} 已在白名单中").send()
args = _args(event, matcher)
group_id = args[0].strip() if args else ""
if not group_id.isdigit():
await UniMessage.text("用法:添加白名单 <群号>").send()
return
if STORE.is_whitelisted(group_id):
await UniMessage.text(f"群 {group_id} 已在白名单中").send()
return
await STORE.update_group(group_id, create=True)
await UniMessage.text(f"群 {group_id} 已加入白名单").send()
@remove_white_list.handle()
async def handle_remove_white(event: Event, matcher: Matcher):
args = _args(event, matcher)
group_id = args[0].strip() if args else ""
if not group_id.isdigit():
await UniMessage.text("用法:移除白名单 <群号>").send()
return
ok = await STORE.remove_group(group_id)
await UniMessage.text(
f"群 {group_id} 已移出白名单" if ok else f"群 {group_id} 不在白名单中"
).send()
@add_black_list.handle()
async def handle_add_black(event: Event, matcher: Matcher):
args = _args(event, matcher)
user_id = args[0].strip() if args else ""
if not user_id.isdigit():
await UniMessage.text("用法:添加黑名单 <QQ>").send()
return
ok = await STORE.add_blacklist(user_id)
await UniMessage.text(
f"{user_id} 已加入黑名单" if ok else f"{user_id} 已在黑名单中"
).send()
@remove_black_list.handle()
async def handle_remove_black(event: Event, matcher: Matcher):
args = _args(event, matcher)
user_id = args[0].strip() if args else ""
if not user_id.isdigit():
await UniMessage.text("用法:移除黑名单 <QQ>").send()
return
ok = await STORE.remove_blacklist(user_id)
await UniMessage.text(
f"{user_id} 已移出黑名单" if ok else f"{user_id} 不在黑名单中"
).send()
@add_auto_list.handle()
async def handle_add_auto(event: Event, matcher: Matcher):
input_id = event.get_message().extract_plain_text().strip()
cmd = matcher.state["_prefix"]["command"][0]
input_id = input_id.replace(cmd, "").strip()
ok = await add_auto_user(input_id)
if ok:
await UniMessage.text(f"群 {input_id} 已开启自动解析").send()
else:
await UniMessage.text(f"群 {input_id} 已开启自动解析,无需重复设置").send()
args = _args(event, matcher)
group_id = args[0].strip() if args else ""
if not group_id.isdigit():
await UniMessage.text("用法:添加自动名单 <群号>").send()
return
policy = STORE.get(group_id)
if policy.auto and STORE.is_whitelisted(group_id):
await UniMessage.text(f"群 {group_id} 已开启自动解析,无需重复设置").send()
return
await STORE.update_group(group_id, create=True, auto=True)
await UniMessage.text(f"群 {group_id} 已开启自动解析").send()
@set_plan_cmd.handle()
async def handle_set_plan(event: Event, matcher: Matcher):
input_text = event.get_message().extract_plain_text().strip()
cmd = matcher.state["_prefix"]["command"][0]
args = input_text.replace(cmd, "").strip().split()
if len(args) != 2:
await UniMessage.text("用法:设置方案 <群号> A/B").send()
args = _args(event, matcher)
if len(args) != 2 or args[1].upper() not in PLANS:
await UniMessage.text("用法:设置方案 <群号> A/B/C").send()
return
group_id, plan = args[0], args[1].upper()
ok = await set_group_plan(group_id, plan)
if ok:
await UniMessage.text(f"群 {group_id} 存储方案已设为 {plan}").send()
elif plan not in ("A", "B"):
await UniMessage.text("方案必须是 A 或 B").send()
else:
policy = await STORE.update_group(group_id, plan=plan)
if policy is None:
await UniMessage.text(f"群 {group_id} 不在白名单中,请先添加白名单").send()
return
await UniMessage.text(f"群 {group_id} 存储方案已设为 {plan}").send()
@set_auto_link_cmd.handle()
async def handle_set_auto_link(event: Event, matcher: Matcher):
input_text = event.get_message().extract_plain_text().strip()
cmd = matcher.state["_prefix"]["command"][0]
args = input_text.replace(cmd, "").strip().split()
if len(args) < 2:
args = _args(event, matcher)
if not args:
await UniMessage.text(
f"用法:设置自动链接 <群号> <关键词...>(关键词:{'/'.join(AUTO_LINK_KEYWORDS)})"
f"用法:设置自动链接 <群号> <平台...>(平台:{'、'.join(PLATFORMS)})"
).send()
return
group_id = args[0]
keywords = args[1:]
ok = await set_group_auto_link(group_id, keywords)
if ok:
if keywords:
platforms = normalize_platforms(args[1:]) if len(args) > 1 else []
if len(args) > 1 and not platforms:
await UniMessage.text(f"平台标签无效,可选:{'、'.join(PLATFORMS)}").send()
return
if await STORE.update_group(group_id, auto_link=platforms) is None:
await UniMessage.text(f"群 {group_id} 不在白名单中,请先添加白名单").send()
return
if platforms:
await UniMessage.text(
f"群 {group_id} 自动链接关键词已设为: {'、'.join(keywords)}\n"
f"群 {group_id} 自动策略已设为:{_fmt_platforms(platforms)}\n"
"匹配到对应平台链接时,即使未开启自动解析也会自动下载"
).send()
else:
await UniMessage.text(f"群 {group_id} 的自动链接关键词已清空").send()
else:
await UniMessage.text(f"群 {group_id} 不在白名单中,请先添加白名单").send()
async def _safe_write_json(data, file_path: str):
tmp_path = file_path + ".tmp"
# 在线程池中执行耗时的文件写入
await asyncio.to_thread(_write_json_sync, data, tmp_path)
# os.replace 是轻量级系统调用,通常很快,可直接在主线程执行
# (也可放 to_thread,但一般没必要)
os.replace(tmp_path, file_path)
def _write_json_sync(data, tmp_path: str):
"""同步写入函数,供 to_thread 调用"""
with open(tmp_path, "w", encoding="utf-8") as f:
json.dump(data, f, ensure_ascii=False, indent=2)
def _migrate_to_v2(data: dict) -> tuple[dict, bool]:
"""将旧格式(平铺数组)转换为新格式(group-centric map)"""
if "groups" in data:
return data, False # 已是 v2
white = data.get("WHITE_LIST", [])
auto_list = data.get("AUTO_ANALYSIS", [])
pa = data.get("PLANA", [])
pb = data.get("PLANB", [])
groups: dict[str, dict] = {}
for gid in white:
entry: dict = {}
if gid in auto_list:
entry["auto"] = True
if gid in pa:
entry["plan"] = "A"
elif gid in pb:
entry["plan"] = "B"
groups[gid] = entry
return {
"groups": groups,
"blacklist": data.get("BLACK_LIST", []),
}, True
async def load_list(file_path: str = FILE_PATH):
global USER_DATA, _LOADED
if _LOADED:
return
async with LOCK:
if _LOADED:
return
exists = await asyncio.to_thread(os.path.exists, file_path)
if not exists:
USER_DATA = {"groups": {}, "blacklist": []}
await _safe_write_json(USER_DATA, file_path)
else:
content = await asyncio.to_thread(_read_file_sync, file_path)
raw = json.loads(content)
USER_DATA, migrated = _migrate_to_v2(raw)
if migrated:
await _safe_write_json(USER_DATA, file_path)
logger.info("list.json 已从旧格式迁移为新 group-centric 格式")
_LOADED = True
def _read_file_sync(file_path: str) -> str:
with open(file_path, "r", encoding="utf-8") as f:
return f.read()
async def verify_user(event: Event) -> tuple[bool, bool, bool, str | None]:
"""
验证用户/群权限
Returns:
is_white — 群在白名单
is_black — 用户在黑名单
is_auto — 群开启自动解析
plan — 存储方案 "A" / "B" / None
"""
await load_list()
user_id = event.get_user_id()
groups = USER_DATA.get("groups", {})
blacklist = USER_DATA.get("blacklist", [])
is_black = user_id in blacklist
plan = None
is_auto = False
if get_target(event).private:
logger.debug(f"[DEBUG] private chat user_id: {repr(user_id)}")
return True, is_black, False, None
group_id = str(event.group_id)
group_config = groups.get(group_id, {})
is_white = group_id in groups
is_auto = group_config.get("auto", False)
plan = group_config.get("plan")
logger.debug(f"[DEBUG] user_id: {repr(user_id)} group_id: {group_id}")
logger.debug(f"[DEBUG] is_white: {is_white} is_black: {is_black} "
f"is_auto: {is_auto} plan: {plan}")
return is_white, is_black, is_auto, plan
async def _add_to_blacklist(new_id: str) -> bool:
await load_list()
async with LOCK:
lst: list = USER_DATA.get("blacklist", [])
if new_id in lst:
return False
lst.append(new_id)
USER_DATA["blacklist"] = lst
await _safe_write_json(USER_DATA, FILE_PATH)
return True
async def add_white_user(new_id: str) -> bool:
"""添加群到白名单(groups map)"""
await load_list()
async with LOCK:
groups: dict = USER_DATA.get("groups", {})
if new_id in groups:
return False
groups[new_id] = {}
USER_DATA["groups"] = groups
await _safe_write_json(USER_DATA, FILE_PATH)
return True
async def add_auto_user(new_id: str) -> bool:
"""设置群自动解析(群不在白名单则自动加入)"""
await load_list()
async with LOCK:
groups: dict = USER_DATA.get("groups", {})
if new_id not in groups:
groups[new_id] = {}
if groups[new_id].get("auto"):
return False
groups[new_id]["auto"] = True
USER_DATA["groups"] = groups
await _safe_write_json(USER_DATA, FILE_PATH)
return True
async def set_group_plan(group_id: str, plan: str) -> bool:
"""设置群的存储方案(A 或 B),群必须在白名单中"""
if plan not in ("A", "B"):
return False
await load_list()
async with LOCK:
groups: dict = USER_DATA.get("groups", {})
if group_id not in groups:
return False
groups[group_id]["plan"] = plan
USER_DATA["groups"] = groups
await _safe_write_json(USER_DATA, FILE_PATH)
return True
async def get_group_auto_link(event: Event) -> list[str]:
"""获取群配置的自动链接关键词(auto_link),私聊返回空列表"""
await load_list()
if get_target(event).private:
return []
group_id = str(event.group_id)
return USER_DATA.get("groups", {}).get(group_id, {}).get("auto_link", [])
async def set_group_auto_link(group_id: str, keywords: list[str]) -> bool:
"""设置群的自动链接关键词(auto_link),群必须在白名单中
keywords 为空列表时清空该配置。
"""
await load_list()
async with LOCK:
groups: dict = USER_DATA.get("groups", {})
if group_id not in groups:
return False
if keywords:
groups[group_id]["auto_link"] = keywords
else:
groups[group_id].pop("auto_link", None)
USER_DATA["groups"] = groups
await _safe_write_json(USER_DATA, FILE_PATH)
return True
# ── Web「分组配置」同步读写(list.json) ─────────────────────────────
def get_group_config_sync() -> dict:
"""Web 读取用:确保已加载并返回 list.json 的完整结构(dict)。"""
global USER_DATA, _LOADED
if not _LOADED:
if os.path.exists(FILE_PATH):
try:
raw = json.loads(_read_file_sync(FILE_PATH))
USER_DATA, _ = _migrate_to_v2(raw)
except Exception: # noqa: BLE001
USER_DATA = {"groups": {}, "blacklist": []}
else:
USER_DATA = {"groups": {}, "blacklist": []}
_LOADED = True
return USER_DATA
def set_group_config_sync(new_data: dict) -> bool:
"""Web 保存用:校验结构 → 更新内存 → 同步写回 list.json。
只保留合法字段(auto/plan/auto_link/blacklist),避免脏数据。
"""
global USER_DATA
if not isinstance(new_data, dict):
return False
groups = new_data.get("groups", {})
blacklist = new_data.get("blacklist", [])
if not isinstance(groups, dict) or not isinstance(blacklist, list):
return False
norm_groups: dict[str, dict] = {}
for gid, entry in groups.items():
if not isinstance(entry, dict):
continue
e: dict = {}
if entry.get("auto"):
e["auto"] = True
plan = str(entry.get("plan", "")).upper()
if plan in ("A", "B"):
e["plan"] = plan
if isinstance(entry.get("auto_link"), list):
e["auto_link"] = [str(x) for x in entry["auto_link"]]
norm_groups[str(gid)] = e
norm = {"groups": norm_groups, "blacklist": [str(x) for x in blacklist]}
USER_DATA = norm
_write_json_sync(norm, FILE_PATH + ".tmp")
os.replace(FILE_PATH + ".tmp", FILE_PATH)
return True
await UniMessage.text(f"群 {group_id} 的自动策略已清空").send()
@@ -0,0 +1,486 @@
"""群策略模型与存储(list.json v3)。
策略以「群」为单位,字段全部可选,缺省值见 `Policy`:
auto 自动解析 默认 False
auto_link 自动策略(命中平台即解析) 默认 []
ban_link 禁用策略(命中平台不解析) 默认 []
plan 存储策略 A/B/C 默认 C
upload_public 上传公网 默认 False
send_link 发送下载链接 默认 False(依赖 upload_public)
upload_group_file 上传群文件(并行通道) 默认 False
group_file_platforms 群文件限定平台 默认 [](空 = 全部平台)
文件结构(v3)::
{
"groups": {"<群号>": {<Policy 字段>}},
"default": {<Policy 字段>}, # 私聊 + 群条目缺字段时的兜底
"blacklist": ["<QQ>", ...] # 全局用户黑名单
}
读路径全内存:`verify_user` 在每条消息的热路径上,首次 `load()` 之后不再读盘。
写路径走 `asyncio.Lock` + `.tmp` 原子替换;Web 子应用与 bot 同进程同事件循环,
asyncio 锁即可覆盖两边并发写(Web 不在独立线程里跑写操作)。
换 ORM 时只需再实现一个同样接口的 Store,调用方零改动。
"""
from __future__ import annotations
import asyncio
import json
import os
import re
from collections.abc import Iterable
from dataclasses import dataclass, field, replace
from pathlib import Path
from typing import Any
from nonebot import logger
# ─────────────────────────── 平台 ───────────────────────────
XHS = "小红书"
BILIBILI = "哔哩哔哩"
DOUYIN = "抖音"
YOUTUBE = "Youtube"
X = "X"
#: 规范标签(存储值 = 展示值,命令/Web 输入同一个词)
PLATFORMS: tuple[str, ...] = (XHS, BILIBILI, DOUYIN, YOUTUBE, X)
#: 平台 → 域名片段(判定 auto_link / ban_link 命中用)
PLATFORM_DOMAINS: dict[str, tuple[str, ...]] = {
XHS: ("xiaohongshu.com", "xhslink.com", "xhslink.cn"),
BILIBILI: ("bilibili.com", "b23.tv", "bili2233.cn"),
DOUYIN: (
"douyin.com",
"iesdouyin.com",
"m.douyin.com",
"jingxuan.douyin.com",
),
YOUTUBE: ("youtube.com", "youtu.be"),
X: ("x.com", "twitter.com"),
}
#: 输入别名 → 规范标签(统一小写比较)
_PLATFORM_ALIASES: dict[str, str] = {
"xhs": XHS,
"小红书": XHS,
"bilibili": BILIBILI,
"b23": BILIBILI,
"b站": BILIBILI,
"哔哩哔哩": BILIBILI,
"douyin": DOUYIN,
"抖音": DOUYIN,
"yt": YOUTUBE,
"youtube": YOUTUBE,
"油管": YOUTUBE,
"x": X,
"twitter": X,
"推特": X,
}
#: 分隔符:兼容手写 JSON 里的 "小红书,xhs 抖音" 这类写法
_SPLIT_RE = re.compile(r"[,,、/\s]+")
def normalize_platform(raw: Any) -> str | None:
"""用户输入 → 规范标签,未知返回 None。"""
if not isinstance(raw, str):
return None
return _PLATFORM_ALIASES.get(raw.strip().lower())
def normalize_platforms(values: Any) -> list[str]:
"""归一 + 去重 + 丢弃非法值,结果按 PLATFORMS 顺序排列。
接受字符串(按分隔符拆)或列表。
"""
if values is None:
items: Iterable[Any] = ()
elif isinstance(values, str):
items = _SPLIT_RE.split(values)
elif isinstance(values, (list, tuple, set)):
items = list(values)
else:
return []
picked = {p for p in (normalize_platform(v) for v in items) if p}
return [p for p in PLATFORMS if p in picked]
def apply_platform_diff(current: Iterable[str], tokens: Iterable[str]) -> list[str] | None:
"""按 `+平台` / `-平台` 增删;出现裸平台名时整体覆盖。
全部 token 都带 +/- 时做增量,否则视为覆盖(命令 `视频策略 自动策略 …` 用)。
含未知平台(或覆盖时没有合法平台)返回 None,由调用方提示用法。
"""
tokens = [t for t in tokens if str(t).strip()]
if not tokens:
return None
if all(str(t)[:1] in "+-" for t in tokens):
result = normalize_platforms(current)
for token in tokens:
picked = normalize_platforms([str(token)[1:]])
if not picked:
return None
if str(token)[0] == "+" and picked[0] not in result:
result.append(picked[0])
elif str(token)[0] == "-" and picked[0] in result:
result.remove(picked[0])
return normalize_platforms(result)
picked = normalize_platforms(tokens)
return picked or None
def match_platform(url: str) -> str | None:
"""URL → 平台规范标签;不属于任何已支持平台时返回 None。"""
for name, domains in PLATFORM_DOMAINS.items():
if any(domain in url for domain in domains):
return name
return None
# ─────────────────────────── 策略 ───────────────────────────
PLANS: tuple[str, ...] = ("A", "B", "C")
DEFAULT_PLAN = "C"
POLICY_FIELDS: tuple[str, ...] = (
"auto",
"auto_link",
"ban_link",
"plan",
"upload_public",
"send_link",
"upload_group_file",
"group_file_platforms",
)
@dataclass
class Policy:
"""单个群的策略(字段缺省即默认值,见类文档)。"""
auto: bool = False
auto_link: list[str] = field(default_factory=list)
ban_link: list[str] = field(default_factory=list)
plan: str = DEFAULT_PLAN
upload_public: bool = False
send_link: bool = False
upload_group_file: bool = False
#: 群文件限定平台;空列表 = 所有平台都传群文件
group_file_platforms: list[str] = field(default_factory=list)
def __post_init__(self) -> None:
"""构造即归一:直接 Policy(...) 传入平台别名/非法 plan 也不至于静默失效。"""
self.auto_link = normalize_platforms(self.auto_link)
self.ban_link = normalize_platforms(self.ban_link)
self.group_file_platforms = normalize_platforms(self.group_file_platforms)
if self.plan not in PLANS:
self.plan = DEFAULT_PLAN
@property
def sends_link(self) -> bool:
"""是否真的发下载链接:没有公网链接可发时恒为 False。"""
return self.send_link and self.upload_public
def allows_group_file(self, platform: str | None) -> bool:
"""这个平台的作品要不要传群文件。
开关关着 → 不传;清单为空 → 全部平台都传;否则只认清单里的平台
(平台识别不出来时按"不在清单里"处理)。
"""
if not self.upload_group_file:
return False
if not self.group_file_platforms:
return True
return platform is not None and platform in self.group_file_platforms
def banned(self, url: str) -> str | None:
"""URL 命中的禁用平台(未命中返回 None)。"""
platform = match_platform(url)
return platform if platform and platform in self.ban_link else None
def auto_matched(self, url: str) -> str | None:
"""URL 命中的自动策略平台(未命中返回 None)。"""
platform = match_platform(url)
return platform if platform and platform in self.auto_link else None
def to_dict(self) -> dict[str, Any]:
return {
"auto": self.auto,
"auto_link": list(self.auto_link),
"ban_link": list(self.ban_link),
"plan": self.plan,
"upload_public": self.upload_public,
"send_link": self.send_link,
"upload_group_file": self.upload_group_file,
"group_file_platforms": list(self.group_file_platforms),
}
@classmethod
def from_dict(cls, raw: Any) -> Policy:
"""脏数据归一:非法 plan 回退 C,平台别名转规范标签,多余键丢弃。"""
if not isinstance(raw, dict):
return cls()
plan = str(raw.get("plan", DEFAULT_PLAN)).strip().upper()
return cls(
auto=bool(raw.get("auto", False)),
auto_link=normalize_platforms(raw.get("auto_link")),
ban_link=normalize_platforms(raw.get("ban_link")),
plan=plan if plan in PLANS else DEFAULT_PLAN,
upload_public=bool(raw.get("upload_public", False)),
send_link=bool(raw.get("send_link", False)),
upload_group_file=bool(raw.get("upload_group_file", False)),
group_file_platforms=normalize_platforms(raw.get("group_file_platforms")),
)
# ───────────────────────── 迁移 ─────────────────────────
def _v1_to_v2(raw: dict) -> dict:
"""旧格式(平铺数组)→ group-centric。"""
white = raw.get("WHITE_LIST", [])
auto_list = raw.get("AUTO_ANALYSIS", [])
pa = raw.get("PLANA", [])
pb = raw.get("PLANB", [])
groups: dict[str, dict] = {}
for gid in white:
entry: dict = {}
if gid in auto_list:
entry["auto"] = True
if gid in pa:
entry["plan"] = "A"
elif gid in pb:
entry["plan"] = "B"
groups[str(gid)] = entry
return {"groups": groups, "blacklist": raw.get("BLACK_LIST", [])}
def migrate(raw: Any) -> tuple[dict, bool]:
"""v1 / v2 → v3,返回 (数据, 是否发生迁移)。
v2 → v3 的关键一步:v2 的 `plan=B` 隐含"上传公网",解耦后给它显式补上
`upload_public`,保证已有群行为不变。
"""
if not isinstance(raw, dict):
return _empty_data(), False
changed = False
if "groups" not in raw:
raw = _v1_to_v2(raw)
changed = True
groups: dict[str, dict] = {}
for gid, entry in (raw.get("groups") or {}).items():
if not isinstance(entry, dict):
changed = True
continue
policy = Policy.from_dict(entry)
if policy.plan == "B" and "upload_public" not in entry:
policy.upload_public = True
if entry != policy.to_dict():
changed = True
groups[str(gid)] = policy.to_dict()
if "default" not in raw:
changed = True
data = {
"groups": groups,
"default": Policy.from_dict(raw.get("default")).to_dict(),
"blacklist": [str(x) for x in (raw.get("blacklist") or [])],
}
return data, changed
def _empty_data() -> dict:
return {
"groups": {},
"default": Policy().to_dict(),
"blacklist": [],
}
# ─────────────────────────── 存储 ───────────────────────────
class PolicyStore:
"""list.json 读写:读全内存,写加锁 + 原子替换。"""
def __init__(self, path: Path) -> None:
self.path = Path(path)
self._groups: dict[str, Policy] = {}
self._default = Policy()
self._blacklist: list[str] = []
self._loaded = False
self._lock = asyncio.Lock()
# ── 读 ──────────────────────────────────────────────
def load(self) -> None:
"""首次调用读盘 + 迁移(幂等,之后不再读盘)。"""
if self._loaded:
return
raw: Any = {}
if self.path.exists():
try:
raw = json.loads(self.path.read_text(encoding="utf-8"))
except Exception:
logger.exception(f"读取 {self.path.name} 失败,改用空配置")
data, migrated = migrate(raw)
self._apply(data)
self._loaded = True
if migrated:
self._write_sync(self._snapshot(data))
logger.info(f"{self.path.name} 已迁移为 v3 格式(群 {len(self._groups)} 个)")
def _apply(self, data: dict) -> None:
self._groups = {
str(gid): Policy.from_dict(entry)
for gid, entry in (data.get("groups") or {}).items()
}
self._default = Policy.from_dict(data.get("default"))
self._blacklist = [str(x) for x in (data.get("blacklist") or [])]
def is_whitelisted(self, group_id: Any) -> bool:
"""群是否在白名单里(白名单即 groups 的键,仍是准入门槛)。"""
self.load()
return str(group_id) in self._groups
def get(self, group_id: Any) -> Policy:
"""群策略:未配置的群回落到 default 节。"""
self.load()
return self._groups.get(str(group_id), self._default)
def all_groups(self) -> dict[str, Policy]:
self.load()
return dict(self._groups)
def default_policy(self) -> Policy:
self.load()
return self._default
def blacklist(self) -> list[str]:
self.load()
return list(self._blacklist)
def is_blacklisted(self, user_id: Any) -> bool:
self.load()
return str(user_id) in self._blacklist
# ── 写 ──────────────────────────────────────────────
def _snapshot(self, data: dict | None = None) -> dict:
"""在事件循环线程内构造完整快照,写线程不再触碰共享状态。"""
if data is not None:
return data
return {
"groups": {gid: p.to_dict() for gid, p in self._groups.items()},
"default": self._default.to_dict(),
"blacklist": list(self._blacklist),
}
def _write_sync(self, data: dict) -> None:
tmp = self.path.with_name(self.path.name + ".tmp")
tmp.parent.mkdir(parents=True, exist_ok=True)
tmp.write_text(
json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8"
)
os.replace(tmp, self.path)
async def _save(self) -> None:
snapshot = self._snapshot()
async with self._lock:
await asyncio.to_thread(self._write_sync, snapshot)
async def update_group(
self, group_id: Any, *, create: bool = False, **changes: Any
) -> Policy | None:
"""更新群策略的部分字段;群不存在且 create=False 时返回 None。
changes 传最终值(列表类字段由调用方算好新列表)。
"""
self.load()
gid = str(group_id)
current = self._groups.get(gid)
if current is None:
if not create:
return None
current = Policy()
valid = {k: v for k, v in changes.items() if k in POLICY_FIELDS}
policy = replace(current, **valid) if valid else current
policy = Policy.from_dict(policy.to_dict()) # 归一化(平台别名/非法 plan)
self._groups[gid] = policy
await self._save()
return policy
async def set_group(self, group_id: Any, policy: Policy) -> Policy:
"""整体覆盖群策略(Web 编辑用)。"""
self.load()
gid = str(group_id)
policy = Policy.from_dict(policy.to_dict())
self._groups[gid] = policy
await self._save()
return policy
async def remove_group(self, group_id: Any) -> bool:
self.load()
gid = str(group_id)
if gid not in self._groups:
return False
del self._groups[gid]
await self._save()
return True
async def set_default(self, policy: Policy) -> Policy:
self.load()
self._default = Policy.from_dict(policy.to_dict())
await self._save()
return self._default
async def add_blacklist(self, user_id: Any) -> bool:
self.load()
uid = str(user_id)
if uid in self._blacklist:
return False
self._blacklist.append(uid)
await self._save()
return True
async def set_blacklist(self, values: Iterable[Any]) -> list[str]:
"""整体替换黑名单(去重保序,一次落盘)。"""
self.load()
cleaned: list[str] = []
for value in values:
uid = str(value).strip()
if uid and uid not in cleaned:
cleaned.append(uid)
self._blacklist = cleaned
await self._save()
return list(self._blacklist)
async def remove_blacklist(self, user_id: Any) -> bool:
self.load()
uid = str(user_id)
if uid not in self._blacklist:
return False
self._blacklist.remove(uid)
await self._save()
return True
#: 单例(插件自己的 data/ 目录,与历史 list.json 同路径,原地迁移)
STORE = PolicyStore(Path(__file__).resolve().parent / "data" / "list.json")
@@ -14,15 +14,19 @@
"""
import re
import tempfile
from datetime import datetime
from pathlib import Path
from typing import Optional, Union
from nonebot import logger
from ...models import ContentFetchError
from ...utils import get_data_dir, parse_netscape_cookies, slugify
from ...utils import (
build_author_dir,
build_work_stem,
get_data_dir,
get_temp_root,
parse_netscape_cookies,
)
DATA_DIR = get_data_dir()
@@ -81,10 +85,10 @@ async def _parse(url: str):
# 4. 文章动态 → 转 opus
if await dynamic.is_article():
return await _parse_opus(dynamic.turn_to_opus(), "文章")
return await _parse_opus(dynamic.turn_to_opus(), "文章", url)
info = await dynamic.get_info()
return await _parse_dynamic_info(info)
return await _parse_dynamic_info(info, url)
async def _parse_article(read_id: int) -> tuple[str, Union[Path, list[Path]]]:
@@ -94,10 +98,12 @@ async def _parse_article(read_id: int) -> tuple[str, Union[Path, list[Path]]]:
# 文章接口对匿名请求风控更严(-509),必须带凭证
article = Article(read_id, _build_credential())
opus = await article.turn_to_opus()
return await _parse_opus(opus, "文章")
return await _parse_opus(opus, "文章", f"cv{read_id}")
async def _parse_opus(opus, kind: str) -> tuple[str, Union[Path, list[Path]]]:
async def _parse_opus(
opus, kind: str, source: str = ""
) -> tuple[str, Union[Path, list[Path]]]:
"""图文动态/专栏解析(opus 接口返回 dict,直接访问)"""
info = await opus.get_info()
item = info.get("item") or {}
@@ -107,10 +113,12 @@ async def _parse_opus(opus, kind: str) -> tuple[str, Union[Path, list[Path]]]:
images: list[str] = []
texts: list[str] = []
author = ""
author_id = ""
for module in item.get("modules") or []:
if module.get("module_type") == "MODULE_TYPE_AUTHOR":
author_info = module.get("module_author") or {}
author = author_info.get("name", "")
author_id = str(author_info.get("mid") or "")
elif module.get("module_type") == "MODULE_TYPE_CONTENT":
content = module.get("module_content") or {}
for para in content.get("paragraphs") or []:
@@ -127,16 +135,20 @@ async def _parse_opus(opus, kind: str) -> tuple[str, Union[Path, list[Path]]]:
if not images:
return text or f"B站{kind}", []
file_name = _build_file_name(author, text or f"B站{kind}", kind)
file_paths = await _download_images(images, file_name)
rel_stem = _build_rel_stem(author, author_id, text or f"B站{kind}", source)
file_paths = await _download_images(images, rel_stem)
return text, file_paths
async def _parse_dynamic_info(info: dict) -> tuple[str, Union[Path, list[Path]]]:
async def _parse_dynamic_info(
info: dict, source: str = ""
) -> tuple[str, Union[Path, list[Path]]]:
"""动态解析(图文 / 视频 / 纯文字)"""
item = info.get("item") or {}
modules = item.get("modules") or {}
author = ((modules.get("module_author") or {}).get("name")) or "B站用户"
module_author = modules.get("module_author") or {}
author = module_author.get("name") or "B站用户"
author_id = str(module_author.get("mid") or "")
module_dynamic = modules.get("module_dynamic") or {}
major = module_dynamic.get("major") or {}
major_type = major.get("type", "")
@@ -151,7 +163,13 @@ async def _parse_dynamic_info(info: dict) -> tuple[str, Union[Path, list[Path]]]
title = archive.get("title") or desc or "B站视频动态"
logger.info(f"B站视频动态: bvid={bvid} 标题={title[:40]}")
video_path = await download_video(f"https://www.bilibili.com/video/{bvid}")
# 动态数据里已有 up 的昵称/mid,传下去才能和同一位 up 的图文
# 落在同一个作者目录(否则要赌 yt-dlp 返回的 id 对得上)
video_path, _ = await download_video(
f"https://www.bilibili.com/video/{bvid}",
author=author,
author_id=author_id,
)
if video_path:
return title, video_path
raise ContentFetchError(f"视频动态下载失败: {bvid}")
@@ -173,8 +191,8 @@ async def _parse_dynamic_info(info: dict) -> tuple[str, Union[Path, list[Path]]]
images = [u for u in images if u]
if images:
file_name = _build_file_name(author, title, "动态")
file_paths = await _download_images(images, file_name)
rel_stem = _build_rel_stem(author, author_id, title, source)
file_paths = await _download_images(images, rel_stem)
logger.info(f"B站图文动态: 作者={author}, 标题={title[:40]}, 图片={len(images)} 张")
return title, file_paths
@@ -197,14 +215,17 @@ def _extract_text(nodes: list) -> str:
return "".join(parts)
def _build_file_name(author: str, title: str, kind: str) -> str:
"""构建文件名 stem: {作者}_{标题}_{类型}_{时间}"""
slug_author = slugify(author)
slug_title = slugify(title or "", max_length=15)
if not slug_title:
slug_title = datetime.now().strftime("%H%M%S")
time_suffix = datetime.now().strftime("%H%M%S")
return f"{slug_author}_{slug_title}_{kind}_{time_suffix}"
def _build_rel_stem(
author: str, author_id: str, title: str, source: str = ""
) -> str:
"""相对平台根的路径词干:`{作者}_{mid}/{作品名}`
昵称/mid 都拿不到时用 source(动态链接)当来源码(见 utils.build_author_dir)。
"""
return (
f"{build_author_dir(author, author_id, source=source)}"
f"/{build_work_stem(title)}"
)
def _build_credential():
@@ -227,20 +248,21 @@ def _build_credential():
)
async def _download_images(image_urls: list[str], file_name: str) -> list[Path]:
"""并发下载图片(复用抖音图文的下载流程)"""
import httpx
async def _download_images(image_urls: list[str], rel_stem: str) -> list[Path]:
"""并发下载图片(复用抖音图文的下载流程)
落 hexi/data/temp/bilibili(原先落系统 temp,cleanup 扫不到、永不清理)。
"""
from .douyin_api import _process_note_with_parsed
tmp_root = Path(tempfile.gettempdir()) / "bilibili"
tmp_root.mkdir(parents=True, exist_ok=True)
tmp_root = get_temp_root("bilibili")
headers = {
"Referer": BILI_REFERER,
"User-Agent": BILI_UA,
}
return await _process_note_with_parsed(
[[u] for u in image_urls], None, tmp_root, file_name, headers
[[u] for u in image_urls], None, tmp_root, rel_stem, headers
)
@@ -10,11 +10,9 @@ from nonebot import logger
from playwright.async_api import async_playwright
from ...models import DouyinFetchError
from ...utils import ensure_unique_path, get_temp_root
from ...utils import get_temp_root, unique_media_path
from .douyin_parser import (
ParsedDouyinContent,
extract_trailing_digits,
is_animated_note,
parse_animated_note_videos,
parse_douyin_response,
parse_note_images,
@@ -194,9 +192,9 @@ async def fetch_douyin_content(
if parsed.media_type == "视频":
content = await _process_video(
api_response, tmp_root, parsed.file_name, aweme_id, headers
api_response, tmp_root, parsed.rel_stem, aweme_id, headers
)
return parsed.file_name, content
return parsed.raw_title, content
elif parsed.media_type == "图片":
# 先解析 images 列表,区分纯动图和图文/图+视频
@@ -209,15 +207,15 @@ async def fetch_douyin_content(
images_urls,
video_url,
tmp_root,
parsed.file_name,
parsed.rel_stem,
headers,
)
else:
# 纯动图(所有项都是视频)
content = await _process_animated_note(
api_response, tmp_root, parsed.file_name, headers
api_response, tmp_root, parsed.rel_stem, headers
)
return parsed.file_name, content
return parsed.raw_title, content
return None, None
@@ -228,11 +226,14 @@ async def fetch_douyin_content(
async def _process_video(
api_response: dict,
tmp_root: Path,
file_name: str,
rel_stem: str,
aweme_id: str,
headers: Dict[str, str],
) -> Path:
"""处理视频内容,返回本地文件路径"""
"""处理视频内容,返回本地文件路径
rel_stem 是相对平台根的路径词干 `{作者目录}/{作品名}`(见 ParsedDouyinContent.rel_stem)。
"""
groups = parse_video_urls(api_response)
best_group = None
@@ -251,7 +252,7 @@ async def _process_video(
best = max(full, key=lambda x: x["br"])
logger.info(f"选择码率: {best['br']} - {best['url'][:60]}...")
output_path = ensure_unique_path(tmp_root / f"{file_name}.mp4")
output_path = unique_media_path(tmp_root / f"{rel_stem}.mp4")
async with httpx.AsyncClient(headers=headers) as client:
async with client.stream("GET", best["url"]) as resp:
resp.raise_for_status()
@@ -271,9 +272,10 @@ async def _process_video(
logger.info(f"选择视频码率: {video['br']}")
logger.info(f"选择音频码率: {audio['br']}")
video_path = tmp_root / f"{file_name}_v.mp4"
audio_path = tmp_root / f"{file_name}_a.mp4"
output_path = ensure_unique_path(tmp_root / f"{file_name}.mp4")
# 先定下产物名(顺带建好作者目录),分轨中间文件与产物同目录
output_path = unique_media_path(tmp_root / f"{rel_stem}.mp4")
video_path = output_path.parent / f"{output_path.stem}_v.mp4"
audio_path = output_path.parent / f"{output_path.stem}_a.mp4"
async with httpx.AsyncClient(headers=headers) as client:
logger.info("开始下载视频...")
@@ -291,7 +293,9 @@ async def _process_video(
f.write(chunk)
logger.info("合并视频和音频...")
merge_video_audio(video_path, audio_path, output_path)
# ffmpeg 是同步子进程,直接 await 不了:不丢线程池会卡住整个事件循环
# (合并期间 Web 轮询、群消息全都停摆)
await asyncio.to_thread(merge_video_audio, video_path, audio_path, output_path)
video_path.unlink()
audio_path.unlink()
@@ -306,11 +310,14 @@ async def _process_note_with_parsed(
images_urls: List[List[str]],
video_url: Optional[str],
tmp_root: Path,
file_name: str,
rel_stem: str,
headers: Dict[str, str],
) -> List[Path]:
"""根据已解析的图片/视频 URL 列表,并行下载"""
note_dir = ensure_unique_path(tmp_root / file_name)
"""根据已解析的图片/视频 URL 列表,并行下载
一个作品一个目录:`{平台根}/{作者目录}/{作品名}[_{短码}]/001.jpg…`
"""
note_dir = unique_media_path(tmp_root / rel_stem)
note_dir.mkdir(parents=True, exist_ok=True)
logger.info(f"图文保存目录: {note_dir}")
@@ -353,7 +360,7 @@ async def _process_note(
api_response: dict,
api_response_favorite: dict,
tmp_root: Path,
file_name: str,
rel_stem: str,
aweme_id: str,
headers: Dict[str, str],
) -> List[Path]:
@@ -367,7 +374,7 @@ async def _process_note(
raise DouyinFetchError("未找到图文链接")
return await _process_note_with_parsed(
images_urls, video_url, tmp_root, file_name, headers
images_urls, video_url, tmp_root, rel_stem, headers
)
@@ -377,14 +384,14 @@ async def _process_note(
async def _process_animated_note(
api_response: dict,
tmp_root: Path,
file_name: str,
rel_stem: str,
headers: Dict[str, str],
) -> List[Path]:
"""处理动图内容(media_type=42),并行下载所有无声 mp4 视频"""
video_urls = parse_animated_note_videos(api_response)
logger.info(f"解析到的动图视频链接: {video_urls}")
note_dir = ensure_unique_path(tmp_root / file_name)
note_dir = unique_media_path(tmp_root / rel_stem)
note_dir.mkdir(parents=True, exist_ok=True)
logger.info(f"动图保存目录: {note_dir}")
@@ -4,13 +4,12 @@ import json
import re
import urllib.parse
from dataclasses import dataclass
from datetime import datetime
from typing import Dict, List, Optional
from nonebot import logger
from ...models import DouyinFetchError
from ...utils import slugify
from ...utils import build_author_dir, build_work_stem
# ============================= 数据结构 =============================
@@ -22,7 +21,13 @@ class ParsedDouyinContent:
raw_title: str
raw_nickname: str
media_type: str # "视频" | "图片"
file_name: str # 构建好的文件名 stem
file_name: str # 作品名 stem(落盘文件名/多图子目录名)
author_dir: str = "" # 作者目录 `{昵称}_{uid}`(落盘与 S3 key 的首层)
@property
def rel_stem(self) -> str:
"""相对平台根目录的路径词干:`{作者目录}/{作品名}`"""
return f"{self.author_dir}/{self.file_name}" if self.author_dir else self.file_name
# ============================= URL 工具 =============================
@@ -109,6 +114,29 @@ def extract_author_nickname(api_response: dict) -> str:
return nickname
def extract_author_id(api_response: dict) -> str:
"""从 API 响应提取作者稳定 id,优先级: uid > unique_id(抖音号) > sec_uid
SSR 路径的 `aweme_list[0].author.uid` 与 API 路径的
`aweme_detail.author.uid` 走同一套取值;都拿不到返回空串,
作者目录退化成只用昵称(见 utils.build_author_dir)。
"""
aweme_detail = api_response.get("aweme_detail") or {}
author = aweme_detail.get("author") or {}
if not author:
aweme_list = api_response.get("aweme_list") or []
if aweme_list and isinstance(aweme_list, list):
author = (aweme_list[0] or {}).get("author") or {}
for key in ("uid", "unique_id", "sec_uid"):
value = str(author.get(key) or "").strip()
if value and value != "0":
logger.info(f"RAW作者id({key}):{value}")
return value
logger.info("未获取到作者 id,作者目录只用昵称")
return ""
def detect_media_type(referer_url: str | None) -> str | None:
"""根据页面 URL 检测媒体类型(视频/图片)"""
if referer_url is None:
@@ -326,36 +354,6 @@ def parse_ssr_page(html: str) -> Optional[dict]:
return {"aweme_list": [item]}
# ============================= 文件名构建 =============================
def build_file_name(
raw_title: str,
raw_nickname: str,
media_type: str,
) -> str:
"""
构建文件名 stem,格式: {作者}_{标题}_{类型}_{时间戳}
昵称不限长,标题最多 15 字符(slugify 后),末尾 HHMMSS 防覆盖。
"""
slug_nickname = slugify(raw_nickname)
if raw_title:
slug_title = slugify(raw_title, max_length=15)
else:
slug_title = ""
if not slug_title:
slug_title = datetime.now().strftime("%H%M%S")
logger.info(f"标题为空,使用短时间戳: {slug_title}")
slug_type = slugify(media_type)
time_suffix = datetime.now().strftime("%H%M%S")
return f"{slug_nickname}_{slug_title}_{slug_type}_{time_suffix}"
# ============================= 动图检测 =============================
@@ -410,11 +408,21 @@ def parse_douyin_response(
raw_title = extract_title_from_api(api_response)
raw_nickname = extract_author_nickname(api_response)
file_name = build_file_name(raw_title, raw_nickname, media_type)
raw_author_id = extract_author_id(api_response)
# 昵称/uid 都拿不到时的来源码:优先作品链接,其次响应里的作品 id
aweme_id = str(
(api_response.get("aweme_detail") or {}).get("aweme_id")
or ((api_response.get("aweme_list") or [{}])[0] or {}).get("aweme_id")
or ""
)
author_dir = build_author_dir(
raw_nickname, raw_author_id, source=referer_url or aweme_id
)
file_name = build_work_stem(raw_title)
logger.info(
f"内容标题: {raw_title}, 作者: {raw_nickname}, "
f"类型: {media_type}, 文件名: {file_name}"
f"内容标题: {raw_title}, 作者: {raw_nickname}({raw_author_id}), "
f"类型: {media_type}, 落盘路径: {author_dir}/{file_name}"
)
return ParsedDouyinContent(
@@ -422,4 +430,5 @@ def parse_douyin_response(
raw_nickname=raw_nickname,
media_type=media_type,
file_name=file_name,
author_dir=author_dir,
)
@@ -85,13 +85,13 @@ async def fetch_douyin_note_ssr(
images_urls, video_url = parse_note_images(api_response, None, vid)
if images_urls:
file_paths = await _process_note_with_parsed(
images_urls, video_url, tmp_root, parsed.file_name, headers
images_urls, video_url, tmp_root, parsed.rel_stem, headers
)
else:
file_paths = await _process_animated_note(
api_response, tmp_root, parsed.file_name, headers
api_response, tmp_root, parsed.rel_stem, headers
)
return parsed.file_name, file_paths
return parsed.raw_title, file_paths
async def _fetch_ssr_api_response(
@@ -19,16 +19,21 @@
import asyncio
import json
import re
import tempfile
import urllib.parse
from datetime import datetime
from pathlib import Path
from typing import Optional, Union
from nonebot import logger
from ...models import ContentFetchError
from ...utils import get_data_dir, get_temp_root, parse_netscape_cookies, slugify
from ...utils import (
build_author_dir,
build_work_stem,
get_data_dir,
get_temp_root,
parse_netscape_cookies,
unique_media_path,
)
DATA_DIR = get_data_dir()
@@ -214,6 +219,12 @@ async def _build_result(note: dict) -> tuple[str, Union[Path, list[Path]]]:
desc = note.get("desc") or ""
nickname = ((note.get("user") or {}).get("nickname")) or "小红书用户"
text = title or desc or "小红书笔记"
rel_stem = _build_rel_stem(
nickname,
_extract_author_id(note),
text,
source=str(note.get("noteId") or note.get("id") or ""),
)
# 1. 视频笔记 → 无水印原片优先
if note.get("type") == "video" and note.get("video"):
@@ -223,8 +234,7 @@ async def _build_result(note: dict) -> tuple[str, Union[Path, list[Path]]]:
if okey:
video_url = f"https://sns-video-bd.xhscdn.com/{okey}"
logger.info(f"小红书视频: 无水印原片 originVideoKey={okey[:30]}...")
file_name = _build_file_name(nickname, text, "视频")
video_path = await _download_video(video_url, file_name)
video_path = await _download_video(video_url, rel_stem)
return text, video_path
# 1b. 无 originVideoKey(国内站数据)→ 从 stream 分组选无水印原片
@@ -251,8 +261,7 @@ async def _build_result(note: dict) -> tuple[str, Union[Path, list[Path]]]:
f"{best.get('width')}x{best.get('height')} {best.get('fps')}fps "
f"size={best.get('size')} duration={duration}ms"
)
file_name = _build_file_name(nickname, text, "视频")
video_path = await _download_video(video_url, file_name)
video_path = await _download_video(video_url, rel_stem)
return text, video_path
raise ContentFetchError("小红书视频流解析失败")
@@ -266,25 +275,36 @@ async def _build_result(note: dict) -> tuple[str, Union[Path, list[Path]]]:
logger.info(f"小红书文字笔记: {text[:30]}")
return text, []
file_name = _build_file_name(nickname, text, "笔记")
file_paths = await _download_images(images, file_name)
file_paths = await _download_images(images, rel_stem)
logger.info(f"小红书图文笔记: 作者={nickname}, 图片={len(images)} 张")
return text, file_paths
def _build_file_name(nickname: str, title: str, kind: str) -> str:
"""构建文件名 stem: {作者}_{标题}_{类型}_{时间}"""
slug_nickname = slugify(nickname)
slug_title = slugify(title or "", max_length=15)
if not slug_title:
slug_title = datetime.now().strftime("%H%M%S")
time_suffix = datetime.now().strftime("%H%M%S")
return f"{slug_nickname}_{slug_title}_{kind}_{time_suffix}"
def _extract_author_id(note: dict) -> str:
"""小红书作者稳定 id(页面数据字段未实测,逐个兜底;取不到返回空串)"""
user = note.get("user") or {}
for key in ("userId", "user_id", "id"):
value = user.get(key)
if isinstance(value, (str, int)) and str(value).strip() not in ("", "0"):
return str(value).strip()
return ""
async def _download_images(image_urls: list[str], file_name: str) -> list[Path]:
def _build_rel_stem(
nickname: str, author_id: str, title: str, source: str = ""
) -> str:
"""相对平台根的路径词干:`{作者}_{userId}/{作品名}`
昵称/作者 id 都拿不到时用 source(笔记 id)当来源码(见 utils.build_author_dir)。
"""
return (
f"{build_author_dir(nickname, author_id, source=source)}"
f"/{build_work_stem(title)}"
)
async def _download_images(image_urls: list[str], rel_stem: str) -> list[Path]:
"""并发下载图片(复用抖音图文的下载流程)"""
import httpx
from .douyin_api import _process_note_with_parsed
@@ -294,11 +314,11 @@ async def _download_images(image_urls: list[str], file_name: str) -> list[Path]:
"User-Agent": REDNOTE_UA,
}
return await _process_note_with_parsed(
[[u] for u in image_urls], None, tmp_root, file_name, headers
[[u] for u in image_urls], None, tmp_root, rel_stem, headers
)
async def _download_video(video_url: str, file_name: str) -> Path:
async def _download_video(video_url: str, rel_stem: str) -> Path:
"""流式下载视频
注意:sns-video-bd(无水印原片)不带 Referer 或带 xiaohongshu.com
@@ -307,7 +327,7 @@ async def _download_video(video_url: str, file_name: str) -> Path:
import httpx
tmp_root = get_temp_root("xiaohongshu")
output_path = tmp_root / f"{file_name}.mp4"
output_path = unique_media_path(tmp_root / f"{rel_stem}.mp4")
headers = {"User-Agent": REDNOTE_UA}
async with httpx.AsyncClient(headers=headers, timeout=300) as client:
async with client.stream("GET", video_url) as resp:
@@ -3,9 +3,9 @@
import asyncio
import os
import re
import shutil
import sys
import tempfile
from datetime import datetime
from pathlib import Path
from typing import Optional
@@ -14,7 +14,14 @@ from nonebot import logger
from yt_dlp import YoutubeDL
from yt_dlp.utils import DownloadError
from ...utils import get_data_dir, get_temp_root, slugify, ensure_unique_path
from ...utils import (
build_author_dir,
build_work_stem,
get_data_dir,
get_temp_root,
slugify,
unique_media_path,
)
def detect_platform(url: str) -> str:
@@ -41,6 +48,16 @@ def extract_uploader(info: dict) -> Optional[str]:
)
def extract_uploader_id(info: dict) -> str:
"""作者稳定 id:channel_id > uploader_id(@handle / B站 mid),拿不到返回空串
不用 `id`(那是作品 id,会把同一作者的作品拆到不同目录)。
"""
if not info:
return ""
return str(info.get("channel_id") or info.get("uploader_id") or "").strip()
def get_ffmpeg_path() -> str:
scripts_dir = os.path.dirname(sys.executable)
ffmpeg_path = os.path.join(scripts_dir, "ffmpeg.exe")
@@ -78,19 +95,36 @@ async def _retry_download(
logger.error(
f"yt-dlp 重试 {max_retries} 次后仍失败: {str(e)[:120]}"
)
except Exception as e:
except Exception:
# 非 DownloadError(如 OSError)不重试,直接抛出
raise
raise last_error # type: ignore[misc]
async def download_video(url: str) -> Optional[Path]:
"""下载视频,支持直链和 yt-dlp"""
async def download_video(
url: str,
*,
author: Optional[str] = None,
author_id: Optional[str] = None,
) -> tuple[Optional[Path], str]:
"""下载视频,支持直链和 yt-dlp
author / author_id 可由调用方覆盖(如 B站 视频动态已从动态数据里拿到 mid,
传进来才能和同一位 up 的图文落在同一个作者目录)。
Returns:
(本地文件, 作品标题) — 失败时 (None, "")。
落盘位置:`temp/{平台}/{作者}_{作者id}/{作品名}[_{短码}].ext`
(直链拿不到作者信息,统一进 `未知作者/`)。
"""
platform = detect_platform(url)
temp_root = get_temp_root(platform)
# ---------- 1. 直链探测 ----------
# 不含 m3u8:HLS 播放列表直下只会得到一个文本文件,交给 yt-dlp 处理
direct_media_ext = re.search(
r"\.(mp4|m3u8|ts|webm|mov|flv)(?:$|\?)", url, re.IGNORECASE
r"\.(mp4|ts|webm|mov|flv)(?:$|\?)", url, re.IGNORECASE
)
is_direct = bool(direct_media_ext)
@@ -105,39 +139,35 @@ async def download_video(url: str) -> Optional[Path]:
is_direct = False
if is_direct:
temp_dir = tempfile.mkdtemp(prefix="direct_ytcache_", dir=get_temp_root("ytcache"))
ext = "mp4"
m = re.search(r"\.([a-zA-Z0-9]{2,5})(?:$|\?)", url)
if m and len(m.group(1)) <= 5:
ext = m.group(1)
url_stem = Path(url.split("?")[0]).stem or "video"
slug_stem = slugify(url_stem, max_length=15)
if not slug_stem:
slug_stem = datetime.now().strftime("%H%M%S")
time_suffix = datetime.now().strftime("%H%M%S")
new_name = f"{slug_stem}_视频_{time_suffix}.{ext}"
filename = os.path.join(temp_dir, new_name)
slug_stem = slugify(url_stem, max_length=15) or "视频"
# 直链拿不到作者信息 → `未知作者_{来源短码}`(同一链接稳定、不同链接不撞)
author_dir = build_author_dir(author, author_id, source=url)
final_path = unique_media_path(temp_root / author_dir / f"{slug_stem}.{ext}")
try:
async with AsyncClient(follow_redirects=True, timeout=300) as client:
async with client.stream("GET", url) as resp:
resp.raise_for_status()
with open(filename, "wb") as fh:
with open(final_path, "wb") as fh:
async for chunk in resp.aiter_bytes(chunk_size=8192):
fh.write(chunk)
final_path = ensure_unique_path(Path(filename))
logger.info(f"直接下载完成: {final_path}")
return final_path
return final_path, ""
except Exception:
logger.exception("直接下载失败,回退 yt-dlp")
if os.path.exists(filename):
os.remove(filename)
final_path.unlink(missing_ok=True)
# ---------- 2. yt-dlp 下载 ----------
platform = detect_platform(url)
temp_dir = tempfile.mkdtemp(prefix="ytcache_", dir=get_temp_root("ytcache"))
# 先下到 scratch 目录(outtmpl 必须在拿到 info 之前给定),拿到 info 后再
# 按作者归位到 temp/{平台}/{作者}_{作者id}/
temp_dir = tempfile.mkdtemp(prefix="_dl_", dir=temp_root)
output_path = os.path.join(temp_dir, "%(title).80s.%(ext)s")
base_opts = {
@@ -188,7 +218,10 @@ async def download_video(url: str) -> Optional[Path]:
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8",
}
base_opts["extractor_args"] = {"twitter": {"api": ["syndication"]}}
# 不要强制 twitter:api=syndication:该端点为未登录视角,对敏感/受限推文
# 只返回 tombstone(无 mediaDetails),且会无条件覆盖已登录 GraphQL 的结果,
# 表现为 "No video could be found in this tweet"。默认走 GraphQL + cookies,
# 遇 429 yt-dlp 会自行回退 syndication。
elif platform == "youtube":
base_opts["http_headers"] = {
"User-Agent": ua,
@@ -206,12 +239,20 @@ async def download_video(url: str) -> Optional[Path]:
info = await _retry_download(loop, url, base_opts)
except Exception:
logger.exception("yt-dlp 下载失败")
if platform == "youtube" and "cookiefile" in base_opts:
# YouTube: cookies 可能触发 bot 检测导致只返回图片无视频格式
# 回退无 cookie 模式重试
if platform == "youtube" and "cookiefile" in base_opts:
logger.info("YouTube 回退无 cookies 模式重试...")
base_opts.pop("cookiefile", None)
base_opts.pop("http_headers", None)
elif platform == "twitter":
# X 登录态失效(auth_token 过期)时 GraphQL 会直接拒绝请求;
# 退回未登录的 syndication 端点,公开推文仍可下载(敏感推文会失败)
logger.info("Twitter 回退 syndication 端点重试...")
base_opts["extractor_args"] = {"twitter": {"api": ["syndication"]}}
else:
return None, ""
# 清理失败残留
for f in Path(temp_dir).glob("*.*"):
try:
@@ -221,39 +262,30 @@ async def download_video(url: str) -> Optional[Path]:
try:
info = await _retry_download(loop, url, base_opts, max_retries=2)
except Exception:
logger.exception("yt-dlp 无 cookies 重试也失败")
return None
else:
return None
logger.exception("yt-dlp 回退重试也失败")
return None, ""
files = list(Path(temp_dir).glob("*.*"))
if not files:
return None
return None, ""
original_file = files[0]
# 构建新文件名
uploader = extract_uploader(info or {})
# 归位到作者目录:{作者}_{作者id}/{作品名}[_{短码}].ext
uploader = author or extract_uploader(info or {})
uploader_id = author_id or extract_uploader_id(info or {})
title = ((info or {}).get("title") or "").strip()
slug_title = slugify(title, max_length=15) if title else ""
if not slug_title:
slug_title = datetime.now().strftime("%H%M%S")
time_suffix = datetime.now().strftime("%H%M%S")
if uploader:
slug_uploader = slugify(str(uploader))
new_stem = f"{slug_uploader}_{slug_title}_视频_{time_suffix}"
else:
new_stem = f"{slug_title}_视频_{time_suffix}"
new_path = ensure_unique_path(
original_file.with_name(f"{new_stem}{original_file.suffix}")
author_dir = build_author_dir(uploader, uploader_id, source=url)
new_path = unique_media_path(
temp_root / author_dir / f"{build_work_stem(title)}{original_file.suffix}"
)
original_file.rename(new_path)
shutil.move(str(original_file), str(new_path))
# 下载用的 scratch 目录已空,顺手收掉(cleanup 不删目录)
shutil.rmtree(temp_dir, ignore_errors=True)
logger.info(
f"yt-dlp 下载完成, 标题: {title}, "
f"作者: {uploader}, 重命名: {new_path}"
f"作者: {uploader}({uploader_id}), 落盘: {new_path}"
)
return new_path
return new_path, title
@@ -0,0 +1,271 @@
"""群文件上传 —— 与消息发送并行的第二条投递通道。
策略开关见 `policy.Policy.upload_group_file`:开着的时候,媒体照常发到群里,
同时另传一份到群文件(群友可随时下载、不占聊天记录)。
投递前会按全局配置打包(`video_analysis_group_file_zip`)成一个 zip:
配了解压密码(`video_analysis_group_file_password`)就用 AES-256 加密,
**密码设置了但 pyzipper 不可用时直接放弃上传,绝不退化成传明文**。
(无密码的普通 zip 走标准库,零依赖。)
失败只记日志(bot 没有群文件权限、超出群文件大小上限等都属于预期内的失败),
不影响消息发送链路;上传成功后不额外发消息,避免刷屏。
"""
from __future__ import annotations
import asyncio
import re
import zipfile
from collections.abc import Iterable
from datetime import datetime
from pathlib import Path
from typing import TYPE_CHECKING, Any
from nonebot import get_bot, logger
if TYPE_CHECKING: # 仅类型检查:运行期不导入,单测可裸加载本模块
from ..policy import Policy
try: # 缺失时只影响「加密打包」这一路,见 build_archive
import pyzipper
except ImportError: # pragma: no cover
pyzipper = None # type: ignore[assignment]
def encryption_available() -> bool:
"""加密打包是否可用(pyzipper 已安装)。"""
return pyzipper is not None
def build_archive(
files: Iterable[Path | str],
title: str = "",
*,
password: str = "",
out_dir: Path | None = None,
rel_dir: str = "",
) -> Path:
"""把文件打包成一个 zip,返回产物路径;password 非空则 AES-256 加密。
out_dir 缺省落 `hexi/data/temp/archive/{作者目录}`(rel_dir 由调用方传入,
与源媒体同一套目录结构,见 sender.media_rel_dir_of),由 cleanup 按天清理。
加密需要 pyzipper,缺失时抛 RuntimeError —— 调用方应当**放弃上传**。
"""
paths = [Path(f) for f in files]
if not paths:
raise ValueError("没有可打包的文件")
missing = [p for p in paths if not p.exists()]
if missing:
raise FileNotFoundError(f"待打包文件不存在: {missing[0]}")
if password and pyzipper is None:
raise RuntimeError("配置了解压密码,但 pyzipper 未安装,无法加密打包")
archive = _resolve_target(out_dir, title, rel_dir)
if password:
with pyzipper.AESZipFile(
archive,
"w",
compression=pyzipper.ZIP_DEFLATED,
encryption=pyzipper.WZ_AES,
) as zf:
zf.setpassword(password.encode("utf-8"))
_write_members(zf, paths)
else:
with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as zf:
_write_members(zf, paths)
logger.info(
f"群文件打包完成: {archive.name}({len(paths)} 个文件"
f"{',已加密' if password else ''})"
)
return archive
def _default_archive_dir() -> Path:
"""默认落 temp/archive(延迟导入 utils:单测裸加载本模块时没有包上下文)。"""
from ...utils import get_temp_root
return get_temp_root("archive")
def _safe_stem(title: str, max_length: int = 15) -> str:
"""标题 → 安全的文件名片段。
原始标题(抖音文案、YouTube 标题)可能带 `/`、换行、控制字符与 `#话题`,
这些进不了文件名,统一换成 `_` 后截断。
"""
cleaned = re.sub(r"#\S+", "", title)
cleaned = re.sub(r'[\\/:*?"<>|\s]+', "_", cleaned.strip())
cleaned = re.sub(r"_+", "_", cleaned).strip("_")
return cleaned[:max_length].strip("_")
def _resolve_target(out_dir: Path | None, title: str, rel_dir: str = "") -> Path:
"""产物最终路径:默认 temp/archive/{rel_dir},重名自动加序号。
空标题 / 占位符 / 已带「群文件」前缀的一律回退成默认名。
"""
base = Path(out_dir) if out_dir is not None else _default_archive_dir()
if rel_dir:
base = base / rel_dir
base.mkdir(parents=True, exist_ok=True)
stamp = f"{datetime.now():%H%M%S}"
stem = _safe_stem(title) if title else ""
if not stem or stem in ("title", "video") or stem.startswith("群文件"):
return _unique_path(base / f"群文件_{stamp}.zip")
return _unique_path(base / f"{stem}_群文件_{stamp}.zip")
def _unique_path(path: Path) -> Path:
"""重名追加 _2/_3…(同 utils.ensure_unique_path 的语义,就地实现)。"""
if not path.exists():
return path
index = 2
while True:
candidate = path.with_name(f"{path.stem}_{index}{path.suffix}")
if not candidate.exists():
return candidate
index += 1
def _write_members(zf: Any, paths: list[Path]) -> None:
"""写入成员:同名文件自动加序号,避免在包里互相覆盖。"""
used: set[str] = set()
for path in paths:
name = path.name
if name in used:
index = 2
while f"{path.stem}_{index}{path.suffix}" in used:
index += 1
name = f"{path.stem}_{index}{path.suffix}"
used.add(name)
zf.write(path, arcname=name)
def _file_uri(path: Path) -> str:
"""本地文件 → `file:///D:/a/b.zip`(正斜杠,中文/空格不转义)。
与图片/视频消息发给 NapCat 的形式一致;直接传 Windows 反斜杠路径会被
它的 realpath 判成 ENOENT(见 upload_group_file 的降级说明)。
"""
return "file:///" + str(path.resolve()).replace("\\", "/")
async def _call_upload(
file_value: str, group_id: int, name: str, folder_id: str | None = None
) -> None:
params: dict = {"group_id": group_id, "file": file_value, "name": name}
if folder_id:
params["folder"] = folder_id
await get_bot().call_api("upload_group_file", **params)
def _s3_url(path: Path, policy: "Policy | None") -> str:
"""把文件传到局域网 S3 换预签名链接(延迟导入 s3,便于单测裸加载本模块)。"""
from .s3 import upload_with_plan
url, _ = upload_with_plan(path, policy=policy)
return url
async def upload_group_file(
file_path: Path | str,
group_id: int,
*,
folder_id: str | None = None,
policy: Policy | None = None,
name: str | None = None,
) -> bool:
"""上传单个文件到群文件,返回是否成功。
两级投递,**S3 链接优先、本地直传兜底**:
本环境实测 NapCat 读不到 bot 进程写的本地文件(`realpath ... ENOENT`,
媒体消息的本地直发 55 次全失败、换 S3 链接后次次成功),所以直传只留作
S3 不可用时的兜底。policy 决定 S3 落到哪个桶(plan)。
本地直传用 `file:///D:/…` 形式(正斜杠、不转义中文)——图片/视频消息就是
这么发的,同机部署时能work。
name 可覆盖群文件列表里显示的名字(多图作品的成员是 001.jpg,调用方会补上
作品名前缀,免得多张图在群文件里全叫 001.jpg)。
"""
path = Path(file_path)
if not path.exists():
logger.warning(f"群文件上传:文件不存在 {path}")
return False
display_name = name or path.name
# 1) 局域网 S3 预签名链接(boto3 同步,丢线程池)
url = await asyncio.to_thread(_s3_url, path, policy)
if url:
try:
await _call_upload(url, group_id, display_name, folder_id)
except Exception as e:
logger.warning(
f"群文件 S3 链接上传失败(群 {group_id} / {display_name}): "
f"{str(e)[:160]};改用本地路径重试"
)
else:
logger.info(f"群文件上传成功(群 {group_id},S3 链接): {display_name}")
return True
else:
logger.warning("群文件:S3 中转没拿到链接,改用本地路径直传")
# 2) 兜底:file:/// 直传本地文件
try:
await _call_upload(_file_uri(path), group_id, display_name, folder_id)
except Exception as e:
logger.warning(
f"群文件上传失败(群 {group_id} / {display_name}): {str(e)[:160]}"
)
return False
logger.info(f"群文件上传成功(群 {group_id},本地直传): {display_name}")
return True
async def upload_group_files(
files: Iterable[Path | str],
group_id: int,
*,
title: str = "",
zip_files: bool = True,
password: str = "",
policy: Policy | None = None,
rel_dir: str = "",
) -> bool:
"""群文件投递入口:按配置打包(可加密)后传一个包,或逐个传原文件。
rel_dir 是源媒体所在的 `{作者}_{作者id}[/{作品名}]` 子目录,打包产物落到
`temp/archive/{rel_dir}`(由调用方从文件路径推出来,见 sender)。
"""
paths = [Path(f) for f in files]
if not paths:
return False
if not zip_files:
# 多图作品逐个传时,成员名是 001.jpg…,补上作品名前缀便于在群文件里辨认
work = Path(rel_dir).name if rel_dir else ""
prefix = f"{work}_" if work and len(paths) > 1 else ""
results = [
await upload_group_file(
p, group_id, policy=policy, name=f"{prefix}{p.name}"
)
for p in paths
]
return any(results)
try:
archive = await asyncio.to_thread(
build_archive, paths, title, password=password, rel_dir=rel_dir
)
except Exception as e:
logger.error(f"群文件打包失败,跳过本次群文件上传: {e}")
return False
return await upload_group_file(archive, group_id, policy=policy)
@@ -1,13 +1,15 @@
"""统一 S3 存储模块 — 合并本地局域网 S3 和公网 MinIO"""
from pathlib import Path
from time import strftime, localtime
import boto3
from botocore.config import Config
from nonebot import logger
from hexi.web_config import get_effective_value
from hexi.web_hub.web_config import get_effective_value
from ...policy import Policy
from ...utils import media_key_of
# 该插件模块名,用于读取统一配置值库(Web 修改后生效)
_PLUGIN_ID = "hexi.plugins.nonebot_plugin_video_analysis"
@@ -171,11 +173,13 @@ def upload_to_public_s3(file_path: str | Path) -> tuple[str, str]:
"""
上传到公网 MinIO,返回 (公网URL, file_key)
私聊场景使用,生成可公网访问的链接
私聊场景使用,生成可公网访问的链接。
key 按本地落盘结构推导:`{作者}_{作者id}/{作品名}[_{短码}].ext`
(不再用 `{年月日}/` 前缀,见 utils.media_key_of)。
"""
file_path = Path(file_path)
current_day = strftime("%Y-%m-%d", localtime())
file_key = f"{current_day}/{file_path.name}"
file_key = media_key_of(file_path)
try:
url = _get_public_s3().upload_public(str(file_path), file_key)
return url, file_key
@@ -184,22 +188,19 @@ def upload_to_public_s3(file_path: str | Path) -> tuple[str, str]:
return "", ""
def upload_to_local_s3(
title: str, image_post: bool, file_path: str | Path, plan: str | None = None
) -> str:
def upload_to_local_s3(file_path: str | Path, plan: str | None = None) -> str:
"""
上传到局域网 S3,返回预签名 URL
plan="A" → PLANA 桶
plan="B" → PLANB 桶
None → PLANC 桶(默认)
key 与公网一致:`{作者}_{作者id}/{作品名}[_{短码}].ext`
(不再有 `{年月日}/` 前缀,也不再重复套一层标题目录)。
"""
file_path = Path(file_path)
current_day = strftime("%Y-%m-%d", localtime())
if image_post:
file_key = f"{current_day}/{title}/{file_path.name}"
else:
file_key = f"{current_day}/{file_path.name}"
file_key = media_key_of(file_path)
if plan == "A":
client = _get_local_s3_plana()
@@ -224,35 +225,24 @@ def delete_from_public_s3(file_key: str) -> bool:
def upload_with_plan(
file_path: str | Path,
*,
plan: str | None = None,
is_private: bool = False,
title: str = "",
image_post: bool = False,
policy: Policy | None = None,
) -> tuple[str, str | None]:
"""
统一上传入口:按 plan 路由到对应的本地桶,并按需上传公网
统一上传入口:按策略路由本地桶,并按需上传公网
plan="A" → PLANA(仅本地)
plan="B" → PLANB + 公网
私聊 → PLANB + 公网
默认 → PLANC(仅本地)
policy.plan → PLANA / PLANB / PLANC(默认 C)
policy.upload_public → 是否额外上传公网(决定能否发下载链接)
key 由文件路径推导(utils.media_key_of),调用方不再传标题。
Returns:
(local_url, public_url_or_none)
"""
# 本地上传
if plan == "A":
local_url = upload_to_local_s3(title, image_post, file_path, plan="A")
elif plan == "B":
local_url = upload_to_local_s3(title, image_post, file_path, plan="B")
elif is_private:
local_url = upload_to_local_s3(title, image_post, file_path, plan="B")
else:
local_url = upload_to_local_s3(title, image_post, file_path) # PLANC
policy = policy or Policy()
local_url = upload_to_local_s3(file_path, plan=policy.plan)
# 公网上传(仅 PLANB 或私聊)
public_url = None
if plan == "B" or is_private:
if policy.upload_public:
public_url, _ = upload_to_public_s3(file_path)
return local_url, public_url
@@ -0,0 +1,508 @@
"""Web 管理台的「链接解析 + 预览」任务层(供 web_hub.py 调用)。
群消息链路(`handlers/entry.py::dispatch_url`)从解析到投递都绑在 event/bot 上,
Web 上下文里第一次 `UniMessage.send()` 就会抛 SerializeFailed,所以这里**只复用
纯函数层**:fetchers 的解析/下载 + `storage/s3.py::upload_with_plan`,
全程不发消息、不碰 event、不读群策略。
存储按**默认策略**走(`STORE.default_policy()` 的 plan / upload_public):
产出什么链接就返回什么链接 —— 开了 `upload_public` 才有公网链接,否则只有
局域网 S3 的预签名链接(1 小时过期,靠 `refresh()` 重传换新)。
任务表在内存里(进程重启即清空,前端按"查不到就算了"处理):
submit(urls, force=False) 新建任务;同 URL 已有 queued/running 任务时**复用**
list_jobs() 全部任务,新的在前
refresh(job_id) 对已落盘文件重跑上传,换一批新链接
抖音每次都新起一个 Chrome,所以并发闸门固定 `Semaphore(2)`;单任务 240s 超时兜底。
单文件上传失败只标该文件(`files[i].error`),不整体判失败 —— 多图作品挂一张
不该让整条任务失败,缺的那个文件点「刷新链接」还能补回来。
进度只有两档(解析中 → 上传中):解析和下载都在 fetcher 内部完成,中间没有可挂的钩子。
"""
from __future__ import annotations
import asyncio
import re
import subprocess
import time
import uuid
from collections.abc import Awaitable, Callable, Iterable
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING, Any, Optional
from nonebot import logger
if TYPE_CHECKING: # 仅类型检查:运行期不导入,单测可裸加载本模块
from ..policy import Policy
# ─────────────────────────── 常量 ───────────────────────────
#: URL 的终止符:空白 + 中文标点(全角括号/引号/破折号也要断)
URL_STOP = ",。!?、;:()【】《》「」『』…—“”‘’"
#: 从整段文本里挑 http(s) 链接。形制同 handlers/entry.py::URL_PATTERN,
#: 但把中文标点也当成终止符 —— `链接A,链接B` 中间没有空格也能各自成条
#: (`\S+` 会把它们连同后面的中文连成一个词,整段粘进来时更容易踩到)。
URL_PATTERN = re.compile(r"https?://[^" + re.escape(URL_STOP) + r"\s]+")
#: 兜底再洗一遍收尾标点(entry.py::dispatch_url 同款)
URL_TRAILING = URL_STOP
#: 任务表上限与终态任务的保留时长(秒)
MAX_JOBS = 100
TERMINAL_TTL = 30 * 60
#: 单任务超时(含下载,不含排队等闸门的时间)
JOB_TIMEOUT = 240
#: 并发闸门:抖音每次都新起 Chrome,必须限流
MAX_CONCURRENCY = 2
VIDEO_SUFFIXES = {".mp4", ".webm", ".mov", ".flv", ".mkv", ".ts", ".m4v"}
IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp", ".avif"}
#: 视频封面(关键帧):宽度压到 480 够卡片用,别把 4K 原帧传上云
POSTER_WIDTH = 480
POSTER_SUFFIX = "_封面.jpg"
POSTER_TIMEOUT = 20
#: 状态机:queued → running → done / failed
ACTIVE_STATUSES = ("queued", "running")
# ─────────────────────────── 纯函数 ───────────────────────────
def extract_urls(text: str) -> list[str]:
"""整段文本 → 去重后的 URL 列表(保序,去掉尾随中文标点)。
多行粘贴时一行一条;URL 后面跟的"。"这类标点不能进链接。
"""
urls: list[str] = []
for raw in URL_PATTERN.findall(text or ""):
url = raw.rstrip(URL_TRAILING)
if url and url not in urls:
urls.append(url)
return urls
def kind_of(path: Path | str) -> str:
"""文件类型(前端据此决定内联预览方式)。"""
suffix = Path(path).suffix.lower()
if suffix in VIDEO_SUFFIXES:
return "video"
if suffix in IMAGE_SUFFIXES:
return "image"
return "file"
# ─────────────────────────── 依赖注入点 ───────────────────────────
#: (标题, 落盘文件列表, 是否图文作品)
FetchResult = tuple[Optional[str], list[Path], bool]
Fetcher = Callable[[str], Awaitable[FetchResult]]
#: (局域网链接, 公网链接) —— 拿不到时返回 "" / None
Uploader = Callable[[Path, "Optional[Policy]"], Awaitable[tuple[str, Optional[str]]]]
#: 视频 → 封面图(抽关键帧);抽不出来返回 None
PosterMaker = Callable[[Path], Awaitable[Optional[Path]]]
def ffmpeg_path() -> str:
"""ffmpeg 可执行文件(解析逻辑复用 video_downloader;单测可替换)"""
from .fetchers.video_downloader import get_ffmpeg_path
return get_ffmpeg_path()
async def make_poster(video: Path) -> Optional[Path]:
"""抽一帧当视频封面,落同目录 `{作品名}_封面.jpg`;失败返回 None(不影响任务)。
先取第 1 秒(躲开黑场/淡入),整段不到 1 秒的视频退回第 0 秒再试一次。
"""
out = video.with_name(f"{video.stem}{POSTER_SUFFIX}")
for seek in ("1", "0"):
cmd = [
ffmpeg_path(),
"-ss",
seek,
"-i",
str(video),
"-frames:v",
"1",
"-vf",
f"scale={POSTER_WIDTH}:-2",
"-q:v",
"4",
"-y",
str(out),
]
try:
# 同步子进程必须丢线程池,否则抽帧期间整个事件循环都停摆
await asyncio.to_thread(
subprocess.run, cmd, capture_output=True, timeout=POSTER_TIMEOUT
)
except Exception: # noqa: BLE001 — 封面失败不该拖累任务
logger.warning(f"抽关键帧失败:{video.name}")
return None
if out.exists() and out.stat().st_size > 0:
return out
return None
def platform_of(url: str) -> Optional[str]:
"""URL → 平台规范标签(延迟导入 policy,便于单测裸加载本模块)。"""
from ..policy import match_platform
return match_platform(url)
def default_policy() -> "Optional[Policy]":
"""Web 解析统一按默认策略走(不是某个群的策略)。"""
from ..policy import STORE
return STORE.default_policy()
async def fetch_media(url: str) -> FetchResult:
"""按平台解析并下载媒体(纯函数层:不发消息、不碰 event、不上传)。
分派规则与 `handlers/entry.py::dispatch_url` 一致:抖音 → parse_douyin;
B站动态/专栏 → fetch_bilibili_content;其余 B站链接 → download_video(yt-dlp);
小红书 → fetch_rednote_content;其它平台 → download_video。
b23 / xhslink / 抖音短链的重定向在各 fetcher 内部自己处理。
"""
from ..handlers.douyin import parse_douyin
from ..policy import BILIBILI, DOUYIN, XHS
from .fetchers.bilibili_content import fetch_bilibili_content
from .fetchers.rednote_content import fetch_rednote_content
from .fetchers.video_downloader import download_video
platform = platform_of(url)
title: Optional[str] = None
parsed: Path | list[Path] | None = None
image_post = False
if platform == DOUYIN:
title, parsed, image_post = await parse_douyin(url)
elif platform == XHS:
title, parsed = await fetch_rednote_content(url)
image_post = isinstance(parsed, list)
elif platform == BILIBILI and any(
kw in url
for kw in (
"bilibili.com/opus",
"bilibili.com/dynamic",
"t.bilibili.com",
"bilibili.com/read",
)
):
title, parsed = await fetch_bilibili_content(url)
image_post = isinstance(parsed, list)
else:
video_file, title = await download_video(url)
parsed = video_file
if isinstance(parsed, list):
files = [Path(p) for p in parsed]
else:
files = [Path(parsed)] if parsed else []
return title, files, image_post
async def upload_file(
path: Path, policy: "Optional[Policy]"
) -> tuple[str, Optional[str]]:
"""上传单个文件 → (局域网链接, 公网链接)。
boto3 是同步的,必须丢线程池:直接在协程里跑会卡死整个事件循环
(Web 轮询、群消息全都停摆)。
"""
from .storage.s3 import upload_with_plan
return await asyncio.to_thread(upload_with_plan, path, policy=policy)
# ─────────────────────────── 任务模型 ───────────────────────────
@dataclass
class WebJob:
"""一条解析任务;`to_dict()` 就是 API 的返回体。"""
id: str
url: str
platform: Optional[str] = None
title: str = ""
status: str = "queued"
#: 运行中的进度文案:解析中 / 上传中(见模块文档:解析与下载不细分)
stage: str = ""
#: 每个文件:{name, kind, size, local_url, public_url, error}
files: list[dict[str, Any]] = field(default_factory=list)
#: 卡片封面:有图取第一张图,全是视频则抽第一帧(见 `_resolve_cover`)
cover_url: str = ""
error: str = ""
created_at: float = field(default_factory=time.time)
updated_at: float = field(default_factory=time.time)
#: 落盘的源文件(不出 API;`refresh()` 重传要用)
paths: list[Path] = field(default_factory=list, repr=False)
#: 抽出来的封面文件(不出 API;`refresh()` 重传要用)
poster_path: Optional[Path] = field(default=None, repr=False)
@property
def active(self) -> bool:
"""还在排队或运行(终态任务才会被 prune 掉)。"""
return self.status in ACTIVE_STATUSES
def to_dict(self) -> dict[str, Any]:
return {
"id": self.id,
"url": self.url,
"platform": self.platform,
"title": self.title,
"status": self.status,
"stage": self.stage,
"files": self.files,
"cover_url": self.cover_url,
"error": self.error,
"created_at": self.created_at,
"updated_at": self.updated_at,
}
class JobManager:
"""内存任务表:上限 MAX_JOBS 条,终态任务 TTL 30 分钟后自动清掉。
fetch / upload / poster / policy 四个依赖可注入(单测塞假实现,不碰网络与磁盘)。
"""
def __init__(
self,
*,
fetch: Optional[Fetcher] = None,
upload: Optional[Uploader] = None,
poster: Optional[PosterMaker] = None,
policy: Optional[Callable[[], "Optional[Policy]"]] = None,
) -> None:
self._jobs: dict[str, WebJob] = {}
self._tasks: dict[str, asyncio.Task[None]] = {}
self._sem: Optional[asyncio.Semaphore] = None
self._fetch: Fetcher = fetch or fetch_media
self._upload: Uploader = upload or upload_file
self._poster: PosterMaker = poster or make_poster
self._policy: Callable[[], "Optional[Policy]"] = policy or default_policy
# ── 查询 ──────────────────────────────────────────────
def list_jobs(self) -> list[WebJob]:
"""全部任务,新的在前。"""
self._prune()
return sorted(self._jobs.values(), key=lambda j: j.created_at, reverse=True)
def get(self, job_id: str) -> Optional[WebJob]:
return self._jobs.get(job_id)
def _find_active(self, url: str) -> Optional[WebJob]:
for job in self._jobs.values():
if job.url == url and job.active:
return job
return None
# ── 提交与执行 ────────────────────────────────────────
async def submit(self, urls: Iterable[str], *, force: bool = False) -> list[WebJob]:
"""提交一批 URL;同 URL 已有在跑的任务时直接复用(force=True 强行新建)。
复用不只是省一次解析:抖音双开浏览器毫无意义,而且两个任务同时往
`unique_media_path` 的同一个路径写会撞车(它是 exists → 改名的写法)。
"""
self._prune()
jobs: list[WebJob] = []
for url in urls:
job = None if force else self._find_active(url)
if job is None:
job = WebJob(
id=uuid.uuid4().hex[:12], url=url, platform=platform_of(url)
)
self._jobs[job.id] = job
self._spawn(job)
jobs.append(job)
return jobs
async def refresh(self, job_id: str) -> Optional[WebJob]:
"""对已落盘文件重跑上传,换一批新的预签名链接(旧的 1 小时过期)。
任务不存在 / 还没有文件时返回 None,由调用方给提示。
"""
job = self._jobs.get(job_id)
if job is None or not job.files:
return None
policy = self._policy()
job.files = [await self._upload_one(path, policy) for path in job.paths]
job.cover_url = await self._resolve_cover(job, policy)
job.updated_at = time.time()
return job
# ── 清理 ──────────────────────────────────────────────
def remove(self, job_id: str) -> bool:
"""删掉一条任务;还在跑的一并取消(Chrome 那头由 playwright 自己收尾)。"""
job = self._jobs.pop(job_id, None)
if job is None:
return False
task = self._tasks.pop(job_id, None)
if task is not None and not task.done():
task.cancel()
return True
def clear_finished(self) -> int:
"""清掉所有终态任务(排队/运行中的不动),返回清掉几条。"""
finished = [jid for jid, job in self._jobs.items() if not job.active]
for job_id in finished:
self._jobs.pop(job_id, None)
return len(finished)
def _spawn(self, job: WebJob) -> None:
task = asyncio.create_task(self._run(job))
self._tasks[job.id] = task
task.add_done_callback(lambda _t: self._tasks.pop(job.id, None))
async def _run(self, job: WebJob) -> None:
"""排队等闸门 → 执行;任何异常都落到任务状态里,不外抛。"""
try:
async with self._gate():
await asyncio.wait_for(self._execute(job), JOB_TIMEOUT)
except asyncio.CancelledError:
raise
except asyncio.TimeoutError: # 3.11+ 就是内置 TimeoutError,wait_for 抛的
self._update(job, status="failed", error=f"解析超时(超过 {JOB_TIMEOUT}s)")
except Exception as e: # noqa: BLE001 — 任务边界,失败即任务状态
logger.exception(f"Web 解析任务失败:{job.url}")
self._update(job, status="failed", error=str(e) or type(e).__name__)
async def _execute(self, job: WebJob) -> None:
self._update(job, status="running", stage="解析中")
policy = self._policy()
title, files, _ = await self._fetch(job.url)
job.title = title or job.url
if not files:
raise RuntimeError("无法解析到媒体(链接失效 / 风控 / cookies 过期)")
job.paths = list(files)
self._update(job, stage="上传中")
entries: list[dict[str, Any]] = []
for path in files:
entries.append(await self._upload_one(path, policy))
job.files = entries # 落一个刷一个,前端能看着进度出图
job.cover_url = await self._resolve_cover(job, policy)
self._update(job, status="done", stage="")
async def _resolve_cover(self, job: WebJob, policy: "Optional[Policy]") -> str:
"""任务封面:有图就用第一张图;全是视频就抽第一帧上传当封面。
抽帧/上传失败都只返回空串 —— 卡片那边退化成占位块,不影响任务本身。
"""
for entry in job.files:
if entry["kind"] == "image":
url = entry["public_url"] or entry["local_url"]
if url:
return url
if job.poster_path and job.poster_path.exists():
entry = await self._upload_one(job.poster_path, policy)
return entry["public_url"] or entry["local_url"]
for path in job.paths:
if kind_of(path) != "video" or not path.exists():
continue
poster = await self._poster(path)
if poster is None:
return ""
job.poster_path = poster
entry = await self._upload_one(poster, policy)
return entry["public_url"] or entry["local_url"]
return ""
async def _upload_one(
self, path: Path, policy: "Optional[Policy]"
) -> dict[str, Any]:
"""上传单个文件 → files 里的一项;失败只标这项。"""
if not path.exists():
return _file_entry(path, error="本地文件不存在(可能已被 temp 清理)")
try:
local_url, public_url = await self._upload(path, policy)
except Exception as e: # noqa: BLE001 — 单文件失败不拖累整条任务
logger.exception(f"Web 任务上传失败:{path}")
return _file_entry(path, error=f"上传失败:{str(e)[:120]}")
entry = _file_entry(path, local_url=local_url, public_url=public_url)
if not entry["local_url"] and not entry["public_url"]:
entry["error"] = "上传失败(S3 没返回链接)"
return entry
# ── 内部工具 ──────────────────────────────────────────
def _gate(self) -> asyncio.Semaphore:
"""惰性建闸门:构造必须发生在跑着的事件循环里。"""
if self._sem is None:
self._sem = asyncio.Semaphore(MAX_CONCURRENCY)
return self._sem
def _update(
self,
job: WebJob,
*,
status: Optional[str] = None,
stage: Optional[str] = None,
error: Optional[str] = None,
) -> None:
if status is not None:
job.status = status
if stage is not None:
job.stage = stage
if error is not None:
job.error = error
job.updated_at = time.time()
def _prune(self) -> None:
"""先清超龄的终态任务,再按上限砍掉最旧的终态任务(在跑的不动)。"""
now = time.time()
for job_id, job in list(self._jobs.items()):
if not job.active and now - job.updated_at > TERMINAL_TTL:
self._jobs.pop(job_id, None)
overflow = len(self._jobs) - MAX_JOBS
if overflow <= 0:
return
finished = sorted(
(j for j in self._jobs.values() if not j.active),
key=lambda j: j.created_at,
)
for job in finished[:overflow]:
self._jobs.pop(job.id, None)
def _file_entry(
path: Path,
*,
local_url: str = "",
public_url: Optional[str] = None,
error: str = "",
) -> dict[str, Any]:
try:
size = path.stat().st_size
except OSError:
size = 0
return {
"name": path.name,
"kind": kind_of(path),
"size": size,
"local_url": local_url or "",
"public_url": public_url or "",
"error": error,
}
#: 全局单例(web_hub.py 的路由共用)
JOBS = JobManager()
@@ -1,8 +1,9 @@
import hashlib
import re
import time
import unicodedata
from pathlib import Path
from time import strftime, localtime
from typing import List
from typing import List, Optional, Tuple
def get_data_dir() -> Path:
@@ -72,6 +73,9 @@ def ensure_unique_path(base_path: Path) -> Path:
"""
确保路径不冲突:如已存在则追加 _2, _3... 后缀
适用于文件和目录
注:媒体落盘已统一走 `unique_media_path`(同名加 4 位短码),本函数仅作
通用兜底保留。
"""
if not base_path.exists():
return base_path
@@ -88,52 +92,139 @@ def ensure_unique_path(base_path: Path) -> Path:
counter += 1
def clean_filename(filename: str, max_length: int = 120) -> str:
_B36_ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyz"
#: 同名短码的模数 —— base36 四位(36**4 ≈ 168 万秒 ≈ 19.4 天一轮)
_B36_MOD = 36**4
def _to_base36(value: int, width: int = 4) -> str:
"""整数 → 定宽 base36(不足左侧补 0)"""
if value <= 0:
return "0" * width
digits = []
while value:
value, rem = divmod(value, 36)
digits.append(_B36_ALPHABET[rem])
return "".join(reversed(digits)).rjust(width, "0")
def short_time_code(at: Optional[float] = None) -> str:
"""4 位 base36 短码(同名兜底用,不可读时间,仅作区分码)"""
return _to_base36(int(time.time() if at is None else at) % _B36_MOD)
#: 各平台把"没有 id"写成过这些值,别让它们进目录名
_JUNK_AUTHOR_IDS = {"", "0", "none", "null", "na", "nan", "undefined"}
#: 抓取层拿不到昵称时的兜底串(slugify 后)—— 它们等于"没有作者信息"
_PLACEHOLDER_AUTHORS = {"未知作者", "小红书用户", "b站用户"}
def _clean_author_id(author_id: Optional[str]) -> str:
"""作者 id 归一:剔除占位值(yt-dlp 缺字段常给 `NA`),再 slugify。"""
raw = str(author_id or "").strip()
if raw.lower() in _JUNK_AUTHOR_IDS:
return ""
return slugify(raw, max_length=40)
def short_source_code(source: str) -> str:
"""来源串(作品 URL/id)→ 4 位 base36 短码。
与 `short_time_code` 的区别:同一个来源永远得到同一个码(md5 取摘要,
不能用内置 `hash()` —— 它有随机盐,重启后目录名会变)。
"""
清理文件名并添加时间前缀
支持多扩展名,如 .tar.gz
digest = hashlib.md5(source.encode("utf-8")).digest()
return _to_base36(int.from_bytes(digest[:4], "big") % _B36_MOD)
def build_author_dir(
author: Optional[str],
author_id: Optional[str] = None,
*,
source: Optional[str] = None,
at: Optional[float] = None,
) -> str:
"""`{作者}_{作者id}` 作者目录名(拿不到 id 时追加码值避免同名混目录)。
- 昵称 + 作者 id → `{昵称}_{id}`(昵称上限 30、id 上限 40;**不能**把 id 截到
20:`MS4wLjABAAAA…` 这类 sec_uid 公共前缀就有 17 字符,再截断必撞车)
- 只有昵称 → `{昵称}_{4 位时间短码}`:没有 id 就分不清同名作者,宁可不聚合
(同一作者的不同作品会各成一个目录)也不能把两个人混进同一个目录
- 连昵称都没有 → `未知作者_{来源短码}`(`source` 给作品 URL/id,同一来源
稳定、不同来源不撞);连 source 都没有 → 退化成时间短码
"""
slug_author = slugify(author or "", max_length=30)
if slug_author in _PLACEHOLDER_AUTHORS:
slug_author = ""
slug_id = _clean_author_id(author_id)
current_time = strftime("%H-%M-%S", localtime())
if slug_author and slug_id:
# 某些站点上传者名就是 handle(X 的 @someone),别产出 someone_someone
if slug_author == slug_id:
return slug_author
return f"{slug_author}_{slug_id}"
if slug_id:
return f"{slug_author or '未知作者'}_{slug_id}"
if slug_author:
return f"{slug_author}_{short_time_code(at)}"
if source:
return f"未知作者_{short_source_code(source)}"
return f"未知作者_{short_time_code(at)}"
p = Path(filename)
# 主文件名
name = p.stem
def build_work_stem(title: Optional[str]) -> str:
"""作品名做文件名/子目录名:slugify(沿用 15 字上限),空则 `作品`"""
return slugify(title or "", max_length=15) or "作品"
# 完整扩展名 (.tar.gz)
ext = "".join(p.suffixes)
# Unicode 标准化
name = unicodedata.normalize("NFKC", name)
def unique_media_path(path: Path, *, at: Optional[float] = None) -> Path:
"""同名才加 4 位短码:`{stem}_{码}{后缀}`,仍撞则再叠 `_2/_3…`
# 去掉 #tag
name = re.sub(r"#\S+", "", name)
文件与目录通用(目录无后缀)。命名发生在落盘前,因此"不存在"即直接采用;
顺带确保父目录存在(作者目录是按需创建的)。
"""
if path.exists():
code = short_time_code(at)
candidate = path.with_name(f"{path.stem}_{code}{path.suffix}")
index = 2
while candidate.exists():
candidate = path.with_name(f"{path.stem}_{code}_{index}{path.suffix}")
index += 1
path = candidate
# 非法字符替换
name = re.sub(r'[\\/:*?"<>|]', "_", name)
path.parent.mkdir(parents=True, exist_ok=True)
return path
# 中英文标点
name = re.sub(r"[&'\"。,:?!《》【】|]", "_", name)
# 空白 -> _
name = re.sub(r"\s+", "_", name)
def _temp_rel_parts(file_path: Path | str) -> Tuple[str, ...]:
"""相对 temp 根拆路径:`{平台}/{作者目录}/…`(归档目录同理)
# 只保留:中文、字母、数字、_
name = re.sub(r"[^\w一-鿿_]", "", name)
不在 temp 下、或没到"平台 + 作者目录"这一层(老数据 / 第三方产物)→ 空元组,
调用方退化为只用文件名。
"""
try:
rel = Path(file_path).resolve().relative_to(get_temp_root().resolve())
except (ValueError, OSError):
return ()
parts = rel.parts
return parts[1:] if len(parts) >= 3 else ()
# 合并 _
name = re.sub(r"_+", "_", name)
# 去首尾 _
name = name.strip("_")
def media_key_of(file_path: Path | str) -> str:
"""媒体文件的 S3 对象 key:`{作者}_{作者id}/{作品名}[_{码}].后缀`
# 长度控制
max_name_length = max_length - len(ext) - len(current_time) - 1
if len(name) > max_name_length:
name = name[:max_name_length].rstrip("_")
由本地路径反推(去掉平台层),保证桶里和 temp 里结构一致。
"""
parts = _temp_rel_parts(file_path)
return "/".join(parts) if parts else Path(file_path).name
return f"{current_time}_{name}{ext}"
def media_rel_dir_of(file_path: Path | str) -> str:
"""媒体文件所在的作者/作品子目录(相对 temp 根、去掉平台层),供归档复用"""
parts = _temp_rel_parts(file_path)
return "/".join(parts[:-1]) if len(parts) > 1 else ""
def parse_netscape_cookies(file_path: str) -> List[dict]:
@@ -0,0 +1,176 @@
"""视频解析 Web API 子应用(挂载到 /api/video_analysis)。
群策略(data/list.json v3,见 policy.py)的唯一 Web 读写入口,
鉴权走 hexi.web_hub.web_auth(OAuth2 + SQLite),与统一管理台 /hub 共用登录态。
前端页面:hexi/web/src/plugins/video_analysis/index.tsx。
写入全部落在 `policy.PolicyStore` 上(加锁 + 原子替换 + 归一化),
所以这里不需要再做字段校验,只要校验群号形态。
另有一组「链接解析 + 预览」接口(/parse、/jobs、/jobs/{id}/refresh):
粘链接 → 起任务 → 出下载链接 + 页面内预览,实现在 services/web_jobs.py。
**每条路由都要自带 `dependencies=[auth]`** —— mount 层没有兜底,
漏一条就是匿名可访问(包括这条"让服务器去抓任意 URL"的接口)。
"""
from __future__ import annotations
from fastapi import FastAPI
from fastapi.responses import JSONResponse
from nonebot import get_adapter
from nonebot.adapters.onebot.v11 import Adapter
from hexi.web_hub.web_auth import require_admin
from .policy import PLANS, PLATFORMS, STORE, Policy
from .services.web_jobs import JOBS, extract_urls
API = require_admin
def _ok(data=None, msg: str = "ok") -> JSONResponse:
return JSONResponse({"status": 0, "msg": msg, "data": data})
def _fail(msg: str, status: int = 400) -> JSONResponse:
return JSONResponse({"status": status, "msg": msg})
async def _group_names() -> dict[str, str]:
"""群号 → 群名;拿不到 bot(未连接)时返回空表,前端只显示群号。"""
try:
bots = get_adapter(Adapter).bots
bot = next(iter(bots.values()), None)
if bot is None:
return {}
return {
str(g["group_id"]): g.get("group_name") or ""
for g in await bot.get_group_list()
}
except Exception: # noqa: BLE001 — 未连接/适配器未加载都按拿不到处理
return {}
def _payload(policy: Policy) -> dict:
"""策略的 JSON 视图;sends_link 是"实际会不会发链接"(供前端置灰)。"""
data = policy.to_dict()
data["sends_link"] = policy.sends_link
return data
def build_admin_app() -> FastAPI | None:
"""构建群策略管理 API 子应用(挂载到 /api/video_analysis)。"""
app = FastAPI(title="Video Analysis API")
auth = require_admin
@app.get("/overview", response_class=JSONResponse, dependencies=[auth])
async def overview():
"""一次拿全:群策略列表 + 默认节 + 黑名单 + 平台/方案选项。"""
names = await _group_names()
groups = [
{
"group_id": gid,
"group_name": names.get(gid, ""),
"online": gid in names,
"policy": _payload(policy),
}
for gid, policy in sorted(STORE.all_groups().items(), key=_sort_key)
]
return _ok(
{
"groups": groups,
"default": _payload(STORE.default_policy()),
"blacklist": STORE.blacklist(),
"platforms": list(PLATFORMS),
"plans": list(PLANS),
"online": bool(names),
}
)
@app.post("/group", response_class=JSONResponse, dependencies=[auth])
async def save_group(data: dict):
"""新增/覆盖单个群的策略(群不存在则加入白名单)。"""
gid = str(data.get("group_id", "")).strip()
if not gid.isdigit():
return _fail("群号必须是纯数字")
policy = await STORE.set_group(gid, Policy.from_dict(data.get("policy")))
return _ok(_payload(policy), f"群 {gid} 策略已保存")
@app.delete("/group/{group_id}", response_class=JSONResponse, dependencies=[auth])
async def remove_group(group_id: str):
"""移出白名单(该群不再解析)。"""
if not await STORE.remove_group(group_id):
return _fail(f"群 {group_id} 不在白名单中")
return _ok({"group_id": group_id}, f"群 {group_id} 已移出白名单")
@app.post("/default", response_class=JSONResponse, dependencies=[auth])
async def save_default(data: dict):
"""保存默认策略(私聊与未配置群使用)。"""
policy = await STORE.set_default(Policy.from_dict(data.get("policy", data)))
return _ok(_payload(policy), "默认策略已保存")
@app.post("/blacklist", response_class=JSONResponse, dependencies=[auth])
async def save_blacklist(data: dict):
"""整体替换全局黑名单(QQ 列表)。"""
raw = data.get("blacklist", [])
if not isinstance(raw, list):
return _fail("blacklist 必须是列表")
values = [str(x).strip() for x in raw if str(x).strip()]
invalid = [v for v in values if not v.isdigit()]
if invalid:
return _fail(f"黑名单只能是 QQ 号:{'、'.join(invalid)}")
stored = await STORE.set_blacklist(values)
return _ok({"blacklist": stored}, "黑名单已保存")
# ── 链接解析 + 预览(services/web_jobs.py) ──────────────
# 与群策略无关:解析结果按默认策略(default 节)的 plan / upload_public 存储。
@app.post("/parse", response_class=JSONResponse, dependencies=[auth])
async def parse(data: dict):
"""整段文本里挑链接 → 逐个起任务;同 URL 已有在跑的任务时直接复用。"""
urls = extract_urls(str(data.get("text") or ""))
if not urls:
return _fail("没找到链接")
jobs = await JOBS.submit(urls, force=bool(data.get("force")))
return _ok(
{"jobs": [j.to_dict() for j in jobs], "urls": urls},
f"已提交 {len(jobs)} 个任务",
)
@app.get("/jobs", response_class=JSONResponse, dependencies=[auth])
async def list_jobs():
"""全部任务,新的在前(前端按 1.5s 轮询这一个接口)。"""
return _ok({"jobs": [j.to_dict() for j in JOBS.list_jobs()]})
@app.post(
"/jobs/{job_id}/refresh", response_class=JSONResponse, dependencies=[auth]
)
async def refresh_job(job_id: str):
"""重传已落盘文件换一批新链接(预签名链接 1 小时过期)。"""
job = await JOBS.refresh(job_id)
if job is None:
return _fail("任务不存在,或还没有可刷新的文件")
return _ok({"job": job.to_dict()}, "链接已刷新")
@app.delete("/jobs/{job_id}", response_class=JSONResponse, dependencies=[auth])
async def remove_job(job_id: str):
"""删掉一条任务(还在跑的一并取消);只清任务表,temp 里的文件交给清理任务。"""
if not JOBS.remove(job_id):
return _fail("任务不存在")
return _ok({"job_id": job_id}, "任务已清理")
@app.post("/jobs/clear", response_class=JSONResponse, dependencies=[auth])
async def clear_jobs():
"""清掉所有已完成/失败的任务(排队、运行中的不动)。"""
removed = JOBS.clear_finished()
return _ok({"removed": removed}, f"已清理 {removed} 条任务")
return app
def _sort_key(item: tuple[str, Policy]) -> tuple[int, int, str]:
"""群号按数值排序(非数字的排最后,不参与数值比较)。"""
gid = item[0]
return (1, 0, gid) if not gid.isdigit() else (0, int(gid), "")
Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

-4
View File
@@ -1,4 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="-54 -44.52999973297119 216.1999969482422 143.78999710083008" data-asc="0.9052734375" width="216.1999969482422" height="143.78999710083008"><defs><filter id="toxic-glow-filter" x="-54" y="-44.52999973297119" width="216.1999969482422" height="143.78999710083008" filterUnits="userSpaceOnUse">
<feDropShadow dx="0" dy="0" stdDeviation="6" flood-color="#acdba4" flood-opacity="0.9"/>
<feDropShadow dx="0" dy="0" stdDeviation="18" flood-color="#a294b8" flood-opacity="0.6"/>
</filter></defs><g fill="#0f0f0f" stroke="#686a68" stroke-width="1.5" filter="url(#toxic-glow-filter)"><g transform="translate(0, 0)"><path d="M0 45.26L0 9.47L4.74 9.47L4.74 24.17L23.34 24.17L23.34 9.47L28.08 9.47L28.08 45.26L23.34 45.26L23.34 28.39L4.74 28.39L4.74 45.26L0 45.26ZM36.06 45.26L36.06 9.47L61.94 9.47L61.94 13.70L40.80 13.70L40.80 24.66L60.60 24.66L60.60 28.86L40.80 28.86L40.80 41.04L62.77 41.04L62.77 45.26L36.06 45.26ZM65.67 45.26L79.52 26.61L67.31 9.47L72.95 9.47L79.44 18.65Q81.47 21.51 82.32 23.05Q83.52 21.09 85.16 18.97L92.36 9.47L97.51 9.47L84.94 26.34L98.49 45.26L92.63 45.26L83.62 32.50Q82.86 31.40 82.06 30.10Q80.86 32.06 80.35 32.79L71.36 45.26L65.67 45.26ZM103.47 45.26L103.47 9.47L108.20 9.47L108.20 45.26L103.47 45.26Z"/></g></g></svg>

Before

Width:  |  Height:  |  Size: 1.3 KiB

+1 -1
View File
@@ -3,7 +3,7 @@
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<link rel="icon" type="image/svg+xml" href="/hub/favicon.svg" />
<link rel="icon" type="image/svg+xml" href="/hub/favicon.png" />
<title>HeXi · 统一管理台</title>
</head>
<body>
+40 -11
View File
@@ -13,7 +13,8 @@
"@heroui/styles": "^3.2.4",
"react": "^19.2.8",
"react-dom": "^19.2.8",
"react-router-dom": "^7.18.3"
"react-router-dom": "^7.18.3",
"yet-another-react-lightbox": "^3.32.2"
},
"devDependencies": {
"@tailwindcss/vite": "^4.3.3",
@@ -30,6 +31,7 @@
"resolved": "https://registry.npmjs.org/@adobe/react-spectrum/-/react-spectrum-3.47.3.tgz",
"integrity": "sha512-tWZG59+xTbXIqeZB4uyuo2qljk3nRK66IkdBmMgbl2p6peApkKbrtJ/YOM8rW4gemdmv7qoJlbIHvy8aJkYsCQ==",
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"@internationalized/date": "^3.12.3",
"@react-types/shared": "^3.36.1",
@@ -83,7 +85,6 @@
"resolved": "https://registry.npmjs.org/@formatjs/ecma402-abstract/-/ecma402-abstract-2.3.6.tgz",
"integrity": "sha512-HJnTFeRM2kVFVr5gr5kH1XP6K0JcJtE7Lzvtr3FS/so5f1kpsqqqxy5JF+FRaO6H2qmcMfAUIox7AJteieRtVw==",
"license": "MIT",
"peer": true,
"dependencies": {
"@formatjs/fast-memoize": "2.2.7",
"@formatjs/intl-localematcher": "0.6.2",
@@ -96,7 +97,6 @@
"resolved": "https://registry.npmjs.org/@formatjs/fast-memoize/-/fast-memoize-2.2.7.tgz",
"integrity": "sha512-Yabmi9nSvyOMrlSeGGWDiH7rf3a7sIwplbvo/dlz9WCIjzIQAfy1RMf4S0X3yG724n5Ghu2GmEl5NJIV6O9sZQ==",
"license": "MIT",
"peer": true,
"dependencies": {
"tslib": "^2.8.0"
}
@@ -106,7 +106,6 @@
"resolved": "https://registry.npmjs.org/@formatjs/icu-messageformat-parser/-/icu-messageformat-parser-2.11.4.tgz",
"integrity": "sha512-7kR78cRrPNB4fjGFZg3Rmj5aah8rQj9KPzuLsmcSn4ipLXQvC04keycTI1F7kJYDwIXtT2+7IDEto842CfZBtw==",
"license": "MIT",
"peer": true,
"dependencies": {
"@formatjs/ecma402-abstract": "2.3.6",
"@formatjs/icu-skeleton-parser": "1.8.16",
@@ -118,7 +117,6 @@
"resolved": "https://registry.npmjs.org/@formatjs/icu-skeleton-parser/-/icu-skeleton-parser-1.8.16.tgz",
"integrity": "sha512-H13E9Xl+PxBd8D5/6TVUluSpxGNvFSlN/b3coUp0e0JpuWXXnQDiavIpY3NnvSp4xhEMoXyyBvVfdFX8jglOHQ==",
"license": "MIT",
"peer": true,
"dependencies": {
"@formatjs/ecma402-abstract": "2.3.6",
"tslib": "^2.8.0"
@@ -129,7 +127,6 @@
"resolved": "https://registry.npmjs.org/@formatjs/intl-localematcher/-/intl-localematcher-0.6.2.tgz",
"integrity": "sha512-XOMO2Hupl0wdd172Y06h6kLpBz6Dv+J4okPLl4LPtzbr8f66WbIoy4ev98EBuZ6ZK4h5ydTN6XneT4QVpD7cdA==",
"license": "MIT",
"peer": true,
"dependencies": {
"tslib": "^2.8.0"
}
@@ -204,7 +201,6 @@
"resolved": "https://registry.npmjs.org/@internationalized/message/-/message-3.1.10.tgz",
"integrity": "sha512-nc0Or6EdWHqZRcsXb6P9hBIpLsfSl/ILh0rk5h/OVBpzmhdExXtPy2cQtWsq8XKRBpRHwDNnAHt4OpolcB7dog==",
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"@swc/helpers": "^0.5.0",
"intl-messageformat": "^10.1.0"
@@ -1163,6 +1159,7 @@
"integrity": "sha512-AnzbBERsrLKtk2XSfTbYRLjQPdy116Sty4q+T+Bp3IC4l6jNBvreVPAHmpq9qhXQM7CXZPjLVmGMw9sy+hxQ3w==",
"devOptional": true,
"license": "MIT",
"peer": true,
"dependencies": {
"csstype": "^3.2.2"
}
@@ -1173,6 +1170,7 @@
"integrity": "sha512-fMPwH9v7r/pp43yUd2/Mbiex5KouJwwR3dzHkhLREUC6764VyDsqxhAxv6OFEYR1RhjOyD1naqba8ECDBe7ZQg==",
"devOptional": true,
"license": "MIT",
"peer": true,
"peerDependencies": {
"@types/react": "^19.2.0"
}
@@ -1257,8 +1255,7 @@
"version": "10.6.0",
"resolved": "https://registry.npmjs.org/decimal.js/-/decimal.js-10.6.0.tgz",
"integrity": "sha512-YpgQiITW3JXGntzdUmyUR1V812Hn8T1YVXhCu+wO3OpS4eU9l4YdD3qjyiKdV6mvV29zapkMeD390UVEf2lkUg==",
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/detect-libc": {
"version": "2.1.2",
@@ -1349,7 +1346,6 @@
"resolved": "https://registry.npmjs.org/intl-messageformat/-/intl-messageformat-10.7.18.tgz",
"integrity": "sha512-m3Ofv/X/tV8Y3tHXLohcuVuhWKo7BBq62cqY15etqmLxg2DZ34AGGgQDeR+SCta2+zICb1NX83af0GJmbQ1++g==",
"license": "BSD-3-Clause",
"peer": true,
"dependencies": {
"@formatjs/ecma402-abstract": "2.3.6",
"@formatjs/fast-memoize": "2.2.7",
@@ -1697,6 +1693,7 @@
"integrity": "sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=12"
},
@@ -1749,6 +1746,7 @@
"resolved": "https://registry.npmjs.org/react/-/react-19.2.8.tgz",
"integrity": "sha512-PWaYA1L/q9u2u7xYQi+Y3L3Yfnie7XyLeaJICV1MGD6LprsBxcAqGjYyr0eY3p+QdsA+x/Irkt4Qif8D63+Sbw==",
"license": "MIT",
"peer": true,
"engines": {
"node": ">=0.10.0"
}
@@ -1758,6 +1756,7 @@
"resolved": "https://registry.npmjs.org/react-aria/-/react-aria-3.51.0.tgz",
"integrity": "sha512-AyWLw0XR38cFPwBu/ErgGaVrc5dupLEKmRlMXTGvFKOtbaGRQ2+yQJkjVhpdHhoRhU4+G+tJDFeHDTS8tK3bfQ==",
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"@internationalized/date": "^3.12.3",
"@internationalized/number": "^3.6.7",
@@ -1779,6 +1778,7 @@
"resolved": "https://registry.npmjs.org/react-aria-components/-/react-aria-components-1.20.0.tgz",
"integrity": "sha512-BMbpIgoV9aELeBrB0Y120NgoigHb5OdcJwc+4e7uSnbTbamea6lo+gqcc4LAxzMaK3Jf+7LI1oCDE6yANsmxIQ==",
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"@internationalized/date": "^3.12.3",
"@internationalized/string": "^3.2.10",
@@ -1798,6 +1798,7 @@
"resolved": "https://registry.npmjs.org/react-dom/-/react-dom-19.2.8.tgz",
"integrity": "sha512-rVprimfGBG3DR+Tq0IQG2DT5PxKth1WIGDmj5yPmlzr4YBe7uyE+Du4oVqTDXZSHGGGXRtTJEGSSePyQCMBglQ==",
"license": "MIT",
"peer": true,
"dependencies": {
"scheduler": "^0.27.0"
},
@@ -1974,7 +1975,8 @@
"version": "4.3.3",
"resolved": "https://registry.npmjs.org/tailwindcss/-/tailwindcss-4.3.3.tgz",
"integrity": "sha512-gOhV3P7ufE62QDGg1zVaTgCR+EtPv92k2nIhVcVKcLmxT1sUBsQGhnZj175j+MqRt4zLF7ic+sCYjfhxMxj7YQ==",
"license": "MIT"
"license": "MIT",
"peer": true
},
"node_modules/tapable": {
"version": "2.3.3",
@@ -2051,6 +2053,7 @@
"integrity": "sha512-cFKLV/PRgAUlIRm5WjMjJ86jrftzpqcgH+Us+DS8mI3CDNiH30Whrz8uHL3+MOLPAgqbMBAqWdAHAphOAM+z/Q==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"lightningcss": "^1.33.0",
"picomatch": "^4.0.5",
@@ -2383,6 +2386,32 @@
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/yet-another-react-lightbox": {
"version": "3.32.2",
"resolved": "https://registry.npmjs.org/yet-another-react-lightbox/-/yet-another-react-lightbox-3.32.2.tgz",
"integrity": "sha512-F4HtHQfUNpvkj+AmECgWM4XRdCqMY5gXpKgOUx39+T+FyxLe8II4SK/pwMyYj2X54KH9lFSQeHYY1/GfYf3SdA==",
"license": "MIT",
"engines": {
"node": ">=14"
},
"funding": {
"url": "https://github.com/sponsors/igordanchenko"
},
"peerDependencies": {
"@types/react": "^16 || ^17 || ^18 || ^19",
"@types/react-dom": "^16 || ^17 || ^18 || ^19",
"react": "^16.8.0 || ^17 || ^18 || ^19",
"react-dom": "^16.8.0 || ^17 || ^18 || ^19"
},
"peerDependenciesMeta": {
"@types/react": {
"optional": true
},
"@types/react-dom": {
"optional": true
}
}
}
}
}
+2 -1
View File
@@ -14,7 +14,8 @@
"@heroui/styles": "^3.2.4",
"react": "^19.2.8",
"react-dom": "^19.2.8",
"react-router-dom": "^7.18.3"
"react-router-dom": "^7.18.3",
"yet-another-react-lightbox": "^3.32.2"
},
"devDependencies": {
"@tailwindcss/vite": "^4.3.3",
Binary file not shown.

After

Width:  |  Height:  |  Size: 810 KiB

+5 -1
View File
@@ -140,13 +140,17 @@ function makeSSEStream(opts: SSEStreamOpts): () => void {
export function hubLogStream(opts: {
since: number
onLine: (line: string) => void
// 服务端检测到日志被轮转/清空时回调,调用方应丢弃已失效的旧窗口
onReset?: () => void
onStatus?: (s: 'open' | 'reconnecting' | 'closed') => void
}): () => void {
let since = opts.since
return makeSSEStream({
url: () => '/hub/api/logs/stream?since=' + since,
onData: (obj) => {
if (obj && obj.line !== undefined) {
if (!obj) return
if (obj.reset) { opts.onReset?.(); return }
if (obj.line !== undefined) {
opts.onLine(obj.line)
if (typeof obj.offset === 'number') since = obj.offset
}
+109
View File
@@ -0,0 +1,109 @@
import {
createContext, useCallback, useContext, useEffect, useRef, useState, type ReactNode,
} from 'react'
import { Button, Card, Spinner, toast } from '@heroui/react'
import { hubBotAction } from './api/client'
import { watchBotReconnect } from './lib/botAction'
/** 停止/重启 bot 的全局相位(本文件是 Provider;纯等待逻辑在 lib/botAction.ts)
*
* 为什么必须是全局的:重启入口有两个 —— 顶栏账号菜单(任何页面都能点)和
* 设置页的「进程控制」。以前两边各写一套:设置页只有**本页**的 fixed 覆盖层,
* 顶栏只弹个 toast。于是从插件页/日志页点「重启 bot」时界面照常可点,请求全
* 打在已经退出的旧进程上。现在相位收进这一个 provider,覆盖层由 AppLayout
* 统一渲染,任何页面触发都挡得住。
*/
export type BotPhase = null | 'stop' | 'restart'
interface BotActionState {
/** 非 null 表示正处于「已停止」/「等待回连」,全局覆盖层生效 */
phase: BotPhase
/** 成功回连的累计次数:页面据此在回连后重新拉数据 */
readyCount: number
/** 请求停止/重启。进入等待相位;失败原样抛出(由调用方 toast) */
run: (action: 'stop' | 'restart') => Promise<void>
/** 停止后手动收起覆盖层(重启不用,回连或超时会自动收) */
dismiss: () => void
}
const Ctx = createContext<BotActionState>({
phase: null,
readyCount: 0,
run: async () => { throw new Error('BotActionProvider 未挂载') },
dismiss: () => {},
})
export function BotActionProvider({ children }: { children: ReactNode }) {
const [phase, setPhase] = useState<BotPhase>(null)
const [readyCount, setReadyCount] = useState(0)
const reconnectRef = useRef<(() => void) | null>(null)
// 登出/布局卸载时别把等待回连的 SSE 与定时器留在外面
useEffect(() => () => { reconnectRef.current?.() }, [])
const run = useCallback(async (action: 'stop' | 'restart') => {
// 失败直接抛给调用方,这里不改相位(免得把界面锁在一个没发生的操作上)
await hubBotAction(action)
if (action === 'stop') {
// 停了就不会再回连,覆盖层留到用户点「知道了」
setPhase('stop')
return
}
setPhase('restart')
reconnectRef.current?.()
reconnectRef.current = watchBotReconnect({
onReady: () => {
reconnectRef.current = null
setPhase(null)
setReadyCount(n => n + 1)
toast.success('bot 已重启并回连')
},
onTimeout: () => {
reconnectRef.current = null
setPhase(null)
toast.danger('等待回连超时,请检查 bot 是否已启动')
},
})
}, [])
const dismiss = useCallback(() => setPhase(null), [])
return (
<Ctx.Provider value={{ phase, readyCount, run, dismiss }}>{children}</Ctx.Provider>
)
}
export const useBotAction = () => useContext(Ctx)
/** 全局覆盖层:铺满视口(含侧栏与顶栏),等待期间挡掉一切点击。
*
* 刻意用裸 div 而不是 Modal —— Modal 支持 Esc 与点遮罩关闭,而「正在重启」
* 期间放行操作恰恰是这里要防的事。
*/
export function BotActionOverlay() {
const { phase, dismiss } = useBotAction()
if (phase === null) return null
return (
<div className="fixed inset-0 z-50 flex items-center justify-center bg-slate-900/60 p-4">
<Card className="border border-slate-200 bg-white">
<Card.Content className="space-y-3 text-center">
{phase === 'stop' ? (
<>
<div className="text-sm font-medium text-slate-800">正在停止 bot…</div>
<p className="text-xs text-slate-400">Web 与 QQ 连接已断开,请在服务器重新启动 bot。</p>
<Button size="sm" variant="tertiary" onPress={dismiss}>知道了</Button>
</>
) : (
<>
<div className="text-sm font-medium text-slate-800">正在重启 bot,等待回连…</div>
<p className="text-xs text-slate-400">期间已禁用所有操作,恢复后自动继续。</p>
<div className="flex justify-center"><Spinner /></div>
</>
)}
</Card.Content>
</Card>
</div>
)
}
+2 -2
View File
@@ -1,7 +1,7 @@
import { createContext, useContext, useEffect, useState, type ReactNode } from 'react'
import { hubDashboardStream } from './api/client'
export type ConnStatus = '连接中' | '实时' | '重连中' | '已断开'
export type ConnStatus = '连接中' | '已连接' | '重连中' | '已断开'
interface ConnState {
status: ConnStatus
@@ -28,7 +28,7 @@ export function ConnectionProvider({ children }: { children: ReactNode }) {
if (d && d.ok !== false) { setData(d); setError(''); setLastUpdate(Date.now()) }
else if (d && d.ok === false) setError(d.msg || '采集失败')
},
onStatus: (s) => setStatus(s === 'open' ? '实时' : s === 'reconnecting' ? '重连中' : '已断开'),
onStatus: (s) => setStatus(s === 'open' ? '已连接' : s === 'reconnecting' ? '重连中' : '已断开'),
})
return dispose
}, [])
+89 -9
View File
@@ -1,11 +1,18 @@
import { useLayoutEffect, useEffect, useRef, useState, Suspense } from 'react'
import { Navigate, NavLink, Outlet, useLocation, useNavigate } from 'react-router-dom'
import { Avatar, Button, Separator, Spinner } from '@heroui/react'
import { House, Rocket, Gear, FileText, Bars } from '@gravity-ui/icons'
import {
Avatar, Button, Dropdown, Modal, Spinner, toast,
} from '@heroui/react'
import {
ArrowRightFromSquare, Bars, ChevronDown, FileText, Gear, House, Power, Rocket,
ArrowRotateLeft,
} from '@gravity-ui/icons'
import { useAuth } from '../auth/auth'
import { hubPlugins } from '../api/client'
import { ErrorBoundary } from '../components/ErrorBoundary'
import { useConnection } from '../connection'
import { BOT_ACTION_TEXT } from '../lib/botAction'
import { BotActionOverlay, useBotAction } from '../botAction'
interface NavItem { id: string; label: string; description: string; icon: any }
@@ -174,7 +181,7 @@ export function AppLayout() {
<Bars className="size-5" />
</Button>
{isPluginSub && (
<Button size="sm" variant="tertiary" className="max-w-24 shrink-0 truncate" onPress={() => navigate('/hub/plugins')}>← 返回</Button>
<Button size="sm" variant="tertiary" className="max-w-24 shrink-0 truncate" onPress={() => navigate('/hub/plugins')}>返回</Button>
)}
{isPluginSub ? (
<div className="flex min-w-0 items-center gap-2">
@@ -192,16 +199,16 @@ export function AppLayout() {
<div className="flex shrink-0 items-center gap-2 md:gap-3">
<span
title={lastUpdate ? '最后更新 ' + new Date(lastUpdate).toLocaleTimeString('zh-CN', { hour12: false }) : '正在连接后端…'}
className={'inline-flex items-center gap-1.5 rounded-full border px-2.5 py-1 text-xs transition-colors duration-300 ' + (status === '实时' ? 'border-emerald-200 bg-emerald-50 text-emerald-600' : status === '重连中' ? 'border-amber-200 bg-amber-50 text-amber-600' : 'border-slate-200 bg-slate-50 text-slate-400')}
className={'inline-flex items-center gap-1.5 rounded-full border px-2.5 py-1 text-xs transition-colors duration-300 ' + (status === '已连接' ? 'border-emerald-200 bg-emerald-50 text-emerald-600' : status === '重连中' ? 'border-amber-200 bg-amber-50 text-amber-600' : 'border-slate-200 bg-slate-50 text-slate-400')}
>
<span className={'size-1.5 rounded-full ' + (status === '实时' ? 'bg-emerald-500' : status === '重连中' ? 'animate-pulse bg-amber-500' : 'bg-slate-300')} />
<span className={'size-1.5 rounded-full ' + (status === '已连接' ? 'bg-emerald-500' : status === '重连中' ? 'animate-pulse bg-amber-500' : 'bg-slate-300')} />
{status}
{lastUpdate && <span className="hidden text-[10px] opacity-70 sm:inline">{new Date(lastUpdate).toLocaleTimeString('zh-CN', { hour12: false })}</span>}
</span>
<Avatar size="sm"><Avatar.Fallback>{(username || 'U').charAt(0).toUpperCase()}</Avatar.Fallback></Avatar>
<span className="hidden text-sm text-slate-600 sm:inline">{username || '未登录'}</span>
<Separator orientation="vertical" className="hidden h-5 sm:block" />
<Button size="sm" variant="tertiary" onPress={() => { logout(); navigate('/hub/login') }}>退出</Button>
<AccountMenu
username={username}
onLogout={() => { logout(); navigate('/hub/login') }}
/>
</div>
</header>
<main className="min-h-0 flex-1 overflow-y-auto p-4 md:p-6">
@@ -215,6 +222,79 @@ export function AppLayout() {
</div>
</main>
</div>
{/* 停止/重启 bot 的全局遮挡:放在 Outlet 之外,任何页面触发都盖得住 */}
<BotActionOverlay />
</div>
)
}
/** 顶栏账号菜单:头像 + 用户名点开下拉,重启/关闭 bot 与退出登录都收进去。
*
* 重启/关闭都是不可逆的破坏性操作(关掉之后得去服务器手动拉起),
* 所以先弹确认框再发请求;等待回连与全局覆盖层交给 `BotActionProvider`,
* 这样从**任何页面**触发都不会出现「看着没事、点哪都在打已退出的进程」。
*/
function AccountMenu({ username, onLogout }: { username: string; onLogout: () => void }) {
const [pending, setPending] = useState<'stop' | 'restart' | null>(null)
const [busy, setBusy] = useState(false)
const { run } = useBotAction()
const confirm = async () => {
const action = pending
if (!action) return
setBusy(true)
try {
await run(action)
setPending(null)
} catch (e: any) {
toast.danger(e.message || '操作失败')
} finally { setBusy(false) }
}
return (
<>
<Dropdown>
<Dropdown.Trigger className="flex cursor-pointer items-center gap-2 rounded-lg px-1.5 py-1 transition-colors hover:bg-slate-100">
<Avatar size="sm"><Avatar.Fallback>{(username || 'U').charAt(0).toUpperCase()}</Avatar.Fallback></Avatar>
<span className="hidden max-w-24 truncate text-sm text-slate-600 sm:inline">{username || '未登录'}</span>
<ChevronDown className="size-4 shrink-0 text-slate-400" />
</Dropdown.Trigger>
<Dropdown.Popover>
<Dropdown.Menu
onAction={(key) => {
if (key === 'logout') onLogout()
else setPending(key === 'stop' ? 'stop' : 'restart')
}}
>
<Dropdown.Item id="restart">
<span className="flex items-center gap-2"><ArrowRotateLeft className="size-4" />重启 bot</span>
</Dropdown.Item>
<Dropdown.Item id="stop">
<span className="flex items-center gap-2"><Power className="size-4" />关闭 bot</span>
</Dropdown.Item>
<Dropdown.Item id="logout">
<span className="flex items-center gap-2"><ArrowRightFromSquare className="size-4" />退出登录</span>
</Dropdown.Item>
</Dropdown.Menu>
</Dropdown.Popover>
</Dropdown>
<Modal isOpen={pending !== null} onOpenChange={(open) => { if (!open) setPending(null) }}>
<Modal.Backdrop><Modal.Container><Modal.Dialog>
<Modal.Header><Modal.Heading>{pending ? BOT_ACTION_TEXT[pending].title : ''}</Modal.Heading></Modal.Header>
<Modal.Body>
<p className="text-sm text-slate-600">{pending ? BOT_ACTION_TEXT[pending].confirm : ''}</p>
</Modal.Body>
<Modal.Footer>
<Button variant="tertiary" onPress={() => setPending(null)}>取消</Button>
<Button isDisabled={busy} className={pending === 'stop' ? 'text-red-500' : ''} onPress={confirm}>
{busy ? '执行中…' : '确定'}
</Button>
</Modal.Footer>
</Modal.Dialog></Modal.Container></Modal.Backdrop>
</Modal>
</>
)
}
+52
View File
@@ -0,0 +1,52 @@
import { hubDashboardStream } from '../api/client'
/** 请求停止/重启 bot 后,等它回连的那段等待逻辑(设置页与顶栏账号菜单共用)。
*
* 重启分两步回来:旧进程先退出(让出端口),新进程才起来,所以先干等一会儿
* 再开始探活;探活以 dashboard SSE **真正收到一帧**为准 —— 只看 HTTP 200 会在
* 「端口起来了但插件还没挂接」时就放行,紧接着的请求全 404。
*
* 返回 dispose:重复触发或组件卸载时调用,内部会撤掉 SSE 与超时定时器。
*/
export function watchBotReconnect(opts: {
onReady: () => void
onTimeout: () => void
/** 等旧进程退出的时间 */
startupDelayMs?: number
/** 探活总超时 */
timeoutMs?: number
}): () => void {
const { onReady, onTimeout, startupDelayMs = 4000, timeoutMs = 60000 } = opts
let stopped = false
let timer: number | null = null
let disposeStream: (() => void) | null = null
const finish = () => {
stopped = true
if (timer != null) { window.clearTimeout(timer); timer = null }
if (disposeStream) { disposeStream(); disposeStream = null }
}
timer = window.setTimeout(() => {
if (stopped) return
disposeStream = hubDashboardStream({
onData: (d) => {
if (stopped) return
if (d && d.ok !== false) { finish(); onReady() }
},
})
timer = window.setTimeout(() => {
if (stopped) return
finish()
onTimeout()
}, timeoutMs)
}, startupDelayMs)
return finish
}
/** 菜单/按钮上共用的文案,避免两处写法漂移 */
export const BOT_ACTION_TEXT = {
restart: { title: '重启 bot', confirm: '确定重启 bot 进程吗?将先拉起新进程再退出旧进程,期间 Web 与 QQ 连接会短暂断开。' },
stop: { title: '关闭 bot', confirm: '确定关闭当前 bot 进程吗?Web 管理台与 QQ 连接都会断开,需要在服务器上手动重新启动。' },
} as const
+119 -43
View File
@@ -1,22 +1,36 @@
import { createContext, useCallback, useContext, useEffect, useRef, useState, type ReactNode } from 'react'
import { hubLogs, hubLogStream } from './api/client'
// 保留的最大行数(窗口较大以容纳「加载更早」预置的历史行)
const MAX_LINES = 6000
// 渲染窗口:日志页**始终只渲染这么多行**,先入先出——新行从尾部挤入,最旧的行从头部被挤掉。
// 行数直接决定每次更新的 DOM 协调与布局开销,是页面的主要成本,不要随意调大。
const WINDOW_LINES = 1000
// 「向前翻页」每次额外取的行数:比窗口小,翻页时保留一半旧内容做重叠,
// 配合浏览器的滚动锚定,视线里的那行不会跳走。
const PAGE_LINES = 500
// 新行合并刷新间隔(ms):日志突发时避免「一行一次 setState」
const FLUSH_MS = 120
export interface LogLine { id: number; text: string }
export interface LogInfo { file: string; size: number; total: number }
interface LogState {
lines: string[]
lines: LogLine[]
info: LogInfo
error: string
loading: boolean
hasMore: boolean
loadingEarlier: boolean
paused: boolean
setPaused: (v: boolean) => void
/** true = 自动滚动(跟随最新,新行实时入列);false = 冻结视图(新行暂存,回到底部再补上) */
follow: boolean
setFollow: (v: boolean) => void
/** 自动换行:false 时长行不折行,改为横向滚动 */
wrap: boolean
setWrap: (v: boolean) => void
refresh: () => Promise<void>
loadEarlier: () => Promise<void>
/** 向前翻一页;返回本次前置进来的行数(0 表示没翻成),调用方据此把视图锚回原处 */
loadEarlier: () => Promise<number>
}
const Ctx = createContext<LogState>({
@@ -26,62 +40,110 @@ const Ctx = createContext<LogState>({
loading: false,
hasMore: false,
loadingEarlier: false,
paused: false,
setPaused: () => {},
follow: true,
setFollow: () => {},
wrap: true,
setWrap: () => {},
refresh: async () => {},
loadEarlier: async () => {},
loadEarlier: async () => 0,
})
/**
* 日志数据与日志流常驻 Provider:首次进入(任意页面)时拉取一次,
* SSE 流保持不断,页面之间切换只读缓存数据,不重建连接、不显示加载动画。
*
* 窗口模型:内存里始终只有 WINDOW_LINES 行(先入先出)。
* - 自动滚动:新行进窗口尾部,最旧的行被挤掉(tail -f 行为)
* - 冻结视图:新行只进暂存区(同样保留最新 WINDOW_LINES 行),窗口不变,
* 回到底部时一次性补上
* - 向前翻页:滚动到顶部时取更早的 PAGE_LINES 行,从尾部挤掉同样多的新行,
* 窗口行数不变(反过来的先入先出)
*/
export function LogProvider({ children }: { children: ReactNode }) {
const [lines, setLines] = useState<string[]>([])
const [lines, setLines] = useState<LogLine[]>([])
const [info, setInfo] = useState<LogInfo>({ file: '', size: 0, total: 0 })
const [error, setError] = useState('')
const [loading, setLoading] = useState(false)
const [hasMore, setHasMore] = useState(false)
const [loadingEarlier, setLoadingEarlier] = useState(false)
const [paused, setPaused] = useState(false)
const [follow, setFollow] = useState(true)
const [wrap, setWrap] = useState(true)
const sinceRef = useRef(0)
const earliestRef = useRef(0)
const pausedRef = useRef(false)
const pendingRef = useRef<string[]>([])
const followRef = useRef(true)
const frozenRef = useRef<string[]>([])
const disposeRef = useRef<(() => void) | null>(null)
const loadingEarlierRef = useRef(false)
const append = useCallback((line: string) => {
setLines(prev => [...prev, line].slice(-MAX_LINES))
// 行 id:作为 React key 保持稳定(用下标做 key 时,窗口一滑动整列表都要 diff)
const idRef = useRef(0)
// 待入列的新行与合并刷新定时器
const bufferRef = useRef<string[]>([])
const timerRef = useRef<number | null>(null)
const toItems = useCallback((texts: string[]): LogLine[] => {
const items: LogLine[] = []
for (const text of texts) items.push({ id: idRef.current++, text })
return items
}, [])
const flush = useCallback(() => {
timerRef.current = null
const buffered = bufferRef.current
if (!buffered.length) return
bufferRef.current = []
// 一次涌入超过整个窗口时(日志突发/整文件重放),只有最后 WINDOW_LINES 行留得下,
// 先裁掉前面的,避免为注定被挤掉的行白渲染一遍
const kept = buffered.length > WINDOW_LINES ? buffered.slice(-WINDOW_LINES) : buffered
// 副作用放在 setState 之外:StrictMode 下 updater 可能被调用两次
const items = toItems(kept)
setLines(prev => {
const next = prev.concat(items)
return next.length > WINDOW_LINES ? next.slice(-WINDOW_LINES) : next
})
}, [toItems])
const enqueue = useCallback((line: string) => {
if (!followRef.current) {
// 冻结视图期间只暂存,且同样只保留最新 WINDOW_LINES 行
frozenRef.current.push(line)
if (frozenRef.current.length > WINDOW_LINES) {
frozenRef.current.splice(0, frozenRef.current.length - WINDOW_LINES)
}
return
}
bufferRef.current.push(line)
if (timerRef.current == null) timerRef.current = window.setTimeout(flush, FLUSH_MS)
}, [flush])
const connect = useCallback(() => {
if (disposeRef.current) disposeRef.current()
disposeRef.current = hubLogStream({
since: sinceRef.current,
onLine: (line) => {
if (pausedRef.current) {
pendingRef.current.push(line)
// 暂停期间只保留最近若干行,避免恢复时一次性渲染超大数组
if (pendingRef.current.length > 500) pendingRef.current.splice(0, pendingRef.current.length - 500)
return
}
append(line)
onLine: enqueue,
onReset: () => {
// 服务端日志被轮转/清空:丢弃已失效的旧窗口,从新文件继续
if (timerRef.current != null) { window.clearTimeout(timerRef.current); timerRef.current = null }
bufferRef.current = []
frozenRef.current = []
setLines([])
earliestRef.current = 0
setHasMore(false)
},
})
}, [append])
}, [enqueue])
const refresh = useCallback(async () => {
setLoading(true)
setError('')
try {
const d = await hubLogs(500)
const d = await hubLogs(WINDOW_LINES)
if (!d) throw new Error('读取日志失败')
if (d.ok === false) throw new Error(d.msg || '读取日志失败')
setLines(d.lines || [])
setLines(toItems(d.lines || []))
setInfo({ file: d.file || '', size: d.size || 0, total: d.total || 0 })
// stream 从文件尾续读(end);「加载更早」从窗口起点向前翻(offset)
// stream 从文件尾续读(end);向前翻页从窗口起点继续往前(offset)
sinceRef.current = d.end || 0
earliestRef.current = d.offset || 0
setHasMore((d.offset || 0) > 0)
@@ -89,47 +151,61 @@ export function LogProvider({ children }: { children: ReactNode }) {
} catch (e: any) {
setError(e.message || '读取日志失败')
} finally { setLoading(false) }
}, [connect])
}, [connect, toItems])
const loadEarlier = useCallback(async () => {
const loadEarlier = useCallback(async (): Promise<number> => {
const before = earliestRef.current
if (!before || loadingEarlierRef.current) return
if (!before || loadingEarlierRef.current) return 0
loadingEarlierRef.current = true
setLoadingEarlier(true)
setError('')
try {
const d = await hubLogs(500, before)
const d = await hubLogs(PAGE_LINES, before)
if (!d || d.ok === false) throw new Error((d && d.msg) || '读取日志失败')
setLines(prev => [...(d.lines || []), ...prev].slice(-MAX_LINES))
const items = toItems(d.lines || [])
setLines(prev => {
const next = items.concat(prev)
// 往前翻时从尾部(较新的行)挤掉:窗口行数不变,新的行让位给历史
return next.length > WINDOW_LINES ? next.slice(0, WINDOW_LINES) : next
})
earliestRef.current = d.offset || 0
setHasMore((d.offset || 0) > 0)
return items.length
} catch (e: any) {
setError(e.message || '读取日志失败')
return 0
} finally {
loadingEarlierRef.current = false
setLoadingEarlier(false)
}
}, [])
}, [toItems])
// 首次进入就建立并保持;provider 卸载(登出/离开 /hub)时才断开
useEffect(() => {
refresh()
return () => { if (disposeRef.current) disposeRef.current() }
return () => {
if (disposeRef.current) disposeRef.current()
// 组件卸载时清掉待刷新定时器,避免对已卸载组件 setState
if (timerRef.current != null) { window.clearTimeout(timerRef.current); timerRef.current = null }
}
}, [refresh])
useEffect(() => { pausedRef.current = paused }, [paused])
useEffect(() => {
// 恢复接收时,把暂停期间暂存的行补进来
if (!paused && pendingRef.current.length) {
const flush = pendingRef.current
pendingRef.current = []
setLines(prev => [...prev, ...flush].slice(-MAX_LINES))
followRef.current = follow
// 恢复自动滚动时,把冻结期间暂存的行一次性补上(仍只保留 WINDOW_LINES 行)
if (follow && frozenRef.current.length) {
const frozen = frozenRef.current
frozenRef.current = []
const items = toItems(frozen)
setLines(prev => {
const next = prev.concat(items)
return next.length > WINDOW_LINES ? next.slice(-WINDOW_LINES) : next
})
}
}, [paused])
}, [follow, toItems])
return (
<Ctx.Provider value={{ lines, info, error, loading, hasMore, loadingEarlier, paused, setPaused, refresh, loadEarlier }}>
<Ctx.Provider value={{ lines, info, error, loading, hasMore, loadingEarlier, follow, setFollow, wrap, setWrap, refresh, loadEarlier }}>
{children}
</Ctx.Provider>
)

Some files were not shown because too many files have changed in this diff Show More