Pyrecall: Open Source Tool for Detecting Catastrophic Forgetting During LLM Fine-Tuning
English summary
Pyrecall is a new open-source tool built to address the lack of practical tooling for continual learning research. It snapshots skill scores before and after fine-tuning, flags performance regressions, and supports rolling back LoRA adapters by name. The tool runs fully locally, is released under the MIT license at v0.1.0, and can be installed via pip. The developer is seeking community feedback on the benchmark design.
Chinese summary
Pyrecall 是一款全新的开源工具，旨在填补持续学习研究中实用工具的空白。它会在微调前后记录技能评分，标记性能退化，并支持按名称回滚 LoRA 适配器。该工具完全本地运行，采用 MIT 许可证发布 v0.1.0 版本，可通过 pip 安装。开发者正在就基准设计征求社区反馈。
Key points
Open-source tool for detecting catastrophic forgetting in LLM fine-tuning.
用于检测 LLM 微调中灾难性遗忘的开源工具。
Captures skill score snapshots before and after fine-tuning to identify regressions.
在微调前后捕获技能评分快照以识别性能退化。
Supports rolling back LoRA adapters by name.
支持按名称回滚 LoRA 适配器。
Fully local operation with no external APIs; v0.1.0, MIT license, installable via pip.
完全本地运行，无外部 API；v0.1.0 版本，MIT 许可证，可通过 pip 安装。
Author is seeking feedback on the benchmark design.
作者正寻求关于基准设计的反馈。