Compare commits

..
34 Commits
Author SHA1 Message Date
OpenClaw Agent b116303abe feat: harden multi-user download concurrency 2026-09-18 09:06:59 +08:00
OpenClaw Agent 0735cb2e09 docs: update UI redesign docs with mobile nav and error localization details 2026-09-17 07:11:09 +08:00
OpenClaw Agent a295f2d610 fix(ui): attach mobile nav to viewport bottom and localize api error messages 2026-09-17 07:09:05 +08:00
OpenClaw Agent d8cc416b20 fix: preserve credit action handlers in user drawer 2026-09-17 07:04:28 +08:00
OpenClaw Agent 215c38f4fb fix: isolate admin WebDAV path by username 2026-09-17 06:50:33 +08:00
OpenClaw Agent 020944d4a0 fix: unify WebDAV upload behavior across roles 2026-09-17 06:47:34 +08:00
OpenClaw Agent 40d99e1e22 fix: make WebDAV upload retry reliable 2026-09-17 06:41:35 +08:00
OpenClaw Agent efb2ff33b4 fix: preserve global WebDAV password when settings save omits it 2026-09-17 06:24:29 +08:00
OpenClaw Agent 684665f0be feat: assign per-user WebDAV paths 2026-09-17 06:16:57 +08:00
OpenClaw Agent 3025019504 feat: wire admin UI to credit ledger APIs 2026-09-17 06:08:20 +08:00
OpenClaw Agent d9df512e0a docs: update UI_REDESIGN.md and clean up nav buttons 2026-09-17 06:04:25 +08:00
OpenClaw Agent 221ce7b6bb feat: refactor admin with segmented tabs, drawer details, and unified settings 2026-09-17 05:56:13 +08:00
OpenClaw Agent aaf6790afe style: add mobile settings nav and modernize UI design 2026-09-17 04:44:31 +08:00
OpenClaw Agent ceec0d797c style: refine mobile admin navigation 2026-09-17 03:45:46 +08:00
OpenClaw Agent 6ccaec53d7 refactor: clean admin navigation and user details 2026-09-17 03:42:24 +08:00
OpenClaw Agent fef1cb7057 fix: show admin entry on mobile navigation 2026-09-17 03:36:59 +08:00
OpenClaw Agent 9f57addec8 feat: group admin users by role 2026-09-17 03:35:20 +08:00
OpenClaw Agent 7507ffa02b feat: expand admin user management 2026-09-17 03:34:41 +08:00
OpenClaw Agent 5a5f1e6d88 feat: automatically resync completed downloads to webdav 2026-09-17 03:25:06 +08:00
OpenClaw Agent 5a624a5d60 fix: privileged webdav falls back to global storage 2026-09-17 03:18:30 +08:00
OpenClaw Agent 44a365822b fix: give regular users global webdav access 2026-09-17 03:11:55 +08:00
OpenClaw Agent 7169a7b5fd feat: make database migration testable and backfill assets 2026-09-17 02:42:38 +08:00
OpenClaw Agent 4607312b7a docs: update multi-user API and migration notes 2026-09-17 02:34:38 +08:00
OpenClaw Agent a60801173f feat: add password change and session invalidation 2026-09-17 02:33:51 +08:00
OpenClaw Agent dea1473424 fix: recover stale shared asset locks 2026-09-17 02:31:05 +08:00
OpenClaw Agent b853a239ff feat: add shared media assets and fix credit lifecycle 2026-09-17 02:30:31 +08:00
OpenClaw Agent 7d79d43b61 feat: add account and admin frontend 2026-09-17 02:21:50 +08:00
OpenClaw Agent c79381555a feat: record admin and redeem audit events 2026-09-17 02:20:54 +08:00
OpenClaw Agent 5fbf9fe252 feat: add login attempt rate limiting 2026-09-17 02:20:38 +08:00
OpenClaw Agent dcf01e8f4c feat: add admin audit log endpoint 2026-09-17 02:20:24 +08:00
OpenClaw Agent c4aa30e564 feat: add redeem batch revocation 2026-09-17 02:20:11 +08:00
OpenClaw Agent 8b31956c6d feat: implement multi-user credits redeem and storage isolation 2026-09-17 02:18:49 +08:00
OpenClaw Agent 7628390966 feat: add multi-user auth and permission schema 2026-09-17 02:09:24 +08:00
OpenClaw Agent 72fdd3b74c docs: define multi-user V1 architecture 2026-09-17 02:07:03 +08:00
26 changed files with 4429 additions and 126 deletions
+90
View File
@@ -0,0 +1,90 @@
# AM Downloader 多用户 V1 方案
## 目标
在现有 Go + Gin + GORM + SQLite + 下载队列 + WebDAV 架构上增量加入多用户、权限、下载额度和兑换码,不推倒重写现有 Downloader。
## 角色与权限
- `admin`:全部权限;用户、兑换码、系统设置、日志、全局 WebDAV Storage 管理。
- `privileged`:普通下载能力 + WebDAV;可以配置自己的 WebDAV。
- `user`:下载、历史、兑换额度;不能访问 WebDAV、系统设置或其他用户。
- 底层使用 capability:`download`、`redeem`、`webdav`、`webdav_config`、`api`、`advanced_download`,避免以后扩展时绑定死角色。
## 设置隔离
- 用户设置:账户、外观;特权用户额外有自己的 WebDAV。
- 管理设置:`/admin/*`,只有管理员可见且后端强制校验。
- 现有全局 AM API、下载目录、默认音质、WebDAV Storage 等属于管理员配置。
- 前端隐藏不是安全措施;API 必须返回 401/403。
## WebDAV
- 管理员定义 Storage Root,例如 `/data/webdav`。
- 特权用户拥有自己的 WebDAV 配置和独立根目录 `/data/webdav/{user_id}/`。
- 禁止用户通过 `..`、绝对路径等逃逸 Storage Root。
- WebDAV 权限和 WebDAV 配置权限分离,方便以后扩展。
- 现有全局 WebDAV 继续作为系统级远程 Storage,逐步迁移为管理员配置。
## Credits
- 一首实际成功下载的歌曲消耗 1 credit。
- 已存在且可复用的文件不扣 credit。
- 新下载创建任务时预留 credit;成功 commit,失败 refund。
- 同一任务重试不能重复扣费。
- 批量任务按实际成功数量结算。
- 所有额度变化进入不可变流水,不允许直接改余额。
## Redeem Code
- 管理员可按批次批量生成兑换码。
- 每个兑换码可以有自定义 credits、有效期、备注。
- 状态:`unused`、`used`、`expired`、`revoked`。
- 数据库只保存 code hash;生成后提供复制/导出明文。
- 兑换必须事务化,防止并发双兑。
- 管理员可以作废兑换码、查看批次统计和兑换记录。
## 数据模型
- `users`
- `user_permissions`
- `credit_accounts`
- `credit_transactions`
- `credit_reservations`
- `redeem_batches`
- `redeem_codes`
- `redeem_records`
- `webdav_configs`
- `audit_logs`
- `sessions`
- 现有 `tasks` 增加 `user_id`、`credit_reservation_id`,继续保留全局资源去重索引。
- `settings` 继续保存管理员系统设置。
## 下载与去重
不要简单把现有唯一键改成 `user_id + source + rid + quality`。用户任务与全局资源/文件应逻辑分离:同一资源可以被多个用户请求,但每个用户拥有自己的任务、额度流水和访问边界。第一阶段保持现有 Task 模型兼容,再逐步增加资源复用层。
## API 分层
- `/api/auth/*`:登录、登出、当前用户。
- `/api/tasks`:当前用户任务;所有查询按 user_id 隔离。
- `/api/credits/*`:余额、流水、兑换。
- `/api/webdav/*`:仅拥有 webdav 权限的用户。
- `/api/admin/users/*`:管理员。
- `/api/admin/redeem/*`:管理员。
- `/api/admin/settings/*`:管理员。
- `/api/admin/audit/*`:管理员。
## 安全
- 密码只保存强哈希。
- Session 使用随机不可预测 token,数据库保存 hash。
- 登录限速。
- 管理员敏感操作记录 audit log。
- 额度操作必须事务化。
- 兑换码不可明文持久化。
- WebDAV 路径必须做目录边界校验。
- 用户绝不能通过 API 读取其他用户 Task、Credit、WebDAV 配置。
## 实施顺序
1. 数据模型与数据库迁移
2. Session/Auth
3. 权限中间件
4. Task 用户隔离
5. Credit Ledger + Reservation
6. 下载成功/失败扣退额度
7. Redeem Code
8. WebDAV 特权用户隔离
9. 管理后台 API
10. 前端用户/特权/管理员界面
11. 测试、迁移兼容性、备份与部署
+51 -3
View File
@@ -4,8 +4,11 @@
## 特性
- 多用户登录、角色与权限隔离(admin / privileged / user)
- 积分账户、预扣、成功扣除、失败退款
- 兑换码批次生成、使用、撤销与审计
- 音乐搜索与分页
- 下载队列、去重、失败重试
- 下载队列、用户任务去重、跨用户媒体资源去重、失败重试
- 自动识别实际音频格式,避免错误扩展名
- 标记解析后的音质与实际音质
- WebDAV/NAS 同步
@@ -65,11 +68,56 @@ go build -o am-downloader .
## API
认证:
- `POST /api/auth/login`
- `POST /api/auth/logout`
- `POST /api/auth/password`
- `GET /api/auth/me`
用户功能:
- `GET /api/search`
- `GET /api/tasks`
- `POST /api/tasks`
- `POST /api/tasks/:id/retry`
- `DELETE /api/tasks/:id`
- `GET /api/settings`
- `PUT /api/settings`
- `GET /api/credits`
- `POST /api/redeem`
- `GET/PUT /api/webdav/config`
- `POST /api/webdav/test`
管理员功能:
- `GET/POST /api/admin/users`
- `PUT /api/admin/users/:id`
- `GET /api/admin/redeem`
- `POST /api/admin/redeem`
- `POST /api/admin/redeem/:id/revoke`
- `GET /api/admin/audit`
- `GET/PUT /api/admin/settings`
- `POST /api/admin/webdav/test`
## 数据库迁移注意事项
首次从旧版本升级时,启动过程会自动执行 GORM schema migration,并将旧任务归属到首次创建的管理员账号;同时把旧的全局任务唯一索引改为按用户隔离的唯一索引,并创建共享媒体资源表。
升级前仍建议先备份 `data/downloader.db`。Docker 部署时不要把 `data/` 或 `downloads/` 加入 Git。
当前版本不会主动删除旧媒体文件;媒体资源记录缺失本地文件时会在下一次请求中重新取得资源。
## 多用户并发与恢复
下载任务和共享媒体资源分离:
- 同一 source + rid + quality 只允许一个 Asset 物理下载,多个用户 Task 可以共享。
- 每个用户 Task 保留独立 WebDAV 目标路径、状态和额度预留。
- Task 使用数据库条件更新抢占状态,重复点击不会启动多个 worker。
- Asset 使用数据库租约(heartbeat/lease)防止 worker 崩溃后永久占锁;过期租约会自动恢复旧任务。
- 下载文件使用按 Asset/generation 隔离的缓存目录和 task-specific .part 临时文件,避免并发用户覆盖同一个本地文件。
- WebDAV 上传采用独立的 upload_pending -> uploading -> done 状态机;上传失败只重试上传,不重新扣积分。
- 已提交额度的历史失败任务会自动恢复;已退款任务不会因为后台 resync 被无条件重新扣费。
- SQLite 使用 WAL + busy timeout,并对关键事务做锁冲突重试;高并发写入仍由 SQLite 单写者模型串行化。
- 本地共享文件只有在没有其他活动 Task 需要时才允许清理。
并发回归测试覆盖:Task 抢占、共享 Asset 租约恢复、已提交额度重试、已退款额度重试、并发积分预留以及 Asset generation 隔离。
+78
View File
@@ -0,0 +1,78 @@
# AM Downloader 交互界面 (UI) 重构与优化文档
## 1. 概述与背景
`am-downloader` 原先的 `index.html` 采用单页平铺逻辑,随着多用户鉴权、积分兑换、用户级 WebDAV 与管理员后台的引入,旧版界面存在层级分散、导航冗余、缺乏现代感与移动端适配生硬等问题。
本项目已完成系统级的前端现代化重构,并保持对后端原有 API 契约和前端 DOM 事件完全兼容,目前已在运行中的 Docker 容器环境(端口 `9540`)热更新上线生效。
---
## 2. 视觉设计系统规范
### 2.1 配色与质感(Modern Dark Glassmorphism)
- **深色基底**:主背景采用 `#080c14`,配合双色极光渐变暗影(翡翠绿 `#10b981` 与天蓝 `#38bdf8`),提升流媒体科技质感。
- **毛玻璃面板**:卡片与导航栏统一采用 `rgba(17, 26, 42, 0.75)` 配以 `backdrop-filter: blur(16px)`,边框引入半透明微发光分割线(`rgba(255, 255, 255, 0.08)`)。
- **字体与层级**:接入 `Plus Jakarta Sans` 作为主要无衬线英文字体,`JetBrains Mono` 作为代码、卡密与音质参数等宽字体。
---
## 3. 核心功能与架构重构细节
### 3.1 导航体系净化与精简
1. **移除顶部及移动端独立「设置」按钮**:
- 彻底移除了原先顶栏导航与移动端底栏的独立「设置」入口。
- 所有系统运行配置统一纳管在管理员后台内部,避免入口碎片化。
2. **下载按钮去除角标数字**:
- 移除桌面端导航与移动端底栏「下载」按钮后方突兀的数字徽章(Badge),保持导航栏整体视觉纯粹和谐。
- 保留 DOM 节点以保证后台任务轮询更新逻辑的向后兼容,不报错不中断。
### 3.2 管理控制台(Admin Center)全面重构
1. **分段式选项卡(Segmented Tabs)**:
- 废弃原有的“点击磁贴切换子面板再返回”繁琐交互,重构为顶部毛玻璃分段 Tab 栏:
- `👥 用户管理`
- `◎ 积分账本`
- `◇ 卡密批次`
- `≡ 审计日志`
- `⚙️ 全局设置`(由原“纳管设置”更名,系统 API、下载存储策略、全局 WebDAV 完整收拢在此)
- Tab 切换平滑无刷新,记忆当前选中状态。
2. **抽屉详情交互(Slide-over Drawer)**:
- 用户详情不再替换整页或跳转离开当前列表。
- 点击任意用户的「查看详情」时,桌面端由右侧平滑滑出毛玻璃抽屉面板,移动端自适应为底部滑起抽屉(Bottom Sheet)。
- 抽屉内置:账号角色与状态、密码重置、账号启停、积分增减与实时流水。
- 配合磨砂半透明背景遮罩(Backdrop),点击空白处或「✕ 关闭」按钮即可无缝收起,列表浏览上下文不丢失。
---
## 4. 兼容性与上线记录
- **DOM 契约兼容**:保留所有核心配置 ID(`amApi`、`downloadDir`、`localFilePolicy`、`defaultQuality`、`davUrl` 等)以及事件回调机制。
- **上线状态**:最新代码已提交 Git(分支 `master`),并同步热更新至运行中的 `am-downloader` 容器内 `/app/index.html`。
---
## 5. 交互与细节修复记录(2026-09-17)
### 5.1 移动端底部导航栏贴底优化
- **改动说明**:解决移动端底栏浮岛胶囊式设计(悬空 `12px` 且带圆角)在滚动时产生的文字从缝隙穿插、视觉割裂问题。
- **调整实现**:
- 改为全宽贴底原生 TabBar 布局(`left: 0; right: 0; bottom: 0; border-radius: 0;`)。
- 顶部微弱边框隔离(`border-top: 1px solid var(--border-subtle)`),四周无多余外边框。
- 适配安全区域与深色毛玻璃(`padding: 6px 8px calc(6px + env(safe-area-inset-bottom, 0px)); background: rgba(8, 12, 20, 0.95); backdrop-filter: blur(20px);`)。
### 5.2 全局接口与弹窗提示中文语义化
- **改动说明**:解决兑换失败、加入队列失败时直接将后端原始错误 JSON(如 `["error""code already used or unavailable"]`)弹给用户的粗糙体验。
- **调整实现**:
- 在底层封装统一错误解析器 `parseErrorMsg(msg)`,自动解包后端返回的 JSON 结构。
- 建立错误语义字典映射:
- `code already used or unavailable` → **该兑换码已被使用或已失效**
- `record not found` → **兑换码不存在,请核对后重试**
- `insufficient credits` → **积分不足,请先兑换或充值积分**
- `code expired` → **该兑换码已过期**
- `user disabled` → **账户已被禁用,无法执行此操作**
- `only failed tasks can be retried` → **只有失败的任务才支持重试**
- 其他校验项(用户名/密码规则、金额非零等)同步完成中文友好化。
### 5.3 用户管理抽屉积分调整事件委托审核
- **问题根因**:原代码在为 `detailAdd` 与 `detailSub` 绑定 `onclick` 后,通过 `innerHTML += ...` 追加积分流水,导致整个容器内 DOM 被全量重构,先前绑定的监听器失效。
- **修复方案与审核**:通过 `document.createElement('div')` + `appendChild(ledger)` 实现流水平行追加,既保留了已渲染节点的事件监听,又保证了 DOM 结构与 ID 契约完整无副作用。
+91
View File
@@ -0,0 +1,91 @@
package main
import (
"github.com/gin-gonic/gin"
"strconv"
)
func (a *App) adminUserCreditTransactions(c *gin.Context) {
id, err := strconv.ParseUint(c.Param("id"), 10, 64)
if err != nil {
c.JSON(400, gin.H{"error": "invalid user id"})
return
}
limit := 50
if n, e := strconv.Atoi(c.DefaultQuery("limit", "50")); e == nil && n > 0 && n <= 200 {
limit = n
}
var txs []CreditTransaction
q := a.db.Where("user_id = ?", id).Order("id desc").Limit(limit)
if before := c.Query("before"); before != "" {
if n, e := strconv.ParseUint(before, 10, 64); e == nil {
q = q.Where("id < ?", n)
}
}
if err := q.Find(&txs).Error; err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
type item struct {
ID uint `json:"id"`
Type string `json:"type"`
Amount int64 `json:"amount"`
BalanceAfter int64 `json:"balanceAfter"`
ReferenceType string `json:"referenceType"`
ReferenceID string `json:"referenceId"`
Reason string `json:"reason"`
CreatedAt interface{} `json:"createdAt"`
}
out := make([]item, 0, len(txs))
for _, t := range txs {
out = append(out, item{t.ID, t.Type, t.Amount, t.BalanceAfter, t.ReferenceType, t.ReferenceID, t.Reason, t.CreatedAt})
}
next := uint(0)
if len(txs) == limit {
next = txs[len(txs)-1].ID
}
c.JSON(200, gin.H{"data": out, "nextBefore": next})
}
func (a *App) adminCreditTransactions(c *gin.Context) {
limit := 100
if n, e := strconv.Atoi(c.DefaultQuery("limit", "100")); e == nil && n > 0 && n <= 300 {
limit = n
}
var txs []CreditTransaction
q := a.db.Order("id desc").Limit(limit)
if uid := c.Query("user_id"); uid != "" {
if n, e := strconv.ParseUint(uid, 10, 64); e == nil {
q = q.Where("user_id = ?", n)
}
}
if before := c.Query("before"); before != "" {
if n, e := strconv.ParseUint(before, 10, 64); e == nil {
q = q.Where("id < ?", n)
}
}
if err := q.Find(&txs).Error; err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
type item struct {
ID uint `json:"id"`
UserID uint `json:"userId"`
Type string `json:"type"`
Amount int64 `json:"amount"`
BalanceAfter int64 `json:"balanceAfter"`
ReferenceType string `json:"referenceType"`
ReferenceID string `json:"referenceId"`
Reason string `json:"reason"`
CreatedAt interface{} `json:"createdAt"`
}
out := make([]item, 0, len(txs))
for _, t := range txs {
out = append(out, item{t.ID, t.UserID, t.Type, t.Amount, t.BalanceAfter, t.ReferenceType, t.ReferenceID, t.Reason, t.CreatedAt})
}
next := uint(0)
if len(txs) == limit {
next = txs[len(txs)-1].ID
}
c.JSON(200, gin.H{"data": out, "nextBefore": next})
}
+362
View File
@@ -0,0 +1,362 @@
package main
import (
"errors"
"fmt"
"regexp"
"strconv"
"strings"
"github.com/gin-gonic/gin"
"gorm.io/gorm"
)
func (a *App) audit(operator uint, action, targetType, targetID, metadata string) {
_ = a.db.Create(&AuditLog{OperatorID: operator, Action: action, TargetType: targetType, TargetID: targetID, Metadata: metadata}).Error
}
var usernamePattern = regexp.MustCompile(`^[A-Za-z]{4,10}$`)
func defaultPermissions(role string) []string {
if role == RolePrivileged {
return []string{PermDownload, PermRedeem, PermWebDAV, PermWebDAVConfig}
}
if role == RoleUser {
return []string{PermDownload, PermRedeem, PermWebDAV}
}
return nil
}
func (a *App) adminListUsers(c *gin.Context) {
var users []User
if err := a.db.Order("id asc").Find(&users).Error; err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
type userView struct {
User
Balance int64 `json:"balance"`
Reserved int64 `json:"reserved"`
Available int64 `json:"available"`
TaskCount int64 `json:"taskCount"`
CompletedCount int64 `json:"completedCount"`
}
out := make([]userView, 0, len(users))
for i := range users {
u := users[i]
u.PasswordHash = ""
var account CreditAccount
_ = a.db.Where("user_id = ?", u.ID).First(&account).Error
var taskCount, completedCount int64
_ = a.db.Model(&Task{}).Where("user_id = ?", u.ID).Count(&taskCount).Error
_ = a.db.Model(&Task{}).Where("user_id = ? AND status = ?", u.ID, "done").Count(&completedCount).Error
out = append(out, userView{User: u, Balance: account.Balance, Reserved: account.Reserved, Available: account.Balance - account.Reserved, TaskCount: taskCount, CompletedCount: completedCount})
}
c.JSON(200, gin.H{"data": out})
}
func (a *App) adminCreateUser(c *gin.Context) {
var in struct {
Username string `json:"username"`
Password string `json:"password"`
Role string `json:"role"`
Credits int64 `json:"credits"`
}
if c.BindJSON(&in) != nil {
c.JSON(400, gin.H{"error": "invalid request body"})
return
}
in.Username = strings.TrimSpace(in.Username)
if !usernamePattern.MatchString(in.Username) || len(in.Password) < 8 {
c.JSON(400, gin.H{"error": "username must be 4-10 English letters and password must be 8+ characters"})
return
}
if in.Role == "" {
in.Role = RoleUser
}
if in.Role != RoleUser && in.Role != RolePrivileged && in.Role != RoleAdmin {
c.JSON(400, gin.H{"error": "invalid role"})
return
}
hash, err := hashPassword(in.Password)
if err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
admin := c.MustGet("user").(*User)
var u User
err = a.db.Transaction(func(tx *gorm.DB) error {
u = User{Username: in.Username, PasswordHash: hash, Role: in.Role, Status: UserActive}
if err := tx.Create(&u).Error; err != nil {
return err
}
if in.Role != RoleAdmin {
root := cleanWebDAVRoot(strings.TrimRight(a.cfg.WebDAVPath, "/") + "/" + u.Username)
if err := tx.Create(&WebDAVConfig{UserID: u.ID, Enabled: true, RootPath: root}).Error; err != nil {
return err
}
}
if err := tx.Create(&CreditAccount{UserID: u.ID, Balance: in.Credits}).Error; err != nil {
return err
}
for _, p := range defaultPermissions(in.Role) {
if err := tx.Create(&UserPermission{UserID: u.ID, Permission: p, Enabled: true}).Error; err != nil {
return err
}
}
if in.Credits > 0 {
if err := tx.Create(&CreditTransaction{UserID: u.ID, Type: "grant", Amount: in.Credits, BalanceAfter: in.Credits, ReferenceType: "admin", ReferenceID: strconv.FormatUint(uint64(c.MustGet("user").(*User).ID), 10), Reason: "initial credits"}).Error; err != nil {
return err
}
}
return nil
})
if err != nil {
c.JSON(400, gin.H{"error": err.Error()})
return
}
u.PasswordHash = ""
a.audit(admin.ID, "user.create", "user", strconv.FormatUint(uint64(u.ID), 10), "role="+u.Role)
c.JSON(201, u)
}
func (a *App) adminAdjustUserCredits(c *gin.Context) {
id, err := strconv.ParseUint(c.Param("id"), 10, 64)
if err != nil {
c.JSON(400, gin.H{"error": "invalid user id"})
return
}
var in struct {
Amount int64 `json:"amount"`
Reason string `json:"reason"`
}
if c.BindJSON(&in) != nil || in.Amount == 0 {
c.JSON(400, gin.H{"error": "amount must not be zero"})
return
}
operator := c.MustGet("user").(*User)
var account CreditAccount
err = a.db.Transaction(func(tx *gorm.DB) error {
var u User
if err := tx.First(&u, id).Error; err != nil {
return err
}
if err := tx.Where("user_id = ?", u.ID).First(&account).Error; err != nil {
if err != gorm.ErrRecordNotFound {
return err
}
account = CreditAccount{UserID: u.ID}
if err := tx.Create(&account).Error; err != nil {
return err
}
}
newBalance := account.Balance + in.Amount
if newBalance < account.Reserved {
return errors.New("balance cannot be lower than reserved credits")
}
account.Balance = newBalance
if err := tx.Save(&account).Error; err != nil {
return err
}
reason := strings.TrimSpace(in.Reason)
if reason == "" {
reason = "admin credit adjustment"
}
return tx.Create(&CreditTransaction{UserID: u.ID, Type: "adjust", Amount: in.Amount, BalanceAfter: account.Balance, ReferenceType: "admin", ReferenceID: strconv.FormatUint(uint64(operator.ID), 10), Reason: reason}).Error
})
if err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
c.JSON(404, gin.H{"error": "user not found"})
return
}
c.JSON(400, gin.H{"error": err.Error()})
return
}
a.audit(operator.ID, "user.credit.adjust", "user", strconv.FormatUint(id, 10), fmt.Sprintf("amount=%d;reason=%s", in.Amount, strings.TrimSpace(in.Reason)))
c.JSON(200, gin.H{"balance": account.Balance, "reserved": account.Reserved, "available": account.Balance - account.Reserved})
}
func (a *App) adminAudit(c *gin.Context) {
var logs []AuditLog
q := a.db.Order("id desc").Limit(200)
if action := c.Query("action"); action != "" {
q = q.Where("action = ?", action)
}
if err := q.Find(&logs).Error; err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
c.JSON(200, gin.H{"data": logs})
}
func (a *App) adminUpdateUser(c *gin.Context) {
id, err := strconv.ParseUint(c.Param("id"), 10, 64)
if err != nil {
c.Status(400)
return
}
var in struct {
Password *string `json:"password"`
Role *string `json:"role"`
Status *string `json:"status"`
}
if c.BindJSON(&in) != nil {
c.Status(400)
return
}
var u User
if a.db.First(&u, id).Error != nil {
c.Status(404)
return
}
if in.Password != nil {
if len(*in.Password) < 8 {
c.JSON(400, gin.H{"error": "password must be 8+"})
return
}
h, e := hashPassword(*in.Password)
if e != nil {
c.JSON(500, gin.H{"error": e.Error()})
return
}
u.PasswordHash = h
}
if in.Role != nil {
if *in.Role != RoleUser && *in.Role != RolePrivileged && *in.Role != RoleAdmin {
c.JSON(400, gin.H{"error": "invalid role"})
return
}
u.Role = *in.Role
}
if in.Status != nil {
if *in.Status != UserActive && *in.Status != UserDisabled {
c.JSON(400, gin.H{"error": "invalid status"})
return
}
u.Status = *in.Status
}
operator := c.MustGet("user").(*User)
if u.ID == operator.ID && (u.Role != RoleAdmin || u.Status == UserDisabled) {
var admins int64
if err := a.db.Model(&User{}).Where("role = ? AND status = ?", RoleAdmin, UserActive).Count(&admins).Error; err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
if admins <= 1 {
c.JSON(400, gin.H{"error": "cannot remove the last active admin account"})
return
}
}
if err := a.db.Transaction(func(tx *gorm.DB) error {
if err := tx.Save(&u).Error; err != nil {
return err
}
if in.Password != nil {
if err := tx.Where("user_id = ?", u.ID).Delete(&Session{}).Error; err != nil {
return err
}
}
if in.Role != nil {
if err := tx.Where("user_id = ?", u.ID).Delete(&UserPermission{}).Error; err != nil {
return err
}
for _, p := range defaultPermissions(u.Role) {
if err := tx.Create(&UserPermission{UserID: u.ID, Permission: p, Enabled: true}).Error; err != nil {
return err
}
}
}
return nil
}); err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
u.PasswordHash = ""
changes := []string{}
if in.Password != nil {
changes = append(changes, "password")
}
if in.Role != nil {
changes = append(changes, "role="+u.Role)
}
if in.Status != nil {
changes = append(changes, "status="+u.Status)
}
a.audit(operator.ID, "user.update", "user", strconv.FormatUint(uint64(u.ID), 10), strings.Join(changes, ","))
c.JSON(200, u)
}
func (a *App) adminUserCredits(c *gin.Context) {
id, err := strconv.ParseUint(c.Param("id"), 10, 64)
if err != nil {
c.Status(400)
return
}
var account CreditAccount
if err := a.db.Where("user_id = ?", id).First(&account).Error; err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
c.JSON(404, gin.H{"error": "credit account not found"})
return
}
c.JSON(500, gin.H{"error": err.Error()})
return
}
var tasks int64
if err := a.db.Model(&Task{}).Where("user_id = ?", id).Count(&tasks).Error; err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
c.JSON(200, gin.H{"balance": account.Balance, "reserved": account.Reserved, "available": account.Balance - account.Reserved, "tasks": tasks})
}
func (a *App) adminAdjustCredits(c *gin.Context) {
id, err := strconv.ParseUint(c.Param("id"), 10, 64)
if err != nil {
c.Status(400)
return
}
var in struct {
Amount int64 `json:"amount"`
Reason string `json:"reason"`
}
if c.BindJSON(&in) != nil || in.Amount == 0 {
c.JSON(400, gin.H{"error": "amount cannot be zero"})
return
}
operator := c.MustGet("user").(*User)
err = a.db.Transaction(func(tx *gorm.DB) error {
var account CreditAccount
if err := tx.Where("user_id = ?", id).First(&account).Error; err != nil {
return err
}
if in.Amount < 0 && account.Balance+in.Amount < account.Reserved {
return errors.New("cannot reduce balance below reserved credits")
}
account.Balance += in.Amount
if account.Balance < 0 {
return errors.New("insufficient balance")
}
if err := tx.Save(&account).Error; err != nil {
return err
}
typ, reason := "grant", strings.TrimSpace(in.Reason)
if reason == "" {
reason = "admin credit adjustment"
}
if in.Amount < 0 {
typ = "admin_debit"
if strings.TrimSpace(in.Reason) == "" {
reason = "admin credit deduction"
}
}
return tx.Create(&CreditTransaction{UserID: uint(id), Type: typ, Amount: in.Amount, BalanceAfter: account.Balance, ReferenceType: "admin", ReferenceID: strconv.FormatUint(uint64(operator.ID), 10), Reason: reason}).Error
})
if err != nil {
c.JSON(400, gin.H{"error": err.Error()})
return
}
a.audit(operator.ID, "user.credits.adjust", "user", strconv.FormatUint(id, 10), "amount="+strconv.FormatInt(in.Amount, 10))
var account CreditAccount
_ = a.db.Where("user_id = ?", id).First(&account)
c.JSON(200, gin.H{"ok": true, "balance": account.Balance, "reserved": account.Reserved, "available": account.Balance - account.Reserved})
}
+65
View File
@@ -0,0 +1,65 @@
package main
import (
"errors"
"strings"
"time"
"gorm.io/gorm"
)
type MediaAsset struct {
ID uint `gorm:"primaryKey" json:"id"`
AssetKey string `gorm:"uniqueIndex;size:512" json:"assetKey"`
Source string `gorm:"index;size:32" json:"source"`
RID string `gorm:"index;size:256" json:"rid"`
Quality string `gorm:"index;size:32" json:"quality"`
LocalPath string `json:"localPath,omitempty"`
Format string `json:"format,omitempty"`
ActualQuality string `json:"actualQuality,omitempty"`
Size int64 `json:"size"`
Status string `gorm:"index;size:20" json:"status"`
OwnerTaskID uint `gorm:"index" json:"ownerTaskId,omitempty"`
LeaseUntil time.Time `gorm:"index" json:"-"`
Generation uint64 `json:"generation"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
func assetKey(source, rid, quality string) string {
return strings.Join([]string{source, rid, quality}, "\x00")
}
func (a *App) ensureAsset(t *Task) (*MediaAsset, error) {
if t.AssetID != 0 {
var asset MediaAsset
if err := a.db.First(&asset, t.AssetID).Error; err == nil {
return &asset, nil
}
}
key := assetKey(t.Source, t.RID, t.Quality)
var asset MediaAsset
err := a.db.Where("asset_key = ?", key).First(&asset).Error
if errors.Is(err, gorm.ErrRecordNotFound) {
asset = MediaAsset{AssetKey: key, Source: t.Source, RID: t.RID, Quality: t.Quality, Status: "new"}
if err := a.db.Create(&asset).Error; err != nil {
if a.db.Where("asset_key = ?", key).First(&asset).Error != nil {
return nil, err
}
}
} else if err != nil {
return nil, err
}
t.AssetID = asset.ID
if err := a.db.Model(t).Update("asset_id", asset.ID).Error; err != nil {
return nil, err
}
return &asset, nil
}
// acquireAsset is the compatibility entry point.
func (a *App) acquireAsset(t *Task) (*MediaAsset, bool, error) { return a.claimAsset(t) }
func (a *App) markAssetDone(t *Task) { _ = a.finishAsset(t) }
func (a *App) releaseAsset(t *Task) { a.invalidateAsset(t) }
+236
View File
@@ -0,0 +1,236 @@
package main
import (
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"fmt"
"net/http"
"strings"
"sync"
"time"
"github.com/gin-gonic/gin"
"golang.org/x/crypto/bcrypt"
"gorm.io/gorm"
)
const sessionCookie = "am_session"
var loginLimiter = struct {
sync.Mutex
attempts map[string][]time.Time
}{attempts: map[string][]time.Time{}}
func loginAllowed(key string) bool {
now := time.Now()
loginLimiter.Lock()
defer loginLimiter.Unlock()
items := loginLimiter.attempts[key]
cutoff := now.Add(-15 * time.Minute)
kept := items[:0]
for _, t := range items {
if t.After(cutoff) {
kept = append(kept, t)
}
}
if len(kept) >= 10 {
return false
}
loginLimiter.attempts[key] = append(kept, now)
return true
}
func hashToken(token string) string {
sum := sha256.Sum256([]byte(token))
return hex.EncodeToString(sum[:])
}
func randomToken(n int) (string, error) {
b := make([]byte, n)
if _, err := rand.Read(b); err != nil {
return "", err
}
return hex.EncodeToString(b), nil
}
func hashPassword(password string) (string, error) {
b, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
return string(b), err
}
func checkPassword(hash, password string) error {
return bcrypt.CompareHashAndPassword([]byte(hash), []byte(password))
}
func (a *App) currentUser(c *gin.Context) *User {
value, ok := c.Get("user")
if !ok {
return nil
}
u, _ := value.(*User)
return u
}
func (a *App) requireAuth(c *gin.Context) {
token, err := c.Cookie(sessionCookie)
if err != nil || token == "" {
c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "authentication required"})
return
}
var session Session
if a.db.Where("token_hash = ? AND expires_at > ?", hashToken(token), time.Now()).First(&session).Error != nil {
c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "invalid or expired session"})
return
}
var user User
if a.db.First(&user, session.UserID).Error != nil || user.Status != UserActive {
c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "user unavailable"})
return
}
c.Set("user", &user)
c.Next()
}
func (a *App) requireAdmin(c *gin.Context) {
u := a.currentUser(c)
if u == nil || u.Role != RoleAdmin {
c.AbortWithStatusJSON(http.StatusForbidden, gin.H{"error": "admin access required"})
return
}
c.Next()
}
func (a *App) requirePermission(permission string) gin.HandlerFunc {
return func(c *gin.Context) {
u := a.currentUser(c)
if u == nil {
c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "authentication required"})
return
}
if u.Role == RoleAdmin || a.hasPermission(u.ID, permission) {
c.Next()
return
}
c.AbortWithStatusJSON(http.StatusForbidden, gin.H{"error": "permission denied"})
}
}
func (a *App) hasPermission(userID uint, permission string) bool {
var p UserPermission
return a.db.Where("user_id = ? AND permission = ? AND enabled = ?", userID, permission, true).First(&p).Error == nil
}
func (a *App) login(c *gin.Context) {
var in struct {
Username string `json:"username"`
Password string `json:"password"`
}
if c.BindJSON(&in) != nil || strings.TrimSpace(in.Username) == "" || in.Password == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "username and password required"})
return
}
key := c.ClientIP() + ":" + strings.ToLower(strings.TrimSpace(in.Username))
if !loginAllowed(key) {
c.JSON(http.StatusTooManyRequests, gin.H{"error": "too many login attempts; try again later"})
return
}
var user User
if a.db.Where("username = ?", strings.TrimSpace(in.Username)).First(&user).Error != nil || user.Status != UserActive || checkPassword(user.PasswordHash, in.Password) != nil {
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid username or password"})
return
}
token, err := randomToken(32)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "session generation failed"})
return
}
// Keep the session table bounded without requiring a background scheduler.
a.db.Where("expires_at <= ?", time.Now()).Delete(&Session{})
session := Session{UserID: user.ID, TokenHash: hashToken(token), ExpiresAt: time.Now().Add(7 * 24 * time.Hour)}
if err := a.db.Create(&session).Error; err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "session creation failed"})
return
}
http.SetCookie(c.Writer, &http.Cookie{Name: sessionCookie, Value: token, Path: "/", HttpOnly: true, SameSite: http.SameSiteLaxMode, Secure: c.Request.TLS != nil, Expires: session.ExpiresAt})
user.PasswordHash = ""
c.JSON(http.StatusOK, gin.H{"user": user})
}
func (a *App) logout(c *gin.Context) {
if token, err := c.Cookie(sessionCookie); err == nil && token != "" {
a.db.Where("token_hash = ?", hashToken(token)).Delete(&Session{})
}
http.SetCookie(c.Writer, &http.Cookie{Name: sessionCookie, Value: "", Path: "/", MaxAge: -1, HttpOnly: true, SameSite: http.SameSiteLaxMode})
c.Status(http.StatusNoContent)
}
func (a *App) changePassword(c *gin.Context) {
user := c.MustGet("user").(*User)
var in struct {
CurrentPassword string `json:"currentPassword"`
NewPassword string `json:"newPassword"`
}
if c.BindJSON(&in) != nil || in.CurrentPassword == "" || len(in.NewPassword) < 8 {
c.JSON(http.StatusBadRequest, gin.H{"error": "currentPassword and newPassword(8+) required"})
return
}
var stored User
if err := a.db.First(&stored, user.ID).Error; err != nil || checkPassword(stored.PasswordHash, in.CurrentPassword) != nil {
c.JSON(http.StatusUnauthorized, gin.H{"error": "current password is incorrect"})
return
}
hash, err := hashPassword(in.NewPassword)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "password hashing failed"})
return
}
if err := a.db.Transaction(func(tx *gorm.DB) error {
if err := tx.Model(&User{}).Where("id = ?", user.ID).Update("password_hash", hash).Error; err != nil {
return err
}
return tx.Where("user_id = ?", user.ID).Delete(&Session{}).Error
}); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "password change failed"})
return
}
http.SetCookie(c.Writer, &http.Cookie{Name: sessionCookie, Value: "", Path: "/", MaxAge: -1, HttpOnly: true, SameSite: http.SameSiteLaxMode})
a.audit(user.ID, "auth.password.change", "user", fmt.Sprint(user.ID), "password changed; sessions revoked")
c.Status(http.StatusNoContent)
}
func (a *App) me(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"user": a.currentUser(c)})
}
func (a *App) bootstrapAdmin() {
var count int64
a.db.Model(&User{}).Count(&count)
if count != 0 {
return
}
username := getenv("ADMIN_USERNAME", "admin")
password := getenv("ADMIN_PASSWORD", "")
if password == "" {
generated, err := randomToken(12)
if err != nil {
panic(err)
}
password = generated
fmt.Printf("AM Downloader initial admin password: %s\n", password)
}
hash, err := hashPassword(password)
if err != nil {
panic(err)
}
u := User{Username: username, PasswordHash: hash, Role: RoleAdmin, Status: UserActive}
if err := a.db.Create(&u).Error; err != nil {
panic(err)
}
if err := a.db.Model(&Task{}).Where("user_id = 0 OR user_id IS NULL").Update("user_id", u.ID).Error; err != nil {
panic(err)
}
if err := a.db.Create(&CreditAccount{UserID: u.ID}).Error; err != nil {
panic(err)
}
}
+255
View File
@@ -0,0 +1,255 @@
package main
import (
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"os"
"path/filepath"
"time"
"gorm.io/gorm"
)
const assetLease = 15 * time.Minute
var (
errTaskRunning = errors.New("task is already running")
errTaskAlreadyDone = errors.New("task is already completed")
)
func (a *App) claimTask(id uint) (*Task, error) {
result := a.db.Model(&Task{}).Where("id = ? AND status = ?", id, "pending").
Updates(map[string]any{"status": "resolving", "error": "", "progress": 0})
if result.Error != nil {
return nil, result.Error
}
if result.RowsAffected != 1 {
var t Task
if err := a.db.First(&t, id).Error; err != nil {
return nil, err
}
if t.Status == "upload_pending" {
return nil, fmt.Errorf("task %d is waiting for upload", id)
}
if t.Status == "done" {
return nil, errTaskAlreadyDone
}
return nil, errTaskRunning
}
var t Task
if err := a.db.First(&t, id).Error; err != nil {
return nil, err
}
return &t, nil
}
func (a *App) claimUpload(t *Task) bool {
result := a.db.Model(&Task{}).
Where("id = ? AND (status = ? OR (status = ? AND (remote_path = '' OR remote_path IS NULL)))",
t.ID, "upload_pending", "done").
Updates(map[string]any{"status": "uploading", "error": ""})
if result.Error != nil || result.RowsAffected != 1 {
return false
}
t.Status, t.Error = "uploading", ""
return true
}
func (a *App) taskReservationStatus(taskID, reservationID uint) (string, error) {
if reservationID == 0 {
return "", nil
}
var r CreditReservation
if err := a.db.First(&r, reservationID).Error; err != nil {
return "", err
}
if r.TaskID != taskID {
return "", fmt.Errorf("credit reservation does not belong to task")
}
return r.Status, nil
}
func (a *App) repairCommittedTask(t *Task) error {
status, err := a.taskReservationStatus(t.ID, t.CreditReservationID)
if err != nil {
return err
}
if status == "committed" {
if t.Status == "error" {
t.Status, t.Error = "upload_pending", "下载已完成,正在恢复 WebDAV 上传"
return a.db.Model(t).Updates(map[string]any{
"status": t.Status, "error": t.Error,
}).Error
}
return nil
}
return nil
}
func (a *App) markAssetHeartbeat(assetID, taskID uint) {
if assetID == 0 || taskID == 0 {
return
}
now := time.Now()
lease := now.Add(assetLease)
_ = a.db.Model(&MediaAsset{}).
Where("id = ? AND owner_task_id = ? AND status = ?", assetID, taskID, "downloading").
Updates(map[string]any{"lease_until": lease, "updated_at": now}).Error
}
func (a *App) assetLeaseExpired(asset *MediaAsset) bool {
return asset.Status == "downloading" && !asset.LeaseUntil.IsZero() && time.Now().After(asset.LeaseUntil)
}
func (a *App) claimAsset(t *Task) (*MediaAsset, bool, error) {
for i := 0; i < 180; i++ {
asset, err := a.ensureAsset(t)
if err != nil {
return nil, false, err
}
if asset.Status == "done" && asset.LocalPath != "" {
if fi, statErr := os.Stat(asset.LocalPath); statErr == nil && fi.Size() > 0 {
t.LocalPath, t.Format, t.ActualQuality = asset.LocalPath, asset.Format, asset.ActualQuality
t.TotalBytes, t.DownloadedBytes, t.Progress, t.Status = fi.Size(), fi.Size(), 100, "done"
if err := a.db.Save(t).Error; err != nil {
return nil, false, err
}
return asset, true, nil
}
}
claimed := false
now := time.Now()
lease := now.Add(assetLease)
err = a.transactionRetry(func(tx *gorm.DB) error {
var current MediaAsset
if err := tx.First(&current, asset.ID).Error; err != nil {
return err
}
if current.Status == "downloading" && current.OwnerTaskID != t.ID {
if !current.LeaseUntil.IsZero() && now.Before(current.LeaseUntil) {
return nil
}
if current.OwnerTaskID != 0 {
_ = tx.Model(&Task{}).Where("id = ? AND status IN ?", current.OwnerTaskID, []string{"resolving", "downloading"}).Updates(map[string]any{"status": "pending", "error": "共享资源下载锁超时,已自动恢复", "progress": 0})
}
}
wasEvicted := current.Status == "evicted"
wasStolen := current.Status == "downloading" && current.OwnerTaskID != 0 && current.OwnerTaskID != t.ID && (current.LeaseUntil.IsZero() || !now.Before(current.LeaseUntil))
current.Status = "downloading"
current.OwnerTaskID = t.ID
current.LeaseUntil = lease
current.Generation++
if current.LocalPath == "" || wasEvicted || wasStolen {
current.LocalPath = assetCachePath(a.cfg.DownloadDir, current.AssetKey, t.Format, current.Generation)
}
if err := tx.Save(&current).Error; err != nil {
return err
}
*asset = current
claimed = true
return nil
})
if err != nil {
return nil, false, err
}
if claimed {
t.LocalPath = asset.LocalPath
return asset, false, nil
}
time.Sleep(500 * time.Millisecond)
}
return nil, false, fmt.Errorf("timed out waiting for shared media asset")
}
func assetCachePath(root, key, format string, generation uint64) string {
sum := sha256.Sum256([]byte(key))
return filepath.Join(root, ".assets", hex.EncodeToString(sum[:]), fmt.Sprintf("asset-%d.%s", generation, normalizedFormat(format)))
}
func (a *App) assertAssetOwner(t *Task) bool {
if t.AssetID == 0 {
return true
}
var asset MediaAsset
if a.db.First(&asset, t.AssetID).Error != nil {
return false
}
return asset.OwnerTaskID == t.ID && asset.Status == "downloading"
}
func (a *App) finishAsset(t *Task) bool {
if t.AssetID == 0 {
return true
}
result := a.db.Model(&MediaAsset{}).
Where("id = ? AND owner_task_id = ? AND status = ?", t.AssetID, t.ID, "downloading").
Updates(map[string]any{
"status": "done", "owner_task_id": 0, "lease_until": nil,
"local_path": t.LocalPath, "format": t.Format,
"actual_quality": t.ActualQuality, "size": t.DownloadedBytes,
})
return result.Error == nil && result.RowsAffected == 1
}
func (a *App) invalidateAsset(t *Task) {
if t.AssetID == 0 {
return
}
_ = a.db.Model(&MediaAsset{}).
Where("id = ? AND owner_task_id = ?", t.AssetID, t.ID).
Updates(map[string]any{"status": "error", "owner_task_id": 0, "lease_until": nil})
}
func (a *App) prepareRetry(t *Task) error {
status, err := a.taskReservationStatus(t.ID, t.CreditReservationID)
if err != nil {
return err
}
switch status {
case "committed":
t.Status = "upload_pending"
t.Error = "下载已完成,恢复 WebDAV 上传"
return a.db.Model(t).Updates(map[string]any{
"status": "upload_pending", "error": t.Error,
}).Error
case "reserved":
t.Status, t.Error = "pending", ""
return a.db.Model(t).Updates(map[string]any{
"status": "pending", "error": "", "progress": 0,
}).Error
case "refunded", "":
r, reserveErr := a.reserveCredit(t.UserID, t.ID, 1)
if reserveErr != nil {
return reserveErr
}
t.CreditReservationID = r.ID
t.Status, t.Error, t.Progress, t.DownloadedBytes = "pending", "", 0, 0
return a.db.Model(t).Updates(map[string]any{
"status": "pending", "error": "", "progress": 0,
"downloaded_bytes": 0, "credit_reservation_id": r.ID,
}).Error
default:
return errReservationState
}
}
func (a *App) enqueueTask(id uint) {
select {
case a.queue <- id:
default:
go func() { a.queue <- id }()
}
}
func (a *App) transactionRetry(fn func(*gorm.DB) error) error {
var err error
for attempt := 0; attempt < 8; attempt++ {
err = a.db.Transaction(fn)
if !isSQLiteBusy(err) {
return err
}
time.Sleep(time.Duration(25*(1<<attempt)) * time.Millisecond)
}
return err
}
+223
View File
@@ -0,0 +1,223 @@
package main
import (
"os"
"path/filepath"
"strconv"
"sync"
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
"time"
)
func TestClaimTaskIsAtomic(t *testing.T) {
a := newCreditTestApp(t)
task := Task{UserID: 1, Source: "kw", RID: "atomic", Quality: "lossless", Status: "pending"}
if err := a.db.Create(&task).Error; err != nil {
t.Fatal(err)
}
if _, err := a.claimTask(task.ID); err != nil {
t.Fatal(err)
}
if _, err := a.claimTask(task.ID); err != errTaskRunning {
t.Fatalf("second claim error = %v, want task running", err)
}
}
func TestRetryCommittedReservationDoesNotChargeAgain(t *testing.T) {
a := newCreditTestApp(t)
user := User{Username: "retry-committed", Role: RoleUser, Status: UserActive}
if err := a.db.Create(&user).Error; err != nil {
t.Fatal(err)
}
if err := a.db.Create(&CreditAccount{UserID: user.ID, Balance: 1}).Error; err != nil {
t.Fatal(err)
}
task := Task{UserID: user.ID, Source: "kw", RID: "committed", Quality: "lossless", Status: "pending"}
if err := a.db.Create(&task).Error; err != nil {
t.Fatal(err)
}
r, err := a.reserveCredit(user.ID, task.ID, 1)
if err != nil {
t.Fatal(err)
}
task.CreditReservationID = r.ID
if err := a.db.Model(&task).Update("credit_reservation_id", r.ID).Error; err != nil {
t.Fatal(err)
}
if err := a.commitCredit(&task); err != nil {
t.Fatal(err)
}
if err := a.db.Model(&task).Updates(map[string]any{"status": "error", "error": "legacy"}).Error; err != nil {
t.Fatal(err)
}
var current Task
if err := a.db.First(&current, task.ID).Error; err != nil {
t.Fatal(err)
}
if err := a.prepareRetry(&current); err != nil {
t.Fatal(err)
}
var account CreditAccount
if err := a.db.Where("user_id = ?", user.ID).First(&account).Error; err != nil {
t.Fatal(err)
}
if account.Balance != 0 || account.Reserved != 0 {
t.Fatalf("credit changed on committed retry: %+v", account)
}
if current.Status != "upload_pending" {
t.Fatalf("status = %q, want upload_pending", current.Status)
}
}
func TestRetryRefundedReservationCreatesNewReservation(t *testing.T) {
a := newCreditTestApp(t)
user := User{Username: "retry-refunded", Role: RoleUser, Status: UserActive}
if err := a.db.Create(&user).Error; err != nil {
t.Fatal(err)
}
if err := a.db.Create(&CreditAccount{UserID: user.ID, Balance: 1}).Error; err != nil {
t.Fatal(err)
}
task := Task{UserID: user.ID, Source: "kw", RID: "refunded", Quality: "lossless", Status: "error"}
if err := a.db.Create(&task).Error; err != nil {
t.Fatal(err)
}
r1, err := a.reserveCredit(user.ID, task.ID, 1)
if err != nil {
t.Fatal(err)
}
task.CreditReservationID = r1.ID
if err := a.refundCredit(&task); err != nil {
t.Fatal(err)
}
if err := a.prepareRetry(&task); err != nil {
t.Fatal(err)
}
if task.CreditReservationID == r1.ID {
t.Fatal("retry reused refunded reservation")
}
var account CreditAccount
if err := a.db.Where("user_id = ?", user.ID).First(&account).Error; err != nil {
t.Fatal(err)
}
if account.Balance != 1 || account.Reserved != 1 {
t.Fatalf("unexpected account: %+v", account)
}
}
func TestStaleAssetLeaseStealsSafely(t *testing.T) {
a := newCreditTestApp(t)
root := t.TempDir()
a.cfg.DownloadDir = root
first := Task{UserID: 1, Source: "kw", RID: "lease", Quality: "lossless", Format: "flac", Status: "downloading"}
second := Task{UserID: 2, Source: "kw", RID: "lease", Quality: "lossless", Format: "flac", Status: "pending"}
if err := a.db.Create(&first).Error; err != nil {
t.Fatal(err)
}
if err := a.db.Create(&second).Error; err != nil {
t.Fatal(err)
}
asset := MediaAsset{
AssetKey: assetKey("kw", "lease", "lossless"),
Source: "kw", RID: "lease", Quality: "lossless",
LocalPath: filepath.Join(root, "old.flac"), Format: "flac",
Status: "downloading", OwnerTaskID: first.ID,
LeaseUntil: time.Now().Add(-time.Minute), Generation: 1,
}
if err := a.db.Create(&asset).Error; err != nil {
t.Fatal(err)
}
first.AssetID = asset.ID
if err := a.db.Model(&first).Update("asset_id", asset.ID).Error; err != nil {
t.Fatal(err)
}
got, reused, err := a.acquireAsset(&second)
if err != nil || reused {
t.Fatalf("acquire = asset=%+v reused=%v err=%v", got, reused, err)
}
if got.Generation != 2 {
t.Fatalf("generation = %d, want 2", got.Generation)
}
if got.LocalPath == asset.LocalPath {
t.Fatal("stolen generation reused old local path")
}
var old Task
if err := a.db.First(&old, first.ID).Error; err != nil {
t.Fatal(err)
}
if old.Status != "pending" {
t.Fatalf("stale owner status = %q, want pending", old.Status)
}
}
func TestAssetCachePathIsStablePerGeneration(t *testing.T) {
root := t.TempDir()
p1 := assetCachePath(root, "kw-rid-lossless", "flac", 1)
p2 := assetCachePath(root, "kw-rid-lossless", "flac", 2)
if p1 == p2 || filepath.Dir(p1) != filepath.Dir(p2) {
t.Fatalf("unexpected cache paths: %q %q", p1, p2)
}
if err := os.MkdirAll(filepath.Dir(p1), 0755); err != nil {
t.Fatal(err)
}
}
func TestConcurrentCreditReservationsAreAtomic(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "credits.db")
db, err := gorm.Open(sqlite.Open(dbPath+"?_pragma=journal_mode(WAL)&_pragma=busy_timeout(10000)"), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
if err := db.AutoMigrate(&Task{}, &MediaAsset{}, &User{}, &CreditAccount{}, &CreditTransaction{}, &CreditReservation{}); err != nil {
t.Fatal(err)
}
app := &App{db: db}
user := User{Username: "concurrent-credit", Role: RoleUser, Status: UserActive}
if err := db.Create(&user).Error; err != nil {
t.Fatal(err)
}
if err := db.Create(&CreditAccount{UserID: user.ID, Balance: 5}).Error; err != nil {
t.Fatal(err)
}
for i := 0; i < 20; i++ {
if err := db.Create(&Task{UserID: user.ID, Source: "kw", RID: "c-" + strconv.Itoa(i), Quality: "lossless", Status: "pending"}).Error; err != nil {
t.Fatal(err)
}
}
var wg sync.WaitGroup
var mu sync.Mutex
success, unexpected := 0, 0
for i := 0; i < 20; i++ {
var task Task
if err := db.Where("user_id = ? AND r_id = ?", user.ID, "c-"+strconv.Itoa(i)).First(&task).Error; err != nil {
t.Fatal(err)
}
wg.Add(1)
go func(taskID uint) {
defer wg.Done()
if _, err := app.reserveCredit(user.ID, taskID, 1); err == nil {
mu.Lock()
success++
mu.Unlock()
} else if err != errInsufficientCredits {
mu.Lock()
unexpected++
mu.Unlock()
}
}(task.ID)
}
wg.Wait()
if success != 5 || unexpected != 0 {
t.Fatalf("reservations: success=%d unexpected=%d", success, unexpected)
}
var account CreditAccount
if err := db.Where("user_id = ?", user.ID).First(&account).Error; err != nil {
t.Fatal(err)
}
if account.Balance != 5 || account.Reserved != 5 {
t.Fatalf("account after concurrent reservations: %+v", account)
}
}
+167
View File
@@ -0,0 +1,167 @@
package main
import (
"errors"
"fmt"
"strconv"
"strings"
"time"
"gorm.io/gorm"
)
var (
errInsufficientCredits = errors.New("insufficient credits")
errReservationState = errors.New("invalid credit reservation state")
)
// reserveCredit places one credit on hold for a task. The hold is released on
// failure or converted into a debit after a successful new download.
func reserveCreditTx(tx *gorm.DB, userID, taskID uint, amount int64) (*CreditReservation, error) {
if amount <= 0 {
return nil, fmt.Errorf("invalid credit amount")
}
var account CreditAccount
if err := tx.Where("user_id = ?", userID).First(&account).Error; err != nil {
if !errors.Is(err, gorm.ErrRecordNotFound) {
return nil, err
}
account = CreditAccount{UserID: userID}
if err := tx.Create(&account).Error; err != nil {
if err := tx.Where("user_id = ?", userID).First(&account).Error; err != nil {
return nil, err
}
}
}
var existing CreditReservation
if err := tx.Where("task_id = ? AND status = ?", taskID, "reserved").
Order("id desc").First(&existing).Error; err == nil {
return &existing, nil
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
return nil, err
}
if account.Balance-account.Reserved < amount {
return nil, errInsufficientCredits
}
account.Reserved += amount
if err := tx.Save(&account).Error; err != nil {
return nil, err
}
reservation := &CreditReservation{
UserID: userID, TaskID: taskID, Amount: amount, Status: "reserved",
}
if err := tx.Create(reservation).Error; err != nil {
return nil, err
}
if err := tx.Create(&CreditTransaction{
UserID: userID, Type: "reserve", Amount: 0,
BalanceAfter: account.Balance, ReferenceType: "task",
ReferenceID: strconv.FormatUint(uint64(taskID), 10),
Reason: "download reservation",
}).Error; err != nil {
return nil, err
}
return reservation, nil
}
func isSQLiteBusy(err error) bool {
if err == nil {
return false
}
msg := strings.ToLower(err.Error())
return strings.Contains(msg, "sqlite_busy") || strings.Contains(msg, "database is locked")
}
func (a *App) reserveCredit(userID, taskID uint, amount int64) (*CreditReservation, error) {
var reservation *CreditReservation
err := a.transactionRetry(func(tx *gorm.DB) error {
var txErr error
reservation, txErr = reserveCreditTx(tx, userID, taskID, amount)
return txErr
})
return reservation, err
}
func (a *App) commitCredit(task *Task) error {
if task.CreditReservationID == 0 {
return nil
}
return a.transactionRetry(func(tx *gorm.DB) error {
var r CreditReservation
if err := tx.First(&r, task.CreditReservationID).Error; err != nil {
return err
}
if r.Status == "committed" {
return nil
}
if r.Status != "reserved" {
return errReservationState
}
var account CreditAccount
if err := tx.Where("user_id = ?", r.UserID).First(&account).Error; err != nil {
return err
}
account.Reserved -= r.Amount
account.Balance -= r.Amount
if account.Reserved < 0 || account.Balance < 0 {
return errReservationState
}
if err := tx.Save(&account).Error; err != nil {
return err
}
now := time.Now()
r.Status, r.CompletedAt = "committed", &now
if err := tx.Save(&r).Error; err != nil {
return err
}
return tx.Create(&CreditTransaction{UserID: r.UserID, Type: "debit", Amount: -r.Amount, BalanceAfter: account.Balance, ReferenceType: "task", ReferenceID: strconv.FormatUint(uint64(task.ID), 10), Reason: "successful download"}).Error
})
}
// refundCredit releases a reservation without debiting the user's balance.
// It is idempotent so download failure paths can safely call fail repeatedly.
func (a *App) refundCredit(task *Task) error {
if task.CreditReservationID == 0 {
return nil
}
return a.transactionRetry(func(tx *gorm.DB) error {
var r CreditReservation
if err := tx.First(&r, task.CreditReservationID).Error; err != nil {
return err
}
if r.Status == "refunded" {
return nil
}
if r.Status == "committed" {
// A committed reservation belongs to a completed download. A later
// storage/retry error must never debit or refund it a second time.
return nil
}
if r.Status != "reserved" {
return errReservationState
}
var account CreditAccount
if err := tx.Where("user_id = ?", r.UserID).First(&account).Error; err != nil {
return err
}
account.Reserved -= r.Amount
if account.Reserved < 0 {
return errReservationState
}
if err := tx.Save(&account).Error; err != nil {
return err
}
now := time.Now()
r.Status, r.CompletedAt = "refunded", &now
if err := tx.Save(&r).Error; err != nil {
return err
}
return tx.Create(&CreditTransaction{UserID: r.UserID, Type: "refund", Amount: 0, BalanceAfter: account.Balance, ReferenceType: "task", ReferenceID: strconv.FormatUint(uint64(task.ID), 10), Reason: "download failed"}).Error
})
}
func (a *App) creditBalance(userID uint) (CreditAccount, error) {
var account CreditAccount
err := a.db.Where("user_id = ?", userID).First(&account).Error
return account, err
}
+125
View File
@@ -0,0 +1,125 @@
package main
import (
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
)
func newCreditTestApp(t *testing.T) *App {
t.Helper()
db, err := gorm.Open(sqlite.Open("file::memory:?cache=shared"), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
if err := db.AutoMigrate(&Task{}, &MediaAsset{}, &User{}, &CreditAccount{}, &CreditTransaction{}, &CreditReservation{}); err != nil {
t.Fatal(err)
}
return &App{db: db}
}
func TestCreditReservationRetryAfterRefund(t *testing.T) {
a := newCreditTestApp(t)
u := User{Username: "u", Role: RoleUser, Status: UserActive}
if err := a.db.Create(&u).Error; err != nil {
t.Fatal(err)
}
if err := a.db.Create(&CreditAccount{UserID: u.ID, Balance: 1}).Error; err != nil {
t.Fatal(err)
}
task := Task{UserID: u.ID, Source: "kw", RID: "1", Quality: "standard", Status: "error"}
if err := a.db.Create(&task).Error; err != nil {
t.Fatal(err)
}
r1, err := a.reserveCredit(u.ID, task.ID, 1)
if err != nil {
t.Fatal(err)
}
task.CreditReservationID = r1.ID
if err := a.refundCredit(&task); err != nil {
t.Fatal(err)
}
r2, err := a.reserveCredit(u.ID, task.ID, 1)
if err != nil {
t.Fatal(err)
}
if r2.ID == r1.ID {
t.Fatal("retry reused refunded reservation")
}
var account CreditAccount
if err := a.db.Where("user_id = ?", u.ID).First(&account).Error; err != nil {
t.Fatal(err)
}
if account.Balance != 1 || account.Reserved != 1 {
t.Fatalf("unexpected account: %+v", account)
}
}
func TestSharedAssetIsReusedAcrossUsers(t *testing.T) {
a := newCreditTestApp(t)
u1 := User{Username: "a1", Role: RoleUser, Status: UserActive}
u2 := User{Username: "a2", Role: RoleUser, Status: UserActive}
if err := a.db.Create(&u1).Error; err != nil {
t.Fatal(err)
}
if err := a.db.Create(&u2).Error; err != nil {
t.Fatal(err)
}
t1 := Task{UserID: u1.ID, Source: "kw", RID: "same", Quality: "standard", Title: "Song", Artist: "Artist"}
t2 := Task{UserID: u2.ID, Source: "kw", RID: "same", Quality: "standard", Title: "Song", Artist: "Artist"}
if err := a.db.Create(&t1).Error; err != nil {
t.Fatal(err)
}
if err := a.db.Create(&t2).Error; err != nil {
t.Fatal(err)
}
asset, reused, err := a.acquireAsset(&t1)
if err != nil || reused {
t.Fatalf("first acquire: asset=%+v reused=%v err=%v", asset, reused, err)
}
if asset.ID == 0 {
t.Fatal("missing asset id")
}
if err := a.db.Model(&MediaAsset{}).Where("id = ?", asset.ID).Updates(map[string]any{"status": "done", "owner_task_id": 0, "local_path": "/etc/hosts", "format": "mp3", "size": 123}).Error; err != nil {
t.Fatal(err)
}
t2.AssetID = asset.ID
asset2, reused, err := a.acquireAsset(&t2)
if err != nil || !reused || asset2.ID != asset.ID {
t.Fatalf("second acquire: asset=%+v reused=%v err=%v", asset2, reused, err)
}
}
func TestCommittedCreditIsNotRefunded(t *testing.T) {
a := newCreditTestApp(t)
u := User{Username: "u2", Role: RoleUser, Status: UserActive}
if err := a.db.Create(&u).Error; err != nil {
t.Fatal(err)
}
if err := a.db.Create(&CreditAccount{UserID: u.ID, Balance: 1}).Error; err != nil {
t.Fatal(err)
}
task := Task{UserID: u.ID, Source: "kw", RID: "2", Quality: "standard", Status: "pending"}
if err := a.db.Create(&task).Error; err != nil {
t.Fatal(err)
}
r, err := a.reserveCredit(u.ID, task.ID, 1)
if err != nil {
t.Fatal(err)
}
task.CreditReservationID = r.ID
if err := a.commitCredit(&task); err != nil {
t.Fatal(err)
}
if err := a.refundCredit(&task); err != nil {
t.Fatal(err)
}
var account CreditAccount
if err := a.db.Where("user_id = ?", u.ID).First(&account).Error; err != nil {
t.Fatal(err)
}
if account.Balance != 0 || account.Reserved != 0 {
t.Fatalf("unexpected account after committed refund: %+v", account)
}
}
+123 -28
View File
@@ -7,6 +7,7 @@ import (
"os"
"path/filepath"
"strings"
"time"
)
func (a *App) startWorkers() {
@@ -15,21 +16,36 @@ func (a *App) startWorkers() {
go func() {
defer a.wg.Done()
for id := range a.queue {
a.download(id)
a.processTask(id)
}
}()
}
}
func (a *App) download(id uint) {
func (a *App) processTask(id uint) {
var t Task
if a.db.First(&t, id).Error != nil || t.Status != "pending" {
if a.db.First(&t, id).Error != nil {
return
}
a.db.Model(&t).Updates(map[string]any{"status": "resolving", "error": "", "progress": 0})
if t.Status == "upload_pending" {
a.upload(&t)
return
}
if t.Status != "pending" {
return
}
a.download(id)
}
func (a *App) download(id uint) {
t, err := a.claimTask(id)
if err != nil {
return
}
info, err := a.am.Resolve(t.Source, t.RID, t.Quality)
if err != nil {
a.fail(&t, err)
a.fail(t, err)
return
}
@@ -40,18 +56,31 @@ func (a *App) download(id uint) {
if ext == "" {
ext = "mp3"
}
filename := safe(t.Artist) + " - " + safe(t.Title) + "." + ext
local := filepath.Join(a.cfg.DownloadDir, filename)
t.Format, t.ResolvedQuality, t.ActualQuality, t.LocalPath = ext, info.Level, "", local
// Reuse only the path recorded on this task. Never borrow another task's quality.
if t.LocalPath != "" {
if fi, statErr := os.Stat(t.LocalPath); statErr == nil && fi.Size() > 0 {
a.finishExisting(&t, fi.Size())
t.Format, t.ResolvedQuality, t.ActualQuality = ext, info.Level, ""
if err := a.db.Model(t).Updates(map[string]any{
"format": t.Format, "resolved_quality": t.ResolvedQuality,
}).Error; err != nil {
a.fail(t, err)
return
}
asset, reused, err := a.acquireAsset(t)
if err != nil {
a.fail(t, err)
return
}
a.downloadFile(&t, info)
if reused {
a.finishExisting(t, asset.Size)
return
}
if err := a.db.Model(t).Updates(map[string]any{
"local_path": t.LocalPath, "status": "downloading",
}).Error; err != nil {
a.invalidateAsset(t)
a.fail(t, err)
return
}
a.downloadFile(t, info)
}
func (a *App) finishExisting(t *Task, size int64) {
@@ -70,12 +99,17 @@ func (a *App) finishExisting(t *Task, size int64) {
t.ActualQuality = actualQuality(t.Quality, t.ResolvedQuality, actual)
}
t.TotalBytes, t.DownloadedBytes, t.Progress, t.Status = size, size, 100, "done"
a.db.Save(t)
if err := a.db.Save(t).Error; err != nil {
return
}
if err := a.refundCredit(t); err != nil {
a.fail(t, err)
return
}
a.upload(t)
}
func (a *App) downloadFile(t *Task, info playInfo) {
a.db.Model(t).Updates(map[string]any{"status": "downloading", "format": t.Format, "local_path": t.LocalPath})
req, err := http.NewRequest("GET", info.URL, nil)
if err != nil {
a.fail(t, err)
@@ -94,27 +128,62 @@ func (a *App) downloadFile(t *Task, info playInfo) {
a.fail(t, fmt.Errorf("media http %d", resp.StatusCode))
return
}
t.Status, t.TotalBytes, t.DownloadedBytes, t.Progress = "downloading", resp.ContentLength, 0, 0
a.db.Save(t)
tmp := t.LocalPath + ".part"
t.Status = "downloading"
t.TotalBytes = resp.ContentLength
t.DownloadedBytes = 0
t.Progress = 0
_ = a.db.Model(t).Updates(map[string]any{
"status": t.Status, "total_bytes": t.TotalBytes,
"downloaded_bytes": 0, "progress": 0,
}).Error
if err := os.MkdirAll(filepath.Dir(t.LocalPath), 0755); err != nil {
a.fail(t, err)
return
}
tmp := fmt.Sprintf("%s.task-%d.part", t.LocalPath, t.ID)
_ = os.Remove(tmp)
f, err := os.Create(tmp)
if err != nil {
a.fail(t, err)
return
}
n, err := a.copyWithProgress(f, resp.Body, t)
stop := make(chan struct{})
ticker := time.NewTicker(30 * time.Second)
go func() {
for {
select {
case <-ticker.C:
a.markAssetHeartbeat(t.AssetID, t.ID)
case <-stop:
return
}
}
}()
n, copyErr := a.copyWithProgress(f, resp.Body, t)
closeErr := f.Close()
if err != nil {
a.fail(t, err)
ticker.Stop()
close(stop)
if copyErr != nil {
a.fail(t, copyErr)
_ = os.Remove(tmp)
return
}
if closeErr != nil {
a.fail(t, closeErr)
_ = os.Remove(tmp)
return
}
if t.TotalBytes > 0 && n != t.TotalBytes {
a.fail(t, fmt.Errorf("incomplete download: got %d of %d bytes", n, t.TotalBytes))
_ = os.Remove(tmp)
return
}
if !a.assertAssetOwner(t) {
_ = os.Remove(tmp)
return
}
@@ -126,19 +195,40 @@ func (a *App) downloadFile(t *Task, info playInfo) {
}
actual = normalizedFormat(actual)
finalPath := strings.TrimSuffix(t.LocalPath, filepath.Ext(t.LocalPath)) + "." + actual
if err = os.Rename(tmp, finalPath); err != nil {
if err := os.Rename(tmp, finalPath); err != nil {
a.fail(t, err)
_ = os.Remove(tmp)
return
}
t.Format, t.LocalPath, t.ActualQuality, t.DownloadedBytes = actual, finalPath, actualQuality(t.Quality, t.ResolvedQuality, actual), n
if t.TotalBytes > 0 {
t.LocalPath = finalPath
t.Format = actual
t.ActualQuality = actualQuality(t.Quality, t.ResolvedQuality, actual)
t.DownloadedBytes = n
t.Progress = 100
if err := a.db.Model(t).Updates(map[string]any{
"format": t.Format, "local_path": t.LocalPath,
"actual_quality": t.ActualQuality,
"downloaded_bytes": n, "progress": 100,
}).Error; err != nil {
a.invalidateAsset(t)
return
}
if !a.finishAsset(t) {
a.fail(t, fmt.Errorf("shared asset ownership lost before completion"))
return
}
if err := a.commitCredit(t); err != nil {
a.fail(t, err)
return
}
t.Status, t.Error = "upload_pending", ""
if err := a.db.Model(t).Updates(map[string]any{
"status": "upload_pending", "error": "",
}).Error; err != nil {
return
}
a.db.Save(t)
a.upload(t)
}
func (a *App) copyWithProgress(dst *os.File, src io.Reader, t *Task) (int64, error) {
var n, last int64
buf := make([]byte, 256*1024)
@@ -156,7 +246,12 @@ func (a *App) copyWithProgress(dst *os.File, src io.Reader, t *Task) (int64, err
progress = float64(n) * 100 / float64(t.TotalBytes)
}
t.DownloadedBytes, t.Progress = n, progress
a.db.Model(t).Updates(map[string]any{"downloaded_bytes": n, "progress": progress})
if err := a.db.Model(t).Updates(map[string]any{
"downloaded_bytes": n, "progress": progress,
}).Error; err != nil {
return n, err
}
a.markAssetHeartbeat(t.AssetID, t.ID)
last = n
}
}
+103 -14
View File
@@ -6,6 +6,7 @@ import (
"strings"
"github.com/gin-gonic/gin"
"gorm.io/gorm"
)
func (a *App) search(c *gin.Context) {
@@ -35,7 +36,12 @@ func (a *App) search(c *gin.Context) {
func (a *App) listTasks(c *gin.Context) {
var ts []Task
if err := a.db.Order("id desc").Limit(100).Find(&ts).Error; err != nil {
user := c.MustGet("user").(*User)
query := a.db.Where("user_id = ?", user.ID)
if user.Role == RoleAdmin && c.Query("all") == "1" {
query = a.db
}
if err := query.Order("id desc").Limit(100).Find(&ts).Error; err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
@@ -54,45 +60,116 @@ func (a *App) addTask(c *gin.Context) {
if in.Quality == "" {
in.Quality = a.cfg.DefaultQuality
}
user := c.MustGet("user").(*User)
var existing Task
err := a.db.Where("source = ? AND r_id = ? AND quality = ?", in.Source, in.RID, in.Quality).Order("id desc").First(&existing).Error
err := a.db.Where("user_id = ? AND source = ? AND r_id = ? AND quality = ?",
user.ID, in.Source, in.RID, in.Quality).Order("id desc").First(&existing).Error
if err == nil {
if existing.Status != "error" {
existing.Title, existing.Artist, existing.Album, existing.Pic = in.Title, in.Artist, in.Album, in.Pic
switch existing.Status {
case "pending", "resolving", "downloading", "uploading", "upload_pending":
c.JSON(200, existing)
return
case "done":
c.JSON(200, existing)
return
case "error":
if err := a.prepareRetry(&existing); err != nil {
if err == errInsufficientCredits {
c.JSON(402, gin.H{"error": err.Error()})
} else {
c.JSON(409, gin.H{"error": err.Error()})
}
a.db.Model(&existing).Updates(map[string]any{"title": in.Title, "artist": in.Artist, "album": in.Album, "pic": in.Pic, "status": "pending", "error": ""})
a.queue <- existing.ID
return
}
a.enqueueTask(existing.ID)
c.JSON(202, existing)
return
}
t := Task{Source: in.Source, RID: in.RID, Title: in.Title, Artist: in.Artist, Album: in.Album, Pic: in.Pic, Quality: in.Quality, Status: "pending"}
if err := a.db.Create(&t).Error; err != nil {
if a.db.Where("source = ? AND r_id = ? AND quality = ?", in.Source, in.RID, in.Quality).Order("id desc").First(&existing).Error == nil {
}
t := Task{
UserID: user.ID, Source: in.Source, RID: in.RID,
Title: in.Title, Artist: in.Artist, Album: in.Album,
Pic: in.Pic, Quality: in.Quality, Status: "pending",
}
var reservation *CreditReservation
err = a.transactionRetry(func(tx *gorm.DB) error {
if err := tx.Create(&t).Error; err != nil {
return err
}
var reserveErr error
reservation, reserveErr = reserveCreditTx(tx, user.ID, t.ID, 1)
if reserveErr != nil {
return reserveErr
}
t.CreditReservationID = reservation.ID
return tx.Model(&t).Update("credit_reservation_id", reservation.ID).Error
})
if err != nil {
if a.db.Where("user_id = ? AND source = ? AND r_id = ? AND quality = ?",
user.ID, in.Source, in.RID, in.Quality).First(&existing).Error == nil {
c.JSON(200, existing)
return
}
if err == errInsufficientCredits {
c.JSON(402, gin.H{"error": err.Error()})
return
}
c.JSON(500, gin.H{"error": err.Error()})
return
}
a.queue <- t.ID
a.enqueueTask(t.ID)
c.JSON(202, t)
}
func (a *App) creditInfo(c *gin.Context) {
user := c.MustGet("user").(*User)
account, err := a.creditBalance(user.ID)
if err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
c.JSON(200, gin.H{"balance": account.Balance, "reserved": account.Reserved, "available": account.Balance - account.Reserved})
}
func (a *App) retryTask(c *gin.Context) {
id, err := strconv.ParseUint(c.Param("id"), 10, 64)
if err != nil {
c.Status(400)
return
}
user := c.MustGet("user").(*User)
var t Task
if a.db.First(&t, id).Error != nil {
if a.db.Where("id = ? AND user_id = ?", id, user.ID).First(&t).Error != nil {
c.Status(404)
return
}
a.db.Model(&t).Updates(map[string]any{"status": "pending", "error": "", "progress": 0, "downloaded_bytes": 0})
t.Status, t.Error, t.Progress, t.DownloadedBytes = "pending", "", 0, 0
a.queue <- t.ID
switch t.Status {
case "done":
c.JSON(409, gin.H{"error": "task is already completed"})
return
case "pending", "resolving", "downloading", "uploading":
c.JSON(409, gin.H{"error": "task is already running"})
return
case "upload_pending", "error":
if t.Status == "error" {
if err := a.prepareRetry(&t); err != nil {
if err == errInsufficientCredits {
c.JSON(402, gin.H{"error": err.Error()})
} else {
c.JSON(409, gin.H{"error": err.Error()})
}
return
}
}
default:
c.JSON(409, gin.H{"error": "task cannot be retried from current state"})
return
}
a.enqueueTask(t.ID)
c.JSON(202, t)
}
@@ -102,7 +179,19 @@ func (a *App) deleteTask(c *gin.Context) {
c.Status(400)
return
}
if err := a.db.Delete(&Task{}, id).Error; err != nil {
user := c.MustGet("user").(*User)
var t Task
if err := a.db.Where("id = ? AND user_id = ?", id, user.ID).First(&t).Error; err != nil {
c.Status(404)
return
}
if t.CreditReservationID != 0 {
if err := a.refundCredit(&t); err != nil {
c.JSON(409, gin.H{"error": "cannot release task credit: " + err.Error()})
return
}
}
if err := a.db.Delete(&t).Error; err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
+1439 -17
View File
File diff suppressed because it is too large Load Diff
+49 -10
View File
@@ -13,32 +13,71 @@ import (
func main() {
dbPath := getenv("DB_PATH", "/data/downloader.db")
_ = os.MkdirAll(filepath.Dir(dbPath), 0755)
db, err := gorm.Open(sqlite.Open(dbPath), &gorm.Config{})
dsn := dbPath + "?_pragma=journal_mode(WAL)&_pragma=busy_timeout(10000)"
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{})
must(err)
must(db.AutoMigrate(&Task{}, &Setting{}))
sqlDB, err := db.DB()
must(err)
sqlDB.SetMaxOpenConns(8)
sqlDB.SetMaxIdleConns(8)
must(migrateDatabase(db))
client := &http.Client{Timeout: 2 * 60 * 1e9}
cfg := loadConfig(db)
a := &App{db: db, cfg: cfg, queue: make(chan uint, 100), httpClient: client}
a.loadSettings()
a.bootstrapAdmin()
must(a.initializeUserWebDAVDefaults())
must(backfillMediaAssets(db, a.cfg.DownloadDir))
if os.Getenv("MIGRATION_CHECK") == "1" {
var users, tasks, assets int64
db.Model(&User{}).Count(&users)
db.Model(&Task{}).Count(&tasks)
db.Model(&MediaAsset{}).Count(&assets)
println("migration check ok", "users", users, "tasks", tasks, "assets", assets)
return
}
normalizeConfig(&a.cfg)
a.am = &AMAPIClient{BaseURL: a.cfg.AMAPI, HTTP: client}
if a.cfg.WebDAVURL != "" {
a.storage = NewWebDAVStorage(a.cfg)
}
a.startWorkers()
a.startWebDAVResync()
r := gin.Default()
r.StaticFile("/", "/app/index.html")
api := r.Group("/api")
api.GET("/search", a.search)
api.GET("/tasks", a.listTasks)
api.POST("/tasks", a.addTask)
api.POST("/tasks/:id/retry", a.retryTask)
api.DELETE("/tasks/:id", a.deleteTask)
api.GET("/settings", a.getSettings)
api.PUT("/settings", a.putSettings)
api.POST("/webdav/test", a.testWebDAV)
api.POST("/auth/login", a.login)
api.POST("/auth/logout", a.requireAuth, a.logout)
api.POST("/auth/password", a.requireAuth, a.changePassword)
api.GET("/auth/me", a.requireAuth, a.me)
auth := api.Group("", a.requireAuth)
auth.GET("/search", a.requirePermission(PermDownload), a.search)
auth.GET("/tasks", a.requirePermission(PermDownload), a.listTasks)
auth.GET("/credits", a.creditInfo)
auth.POST("/redeem", a.requirePermission(PermRedeem), a.redeem)
auth.GET("/webdav/config", a.requirePermission(PermWebDAVConfig), a.getUserWebDAV)
auth.PUT("/webdav/config", a.requirePermission(PermWebDAVConfig), a.putUserWebDAV)
auth.POST("/webdav/test", a.requirePermission(PermWebDAVConfig), a.testUserWebDAV)
auth.POST("/tasks", a.requirePermission(PermDownload), a.addTask)
auth.POST("/tasks/:id/retry", a.requirePermission(PermDownload), a.retryTask)
auth.DELETE("/tasks/:id", a.requirePermission(PermDownload), a.deleteTask)
admin := api.Group("/admin", a.requireAuth, a.requireAdmin)
admin.GET("/users", a.adminListUsers)
admin.GET("/users/:id/credits", a.adminUserCredits)
admin.POST("/users/:id/credits", a.adminAdjustCredits)
admin.GET("/users/:id/credits/transactions", a.adminUserCreditTransactions)
admin.GET("/credits/transactions", a.adminCreditTransactions)
admin.GET("/audit", a.adminAudit)
admin.POST("/users", a.adminCreateUser)
admin.PUT("/users/:id", a.adminUpdateUser)
admin.GET("/redeem", a.adminListRedeemBatches)
admin.POST("/redeem", a.adminCreateRedeemBatch)
admin.POST("/redeem/:id/revoke", a.adminRevokeRedeemBatch)
admin.GET("/settings", a.getSettings)
admin.PUT("/settings", a.putSettings)
admin.POST("/webdav/test", a.testWebDAV)
must(r.Run(":" + getenv("PORT", "9540")))
}
+78
View File
@@ -0,0 +1,78 @@
package main
import (
"fmt"
"os"
"path/filepath"
"gorm.io/gorm"
)
// migrateDatabase applies schema changes and repairs legacy indexes. It is
// intentionally separate from startup so migration behavior can be tested on
// a disposable SQLite database before touching production data.
func migrateDatabase(db *gorm.DB) error {
if err := db.AutoMigrate(&Task{}, &MediaAsset{}, &Setting{}, &User{}, &UserPermission{}, &CreditAccount{}, &CreditTransaction{}, &CreditReservation{}, &RedeemBatch{}, &RedeemCode{}, &RedeemRecord{}, &WebDAVConfig{}, &Session{}, &AuditLog{}); err != nil {
return err
}
if err := db.Exec("DROP INDEX IF EXISTS idx_tasks_source_rid_quality").Error; err != nil {
return err
}
if err := db.Exec("CREATE UNIQUE INDEX IF NOT EXISTS idx_tasks_user_source_rid_quality ON tasks(user_id, source, r_id, quality)").Error; err != nil {
return err
}
if err := db.Exec("DROP INDEX IF EXISTS idx_credit_reservations_task_id").Error; err != nil {
return err
}
return db.Exec("CREATE INDEX IF NOT EXISTS idx_credit_reservations_task_id ON credit_reservations(task_id)").Error
}
// backfillMediaAssets turns legacy completed tasks into shared media assets.
// Existing physical files are reused; no media is moved or deleted.
func backfillMediaAssets(db *gorm.DB, downloadDir string) error {
var tasks []Task
if err := db.Where("(asset_id = 0 OR asset_id IS NULL) AND status = ?", "done").Find(&tasks).Error; err != nil {
return err
}
for i := range tasks {
t := &tasks[i]
key := assetKey(t.Source, t.RID, t.Quality)
var asset MediaAsset
result := db.Where("asset_key = ?", key).Limit(1).Find(&asset)
if result.Error != nil {
return result.Error
}
if result.RowsAffected == 0 {
asset = MediaAsset{
AssetKey: key,
Source: t.Source,
RID: t.RID,
Quality: t.Quality,
LocalPath: t.LocalPath,
Format: t.Format,
ActualQuality: t.ActualQuality,
Size: t.DownloadedBytes,
Status: "done",
}
if asset.LocalPath != "" {
if fi, statErr := os.Stat(asset.LocalPath); statErr == nil && fi.Size() > 0 {
asset.Size = fi.Size()
}
} else if downloadDir != "" {
candidate := filepath.Join(downloadDir, safe(t.Artist)+" - "+safe(t.Title)+"."+t.Format)
if fi, statErr := os.Stat(candidate); statErr == nil && fi.Size() > 0 {
asset.LocalPath, asset.Size = candidate, fi.Size()
}
}
if err := db.Create(&asset).Error; err != nil {
return fmt.Errorf("create media asset for task %d: %w", t.ID, err)
}
}
if t.AssetID != asset.ID {
if err := db.Model(t).Update("asset_id", asset.ID).Error; err != nil {
return fmt.Errorf("link media asset for task %d: %w", t.ID, err)
}
}
}
return nil
}
+98
View File
@@ -0,0 +1,98 @@
package main
import (
"os"
"path/filepath"
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
)
func openMigrationTestDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open(filepath.Join(t.TempDir(), "migration.db")), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
return db
}
func TestMigrateDatabaseLegacySchema(t *testing.T) {
db := openMigrationTestDB(t)
if err := db.Exec(`CREATE TABLE tasks (
id integer PRIMARY KEY AUTOINCREMENT,
source text, r_id text, title text, artist text, album text, pic text,
quality text, format text, status text, error text, local_path text,
remote_path text, created_at datetime, updated_at datetime, total_bytes integer,
downloaded_bytes integer, progress real, resolved_quality text, actual_quality text
)`).Error; err != nil {
t.Fatal(err)
}
if err := db.Exec("CREATE UNIQUE INDEX idx_tasks_source_rid_quality ON tasks(source, r_id, quality)").Error; err != nil {
t.Fatal(err)
}
if err := db.Exec("INSERT INTO tasks(source,r_id,quality,status) VALUES('kw','legacy-1','standard','done')").Error; err != nil {
t.Fatal(err)
}
if err := migrateDatabase(db); err != nil {
t.Fatal(err)
}
var task Task
if err := db.First(&task).Error; err != nil {
t.Fatal(err)
}
if task.UserID != 0 {
t.Fatalf("legacy task user id = %d, want 0 before bootstrap", task.UserID)
}
var indexes []struct{ Name string }
if err := db.Raw("SELECT name FROM sqlite_master WHERE type='index' AND tbl_name='tasks'").Scan(&indexes).Error; err != nil {
t.Fatal(err)
}
found := false
for _, idx := range indexes {
if idx.Name == "idx_tasks_user_source_rid_quality" {
found = true
}
if idx.Name == "idx_tasks_source_rid_quality" {
t.Fatalf("legacy global task index still exists")
}
}
if !found {
t.Fatalf("scoped task index not created")
}
}
func TestBackfillMediaAssetsLinksExistingFiles(t *testing.T) {
db := openMigrationTestDB(t)
if err := migrateDatabase(db); err != nil {
t.Fatal(err)
}
dir := t.TempDir()
path := filepath.Join(dir, "Artist - Title.mp3")
if err := os.WriteFile(path, []byte("audio"), 0644); err != nil {
t.Fatal(err)
}
task := Task{UserID: 1, Source: "kw", RID: "rid-1", Title: "Title", Artist: "Artist", Quality: "standard", Format: "mp3", Status: "done", LocalPath: path}
if err := db.Create(&task).Error; err != nil {
t.Fatal(err)
}
if err := backfillMediaAssets(db, dir); err != nil {
t.Fatal(err)
}
var asset MediaAsset
if err := db.Where("asset_key = ?", assetKey(task.Source, task.RID, task.Quality)).First(&asset).Error; err != nil {
t.Fatal(err)
}
if asset.Status != "done" || asset.LocalPath != path || asset.Size != int64(len("audio")) {
t.Fatalf("unexpected asset: %+v", asset)
}
var got Task
if err := db.First(&got, task.ID).Error; err != nil {
t.Fatal(err)
}
if got.AssetID != asset.ID {
t.Fatalf("task asset id = %d, want %d", got.AssetID, asset.ID)
}
}
+6 -3
View File
@@ -4,13 +4,16 @@ import "time"
type Task struct {
ID uint `gorm:"primaryKey" json:"id"`
Source string `gorm:"uniqueIndex:idx_tasks_source_rid_quality" json:"source"`
RID string `gorm:"uniqueIndex:idx_tasks_source_rid_quality" json:"rid"`
UserID uint `gorm:"uniqueIndex:idx_tasks_user_source_rid_quality" json:"userId"`
AssetID uint `gorm:"index" json:"assetId,omitempty"`
CreditReservationID uint `gorm:"index" json:"creditReservationId,omitempty"`
Source string `gorm:"uniqueIndex:idx_tasks_user_source_rid_quality" json:"source"`
RID string `gorm:"uniqueIndex:idx_tasks_user_source_rid_quality" json:"rid"`
Title string `json:"title"`
Artist string `json:"artist"`
Album string `json:"album"`
Pic string `json:"pic"`
Quality string `gorm:"uniqueIndex:idx_tasks_source_rid_quality" json:"quality"`
Quality string `gorm:"uniqueIndex:idx_tasks_user_source_rid_quality" json:"quality"`
ResolvedQuality string `json:"resolvedQuality,omitempty"`
ActualQuality string `json:"actualQuality,omitempty"`
Format string `json:"format"`
+125
View File
@@ -0,0 +1,125 @@
package main
import "time"
type User struct {
ID uint `gorm:"primaryKey" json:"id"`
Username string `gorm:"uniqueIndex;size:64" json:"username"`
PasswordHash string `json:"-"`
Role string `gorm:"size:20;index" json:"role"`
Status string `gorm:"size:20;index" json:"status"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
type UserPermission struct {
ID uint `gorm:"primaryKey"`
UserID uint `gorm:"uniqueIndex:idx_user_permission" json:"userId"`
Permission string `gorm:"uniqueIndex:idx_user_permission;size:64" json:"permission"`
Enabled bool `json:"enabled"`
}
type CreditAccount struct {
ID uint `gorm:"primaryKey"`
UserID uint `gorm:"uniqueIndex" json:"userId"`
Balance int64 `json:"balance"`
Reserved int64 `json:"reserved"`
UpdatedAt time.Time `json:"updatedAt"`
}
type CreditTransaction struct {
ID uint `gorm:"primaryKey"`
UserID uint `gorm:"index" json:"userId"`
Type string `gorm:"index;size:32" json:"type"`
Amount int64 `json:"amount"`
BalanceAfter int64 `json:"balanceAfter"`
ReferenceType string `gorm:"size:32" json:"referenceType"`
ReferenceID string `gorm:"size:64" json:"referenceId"`
Reason string `json:"reason,omitempty"`
CreatedAt time.Time `json:"createdAt"`
}
type CreditReservation struct {
ID uint `gorm:"primaryKey"`
UserID uint `gorm:"index" json:"userId"`
TaskID uint `gorm:"index" json:"taskId"`
Amount int64 `json:"amount"`
Status string `gorm:"index;size:20" json:"status"`
CreatedAt time.Time `json:"createdAt"`
CompletedAt *time.Time `json:"completedAt,omitempty"`
}
type RedeemBatch struct {
ID uint `gorm:"primaryKey"`
Code string `gorm:"uniqueIndex;size:64" json:"code"`
Credits int64 `json:"credits"`
Count int `json:"count"`
ExpiresAt *time.Time `json:"expiresAt,omitempty"`
Note string `json:"note,omitempty"`
CreatedBy uint `json:"createdBy"`
CreatedAt time.Time `json:"createdAt"`
}
type RedeemCode struct {
ID uint `gorm:"primaryKey"`
BatchID uint `gorm:"index" json:"batchId"`
CodeHash string `gorm:"uniqueIndex;size:64" json:"-"`
Prefix string `gorm:"size:24" json:"prefix"`
Credits int64 `json:"credits"`
Status string `gorm:"index;size:20" json:"status"`
ExpiresAt *time.Time `json:"expiresAt,omitempty"`
RedeemedBy *uint `json:"redeemedBy,omitempty"`
RedeemedAt *time.Time `json:"redeemedAt,omitempty"`
CreatedAt time.Time `json:"createdAt"`
}
type RedeemRecord struct {
ID uint `gorm:"primaryKey"`
CodeID uint `gorm:"index" json:"codeId"`
BatchID uint `gorm:"index" json:"batchId"`
UserID uint `gorm:"index" json:"userId"`
Credits int64 `json:"credits"`
CreatedAt time.Time `json:"createdAt"`
}
type WebDAVConfig struct {
ID uint `gorm:"primaryKey"`
UserID uint `gorm:"uniqueIndex" json:"userId"`
Enabled bool `json:"enabled"`
URL string `json:"url"`
Username string `json:"username"`
Password string `json:"-"`
RootPath string `json:"rootPath"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
type Session struct {
ID uint `gorm:"primaryKey"`
UserID uint `gorm:"index" json:"userId"`
TokenHash string `gorm:"uniqueIndex;size:64" json:"-"`
ExpiresAt time.Time `gorm:"index" json:"expiresAt"`
CreatedAt time.Time `json:"createdAt"`
}
type AuditLog struct {
ID uint `gorm:"primaryKey"`
OperatorID uint `gorm:"index" json:"operatorId"`
Action string `gorm:"index;size:64" json:"action"`
TargetType string `gorm:"size:32" json:"targetType"`
TargetID string `gorm:"size:64" json:"targetId"`
Metadata string `json:"metadata,omitempty"`
CreatedAt time.Time `json:"createdAt"`
}
const (
RoleAdmin = "admin"
RolePrivileged = "privileged"
RoleUser = "user"
UserActive = "active"
UserDisabled = "disabled"
PermDownload = "download"
PermRedeem = "redeem"
PermWebDAV = "webdav"
PermWebDAVConfig = "webdav_config"
)
+152
View File
@@ -0,0 +1,152 @@
package main
import (
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"strings"
"time"
"github.com/gin-gonic/gin"
"gorm.io/gorm"
)
func hashRedeemCode(code string) string {
s := sha256.Sum256([]byte(strings.ToUpper(strings.TrimSpace(code))))
return hex.EncodeToString(s[:])
}
func generateRedeemCode() (string, error) {
b := make([]byte, 9)
if _, err := rand.Read(b); err != nil {
return "", err
}
const alphabet = "ABCDEFGHJKLMNPQRSTUVWXYZ23456789"
out := make([]byte, 12)
for i := range out {
out[i] = alphabet[int(b[i%len(b)])%len(alphabet)]
}
return string(out[:4]) + "-" + string(out[4:8]) + "-" + string(out[8:]), nil
}
func (a *App) adminCreateRedeemBatch(c *gin.Context) {
var in struct {
Credits int64 `json:"credits"`
Count int `json:"count"`
ExpiresAt *time.Time `json:"expiresAt"`
Note string `json:"note"`
}
if c.BindJSON(&in) != nil || in.Credits <= 0 || in.Count < 1 || in.Count > 10000 {
c.JSON(400, gin.H{"error": "credits/count invalid"})
return
}
admin := c.MustGet("user").(*User)
batch := RedeemBatch{Code: fmt.Sprintf("B-%d", time.Now().UnixNano()), Credits: in.Credits, Count: in.Count, ExpiresAt: in.ExpiresAt, Note: in.Note, CreatedBy: admin.ID}
codes := make([]string, 0, in.Count)
err := a.db.Transaction(func(tx *gorm.DB) error {
if err := tx.Create(&batch).Error; err != nil {
return err
}
for i := 0; i < in.Count; i++ {
code, err := generateRedeemCode()
if err != nil {
return err
}
row := RedeemCode{BatchID: batch.ID, CodeHash: hashRedeemCode(code), Prefix: code[:4], Credits: in.Credits, Status: "unused", ExpiresAt: in.ExpiresAt}
if err := tx.Create(&row).Error; err != nil {
return err
}
codes = append(codes, code)
}
return nil
})
if err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
a.audit(admin.ID, "redeem.batch.create", "redeem_batch", fmt.Sprint(batch.ID), fmt.Sprintf("credits=%d,count=%d", batch.Credits, batch.Count))
c.JSON(201, gin.H{"batch": batch, "codes": codes})
}
func (a *App) redeem(c *gin.Context) {
user := c.MustGet("user").(*User)
var in struct {
Code string `json:"code"`
}
if c.BindJSON(&in) != nil || strings.TrimSpace(in.Code) == "" {
c.JSON(400, gin.H{"error": "code required"})
return
}
now := time.Now()
var amount int64
err := a.db.Transaction(func(tx *gorm.DB) error {
var code RedeemCode
if err := tx.Where("code_hash = ?", hashRedeemCode(in.Code)).First(&code).Error; err != nil {
return err
}
if code.Status != "unused" {
return errors.New("code already used or unavailable")
}
if code.ExpiresAt != nil && code.ExpiresAt.Before(now) {
return errors.New("code expired")
}
var account CreditAccount
if err := tx.Where("user_id = ?", user.ID).First(&account).Error; err != nil {
if !errors.Is(err, gorm.ErrRecordNotFound) {
return err
}
account = CreditAccount{UserID: user.ID}
if err := tx.Create(&account).Error; err != nil {
return err
}
}
account.Balance += code.Credits
if err := tx.Save(&account).Error; err != nil {
return err
}
code.Status, code.RedeemedBy, code.RedeemedAt = "used", &user.ID, &now
if err := tx.Save(&code).Error; err != nil {
return err
}
if err := tx.Create(&RedeemRecord{CodeID: code.ID, BatchID: code.BatchID, UserID: user.ID, Credits: code.Credits}).Error; err != nil {
return err
}
if err := tx.Create(&CreditTransaction{UserID: user.ID, Type: "redeem", Amount: code.Credits, BalanceAfter: account.Balance, ReferenceType: "redeem_code", ReferenceID: fmt.Sprint(code.ID), Reason: "redeem code"}).Error; err != nil {
return err
}
amount = code.Credits
return nil
})
if err != nil {
status := 500
if errors.Is(err, gorm.ErrRecordNotFound) {
status = 404
}
c.JSON(status, gin.H{"error": err.Error()})
return
}
a.audit(user.ID, "redeem.use", "redeem_code", "hash:"+hashRedeemCode(in.Code)[:12], fmt.Sprintf("credits=%d", amount))
c.JSON(200, gin.H{"credits": amount})
}
func (a *App) adminRevokeRedeemBatch(c *gin.Context) {
id := c.Param("id")
if err := a.db.Model(&RedeemCode{}).Where("batch_id = ? AND status = ?", id, "unused").Update("status", "revoked").Error; err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
admin := c.MustGet("user").(*User)
a.audit(admin.ID, "redeem.batch.revoke", "redeem_batch", id, "unused codes revoked")
c.JSON(200, gin.H{"ok": true})
}
func (a *App) adminListRedeemBatches(c *gin.Context) {
var batches []RedeemBatch
if err := a.db.Order("id desc").Limit(100).Find(&batches).Error; err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
c.JSON(200, gin.H{"data": batches})
}
+47 -3
View File
@@ -1,6 +1,8 @@
package main
import (
"crypto/rand"
"fmt"
"strings"
"time"
@@ -12,7 +14,7 @@ func (a *App) getSettings(c *gin.Context) {
c.JSON(200, gin.H{
"amApi": a.cfg.AMAPI, "downloadDir": a.cfg.DownloadDir,
"webdavUrl": a.cfg.WebDAVURL, "webdavUser": a.cfg.WebDAVUser,
"webdavPass": a.cfg.WebDAVPass, "webdavPath": a.cfg.WebDAVPath,
"webdavPass": "", "hasWebdavPassword": a.cfg.WebDAVPass != "", "webdavPath": a.cfg.WebDAVPath,
"quality": a.cfg.DefaultQuality, "webdavVersionPolicy": a.cfg.WebDAVVersionPolicy,
"localFilePolicy": a.cfg.LocalFilePolicy,
})
@@ -45,7 +47,9 @@ func (a *App) putSettings(c *gin.Context) {
if x, ok := v["webdavUser"]; ok {
a.cfg.WebDAVUser = x
}
if x, ok := v["webdavPass"]; ok {
// The GET endpoint intentionally never returns the WebDAV password.
// Therefore an empty webdavPass means "keep the existing password", not clear it.
if x, ok := v["webdavPass"]; ok && x != "" {
a.cfg.WebDAVPass = x
}
if x, ok := v["webdavPath"]; ok {
@@ -66,6 +70,9 @@ func (a *App) putSettings(c *gin.Context) {
} else {
a.storage = nil
}
// Credentials/config changes should repair already-downloaded local files
// immediately instead of waiting for the periodic retry interval.
go a.resyncPendingUploads()
c.JSON(200, gin.H{"ok": true})
}
@@ -75,11 +82,48 @@ func (a *App) testWebDAV(c *gin.Context) {
c.Status(400)
return
}
// An empty password means "use the saved password". The password is never
// returned to the browser, so this also makes a fresh settings page test
// the credential that is actually stored by the server.
if v.URL == "" {
v.URL = a.cfg.WebDAVURL
}
if v.User == "" {
v.User = a.cfg.WebDAVUser
}
if v.Pass == "" {
v.Pass = a.cfg.WebDAVPass
}
v.Path = strings.Trim(v.Path, "/")
if v.Path == "" {
v.Path = a.cfg.WebDAVPath
}
if v.URL == "" || v.User == "" || v.Pass == "" {
c.JSON(400, gin.H{"ok": false, "error": "WebDAV URL、用户名或密码未配置"})
return
}
client := gowebdav.NewClient(v.URL, v.User, v.Pass)
client.SetTimeout(20 * time.Second)
if err := client.Connect(); err != nil {
c.JSON(502, gin.H{"ok": false, "error": err.Error()})
c.JSON(502, gin.H{"ok": false, "error": "WebDAV authentication failed: " + err.Error()})
return
}
if v.Path != "" {
if err := client.MkdirAll(v.Path, 0755); err != nil {
c.JSON(502, gin.H{"ok": false, "error": "WebDAV target path is not writable: " + err.Error()})
return
}
}
var token [12]byte
if _, err := rand.Read(token[:]); err != nil {
c.JSON(500, gin.H{"ok": false, "error": "failed to create test token"})
return
}
testPath := strings.Trim(v.Path, "/") + "/.am-downloader-test-" + fmt.Sprintf("%x", token[:])
if err := client.Write(testPath, []byte("ok"), 0644); err != nil {
c.JSON(502, gin.H{"ok": false, "error": "WebDAV write test failed: " + err.Error()})
return
}
_ = client.Remove(testPath)
c.JSON(200, gin.H{"ok": true, "path": v.Path})
}
+132
View File
@@ -0,0 +1,132 @@
package main
import (
"path"
"strings"
"time"
"github.com/gin-gonic/gin"
"github.com/studio-b12/gowebdav"
"gorm.io/gorm"
)
func cleanWebDAVRoot(root string) string {
root = strings.TrimSpace(strings.ReplaceAll(root, "\\", "/"))
root = path.Clean("/" + root)
if root == "/" {
return ""
}
return strings.Trim(root, "/")
}
func (a *App) initializeUserWebDAVDefaults() error {
var users []User
if err := a.db.Find(&users).Error; err != nil {
return err
}
for _, user := range users {
var cfg WebDAVConfig
err := a.db.Where("user_id = ?", user.ID).First(&cfg).Error
if err == nil {
continue
}
if err != gorm.ErrRecordNotFound {
return err
}
root := cleanWebDAVRoot(strings.TrimRight(a.cfg.WebDAVPath, "/") + "/" + user.Username)
if err := a.db.Create(&WebDAVConfig{UserID: user.ID, Enabled: true, RootPath: root}).Error; err != nil {
return err
}
}
return nil
}
func (a *App) getUserWebDAV(c *gin.Context) {
user := c.MustGet("user").(*User)
var cfg WebDAVConfig
if err := a.db.Where("user_id = ?", user.ID).First(&cfg).Error; err != nil {
c.JSON(200, gin.H{"enabled": false, "url": "", "username": "", "rootPath": "", "hasPassword": false})
return
}
c.JSON(200, gin.H{"enabled": cfg.Enabled, "url": cfg.URL, "username": cfg.Username, "rootPath": cfg.RootPath, "hasPassword": cfg.Password != ""})
}
func (a *App) putUserWebDAV(c *gin.Context) {
user := c.MustGet("user").(*User)
var in struct {
Enabled bool `json:"enabled"`
URL string `json:"url"`
Username string `json:"username"`
Password string `json:"password"`
RootPath string `json:"rootPath"`
}
if c.BindJSON(&in) != nil {
c.Status(400)
return
}
root := cleanWebDAVRoot(in.RootPath)
var cfg WebDAVConfig
err := a.db.Where("user_id = ?", user.ID).First(&cfg).Error
if err != nil {
cfg = WebDAVConfig{UserID: user.ID, Enabled: in.Enabled, URL: strings.TrimRight(strings.TrimSpace(in.URL), "/"), Username: strings.TrimSpace(in.Username), Password: in.Password, RootPath: root}
} else {
cfg.Enabled = in.Enabled
cfg.URL = strings.TrimRight(strings.TrimSpace(in.URL), "/")
cfg.Username = strings.TrimSpace(in.Username)
cfg.RootPath = root
if in.Password != "" {
cfg.Password = in.Password
}
}
if err := a.db.Save(&cfg).Error; err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
c.JSON(200, gin.H{"ok": true})
}
func (a *App) testUserWebDAV(c *gin.Context) {
user := c.MustGet("user").(*User)
var in struct{ URL, Username, Password, RootPath string }
if c.BindJSON(&in) != nil {
c.Status(400)
return
}
if in.URL == "" {
var cfg WebDAVConfig
if a.db.Where("user_id = ?", user.ID).First(&cfg).Error == nil {
in.URL, in.Username, in.Password, in.RootPath = cfg.URL, cfg.Username, cfg.Password, cfg.RootPath
}
}
client := gowebdav.NewClient(in.URL, in.Username, in.Password)
client.SetTimeout(20 * time.Second)
if err := client.Connect(); err != nil {
c.JSON(502, gin.H{"ok": false, "error": err.Error()})
return
}
c.JSON(200, gin.H{"ok": true, "rootPath": cleanWebDAVRoot(in.RootPath)})
}
func (a *App) storageForTask(t *Task) (RemoteStorage, string, bool) {
var user User
if a.db.First(&user, t.UserID).Error != nil {
return nil, "", false
}
if !a.hasPermission(user.ID, PermWebDAV) && user.Role != RoleAdmin {
return nil, "", false
}
// All users share the same global WebDAV connection by default. Each user
// gets an isolated destination directory, so upload/retry behavior is
// identical for admin, privileged, and regular users.
if a.cfg.WebDAVURL == "" || a.storage == nil {
return nil, "", false
}
var cfg WebDAVConfig
if a.db.Where("user_id = ?", user.ID).First(&cfg).Error == nil && cfg.RootPath != "" {
return a.storage, cfg.RootPath, true
}
root := cleanWebDAVRoot(strings.TrimRight(a.cfg.WebDAVPath, "/") + "/" + user.Username)
return a.storage, root, true
}
+15
View File
@@ -13,11 +13,26 @@ func (a *App) fail(t *Task, err error) {
if t.LocalPath != "" {
_ = os.Remove(t.LocalPath + ".part")
}
a.releaseAsset(t)
if err == nil {
err = errors.New("unknown download error")
}
t.Status, t.Error = "error", err.Error()
a.db.Save(t)
if refundErr := a.refundCredit(t); refundErr != nil {
t.Error += "; credit refund failed: " + refundErr.Error()
a.db.Save(t)
}
}
func (a *App) failUpload(t *Task, err error) {
if err == nil {
err = errors.New("unknown WebDAV upload error")
}
t.RemotePath = ""
t.Status = "upload_pending"
t.Error = "WebDAV 上传失败,等待自动重试: " + err.Error()
a.db.Save(t)
}
var bad = regexp.MustCompile(`[/\\:*?"<>|]+`)
+80 -25
View File
@@ -7,35 +7,63 @@ import (
)
func (a *App) upload(t *Task) {
if a.storage == nil || a.cfg.WebDAVURL == "" {
t.Status, t.Progress = "done", 100
a.db.Save(t)
if t.RemotePath != "" && t.Status == "done" {
return
}
a.db.Model(t).Updates(map[string]any{"status": "uploading"})
remote := strings.TrimRight(a.cfg.WebDAVPath, "/") + "/" + safe(t.Artist) + "/" + safe(t.Album) + "/" + safe(t.Title) + "." + safe(t.Format)
storage, root, enabled := a.storageForTask(t)
if !enabled || storage == nil || root == "" {
if t.Status == "upload_pending" {
if err := a.db.Model(t).Updates(map[string]any{
"status": "done", "error": "",
}).Error; err != nil {
return
}
t.Status, t.Error = "done", ""
}
return
}
if !a.claimUpload(t) {
return
}
remote := strings.TrimRight(root, "/") + "/" + safe(t.Artist) + "/" +
safe(t.Album) + "/" + safe(t.Title) + "." + safe(t.Format)
if err := storage.MkdirAll(filepath.ToSlash(filepath.Dir(remote))); err != nil {
a.failUpload(t, err)
return
}
fi, statErr := os.Stat(t.LocalPath)
if statErr != nil {
if a.recoverMissingLocalUpload(t) {
return
}
a.failUpload(t, statErr)
return
}
if same, _ := storage.ExistsSameSize(remote, fi.Size()); same {
t.RemotePath = remote
if err := a.storage.MkdirAll(filepath.ToSlash(filepath.Dir(remote))); err != nil {
a.fail(t, err)
return
}
if fi, err := os.Stat(t.LocalPath); err == nil {
if same, _ := a.storage.ExistsSameSize(remote, fi.Size()); same {
t.Status, t.Progress = "done", 100
a.db.Save(t)
t.Status, t.Progress, t.Error = "done", 100, ""
_ = a.db.Model(t).Updates(map[string]any{
"remote_path": remote, "status": "done",
"progress": 100, "error": "",
}).Error
a.cleanupLocalFile(t)
return
}
if err := storage.Upload(t.LocalPath, remote); err != nil {
a.failUpload(t, err)
return
}
if err := a.storage.Upload(t.LocalPath, remote); err != nil {
a.fail(t, err)
t.RemotePath = remote
t.Status, t.Progress, t.Error = "done", 100, ""
if err := a.db.Model(t).Updates(map[string]any{
"remote_path": remote, "status": "done",
"progress": 100, "error": "",
}).Error; err != nil {
return
}
if a.cfg.WebDAVVersionPolicy == "replace_lower" {
a.cleanupLowerQualityVersions(t)
a.cleanupLowerQualityVersions(t, storage)
}
t.Status, t.Progress = "done", 100
a.db.Save(t)
a.cleanupLocalFile(t)
}
@@ -43,26 +71,53 @@ func (a *App) cleanupLocalFile(t *Task) {
if a.cfg.LocalFilePolicy != "delete_after_upload" || t.LocalPath == "" {
return
}
if err := os.Remove(t.LocalPath); err != nil && !os.IsNotExist(err) {
t.Error = "WebDAV 上传成功,但删除本地文件失败: " + err.Error()
a.db.Save(t)
if t.AssetID != 0 {
var pending int64
if err := a.db.Model(&Task{}).
Where("asset_id = ? AND id <> ? AND local_path = ? AND "+
"(status IN ? OR (status = ? AND (remote_path = '' OR remote_path IS NULL)))",
t.AssetID, t.ID, t.LocalPath,
[]string{"pending", "resolving", "downloading", "uploading", "upload_pending"},
"done").Count(&pending).Error; err != nil {
return
}
if pending > 0 {
return
}
}
func (a *App) cleanupLowerQualityVersions(t *Task) {
levels := map[string]int{"standard": 1, "exhigh": 2, "sq": 3, "lossless": 4, "hires": 5, "jyeffect": 6, "sky": 7, "jymaster": 8, "atmos": 9, "atmos_plus": 10, "master": 11}
if err := os.Remove(t.LocalPath); err != nil && !os.IsNotExist(err) {
t.Error = "WebDAV 上传成功,但删除本地文件失败: " + err.Error()
_ = a.db.Save(t)
return
}
if t.AssetID != 0 {
_ = a.db.Model(&MediaAsset{}).
Where("id = ? AND local_path = ?", t.AssetID, t.LocalPath).
Updates(map[string]any{
"status": "evicted", "local_path": "",
"size": 0, "owner_task_id": 0, "lease_until": nil,
}).Error
}
}
func (a *App) cleanupLowerQualityVersions(t *Task, storage RemoteStorage) {
levels := map[string]int{"standard": 1, "exhigh": 2, "sq": 3,
"lossless": 4, "hires": 5, "jyeffect": 6, "sky": 7,
"jymaster": 8, "atmos": 9, "atmos_plus": 10, "master": 11}
current := qualityRank(levels, t.ActualQuality, t.ResolvedQuality, t.Quality)
if current == 0 {
return
}
var old []Task
a.db.Where("artist = ? AND title = ? AND album = ? AND status = ? AND id <> ?", t.Artist, t.Title, t.Album, "done", t.ID).Find(&old)
a.db.Where("user_id = ? AND artist = ? AND title = ? AND album = ? "+
"AND status = ? AND id <> ?", t.UserID, t.Artist, t.Title, t.Album, "done", t.ID).Find(&old)
for _, candidate := range old {
q := qualityRank(levels, candidate.ActualQuality, candidate.ResolvedQuality, candidate.Quality)
if q == 0 || q >= current || candidate.RemotePath == "" || candidate.RemotePath == t.RemotePath {
continue
}
_ = a.storage.Remove(candidate.RemotePath)
_ = storage.Remove(candidate.RemotePath)
}
}
+137
View File
@@ -0,0 +1,137 @@
package main
import (
"os"
"time"
"gorm.io/gorm"
)
// recoverMissingLocalUpload re-queues a task when its shared media file has
// disappeared. This can happen after an older version deleted a shared file
// after another user's upload. The asset is invalidated so the next worker
// downloads it again instead of retrying an impossible WebDAV upload forever.
func (a *App) recoverMissingLocalUpload(t *Task) bool {
if t.AssetID == 0 || t.LocalPath == "" {
return false
}
if _, err := os.Stat(t.LocalPath); err == nil || !os.IsNotExist(err) {
return false
}
var queued bool
err := a.transactionRetry(func(tx *gorm.DB) error {
var asset MediaAsset
if err := tx.First(&asset, t.AssetID).Error; err != nil {
return err
}
if asset.Status == "done" && asset.LocalPath != "" {
if _, statErr := os.Stat(asset.LocalPath); statErr == nil {
if err := tx.Model(&Task{}).Where("id = ?", t.ID).Updates(map[string]any{
"local_path": asset.LocalPath, "format": asset.Format,
"actual_quality": asset.ActualQuality,
"status": "upload_pending", "error": "共享资源已恢复,正在继续上传",
}).Error; err != nil {
return err
}
queued = true
return nil
}
}
if asset.LocalPath != "" && asset.LocalPath != t.LocalPath {
return nil
}
asset.Status = "error"
asset.OwnerTaskID = 0
asset.LeaseUntil = time.Time{}
asset.LocalPath = ""
asset.Size = 0
if err := tx.Save(&asset).Error; err != nil {
return err
}
if err := tx.Model(&Task{}).Where("id = ?", t.ID).Updates(map[string]any{
"status": "pending", "error": "本地共享文件不存在,正在重新下载",
"progress": 0, "downloaded_bytes": 0,
}).Error; err != nil {
return err
}
queued = true
return nil
})
if err != nil || !queued {
return false
}
a.enqueueTask(t.ID)
return true
}
// resyncPendingUploads repairs tasks that finished locally while WebDAV was
// unavailable or the user's WebDAV permission was not yet configured.
func (a *App) resyncPendingUploads() {
var tasks []Task
if err := a.db.Where("local_path <> ? AND ((status IN ? AND (remote_path = '' OR remote_path IS NULL)) OR status = ?)",
"", []string{"done", "upload_pending"}, "error").Order("id asc").Limit(100).Find(&tasks).Error; err != nil {
return
}
for i := range tasks {
t := tasks[i]
if t.Status == "error" {
status, err := a.taskReservationStatus(t.ID, t.CreditReservationID)
if err != nil {
continue
}
if status == "committed" {
if err := a.repairCommittedTask(&t); err != nil {
continue
}
} else {
var asset MediaAsset
if t.AssetID == 0 || a.db.First(&asset, t.AssetID).Error != nil ||
asset.Status != "done" || asset.LocalPath == "" {
continue
}
if _, statErr := os.Stat(asset.LocalPath); statErr != nil {
continue
}
t.LocalPath = asset.LocalPath
t.Format = asset.Format
t.ActualQuality = asset.ActualQuality
t.Status = "upload_pending"
t.Error = "共享资源已完成,历史失败任务恢复上传"
if err := a.db.Model(&t).Updates(map[string]any{
"local_path": t.LocalPath, "format": t.Format,
"actual_quality": t.ActualQuality,
"status": t.Status, "error": t.Error,
}).Error; err != nil {
continue
}
}
}
if _, _, ok := a.storageForTask(&t); !ok {
continue
}
if a.recoverMissingLocalUpload(&t) {
continue
}
var current Task
if a.db.First(&current, t.ID).Error != nil {
continue
}
if current.Status == "done" || current.Status == "upload_pending" {
a.enqueueTask(current.ID)
}
}
}
// startWebDAVResync runs a small periodic repair loop. It is intentionally
// conservative: only tasks with a completed local file and no confirmed remote path are retried.
func (a *App) startWebDAVResync() {
go func() {
a.resyncPendingUploads()
ticker := time.NewTicker(5 * time.Minute)
defer ticker.Stop()
for range ticker.C {
a.resyncPendingUploads()
}
}()
}
+79
View File
@@ -0,0 +1,79 @@
package main
import (
"os"
"path/filepath"
"testing"
)
func TestCleanupLocalFileKeepsSharedAssetForPendingTask(t *testing.T) {
db := openMigrationTestDB(t)
if err := migrateDatabase(db); err != nil {
t.Fatal(err)
}
path := filepath.Join(t.TempDir(), "shared.flac")
if err := os.WriteFile(path, []byte("audio"), 0644); err != nil {
t.Fatal(err)
}
asset := MediaAsset{AssetKey: assetKey("kw", "shared", "lossless"), Source: "kw", RID: "shared", Quality: "lossless", LocalPath: path, Format: "flac", Status: "done", Size: 5}
if err := db.Create(&asset).Error; err != nil {
t.Fatal(err)
}
first := Task{UserID: 1, AssetID: asset.ID, Source: "kw", RID: "shared", Quality: "lossless", Status: "done", LocalPath: path, RemotePath: "Music/shared.flac"}
second := Task{UserID: 2, AssetID: asset.ID, Source: "kw", RID: "shared", Quality: "lossless", Status: "upload_pending", LocalPath: path}
if err := db.Create(&first).Error; err != nil {
t.Fatal(err)
}
if err := db.Create(&second).Error; err != nil {
t.Fatal(err)
}
a := &App{db: db, cfg: Config{LocalFilePolicy: "delete_after_upload"}}
a.cleanupLocalFile(&first)
if _, err := os.Stat(path); err != nil {
t.Fatalf("shared file deleted while needed: %v", err)
}
if err := db.Model(&second).Updates(map[string]any{"status": "done", "remote_path": "Music/shared.flac"}).Error; err != nil {
t.Fatal(err)
}
a.cleanupLocalFile(&first)
if _, err := os.Stat(path); !os.IsNotExist(err) {
t.Fatalf("shared file still exists: %v", err)
}
var gotAsset MediaAsset
if err := db.First(&gotAsset, asset.ID).Error; err != nil {
t.Fatal(err)
}
if gotAsset.Status != "evicted" || gotAsset.LocalPath != "" {
t.Fatalf("asset not evicted: %+v", gotAsset)
}
}
func TestRecoverMissingLocalUploadRequeuesTask(t *testing.T) {
db := openMigrationTestDB(t)
if err := migrateDatabase(db); err != nil {
t.Fatal(err)
}
path := filepath.Join(t.TempDir(), "missing.flac")
asset := MediaAsset{AssetKey: assetKey("kw", "missing", "lossless"), Source: "kw", RID: "missing", Quality: "lossless", LocalPath: path, Format: "flac", Status: "done", Size: 10}
if err := db.Create(&asset).Error; err != nil {
t.Fatal(err)
}
task := Task{UserID: 1, AssetID: asset.ID, Source: "kw", RID: "missing", Quality: "lossless", Status: "upload_pending", LocalPath: path}
if err := db.Create(&task).Error; err != nil {
t.Fatal(err)
}
a := &App{db: db, queue: make(chan uint, 1)}
if !a.recoverMissingLocalUpload(&task) {
t.Fatal("missing local file was not requeued")
}
var got Task
if err := db.First(&got, task.ID).Error; err != nil {
t.Fatal(err)
}
if got.Status != "pending" {
t.Fatalf("task status = %q, want pending", got.Status)
}
if <-a.queue != task.ID {
t.Fatal("queued task id mismatch")
}
}