mirror of
https://github.com/Tencent/WeKnora.git
synced 2026-06-04 13:30:32 +08:00
feat: add knowledge parse cancellation with finalizing post-process state
Lets users stop an in-flight document parse to free up LLM / worker
resources without losing the chunks and index already written. The
core insight is that the previous parse_status=completed flipped as
soon as primary chunks landed, while the most expensive subtasks
(graph extract = N LLM calls per chunk, plus summary, question
generation) were still running in the background — so "completed"
wasn't actually terminal from a resource standpoint.
State machine
pending -> processing -> finalizing -> completed
|
+-> cancelled (any of the three
in-flight states)
+-> failed
+-> deleting
`finalizing` is the new post-process fan-out window. parse_status
only promotes to `completed` once pending_subtasks_count (a new
column tracking summary + question + per-chunk graph extract)
drains to zero via atomic FinalizeSubtask. Wiki ingest is
intentionally excluded from the counter — it's a KB-scoped
debounced batch and would otherwise pin parse_status in
`finalizing` for the wiki batch window.
Backend
- New ParseStatusFinalizing + pending_subtasks_count column with
migration 000056.
- knowledgeRepository.SetFinalizing transitions processing -> finalizing
conditionally so a racing cancel cannot be clobbered.
- knowledgeRepository.FinalizeSubtask atomically decrements the
counter and self-promotes the row to completed when it hits zero.
- KnowledgePostProcess restructured to compute expected subtask
count up front, flip to finalizing (or completed when no
enrichment is enabled), and only then fan out subtasks. Subtask
handlers (summary, question, graph extract) defer-decrement on
terminal exit using the existing isFinalAsynqAttempt convention.
- New POST /api/v1/knowledge/{id}/cancel-parse handler accepting
pending / processing / finalizing. Marks the row cancelled,
zeroes the counter, best-effort dequeues asynq tasks via a new
TaskInspector abstraction (asynq-mode walks pending/scheduled/
retry queues; Lite-mode noop), and scrubs wiki ingest pending op.
- SpanTracker.AbortAttempt flat-sweeps every still-running span
for the attempt via a new repo.CancelAllOpenSpans helper so the
trace viewer's striped bars all flip to cancelled, even leaf
generations whose parent stage already EndSpan'd (multimodal
fan-out pattern). knowledge_post_process closes its postSpan
via SkipSpan on the cancel/deleting entry guard so a worker
that opens a span AFTER the cancel sweep doesn't leak it.
- Housekeeping and resetPendingTasks sweep finalizing rows
identically to processing so a crash/restart can't strand them.
- DeleteKnowledge/DeleteKnowledgeList proactively dequeue
downstream tasks via the same TaskInspector path.
- ChunkExtractService gets a cancel entry guard so the most
expensive enrichment (graph extract) bails immediately when the
parent knowledge is aborted.
Frontend
- New cancelKnowledgeParse API client + "Stop parsing" entry in
both list view and card view more menus, gated on
pending/processing/finalizing.
- Polling predicate refactored to a shared isParseInFlight helper
that recognises `finalizing` (previously the doc list silently
stopped polling once parse_status flipped from processing).
- Knowledge processing timeline: isPolling includes finalizing,
new isHardTerminal short-circuits LIVE for cancelled/failed/
completed so stranded child spans cannot pin LIVE on.
- DocumentListView.computeStatus distinguishes finalizing
("增强中") from completed and shows the previous "生成摘要中"
copy when summary_status is still pending under finalizing.
Added cancelled badge as well.
- i18n: statusFinalizing / statusCancelled / cancelParse* keys
across zh-CN, en-US, ko-KR, ru-RU.
Docs / SDK
- docs/api/knowledge.md: documents the new finalizing state,
cancel-parse semantics, and which statuses accept cancel.
- client (Go SDK): CancelKnowledgeParse with docstring listing
the cancellable statuses.
This commit is contained in:
wizardchen
lyingbug
@@ -339,6 +339,20 @@ for {
|
||||
}
|
||||
```
|
||||
|
||||
### 示例：取消解析
|
||||
|
||||
```go
|
||||
// 取消正在进行的解析任务（资源紧张 / 上传错误文件时使用）
|
||||
// - 已经 completed / failed 的知识不能取消
|
||||
// - 已写入的分块/索引会保留，可后续调用 ReparseKnowledge 重新解析
|
||||
|
||||
knowledge, err := apiClient.CancelKnowledgeParse(context.Background(), knowledgeID)
|
||||
if err != nil {
|
||||
// 处理错误
|
||||
}
|
||||
fmt.Printf("Parse Status: %s\n", knowledge.ParseStatus) // "cancelled"
|
||||
```
|
||||
|
||||
### 示例：获取会话消息
|
||||
|
||||
```go
|
||||
|
||||
@@ -489,6 +489,56 @@ func (c *Client) ReparseKnowledge(ctx context.Context, knowledgeID string) (*Kno
|
||||
return &response.Data, nil
|
||||
}
|
||||
|
||||
// CancelKnowledgeParse cancels an in-progress knowledge parse.
|
||||
// The server marks the knowledge as cancelled and best-effort dequeues
|
||||
// any pending downstream tasks (multimodal, post-process, summary,
|
||||
// question generation, graph extract) for the same knowledge ID. Any
|
||||
// chunks/index already written are preserved; the user can re-trigger
|
||||
// parsing later via ReparseKnowledge.
|
||||
//
|
||||
// Cancellable parse_status values:
|
||||
// - pending — task has not started
|
||||
// - processing — DocReader / chunking / embedding stage
|
||||
// - finalizing — primary parse done, enrichment subtasks (summary,
|
||||
// question generation, graph extract) still running
|
||||
//
|
||||
// Returns an error when the knowledge is in a terminal state
|
||||
// (completed, failed) or already being deleted.
|
||||
//
|
||||
// Parameters:
|
||||
// - ctx: Context for the request
|
||||
// - knowledgeID: The ID of the knowledge entry whose parse should be cancelled
|
||||
//
|
||||
// Returns:
|
||||
// - *Knowledge: The updated knowledge entry with status set to "cancelled"
|
||||
// - error: Error information if the request fails
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// knowledge, err := client.CancelKnowledgeParse(ctx, "knowledge-id-123")
|
||||
// if err != nil {
|
||||
// log.Fatalf("Failed to cancel parse: %v", err)
|
||||
// }
|
||||
// fmt.Printf("Knowledge parse cancelled, status: %s\n", knowledge.ParseStatus)
|
||||
func (c *Client) CancelKnowledgeParse(ctx context.Context, knowledgeID string) (*Knowledge, error) {
|
||||
if knowledgeID == "" {
|
||||
return nil, fmt.Errorf("knowledge ID cannot be empty")
|
||||
}
|
||||
|
||||
path := fmt.Sprintf("/api/v1/knowledge/%s/cancel-parse", knowledgeID)
|
||||
resp, err := c.doRequest(ctx, http.MethodPost, path, nil, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var response KnowledgeResponse
|
||||
if err := parseResponse(resp, &response); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &response.Data, nil
|
||||
}
|
||||
|
||||
// UpdateChunk updates a chunk's information
|
||||
// Updates information for a specific chunk under a knowledge document
|
||||
// Parameters:
|
||||
|
||||
@@ -17,6 +17,7 @@
|
||||
| DELETE | `/knowledge/:id` | 删除单条知识 |
|
||||
| PUT | `/knowledge/manual/:id` | 更新手工 Markdown 知识 |
|
||||
| POST | `/knowledge/:id/reparse` | 重新解析知识（异步） |
|
||||
| POST | `/knowledge/:id/cancel-parse` | 取消正在进行的解析任务 |
|
||||
| GET | `/knowledge/:id/download` | 下载原始文件（attachment） |
|
||||
| GET | `/knowledge/:id/preview` | 内联预览文件（按扩展名设置 Content-Type） |
|
||||
| PUT | `/knowledge/image/:id/:chunk_id` | 更新分块图像信息 |
|
||||
@@ -28,8 +29,10 @@
|
||||
|
||||
> **公共说明**：
|
||||
> - 路径中的 `:id`（知识库路径下）为**知识库 ID**，`/knowledge/:id` 中的 `:id` 为**知识 ID**。
|
||||
> - 所有写操作（创建、更新、删除、迁移、重新解析）需要当前用户在知识库所属组织内具有 `editor` 或 `admin` 权限；清空知识库内容仅 KB **所有者**（admin 且租户匹配）可操作。
|
||||
> - 关键状态字段：`parse_status` 取值 `pending` / `processing` / `completed` / `failed`；`enable_status` 取值 `enabled` / `disabled`。
|
||||
> - 所有写操作（创建、更新、删除、迁移、重新解析、取消解析）需要当前用户在知识库所属组织内具有 `editor` 或 `admin` 权限；清空知识库内容仅 KB **所有者**（admin 且租户匹配）可操作。
|
||||
> - 关键状态字段：`parse_status` 取值 `pending` / `processing` / `finalizing` / `completed` / `failed` / `cancelled`；`enable_status` 取值 `enabled` / `disabled`。
|
||||
> - `processing` 指 DocReader / 分块 / 向量化阶段；`finalizing` 指主解析已完成、但摘要 / 问题生成 / 图谱抽取等异步增强子任务仍在执行；只有当全部子任务到达终态后才进入 `completed`。
|
||||
> - `cancelled` 表示解析被用户主动取消，可通过 `reparse` 重新触发。`pending` / `processing` / `finalizing` 这三种状态都可通过 `cancel-parse` 终止。
|
||||
|
||||
## POST `/knowledge-bases/:id/knowledge/file` - 上传文件创建知识
|
||||
|
||||
@@ -562,6 +565,47 @@ curl --location --request POST 'http://localhost:8080/api/v1/knowledge/4c4e7c1a-
|
||||
|
||||
调用后 `parse_status` 会先变为 `pending`，再由后台 worker 转为 `processing` → `completed`/`failed`。
|
||||
|
||||
## POST `/knowledge/:id/cancel-parse` - 取消解析
|
||||
|
||||
中止正在进行的解析任务，常用于资源紧张时主动放弃当前文档的解析过程。
|
||||
|
||||
**行为**：
|
||||
|
||||
- 将 `parse_status` 置为 `cancelled`，`error_message` 写入「用户已取消解析」，并把 `pending_subtasks_count` 清零。
|
||||
- 已写入数据库的分块 / 索引保留，可通过 `reparse` 接口在同一记录上重新触发解析。
|
||||
- 后台异步会 best-effort 从队列中删除该知识对应的下游任务（多模态、问题生成、摘要、图谱抽取、Post-Process 等），并对正在执行的 worker 发出停止信号；worker 在下一个检查点退出。
|
||||
- **可取消的状态**：`pending` / `processing` / `finalizing`。`finalizing` 表示主解析已完成、但摘要 / 问题生成 / 图谱抽取等增强子任务仍在执行；在该状态取消可以及时止血最昂贵的增强阶段（图谱抽取按 chunk 调 LLM，开销最大）。
|
||||
- 已经完成 (`completed`) 或失败 (`failed`) 的知识不允许取消；正在删除 (`deleting`) 的知识不允许取消。
|
||||
- 接口幂等：对已经 `cancelled` 的记录重复调用直接返回当前状态。
|
||||
|
||||
**请求**:
|
||||
|
||||
```curl
|
||||
curl --location --request POST 'http://localhost:8080/api/v1/knowledge/4c4e7c1a-09cf-485b-a7b5-24b8cdc5acf5/cancel-parse' \
|
||||
--header 'X-API-Key: sk-xxxxx'
|
||||
```
|
||||
|
||||
**响应**:
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "Knowledge parse cancelled",
|
||||
"data": {
|
||||
"id": "4c4e7c1a-09cf-485b-a7b5-24b8cdc5acf5",
|
||||
"tenant_id": 1,
|
||||
"knowledge_base_id": "kb-00000001",
|
||||
"type": "file",
|
||||
"title": "彗星.txt",
|
||||
"parse_status": "cancelled",
|
||||
"error_message": "用户已取消解析",
|
||||
"enable_status": "disabled",
|
||||
"created_at": "2025-08-12T11:52:36.168632+08:00",
|
||||
"updated_at": "2025-08-12T13:05:00.000000+08:00"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## GET `/knowledge/:id/download` - 下载原始文件
|
||||
|
||||
以 `attachment` 方式下载知识对应的原始文件。
|
||||
|
||||
@@ -223,6 +223,10 @@ export function reparseKnowledge(id: string) {
|
||||
return post(`/api/v1/knowledge/${id}/reparse`);
|
||||
}
|
||||
|
||||
export function cancelKnowledgeParse(id: string) {
|
||||
return post(`/api/v1/knowledge/${id}/cancel-parse`);
|
||||
}
|
||||
|
||||
export function getKnowledgeSpans(id: string, attempt?: number) {
|
||||
const qs = attempt ? `?attempt=${attempt}` : '';
|
||||
return get(`/api/v1/knowledge/${id}/spans${qs}`);
|
||||
|
||||
@@ -194,7 +194,18 @@ function formatRelativeTime(ts: number): string {
|
||||
}
|
||||
|
||||
function isPolling(status?: string): boolean {
|
||||
return status === 'pending' || status === 'processing'
|
||||
// finalizing is the post-process fan-out window — subspans
|
||||
// (summary / question / graph.chunk[*]) are still actively producing
|
||||
// events, so we must keep polling and drawing LIVE.
|
||||
return status === 'pending' || status === 'processing' || status === 'finalizing'
|
||||
}
|
||||
|
||||
// Hard-terminal statuses override traceActive: even if child spans were
|
||||
// left in 'running' state (e.g. a cancel raced ahead of a worker's
|
||||
// FailSpan), the parse pipeline is definitively done for this attempt
|
||||
// and we should stop polling instead of refreshing forever.
|
||||
function isHardTerminal(status?: string): boolean {
|
||||
return status === 'cancelled' || status === 'failed' || status === 'completed'
|
||||
}
|
||||
|
||||
// True if any node in the trace is still running/pending. Crucial for
|
||||
@@ -224,8 +235,14 @@ const traceActive = computed<boolean>(() => spanTreeActive(data.value?.trace))
|
||||
// parseStatus hint so the UI shows LIVE immediately.
|
||||
const isLive = computed<boolean>(() => {
|
||||
if (data.value) {
|
||||
// Hard-terminal parse_status wins over a stale traceActive: cancel
|
||||
// and irrecoverable failure can leave child spans stranded as
|
||||
// 'running' (worker process died, cancel raced FailSpan, etc.),
|
||||
// and we must NOT keep polling forever on those.
|
||||
if (isHardTerminal(data.value.parse_status)) return false
|
||||
return isPolling(data.value.parse_status) || traceActive.value
|
||||
}
|
||||
if (isHardTerminal(props.parseStatus)) return false
|
||||
return isPolling(props.parseStatus)
|
||||
})
|
||||
|
||||
|
||||
@@ -268,6 +268,10 @@ export default {
|
||||
rebuildSubmitted: 'Rebuild task submitted',
|
||||
rebuildFailed: 'Rebuild failed. Please try again later',
|
||||
rebuildInProgress: 'This document is currently being parsed. Please try again later',
|
||||
cancelParse: 'Stop parsing',
|
||||
cancelParseConfirmBody: 'Stop parsing "{title}"? Already-written chunks are kept and can be re-parsed later via "Rebuild"; pending enrichment subtasks (summary / Q&A / knowledge graph) will be dropped immediately.',
|
||||
cancelParseSubmitted: 'Parsing stopped',
|
||||
cancelParseFailed: 'Failed to stop, please try again later',
|
||||
draft: 'Draft',
|
||||
draftTip: 'Temporarily saved and not included in retrieval',
|
||||
untitledDocument: 'Untitled Document',
|
||||
@@ -317,8 +321,10 @@ export default {
|
||||
batchDeleteFailed: 'Batch delete failed',
|
||||
statusCompleted: 'Completed',
|
||||
statusProcessing: 'Processing',
|
||||
statusFinalizing: 'Finalizing',
|
||||
statusPending: 'Pending',
|
||||
statusFailed: 'Failed',
|
||||
statusCancelled: 'Cancelled',
|
||||
statusDraft: 'Draft',
|
||||
selectKnowledgeBaseFirst: 'Please select a knowledge base first',
|
||||
sessionCreationFailed: 'Failed to create chat session',
|
||||
|
||||
@@ -209,6 +209,10 @@ export default {
|
||||
rebuildSubmitted: "재구축 작업이 제출되었습니다",
|
||||
rebuildFailed: "재구축 실패, 나중에 다시 시도해주세요",
|
||||
rebuildInProgress: "현재 문서가 파싱 중입니다. 나중에 다시 시도해주세요",
|
||||
cancelParse: "파싱 중지",
|
||||
cancelParseConfirmBody: '"{title}"의 파싱을 중지하시겠습니까? 이미 저장된 청크는 유지되며 "다시 빌드"를 통해 다시 파싱할 수 있습니다. 보강 단계(요약 / Q&A / 지식 그래프)의 대기 중인 하위 작업은 즉시 취소됩니다.',
|
||||
cancelParseSubmitted: "파싱이 중지되었습니다",
|
||||
cancelParseFailed: "중지에 실패했습니다. 나중에 다시 시도해주세요",
|
||||
characters: "자",
|
||||
segment: "조각",
|
||||
chunkCount: "총 {count}개 조각",
|
||||
@@ -321,8 +325,10 @@ export default {
|
||||
batchDeleteFailed: "일괄 삭제 실패",
|
||||
statusCompleted: "완료",
|
||||
statusProcessing: "처리 중",
|
||||
statusFinalizing: "마무리 중",
|
||||
statusPending: "대기 중",
|
||||
statusFailed: "실패",
|
||||
statusCancelled: "취소됨",
|
||||
statusDraft: "초안",
|
||||
selectKnowledgeBaseFirst: "먼저 지식베이스를 선택하세요",
|
||||
sessionCreationFailed: "세션 생성 실패",
|
||||
|
||||
@@ -247,8 +247,10 @@ export default {
|
||||
batchDeleteFailed: 'Ошибка пакетного удаления',
|
||||
statusCompleted: 'Завершено',
|
||||
statusProcessing: 'Обработка',
|
||||
statusFinalizing: 'Завершение',
|
||||
statusPending: 'Ожидание',
|
||||
statusFailed: 'Ошибка',
|
||||
statusCancelled: 'Отменено',
|
||||
statusDraft: 'Черновик',
|
||||
selectKnowledgeBaseFirst: 'Пожалуйста, сначала выберите базу знаний',
|
||||
sessionCreationFailed: 'Не удалось создать диалог',
|
||||
@@ -356,7 +358,11 @@ export default {
|
||||
rebuildConfirm: 'Подтвердить пересборку документа "{fileName}"? Существующие фрагменты будут удалены и документ будет повторно проанализирован.',
|
||||
rebuildSubmitted: 'Задача пересборки отправлена',
|
||||
rebuildFailed: 'Ошибка пересборки, попробуйте позже',
|
||||
rebuildInProgress: 'Документ сейчас анализируется, попробуйте позже'
|
||||
rebuildInProgress: 'Документ сейчас анализируется, попробуйте позже',
|
||||
cancelParse: 'Остановить разбор',
|
||||
cancelParseConfirmBody: 'Остановить разбор «{title}»? Уже записанные фрагменты сохранятся, и их можно будет разобрать заново через «Пересобрать». Ожидающие подзадачи обогащения (резюме / вопросы и ответы / граф знаний) будут немедленно отменены.',
|
||||
cancelParseSubmitted: 'Разбор остановлен',
|
||||
cancelParseFailed: 'Не удалось остановить, попробуйте позже'
|
||||
},
|
||||
knowledgeStages: {
|
||||
title: 'Конвейер обработки',
|
||||
|
||||
@@ -269,6 +269,10 @@ export default {
|
||||
rebuildSubmitted: "重建任务已提交",
|
||||
rebuildFailed: "重建失败，请稍后再试",
|
||||
rebuildInProgress: "当前文档正在解析中，请稍后重试",
|
||||
cancelParse: "停止解析",
|
||||
cancelParseConfirmBody: '确认停止解析"{title}"？已写入的分块会保留，可稍后通过"重建"重新触发；增强阶段（摘要 / 问答 / 知识图谱）的待执行子任务会被立即丢弃。',
|
||||
cancelParseSubmitted: "已停止解析",
|
||||
cancelParseFailed: "停止失败，请稍后再试",
|
||||
draft: "草稿",
|
||||
draftTip: "暂存内容，未参与检索",
|
||||
untitledDocument: "未命名文档",
|
||||
@@ -318,8 +322,10 @@ export default {
|
||||
batchDeleteFailed: "批量删除失败",
|
||||
statusCompleted: "已完成",
|
||||
statusProcessing: "解析中",
|
||||
statusFinalizing: "增强中",
|
||||
statusPending: "等待中",
|
||||
statusFailed: "失败",
|
||||
statusCancelled: "已取消",
|
||||
statusDraft: "草稿",
|
||||
selectKnowledgeBaseFirst: "请先选择知识库",
|
||||
sessionCreationFailed: "创建会话失败",
|
||||
|
||||
@@ -31,6 +31,7 @@ import {
|
||||
createKnowledgeFromURL,
|
||||
listKnowledgeBases,
|
||||
reparseKnowledge,
|
||||
cancelKnowledgeParse,
|
||||
batchDeleteKnowledge,
|
||||
getKnowledgeSpans,
|
||||
} from "@/api/knowledge-base/index";
|
||||
@@ -305,9 +306,16 @@ function clearTraceAvailabilityCache() {
|
||||
traceProbeInflight.clear();
|
||||
}
|
||||
|
||||
// Parse phases where the backend pipeline is still actively running
|
||||
// (primary parse OR post-process fan-out). Trace data exists and the
|
||||
// UI should treat the row as "in flight" rather than terminal.
|
||||
function isParseInFlight(status?: string): boolean {
|
||||
return status === 'pending' || status === 'processing' || status === 'finalizing';
|
||||
}
|
||||
|
||||
function isTraceMenuVisible(item: KnowledgeCard): boolean {
|
||||
if (!item?.id) return false;
|
||||
if (item.parse_status === 'pending' || item.parse_status === 'processing') {
|
||||
if (isParseInFlight(item.parse_status)) {
|
||||
return true;
|
||||
}
|
||||
return traceAvailableById[item.id] === true;
|
||||
@@ -316,7 +324,7 @@ function isTraceMenuVisible(item: KnowledgeCard): boolean {
|
||||
async function probeTraceAvailable(item: KnowledgeCard) {
|
||||
const id = item.id;
|
||||
if (!id || traceProbeInflight.has(id)) return;
|
||||
if (item.parse_status === 'pending' || item.parse_status === 'processing') {
|
||||
if (isParseInFlight(item.parse_status)) {
|
||||
traceAvailableById[id] = true;
|
||||
return;
|
||||
}
|
||||
@@ -1031,12 +1039,7 @@ watch(() => cardList.value, (newValue) => {
|
||||
|
||||
let analyzeList = [];
|
||||
// Filter items that need polling: parsing in progress OR summary generation in progress
|
||||
analyzeList = newValue.filter(item => {
|
||||
const isParsing = item.parse_status == 'pending' || item.parse_status == 'processing';
|
||||
const isSummaryPending = item.parse_status == 'completed' &&
|
||||
(item.summary_status == 'pending' || item.summary_status == 'processing');
|
||||
return isParsing || isSummaryPending;
|
||||
})
|
||||
analyzeList = newValue.filter(needsStatusPolling);
|
||||
if (timeout !== null) {
|
||||
clearTimeout(timeout);
|
||||
timeout = null;
|
||||
@@ -1064,6 +1067,18 @@ type KnowledgeCard = {
|
||||
error_message?: string;
|
||||
tag_id?: string;
|
||||
};
|
||||
// needsStatusPolling decides whether a card row is still "in flight"
|
||||
// enough that the doc list should keep refreshing it. Keep in sync with
|
||||
// the backend lifecycle: pending / processing are the primary parse
|
||||
// phase, finalizing is the post-process fan-out (summary / question /
|
||||
// graph extract still running), and a `completed` row whose summary
|
||||
// hasn't landed yet keeps polling so the description fills in.
|
||||
const needsStatusPolling = (item: KnowledgeCard) => {
|
||||
if (isParseInFlight(item.parse_status)) return true;
|
||||
return item.parse_status == 'completed' &&
|
||||
(item.summary_status == 'pending' || item.summary_status == 'processing');
|
||||
};
|
||||
|
||||
const updateStatus = (analyzeList: KnowledgeCard[]) => {
|
||||
if (timeout !== null) {
|
||||
clearTimeout(timeout);
|
||||
@@ -1099,23 +1114,13 @@ const updateStatus = (analyzeList: KnowledgeCard[]) => {
|
||||
// If there are no changes, the watch won't trigger, so we must manually poll again
|
||||
// Even if there are changes, we can manually poll again just to be safe.
|
||||
// The watch will clear this timeout if it triggers.
|
||||
const stillPending = cardList.value.filter(item => {
|
||||
const isParsing = item.parse_status == 'pending' || item.parse_status == 'processing';
|
||||
const isSummaryPending = item.parse_status == 'completed' &&
|
||||
(item.summary_status == 'pending' || item.summary_status == 'processing');
|
||||
return isParsing || isSummaryPending;
|
||||
});
|
||||
const stillPending = cardList.value.filter(needsStatusPolling);
|
||||
if (stillPending.length > 0) {
|
||||
updateStatus(stillPending);
|
||||
}
|
||||
}).catch((_err) => {
|
||||
// 错误处理
|
||||
const stillPending = cardList.value.filter(item => {
|
||||
const isParsing = item.parse_status == 'pending' || item.parse_status == 'processing';
|
||||
const isSummaryPending = item.parse_status == 'completed' &&
|
||||
(item.summary_status == 'pending' || item.summary_status == 'processing');
|
||||
return isParsing || isSummaryPending;
|
||||
});
|
||||
const stillPending = cardList.value.filter(needsStatusPolling);
|
||||
if (stillPending.length > 0) {
|
||||
updateStatus(stillPending);
|
||||
}
|
||||
@@ -1773,7 +1778,7 @@ const handleKnowledgeReparse = (index: number, item: KnowledgeCard) => {
|
||||
MessagePlugin.warning(t('knowledgeEditor.messages.missingId'));
|
||||
return;
|
||||
}
|
||||
if (item.parse_status === 'pending' || item.parse_status === 'processing') {
|
||||
if (isParseInFlight(item.parse_status)) {
|
||||
MessagePlugin.info(t('knowledgeBase.rebuildInProgress'));
|
||||
return;
|
||||
}
|
||||
@@ -1959,14 +1964,36 @@ const confirmBatchDelete = async () => {
|
||||
}
|
||||
};
|
||||
|
||||
// Cancel an in-progress parse. Mirrors the backend gate: only the
|
||||
// pending / processing / finalizing states accept cancel. Uses the
|
||||
// native confirm dialog to match the existing delete-tag pattern in
|
||||
// this view and avoid a heavier dialog dependency.
|
||||
const handleKnowledgeCancelParse = async (item: KnowledgeCard) => {
|
||||
if (!item?.id) return;
|
||||
const confirmed = window.confirm(
|
||||
t('knowledgeBase.cancelParseConfirmBody', { title: item.title || item.id }) as string,
|
||||
);
|
||||
if (!confirmed) return;
|
||||
try {
|
||||
await cancelKnowledgeParse(item.id);
|
||||
MessagePlugin.success(t('knowledgeBase.cancelParseSubmitted'));
|
||||
// Refresh so the row reflects parse_status=cancelled and the
|
||||
// menu drops the cancel entry on the next open.
|
||||
loadKnowledgeFiles(kbId.value);
|
||||
} catch (error: any) {
|
||||
MessagePlugin.error(error?.message || t('knowledgeBase.cancelParseFailed'));
|
||||
}
|
||||
};
|
||||
|
||||
// Bridge list-view actions back to existing per-card handlers.
|
||||
const handleListAction = (
|
||||
action: 'edit' | 'reparse' | 'move' | 'delete',
|
||||
action: 'edit' | 'reparse' | 'cancel-parse' | 'move' | 'delete',
|
||||
item: KnowledgeCard,
|
||||
) => {
|
||||
const idx = (cardList.value || []).findIndex((i: KnowledgeCard) => i.id === item.id);
|
||||
if (action === 'edit') return handleManualEdit(idx, item);
|
||||
if (action === 'reparse') return handleKnowledgeReparse(idx, item);
|
||||
if (action === 'cancel-parse') return handleKnowledgeCancelParse(item);
|
||||
if (action === 'move') return handleMoveKnowledge(item);
|
||||
if (action === 'delete') return delCard(idx, item);
|
||||
};
|
||||
@@ -2356,6 +2383,14 @@ async function createNewSession(value: string): Promise<void> {
|
||||
<t-icon class="icon" name="refresh" />
|
||||
<span>{{ t('knowledgeBase.rebuildDocument') }}</span>
|
||||
</div>
|
||||
<div
|
||||
v-if="isParseInFlight(item.parse_status)"
|
||||
class="card-menu-item danger"
|
||||
@click.stop="handleKnowledgeCancelParse(item)"
|
||||
>
|
||||
<t-icon class="icon" name="close-circle" />
|
||||
<span>{{ t('knowledgeBase.cancelParse') }}</span>
|
||||
</div>
|
||||
<div v-if="canMutateKnowledge" class="card-menu-item"
|
||||
@click.stop="handleMoveKnowledge(item)">
|
||||
<t-icon class="icon" name="swap" />
|
||||
@@ -2440,6 +2475,14 @@ async function createNewSession(value: string): Promise<void> {
|
||||
<t-icon name="loading" class="card-analyze-loading"></t-icon>
|
||||
<span class="card-analyze-txt">{{ t('knowledgeBase.parsingInProgress') }}</span>
|
||||
</div>
|
||||
<div v-else-if="item.parse_status === 'finalizing'" class="card-analyze">
|
||||
<t-icon name="loading" class="card-analyze-loading"></t-icon>
|
||||
<span class="card-analyze-txt">{{
|
||||
(item.summary_status === 'pending' || item.summary_status === 'processing')
|
||||
? t('knowledgeBase.generatingSummary')
|
||||
: t('knowledgeBase.statusFinalizing')
|
||||
}}</span>
|
||||
</div>
|
||||
<div v-else-if="item.parse_status === 'failed'" class="card-analyze failure">
|
||||
<t-icon name="close-circle" class="card-analyze-loading failure"></t-icon>
|
||||
<span class="card-analyze-txt failure">{{ t('knowledgeBase.parsingFailed') }}</span>
|
||||
@@ -2511,7 +2554,7 @@ async function createNewSession(value: string): Promise<void> {
|
||||
<template v-if="hoveredCardItem">
|
||||
<div class="card-popover-title">{{ hoveredCardItem.file_name }}</div>
|
||||
<div
|
||||
v-if="hoveredCardItem.parse_status === 'processing' || hoveredCardItem.parse_status === 'pending'"
|
||||
v-if="isParseInFlight(hoveredCardItem.parse_status)"
|
||||
class="card-popover-status parsing">
|
||||
<KnowledgeProcessingTimeline :knowledge-id="hoveredCardItem.id"
|
||||
:parse-status="hoveredCardItem.parse_status" :auto-poll="false" :compact="true" />
|
||||
|
||||
@@ -37,7 +37,7 @@ const emit = defineEmits<{
|
||||
(e: 'open', item: KnowledgeItem): void;
|
||||
(e: 'toggle-row', id: string, checked: boolean, shiftKey: boolean): void;
|
||||
(e: 'toggle-all', checked: boolean): void;
|
||||
(e: 'action', action: 'edit' | 'reparse' | 'move' | 'delete', item: KnowledgeItem): void;
|
||||
(e: 'action', action: 'edit' | 'reparse' | 'cancel-parse' | 'move' | 'delete', item: KnowledgeItem): void;
|
||||
}>();
|
||||
|
||||
const { t } = useI18n();
|
||||
@@ -89,12 +89,30 @@ const computeStatus = (item: KnowledgeItem): StatusInfo => {
|
||||
if (item.parse_status === 'pending' || item.parse_status === 'processing') {
|
||||
return { label: t('knowledgeBase.statusProcessing'), theme: 'primary', icon: 'loading', spin: true };
|
||||
}
|
||||
// finalizing = primary parse done, enrichment subtasks still running.
|
||||
// While in this phase, prefer the specific "summary generating" copy
|
||||
// when summary is what's actually outstanding (preserves the old UX
|
||||
// where this label was tied to completed+summary_pending). Otherwise
|
||||
// fall back to the generic "finalizing" label — covers question gen
|
||||
// and graph extract, which the user historically had no visibility on.
|
||||
if (item.parse_status === 'finalizing') {
|
||||
if (item.summary_status === 'pending' || item.summary_status === 'processing') {
|
||||
return { label: t('knowledgeBase.generatingSummary'), theme: 'primary', icon: 'loading', spin: true };
|
||||
}
|
||||
return { label: t('knowledgeBase.statusFinalizing'), theme: 'primary', icon: 'loading', spin: true };
|
||||
}
|
||||
if (item.parse_status === 'failed') {
|
||||
return { label: t('knowledgeBase.statusFailed'), theme: 'danger', icon: 'close-circle' };
|
||||
}
|
||||
if (item.parse_status === 'cancelled') {
|
||||
return { label: t('knowledgeBase.statusCancelled'), theme: 'warning', icon: 'close-circle' };
|
||||
}
|
||||
if (item.parse_status === 'draft') {
|
||||
return { label: t('knowledgeBase.statusDraft'), theme: 'warning' };
|
||||
}
|
||||
// Legacy completed+summary_pending path: kept as a defensive fallback
|
||||
// for rows that bypassed finalizing (no enrichment configured, or
|
||||
// upgraded mid-flight from a pre-finalizing build).
|
||||
if (
|
||||
item.parse_status === 'completed' &&
|
||||
(item.summary_status === 'pending' || item.summary_status === 'processing')
|
||||
@@ -153,7 +171,14 @@ onBeforeUnmount(() => {
|
||||
stickyObserver = null;
|
||||
});
|
||||
|
||||
const handleAction = (action: 'edit' | 'reparse' | 'move' | 'delete', item: KnowledgeItem) => {
|
||||
// Cancellable parse statuses mirror the backend CancelKnowledgeParse
|
||||
// gate: pending / processing / finalizing all surface the stop entry,
|
||||
// while completed / failed / cancelled / deleting hide it.
|
||||
const CANCELABLE_PARSE_STATUSES = new Set(['pending', 'processing', 'finalizing']);
|
||||
const canCancelParse = (item: KnowledgeItem) =>
|
||||
CANCELABLE_PARSE_STATUSES.has(String(item.parse_status ?? ''));
|
||||
|
||||
const handleAction = (action: 'edit' | 'reparse' | 'cancel-parse' | 'move' | 'delete', item: KnowledgeItem) => {
|
||||
moreOpen.value = null;
|
||||
item.isMore = false;
|
||||
emit('action', action, item);
|
||||
@@ -284,6 +309,14 @@ const handleAction = (action: 'edit' | 'reparse' | 'move' | 'delete', item: Know
|
||||
<t-icon class="icon" name="refresh" />
|
||||
<span>{{ t('knowledgeBase.rebuildDocument') }}</span>
|
||||
</div>
|
||||
<div
|
||||
v-if="canCancelParse(item)"
|
||||
class="row-menu-item danger"
|
||||
@click.stop="handleAction('cancel-parse', item)"
|
||||
>
|
||||
<t-icon class="icon" name="close-circle" />
|
||||
<span>{{ t('knowledgeBase.cancelParse') }}</span>
|
||||
</div>
|
||||
<div class="row-menu-item" @click.stop="handleAction('move', item)">
|
||||
<t-icon class="icon" name="swap" />
|
||||
<span>{{ t('knowledgeBase.moveDocument') }}</span>
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
@@ -308,6 +309,102 @@ func (r *knowledgeRepository) UpdateKnowledgeColumns(
|
||||
return r.db.WithContext(ctx).Model(&types.Knowledge{}).Where("id = ?", id).Updates(values).Error
|
||||
}
|
||||
|
||||
// FinalizeSubtask atomically decrements pending_subtasks_count and, when
|
||||
// the counter reaches zero while parse_status is still 'finalizing',
|
||||
// flips the row to 'completed' in the same statement so concurrent
|
||||
// subtask completions can't race the promotion.
|
||||
//
|
||||
// Returns (newCount, promoted, error). promoted is true iff this caller
|
||||
// was the one whose UPDATE flipped 'finalizing'→'completed'.
|
||||
//
|
||||
// The implementation is two statements (atomic decrement, then a guarded
|
||||
// promote UPDATE) because GORM does not expose a portable RETURNING
|
||||
// across PostgreSQL and SQLite. The promote UPDATE's WHERE clause
|
||||
// (parse_status='finalizing' AND pending_subtasks_count=0) makes it
|
||||
// safe to run from any number of concurrent callers — at most one wins.
|
||||
func (r *knowledgeRepository) FinalizeSubtask(
|
||||
ctx context.Context, id string,
|
||||
) (int, bool, error) {
|
||||
now := time.Now()
|
||||
// 1) Atomic decrement, clamped at zero. The `pending_subtasks_count > 0`
|
||||
// guard is purely a safety net for accounting bugs — under normal
|
||||
// operation each subtask handler decrements at most once per task,
|
||||
// so the counter cannot go negative.
|
||||
res := r.db.WithContext(ctx).Model(&types.Knowledge{}).
|
||||
Where("id = ? AND pending_subtasks_count > 0", id).
|
||||
Updates(map[string]interface{}{
|
||||
"pending_subtasks_count": gorm.Expr("pending_subtasks_count - 1"),
|
||||
"updated_at": now,
|
||||
})
|
||||
if res.Error != nil {
|
||||
return 0, false, res.Error
|
||||
}
|
||||
|
||||
// 2) Re-read to discover the new count and current parse_status.
|
||||
// Reading after the UPDATE gives us a value that is at worst one
|
||||
// decrement stale relative to other callers — the promote step
|
||||
// below is guarded by the same condition at SQL level, so the
|
||||
// stale read only causes us to attempt a no-op promote.
|
||||
var snap struct {
|
||||
PendingSubtasksCount int `gorm:"column:pending_subtasks_count"`
|
||||
ParseStatus string `gorm:"column:parse_status"`
|
||||
}
|
||||
if err := r.db.WithContext(ctx).Model(&types.Knowledge{}).
|
||||
Select("pending_subtasks_count", "parse_status").
|
||||
Where("id = ?", id).Take(&snap).Error; err != nil {
|
||||
return 0, false, err
|
||||
}
|
||||
|
||||
if snap.PendingSubtasksCount != 0 || snap.ParseStatus != types.ParseStatusFinalizing {
|
||||
return snap.PendingSubtasksCount, false, nil
|
||||
}
|
||||
|
||||
// 3) Guarded promote. The WHERE clause ensures only ONE caller wins
|
||||
// when multiple subtasks decrement to zero in the same instant,
|
||||
// and that cancel/delete cannot be clobbered by a late promote.
|
||||
promoteRes := r.db.WithContext(ctx).Model(&types.Knowledge{}).
|
||||
Where("id = ? AND parse_status = ? AND pending_subtasks_count = 0",
|
||||
id, types.ParseStatusFinalizing).
|
||||
Updates(map[string]interface{}{
|
||||
"parse_status": types.ParseStatusCompleted,
|
||||
"processed_at": now,
|
||||
"updated_at": now,
|
||||
})
|
||||
if promoteRes.Error != nil {
|
||||
return snap.PendingSubtasksCount, false, promoteRes.Error
|
||||
}
|
||||
return snap.PendingSubtasksCount, promoteRes.RowsAffected > 0, nil
|
||||
}
|
||||
|
||||
// SetFinalizing atomically transitions a row from 'processing' to
|
||||
// 'finalizing' and seeds pending_subtasks_count. Used by
|
||||
// KnowledgePostProcess.Handle as the single durable handoff between
|
||||
// the synchronous parse stage and the asynchronous enrichment fan-out.
|
||||
//
|
||||
// The transition is conditional on parse_status='processing' so a row
|
||||
// that the user cancelled / deleted between ProcessDocument finishing
|
||||
// and post-process starting will NOT get hijacked into finalizing.
|
||||
// Returns whether the transition happened.
|
||||
func (r *knowledgeRepository) SetFinalizing(
|
||||
ctx context.Context, id string, expectedSubtasks int,
|
||||
) (bool, error) {
|
||||
if expectedSubtasks < 0 {
|
||||
expectedSubtasks = 0
|
||||
}
|
||||
now := time.Now()
|
||||
res := r.db.WithContext(ctx).Model(&types.Knowledge{}).
|
||||
Where("id = ? AND parse_status = ?", id, types.ParseStatusProcessing).
|
||||
Updates(map[string]interface{}{
|
||||
"parse_status": types.ParseStatusFinalizing,
|
||||
"pending_subtasks_count": expectedSubtasks,
|
||||
"updated_at": now,
|
||||
})
|
||||
if res.Error != nil {
|
||||
return false, res.Error
|
||||
}
|
||||
return res.RowsAffected > 0, nil
|
||||
}
|
||||
|
||||
// CountKnowledgeByKnowledgeBaseID counts the number of knowledge items in a knowledge base
|
||||
func (r *knowledgeRepository) CountKnowledgeByKnowledgeBaseID(
|
||||
ctx context.Context,
|
||||
|
||||
@@ -3,6 +3,7 @@ package repository
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"gorm.io/gorm"
|
||||
@@ -29,6 +30,14 @@ type KnowledgeSpanRepository interface {
|
||||
// cascade an upstream failure across a stage's downstream subtree
|
||||
// without iterating in Go memory.
|
||||
CancelDescendants(ctx context.Context, knowledgeID string, attempt int, parentSpanID, reason string) (int64, error)
|
||||
// CancelAllOpenSpans flips every non-terminal (pending/running) span
|
||||
// for (knowledgeID, attempt) to "cancelled" in one statement,
|
||||
// regardless of tree position. Used by the user-cancel path where
|
||||
// fan-out stages (e.g. "多模态识别") flip themselves to done as soon
|
||||
// as they finish dispatching, while their async children are still
|
||||
// running — a tree walk that stops at terminal parents would miss
|
||||
// those orphan leaves.
|
||||
CancelAllOpenSpans(ctx context.Context, knowledgeID string, attempt int, errorCode, reason string) (int64, error)
|
||||
}
|
||||
|
||||
type knowledgeSpanRepository struct {
|
||||
@@ -189,3 +198,33 @@ func (r *knowledgeSpanRepository) CancelDescendants(ctx context.Context, knowled
|
||||
}
|
||||
return totalAffected, nil
|
||||
}
|
||||
|
||||
// CancelAllOpenSpans is the "abort the attempt" counterpart to
|
||||
// CancelDescendants. It avoids the BFS entirely so spans whose parent
|
||||
// is already terminal (typical for stage fan-outs that EndSpan as soon
|
||||
// as they finish dispatching async work) still get flipped to cancelled.
|
||||
// We deliberately do NOT touch finished_at / duration_ms here — the
|
||||
// span row remains observable in the trace tree with its original
|
||||
// start time and gets a cancelled status + reason, which is enough
|
||||
// for the UI to drop the running-bar styling.
|
||||
func (r *knowledgeSpanRepository) CancelAllOpenSpans(
|
||||
ctx context.Context, knowledgeID string, attempt int, errorCode, reason string,
|
||||
) (int64, error) {
|
||||
now := time.Now()
|
||||
updates := map[string]any{
|
||||
"status": types.SpanStatusCancelled,
|
||||
"error_code": errorCode,
|
||||
"error_message": reason,
|
||||
"finished_at": now,
|
||||
"updated_at": now,
|
||||
}
|
||||
res := r.db.WithContext(ctx).Model(&types.KnowledgeProcessingSpan{}).
|
||||
Where("knowledge_id = ? AND attempt = ? AND status IN ?",
|
||||
knowledgeID, attempt,
|
||||
[]string{types.SpanStatusPending, types.SpanStatusRunning}).
|
||||
Updates(updates)
|
||||
if res.Error != nil {
|
||||
return 0, res.Error
|
||||
}
|
||||
return res.RowsAffected, nil
|
||||
}
|
||||
|
||||
@@ -154,6 +154,7 @@ type ChunkExtractService struct {
|
||||
template *types.PromptTemplateStructured
|
||||
modelService interfaces.ModelService
|
||||
knowledgeBaseRepo interfaces.KnowledgeBaseRepository
|
||||
knowledgeRepo interfaces.KnowledgeRepository
|
||||
chunkRepo interfaces.ChunkRepository
|
||||
graphEngine interfaces.RetrieveGraphRepository
|
||||
// spanTracker records this graph-extract task's subspan under the
|
||||
@@ -167,6 +168,7 @@ func NewChunkExtractService(
|
||||
config *config.Config,
|
||||
modelService interfaces.ModelService,
|
||||
knowledgeBaseRepo interfaces.KnowledgeBaseRepository,
|
||||
knowledgeRepo interfaces.KnowledgeRepository,
|
||||
chunkRepo interfaces.ChunkRepository,
|
||||
graphEngine interfaces.RetrieveGraphRepository,
|
||||
spanTracker SpanTracker,
|
||||
@@ -175,6 +177,7 @@ func NewChunkExtractService(
|
||||
template: config.ExtractManager.ExtractGraph,
|
||||
modelService: modelService,
|
||||
knowledgeBaseRepo: knowledgeBaseRepo,
|
||||
knowledgeRepo: knowledgeRepo,
|
||||
chunkRepo: chunkRepo,
|
||||
graphEngine: graphEngine,
|
||||
spanTracker: spanTracker,
|
||||
@@ -220,6 +223,15 @@ func (s *ChunkExtractService) Handle(ctx context.Context, t *asynq.Task) error {
|
||||
var handleErr error
|
||||
graphOut := types.JSONMap{}
|
||||
defer func() {
|
||||
// Decrement the parent's enrichment counter on terminal exit so a
|
||||
// completed (or terminally-failed) per-chunk extract releases its
|
||||
// slot in pending_subtasks_count. KnowledgeID is the new (post-#? )
|
||||
// payload field; legacy in-flight tasks without it are skipped.
|
||||
if (handleErr == nil || isFinalAsynqAttempt(ctx)) && p.KnowledgeID != "" && s.knowledgeRepo != nil {
|
||||
if _, _, ferr := s.knowledgeRepo.FinalizeSubtask(ctx, p.KnowledgeID); ferr != nil {
|
||||
logger.Warnf(ctx, "graph extract: FinalizeSubtask failed for %s: %v", p.KnowledgeID, ferr)
|
||||
}
|
||||
}
|
||||
if gSpan == nil {
|
||||
return
|
||||
}
|
||||
@@ -230,6 +242,22 @@ func (s *ChunkExtractService) Handle(ctx context.Context, t *asynq.Task) error {
|
||||
}
|
||||
}()
|
||||
|
||||
// Short-circuit when the parent knowledge has been cancelled / deleted.
|
||||
// Each graph extract is per-chunk and runs one LLM call — the most
|
||||
// expensive enrichment fan-out in the pipeline. Skipping on cancel
|
||||
// is the whole point of the finalizing-state machinery above.
|
||||
if p.KnowledgeID != "" && s.knowledgeRepo != nil {
|
||||
if k, kerr := s.knowledgeRepo.GetKnowledgeByIDOnly(ctx, p.KnowledgeID); kerr == nil && k != nil {
|
||||
switch k.ParseStatus {
|
||||
case types.ParseStatusCancelled, types.ParseStatusDeleting:
|
||||
logger.Infof(ctx, "graph extract: knowledge %s aborted (%s), skipping chunk %s",
|
||||
p.KnowledgeID, k.ParseStatus, p.ChunkID)
|
||||
graphOut["skipped"] = "knowledge_" + k.ParseStatus
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
chunk, err := s.chunkRepo.GetChunkByID(ctx, p.TenantID, p.ChunkID)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "failed to get chunk: %v", err)
|
||||
|
||||
@@ -130,6 +130,18 @@ func (s *ImageMultimodalService) Handle(ctx context.Context, task *asynq.Task) e
|
||||
ctx = context.WithValue(ctx, types.LanguageContextKey, payload.Language)
|
||||
}
|
||||
|
||||
// Short-circuit when the parent knowledge has been cancelled by the user
|
||||
// or marked for deletion. Skip the VLM call entirely so we don't burn
|
||||
// model quota on already-aborted work.
|
||||
if k, kerr := s.knowledgeRepo.GetKnowledgeByIDOnly(ctx, payload.KnowledgeID); kerr == nil && k != nil {
|
||||
switch k.ParseStatus {
|
||||
case types.ParseStatusCancelled, types.ParseStatusDeleting:
|
||||
logger.Infof(ctx, "[ImageMultimodal] Knowledge %s aborted (%s), skipping image %s",
|
||||
payload.KnowledgeID, k.ParseStatus, payload.ImageURL)
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// Open a per-image subspan under the parent attempt's multimodal
|
||||
// stage. If the parent stage row is missing (legacy in-flight
|
||||
// task, or the upstream code shipped without span tracking), the
|
||||
|
||||
@@ -53,6 +53,7 @@ type knowledgeService struct {
|
||||
fileSvc interfaces.FileService
|
||||
modelService interfaces.ModelService
|
||||
task interfaces.TaskEnqueuer
|
||||
taskInspector interfaces.TaskInspector
|
||||
graphEngine interfaces.RetrieveGraphRepository
|
||||
redisClient *redis.Client
|
||||
kbShareService interfaces.KBShareService
|
||||
@@ -93,6 +94,7 @@ func NewKnowledgeService(
|
||||
fileSvc interfaces.FileService,
|
||||
modelService interfaces.ModelService,
|
||||
task interfaces.TaskEnqueuer,
|
||||
taskInspector interfaces.TaskInspector,
|
||||
graphEngine interfaces.RetrieveGraphRepository,
|
||||
retrieveEngine interfaces.RetrieveEngineRegistry,
|
||||
ownership retriever.TenantStoreOwnership,
|
||||
@@ -118,6 +120,7 @@ func NewKnowledgeService(
|
||||
fileSvc: fileSvc,
|
||||
modelService: modelService,
|
||||
task: task,
|
||||
taskInspector: taskInspector,
|
||||
graphEngine: graphEngine,
|
||||
retrieveEngine: retrieveEngine,
|
||||
ownership: ownership,
|
||||
@@ -297,6 +300,33 @@ func (s *knowledgeService) isKnowledgeDeleting(ctx context.Context, tenantID uin
|
||||
return knowledge.ParseStatus == types.ParseStatusDeleting
|
||||
}
|
||||
|
||||
// isKnowledgeAborted returns (true, status) when the knowledge has been
|
||||
// marked as deleting OR cancelled so async pipeline workers should bail
|
||||
// out. Status is returned so callers can branch on cleanup behavior:
|
||||
// deleting → existing cleanup of partial chunks/index applies;
|
||||
// cancelled → keep partially written data per user expectation.
|
||||
//
|
||||
// When the row is missing or unreadable we conservatively return
|
||||
// (true, ParseStatusDeleting): the existing deleting branch already
|
||||
// handles cleanup-or-no-op semantics safely.
|
||||
func (s *knowledgeService) isKnowledgeAborted(
|
||||
ctx context.Context, tenantID uint64, knowledgeID string,
|
||||
) (bool, string) {
|
||||
knowledge, err := s.repo.GetKnowledgeByID(ctx, tenantID, knowledgeID)
|
||||
if err != nil {
|
||||
logger.Warnf(ctx, "Failed to check knowledge abort status (assuming deleted): %v", err)
|
||||
return true, types.ParseStatusDeleting
|
||||
}
|
||||
if knowledge == nil {
|
||||
return true, types.ParseStatusDeleting
|
||||
}
|
||||
switch knowledge.ParseStatus {
|
||||
case types.ParseStatusDeleting, types.ParseStatusCancelled:
|
||||
return true, knowledge.ParseStatus
|
||||
}
|
||||
return false, knowledge.ParseStatus
|
||||
}
|
||||
|
||||
// checkStorageEngineConfigured verifies that the knowledge base has a storage engine configured
|
||||
// (either at the KB level or via the tenant default).
|
||||
//
|
||||
|
||||
@@ -75,6 +75,17 @@ func (s *knowledgeService) DeleteKnowledge(ctx context.Context, id string) error
|
||||
logger.Infof(ctx, "Marked knowledge %s as deleting (previous status: %s)", id, originalStatus)
|
||||
}
|
||||
|
||||
// Best-effort: purge any queued downstream tasks for this knowledge
|
||||
// (multimodal / post-process / question / summary / graph extract).
|
||||
// Worker checkpoints already drop them on the floor, but dequeuing
|
||||
// here avoids waking workers just to no-op when the parse was still
|
||||
// in flight at delete time. No-op in Lite mode and on completed rows
|
||||
// (no queued descendants anyway).
|
||||
if originalStatus == types.ParseStatusPending ||
|
||||
originalStatus == types.ParseStatusProcessing {
|
||||
s.dequeueKnowledgeTasks(ctx, id)
|
||||
}
|
||||
|
||||
// Resolve file service for this KB before spawning goroutines
|
||||
kb, _ := s.kbService.GetKnowledgeBaseByID(ctx, knowledge.KnowledgeBaseID)
|
||||
kbFileSvc := s.resolveFileService(ctx, kb)
|
||||
@@ -392,8 +403,12 @@ func (s *knowledgeService) DeleteKnowledgeList(ctx context.Context, ids []string
|
||||
return err
|
||||
}
|
||||
|
||||
// Mark all as deleting first to prevent async task conflicts
|
||||
// Mark all as deleting first to prevent async task conflicts.
|
||||
// Remember which entries still had queued / in-flight downstream tasks
|
||||
// so we can dequeue them in one pass after marking.
|
||||
var inFlightIDs []string
|
||||
for _, knowledge := range knowledgeList {
|
||||
prev := knowledge.ParseStatus
|
||||
knowledge.ParseStatus = types.ParseStatusDeleting
|
||||
knowledge.UpdatedAt = time.Now()
|
||||
if err := s.repo.UpdateKnowledge(ctx, knowledge); err != nil {
|
||||
@@ -401,9 +416,19 @@ func (s *knowledgeService) DeleteKnowledgeList(ctx context.Context, ids []string
|
||||
Errorf("DeleteKnowledgeList failed to mark as deleting")
|
||||
// Continue with deletion even if marking fails
|
||||
}
|
||||
if prev == types.ParseStatusPending || prev == types.ParseStatusProcessing {
|
||||
inFlightIDs = append(inFlightIDs, knowledge.ID)
|
||||
}
|
||||
}
|
||||
logger.Infof(ctx, "Marked %d knowledge entries as deleting", len(knowledgeList))
|
||||
|
||||
// Best-effort dequeue of downstream tasks for in-flight entries.
|
||||
// See DeleteKnowledge for the rationale; loop is per-knowledge because
|
||||
// the inspector only filters by knowledge_id, not by ID set.
|
||||
for _, kid := range inFlightIDs {
|
||||
s.dequeueKnowledgeTasks(ctx, kid)
|
||||
}
|
||||
|
||||
// Pre-resolve file services per KB so goroutines don't need DB access
|
||||
kbFileServices := make(map[string]interfaces.FileService)
|
||||
for _, knowledge := range knowledgeList {
|
||||
|
||||
@@ -116,9 +116,16 @@ func (h *HousekeepingService) runSweep(ctx context.Context) {
|
||||
// Knowledge rows with no spans at all (lite mode, in-flight tasks
|
||||
// from before this code shipped) fall back to the simple
|
||||
// updated_at check — they have no heartbeat to consult.
|
||||
// Include 'finalizing' alongside 'processing': finalizing rows still
|
||||
// consume LLM compute via enrichment subtasks (summary/question/graph),
|
||||
// and the same stall modes (subtask worker dies, retry budget exhausted
|
||||
// without decrementing the counter) leave the row hanging just as
|
||||
// visibly. Housekeeping promotes both states to 'failed' once the
|
||||
// span heartbeat is older than the threshold.
|
||||
var candidates []types.Knowledge
|
||||
if err := h.db.WithContext(ctx).
|
||||
Where("parse_status = ? AND updated_at < ?", types.ParseStatusProcessing, cutoff).
|
||||
Where("parse_status IN ? AND updated_at < ?",
|
||||
[]string{types.ParseStatusProcessing, types.ParseStatusFinalizing}, cutoff).
|
||||
Find(&candidates).Error; err != nil {
|
||||
logger.Warnf(ctx, "[Housekeeping] knowledge candidate query failed: %v", err)
|
||||
return
|
||||
@@ -131,10 +138,12 @@ func (h *HousekeepingService) runSweep(ctx context.Context) {
|
||||
stuckIDs = append(stuckIDs, k.ID)
|
||||
}
|
||||
res := h.db.WithContext(ctx).Model(&types.Knowledge{}).
|
||||
Where("id IN ? AND parse_status = ?", stuckIDs, types.ParseStatusProcessing).
|
||||
Where("id IN ? AND parse_status IN ?", stuckIDs,
|
||||
[]string{types.ParseStatusProcessing, types.ParseStatusFinalizing}).
|
||||
Updates(map[string]interface{}{
|
||||
"parse_status": types.ParseStatusFailed,
|
||||
"error_message": "task stuck in processing > " + threshold.String() + ", recovered by housekeeping",
|
||||
"parse_status": types.ParseStatusFailed,
|
||||
"error_message": "task stuck in processing > " + threshold.String() + ", recovered by housekeeping",
|
||||
"pending_subtasks_count": 0,
|
||||
})
|
||||
if res.Error != nil {
|
||||
logger.Warnf(ctx, "[Housekeeping] knowledge sweep update failed: %v", res.Error)
|
||||
|
||||
@@ -27,6 +27,7 @@ CREATE TABLE IF NOT EXISTS knowledges (
|
||||
knowledge_base_id VARCHAR(64),
|
||||
parse_status VARCHAR(32) NOT NULL DEFAULT 'pending',
|
||||
summary_status VARCHAR(32) NOT NULL DEFAULT 'none',
|
||||
pending_subtasks_count INTEGER NOT NULL DEFAULT 0,
|
||||
error_message TEXT,
|
||||
title TEXT,
|
||||
file_type TEXT,
|
||||
|
||||
@@ -100,6 +100,24 @@ func (s *KnowledgePostProcessService) Handle(ctx context.Context, task *asynq.Ta
|
||||
return nil
|
||||
}
|
||||
|
||||
// Skip post-processing entirely when the knowledge has been cancelled
|
||||
// by the user or marked for deletion. We must NOT enqueue summary /
|
||||
// question / graph / wiki child tasks for an aborted knowledge. We
|
||||
// MUST also close postSpan before returning, otherwise it stays in
|
||||
// running state forever and the trace viewer shows an orange bar
|
||||
// long after the user cancelled (the AbortAttempt sweep ran before
|
||||
// we opened postSpan, so the sweep didn't catch this row).
|
||||
switch knowledge.ParseStatus {
|
||||
case types.ParseStatusCancelled, types.ParseStatusDeleting:
|
||||
logger.Infof(ctx,
|
||||
"[KnowledgePostProcess] Knowledge %s aborted (%s), skipping post-processing.",
|
||||
payload.KnowledgeID, knowledge.ParseStatus,
|
||||
)
|
||||
s.tracker().SkipSpan(ctx, postSpan,
|
||||
"knowledge "+knowledge.ParseStatus+" before postprocess started")
|
||||
return nil
|
||||
}
|
||||
|
||||
kb, err := s.kbService.GetKnowledgeBaseByIDOnly(ctx, payload.KnowledgeBaseID)
|
||||
if err != nil || kb == nil {
|
||||
return fmt.Errorf("get knowledge base %s: %w", payload.KnowledgeBaseID, err)
|
||||
@@ -119,42 +137,121 @@ func (s *KnowledgePostProcessService) Handle(ctx context.Context, task *asynq.Ta
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Update ParseStatus to Completed
|
||||
// (Except if it's already completed or if it was marked as failed/deleting, but we'll just set it to completed if it's processing)
|
||||
if knowledge.ParseStatus == types.ParseStatusProcessing {
|
||||
knowledge.ParseStatus = types.ParseStatusCompleted
|
||||
knowledge.UpdatedAt = time.Now()
|
||||
// 3. Compute the enrichment subtask count up front so we can flip to
|
||||
// "finalizing" with the right counter BEFORE spawning any subtasks.
|
||||
// Each subtask handler atomically decrements pending_subtasks_count
|
||||
// on its terminal exit; the row promotes itself to "completed" when
|
||||
// the counter hits zero (see knowledgeRepository.FinalizeSubtask).
|
||||
//
|
||||
// Wiki ingest is NOT counted here — it's a KB-scoped debounced
|
||||
// batch with its own dedup queue; per-knowledge accounting is not
|
||||
// meaningful for it.
|
||||
willSpawnSummary := len(textChunks) > 0
|
||||
willSpawnQuestion := willSpawnSummary && kb.NeedsEmbeddingModel() &&
|
||||
kb.QuestionGenerationConfig != nil && kb.QuestionGenerationConfig.Enabled
|
||||
graphChunkCount := 0
|
||||
if kb.IsGraphEnabled() {
|
||||
graphChunkCount = len(textChunks)
|
||||
}
|
||||
expectedSubtasks := 0
|
||||
if willSpawnSummary {
|
||||
expectedSubtasks++
|
||||
}
|
||||
if willSpawnQuestion {
|
||||
expectedSubtasks++
|
||||
}
|
||||
expectedSubtasks += graphChunkCount
|
||||
|
||||
// Setup summary status
|
||||
if len(textChunks) > 0 {
|
||||
knowledge.SummaryStatus = types.SummaryStatusPending
|
||||
} else {
|
||||
knowledge.SummaryStatus = types.SummaryStatusNone
|
||||
switch {
|
||||
case knowledge.ParseStatus != types.ParseStatusProcessing:
|
||||
// The row was already in some other state (deleting / cancelled /
|
||||
// failed / completed) when we arrived. Don't touch parse_status
|
||||
// and don't spawn enrichment — the upstream that put the row in
|
||||
// that state has already decided this attempt is over.
|
||||
logger.Infof(ctx, "[KnowledgePostProcess] Knowledge %s is in %s, skipping enrichment fan-out.",
|
||||
payload.KnowledgeID, knowledge.ParseStatus)
|
||||
s.tracker().EndSpan(ctx, postSpan, types.JSONMap{
|
||||
"skipped": "non_processing_status",
|
||||
"observed_status": knowledge.ParseStatus,
|
||||
})
|
||||
s.tracker().FinalizeAttempt(ctx, payload.KnowledgeID, attempt,
|
||||
types.SpanStatusDone, types.JSONMap{
|
||||
"skipped": "non_processing_status",
|
||||
"observed_status": knowledge.ParseStatus,
|
||||
}, "", "")
|
||||
return nil
|
||||
case expectedSubtasks == 0:
|
||||
// Nothing to enrich — fast path keeps the previous behavior so
|
||||
// users without summary/question/graph see 'completed' immediately.
|
||||
updates := map[string]interface{}{
|
||||
"parse_status": types.ParseStatusCompleted,
|
||||
"updated_at": time.Now(),
|
||||
}
|
||||
|
||||
if err := s.knowledgeRepo.UpdateKnowledge(ctx, knowledge); err != nil {
|
||||
logger.Warnf(ctx, "[KnowledgePostProcess] Failed to update knowledge status to completed: %v", err)
|
||||
if len(textChunks) > 0 {
|
||||
updates["summary_status"] = types.SummaryStatusNone
|
||||
}
|
||||
if err := s.knowledgeRepo.UpdateKnowledgeColumns(ctx, payload.KnowledgeID, updates); err != nil {
|
||||
logger.Warnf(ctx, "[KnowledgePostProcess] Failed to mark %s completed (no subtasks): %v",
|
||||
payload.KnowledgeID, err)
|
||||
} else {
|
||||
logger.Infof(ctx, "[KnowledgePostProcess] Knowledge %s marked as completed.", payload.KnowledgeID)
|
||||
logger.Infof(ctx, "[KnowledgePostProcess] Knowledge %s marked completed (no enrichment subtasks).",
|
||||
payload.KnowledgeID)
|
||||
}
|
||||
default:
|
||||
// Flip processing → finalizing in one statement so a parallel
|
||||
// cancel/delete cannot race us into completed.
|
||||
promoted, err := s.knowledgeRepo.SetFinalizing(ctx, payload.KnowledgeID, expectedSubtasks)
|
||||
if err != nil {
|
||||
logger.Warnf(ctx, "[KnowledgePostProcess] SetFinalizing failed for %s: %v",
|
||||
payload.KnowledgeID, err)
|
||||
}
|
||||
if promoted {
|
||||
// Reflect summary status separately so the UI shows the
|
||||
// summary as queued for users who already had it visible.
|
||||
summaryStatus := types.SummaryStatusNone
|
||||
if willSpawnSummary {
|
||||
summaryStatus = types.SummaryStatusPending
|
||||
}
|
||||
if err := s.knowledgeRepo.UpdateKnowledgeColumn(ctx,
|
||||
payload.KnowledgeID, "summary_status", summaryStatus); err != nil {
|
||||
logger.Warnf(ctx, "[KnowledgePostProcess] Failed to update summary_status for %s: %v",
|
||||
payload.KnowledgeID, err)
|
||||
}
|
||||
logger.Infof(ctx,
|
||||
"[KnowledgePostProcess] Knowledge %s entered finalizing (pending_subtasks=%d).",
|
||||
payload.KnowledgeID, expectedSubtasks)
|
||||
} else {
|
||||
// Row was no longer 'processing' (cancel / delete won the race).
|
||||
// Skip enrichment entirely so we don't waste LLM quota on a row
|
||||
// the user already abandoned.
|
||||
logger.Infof(ctx,
|
||||
"[KnowledgePostProcess] Knowledge %s no longer in processing, skipping enrichment fan-out.",
|
||||
payload.KnowledgeID)
|
||||
s.tracker().EndSpan(ctx, postSpan, types.JSONMap{
|
||||
"skipped": "knowledge_no_longer_processing",
|
||||
})
|
||||
s.tracker().FinalizeAttempt(ctx, payload.KnowledgeID, attempt,
|
||||
types.SpanStatusDone, types.JSONMap{
|
||||
"skipped": "knowledge_no_longer_processing",
|
||||
}, "", "")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Spawn Summary and Question Tasks
|
||||
enqueuedSummary := false
|
||||
enqueuedQuestion := false
|
||||
if len(textChunks) > 0 {
|
||||
if willSpawnSummary {
|
||||
s.enqueueSummaryGenerationTask(ctx, payload, attempt)
|
||||
enqueuedSummary = true
|
||||
// Question generation only makes sense for RAG indexing (improves chunk recall).
|
||||
// Skip when only Wiki/Graph is enabled without vector/keyword search.
|
||||
if kb.NeedsEmbeddingModel() {
|
||||
if willSpawnQuestion {
|
||||
enqueuedQuestion = s.enqueueQuestionGenerationIfEnabled(ctx, payload, kb, attempt)
|
||||
}
|
||||
}
|
||||
|
||||
// 5. Spawn Graph RAG Tasks — only when graph indexing is enabled in IndexingStrategy
|
||||
enqueuedGraph := false
|
||||
if kb.IsGraphEnabled() {
|
||||
if graphChunkCount > 0 {
|
||||
logger.Infof(ctx, "[KnowledgePostProcess] Spawning Graph RAG extract tasks for %d text-like chunks", len(textChunks))
|
||||
for i, chunk := range textChunks {
|
||||
err := NewChunkExtractTask(ctx, s.taskEnqueuer, payload.TenantID, chunk.ID, kb.SummaryModelID,
|
||||
@@ -163,7 +260,7 @@ func (s *KnowledgePostProcessService) Handle(ctx context.Context, task *asynq.Ta
|
||||
logger.Errorf(ctx, "[KnowledgePostProcess] Failed to create chunk extract task for %s: %v", chunk.ID, err)
|
||||
}
|
||||
}
|
||||
enqueuedGraph = len(textChunks) > 0
|
||||
enqueuedGraph = true
|
||||
}
|
||||
|
||||
// 6. Spawn Wiki Ingest Task if wiki indexing is enabled in IndexingStrategy
|
||||
|
||||
@@ -240,10 +240,12 @@ func (s *knowledgeService) processChunks(ctx context.Context,
|
||||
attribute.Int("chunk_count", len(chunks)),
|
||||
)
|
||||
|
||||
// Check if knowledge is being deleted before processing
|
||||
if s.isKnowledgeDeleting(ctx, knowledge.TenantID, knowledge.ID) {
|
||||
logger.Infof(ctx, "Knowledge is being deleted, aborting chunk processing: %s", knowledge.ID)
|
||||
span.AddEvent("aborted: knowledge is being deleted")
|
||||
// Check if knowledge is being deleted/cancelled before processing.
|
||||
// Both statuses short-circuit identically here — there's nothing to clean
|
||||
// up yet so the branch is purely "stop early".
|
||||
if aborted, status := s.isKnowledgeAborted(ctx, knowledge.TenantID, knowledge.ID); aborted {
|
||||
logger.Infof(ctx, "Knowledge aborted (%s), skipping chunk processing: %s", status, knowledge.ID)
|
||||
span.AddEvent("aborted: knowledge " + status)
|
||||
return
|
||||
}
|
||||
|
||||
@@ -455,10 +457,11 @@ func (s *knowledgeService) processChunks(ctx context.Context,
|
||||
}
|
||||
}
|
||||
|
||||
// Check if knowledge is being deleted before writing to database
|
||||
if s.isKnowledgeDeleting(ctx, knowledge.TenantID, knowledge.ID) {
|
||||
logger.Infof(ctx, "Knowledge is being deleted, aborting before saving chunks: %s", knowledge.ID)
|
||||
span.AddEvent("aborted: knowledge is being deleted before saving")
|
||||
// Check if knowledge is being deleted/cancelled before writing chunks.
|
||||
// Nothing has been persisted yet, so both branches just bail.
|
||||
if aborted, status := s.isKnowledgeAborted(ctx, knowledge.TenantID, knowledge.ID); aborted {
|
||||
logger.Infof(ctx, "Knowledge aborted (%s), skipping chunk write: %s", status, knowledge.ID)
|
||||
span.AddEvent("aborted: knowledge " + status + " before saving")
|
||||
return
|
||||
}
|
||||
|
||||
@@ -550,14 +553,17 @@ func (s *knowledgeService) processChunks(ctx context.Context,
|
||||
}
|
||||
}
|
||||
|
||||
// Check again before batch indexing (this is a heavy operation)
|
||||
if s.isKnowledgeDeleting(ctx, knowledge.TenantID, knowledge.ID) {
|
||||
logger.Infof(ctx, "Knowledge is being deleted, cleaning up and aborting before indexing: %s", knowledge.ID)
|
||||
// Clean up the chunks we just created
|
||||
if err := s.chunkService.DeleteChunksByKnowledgeID(ctx, knowledge.ID); err != nil {
|
||||
logger.Warnf(ctx, "Failed to cleanup chunks after deletion detected: %v", err)
|
||||
// Check again before batch indexing (heavy operation).
|
||||
// deleting → row is going away anyway, drop the chunks we just wrote.
|
||||
// cancelled → user wants to keep what was already persisted, just stop.
|
||||
if aborted, status := s.isKnowledgeAborted(ctx, knowledge.TenantID, knowledge.ID); aborted {
|
||||
logger.Infof(ctx, "Knowledge aborted (%s) before indexing: %s", status, knowledge.ID)
|
||||
if status == types.ParseStatusDeleting {
|
||||
if err := s.chunkService.DeleteChunksByKnowledgeID(ctx, knowledge.ID); err != nil {
|
||||
logger.Warnf(ctx, "Failed to cleanup chunks after deletion detected: %v", err)
|
||||
}
|
||||
}
|
||||
span.AddEvent("aborted: knowledge is being deleted before indexing")
|
||||
span.AddEvent("aborted: knowledge " + status + " before indexing")
|
||||
return
|
||||
}
|
||||
|
||||
@@ -597,17 +603,21 @@ func (s *knowledgeService) processChunks(ctx context.Context,
|
||||
"storage_bytes": totalStorageSize,
|
||||
})
|
||||
|
||||
// Final check before marking as completed - if deleted during processing, don't update status
|
||||
if s.isKnowledgeDeleting(ctx, knowledge.TenantID, knowledge.ID) {
|
||||
logger.Infof(ctx, "Knowledge was deleted during processing, skipping completion update: %s", knowledge.ID)
|
||||
// Clean up the data we just created since the knowledge is being deleted
|
||||
if err := s.chunkService.DeleteChunksByKnowledgeID(ctx, knowledge.ID); err != nil {
|
||||
logger.Warnf(ctx, "Failed to cleanup chunks after deletion detected: %v", err)
|
||||
// Final check before marking as completed.
|
||||
// deleting → drop chunks+index we just wrote.
|
||||
// cancelled → keep persisted data; the row stays in cancelled status
|
||||
// and downstream stages skip via the entry guards.
|
||||
if aborted, status := s.isKnowledgeAborted(ctx, knowledge.TenantID, knowledge.ID); aborted {
|
||||
logger.Infof(ctx, "Knowledge aborted (%s) after indexing: %s", status, knowledge.ID)
|
||||
if status == types.ParseStatusDeleting {
|
||||
if err := s.chunkService.DeleteChunksByKnowledgeID(ctx, knowledge.ID); err != nil {
|
||||
logger.Warnf(ctx, "Failed to cleanup chunks after deletion detected: %v", err)
|
||||
}
|
||||
if err := retrieveEngine.DeleteByKnowledgeIDList(ctx, []string{knowledge.ID}, embeddingModel.GetDimensions(), kb.Type); err != nil {
|
||||
logger.Warnf(ctx, "Failed to cleanup index after deletion detected: %v", err)
|
||||
}
|
||||
}
|
||||
if err := retrieveEngine.DeleteByKnowledgeIDList(ctx, []string{knowledge.ID}, embeddingModel.GetDimensions(), kb.Type); err != nil {
|
||||
logger.Warnf(ctx, "Failed to cleanup index after deletion detected: %v", err)
|
||||
}
|
||||
span.AddEvent("aborted: knowledge was deleted during processing")
|
||||
span.AddEvent("aborted: knowledge " + status + " during processing")
|
||||
return
|
||||
}
|
||||
} else {
|
||||
@@ -899,6 +909,14 @@ func (s *knowledgeService) ProcessSummaryGeneration(ctx context.Context, t *asyn
|
||||
var summaryErr error
|
||||
summaryOut := types.JSONMap{}
|
||||
defer func() {
|
||||
// Decrement the parent's enrichment counter on every terminal
|
||||
// exit (success OR final retry failure). Intermediate retries
|
||||
// skip the decrement so the counter cannot drain prematurely.
|
||||
if (summaryErr == nil || isFinalAsynqAttempt(ctx)) && payload.KnowledgeID != "" {
|
||||
if _, _, ferr := s.repo.FinalizeSubtask(ctx, payload.KnowledgeID); ferr != nil {
|
||||
logger.Warnf(ctx, "summary: FinalizeSubtask failed for %s: %v", payload.KnowledgeID, ferr)
|
||||
}
|
||||
}
|
||||
if span == nil {
|
||||
return
|
||||
}
|
||||
@@ -935,6 +953,16 @@ func (s *knowledgeService) ProcessSummaryGeneration(ctx context.Context, t *asyn
|
||||
summaryErr = err
|
||||
return nil
|
||||
}
|
||||
// Short-circuit when the user cancelled parsing or the row is being deleted.
|
||||
if knowledge != nil {
|
||||
switch knowledge.ParseStatus {
|
||||
case types.ParseStatusCancelled, types.ParseStatusDeleting:
|
||||
logger.Infof(ctx, "Summary generation: knowledge aborted (%s), skipping: %s",
|
||||
knowledge.ParseStatus, payload.KnowledgeID)
|
||||
summaryOut["skipped"] = "knowledge_" + knowledge.ParseStatus
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// Update summary status to processing
|
||||
knowledge.SummaryStatus = types.SummaryStatusProcessing
|
||||
@@ -1172,6 +1200,16 @@ func (s *knowledgeService) ProcessQuestionGeneration(ctx context.Context, t *asy
|
||||
// what we already log to stdout.
|
||||
var qSpan *Span
|
||||
var qErr error
|
||||
// Decrement enrichment counter on terminal exit (success OR final
|
||||
// retry failure). Runs AFTER the stats-log defer below — defers
|
||||
// unwind LIFO, so this one declared first executes last.
|
||||
defer func() {
|
||||
if (qErr == nil || isFinalAsynqAttempt(ctx)) && payload.KnowledgeID != "" {
|
||||
if _, _, ferr := s.repo.FinalizeSubtask(ctx, payload.KnowledgeID); ferr != nil {
|
||||
logger.Warnf(ctx, "question: FinalizeSubtask failed for %s: %v", payload.KnowledgeID, ferr)
|
||||
}
|
||||
}
|
||||
}()
|
||||
defer func() {
|
||||
logger.Infof(
|
||||
ctx,
|
||||
@@ -1287,6 +1325,16 @@ func (s *knowledgeService) ProcessQuestionGeneration(ctx context.Context, t *asy
|
||||
qErr = err
|
||||
return nil
|
||||
}
|
||||
// Short-circuit when the user cancelled parsing or the row is being deleted.
|
||||
if knowledge != nil {
|
||||
switch knowledge.ParseStatus {
|
||||
case types.ParseStatusCancelled, types.ParseStatusDeleting:
|
||||
exitStatus = "knowledge_" + knowledge.ParseStatus
|
||||
logger.Infof(ctx, "Question generation: knowledge aborted (%s), skipping: %s",
|
||||
knowledge.ParseStatus, payload.KnowledgeID)
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// Get text chunks for this knowledge
|
||||
chunks, err := s.chunkService.ListChunksByKnowledgeID(ctx, payload.KnowledgeID)
|
||||
@@ -1591,6 +1639,10 @@ func (s *knowledgeService) ReparseKnowledge(ctx context.Context, knowledgeID str
|
||||
existing.Description = ""
|
||||
existing.ProcessedAt = nil
|
||||
existing.EmbeddingModelID = kb.EmbeddingModelID
|
||||
// Reset the enrichment counter so a leftover value from a
|
||||
// previous attempt (e.g. cancelled before all subtasks decremented)
|
||||
// cannot block the new finalizing transition later.
|
||||
existing.PendingSubtasksCount = 0
|
||||
|
||||
if err := s.repo.UpdateKnowledge(ctx, existing); err != nil {
|
||||
logger.Errorf(ctx, "Failed to update knowledge status before reparse: %v", err)
|
||||
@@ -1621,6 +1673,9 @@ func (s *knowledgeService) ReparseKnowledge(ctx context.Context, knowledgeID str
|
||||
existing.Description = ""
|
||||
existing.ProcessedAt = nil
|
||||
existing.EmbeddingModelID = kb.EmbeddingModelID
|
||||
// Reset the enrichment counter so a leftover value from a previous
|
||||
// attempt cannot block the new finalizing transition later.
|
||||
existing.PendingSubtasksCount = 0
|
||||
|
||||
if err := s.repo.UpdateKnowledge(ctx, existing); err != nil {
|
||||
logger.Errorf(ctx, "Failed to update knowledge status before reparse: %v", err)
|
||||
@@ -1785,6 +1840,117 @@ func (s *knowledgeService) ReparseKnowledge(ctx context.Context, knowledgeID str
|
||||
return existing, nil
|
||||
}
|
||||
|
||||
// CancelKnowledgeParse marks an in-progress parse as cancelled by the user.
|
||||
//
|
||||
// Semantics (kept aligned with the existing deleting path, but partial work
|
||||
// is preserved instead of cleaned up):
|
||||
// - parse_status is set to "cancelled"; partial chunks/index already written
|
||||
// to the database remain on disk. The user can re-trigger parsing via the
|
||||
// existing ReparseKnowledge API, which overwrites status back to pending.
|
||||
// - Any in-flight worker reads the new status at its next checkpoint and
|
||||
// bails (see processChunks / ProcessDocument / downstream handlers).
|
||||
// - The asynq inspector (if available) dequeues pending / scheduled / retry
|
||||
// tasks for this knowledge_id across the default / critical / low queues
|
||||
// and signals active workers to stop. Lite mode (no Redis) skips the
|
||||
// dequeue step — the checkpoint-based abort is the only stop signal there.
|
||||
// - Idempotent: re-calling on an already-cancelled row is a no-op.
|
||||
//
|
||||
// Errors:
|
||||
// - ParseStatusCompleted / ParseStatusFailed: the parse has already finished.
|
||||
// - ParseStatusDeleting: a delete is in progress; cancel cannot supersede it.
|
||||
func (s *knowledgeService) CancelKnowledgeParse(
|
||||
ctx context.Context, knowledgeID string,
|
||||
) (*types.Knowledge, error) {
|
||||
tenantID := ctx.Value(types.TenantIDContextKey).(uint64)
|
||||
existing, err := s.repo.GetKnowledgeByID(ctx, tenantID, knowledgeID)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "CancelKnowledgeParse: failed to load knowledge: %v", err)
|
||||
return nil, err
|
||||
}
|
||||
if existing == nil {
|
||||
return nil, werrors.NewNotFoundError("knowledge not found")
|
||||
}
|
||||
|
||||
switch existing.ParseStatus {
|
||||
case types.ParseStatusCancelled:
|
||||
// Idempotent — still attempt the dequeue in case earlier calls
|
||||
// raced an enqueue, but skip the row update / span close path.
|
||||
s.dequeueKnowledgeTasks(ctx, knowledgeID)
|
||||
return existing, nil
|
||||
case types.ParseStatusCompleted, types.ParseStatusFailed:
|
||||
return nil, werrors.NewBadRequestError("解析已结束，无法取消")
|
||||
case types.ParseStatusDeleting:
|
||||
return nil, werrors.NewBadRequestError("知识正在删除中，无法取消解析")
|
||||
case types.ParseStatusPending, types.ParseStatusProcessing, types.ParseStatusFinalizing:
|
||||
// Cancellable. `finalizing` is the post-process fan-out window
|
||||
// where graph-extract / summary / question subtasks are still
|
||||
// running; cancel here stops the LLM cost they would burn.
|
||||
default:
|
||||
// Unknown status — let it through but log. Should never happen
|
||||
// outside test fixtures or hand-edited rows.
|
||||
logger.Warnf(ctx, "CancelKnowledgeParse: unexpected status %q for %s, proceeding",
|
||||
existing.ParseStatus, knowledgeID)
|
||||
}
|
||||
|
||||
// Flip the row to cancelled and zero the enrichment counter in one
|
||||
// update so a late subtask FinalizeSubtask call can't race-promote
|
||||
// the row back to completed. Persisted partial data is left in
|
||||
// place — the user can reuse it on the next reparse attempt.
|
||||
now := time.Now()
|
||||
if err := s.repo.UpdateKnowledgeColumns(ctx, existing.ID, map[string]interface{}{
|
||||
"parse_status": types.ParseStatusCancelled,
|
||||
"error_message": "用户已取消解析",
|
||||
"pending_subtasks_count": 0,
|
||||
"updated_at": now,
|
||||
}); err != nil {
|
||||
logger.Errorf(ctx, "CancelKnowledgeParse: failed to mark knowledge cancelled: %v", err)
|
||||
return nil, err
|
||||
}
|
||||
existing.ParseStatus = types.ParseStatusCancelled
|
||||
existing.ErrorMessage = "用户已取消解析"
|
||||
existing.PendingSubtasksCount = 0
|
||||
existing.UpdatedAt = now
|
||||
logger.Infof(ctx, "Knowledge %s marked as cancelled by user", knowledgeID)
|
||||
|
||||
// Close the active attempt span tree so the UI stops showing "进行中"
|
||||
// for the cancelled run. AbortAttempt cascade-cancels every still-
|
||||
// running descendant (multimodal per-image, postprocess subtasks,
|
||||
// graph chunks) BEFORE closing the root, otherwise the trace
|
||||
// viewer would leave those striped/running bars hanging forever
|
||||
// because workers exit via their abort-guard without ever calling
|
||||
// FailSpan on their own subspan. Best-effort: nil tracker / missing
|
||||
// attempt no-ops.
|
||||
if attempt := s.tracker().LatestAttempt(ctx, knowledgeID); attempt > 0 {
|
||||
s.tracker().AbortAttempt(ctx, knowledgeID, attempt,
|
||||
"USER_CANCELLED", "用户已取消解析", "用户已取消解析")
|
||||
}
|
||||
|
||||
// Best-effort dequeue. Failures here don't block the cancel — the
|
||||
// downstream tasks will still self-abort at their entry guards.
|
||||
s.dequeueKnowledgeTasks(ctx, knowledgeID)
|
||||
// Wiki ingest lives in its own per-KB pending queue (task_pending_ops)
|
||||
// rather than asynq, so dequeueKnowledgeTasks above can't see it.
|
||||
// Mirror the deletion path's scrub so a cancelled knowledge doesn't
|
||||
// get picked up by the next 30s batch and burn a wiki LLM call on a
|
||||
// doc the user already abandoned. The in-flight worker would skip it
|
||||
// at isWikiKnowledgeAborted anyway, but scrubbing avoids waking the
|
||||
// batch in the first place.
|
||||
s.scrubWikiPendingIngest(ctx, existing.KnowledgeBaseID, knowledgeID, "cancel")
|
||||
return existing, nil
|
||||
}
|
||||
|
||||
// dequeueKnowledgeTasks asks the task inspector to remove any queued
|
||||
// tasks for this knowledge and signal active workers to stop. Safe to
|
||||
// call when the inspector is a no-op (Lite mode).
|
||||
func (s *knowledgeService) dequeueKnowledgeTasks(ctx context.Context, knowledgeID string) {
|
||||
if s.taskInspector == nil {
|
||||
return
|
||||
}
|
||||
if _, _, err := s.taskInspector.CancelTasksForKnowledge(ctx, knowledgeID); err != nil {
|
||||
logger.Warnf(ctx, "CancelKnowledgeParse: dequeue best-effort failed for %s: %v", knowledgeID, err)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *knowledgeService) updateChunkVector(ctx context.Context, kbID string, chunks []*types.Chunk) error {
|
||||
// Get embedding model from knowledge base
|
||||
sourceKB, err := s.kbService.GetKnowledgeBaseByID(ctx, kbID)
|
||||
@@ -2038,6 +2204,10 @@ func (s *knowledgeService) ProcessManualUpdate(ctx context.Context, t *asynq.Tas
|
||||
logger.Infof(ctx, "ProcessManualUpdate: being deleted, skipping: %s", payload.KnowledgeID)
|
||||
return nil
|
||||
}
|
||||
if knowledge.ParseStatus == types.ParseStatusCancelled {
|
||||
logger.Infof(ctx, "ProcessManualUpdate: cancelled by user, skipping: %s", payload.KnowledgeID)
|
||||
return nil
|
||||
}
|
||||
|
||||
kb, err := s.kbService.GetKnowledgeBaseByID(ctx, payload.KnowledgeBaseID)
|
||||
if err != nil {
|
||||
@@ -2049,6 +2219,12 @@ func (s *knowledgeService) ProcessManualUpdate(ctx context.Context, t *asynq.Tas
|
||||
return nil
|
||||
}
|
||||
|
||||
// Re-check abort status right before marking processing — see the same
|
||||
// note in ProcessDocument for the cancel race this guards.
|
||||
if aborted, status := s.isKnowledgeAborted(ctx, knowledge.TenantID, knowledge.ID); aborted {
|
||||
logger.Infof(ctx, "ProcessManualUpdate: knowledge aborted (%s), skipping: %s", status, knowledge.ID)
|
||||
return nil
|
||||
}
|
||||
// Update status to processing
|
||||
knowledge.ParseStatus = "processing"
|
||||
knowledge.UpdatedAt = time.Now()
|
||||
@@ -2117,11 +2293,15 @@ func (s *knowledgeService) ProcessDocument(ctx context.Context, t *asynq.Task) e
|
||||
return nil
|
||||
}
|
||||
|
||||
// 检查是否正在删除 - 如果是则直接退出，避免与删除操作冲突
|
||||
// 检查是否正在删除 / 已被用户取消 - 如果是则直接退出
|
||||
if knowledge.ParseStatus == types.ParseStatusDeleting {
|
||||
logger.Infof(ctx, "Knowledge is being deleted, aborting processing: %s", payload.KnowledgeID)
|
||||
return nil
|
||||
}
|
||||
if knowledge.ParseStatus == types.ParseStatusCancelled {
|
||||
logger.Infof(ctx, "Knowledge cancelled by user, aborting processing: %s", payload.KnowledgeID)
|
||||
return nil
|
||||
}
|
||||
|
||||
// 检查任务状态 - 幂等性处理
|
||||
if knowledge.ParseStatus == types.ParseStatusCompleted {
|
||||
@@ -2159,6 +2339,14 @@ func (s *knowledgeService) ProcessDocument(ctx context.Context, t *asynq.Task) e
|
||||
return nil
|
||||
}
|
||||
|
||||
// Re-check abort status right before flipping to "processing" — closes
|
||||
// the race where the user cancels between the entry guard above and
|
||||
// this write (otherwise the worker would overwrite cancelled→processing
|
||||
// and downstream checkpoints would treat the run as live).
|
||||
if aborted, status := s.isKnowledgeAborted(ctx, knowledge.TenantID, knowledge.ID); aborted {
|
||||
logger.Infof(ctx, "Knowledge aborted (%s) before marking processing: %s", status, knowledge.ID)
|
||||
return nil
|
||||
}
|
||||
knowledge.ParseStatus = "processing"
|
||||
knowledge.UpdatedAt = time.Now()
|
||||
if err := s.repo.UpdateKnowledge(ctx, knowledge); err != nil {
|
||||
|
||||
@@ -102,6 +102,14 @@ type SpanTracker interface {
|
||||
// done; output/error are written verbatim.
|
||||
FinalizeAttempt(ctx context.Context, knowledgeID string, attempt int, status string,
|
||||
output types.JSONMap, errorCode, errorMessage string)
|
||||
|
||||
// AbortAttempt cascade-cancels every still-running descendant of the
|
||||
// attempt's root span and then closes the root as cancelled. Used by
|
||||
// the user-initiated cancel path so the trace viewer doesn't leave
|
||||
// stranded subspans (multimodal images, postprocess subtasks)
|
||||
// looking like they're still in flight forever after the user
|
||||
// stopped the parse. Idempotent.
|
||||
AbortAttempt(ctx context.Context, knowledgeID string, attempt int, errorCode, errorMessage, reason string)
|
||||
}
|
||||
|
||||
type spanTracker struct {
|
||||
@@ -724,6 +732,43 @@ func (t *spanTracker) FinalizeAttempt(ctx context.Context, knowledgeID string, a
|
||||
t.touchKnowledgeHeartbeat(ctx, knowledgeID, types.SpanKindRoot)
|
||||
}
|
||||
|
||||
// AbortAttempt is the user-cancel counterpart to FinalizeAttempt. It
|
||||
// flips every still-running / still-pending span for this attempt to
|
||||
// cancelled — regardless of tree position — and then closes the root.
|
||||
//
|
||||
// Why a flat sweep instead of CancelDescendants' BFS: fan-out stages
|
||||
// (e.g. 多模态识别) call EndSpan on the stage as soon as they finish
|
||||
// DISPATCHING their async per-image work, so by the time the user
|
||||
// hits cancel the stage row is already status=done but its image[*]
|
||||
// children are still status=running. A BFS that stops at terminal
|
||||
// parents would orphan those leaves. The flat sweep doesn't care
|
||||
// about the tree shape — anything not-yet-terminal gets flipped.
|
||||
func (t *spanTracker) AbortAttempt(ctx context.Context, knowledgeID string, attempt int,
|
||||
errorCode, errorMessage, reason string,
|
||||
) {
|
||||
if knowledgeID == "" || attempt <= 0 {
|
||||
return
|
||||
}
|
||||
if reason == "" {
|
||||
reason = "user cancelled"
|
||||
}
|
||||
if errorCode == "" {
|
||||
errorCode = "USER_CANCELLED"
|
||||
}
|
||||
if n, err := t.repo.CancelAllOpenSpans(ctx, knowledgeID, attempt, errorCode, reason); err != nil {
|
||||
logger.Warnf(ctx, "[SpanTracker] AbortAttempt sweep failed kid=%s attempt=%d: %v",
|
||||
knowledgeID, attempt, err)
|
||||
// Fall through to FinalizeAttempt anyway — closing the root
|
||||
// is more important than perfectly closing every child.
|
||||
} else if n > 0 {
|
||||
logger.Infof(ctx,
|
||||
"[SpanTracker] AbortAttempt swept %d open span(s) for kid=%s attempt=%d",
|
||||
n, knowledgeID, attempt)
|
||||
}
|
||||
t.FinalizeAttempt(ctx, knowledgeID, attempt,
|
||||
types.SpanStatusCancelled, nil, errorCode, errorMessage)
|
||||
}
|
||||
|
||||
// noopSpanTracker collapses every method to a no-op for tests/lite.
|
||||
type noopSpanTracker struct{}
|
||||
|
||||
@@ -743,3 +788,4 @@ func (noopSpanTracker) SkipSpan(_ context.Context, _ *Span, _ string)
|
||||
func (noopSpanTracker) LookupStage(_ context.Context, _ string, _ int, _ string) *Span { return nil }
|
||||
func (noopSpanTracker) FinalizeAttempt(_ context.Context, _ string, _ int, _ string, _ types.JSONMap, _, _ string) {
|
||||
}
|
||||
func (noopSpanTracker) AbortAttempt(_ context.Context, _ string, _ int, _, _, _ string) {}
|
||||
|
||||
@@ -1786,7 +1786,11 @@ func (s *wikiIngestService) isKnowledgeGone(ctx context.Context, kbID, knowledge
|
||||
if err != nil || kn == nil {
|
||||
return true
|
||||
}
|
||||
return kn.ParseStatus == types.ParseStatusDeleting
|
||||
switch kn.ParseStatus {
|
||||
case types.ParseStatusDeleting, types.ParseStatusCancelled:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// filterLiveUpdates drops additions/summaries whose source knowledge has been
|
||||
|
||||
@@ -255,10 +255,18 @@ func BuildContainer(container *dig.Container) *dig.Container {
|
||||
if redisAvailable {
|
||||
must(container.Provide(router.NewAsyncqClient, dig.As(new(interfaces.TaskEnqueuer))))
|
||||
must(container.Provide(router.NewAsynqServer))
|
||||
// Asynq inspector for cancel-by-knowledge-id (best-effort
|
||||
// dequeue of pending/scheduled/retry tasks + active-task cancel).
|
||||
must(container.Provide(router.NewAsynqInspector))
|
||||
must(container.Provide(router.NewAsynqTaskInspector))
|
||||
} else {
|
||||
syncExec := router.NewSyncTaskExecutor()
|
||||
must(container.Provide(func() interfaces.TaskEnqueuer { return syncExec }))
|
||||
must(container.Provide(func() *router.SyncTaskExecutor { return syncExec }))
|
||||
// Lite mode: no Redis means no asynq inspector. SyncTaskExecutor
|
||||
// dispatches inline goroutines that the checkpoint-based abort
|
||||
// already handles.
|
||||
must(container.Provide(router.NewNoopTaskInspector))
|
||||
}
|
||||
|
||||
// Chat pipeline components for processing chat requests
|
||||
@@ -643,7 +651,12 @@ func resetPendingTasks(db *gorm.DB) {
|
||||
distributed := os.Getenv("REDIS_ADDR") != ""
|
||||
|
||||
knowledgeQuery := db.Model(&types.Knowledge{}).
|
||||
Where("parse_status IN ?", []string{types.ParseStatusPending, types.ParseStatusProcessing, types.ParseStatusDeleting})
|
||||
Where("parse_status IN ?", []string{
|
||||
types.ParseStatusPending,
|
||||
types.ParseStatusProcessing,
|
||||
types.ParseStatusFinalizing,
|
||||
types.ParseStatusDeleting,
|
||||
})
|
||||
summaryQuery := db.Model(&types.Knowledge{}).
|
||||
Where("summary_status IN ?", []string{types.SummaryStatusPending, types.SummaryStatusProcessing})
|
||||
syncQuery := db.Model(&types.SyncLog{}).
|
||||
@@ -659,10 +672,12 @@ func resetPendingTasks(db *gorm.DB) {
|
||||
syncQuery = syncQuery.Where("start_time < ?", staleCutoff)
|
||||
}
|
||||
|
||||
// 1. Reset knowledge parsing tasks
|
||||
// 1. Reset knowledge parsing tasks (including finalizing rows whose
|
||||
// enrichment subtasks were lost with the process).
|
||||
result := knowledgeQuery.Updates(map[string]interface{}{
|
||||
"parse_status": types.ParseStatusFailed,
|
||||
"error_message": "Task interrupted due to application restart",
|
||||
"parse_status": types.ParseStatusFailed,
|
||||
"error_message": "Task interrupted due to application restart",
|
||||
"pending_subtasks_count": 0,
|
||||
})
|
||||
if result.Error != nil {
|
||||
logger.Warnf(context.Background(), "Failed to reset pending knowledge tasks: %v", result.Error)
|
||||
|
||||
@@ -1582,6 +1582,59 @@ func (h *KnowledgeHandler) ReparseKnowledge(c *gin.Context) {
|
||||
})
|
||||
}
|
||||
|
||||
// CancelKnowledgeParse godoc
|
||||
// @Summary 取消知识解析
|
||||
// @Description 取消进行中的知识解析任务。当前已写入的 chunk / 索引保留，可通过 reparse 接口重新触发解析。已完成 / 已失败 / 删除中的知识不支持取消。
|
||||
// @Tags 知识管理
|
||||
// @Accept json
|
||||
// @Produce json
|
||||
// @Param id path string true "知识ID"
|
||||
// @Success 200 {object} map[string]interface{} "取消已提交"
|
||||
// @Failure 400 {object} errors.AppError "状态不支持取消"
|
||||
// @Failure 403 {object} errors.AppError "权限不足"
|
||||
// @Failure 404 {object} errors.AppError "知识不存在"
|
||||
// @Security Bearer
|
||||
// @Security ApiKeyAuth
|
||||
// @Router /knowledge/{id}/cancel-parse [post]
|
||||
func (h *KnowledgeHandler) CancelKnowledgeParse(c *gin.Context) {
|
||||
ctx := c.Request.Context()
|
||||
logger.Info(ctx, "Start cancelling knowledge parse")
|
||||
|
||||
id := secutils.SanitizeForLog(c.Param("id"))
|
||||
if id == "" {
|
||||
logger.Error(ctx, "Knowledge ID is empty")
|
||||
c.Error(errors.NewBadRequestError("Knowledge ID cannot be empty"))
|
||||
return
|
||||
}
|
||||
|
||||
// Editor permission — same gate as ReparseKnowledge / DeleteKnowledge.
|
||||
_, effCtx, err := h.resolveKnowledgeAndValidateKBAccess(c, id, types.OrgRoleEditor)
|
||||
if err != nil {
|
||||
c.Error(err)
|
||||
return
|
||||
}
|
||||
|
||||
knowledge, err := h.kgService.CancelKnowledgeParse(effCtx, id)
|
||||
if err != nil {
|
||||
if appErr, ok := errors.IsAppError(err); ok {
|
||||
c.Error(appErr)
|
||||
return
|
||||
}
|
||||
logger.ErrorWithFields(ctx, err, map[string]interface{}{
|
||||
"knowledge_id": id,
|
||||
})
|
||||
c.Error(errors.NewInternalServerError(err.Error()))
|
||||
return
|
||||
}
|
||||
|
||||
logger.Infof(ctx, "Knowledge parse cancelled successfully, knowledge ID: %s", id)
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"success": true,
|
||||
"message": "Knowledge parse cancelled",
|
||||
"data": knowledge,
|
||||
})
|
||||
}
|
||||
|
||||
type knowledgeTagBatchRequest struct {
|
||||
Updates map[string]*string `json:"updates" binding:"required,min=1"`
|
||||
KBID string `json:"kb_id"` // Optional: scope to this KB (validates editor access and uses effective tenant for shared KB)
|
||||
|
||||
@@ -289,6 +289,7 @@ func RegisterKnowledgeRoutes(r *gin.RouterGroup, handler *handler.KnowledgeHandl
|
||||
k.PUT("/:id", g.OwnedKnowledgeKBOrAdmin(), g.KBAccessWriteFromKnowledgeIDParam("id"), handler.UpdateKnowledge)
|
||||
k.PUT("/manual/:id", g.OwnedKnowledgeKBOrAdmin(), g.KBAccessWriteFromKnowledgeIDParam("id"), handler.UpdateManualKnowledge)
|
||||
k.POST("/:id/reparse", g.OwnedKnowledgeKBOrAdmin(), g.KBAccessWriteFromKnowledgeIDParam("id"), handler.ReparseKnowledge)
|
||||
k.POST("/:id/cancel-parse", g.OwnedKnowledgeKBOrAdmin(), g.KBAccessWriteFromKnowledgeIDParam("id"), handler.CancelKnowledgeParse)
|
||||
k.GET("/:id/download", g.Viewer(), g.KBAccessReadFromKnowledgeIDParam("id"), handler.DownloadKnowledgeFile)
|
||||
k.GET("/:id/preview", g.Viewer(), g.KBAccessReadFromKnowledgeIDParam("id"), handler.PreviewKnowledgeFile)
|
||||
k.PUT("/image/:id/:chunk_id", g.OwnedKnowledgeKBOrAdmin(), g.KBAccessWriteFromKnowledgeIDParam("id"), handler.UpdateImageInfo)
|
||||
|
||||
254
internal/router/task_inspector.go
Normal file
254
internal/router/task_inspector.go
Normal file
@@ -0,0 +1,254 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/hibiken/asynq"
|
||||
)
|
||||
|
||||
// NewAsynqInspector constructs an *asynq.Inspector pointed at the same
|
||||
// Redis used by the asynq client. Only registered in asynq mode.
|
||||
func NewAsynqInspector() *asynq.Inspector {
|
||||
return asynq.NewInspector(getAsynqRedisClientOpt())
|
||||
}
|
||||
|
||||
// asynqTaskInspector implements interfaces.TaskInspector backed by an
|
||||
// *asynq.Inspector. Scans the queues we actually use ("default",
|
||||
// "critical", "low") and matches tasks whose payload carries the given
|
||||
// knowledge_id. Best-effort: any scan/delete error is logged and
|
||||
// swallowed so the cancel API still returns success even when Redis is
|
||||
// flaky.
|
||||
type asynqTaskInspector struct {
|
||||
inspector *asynq.Inspector
|
||||
}
|
||||
|
||||
// NewAsynqTaskInspector returns a TaskInspector wrapping the given
|
||||
// *asynq.Inspector. nil-safe: a nil inspector degrades to a no-op so
|
||||
// the cancel path remains usable when the inspector failed to init.
|
||||
func NewAsynqTaskInspector(inspector *asynq.Inspector) interfaces.TaskInspector {
|
||||
if inspector == nil {
|
||||
return noopTaskInspector{}
|
||||
}
|
||||
return &asynqTaskInspector{inspector: inspector}
|
||||
}
|
||||
|
||||
// knowledgeIDProbe is the minimal payload shape we need to filter
|
||||
// tasks. All pipeline payload types embed a json:"knowledge_id" field,
|
||||
// so a single struct covers Document / ImageMultimodal / PostProcess /
|
||||
// Question / Summary / Extract / Manual.
|
||||
type knowledgeIDProbe struct {
|
||||
KnowledgeID string `json:"knowledge_id,omitempty"`
|
||||
}
|
||||
|
||||
// queuesScanned is the fixed set of queue names this codebase enqueues
|
||||
// into. Kept tight on purpose — we never scan user-defined queues.
|
||||
var queuesScanned = []string{"default", "critical", "low"}
|
||||
|
||||
// taskTypesForKnowledgeCancel lists every asynq task type that carries
|
||||
// a knowledge_id in its payload and should be cancelable. The set is
|
||||
// deliberately narrow: we don't touch FAQ import / KB-level tasks
|
||||
// because the cancel API is per-knowledge.
|
||||
var taskTypesForKnowledgeCancel = map[string]struct{}{
|
||||
types.TypeDocumentProcess: {},
|
||||
types.TypeManualProcess: {},
|
||||
types.TypeImageMultimodal: {},
|
||||
types.TypeKnowledgePostProcess: {},
|
||||
types.TypeQuestionGeneration: {},
|
||||
types.TypeSummaryGeneration: {},
|
||||
types.TypeChunkExtract: {},
|
||||
}
|
||||
|
||||
// listPageSize caps each Redis LIST call. Asynq pages tasks, so we
|
||||
// loop until a short page comes back. 100 matches asynq's default.
|
||||
const listPageSize = 100
|
||||
|
||||
// CancelTasksForKnowledge removes queued tasks whose payload references
|
||||
// the given knowledge_id and signals active workers running such tasks
|
||||
// to stop.
|
||||
func (a *asynqTaskInspector) CancelTasksForKnowledge(
|
||||
ctx context.Context, knowledgeID string,
|
||||
) (int, int, error) {
|
||||
if a == nil || a.inspector == nil || knowledgeID == "" {
|
||||
return 0, 0, nil
|
||||
}
|
||||
deleted := 0
|
||||
cancelled := 0
|
||||
|
||||
for _, queue := range queuesScanned {
|
||||
// Pending / Scheduled / Retry can all be deleted by task ID.
|
||||
// Archived tasks are NOT touched: dead-letter rows are
|
||||
// already final and should remain visible to operators.
|
||||
deleted += a.deletePendingMatches(ctx, queue, knowledgeID)
|
||||
deleted += a.deleteScheduledMatches(ctx, queue, knowledgeID)
|
||||
deleted += a.deleteRetryMatches(ctx, queue, knowledgeID)
|
||||
cancelled += a.cancelActiveMatches(ctx, queue, knowledgeID)
|
||||
}
|
||||
|
||||
logger.Infof(ctx,
|
||||
"[TaskInspector] knowledge=%s cancel summary: deleted_from_queue=%d active_cancel_signaled=%d",
|
||||
knowledgeID, deleted, cancelled,
|
||||
)
|
||||
return deleted, cancelled, nil
|
||||
}
|
||||
|
||||
// matchesKnowledge returns true when the task type is one we cancel
|
||||
// AND its payload references the target knowledge ID.
|
||||
func matchesKnowledge(taskType string, payload []byte, knowledgeID string) bool {
|
||||
if _, ok := taskTypesForKnowledgeCancel[taskType]; !ok {
|
||||
return false
|
||||
}
|
||||
var probe knowledgeIDProbe
|
||||
if err := json.Unmarshal(payload, &probe); err != nil {
|
||||
return false
|
||||
}
|
||||
return probe.KnowledgeID == knowledgeID
|
||||
}
|
||||
|
||||
func (a *asynqTaskInspector) deletePendingMatches(ctx context.Context, queue, knowledgeID string) int {
|
||||
deleted := 0
|
||||
page := 1
|
||||
for {
|
||||
tasks, err := a.inspector.ListPendingTasks(queue, asynq.PageSize(listPageSize), asynq.Page(page))
|
||||
if err != nil {
|
||||
if !errors.Is(err, asynq.ErrQueueNotFound) {
|
||||
logger.Warnf(ctx, "[TaskInspector] list pending queue=%s page=%d: %v", queue, page, err)
|
||||
}
|
||||
return deleted
|
||||
}
|
||||
if len(tasks) == 0 {
|
||||
return deleted
|
||||
}
|
||||
for _, t := range tasks {
|
||||
if !matchesKnowledge(t.Type, t.Payload, knowledgeID) {
|
||||
continue
|
||||
}
|
||||
if err := a.inspector.DeleteTask(queue, t.ID); err != nil {
|
||||
logger.Warnf(ctx, "[TaskInspector] delete pending type=%s id=%s: %v", t.Type, t.ID, err)
|
||||
continue
|
||||
}
|
||||
deleted++
|
||||
}
|
||||
if len(tasks) < listPageSize {
|
||||
return deleted
|
||||
}
|
||||
page++
|
||||
}
|
||||
}
|
||||
|
||||
func (a *asynqTaskInspector) deleteScheduledMatches(ctx context.Context, queue, knowledgeID string) int {
|
||||
deleted := 0
|
||||
page := 1
|
||||
for {
|
||||
tasks, err := a.inspector.ListScheduledTasks(queue, asynq.PageSize(listPageSize), asynq.Page(page))
|
||||
if err != nil {
|
||||
if !errors.Is(err, asynq.ErrQueueNotFound) {
|
||||
logger.Warnf(ctx, "[TaskInspector] list scheduled queue=%s page=%d: %v", queue, page, err)
|
||||
}
|
||||
return deleted
|
||||
}
|
||||
if len(tasks) == 0 {
|
||||
return deleted
|
||||
}
|
||||
for _, t := range tasks {
|
||||
if !matchesKnowledge(t.Type, t.Payload, knowledgeID) {
|
||||
continue
|
||||
}
|
||||
if err := a.inspector.DeleteTask(queue, t.ID); err != nil {
|
||||
logger.Warnf(ctx, "[TaskInspector] delete scheduled type=%s id=%s: %v", t.Type, t.ID, err)
|
||||
continue
|
||||
}
|
||||
deleted++
|
||||
}
|
||||
if len(tasks) < listPageSize {
|
||||
return deleted
|
||||
}
|
||||
page++
|
||||
}
|
||||
}
|
||||
|
||||
func (a *asynqTaskInspector) deleteRetryMatches(ctx context.Context, queue, knowledgeID string) int {
|
||||
deleted := 0
|
||||
page := 1
|
||||
for {
|
||||
tasks, err := a.inspector.ListRetryTasks(queue, asynq.PageSize(listPageSize), asynq.Page(page))
|
||||
if err != nil {
|
||||
if !errors.Is(err, asynq.ErrQueueNotFound) {
|
||||
logger.Warnf(ctx, "[TaskInspector] list retry queue=%s page=%d: %v", queue, page, err)
|
||||
}
|
||||
return deleted
|
||||
}
|
||||
if len(tasks) == 0 {
|
||||
return deleted
|
||||
}
|
||||
for _, t := range tasks {
|
||||
if !matchesKnowledge(t.Type, t.Payload, knowledgeID) {
|
||||
continue
|
||||
}
|
||||
if err := a.inspector.DeleteTask(queue, t.ID); err != nil {
|
||||
logger.Warnf(ctx, "[TaskInspector] delete retry type=%s id=%s: %v", t.Type, t.ID, err)
|
||||
continue
|
||||
}
|
||||
deleted++
|
||||
}
|
||||
if len(tasks) < listPageSize {
|
||||
return deleted
|
||||
}
|
||||
page++
|
||||
}
|
||||
}
|
||||
|
||||
// cancelActiveMatches signals active workers to abort via
|
||||
// Inspector.CancelProcessing. The worker's ctx becomes Done() so the
|
||||
// next blocking call (or our checkpoint reads) bails. The DB-level
|
||||
// abort flag (parse_status=cancelled) remains the durable signal —
|
||||
// this is a latency optimization, not the correctness mechanism.
|
||||
func (a *asynqTaskInspector) cancelActiveMatches(ctx context.Context, queue, knowledgeID string) int {
|
||||
cancelled := 0
|
||||
page := 1
|
||||
for {
|
||||
tasks, err := a.inspector.ListActiveTasks(queue, asynq.PageSize(listPageSize), asynq.Page(page))
|
||||
if err != nil {
|
||||
if !errors.Is(err, asynq.ErrQueueNotFound) {
|
||||
logger.Warnf(ctx, "[TaskInspector] list active queue=%s page=%d: %v", queue, page, err)
|
||||
}
|
||||
return cancelled
|
||||
}
|
||||
if len(tasks) == 0 {
|
||||
return cancelled
|
||||
}
|
||||
for _, t := range tasks {
|
||||
if !matchesKnowledge(t.Type, t.Payload, knowledgeID) {
|
||||
continue
|
||||
}
|
||||
if err := a.inspector.CancelProcessing(t.ID); err != nil {
|
||||
logger.Warnf(ctx, "[TaskInspector] cancel active type=%s id=%s: %v", t.Type, t.ID, err)
|
||||
continue
|
||||
}
|
||||
cancelled++
|
||||
}
|
||||
if len(tasks) < listPageSize {
|
||||
return cancelled
|
||||
}
|
||||
page++
|
||||
}
|
||||
}
|
||||
|
||||
// noopTaskInspector is the Lite-mode (no Redis) inspector. Inline
|
||||
// goroutines spawned by SyncTaskExecutor cannot be dequeued before
|
||||
// they start; the checkpoint-based abort in worker code is the only
|
||||
// stop signal in that mode.
|
||||
type noopTaskInspector struct{}
|
||||
|
||||
// NewNoopTaskInspector returns a no-op TaskInspector for Lite mode.
|
||||
func NewNoopTaskInspector() interfaces.TaskInspector { return noopTaskInspector{} }
|
||||
|
||||
func (noopTaskInspector) CancelTasksForKnowledge(
|
||||
ctx context.Context, knowledgeID string,
|
||||
) (int, int, error) {
|
||||
return 0, 0, nil
|
||||
}
|
||||
@@ -95,6 +95,14 @@ type KnowledgeService interface {
|
||||
) (*types.Knowledge, error)
|
||||
// ReparseKnowledge deletes existing document content and re-parses the knowledge asynchronously.
|
||||
ReparseKnowledge(ctx context.Context, knowledgeID string) (*types.Knowledge, error)
|
||||
// CancelKnowledgeParse marks an in-progress parse as cancelled by the
|
||||
// user. The knowledge row and any partially written chunks/index are
|
||||
// kept; downstream queued tasks for the same knowledge are best-effort
|
||||
// dequeued and active workers are signalled to stop at their next
|
||||
// checkpoint. Idempotent — returns the existing row when the knowledge
|
||||
// is already cancelled. Returns an error when the knowledge is in a
|
||||
// terminal state (completed / failed) or being deleted.
|
||||
CancelKnowledgeParse(ctx context.Context, knowledgeID string) (*types.Knowledge, error)
|
||||
// CloneKnowledgeBase clones knowledge to another knowledge base.
|
||||
CloneKnowledgeBase(ctx context.Context, srcID, dstID string) error
|
||||
// UpdateImageInfo updates image information for a knowledge chunk.
|
||||
@@ -214,6 +222,17 @@ type KnowledgeRepository interface {
|
||||
// statement so callers that flip several related fields (e.g. parse_status +
|
||||
// error_message) cannot leave the row in a half-updated state.
|
||||
UpdateKnowledgeColumns(ctx context.Context, id string, values map[string]interface{}) error
|
||||
// FinalizeSubtask atomically decrements pending_subtasks_count for the
|
||||
// given knowledge and promotes parse_status from "finalizing" to
|
||||
// "completed" when the count reaches zero. Returns the post-decrement
|
||||
// count, whether this caller's UPDATE was the one that promoted the
|
||||
// row, and any error.
|
||||
FinalizeSubtask(ctx context.Context, id string) (int, bool, error)
|
||||
// SetFinalizing atomically transitions a row from "processing" to
|
||||
// "finalizing" and writes the initial pending_subtasks_count. Returns
|
||||
// whether the transition took place (false when the row's parse_status
|
||||
// was no longer "processing", e.g. user cancelled / deleted in flight).
|
||||
SetFinalizing(ctx context.Context, id string, expectedSubtasks int) (bool, error)
|
||||
// CountKnowledgeByKnowledgeBaseID counts the number of knowledge items in a knowledge base.
|
||||
CountKnowledgeByKnowledgeBaseID(ctx context.Context, tenantID uint64, kbID string) (int64, error)
|
||||
// CountKnowledgeByStatus counts the number of knowledge items with the specified parse status.
|
||||
|
||||
26
internal/types/interfaces/task_inspector.go
Normal file
26
internal/types/interfaces/task_inspector.go
Normal file
@@ -0,0 +1,26 @@
|
||||
package interfaces
|
||||
|
||||
import "context"
|
||||
|
||||
// TaskInspector abstracts queue inspection / cancellation against the
|
||||
// task backend. It is best-effort: implementations may scan a finite
|
||||
// number of tasks per call and return whatever count they could
|
||||
// affect. Lite mode (no Redis) ships a no-op implementation because
|
||||
// SyncTaskExecutor dispatches inline goroutines that cannot be
|
||||
// dequeued before they start.
|
||||
//
|
||||
// Use cases today: user-initiated cancel of an in-progress knowledge
|
||||
// parse, which must remove downstream multimodal / post-process /
|
||||
// question / summary tasks already enqueued against the same
|
||||
// knowledge_id, plus signal active workers to stop at their next
|
||||
// checkpoint.
|
||||
type TaskInspector interface {
|
||||
// CancelTasksForKnowledge removes pending/scheduled/retry tasks
|
||||
// whose payload references the given knowledge ID, and signals
|
||||
// active workers running such tasks to stop. Returns rough
|
||||
// counts of (deletedFromQueue, activeCancelled) for observability.
|
||||
// Errors are returned but callers should treat the operation as
|
||||
// best-effort: the row-level abort flag remains the source of
|
||||
// truth, this just prevents wasted work.
|
||||
CancelTasksForKnowledge(ctx context.Context, knowledgeID string) (deleted int, cancelled int, err error)
|
||||
}
|
||||
@@ -37,13 +37,30 @@ const (
|
||||
// ParseStatusPending indicates the knowledge is waiting to be processed
|
||||
ParseStatusPending = "pending"
|
||||
// ParseStatusProcessing indicates the knowledge is being processed
|
||||
// (DocReader / chunking / embedding stage).
|
||||
ParseStatusProcessing = "processing"
|
||||
// ParseStatusCompleted indicates the knowledge has been processed successfully
|
||||
// ParseStatusFinalizing indicates the primary parse has finished but
|
||||
// enrichment subtasks (summary, question generation, graph extract)
|
||||
// are still in flight. The user-facing intuition behind this state is
|
||||
// "the document is queryable for vector search but is still spending
|
||||
// resources" — cancel-parse can interrupt enrichment from here.
|
||||
// pending_subtasks_count holds the outstanding subtask count; the
|
||||
// last subtask to finish atomically promotes the row to completed.
|
||||
ParseStatusFinalizing = "finalizing"
|
||||
// ParseStatusCompleted indicates the knowledge has been processed
|
||||
// successfully AND every enrichment subtask has reached a terminal
|
||||
// state. No further resources will be spent on the document until
|
||||
// the user explicitly re-parses it.
|
||||
ParseStatusCompleted = "completed"
|
||||
// ParseStatusFailed indicates the knowledge processing failed
|
||||
ParseStatusFailed = "failed"
|
||||
// ParseStatusDeleting indicates the knowledge is being deleted (used to prevent async task conflicts)
|
||||
ParseStatusDeleting = "deleting"
|
||||
// ParseStatusCancelled indicates parsing was cancelled by the user.
|
||||
// Same short-circuit semantics as ParseStatusDeleting for in-flight and
|
||||
// queued downstream tasks, but the knowledge row and any already-written
|
||||
// chunks/index are kept so the user can re-trigger parsing via reparse.
|
||||
ParseStatusCancelled = "cancelled"
|
||||
)
|
||||
|
||||
// Summary status constants for async summary generation
|
||||
@@ -112,6 +129,10 @@ type Knowledge struct {
|
||||
Channel string `json:"channel" gorm:"type:varchar(50);default:'web'"`
|
||||
// Parse status of the knowledge
|
||||
ParseStatus string `json:"parse_status"`
|
||||
// PendingSubtasksCount is the outstanding enrichment subtask count
|
||||
// (summary + question + graph chunks). Only meaningful while
|
||||
// ParseStatus == "finalizing"; defaults to 0 in any terminal state.
|
||||
PendingSubtasksCount int `json:"pending_subtasks_count" gorm:"type:int;not null;default:0"`
|
||||
// Summary status for async summary generation
|
||||
SummaryStatus string `json:"summary_status" gorm:"type:varchar(32);default:none"`
|
||||
// Enable status of the knowledge
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
ALTER TABLE knowledges DROP COLUMN IF EXISTS pending_subtasks_count;
|
||||
@@ -0,0 +1,31 @@
|
||||
-- Migration: 000056_knowledge_pending_subtasks
|
||||
--
|
||||
-- Add pending_subtasks_count to support the "finalizing" parse status,
|
||||
-- which gates parse_status='completed' until enrichment subtasks
|
||||
-- (summary, question generation, graph extract) finish.
|
||||
--
|
||||
-- Previously, parse_status flipped to 'completed' as soon as primary
|
||||
-- chunks + embeddings were written, even though the user-cancellable
|
||||
-- "expensive" tasks (graph extract = N LLM calls per chunk, question
|
||||
-- gen, summary) were still in flight. That broke the user's intuition
|
||||
-- that 'completed' means "no more resources will be spent on this".
|
||||
--
|
||||
-- New lifecycle:
|
||||
-- pending -> processing -> finalizing -> completed
|
||||
-- ^
|
||||
-- | parse_status='finalizing' AND
|
||||
-- | pending_subtasks_count > 0 mean
|
||||
-- | enrichment is still running and
|
||||
-- | CancelKnowledgeParse can interrupt it.
|
||||
--
|
||||
-- Existing rows: column defaults to 0, so all historical 'completed'
|
||||
-- rows look like "no pending subtasks" — which is correct (they're
|
||||
-- past the enrichment phase by definition).
|
||||
--
|
||||
-- Wiki ingest is NOT counted here: it is debounced and KB-scoped, with
|
||||
-- its own dedup queue; cancelling a single knowledge cannot meaningfully
|
||||
-- shorten an in-flight wiki batch and the wiki worker already short-
|
||||
-- circuits per-knowledge once parse_status is aborted.
|
||||
|
||||
ALTER TABLE knowledges
|
||||
ADD COLUMN IF NOT EXISTS pending_subtasks_count INT NOT NULL DEFAULT 0;
|
||||
Reference in New Issue
Block a user