1970 Commits

Author SHA1 Message Date
ochan.kwon
e9980c6011 fix: deep-copy stored files and images when cloning a knowledge base
Cloning a knowledge base previously copied only the storage path strings
(knowledge.FilePath and chunk.ImageInfo.URL), so the source and the clone
shared the same physical objects in the storage backend. Once the original
file and extracted images are deleted on source removal, the clone is left
with dangling references and its document and images become unreadable —
data loss that occurs even for same-store clones.

Add a CopyFile primitive to the FileService interface and implement it in
every backend: server-side CopyObject on the object stores
(s3/obs/cos/oss/tos/ks3/minio), io.Copy on local, and a no-op on dummy.
Destinations use the knowledge-owned layout and reuse the existing
path/object-key guards; a sentinel ErrCrossBackendCopy is returned when the
source scheme does not match the backend.

Use CopyFile to deep-copy the document file in cloneKnowledge and the
extracted images in CloneChunk and cloneFAQKnowledgeBase via a shared
cloneChunkImageInfo helper that deduplicates identical image URLs per clone
and rewrites them to the new objects. Copied objects are cleaned up
best-effort if a clone fails partway through. A clone-time preflight rejects
cloning into a target bound to a different storage backend when the tenant
pins providers via StorageEngineConfig.

Adds unit tests for local CopyFile (independent copy survives source
deletion, traversal rejection, cross-backend rejection), cloneChunkImageInfo
(empty/multi/dedup/parse-failure/OriginalURL handling), and the storage
provider preflight.
2026-06-03 14:45:59 +08:00
wizardchen
f4af9cca97 feat(contextualGuides): add multiple contextual guide components for agent and knowledge base creation
- Introduced new components: AgentCreateContextualGuide, KbCreateContextualGuide, TenantModelsGuide, and SpotlightGuide to enhance user onboarding.
- Implemented dynamic step configurations for each guide, allowing tailored user experiences based on context.
- Enhanced the existing NewUserGuide component to utilize the new SpotlightGuide for improved visual guidance.
- Updated localization files to include new strings for contextual guides, ensuring a comprehensive user experience across languages.
- Refactored existing components to integrate with the new guide system, improving maintainability and user interaction.
2026-06-03 14:42:46 +08:00
wizardchen
291d72565f feat(NewUserGuide): enhance guide component with improved backdrop and hole calculation
- Added a new visual element for the guide's highlight area, improving user interaction.
- Refactored hole calculation logic to ensure consistent spacing around highlighted elements.
- Introduced a new computed property for dynamic styling of the highlight area.
- Updated styles for backdrop elements to enhance visual clarity and interaction.
- Improved accessibility by ensuring the guide's interactive elements are properly defined.
2026-06-03 14:42:46 +08:00
wizardchen
bbd3f6324a refactor(parser): reorganize Markdown parser and enhance gRPC document reading
- Moved the _SEPARATOR_CELL regex definition to a more appropriate location in the Markdown parser.
- Implemented a fallback mechanism in the gRPC document reader to handle cases where the ReadStream RPC is unimplemented, ensuring compatibility with older versions.
- Added a readUnary method to maintain backward compatibility with the legacy unary Read RPC.
- Improved cancellation handling in the MinerUCloud and PaddleOCR-VL readers to prevent excessive API calls during context cancellation.
2026-06-03 12:29:13 +08:00
wizardchen
ef1047bf67 feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements
Introduce opendataloader and PaddleOCR-VL parser engines with tenant-level
settings UI, replace liteparse, and harden Excel/PPT/Markdown parsing.
Optional odl-hybrid sidecar stays local-build only and is excluded from
default dev-start and full profiles.
2026-06-03 12:29:13 +08:00
wizardchen
7b1bb1054f feat(docreader): speed up scanned-PDF parsing, stream image results, isolate heavy async queues
Large scanned PDFs (hundreds of pages) were slow and fragile end-to-end.
This change addresses the parse, transport, and task-scheduling layers:

docreader (parse + transport):
- Parallelize per-page scanned rendering across processes (forkserver/fork),
  with serial fallback. ~4-7x faster on large scanned PDFs; pdfium is not
  thread-safe so we fan out across processes. Configurable via
  DOCREADER_PDF_RENDER_PARALLELISM.
- Add server-streaming ReadStream RPC: emit one meta frame then one frame per
  image, so documents with many page images are no longer capped by the unary
  gRPC message-size limit (a 874-page PDF produced ~193MiB of images, far over
  the 50MB cap) and memory is bounded on both ends. Unary Read is kept for
  backward compatibility; the Go production reader switches to ReadStream.

VLM:
- Make the VLM HTTP timeout configurable (VLM_HTTP_TIMEOUT_SECONDS) and raise
  the default 90s -> 180s so dense scanned-page OCR does not time out with
  "context deadline exceeded".

Async task queues:
- Isolate high-volume, model-heavy fan-out tasks into dedicated asynq queues so
  a single large document cannot saturate the shared worker pool and block
  user-facing document parsing:
    image:multimodal  -> "multimodal"
    chunk:extract     -> "graph"
    question:generation -> "question"
- Register the new queues in the server weight map and the cancel inspector's
  scanned-queue set (so cancelling a knowledge still purges its pending tasks).
2026-06-03 12:29:13 +08:00
wizardchen
2ba1237598 feat(i18n): add login hints for new users in multiple languages
- Introduced new localization strings for login hints and first-time user prompts in English, Korean, Russian, and Chinese.
- Updated the Login.vue component to enhance user guidance during the login process.

This change aims to improve the onboarding experience for new users by providing clear instructions on how to sign in or create an account.
2026-06-03 08:19:06 +08:00
wizardchen
baef12e682 feat(tour): implement new user guide component and integrate into user menu
- Added a NewUserGuide component to provide a guided tour for new users.
- Integrated the guide into the UserMenu for easy access, allowing users to reopen the guide at any time.
- Updated localization files to include new strings for the user guide steps and actions.
- Enhanced the menu item for the user button to include a data attribute for better accessibility.

This feature aims to improve user onboarding and enhance the overall user experience by providing contextual help throughout the application.
2026-06-03 08:19:06 +08:00
dependabot[bot]
09a8607cd4 chore(deps): bump github.com/mattn/go-runewidth
Bumps the cli-deps group in /cli with 1 update: [github.com/mattn/go-runewidth](https://github.com/mattn/go-runewidth).


Updates `github.com/mattn/go-runewidth` from 0.0.23 to 0.0.24
- [Commits](https://github.com/mattn/go-runewidth/compare/v0.0.23...v0.0.24)

---
updated-dependencies:
- dependency-name: github.com/mattn/go-runewidth
  dependency-version: 0.0.24
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cli-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-02 21:39:10 +08:00
octo-patch
f22995f835 feat: add MiniMax-M3 to provider model list
- Add MiniMax-M3 to the MiniMax provider description across i18n locales
- Remove deprecated MiniMax-M2.5 reference
- Retain MiniMax-M2.7 and MiniMax-M2.7-highspeed as supported models

MiniMax-M3 is the latest flagship model from MiniMax with extended context
and stronger reasoning capabilities. Users can now see it in the provider
selection UI when configuring a new Minimax model.
2026-06-02 21:38:30 +08:00
Yeongpil Yoon
8f8a276120 fix(embedding): support native Gemini embeddings 2026-06-02 21:37:58 +08:00
wizardchen
482686d17e feat(chat): implement local image resolver for multimodal chat
- Added a LocalImageResolver function to resolve local storage URLs to their byte data using the tenant's storage configuration.
- Integrated the resolver into the container setup to support reading local images in multimodal chat scenarios.
- Enhanced the handling of local storage paths to correctly parse tenant IDs and retrieve files based on the configured storage settings.
2026-06-01 20:54:37 +08:00
wizardchen
959eba2136 fix(doc_parser): enhance DOC to DOCX conversion reliability
- Implemented a retry mechanism for DOC to DOCX conversion to handle concurrent `soffice` invocations, ensuring each attempt uses a dedicated user profile directory.
- Added logging for each conversion attempt, including success and failure messages, to improve visibility into the conversion process.
- Adjusted the handling of temporary directories for both conversion output and user profiles, enhancing robustness against conversion failures.
2026-06-01 20:50:02 +08:00
wizardchen
65f9018f73 refactor(settings): enhance card interactions and accessibility across components
Updated the McpSettings, ModelSettings, VectorStoreSettings, and WebSearchSettings components to improve user interactions with service, model, store, and provider cards. Implemented click and keyboard event handling for better accessibility, allowing cards to be clickable based on user roles. Enhanced UI elements with appropriate roles and tabindex attributes, ensuring a consistent and user-friendly experience. Adjusted styles to reflect clickable states and improved focus visibility for better usability.
2026-06-01 14:30:17 +08:00
wizardchen
a4557c3a80 refactor(knowledge): streamline event handling for tag and search inputs
Updated event handling in KnowledgeBase and FAQEntryManager components to utilize the `@enter` directive for form submissions, enhancing clarity and consistency. Implemented custom keydown handling for the Escape key to improve user experience when editing tags and searching documents. This change simplifies the input event management across various components.
2026-05-31 15:43:30 +08:00
wizardchen
d56370b30f refactor(settings): update event handling and UI interactions in various components
Modified event handling for form submissions in CreateTenantDialog and CredentialResource components to use the `@enter` directive for improved clarity. Removed console log from i18n initialization and updated localization files to include new section labels for the redesigned drawer across multiple languages. Enhanced the UI in Login and McpSettings views by adjusting input types and integrating test connection functionality into the drawer components. Streamlined the layout and interactions in ModelSettings, ParserEngineSettings, StorageEngineSettings, and VectorStoreSettings to ensure consistency and better user experience.
2026-05-31 15:43:30 +08:00
wizardchen
9d1f3001e8 feat(model-editor): enhance ModelEditorDialog with connection test and improved layout
Added a connection test button in the ModelEditorDialog for remote sources, allowing users to verify API connectivity before saving. Improved the layout by restructuring the source selection section and updating the styling for better usability. Enhanced the drawer component with a customizable header and resizable width, improving the overall user experience. Updated localization files to reflect new UI elements and labels.
2026-05-31 15:43:30 +08:00
wizardchen
d91bda0fb0 refactor(doc-content): streamline file and manual type handling in document viewer
Removed redundant sections for file and manual types in the document content component. Consolidated download actions into a unified header actions area, enhancing the user interface for file downloads and timeline access. Updated styles for improved layout and responsiveness, ensuring a cleaner and more efficient document viewer experience.
2026-05-31 15:43:30 +08:00
wizardchen
e35272161d feat(doc-content): implement adjustable main drawer width with local storage persistence
Added functionality to allow users to resize the main drawer in the document content view. The new width is adjustable via a drag handle, and the selected width is saved in local storage for persistence across sessions. Updated styles to support the new resize handle and ensure smooth user experience during resizing. This enhancement improves the usability of the document viewer by providing a customizable layout.
2026-05-30 17:53:47 +08:00
wizardchen
f92302cc64 fix(knowledge): stitch chunks by text overlap, not by position
Reconstructing full document content from chunks relied on position math
(offset = content length - (EndAt - lastEndAt)) that assumes
len([]rune(Content)) == EndAt - StartAt. Two upstream behaviors break that
invariant and caused the merged "全文" view to drop or duplicate text:

- The parent-child chunker prepends a synthetic table header to split
  tables; that header is zero-width in position space (start == end), so
  Content is longer than EndAt - StartAt.
- Chunk Content can retain HTML entities (&#34;, &gt; ...), inflating the
  rune count relative to the source span.

Replace position-based trimming with text-overlap matching: find where the
accumulated text's suffix reappears at the head of the next chunk and join
there. Positions are only used to size the search window. This naturally
skips the prepended header and is immune to entity length drift.

Extract the logic into searchutil.AppendWithOverlap / MergeTextChunks and
reuse it across the three backend stitching sites (wiki_ingest
reconstructContent, graph mergeChunkContents, chat_pipeline
mergeOverlappingChunks). Mirror the same algorithm in the frontend
doc-content merge. Also fix the markdown hr/heading styles (thin solid
divider, primary heading color).
2026-05-30 17:53:47 +08:00
jackson.jia
c8645843c3 fix(knowledge): wrap locale code with LanguageLocaleName in wiki ingest prompts
The wiki ingest pipeline was passing raw locale codes (e.g. "zh-CN",
"en-US") directly into LLM prompts where a human-readable language name
is expected. The model would see "zh-CN" instead of "Chinese (Simplified)"
in its instructions, which could degrade output quality.

Two call sites were affected:

- SlugUpdate.Language (ProcessWikiIngest, retract path): consumed by the
  reduce phase to tell the page editor model what language to write in.
- lang variable (mapOneDocument): consumed by entity/concept extraction,
  chunk citation classification, and index page rebuild — all LLM prompts.

Wrap both with types.LanguageLocaleName() which maps locale codes to
descriptive names (e.g. "zh-CN" → "Chinese (Simplified)"). Unknown
locales fall through unchanged so no information is lost.
2026-05-30 16:56:32 +08:00
wizardchen
cea6ef0ce3 test(knowledge): add comprehensive tests for finalizing subtask behavior
Introduced a new test file for the knowledge repository, implementing various scenarios to ensure the reliability of the finalizing subtask counter. Key tests include:

- Concurrent promotion of subtasks to verify atomic behavior.
- Handling of partial decrements to ensure the row remains in "finalizing" state.
- Safety checks to prevent underflow of the pending subtask count.
- Regression tests to confirm that updates do not overwrite the pending counter.

These tests address previous issues with the finalizing process and enhance overall stability.
2026-05-30 01:00:15 +08:00
wizardchen
84bd95275d fix(knowledge): make finalizing subtask counter reliable + async question fan-out
The finalizing subtask counter (pending_subtasks_count) could get stuck at a
non-zero value, leaving knowledge permanently in "finalizing" and never
promoted to "completed". Root cause and related hardening:

- UpdateKnowledge does a full-row Save; pending_subtasks_count was NOT in the
  omit list, so any concurrent enrichment subtask that loaded the row, did slow
  work (LLM call), then saved an unrelated field wrote back the STALE counter,
  clobbering decrements other subtasks performed in between. Add
  PendingSubtasksCount to omitFieldsOnUpdate so only the atomic helpers
  (SetFinalizing / FinalizeSubtask / explicit column writes) ever touch it.
- FinalizeSubtask no longer gates the finalizing->completed promote on a
  separate re-read of the counter. Every caller unconditionally runs the
  guarded promote UPDATE (WHERE pending_subtasks_count = 0), the single
  authoritative atomic check, so a racy/stale read can't strand the row.
- Decrements run on a context detached from the caller's cancellation
  (context.WithoutCancel + timeout) so graceful shutdown / preemption can't
  silently skip a decrement.
- Reconcile seeded vs actually-enqueued subtask slots (summary/question/graph)
  and release any shortfall so un-enqueued planned slots (e.g. graph with
  NEO4J off) don't strand the row.
- Reparse paths reset pending_subtasks_count via an explicit column write since
  full-row Save now omits it.

Question generation is also moved off the synchronous single-task path onto a
batched async fan-out (windows of text chunks), each batch independently
queued / retried / traced, grouped under a postprocess.question span.
2026-05-30 01:00:15 +08:00
wizardchen
9ffb886e40 fix(chat): keep in-progress message reactive so continue-stream renders
When reopening a session whose last assistant message is still being
generated, handleMsgList marked every loaded message's agentEventStream /
_eventMap / _pendingToolCalls with markRaw for history-rendering
performance. But the last, not-yet-completed message is then resumed via
the continue-stream SSE endpoint, and handleAgentChunk keeps mutating
those exact structures. markRaw detaches them from Vue reactivity, so the
backend was streaming events but the UI never re-rendered — users only saw
the pre-refresh snapshot, and the full content appeared only after the run
finished and a later refresh rebuilt it from persisted agent_steps.

Skip markRaw for the in-progress (is_completed === false) message and keep
it reactive; completed history messages still use markRaw. Regression from
8f462615 ("improve history rendering stability").
2026-05-30 00:37:31 +08:00
wizardchen
4a0a151d41 fix(knowledge): drain finalizing counter on all terminal subtask exits
The finalizing subtask counter (introduced when wiki ingest was counted)
could leak slots, leaving a fully-parsed doc stuck in "finalizing" until
the housekeeping sweep wrongly marked it "failed".

- wiki ingest: a doc skipped in map (knowledge deleted / no chunks /
  insufficient text) produced no docResult and was not a failedOp, so
  neither the success nor the dead-letter drain fired. Drain the slot on
  that terminal skip path.
- summary & question: the drain was keyed on the span-error variable,
  which assumes "err != nil => asynq will retry". Several branches set
  that variable yet `return nil` (insufficient text content, KB/knowledge
  fetch failures) - terminal, no retry - so the drain was skipped. Key
  the drain on the value actually returned to asynq (named retErr)
  instead, so terminal nil-returns drain and only retried errors wait
  for the final attempt.

Also fix the trace panel header flashing "已完成" mid-wiki: the latest
attempt's root span closes while async post-pipeline subspans keep
running, so trace.status read terminal while the row was still
"finalizing". Prefer parse_status on the latest attempt while it is
non-terminal so the panel header, LIVE badge and doc card agree, and add
the "finalizing" status label to all locales.
2026-05-29 19:32:13 +08:00
wizardchen
72272b2d62 feat(knowledge): count wiki ingest in the finalizing subtask counter
Wiki ingest runs asynchronously after the parse pipeline and was not
counted in pending_subtasks_count, so a document flipped to "completed"
while wiki generation was still minutes away (30s debounce + batch +
retries). That hid the in-progress state and dropped the stop-parse
affordance before wiki actually finished.

Count wiki as a single enrichment subtask (when WikiEnabled and the doc
has text chunks, matching the enqueue condition) so the row stays in
"finalizing" until wiki is done. The batch worker drains the slot at the
op's terminal state: once on successful map, and once when an op is
dead-lettered after exhausting in-batch retries. Retract ops (deleted
knowledge) are skipped. FinalizeSubtask guards both the decrement and the
promote, so a wiki op enqueued before this accounting shipped is a
harmless no-op on an already-completed row. A wiki op that never drains is
bounded by the existing housekeeping finalizing sweep.
2026-05-29 19:32:13 +08:00
wizardchen
95f7cd7d61 feat(knowledge): add stop-parse button to the trace timeline panel
The trace drawer had no way to cancel an in-flight parse — users had to
go back to the card/list menu. Add a stop-parse control in the timeline
header, shown while parse_status is pending / processing / finalizing
(mirroring the backend CancelKnowledgeParse gate and the card/list menus).

It is a quiet icon-only button matching the other header controls
(refresh / close), revealing its destructive error tint only on hover,
and opens an in-app TDesign confirm dialog (not the native window.confirm)
before calling the existing cancel-parse API. After cancelling it refetches
the spans so the trace reflects the cancelled state. Reuses the existing
cancel-parse i18n keys, already present in all locales.
2026-05-29 18:16:38 +08:00
wizardchen
9cb9505114 fix(knowledge): unstick summary and isolate manual re-index trace attempts
Manually uploaded text documents could get stuck showing "generating
summary" forever, and editing/re-saving a document mixed its trace into
the previous attempt.

Three related fixes:

1. finalizeIndexedKnowledgeState no longer marks a text document
   completed the moment chunks are indexed. Doing so made
   KnowledgePostProcess hit its non-processing guard and skip the
   summary/question/graph fan-out, stranding summary_status on
   "pending". The row now stays "processing" (still retrievable via
   enable_status=enabled) and post-process remains the sole authority
   that drives processing -> finalizing -> completed.

2. ProcessManualUpdate now allocates a fresh span-tracking attempt via
   OpenAttempt. Previously it ran with attempt 0, so processChunks
   dropped every stage span and KnowledgePostProcess fell back to
   LatestAttempt, piling the new run's summary/wiki subspans onto the
   prior attempt's trace.

3. Enrichment workers (summary, question, graph extract) skip when a
   newer attempt has superseded theirs. A stale subtask from a previous
   upload/edit/reparse must not read deleted chunks or decrement the new
   attempt's pending_subtasks_count, which would race-promote the row to
   completed before the new attempt finishes.

Updated TestFinalizeIndexedKnowledgeState to expect the deferred
completion for text documents.
2026-05-29 17:52:01 +08:00
wizardchen
9f139ff4d8 feat(migrations): add display_name column to models table
- Introduced a new column `display_name` to the `models` table for optional user-facing display names.
- Created migration scripts for both adding and removing the column, ensuring backward compatibility.
2026-05-29 17:03:42 +08:00
wizardchen
2241f9579d refactor(keywords_vector_hybrid_indexer): enhance image payload sanitization logic
- Added a check to skip regex processing for non-image content, improving performance when no inline base64 payload is present.
- Updated regex pattern to ensure it correctly matches base64 image data URIs.
2026-05-29 16:56:53 +08:00
wolfkill
6b3ad07543 fix embedding input image payload safety 2026-05-29 16:53:12 +08:00
wolfkill
2a54bb2d45 fix: avoid stale knowledge records on upload failure 2026-05-29 16:49:07 +08:00
mileslai
b603b1dcfa fix(mcp-server): address Copilot PR review comments
- SSL verification now defaults to enabled; set WEKNORA_VERIFY_SSL=false to
  opt out (with a logged warning). Fixes MITM risk from default-off TLS.
- WEKNORA_CHAT_TIMEOUT parse is now guarded with try/except ValueError so a
  bad env value falls back to 300s instead of crashing at import.
- SSE streaming response is now closed via context manager (with response:)
  to guarantee connection pool return even on early break.
- Replace asyncio.get_event_loop() (deprecated) with asyncio.get_running_loop()
  in both chat and agent_chat handlers.
- create_session now calls resolve_kb_id() so KB names are accepted in addition
  to UUIDs (consistent with chat / hybrid_search).
- knowledge_base_ids description changed from REQUIRED to Strongly recommended
  to match actual schema optionality.
- run_sse() handle_sse rewritten as raw ASGI callable (scope, receive, send) to
  avoid accessing Starlette private _send attribute.
- Fix main.py comment: http transport is Streamable HTTP (MCP spec), not long-polling.
2026-05-29 16:40:29 +08:00
mileslai
835148626b chore(docker): make mcp service optional via --profile full 2026-05-29 16:40:29 +08:00
mileslai
8ffd1ee2d6 fix(mcp-server): restore all create_session parameters (kb_id, max_rounds, enable_rewrite, fallback_response, summary_model_id)
Restore parameters that were inadvertently removed during refactoring.
- kb_id: Required knowledge base ID (architectural shift from KB-agnostic back to KB-bound sessions)
- max_rounds, enable_rewrite, fallback_response: Session strategy configuration
- summary_model_id: Model for response summarization
- title, description: Optional session metadata

These parameters enable AI agents to fully configure session behavior.
2026-05-29 16:40:29 +08:00
mileslai
e9a242c25f feat(mcp-server): add multi-transport support (stdio / SSE / HTTP) 2026-05-29 16:40:29 +08:00
yaol
00af694c52 feat(mcp): expose read-only wiki tools via Python MCP server
Add 3 read-only wiki tools (wiki_search, wiki_read_page,
wiki_index_view) to the Python MCP server, enabling external agents
like Claude Code and Codex to query WeKnora's LLM-generated wiki
pages following the LLM Wiki pattern.

Closes #1501

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-29 16:37:12 +08:00
ochan.kwon
40b74e2efa feat(retriever): activate OpenSearch k-NN driver (PR 3 of 3)
Phase 3 (#1440) gate flip. PR 1 (#1445) + PR 2a (#1481) + PR 2b (#1482)
laid the type prep + driver skeleton + read/write paths as gated dead
code; this PR wires every activation surface so opensearch becomes a
registerable VectorStore engine.

Activation wiring
- internal/types: validEngineTypes / GetVectorStoreTypes (with HNSW
  bounds + knn_engine enum + Immutable hints) / retrieverEngineMapping /
  buildEnvStoreForDriver — every gated surface now recognises
  "opensearch". IndexConfig grows four omitempty HNSW fields (HNSWM /
  HNSWEFConstruction / HNSWEFSearch / KNNEngine), keeping other engines'
  serialised config byte-identical.
- internal/container: createOpenSearchEngine + the switch case in
  createEngineServiceFromStore; the RETRIEVE_DRIVER=opensearch env path
  in initRetrieveEngineRegistry; NewEngineFactory now closes over the
  AuditLogService (the EngineFactory type itself is unchanged).
- internal/application/service/vectorstore_healthcheck.go: a
  testOpenSearchConnection case so CreateStore's connectivity probe
  accepts opensearch instead of returning 400.
- internal/application/repository/retriever/opensearch/transport.go:
  NewOpenSearchClient is exported so the factory and env path can build
  the TLS-hardened client; healthcheck.go reuses the unexported
  probeVersion / probeKNNPlugin for the service-layer probe.

Service-layer validation
- validateOpenSearchIndexConfig validates the HNSW caps (m 2-100,
  ef_construction 2-4096, ef_search 1-10000, knn_engine ∈ lucene|faiss).
  Shards/replicas continue to be enforced by the flat ValidateIndexConfig.
  Create-only: UpdateStore mutates the name only.
- validateConnectionConfig requires addr for opensearch.

Sync implementations (stubs.go shrinks)
- CopyIndices (copy.go) mirrors the Elasticsearch / Qdrant pattern —
  search → BatchSave with the source_id remap for generated questions —
  so dim/keyword routing and the source_id contract come from BatchSave
  for free. embeddingMap is keyed by the *target* SourceID because
  OpenSearch's BatchSave looks up embeddings by SourceID
  (lookupEmbedding), not by chunk_id (the ES driver's convention).
  Pagination is from/size; copies larger than max_result_window
  (default 10000) need the scroll-based async path that lands later.
- BatchUpdateChunkEnabledStatus / BatchUpdateChunkTagID (bulk_update.go)
  group the input by target value and issue one _update_by_query per
  group over the cross-dim <base>_* pattern. Caller values flow through
  bound script params only — never string-interpolated into the Painless
  source — closing the script-injection surface.
- inspectByQueryResponse (byquery.go) mirrors inspectBulkResponse: the
  full failure reason goes to the debug log only; the returned error
  carries the bounded id + type.
- UpdateByQueryParams.Refresh is *bool in opensearch-go v4.6.0 (the same
  shape as DeleteByQuery's quirk), so refresh=wait_for is not
  expressible; we use refresh=true.

Driver-owned audit (DIP)
- A new opensearch.AuditSink interface (with nopSink + WithAuditSink
  functional option) lets the driver emit opensearch.index_created and
  opensearch.reindex_executed events without importing any service
  package — the service layer implements the interface. NewRepository
  takes opts, so existing 4-arg test call sites keep compiling unchanged.
- internal/container/audit_sink.go bridges AuditSink to AuditLogService.
  When the context carries no tenant (the env-path registration ctx
  during boot, for example) the adapter skips the emit with a warning
  rather than silently writing tenant_id=0, which would collide with the
  system-scope sentinel.

Frontend + polish
- FieldSchema (frontend/src/api/vector-store.ts) gains min/max/enum/
  immutable. VectorStoreSettings.vue is now schema-driven: a closed
  `enum` renders a t-select; number inputs use the schema's `:min`/`:max`
  and fall back to the legacy replica-vs-shard heuristic only when the
  schema does not pin them; a danger-coloured warning fires when
  insecure_skip_verify is toggled on (the switch and warning are wrapped
  in a vertical stack so the warning sits on its own row below the switch).
- i18n: labels for hnsw_m / hnsw_ef_construction / hnsw_ef_search /
  knn_engine / insecure_skip_verify plus the warning copy in en-US,
  ko-KR, zh-CN, ru-RU.
- docker-compose.dev.yml: an opensearch profile (single-node 3.3.2 with
  security plugin disabled for dev only). OpenSearch Dashboards lives in a
  separate, opt-in opensearch-ui profile so the heavy UI container is not
  forced up alongside the cluster (the driver e2e is fully curl-verifiable
  against :9200). The new docs/dev/opensearch-integration-test.md covers the
  end-to-end exercise and the single-node guidance (set replicas=0 to keep
  the cluster Green).

Gating-guard tests flipped
- The "OpenSearch is NOT in validEngineTypes / mapping / types list /
  env builder / stubs" guard tests from PR 1 / PR 2 are replaced by
  their positive counterparts in this PR. The test suite was the
  activation checklist; the activation flip is its diff.

Backward compatibility
- Additive everywhere. IndexConfig's new HNSW fields are omitempty so
  other engines' serialised config is byte-identical. Existing
  Elasticsearch / Qdrant / Milvus / Weaviate / Doris / TencentVectorDB
  stores are untouched. No migrations.

Test plan
- go build ./... clean
- go vet ./... clean
- gofmt -l clean on touched files
- go test ./... — only TestOssEnsureBucket_CreateFails (Aliyun OSS
  endpoint), the docreader gRPC tests, and the doris SQL-shape tests
  fail; all three are pre-existing on upstream/main and untouched by
  this PR.
- New tests across internal/types, opensearch, service and container —
  including a full end-to-end env-path test that exercises
  initRetrieveEngineRegistry with RETRIEVE_DRIVER=opensearch against an
  httptest cluster.
2026-05-29 16:32:27 +08:00
wizardchen
19a1b15106 fix(container): correct resetPendingTasks SQL on startup
Split knowledge list/update queries to avoid GORM UPDATE...FROM
duplicate-table errors after Find, and use sync_logs started_at/
finished_at column names instead of start_time/end_time.
2026-05-29 15:46:11 +08:00
wizardchen
7d1dfc78b7 fix(knowledgeBase): improve tag loading logic and ensure consistent behavior
- Updated the loadTags function to prevent unnecessary calls when tags are already loading, enhancing performance and user experience.
- Modified tag loading calls in various tag-related functions to ensure the reset parameter is consistently set to true, ensuring the tag list is refreshed correctly after operations like create, edit, and delete.
- Improved the FAQEntryManager component to handle tag loading more efficiently during scrolling and batch operations.
2026-05-29 15:42:44 +08:00
wizardchen
c07ab6988c feat(frontend): enhance IMChannelsOverviewPanel layout and functionality
This update improves the layout and user experience of the IMChannelsOverviewPanel component. Key changes include:

- Added tooltips for subtitles and agent names for better accessibility.
- Refactored channel and agent display logic to improve clarity and consistency.
- Adjusted styling for better visual hierarchy and responsiveness.
- Enhanced toggle functionality for IM channels to ensure state consistency during updates.

These changes aim to provide a more intuitive interface for users managing instant messaging channels.
2026-05-29 15:40:38 +08:00
wizardchen
bb202a203a feat: implement tenant validation for file access and enhance related tests
This commit introduces a new validation mechanism to ensure that file access paths include the correct tenant segment, preventing cross-tenant access. The `ValidateStoragePathTenant` function has been added to enforce this rule, and the `serveFiles` function has been updated to return a forbidden status for invalid paths. Additionally, new tests have been added to verify the behavior of the file service under various tenant scenarios, ensuring robust handling of file access permissions.
2026-05-29 13:47:32 +08:00
wizardchen
48381fbaf5 feat: implement tenant default storage provider handling in Knowledge Base creation
This commit introduces functionality to utilize the tenant's default storage provider when creating a Knowledge Base. It includes updates to the frontend to load the default provider from settings and apply it during Knowledge Base initialization. Additionally, the backend has been enhanced to ensure that the storage provider is set correctly based on tenant configuration, improving consistency across the application. Tests have been added to verify the correct application of the default storage provider in various scenarios.
2026-05-29 13:47:32 +08:00
wizardchen
e0823eff31 feat: enhance IM file service tests and refactor storage URL handling
This commit introduces a new test suite for the IM file service, including a stub implementation for testing purposes. It adds tests for resolving file services based on storage providers and ensures proper fallback mechanisms for MinIO URLs. Additionally, the `rewriteStorageURLs` and `cleanIMContent` functions have been refactored to utilize a resolver for improved caching and efficiency. These changes enhance the robustness of file service handling and improve test coverage for various storage scenarios.
2026-05-29 13:47:32 +08:00
wizardchen
bdb164d432 feat: add tests for incomplete Markdown image handling and improve stream flush logic
This commit introduces a new test, `TestFindIncompleteMarkdownImage`, to validate the detection of incomplete Markdown images in various scenarios. Additionally, it enhances the `holdbackCutoff` function to prioritize handling incomplete Markdown images, ensuring that they are correctly managed during stream flush operations. The changes improve the robustness of image processing in the application, addressing potential issues with unclosed image URLs in Markdown content.
2026-05-29 13:47:32 +08:00
wizardchen
5d02404fdd feat: add unit tests for COS object name parsing and enhance error handling
This commit introduces unit tests for the `parseCosObjectName` method in the `cosFileService`, ensuring it correctly rejects local scheme URLs and properly parses COS scheme URLs. Additionally, the `parseCosObjectName` method has been updated to return an error for unsupported schemes, improving error handling in the `GetFile` and `DeleteFile` methods. This enhancement ensures more robust handling of file paths in the application.
2026-05-29 13:47:32 +08:00
wizardchen
83808cc5b7 feat(frontend): improve agent and KB editor ID display and intent prompts UX
Expose copyable resource IDs in edit modals and replace the intent prompt
dropdown with independent toggle buttons so multi-intent selection wraps cleanly.
2026-05-28 21:11:49 +08:00
wizardchen
0e1282c2da test: add document process task options tests
Centralize DefaultDocumentProcessTimeout in the config package and
reuse DocumentProcessTimeout() from documentProcessTaskOptions.
2026-05-28 20:27:53 +08:00
HelloWeit
edfabf3135 bugfix 2026-05-28 20:22:55 +08:00
wizardchen
f1a27e0e18 feat: implement CancelOpenSpansByName method and related tests
This commit introduces the CancelOpenSpansByName method in the KnowledgeSpanRepository, allowing for the cancellation of open spans by their name for a specific knowledge ID and attempt. This functionality is crucial for managing spans during retries or server restarts, preventing duplicate entries in the trace tree. Additionally, a new test case, TestKnowledgeSpanRepo_CancelOpenSpansByName, has been added to ensure the correct behavior of this method, verifying that only the intended spans are cancelled while others remain unaffected. This enhancement improves the robustness of span management in the application.
2026-05-28 20:16:02 +08:00