Commit Graph

61 Commits

Author SHA1 Message Date
wizardchen
3475af1707 feat(frontend): configure API proxy target for development environment
Updated the Vite configuration to allow dynamic setting of the API proxy target based on environment variables. The default target is now configurable via VITE_DEV_PROXY_TARGET or FRONTEND_BACKEND_URL, enhancing flexibility for different development setups. Additionally, the development script logs the current API proxy target for better visibility during startup.
2026-05-20 21:00:16 +08:00
wizardchen
80bd268862 chore: release v0.5.2
Bump version to v0.5.2 across VERSION, Helm chart, frontend package
files, and cloud-image script comment. Update CHANGELOG and all four
language READMEs with v0.5.2 highlights (Wiki Mode at scale, MCP
human-in-the-loop approval, new LLM/vector/storage/search backends,
adaptive 3-tier chunking, global command palette, CLI preview, etc.).
2026-05-13 15:04:15 +08:00
wizardchen
bd68a0c377 feat(cloud-image): support apt-based docker install for restricted-egress hosts
Mainland China cloud VMs (Tencent Lighthouse, Aliyun, etc.) frequently
cannot reach get.docker.com, github.com, or even community GitHub
mirrors like gh-proxy.com. The cloud-image bootstrap previously had no
escape hatch for this and failed at the very first curl.

This adds a new DOCKER_INSTALL_MIRROR env var to prepare.sh. When set,
it skips get.docker.com and installs docker-ce + compose-plugin from an
apt mirror of Docker's official repo (e.g. mirrors.tencent.com,
mirrors.aliyun.com).

README.md also gets:
- A GH_PROXY env var threaded through bootstrap methods A and B so the
  initial script pull can route through gh-proxy / ghfast.
- An explicit recommendation to prefer method C (scp from local) on
  mainland China VMs.
- A consolidated "三件套" table mapping WEKNORA_GH_PROXY /
  DOCKER_INSTALL_MIRROR / DOCKER_REGISTRY_MIRROR to per-cloud
  endpoints, so users hit one place to copy the full env.
2026-05-11 21:14:08 +08:00
wizardchen
0cfbad7f97 fix(cloud-image): enhance firstboot and cleanup scripts for improved security and functionality
- Updated cleanup.sh to avoid recreating .env during cleanup, preventing exposure of default passwords before firstboot.
- Modified firstboot.sh to create .env from .env.example only if it doesn't exist, ensuring no sensitive data is present before initialization.
- Added support for Docker Hub and GitHub tarball download acceleration via new environment variables WEKNORA_GH_PROXY and DOCKER_REGISTRY_MIRROR.
- Implemented a mechanism to prune old WeKnora images based on the current version, reducing image size and maintaining a clean environment.
- Enhanced README.md with instructions for using the new acceleration features and image pruning options.
2026-05-11 15:51:11 +08:00
wizardchen
1e3aa337db fix(cloud-image): refine cleanup process to prevent unwanted image deletions
Updated the cleanup script to replace the broad `docker system prune` command with more targeted commands. This change ensures that only unused containers, build cache, and dangling volumes are removed, preventing the accidental deletion of important images like `wechatopenai/weknora-*` that are pre-pulled for firstboot. The script now explicitly prunes containers, builders, and dangling volumes to maintain a clean environment without disrupting the intended image setup.
2026-05-11 12:25:19 +08:00
wizardchen
cdfbf05524 fix(cloud-image): make firstboot idempotent and pin image versions
Address review feedback on PR #1249:

- prepare.sh: when WEKNORA_REF looks like a version tag (v*), write the
  matching WEKNORA_VERSION into .env so docker compose pulls images that
  match the compose YAML's git ref (previously stuck on :latest).
- prepare.sh: detect docker binary path via `command -v docker` and
  template it into weknora.service (replacing hardcoded /usr/bin/docker
  that fails when docker lives in /usr/local/bin).
- firstboot.sh: write a /opt/WeKnora/.firstboot.done marker immediately
  after rewriting .env, before `docker compose up -d`. If compose fails
  mid-run, the next boot is gated by ConditionPathExists=!marker so we
  never regenerate DB_PASSWORD against an already-initialized postgres
  volume (which previously bricked the database).
- firstboot.sh: stop deleting its own unit file / script while the
  oneshot is still executing; rely on the marker + `systemctl disable`
  instead, avoiding "job failed" markings from systemd.
- firstboot.sh: use detected docker path instead of /usr/bin/docker;
  add note in credentials file that .env is the source of truth.
- weknora-firstboot.service: add ConditionPathExists=!.firstboot.done.
- cleanup.sh: scope docker volume deletion to compose project label
  (com.docker.compose.project=<name>) instead of fuzzy substring match
  that could nuke unrelated postgres/redis volumes.
- cleanup.sh: also remove .firstboot.done marker, firstboot log, and
  any leftover /root/weknora-credentials.txt so the image is clean.
- README.md: clarify how to actually disable registration (edit the
  `replace` call list in firstboot.sh, not run that command in shell).
2026-05-11 12:25:19 +08:00
wizardchen
155f3b3e72 docs(cloud-image): clarify sudo + redirection pitfall in setup steps
Recommend `sudo -i` to avoid the classic `sudo cmd >> file` failure
where the shell redirection runs as the unprivileged user. Also document
the `sudo tee -a` workaround and add a scp option C.
2026-05-11 12:25:19 +08:00
wizardchen
afd7d1fdf8 docs(cloud-image): add cloud-agnostic image packaging scripts
Add scripts and docs for packaging WeKnora into cloud images (AMI,
custom images, snapshots) so users can distribute one-click deployable
templates on any cloud provider.

- scripts/cloud-image/: cloud-agnostic prepare/cleanup/firstboot scripts
  plus systemd units. Downloads only the 4 runtime files needed by the
  compose stack (~100KB) instead of cloning the full repo, and pins to
  any git ref via WEKNORA_REF for reproducible builds.
- firstboot.sh randomizes DB/Redis/JWT/AES secrets on first boot,
  writes credentials to /root/weknora-credentials.txt and self-removes.
- docs/cloud-image/: per-platform packaging guides. Includes a guide
  for Tencent Cloud Lighthouse / CVM covering image creation, sharing,
  and marketplace listing.

Default-on services match the unprofiled compose stack (frontend, app,
docreader, postgres, redis); optional services (qdrant, milvus,
neo4j, langfuse, etc.) remain opt-in via compose profiles to keep the
image size small.
2026-05-11 12:25:19 +08:00
wizardchen
fc6f160eff fix(retriever/doris): code review cleanup
针对 4cce6f2e(接入 Apache Doris)的 code review 修复,主要修正若干阻断性
问题与可读性问题,并剔除不应进入主仓的本地工作流文件。

阻断性修复:
- docker-compose: Doris 镜像由 2.1.0 升至 4.1.0。原 2.1.0 不支持 HNSW
  ANN、cosine_distance_approximate 与 Stream Load partial_columns,
  按当前 DDL 一启动就会失败。
- DSN 字面量拼接改用 mysql.Config.FormatDSN()。原 fmt.Sprintf 在用户名/
  密码包含 `@`/`:`/`/` 等字符时会跑偏。覆盖 health check 与 engine
  factory 两处。

健壮性修复:
- 新增 validateEmbedding,写入与查询前拒绝 NaN/±Inf;strconv.FormatFloat
  对非有限值会输出 "NaN"/"+Inf" 拼成无效 SQL。
- waitANNReady 改为后台 goroutine + 独立 context,避免新维度首次写入路径
  阻塞最长 30s;ANN 未就绪时 Doris 会自动退化为 brute-force。

清理:
- annIndexReady 移除最终两个分支都 return true 的冗余写法。
- Save 移除冗余的双重 toDorisVectorEmbedding。
- testDorisConnection 把 "5.7.99 Doris-4.1.0" 解析为裸 "4.1.0",与
  Postgres/ES 的版本格式对齐。

剔除(不应合入主仓):
- docs/wiki/集成扩展/Doris改动与上游同步.md:纯 fork 维护工作流文档。
- scripts/e2e-doris.sh:作者本地 E2E 验证清单。

测试:
- repository_test 用 require.Eventually 适配 ANN 异步轮询。
- 现有 doris 单测全部通过。
2026-05-09 00:31:03 +08:00
issunion
4cce6f2e99 feat(retriever): 接入 Apache Doris 4.1 作为向量数据库
为 RetrieveEngine 体系新增 Doris 后端,与现有 Qdrant/Milvus/Weaviate
等保持完整能力对齐:向量检索、关键词检索、健康检查、环境变量与多实例
DB 配置、前端类型注册、单元测试、Docker Compose 模板。

实现要点:
- 协议分工:主链路用 MySQL 协议(database/sql + go-sql-driver/mysql)
  做 DDL / 查询 / 删除;批量更新走 Stream Load HTTP API,并启用
  partial_update=true、merge_type=APPEND,自动按 1MiB 切分批次并处理
  307 重定向。
- 表结构:UNIQUE KEY(id) + enable_unique_key_merge_on_write=true 以
  支持 upsert/部分列更新;按维度分表(<base>_<dim>),每张表上建
  HNSW ANN 索引(metric_type=cosine_distance)和 INVERTED 索引
  (parser=chinese)。
- 分数语义:使用 cosine_distance_approximate,再以 1 - dist 转换为
  "越大越相似",与现有 KVHybridRetrieveEngine 约定一致。
- 异步索引:ANN 索引为后台构建,ensureTable 通过轮询 SHOW INDEX 等
  待索引就绪后再放行写入,避免首次检索召回为空。
- ARRAY<FLOAT> 序列化:go-sql-driver/mysql 不支持数组占位符,
  embeddingLiteral 将 []float32 转成 SQL 字面量字符串再拼接。

新增文件:
- internal/application/repository/retriever/doris/{structs,schema,
  query,repository,streamload,repository_test}.go
- scripts/e2e-doris.sh:E2E 验证清单
- docs/wiki/集成扩展/Doris改动与上游同步.md:fork-and-rebase 工作流
  与改动清单

修改文件(接线 + 文档):
- internal/types/{retriever,tenant,vectorstore}.go:新增
  DorisRetrieverEngineType、env 解析、表单 schema 与索引参数校验
- internal/container/{container,engine_factory}.go:环境变量驱动
  与 VectorStore 配置驱动两条路径都支持 Doris
- internal/application/service/vectorstore{,_healthcheck}.go:连接
  校验 + Ping/Version 健康检查
- docker-compose.yml:新增 doris-fe / doris-be 服务(profile=doris)
- .env.example:DORIS_* 环境变量与示例
- docs/{使用其他向量数据库,wiki/集成扩展/集成向量数据库}.md:
  使用说明与索引/分数行为说明

依赖:go.mod/go.sum 新增 github.com/go-sql-driver/mysql(运行时)和
github.com/DATA-DOG/go-sqlmock(测试)。

测试:repository 层 SQL 形状、Stream Load HTTP 行为、whereBuilder
逻辑、embeddingLiteral 往返、健康检查错误路径均有单测覆盖。
2026-05-08 21:59:35 +08:00
yy
6accdc2a60 fix:修复make dev-app里面脚本问题 2026-04-22 16:38:50 +08:00
ochan.kwon
8fd1d61333 feat: add VectorStore management UI settings page 2026-04-17 21:51:25 +08:00
wizardchen
52bdfd388a feat: implement desktop app structure with API base URL handling and bindings for Wails integration 2026-04-11 18:51:16 +08:00
wizardchen
0a964ac0b7 fix: resolve desktop app crash on launch by fixing working directory and loading resources 2026-04-11 18:51:16 +08:00
wizardchen
026c4c7662 fix: resolve neo4j parsing issue during macOS desktop app build by skipping bindings and fixing paths 2026-04-11 18:51:16 +08:00
wizardchen
96fb36ebb3 feat: introduce macOS desktop app using Wails wrapper 2026-04-11 18:51:16 +08:00
wizardchen
d005bd1db8 Revert "chore: make Lite edition support and related configurations hidden"
This reverts commit 88d5b303b8.
2026-04-11 18:51:16 +08:00
Windfarer
c1816fe6d6 add oidc 2026-03-30 11:13:44 +08:00
Dounx
cf9b935144 fix(dev): add milvus env and versioned docreader image 2026-03-20 15:50:16 +08:00
DaWesen
e309e0bed8 feat(storage): 集成S3存储适配器
添加对AWS S3及兼容存储服务的支持:
- 实现完整的S3FileService接口
- 支持文件上传、下载、删除功能
- 添加配置支持和环境变量检查
- 实现连接测试功能
- 遵循与其他存储适配器相同的代码风格
2026-03-09 10:39:46 +08:00
Manx98
1d1d3de76a fix: make dev-app command error on Linux 2026-03-09 10:33:52 +08:00
wizardchen
90c32b5926 feat: enhance Docker setup with entrypoint script and skill management
- Added a new entrypoint script to manage ownership of bind-mounted directories and merge built-in skills into preloaded ones.
- Updated the Dockerfile to include the `gosu` package for privilege management and to set the entrypoint to the new script.
- Ensured built-in skills are preserved and accessible after bind-mounting user directories, improving the application's flexibility and usability.

These changes streamline the container initialization process and enhance the management of skills within the application.
2026-03-05 18:14:14 +08:00
wizardchen
88d5b303b8 chore: make Lite edition support and related configurations hidden 2026-03-02 21:21:49 +08:00
wizardchen
aefa1c6fe8 feat: enhance system information display with database version
- Added `db_version` field to the `SystemInfo` interface to expose the current database migration version.
- Updated the system information response to include the database version, reflecting its state during application runtime.
- Enhanced the UI in the SystemInfo component to display the database version with appropriate labels and descriptions in multiple languages.

This update improves transparency regarding the database state within the system information settings.
2026-03-02 21:21:49 +08:00
wizardchen
6d88619869 feat: enhance Dockerfile and build scripts for customizable APT mirror
- Added support for customizable APT mirror in the Dockerfile for the docreader service, allowing users to specify a mirror via build arguments.
- Updated docker-compose.yml to pass the APT_MIRROR argument during the build process.
- Modified build_images.sh script to include the APT_MIRROR argument when building the docreader image.
- Updated .gitignore to exclude .cursor/ directory.

This update improves flexibility in package management during the image build process.
2026-03-02 21:21:49 +08:00
wizardchen
469f320d10 feat: enhance storage engine configuration and UI updates
- Updated `.air.toml` to include additional CGO flags for improved build settings.
- Expanded `StorageEngineConfig` interface to support "tos" (火山引擎 TOS) as a new storage provider.
- Modified related components and views to accommodate the new storage engine, including updates to `ListSpaceSidebar`, `AgentList`, `KnowledgeBaseList`, and `OrganizationList`.
- Improved UI elements for batch management in the menu and sidebar components.
- Added internationalization support for new memory features and storage engine descriptions across multiple languages.

This update enhances the flexibility of storage options and improves user experience with better UI interactions.
2026-03-02 21:21:49 +08:00
wizardchen
397689d2f3 feat: introduce WeKnora Lite edition with lightweight configuration and deployment
- Added a new `.env.lite.example` file for the Lite version, providing a minimal configuration template.
- Updated `.env.example` to remove deprecated variables and include new Docreader settings.
- Enhanced Docker configurations to support the Lite version, including a new Dockerfile for the Docreader service.
- Introduced a Makefile target for building and running the Lite version, along with packaging capabilities.
- Created GitHub workflows for building and releasing Lite binaries, including Homebrew formula support.
- Implemented a new service file for managing the Lite version as a system service.

This update enables a streamlined, single-binary deployment of WeKnora, reducing external dependencies and simplifying setup.
2026-03-02 21:21:49 +08:00
Dounx
6adecdb30e feat: add volcengine tos support 2026-02-25 14:24:50 +08:00
nullkey
43a2c647ea feat: optimize security and deployment of agent skills 2026-02-04 20:08:49 +08:00
MEKXH
9a3f09c983 docs: update Air installation command from cosmtrek/air to air-verse/air 2026-02-04 19:27:44 +08:00
wizardchen
1f6bb3f841 feat: 支持Qdrant多维度向量存储和全文检索优化,添加Docker Compose配置 2025-12-08 19:38:16 +08:00
wizardchen
c67a5d3949 feat: 支持数据库迁移脏状态自动恢复和Neo4j连接重试机制 2025-12-08 19:38:16 +08:00
wizardchen
ae58801baf Merge remote-tracking branch 'github_public/main' 2025-11-13 14:28:41 +08:00
fuyaozong
a625eff525 修复bug:脚本检查系统环境时如果操作系统是Darwin,检查内存使用情况输出为空,原因是正则表达式有问题 2025-11-13 10:30:57 +08:00
wizardchen
9f6b903ef2 feat: Integrate LLM-based reranking in KnowledgeSearchTool
- Added optional chat model to KnowledgeSearchTool for enhanced reranking using LLM prompt scoring.
- Updated rerankResults method to prioritize LLM-based reranking over traditional rerank model.
- Implemented new functions for LLM scoring and parsing scores from responses, improving search result relevance.
- Enhanced migration script to handle database versioning and dirty state checks more robustly.
2025-11-13 01:22:08 +08:00
wizardchen
77b6133e2a feat: Add MCP services support and enhance migration scripts
- Introduced new MCP services functionality, including management routes and integration with the agent service.
- Updated Docker Compose configuration to include a new SQL migration for MCP services.
- Enhanced Makefile with new migration commands for versioning and creating migrations.
- Improved chat component to handle selected MCP services in message processing.
- Updated migration documentation to reflect new strategies and added MCP services migration files.
2025-11-09 03:27:24 +08:00
wizardchen
83721f98d4 feat: Integrate Swiper library and enhance chat functionality
- Added Swiper library to package dependencies for improved UI components.
- Updated chat API to include summary model ID in message handling, allowing for more flexible model selection.
- Enhanced InputField component to manage model selection and state more effectively.
- Improved event handling in chat components to support new model features and ensure accurate message processing.
- Refactored SVG assets for better visual consistency and updated styles for improved user experience.
2025-11-09 00:02:31 +08:00
wizardchen
d387fd672f chore: Update .gitignore and Makefile for development enhancements
- Added 'logs/' and '*.pid' to .gitignore to exclude log files and process ID files from version control.
- Expanded Makefile with new development commands for easier local environment management, including 'dev-start', 'dev-stop', 'dev-restart', 'dev-logs', 'dev-status', 'dev-app', and 'dev-frontend'.
- Updated README_CN.md to include instructions for the new development mode and commands for improved developer experience.
2025-11-07 15:52:52 +08:00
begoniezhao
07c3453e1a chore: 移除Docker构建参数并更新.gitignore规则 2025-11-07 10:48:38 +08:00
wizardchen
4fa3adbf3b feat: Add agent configuration and cleanup scripts for database migrations 2025-11-05 23:18:44 +08:00
wizardchen
9756b4aa82 fix: Get version script 2025-09-17 16:40:48 +08:00
wizardchen
5e0a99b127 fix: Get version script 2025-09-17 16:40:48 +08:00
wizardchen
98e47e83d2 feat(ui): Support multi knowledgebases operation 2025-09-17 16:02:08 +08:00
wizardchen
91e65d6445 feat(ui): Support multi knowledgebases operation 2025-09-17 16:02:08 +08:00
lyingbug
e8b8215f1e Merge branch 'main' into login_page 2025-09-16 10:24:05 +08:00
lyingbug
c3744866fd Merge branch 'main' into login_page 2025-09-16 10:24:05 +08:00
wizardchen
81bd2e6c2c feat: Add Login Page 2025-09-16 02:47:39 +08:00
wizardchen
cd249df8c8 fix(ui): Ignore showing APIKEY for security 2025-09-15 14:52:00 +08:00
wizardchen
74c121f7fb feat: Adjust App & Docreader log output 2025-09-11 23:14:23 +08:00
wizardchen
fd6c50059e feat: support modify listening port 2025-09-10 10:14:43 +08:00