pub_soft/WeKnora
3
0
Fork 0
mirror of https://github.com/Tencent/WeKnora.git synced 2026-06-04 13:30:32 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,686 Commits 17 Branches 36 Tags
7611d59d715d7bbbdf50f4faa178d036fdb9ab7b
Commit Graph

8 Commits

Author SHA1 Message Date
nullkey
7611d59d71 docs(cli): README / AGENTS.md / CHANGELOG + CI parity test 
Wire-contract documentation and the CI check that keeps it honest.

* cli/README.md gains a verbatim --help block (top-level + subtrees),
  an Exit codes table covering 0/1/2/3/4/5/6/7/10/124/130, a "Status
  vs check" verb-pair subtable, and a "doc wait" paragraph spelling out
  the four exit codes (0 / 1 / 124 / 130). The api passthrough note
  trims storage provider out of the deep-config list now that
  kb create --storage-provider is a polished flag.
* cli/AGENTS.md becomes the contributor guide: build/test, CRUD flag
  conventions, the status/check verb pattern, long-poll wait commands,
  the SetAgentHelp pattern, and a full Error code reference with 35
  typed codes mapped to namespaces, exit codes, retryable / hint
  guidance. Reference section is bracketed by HTML markers so a CI
  parity test can keep it in sync with AllCodes().
* cli/internal/cmdutil/errors_doc_test.go enforces parity: every code
  in AllCodes() must appear in AGENTS.md inside the markers, and
  AGENTS.md must not reference codes that no longer exist. Fails CI
  if a new typed code is added without documentation.
* CHANGELOG.md gets the v0.6 entry: BREAKING (--json / --no-stream /
  WEKNORA_SDK_DEBUG / kb create --name), Added (--format / --jq /
  doc wait / --log-level / kb-and-agent status & check / multi-id
  delete / api --paginate / MCP schema extension / SetAgentHelp /
  signal-aware ctx / kb create --storage-provider / new operation.*
  namespace), Changed (multi-id partial-failure exit code, doc upload
  FlagError, --log-level FlagError, multi-id stdout cleanup, README /
  AGENTS.md changes), with a Migration from v0.5 section walking
  every BREAKING through its v0.6 replacement.
 2026-05-18 11:10:19 +08:00
nullkey
c87e35b34b chore(cli): polish + docs sync + pre-PR audit fixes 
Code-reuse polish (post-implementation review pass):
- Extract text.OneLine(maxWidth, s) helper combining preview-row
  normalization (newline/CR/tab → space) with text.Truncate's
  UTF-8-safe truncation. Replaces agent/view.go truncate1Line (ASCII
  '...' + byte-slice CJK-unsafe) and chunk/list.go singleLine.
- Lift cmdutil.OpenInput(path) for the '-' = stdin / else os.Open
  pattern shared across agent create/edit and the api command.
  Replaces agent/create.go's private openInput.
- Strip inline doc-spec parentheticals from source comments — those
  belong in commit messages and project docs, not in source where
  they rot.

Pre-PR audit fixes:
- doc upload: reject `--metadata` paired with `--from-url` as
  input.invalid_argument up-front (the URL-ingest request type has
  no metadata field server-side, so the pair would otherwise silently
  drop). Long help and CHANGELOG updated to call out the asymmetry.
- doc upload (file path): map sdk.ErrDuplicateFile sentinel to
  resource.already_exists. The sentinel arrives with no "HTTP error <n>:"
  prefix because the SDK short-circuits on file-hash before reading the
  HTTP status, so the previous WrapHTTP fall-through misclassified it
  as network.error with a misleading "check base URL reachability" hint.
  The --from-url branch already handled ErrDuplicateURL this way; this
  closes the asymmetry. Caught by e2e re-upload of an already-ingested
  file; regression test added.
- README exit-10 enumeration adds `agent delete` and `chunk delete`
  (these were missing alongside the v0.5 destructive verbs they were
  meant to gate).

Docs sync:
- cli/README.md: command tree now includes the chunk subtree; adds
  agent / chunk lines to the 5-minute quickstart; adds a "Contributing
  / Reporting issues" section pointing at the repo's SECURITY.md and
  AGENTS.md; drops third-party CLI parallels from the surface
  description.
- cli/AGENTS.md: "Command surface design SOP" gains the
  flag-vs-escape-hatch step. "CRUD command flag canon" renamed to the
  hard-required-flags pattern with the contrast (TTY-prompts-fill)
  defined inline rather than via opaque shorthand.
- cli/CHANGELOG.md: search docs case-sensitivity shift promoted to its
  own #### Breaking changes subsection. MCP doc_list filter count
  corrected from 5 to 6. Drops the bogus go.mod yaml.v3 entry (yaml.v3
  was already a dependency on main; v0.5 added zero go.mod lines).
  Replaces internal-Go identifiers (fuzzyTime, NoOptDefVal) with
  user-language and drops the § section-symbol jargon.
 2026-05-16 16:56:33 +08:00
nullkey
f2e8e3f56c refactor(cli): drop aiclient package; align AGENTS.md with mainstream 
Survey of 10 mainstream CLIs (gh, lark, stripe, vercel, supabase, aws,
azure, gcloud, openai/codex, github-copilot-cli) showed env-gated
per-command --help blurbs are a Stripe-only pattern; gh uses env detect
for telemetry only, and lark relies on installed agent Skills + MCP.
Our cmd/mcp/serve already covers the dominant 2025/26 path, so
internal/aiclient/ (136 LOC + 38 callsites) is net maintenance burden
without precedent.

- Drop internal/aiclient/ entirely (annotations + detect + tests)
- Remove 38 SetAgentHelp callsites + agentAwareHelpFunc / SetHelpFunc
  wiring in cmd/root.go
- Migrate 4 command-level rules to standard Long help (visible to all,
  not env-gated): doc upload mode mutex, kb edit at-least-one,
  kb pin idempotent, search chunks channel mutex
- Rewrite AGENTS.md as a developer guide (gh-style 6 H2 / 167 lines):
  audience preamble + Build / Architecture / Command Structure /
  Testing / Code Style / Error Handling. Drops sections absent in
  surveyed projects (Commit & PR Conventions, Who Uses This CLI)
- Clean 14 internal doc refs (ADR-N, spec §X, v0.X) in source comments
  and docs that pointed at docs/superpowers/ — that directory is
  local-only / uncommitted, so refs are dead for outside readers
- Drop forward-looking "once v0.2 ships" from README
 2026-05-15 12:03:56 +08:00
nullkey
e623e8208f refactor(cli): delete envelope infrastructure, errors to stderr 
Removes the entire envelope machinery now that every success path
emits bare JSON:

- cli/internal/format/envelope.go (Envelope, Success, Failure,
  SuccessWithRisk, WriteEnvelope, Meta, Notice, UpdateNotice,
  VersionSkewNotice, Risk, RiskLevel, ErrorBody) + tests.
- cli/internal/format/filter.go envelope-specific helpers
  (WriteEnvelopeFiltered, marshalEnvelope, applyFieldFilter,
  filterDataPayload, filterObjectData); the reusable
  filterArrayItems / filterObjectKeys / writeJQ stay for bare.go.
- cli/internal/cmdutil/exporter.go + tests (envelope-only).
- cli/internal/cmdutil/PrintErrorEnvelope + ToErrorBody +
  operationRiskOf + Error.OperationRisk field + OperationRisk struct.

Error path: all errors now go to stderr via cmdutil.PrintError in
`code: message\nhint: ...` form, regardless of --json. Stdout stays
empty (or holds the partial-success the command already wrote) so
downstream `--json | jq` pipelines never have to filter error shapes
out of the success stream. Typed exit codes (3 auth.* / 4
resource.not_found / 5 input.* / 6 server.rate_limited / 7 server.*
+ network.* / 10 input.confirmation_required) carry the failure
class for agents that branch on it.

Acceptance contract:
- envelope_test.go → wire_test.go (TestEnvelopeGolden → TestWireGolden).
- testdata/envelopes/ → testdata/wire/.
- Error-path cases assert the typed code substring on stderr.
- Orphan whoami.*.json goldens deleted.

AGENTS.md + README.md rewritten for the bare-data contract:
- Drop envelope schema section + dry-run rule.
- Document bare JSON on stdout + `code: msg\nhint: …` on stderr.
- ADR-3 reframed around bare data and why error separation matters
  for `--json | jq` pipelines.

WriteJSONFiltered short-circuits to WriteJSON when both filters are
empty (skip the marshal-buffer round-trip for the common case).

Final review pass:
- Fix wire-contract bug: `--json id,name` (space form) is broken by
  pflag's NoOptDefVal; AGENTS.md / README.md / SetAgentHelp + the
  field-discovery help text all switched to `--json=id,name`.
- Fix `weknora api --jq` silently ignored: api.go now routes through
  WriteJSONFiltered with jopts.JQ.
- AGENTS.md: drop the false claim that `auth logout` honors `-y`
  (logout is local-only with no ConfirmDestructive guard); list the
  actual destructive commands instead.
- Rewrite cli/acceptance/e2e/e2e_test.go for the bare-data wire shape
  (was still parsing `out["data"]` / `env["ok"]`).
- Add `JSONOptions.Emit(w, v)` helper; collapse ~33 repeated
  `format.WriteJSONFiltered(iostreams.IO.Out, X, jopts.Fields,
  jopts.JQ)` sites to `jopts.Emit(iostreams.IO.Out, X)` — drops the
  format import from 22 cmd/* files.
- Delete single-caller `cmdutil.MustRequireFlag`; inline as
  `_ = cmd.MarkFlagRequired(...)` everywhere.
- Add `_ = cmd.MarkFlagRequired("name")` to `kb create`; it was the
  only write command relying on runtime --name validation while
  `context add` already used the cobra-level mark.
- `context use`: register `--json` / `--jq` (was always emitting JSON
  unconditionally with no human path and no flag — diverged from
  every other write command); human mode now prints
  `✓ Switched context to X (was Y)`.
- Replace per-package `confirmPrompter` / `scriptedConfirm` /
  `errPrompter` test doubles with `testutil.ConfirmPrompter`.
- Rename `chatService` → `ChatService` (export to match siblings
  `ListService` / `ViewService`); rename `printUploadSuccess` →
  `renderUploadSuccess` (siblings use `render*`).
- `defaultHint(CodeResourceNotFound)`: drop the hardcoded
  "list available with `weknora kb list`" — misleading on agent /
  doc / session 404. Replaced with "verify the resource ID and try
  again".
- Strip stale `v0.2/v0.3` / "envelope" / "v0.0/v0.1 supports only"
  historical tags from production comments and a few test
  descriptions.
 2026-05-15 12:03:56 +08:00
nullkey
cc8254f862 refactor(cli): drop --dry-run + introduce bare-JSON output path 
Two intertwined mainstream-alignment moves bundled because they share
the migration target (every command's --json path):

1. Drop --dry-run entirely. Survey of comparable API-wrapper CLIs
   (gh, aws, stripe, lark): none expose --dry-run. The mainstream that
   does (kubectl/git/helm/ansible) operates on declarative manifests
   or local state where the preview is materially different from the
   executed action. WeKnora's CLI just echoed the same parameters
   that would have gone on the wire — the preview added no real
   signal over `--help` + reading the call site. Removes:
   - root --dry-run persistent flag + cmdutil/dryrun.go
   - DryRun fields + EmitDryRun calls in 12 write commands
   - format.Envelope.DryRun field
   - 8 corresponding *_test.go cases
   - --dry-run mention from README.md and CHANGELOG.md
   - "dry_run":false from 16 golden envelopes

2. Migrate every --json output to bare data:
   - New format.WriteJSON / WriteJSONFiltered helpers
     (cli/internal/format/bare.go) share filterArrayItems /
     filterObjectKeys / writeJQ with the (still-live for now) envelope
     filter helpers.
   - Read commands (kb/doc/session list+view, search chunks/docs/
     sessions/kb, auth list/status, agent list/view, context list,
     doctor) emit bare arrays / objects on stdout.
   - Write commands (kb create/edit/delete/pin/empty, doc upload/
     upload_recursive/delete, session delete, auth login/logout/
     refresh/token, link/unlink, context add/use/remove, agent
     invoke, chat, api, version) emit bare result objects. Risk
     classification dropped — the resource + exit code already
     convey the action.

Per-command shape changes:
   list / search       → []T   (was {ok, data:{items:[…]}})
   view                → T     (was {ok, data:T, _meta:…})
   create / edit       → T
   delete / pin / etc. → {id, …action result…}
   doctor              → {summary, checks}
   api                 → {status, headers, body}

_meta dropped on the read path:
   pagination (page/page_size/total/has_more) — agents iterate with
   --all-pages or accept --limit (gh CLI parity);
   kb_id / context echo — caller already knows what it asked for.

Acceptance contract goldens regenerated for the new bare shape.
Error envelope on stdout (PrintErrorEnvelope) stays live for now —
the envelope-infra deletion lands in the next commit.
 2026-05-15 12:03:56 +08:00
nullkey
c9b837dfce docs(cli): sync README + AGENTS.md, add cli/CHANGELOG.md, clear stale e2e refs 
v0.3 feature commits didn't update the docs alongside; this commit
syncs them and introduces a CLI-local changelog so v0.3+ release
notes stop crowding the project root file.

cli/CHANGELOG.md (new):
- Subsystem-local pattern, mirroring mcp-server/CHANGELOG.md. CLI
  versions independently from server / frontend cadence; reduces
  merge-conflict surface on the shared root file.
- Scope: Added + SDK additions only. v0.3-internal dev churn
  (--top-k → --limit, kb clear-contents → kb empty, link --context
  introduce-then-drop, internal Go type-name leaks) never reached a
  shipped release so it doesn't belong in Changed / Fixed sections.
  mcp-server's v1.0.0 changelog is Added-only for the same reason.
- v0.0–v0.2 history stays in the project root CHANGELOG.md;
  cross-referenced from the top of cli/CHANGELOG.md.

Stale --help / quickstart examples fixed in cli/cmd/root.go,
cli/README.md, and cli/AGENTS.md — all three showed the dropped bare
`weknora search "<q>" --kb=...` form; updated to `search chunks ...`.

AGENTS.md updates:
- Verb canon table gained edit / empty / download / pin / unpin /
  add / remove.
- `auth` subtree description gained `refresh` and the transparent
  401-retry transport (replacing the now-inverted "deferred to v0.3"
  sentence).
- `search` and `session` subtree paragraphs added; top-level
  verb list gained `context` and `session`.

cli/README.md top-level command list gained `session`; `search`
short retitled to the parent description ("Search across chunks,
knowledge bases, documents, or sessions") since search is now a
pure dispatcher.

Pre-existing stale e2e refs swept up while syncing:
- cli/acceptance/doc.go listed e2e/ under "Future v0.2+:" — moved
  into the present-tense Sub-packages block.
- envelope_test.go preamble "Deferred to v0.2 e2e" rephrased to
  "Deferred to the e2e harness" so it isn't pinned to a past version.

Not changed (out of scope, flagged for future PRs):
- envelope_test.go "Implemented count: 16" vs the actual 14 named
  entries — could be a different counting rule; verify with PR-8
  author before editing.
- envelope_test.go context_use deferred-cases narrative is loose
  (context_use.success IS golden-pinned today) but rewriting needs
  careful re-derivation of which error scenarios are still deferred.
- cli/README.md:50 "once v0.2 ships" — v0.2-PR-original wording;
  not load-bearing once a release tag exists.

No project-root CHANGELOG.md change in this commit.
 2026-05-14 10:57:17 +08:00
nullkey
e236be1ced fix(cli): correct KB id detection, SSE terminal-frame, and CI test isolation 
Three defects surfaced during end-to-end RAG verification — the first two
block real chat usage, the third makes Linux CI flaky:

1. KB id detection — `IsKBID` was checking
   `strings.HasPrefix(s, "kb_")`, but WeKnora generates KB ids as bare
   UUIDs (internal/types/knowledge_base.go: `uuid.New().String()` stored
   in a `varchar(36)` column). Real ids therefore fell through to the
   name-resolution path:

     $ weknora chat ... --kb a32a63ff-fb36-4874-bcaa-30f48570a694
     Error: knowledge base not found: a32a63ff-...

   Switched the discriminator to a UUID regex
   (`^[0-9a-fA-F]{8}-…-[0-9a-fA-F]{12}$`). KB names are arbitrary
   user-supplied strings, so the canonical 8-4-4-4-12 form is an
   unambiguous signal. Mirrors gcloud `--project`'s id-vs-name detection.

2. SSE terminal-frame — the accumulator's `Append` was gating
   finalization on `r.Done`, but the server's KnowledgeQAStream protocol
   emits a leading `agent_query` frame with `done=true` to deliver
   session + message metadata *before* the answer fragments arrive:

     event: message
     data: {"response_type":"agent_query","content":"","done":true,…}

     event: message
     data: {"response_type":"answer","content":"你好","done":false}
     …
     event: message
     data: {"response_type":"complete","content":"","done":true}

   The accumulator therefore flipped to `finished=true` on frame #1 and
   discarded every subsequent answer fragment — `weknora chat … --json`
   returned `answer: ""` even though the LLM reported completion_tokens
   > 0. Fixed: terminate only on `response_type == complete`.
   References still captured opportunistically (they may arrive on a
   dedicated `references` event before the terminator).

3. doctor credential_storage CI isolation — the check probes the real
   OS keyring via `secrets.NewBestEffortStore()`: present on macOS dev
   machines → StatusOK; absent on Linux CI runners without libsecret /
   Gnome-Keyring → StatusWarn ("falling back to file store"). That
   host-dependence was leaking into two test classes that assumed
   StatusOK:

     * cmd/doctor/doctor_test.go: TestDoctor_AllOK and
       TestDoctor_NoConfig_StillRunsCredentialStorage already had a
       withCredStoreFactory seam but didn't use it. Added the pin.

     * acceptance/contract/envelope_test.go: doctor.success_offline
       and doctor.error_network golden cases. The contract test runs
       through the cobra tree in-process and shares cmd/doctor's
       package-level credStoreFactory var — but couldn't reach it
       because the existing seam was unexported.

   Fix: export `doctor.SetCredStoreFactoryForTest(fn) (restore func())`
   for out-of-package tests; acceptance/contract/helpers_test.go adds
   a TestMain that pins the factory to a MemStore-returning closure
   for the whole suite (MemStore is neither *FileStore nor a real
   keyring, so doctor's type-switch hits StatusOK). Production stays
   at secrets.NewBestEffortStore — only the test hook is now reachable
   from across packages.

Test fixtures and goldens that used the old `kb_xxx` literals or
`Done: true` terminators were rewritten to use real UUIDs and
`ResponseType: ResponseTypeComplete` respectively. Per-command --help
text and Long descriptions / Examples now show a UUID rather than
`kb_…` so users see the correct shape from the start. New
TestAccumulator_IgnoresAgentQueryDone pins the SSE terminator bug so
it can't regress.

Tests: 24 cli packages green on macOS dev + Linux/macOS/Windows CI
matrix. Verified end-to-end against a live WeKnora server: `weknora
chat "..." --kb <UUID> --no-stream --json` returns the full LLM answer
in the envelope, live token streaming in TTY mode works, and the
credential_storage check renders deterministic envelopes across hosts.
 2026-05-12 13:20:42 +08:00
nullkey
bdbd15bf75 docs(cli): add CLI README, top-level mention, CHANGELOG, ADR section 
Discoverability gaps surfaced by the pre-PR review:

- New cli/README.md: install (build-from-source / pre-built once shipped)
  + 5-minute quickstart (auth login → kb list → link → doc upload →
  chat) + multi-context walkthrough + JSON envelope shape + agent /
  scripting integration overview + dev workflow. Points readers at
  cli/AGENTS.md for the full operational contract.

- Top-level README.md: new "⌨️ Command-Line Interface" section between
  Key Features and Getting Started, with a one-paragraph pitch + four
  representative commands and links to cli/README.md and cli/AGENTS.md.
  English README only this round; CN / JA / KO translations to follow
  in v0.3 to match the existing four-language pattern.

- CHANGELOG.md [Unreleased] gets a "weknora CLI v0.2" bullet listing
  the headline capabilities (10-command surface, project-link,
  envelope, agent affordance, multi-context auth, doctor) and pointing
  at cli/README.md.

- cli/AGENTS.md gains an "Architecture decisions" section documenting
  ADR-3 (gh as primary mainstream north star + the four documented
  deviations: link, chat/search, context use, doctor) and ADR-4
  (Factory closures + narrow Service interfaces). The in-source
  references (`(v0.2 ADR-3)`, `(per ADR-4)`) now point at committed
  prose rather than dangling.
 2026-05-12 13:20:42 +08:00