mirror of
https://github.com/huggingface/xet-core.git
synced 2026-06-04 13:30:29 +08:00
c90f0a7bd9
Previously, upload and download paths each had their own ad-hoc state
tracking, cancellation, and runtime bridging logic. TaskRuntime
consolidates this into a single type that owns a CancellationToken tree,
tracks Running/Finished/Cancelled state with recursive propagation to
children, and provides bridge_async/bridge_sync wrappers that
automatically wire up tokio::select! cancellation. Session →
commit/group → per-file handles form a parent-child token tree, so
aborting a session cancels all descendant work.
The upload path gets new UploadFileHandle and UploadStreamHandle wrapper
types (replacing the old UploadTaskHandle), with inner/wrapper pattern
for cheap cloning. UploadCommit::commit() now returns a CommitReport
containing aggregate dedup metrics, progress, and per-file FileMetadata.
The download path mirrors this structure: FileDownloadGroup uses
TaskRuntime for state gating and owns bespoke DownloadTaskHandle
instances with per-task status and result access.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **High Risk**
> High risk due to a breaking redesign of the public `xet_session` API
(new handle/report types and renamed methods) plus new
cancellation/state machinery that changes how uploads/downloads are
coordinated and terminated.
>
> **Overview**
> **Redesigns `xet_pkg::xet_session` around a new hierarchical
`TaskRuntime`** (using `tokio-util` cancellation tokens) to unify state,
bridging, and cancellation across session → commit/group → per-file
handles.
>
> **Replaces the old task-handle/result model** (`tasks.rs`,
`UploadResult`/`DownloadResult`, `TaskStatus`, group/session state
enums) with explicit handle/report types: `XetFileUpload`,
`XetStreamUpload`, `XetFileDownload`, `XetCommitReport`, and
`XetDownloadGroupReport`, and standardizes task state via
`XetTaskState`.
>
> **Adjusts APIs and error semantics**: `commit()` now returns an
aggregate report (dedup metrics + progress + per-file metadata) and no
longer consumes `self`; progress methods become infallible
(`progress()`); cancellations/errors are consolidated
(`AlreadyCompleted`, `UserCancelled`, `KeyboardInterrupt`,
`TaskError`/`PreviousTaskError`) with updated Python exception mapping.
`xet_data` now returns per-file `DeduplicationMetrics` from upload tasks
and adds a zero-copy `SingleFileCleaner::add_data_from_bytes`.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
153a3ebbbe. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
API changes
This folder contains a record of API changes in main. It's indended for AI agents to read in order to correctly apply merges or update dependencies and PRs.
The updates are listed by date in the form: update_<yymmdd>_<description>.md
When applying a merge, rebase, or downstream update, all AI agents should first scan this folder to understand what relevant information may need to be applied.
When creating a PR that involves an API change potentially requiring downstream updates, an AI agent should create such a file. This file should be humanly readable but contain enough information to correctly apply the needed changes without scanning the code.