mirror of
https://github.com/huggingface/xet-core.git
synced 2026-06-04 13:30:29 +08:00
b43c0aec0e
This PR moves the XetRuntime model away from using thread-local statics
and decouples the XetConfig and XetCommon structs from a single runtime.
It introduces a struct XetContext that gives the runtime context for
operations:
```
struct XetContext {
pub runtime : Arc<XetRuntime>, // The current tokio runtime wrapper, minus the config and common objects..
pub common : Arc<XetCommon>, // The common cache objects, semaphores, rate trackers, etc.
pub config : Arc<XetConfig> // The config
}
```
Now, instead of using functions like `xet_runtime()` and `xet_config()` that examine the thread-local storage, we now explicitly passing through a XetContext instance from the session creation that gets stored in each major processing struct.
This allows decoupling between the runtime, config, and common caches, especially:
- Running multiple config settings and/or endpoints within the same pre-existing tokio runtime.
- Running multiple runtimes that share the same XetCommon object.