thread_pool.h#

namespace onnx_light

Alias that makes onnx-light headers compatible with code that references ONNX_LIGHT_NAMESPACE (the macro used in the standard onnx package).

Set to ONNX_LIGHT_NAMESPACE so both names resolve to the same namespace.

Symbol-visibility attribute for the public onnx-light C++ API.

Maps the upstream compatibility macro to onnx-light’s explicit proto ABI annotation. This keeps declarations from vendored ONNX headers visible when lib_onnx_proto uses hidden visibility by default.

Namespace alias so that ONNX C++ code (and consumers such as onnxruntime) that refers to the literal onnx namespace — rather than the ONNX_NAMESPACE macro — resolves to the onnx-light namespace. The standard onnx package lives in namespace onnx; onnx-light uses onnx_light (via ONNX_LIGHT_NAMESPACE), so this alias keeps onnx-light a true drop-in. It is only introduced when the onnx-light namespace differs from onnx.

namespace utils

class ThreadPool#

#include <thread_pool.h>

Thread pool used to parallelize delayed block loading.

Manages a fixed set of worker threads that pull jobs from a shared queue. Call Start() to launch workers, SubmitTask() to enqueue work, and Wait() to block until all submitted jobs have completed and the workers have stopped. The pool can be restarted by calling Start() again after Wait() returns. Clear() resets internal state when the pool is idle (not started).

Public Functions

ThreadPool()#: Initializes the thread pool in a stopped, idle state.

~ThreadPool()#

Destroys the thread pool by calling Wait().

Blocks until all pending jobs have finished executing, then stops and joins all worker threads. If the pool has already been stopped, this is a no-op.

void Start(int32_t num_threads)#

Starts the pool, deferring worker-thread creation until the first task.

The pool is marked started immediately, but the worker threads are only spawned lazily on the first SubmitTask() call. A parse/serialize that never submits a delayed block (typically a small model whose tensor blocks all stay below the parallelization threshold) therefore never pays the cost of creating and joining threads, keeping load/save latency minimal.

Parameters:: num_threads – Number of worker threads to create. Any negative value (for example -1) is treated as a request to use the value returned by std::thread::hardware_concurrency().

void SubmitTask(std::function<void()> &&job)#

Submits a callable job for asynchronous execution.

If no worker threads have been started, the job is queued and will be run inline when Wait() is called.

Parameters:: job – Callable to execute.

void SubmitTask(const std::function<void()> &job)#

void Wait()#

Blocks until all submitted jobs have finished executing, then stops and joins all worker threads.

If no workers were started, runs any queued jobs inline on the calling thread. If a submitted job throws, Wait() rethrows the first captured exception on the caller thread after all workers have been joined. After Wait() returns, the pool is in a stopped state and can be restarted with Start().

inline size_t GetThreadCount() const#: Returns the number of worker threads the pool has (or will lazily spawn).

inline bool IsStarted() const#: Returns whether the pool has been started (workers may start lazily).

void Clear()#

Resets the pool to an empty, idle state.

Clears the worker list and any pending jobs. Must only be called when the pool is not started (i.e., after Wait() has returned or before Start() has been called).

Private Functions

void WaitImpl(bool rethrow_exceptions)#

Private Members

WorkerPool workers_#

bool is_started_#

int32_t requested_threads_#