TPool package

Submodules

TPool.wildpool module

class TPool.wildpool.WildPool(pool_size=5, logger=None)

Bases: object

A flexible thread pool for managing concurrent tasks.

The WildPool class provides a thread pool that efficiently manages the execution of multiple threads concurrently. Unlike a sequential thread pool, WildPool does not guarantee the order of execution of tasks, making it particularly suited for scenarios where tasks are independent and can be processed in parallel without needing to adhere to a specific sequence.

This thread pool is designed for high-concurrency environments such as messaging queues and daemons, where managing multiple background tasks simultaneously is crucial. By limiting the number of concurrent threads to a specified maximum (pool_size), WildPool ensures that system resources are used efficiently, preventing issues like resource exhaustion or system slowdown.

Key features of WildPool include:

• Concurrency Management: Manages a fixed number of threads, ensuring no more than the specified number of tasks run concurrently, which helps in maintaining optimal system performance.

• Non-Sequential Execution: Tasks are executed as soon as resources are available, without guaranteeing any particular order, which is ideal for tasks that do not depend on each other.

• Automatic Cleanup: Completed threads are automatically removed from the pool, freeing up resources for new tasks.

• Daemon-Friendly: The pool is well-suited for daemon processes and long-running services where tasks need to be processed continuously in the background.

Parameters:

• pool_size (int) – The maximum number of threads in the pool.

• logger (logging.Logger, optional) – Optional custom logger. If not provided, a default logger will be used.

add_thread(thread: Thread)

Add a thread to the pool to be managed and executed.

Parameters:

thread (threading.Thread) – The thread to be added to the pool.

join()

Block until all queued tasks have been processed and the worker thread exits.

This method initiates a graceful shutdown of the thread pool. It waits for all tasks currently in the queue (bench) to be consumed and processed. Once the queue is empty, the worker thread will terminate, and this method will return.

If join() has already been called, subsequent calls will be ignored to prevent unintended side effects or redundant shutdown attempts.

Internally, a None sentinel is enqueued to unblock the worker if it is waiting for tasks, allowing it to check for shutdown conditions and exit cleanly.

Note

• join() and stop() are mutually exclusive; calling both may lead to undefined behavior.

• After calling join(), no new tasks should be added to the pool.

Usage Example:

pool = WildPool(pool_size=4) for task in tasks:

pool.add_thread(threading.Thread(target=task))

pool.start_worker() pool.join() # Waits until all tasks are done

start_worker()

Start a worker thread if none is currently running.

This method ensures that only one worker thread is running to manage the threads in the pool. If a worker thread is already running, it will not start a new one.

stop()

Stop the worker thread and release resources.

This method gracefully shuts down the worker thread, ensuring that all running threads are completed and resources are cleaned up.

Module contents