Skip to content

cdluminate/tasque

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

70 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Tasque -- Zero-Config Single-Node Workload Manager (WORK IN PROGRESS)

Latest Version

Tasque (Task Queue) is a simple workload manager for single-node usage that can be used out-of-the-box. It is resource-aware (e.g. CPU, Memory, GPU, Video Memory), and can automatically schedule the submitted jobs in a sensible way. It is much more light-weight compared to cluster workload managers such as Slurm and PBS, while being much smarter than a casually rushed script using tmux, screen, or alike. Tasque is written by the author for scheduling batches of machine learning experiments (e.g. caffe, pytorch, tensorflow).

Tasque has the following characteristics:

  1. Submitted jobs (command lines) will be automatically scheduled to run when there is enough resource to do so.
  2. The daemon is resource-aware, namely it is able to plan the usage of either CPU, Memory, GPU, or Video memory. When resource is aboundant, tasks may be scheduled to run in parallel.
  3. The default behavior is to execute the given commands in the FIFO order.
  4. Tasks with high priority values will be scheduled to run prior to the rest.
  5. The queue is stored in an SQLite3 database, and will not be lost in case of powerloss.
  6. Users can assign text annotations with the tasks in database.
  7. Requires no configuration and can be used out-of-the-box.

Example

Installation

This tools is available on Pypi. Just issue the following command:

pip3 install tq1

Note that some new language features may be used in the code. There is no plan to support older versions of python3. Hence python3 >= 3.8 is recommended.

In case you want to install it from source:

python3 setup.py install

Usage

Copyright and License

Copyright (C) 2016-2021 Mo Zhou <lumin@debian.org>
Released under the MIT/Expat License.