tpv-metascheduler-api

Metascheduler for TPV as Service

1. Create a venv

   python -m venv .venv
   source .venv/bin/activate
   pip install -r requirements.txt

2. Run the API locally: uvicorn main:app --reload

3. Testing the API with input data:

   1. SwaggerUI: http://127.0.0.1:8000/docs Try out > Fill out the request body: Request body:

      {
         "destinations": [
            {
               "id": "pulsar_italy",
               "abstract": false,
               "runner": "general_pulsar_1",
               "destination_name_override": "pulsar_italy",
               "cores": null,
               "mem": null,
               "gpus": null,
               "min_cores": null,
               "min_mem": null,
               "min_gpus": null,
               "max_cores": null,
               "max_mem": null,
               "max_gpus": null,
               "min_accepted_cores": null,
               "min_accepted_mem": null,
               "min_accepted_gpus": null,
               "max_accepted_cores": 8,
               "max_accepted_mem": 32,
               "max_accepted_gpus": null,
               "env": null,
               "params": null,
               "resubmit": null,
               "scheduling": {
               "require": ["pulsar"],
               "prefer": [],
               "accept": ["general"],
               "reject": []
               },
               "inherits": null,
               "context": {
                  "latitude": 50.0689816,
                  "longitude": 19.9070188 },
               "rules": {},
               "tags": null,
               "queued_job_count": 10
            },
            {
               "id": "slurm_poland",
               "abstract": false,
               "runner": "slurm",
               "destination_name_override": "slurm_poland",
               "cores": null,
               "mem": null,
               "gpus": null,
               "min_cores": null,
               "min_mem": null,
               "min_gpus": null,
               "max_cores": null,
               "max_mem": null,
               "max_gpus": null,
               "min_accepted_cores": null,
               "min_accepted_mem": null,
               "min_accepted_gpus": null,
               "max_accepted_cores": 16,
               "max_accepted_mem": 64,
               "max_accepted_gpus": null,
               "env": null,
               "params": null,
               "resubmit": null,
               "scheduling": {
               "require": [],
               "prefer": [],
               "accept": ["slurm"],
               "reject": []
               },
               "inherits": null,
               "context": {
                  "latitude": 51.9189046,
                  "longitude": 19.1343786 },
               "rules": {},
               "tags": null,
               "queued_job_count": 8
            }
         ],
         "objectstores": {
            "object_store_italy_S3_01": {
               "latitude": 50.0689816,
               "longitude": 19.9070188,
               "other_stuff_that_we_find_useful": "foobar"
            },
            "object_store_poland": {
               "latitude": 51.9189046,
               "longitude": 19.1343786,
               "other_stuff_that_we_find_useful": "foobar"
            }
         },
         "dataset_attributes": {
            "dataset_italy": {
               "object_store_id": "object_store_italy_S3_01",
               "size": 12345678
            },
            "dataset_poland": {
               "object_store_id": "object_store_poland",
               "size": 123456789
            }
         }
      }

   2. curl

      The Swagger UI can give you a curl version of your request after executing

   3. Using a python script with the requests or httpx library

      There is an example of how to do this with TPV: tpv-metascheduler-api/example_tpv_config_locations_api.yml This config can be set up with a galaxy instance or by cloning the TPV repo (the example currently depends on an open PR to this repo) test/test_scenario_esg_group_user_api contains a test that has a similar request as above to test out:

      # Clone the remote repository
      git clone https://github.com/pauldg/total-perspective-vortex.git
      # Change into the cloned repository directory
      cd total-perspective-vortex
      # Checkout the desired remote branch
      git checkout -b location_test origin/location_test
      # Create a venv for testing
      python -m venv .venv
      source .venv/bin/activate
      pip install -r requirements_test.txt
      # Run pytest for the api
      pytest -rPv  tests/test_scenarios_locations.py::TestScenarios::test_scenario_esg_group_user_api