To run simply run the ohsome quality API and the database the provided Docker setup can be used.
docker compose up --detachAfter all services are up they are available under:
- API: http://127.0.0.1:8080/
- Database:
host=localhost port=5445 dbname=oqapi user=oqapi password=oqapi
For development setup please continue reading.
- Python: ≥ 3.10
- Poetry: ≥ 1.5
- R: ≥ 4.0
This project uses Poetry for packaging and dependencies management. Please make sure it is installed on your system.
For development a database and raster datasets on disk might not be needed. In case the database is needed start the database service defined in the docker compose file. If raster datasets are needed please refer to /docs/raster_datasets.md for setting those up.
poetry install
poetry shell # Spawns a shell within the virtual environment.
pre-commit install # Install pre-commit hooks.
# Hack away
pytest # Run all testsFor all possible configuration parameter please refer to the configuration documentation.
For local development no custom configuration is required.
python scripts/start_api.pyGo to http://127.0.0.1:8080/docs and check out the endpoints.
Default host is 127.0.0.1 and port is 8080. To change this, provide the corresponding parameter:
$ cd scripts
$ python start_api.py --help
Usage: start_api.py [OPTIONS]
Options:
--host TEXT [default: 127.0.0.1]
--port INTEGER [default: 8080]
--help Show this message and exit.All relevant components should be tested. Please write tests for newly integrated functionality.
The test framework is pytest.
To run all tests just execute pytest:
pytestAll tests that are calling function, which are dependent on external resources (e.g. ohsome API) have to use the VCR.py module: "VCR.py records all HTTP interactions that take place […]." This ensures that the positive test result is not dependent on the external resource. The cassettes are stored in the test directory within fixtures/vcr_cassettes. These cassettes are supposed to be integrated (committed and pushed) to the repository.
The VCR record mode is configurable through the environment variable VCR_RECORD_MODE. To ensure that every request is covered by cassettes, run the tests with the record mode none. If necessary, the cassettes can be re-recorded by deleting the cassettes and run all tests again, or using the record mode all. This is not necessary in normal cases, because not-yet-stored requests are downloaded automatically.
Writing tests using VCR.py with our custom decorator is as easy as:
from tests import oqapi_vcr
@oqapi_vcr.use_cassette
def test_something(self):
oqapi.do_something(…)Good examples can be found in test_oqt.py.
When writing tests for functions which are asynchronous (using the async/await pattern) such as the preprocess functions of indicator classes, those functions should be called as follows: asyncio.run(indicator.preprocess()).
Logging is enabled by default.
ohsome_quality_api uses the logging module.
The logging module is configured in config.py. Both entry-points to
ohsome_quality_api, the api.py, will call the configuration
function defined in definitions.py. The default log level is INFO. This can be
overwritten by setting the environment variable OQAPI_LOG_LEVEL (See also the
configuration documentation).
import logging
logging.info("Logging message")asyncpg is used as database interface library.
In Psycopg one can execute a query for each element of a list with
executemany(insert_query, vars_list).
To achieve similar results in asyncpg, one would write an SQL query which takes an array
as input:
await conn.fetch('''
INSERT INTO people (name) (SELECT unnest ($1))
RETURNING id
''', ['anne', 'ben', 'charlie'])The package rpy2 is utilized to execute R code.
rpy2is an interface to R running embedded in a Python process.
For an example how the ohsome quality API is using rpy2 have a look the module models.py.
Through this module the Mapping Saturation indicator uses some of the built-in statistical models of R.