General agreement to go ahead, and once datapusher+ is released let's link to it from the docs and consider switching the docker images again

Hi All,

@wardi thanks for the warm welcome.

I've gone looking, but I've not been able to find if the ckan system has the ability to start the job workers. I also don't see any items that runs the email notify sub system.

Since xloader ulitlizes the job's system that ckan core has, having the job queue/s started on the ckan container should be all you need. Plus having the xloader extension available and on the ckan.ini config items.

In the CKAN data.qld.gov.au deployment, we have 3 queues as we have the 'archiver' extension installed.
So that is 'default', 'priority' and 'bulk'.

Just spitballing here, how does this sound:
The base image should have config options to start at least default job worker but also any extra queue listeners in container config options.

As for the plugin being included in the base image, since its not inlined with a release of ckan, it should be loading a version either direct from github or pypi via default container options or overrides to get direct on extending the base image.

on how to achive that please see: https://github.com/qld-gov-au/opswx-ckan-cookbook/blob/934611c7dd8833f18c5bf2d1de7634a5ddd7238e/resources/pip_install_app.rb and https://github.com/qld-gov-au/opswx-ckan-cookbook/blob/934611c7dd8833f18c5bf2d1de7634a5ddd7238e/recipes/ckanweb-deploy-exts.rb#L147 which is what we use to load plugins via chef/opsworks (And yes we are about to migrate off opsworks since its gone after May 26, 2024 )