01_setup_datapusher.sh is broken

[tB0nE]

The start up script (ckan-docker/ckan/docker-entrypoint.d) for datapusher is broken. It is checking whether the API Token is null, rather than the reverse.

This:
if [ -z "$CKAN__DATAPUSHER__API_TOKEN" ] ; then

Should be this:
if [ -n "$CKAN__DATAPUSHER__API_TOKEN" ] ; then

[tB0nE]

So to add on to this, it seems I misunderstood the purpose of this script. The idea is that once you enable the DataPusher plugin (without setting an API Token), you start it up once, which generates the token - you then add that to the .env file. So my complaint I guess would be to improve the documentation so this is more clear, as I was under the impression you should set the API token in the .env, and then when the container was created it would set the token to that value.

[tB0nE]

Last bit on this, once I finally got it to work. So I think the recent change in ckan to make the DataPusher api key a requirement breaks this in some way.

Let me try break it down:

• ckan requires an api key to be set in order to run with DataPusher
• this script creates an api key, but only if the api key is NOT set
• the script does not output the api key once created, so its hard to retrieve
• the script does not set the api key in the clan.ini file like it is suppose to (I think it's being overridden at some point to "xxx"

I am not sure what the best way is to correct this. What I decided to do was set a value to CKAN__DATAPUSHER__API_TOKEN (e.g. "UNSET"), disable the script, and then when ckan is running, execute "ckan -c ckan.ini user token add ckan_admin datapusher | tail -n 1 | tr -d '\t'" - then copy the API key, and then set CKAN__DATAPUSHER__API_TOKEN to this API key

I am also using DataPusher Plus, so I needed to set the api key in that env file too.

Since ckan needs to be running to create the api key, and then you need to set the api in the .env outside the environment, I am not sure how one can automate it.

[kowh-ai]

Are you able to post the specific error message(s) you are getting?

[tB0nE]

Are you able to post the specific error message(s) you are getting?

It's not specifically an error, the error that the DataPusher was throwing was connection refused - because the api key was unset or invalid.

The issue is that the api key wasn't being set or created when ckan was being initiated. The script ckan-docker/ckan/docker-entrypoint.d/01_setup_datapusher.sh requires ckan to be running, but ckan (from version 2.10 up) requires an api_key when you use DataPusher in order to start up. So basically the script cannot run to create the api key, because you need an api key to start ckan.

It's quite a logical issue! As I mentioned in one of my other posts, my solution was to disable the script, set a dummy api key. This allows ckan to start up. Then manually run the api key creation command, copy the api key, and then set ckan and DataPusher plus to use the api key. I think automating this might be an issue, and might just need to be part of installation notes.

[DBATCBS]

@kowh-ai @tB0nE This should be in the readme file. I -and many others- experienced the exact same thing. Thanks to @tB0nE I know now where to look for.

@tB0nE Do you have to restart CKAN after you configured the API token in the ckan.ini? And do you know if you stop the container from running and start again, will it still use the same API token?

@kowh-ai Do you expect people setting up CKAN for the first time to first install and configure CKAN-base. Because I ended up using CKAN-docker instead of base and that is where my research ends, since Datapusher needs an API token to be set which you have to generate in CKAN main site. But CKAN main site will not succesfully boot up if a Datapusher API token has not been set.

[tB0nE]

@DBATCBS I set CKAN__DATAPUSHER__API_TOKEN to something, and disable the 01_setup_datapusher.sh script. Start up ckan, and create my API key, then set CKAN__DATAPUSHER__API_TOKEN to the API key. This involves change the .env, so you will need to rebuild I think. It doesn't change the db, so it retains the API Token on ckan.

[kowh-ai]

I haven’t been able to recreate any error/problem you are facing - maybe I’m missing something but here is my understanding:

As long as the $CKAN__DATAPUSHER__API_TOKEN environment variable is not set

• In the ckan-docker-base repo the start_ckan.sh script creates a ckan.datapusher.api_token=xxx line in the CKAN config file (ckan.ini)
• In the ckan-docker repo the 01_setup_datapusher.sh script creates a real token in the CKAN config file using the 'ckan -c $CKAN_INI user token add ckan_admin datapusher’ command. This replaces the ckan.datapusher.api_token=xxx line

At that point this token can now be referenced from the CKAN config file if required