How to always use dataset UUID for name/URL?

[themowski]

(tl,dr; see summary at the bottom)

My team has a CKAN 2.10.1 instance that uses ckanext-scheming to manage dataset & organization schemas. The name field for datasets is configured the same way as in the example schema from ckanext-scheming, namely:

- field_name: name
  label: URL
  preset: dataset_slug
  form_placeholder: e.g., my-dataset

The name field is a special field that is used to construct a dataset's URL (and possibly for other things?). CKAN made the design decision to allow users to specify name during dataset creation, with the default value being based on the dataset's title. For our purposes, we would prefer that each dataset's name/URL is the same as its UUID (package ID), which we feel is much more robust from a "link sharing" perspective, and reduces the burden on users of coming up with unique dataset names/URL's.

We are aware that if you try to view a dataset by passing in the UUID instead of name, CKAN will "get it right" and redirect you to the correct dataset, but in our ideal scenario, the name and the UUID would always align.

Using the UUID for the URL instead of a user-entered name value was suggested over 10 years ago (!!) in issue #1233, and the resolution was that UUID's would result in ugly URL's, so the idea was rejected. While looking for other issues that relate to this, I found (declined) PR #7425, which would automatically update the name field if a collision was detected. There was some discussion on the PR that suggested a plugin could do this, so I have started looking into whether it is possible to use a plugin to get our desired behavior.

After looking at the source of ckan.logic.action.create.package_create(), I've concluded that the UUID is generated at the end of the dataset creation process, immediately before (or maybe as part of?) inserting the dataset into the CKAN DB. This means that the UUID is not available at the time the user is filling out the form, and we cannot override the name at any point prior to the dataset actually being created.

I think it might be feasible to implement the IPackageController.after_dataset_create() function in a custom plugin, since the input data will have the package ID when that function is called. However, I am not sure what we would have to do after setting data['name'] = data['id']. I believe we would need to re-commit the new data to the CKAN DB, but I'm not familiar with interacting with the database, so help with that would be appreciated. Or, maybe a better approach is to call a patch action with the updated dataset?

Any insight or pointers about this problem & our proposed approach would be greatly appreciated.

Summary:

• We want every dataset's URL to use the dataset's UUID (package ID).
• We think that this means we need to force dataset name to be the package ID during the dataset create process
  • Is this correct?
• It looks like IPackageController.after_dataset_create() is the hook we need to implement to do this, since UUID is generated after dataset validation.
  • Is this correct?
• Is there documentation about how to interact with the CKAN DB?
• Any other ideas for how this could be implemented, or is this the right approach?

[wardi]

This is how we set the name to be the same as the id of our datasets:

https://github.com/open-data/ckanext-canada/blob/69b04b8eaf2750b3e95239783d2768248240e10e/ckanext/canada/schemas/presets.yaml#L29-L34

form_snippet: null hides the name field on the form and if_empty_same_as(id) copies the id value over.

There are some cases where we had an old UUID that we wanted to map to a new id so we allow name to be set to a different value so that our site can accept both, but it still needs to be a UUID because of canada_validate_generate_uuid defined here:

https://github.com/open-data/ckanext-canada/blob/69b04b8eaf2750b3e95239783d2768248240e10e/ckanext/canada/validators.py#L144-L154

Looking at now this there are some old-ckan-isms we need to clean up like the use of unicode as a validator, but the general idea still works.

[themowski]

@wardi -- thanks so much for the quick response, this is exactly what we need! I tried this approach out, and it looks like it's working smoothly. Definitely a lot easier than what I was worried we'd have to do.

For anyone else who reads this: I was correct in my original observation that CKAN generates a UUID automatically after validation... but I didn't realize that that's only done if your dataset does not already have an id field. So, in addition to setting up the name field, you also need to define an id field in your schema, and generate the UUID into that field automatically. The canada_dataset_id preset in the presets.yaml file that Ian linked shows how to set up that field, using the same validator function.