/ ckanext-blob-storage Registry Initialized Error

[zesdatascience]

Hi there,

Wondering if anyone could please point me in the write direction. I am installing CKAN for the first time and really keen to link up to Azure storage. I have got CKAN up and running and now trying to get the blob_storage plugin working. My environment is as follows:

Ubuntu 20.04
Python 3.8
CKAN installed from package (https://docs.ckan.org/en/2.9/maintaining/installing/install-from-package.html)
ckan 2.9.3 /usr/lib/ckan/default/src/ckan
ckanext-authz-service 0.2.0 /usr/lib/ckan/default/src/ckanext-authz-service
ckanext-blob-storage 0.8.2 /usr/lib/ckan/default/src/ckanext-blob-storage

It's all configured and no issues on restarting ckan-uwsgi service. However when I go to the "add data" page I get the errors shown below. Just wondering if anyone can please provide some pointers?

Also not sure if this is because of steps required to add resource fields (https://github.com/datopian/ckanext-blob-storage#required-resource-fields). I wasn't sure of the steps required here? Could anyone please provide pointers on this?

Note, I have successfully test the giftless set up and integration to Azure storage. Just trying to connect ckan through to this using the plugin.

Thanks in advance for any help,

Stu

Errors:

2021-08-30 00:52:06,117 ERROR [ckan.config.middleware.flask_app] Registry initialized.
Traceback (most recent call last):
File "/usr/lib/ckan/default/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib/ckan/default/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "/usr/lib/ckan/default/lib/python3.8/site-packages/flask/views.py", line 89, in view
return self.dispatch_request(*args, **kwargs)
File "/usr/lib/ckan/default/lib/python3.8/site-packages/flask/views.py", line 163, in dispatch_request
return meth(*args, **kwargs)
File "/usr/lib/ckan/default/src/ckan/ckan/config/middleware/../../views/resource.py", line 333, in get
return base.render(template, extra_vars)
File "/usr/lib/ckan/default/src/ckan/ckan/lib/base.py", line 151, in render
return flask_render_template(template_name, **extra_vars)
File "/usr/lib/ckan/default/lib/python3.8/site-packages/flask/templating.py", line 137, in render_template
return _render(
File "/usr/lib/ckan/default/lib/python3.8/site-packages/flask/templating.py", line 120, in _render
rv = template.render(context)
File "/usr/lib/ckan/default/lib/python3.8/site-packages/jinja2/environment.py", line 1090, in render
self.environment.handle_exception()
File "/usr/lib/ckan/default/lib/python3.8/site-packages/jinja2/environment.py", line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "/usr/lib/ckan/default/lib/python3.8/site-packages/jinja2/_compat.py", line 28, in reraise
raise value.with_traceback(tb)
File "/usr/lib/ckan/default/src/ckanext-blob-storage/ckanext/blob_storage/templates/package/new_resource.html", line
3, in top-level template code
{% set package_id = pkg_dict.id %}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/package/new_resource.html", line 3, in top-level template code
{% set logged_in = true if c.userobj else false %}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/package/base_form_page.html", line 1, in top-level template code
{% extends "package/edit_base.html" %}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/package/edit_base.html", line 1, in top-level template code
{% extends 'package/base.html' %}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/package/base.html", line 4, in top-level template code
{% set dataset_type = dataset_type or pkg.type or 'dataset' %}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/page.html", line 1, in top-level template code
{% extends "base.html" %}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/base.html", line 106, in top-level template code
{%- block page %}{% endblock -%}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/page.html", line 19, in block "page"
{%- block content %}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/page.html", line 22, in block "content"
{% block main_content %}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/page.html", line 74, in block "main_content"
{% block primary %}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/page.html", line 87, in block "primary"
{% block primary_content %}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/package/base_form_page.html", line 7, in block "primary_content"
{% block primary_content_inner %}
File "/usr/lib/ckan/default/src/ckan/ckan/templates/package/base_form_page.html", line 8, in block "primary_content_inner"
{% block form %}
File "/usr/lib/ckan/default/src/ckanext-blob-storage/ckanext/blob_storage/templates/package/new_resource.html", line
6, in block "form"
{% snippet 'blob_storage/snippets/upload_module.html', pkg_id=package_id, api_key=c.userobj.apikey, base_url=g.site_url, parent=super %}
File "/usr/lib/ckan/default/src/ckan/ckan/lib/jinja_extensions.py", line 274, in _call
return base.render_snippet(*args, **kwargs)
File "/usr/lib/ckan/default/src/ckan/ckan/lib/base.py", line 95, in render_snippet
output = render(template_name, extra_vars=kw)
File "/usr/lib/ckan/default/src/ckan/ckan/lib/base.py", line 151, in render
return flask_render_template(template_name, **extra_vars)
File "/usr/lib/ckan/default/lib/python3.8/site-packages/flask/templating.py", line 137, in render_template
return _render(
File "/usr/lib/ckan/default/lib/python3.8/site-packages/flask/templating.py", line 120, in _render
rv = template.render(context)
File "/usr/lib/ckan/default/lib/python3.8/site-packages/jinja2/environment.py", line 1090, in render
self.environment.handle_exception()
File "/usr/lib/ckan/default/lib/python3.8/site-packages/jinja2/environment.py", line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "/usr/lib/ckan/default/lib/python3.8/site-packages/jinja2/_compat.py", line 28, in reraise
raise value.with_traceback(tb)
File "/usr/lib/ckan/default/src/ckanext-blob-storage/ckanext/blob_storage/templates/blob_storage/snippets/upload_module.html", line 1, in top-level template code
{% resource 'blob-storage/css/main.3c074c8b.chunk.css' %}
File "/usr/lib/ckan/default/src/ckan/ckan/lib/jinja_extensions.py", line 334, in _call
h.include_resource(args[0], **kwargs)
File "/usr/lib/ckan/default/src/ckan/ckan/lib/helpers.py", line 2085, in include_resource
import ckan.lib.fanstatic_resources as fanstatic_resources
File "/usr/lib/ckan/default/src/ckan/ckan/lib/fanstatic_resources.py", line 252, in
create_library('vendor', os.path.join(base_path, 'vendor'), depend_base=False)
File "/usr/lib/ckan/default/src/ckan/ckan/lib/fanstatic_resources.py", line 243, in create_library
registry.add(library)
File "/usr/lib/ckan/default/lib/python3.8/site-packages/fanstatic/registry.py", line 18, in add
self[item.name] = item
File "/usr/lib/ckan/default/lib/python3.8/site-packages/fanstatic/registry.py", line 72, in setitem
raise ValueError('Registry initialized.')
ValueError: Registry initialized.

[zesdatascience]

Additional note: just testing with filestore configured to a mount using Blobfuse (https://docs.microsoft.com/en-us/azure/storage/blobs/storage-how-to-mount-container-linux). Might be a good workaround.

[amercader]

@zesdatascience Not sure what the actual issue is but this is the symptom:

File "/usr/lib/ckan/default/src/ckanext-blob-storage/ckanext/blob_storage/templates/blob_storage/snippets/upload_module.html", line 1, in top-level template code
{% resource 'blob-storage/css/main.3c074c8b.chunk.css' %}

{% resource %} is a CKAN<=2.8 only tag only. Its CKAN>= 2.9 equivalent is {% asset %} . For multi CKAN version extensions like this one there is generally some logic to include one or the other depending on the version, eg:

https://github.com/ckan/ckanext-harvest/blob/e2e8acbb6d87d44663e92c5ff5cbd67401588c2d/ckanext/harvest/templates/base.html#L6-L7

[zesdatascience]

Thanks very much for your help @amercader with the explanation above . For now I think I will use the blobfuse solution above to mount Azure storage and then use the inbuilt filestore functionality to use a folder under this mount point. I did some initial testing on this and it seems to work OK.