CKAN - Custom plugin/theme error datastore using fluent presets

[miguelangelch]

I have created a custom theme/plugin for CKAN 2.10.1 and it uses fluent and scheming CKAN plugins.

In my the I have defined the following dataset.yaml:

scheming_version: 2
dataset_type: dataset
about: A reimplementation of the default CKAN dataset schema
about_url: http://github.com/ckan/ckanext-scheming


dataset_fields:

- field_name: title_translated
  label: Title
  preset: fluent_core_translated
  classes:
  - form-group
  - col-md-12
  mark_required: true
  form_placeholder: eg. A descriptive title
  form_snippet: extrafields_fluent_title.html


- field_name: name
  label: URL
  preset: dataset_slug
  form_placeholder: eg. my-dataset

When I create a dataset and then, I upload a CSV resource, in the dataset view I get this error:

Error: Unable to add package to search index: Solr returned an error:
Solr responded with an error (HTTP 400): [Reason: [doc=null] missing
required field: site_id]

And also, I see the Datastore active field value is False.

However, when I comment/remove the preset line (Only for fluent) it works and I am able to see the preview table and the Datastore active field value is True without the site_id error.

These are the values I have defined in my CKAN configuration file (.ini):

ckan.site_id = default
scheming.organization_schemas = ckanext.mycustomplugin_theme:schemas/organization.json
scheming.group_schemas = ckanext.mycustomplugin_theme:schemas/group.json
scheming.dataset_schemas = ckanext.mycustomplugin_theme:schemas/dataset.yaml
scheming.presets = ckanext.scheming:presets.json ckanext.fluent:presets.json ckanext.mycustomplugin_theme:schemas/presets.json"
ckan.locale_default = es
ckan.locales_offered = es en
ckan.plugins = envvars image_view text_view datatables_view webpage_view recline_graph_view pdf_view datastore datapusher scheming_nerf_index scheming_datasets scheming_groups scheming_organizations fluent mycustomplugin_theme activity

And this is the code of the form_snippet:

{#

Renders modified fluent_text.html.

Modified `attrs=` to tie the CKAN default language title to the 
slug preview and name input. Without this the slug preview and name
input would appear but would not auto fill like the default CKAN 
setup. This is likely due to the fact that tieing the URL to a 
single languge is not ideal.

Future solutions may include:

* using an ID as the slug / name (preferred)
* changing the slug / url as languages change

#}

{% extends 'scheming/form_snippets/_base.html' %}

{% block snippet_name %}
extrafields_fluent_title.html
{% endblock %}

{% import 'macros/form.html' as form %}

{%- for lang in h.fluent_form_languages(field, entity_type, object_type, schema) -%}
  {% call form.input(
    field.field_name + '-' + lang,
    id='field-' + field.field_name + '-' + lang,
    label=h.fluent_form_label(field, lang),
    placeholder=h.scheming_language_text(field.form_placeholder, lang),
    value=data[field.field_name + '-' + lang]
        or data.get(field.field_name, {})[lang],
    error=errors[field.field_name + '-' + lang],
    classes=['col-md-6']+field.classes,
    attrs={'data-module': 'slug-preview-target', 'id': 'field-' + field.field_name, 'class': 'form-control'} if lang == h.extrafields_default_locale() else {'class': 'form-control'},    is_required=h.scheming_field_required(field)
    ) %}
    {%- snippet 'scheming/form_snippets/fluent_help_text.html',
      field=field,
      lang=lang -%}
  {% endcall %}
{%- endfor -%}

And this is the plugin.py content file:

import ckan.plugins as plugins
import ckan.plugins.toolkit as toolkit
from ckan.lib.plugins import DefaultTranslation
from ckan.common import config
from ckan.model import Package
import ckan.lib.helpers as h

def default_locale():
    '''Wrap the ckan default locale in a helper function to access
    in templates.
    Returns 'en' by default.
    :rtype: string
    '''
    value = config.get('ckan.locale_default', 'en')
    return value

def mycustomplugin_portal_url():
    portal_url = config.get('ckanext.mycustomplugin_theme.portal_url', 'https://dev.myckanwebsite.com/')
    if (h.lang() != 'es'):
      portal_url += h.lang() + '/'
    return portal_url

def remove_html_tags(html):
    return ''.join(xml.etree.ElementTree.fromstring(html).itertext())

def get_license(license_id):
    '''Helper to return license based on id.
    '''
    return Package.get_license_register().get(license_id)

def get_group(group_id):
    '''Helper to return the group.
    CKAN core has a get_organization helper but does not have one for groups.
    This also allows us to access the group with all extras which are needed to 
    access the scheming/fluent fields.
    '''
    group_dict = toolkit.get_action('group_show')(
        data_dict={'id': group_id})
    return group_dict

class MyCustomPluginThemePlugin(plugins.SingletonPlugin, DefaultTranslation):
    plugins.implements(plugins.ITranslation)
    plugins.implements(plugins.IConfigurer)
    plugins.implements(plugins.ITemplateHelpers)
    plugins.implements(plugins.IPackageController, inherit=True)

    # IConfigurer
    def update_config(self, config_):
        toolkit.add_template_directory(config_, "templates")
        toolkit.add_public_directory(config_, "public")
        toolkit.add_resource("assets", "mycustomplugin_theme")

    # ITemplateHelpers
    def get_helpers(self):
        """
        Provide template helper functions
        """
        return {
            'extrafields_default_locale': default_locale,
            'mycustomplugin_portal_url': mycustomplugin_portal_url,
            'mycustomplugin_theme_get_license': get_license,
            'mycustomplugin_theme_get_group': get_group
        }
    
    # IPackageController
    def before_dataset_search(self, search_params):
        search_params['q'] = search_params.get('q', '').lower()
        return search_params

Based on that information, what can be wrong or what I need to add to be able tu use the fluent presets in my CKAN custom plugin?

[kumarvivek1752]

@miguelangelch how did you solve this issue?