Support for DCAT-AP 2.0

[aidig]

DCAT v.2.0 is now Proposed Recommendation by W3C (https://www.w3.org/TR/vocab-dcat-2/)

Also, DCAT-AP v.2.0 has been completed in the context of the European Commission’s ISA2 programme (https://joinup.ec.europa.eu/solution/dcat-application-profile-data-portals-europe/release/200)

What are the plans of updating this extension to be compatible with DCAT-AP 2.0 / alternatively creating a new extension?

[metaodi]

This extension has the ability to use so called profiles. So in order to use DCAT-AP 2.0 someone would need to create a profile for it.
There is no need for a new extension.

[aidig]

I've tried to summarize the added properties (from DCAT-AP 1.1 to 2.0) below:

dcat:Dataset

Add dct:creator (Optional property)
Add dct:isReferencedBy (Optional property)
Add prov:qualifiedAttribution (Optional property)
Add dcat:spatialResolutionInMeters (Optional property)
Add dcat:temporalResolution (Optional property)
Add adms:versionNotes (Optional property)
Add prov:wasGeneratedBy (Optional property)
Note that dct:spatial is now a recommended property (instead of optional)
Note that dct:temporal is now a recommended property (instead of optional)

dcat:Distribution

Add dcatap:availability (Recommended property)
Add dcat:accessService (Optional property)
Add dcat:compressFormat (Optional property)
Add odrl:hasPolicy (Optional property)
Add dcat:packageFormat (Optional property)
Add prov:qualifiedAttribution (Optional property)
Add dcat:spatialResolutionInMeters (Optional property)

[amercader]

Thanks @aidig, that's really useful in case someone wants to jump in and update the current profile or create a new one.

Are the new fields for Distribution (dcatap:availability, dcat:accessService, etc) mandatory?

[aidig]

Thanks @aidig, that's really useful in case someone wants to jump in and update the current profile or create a new one.
Are the new fields for Distribution (dcatap:availability, dcat:accessService, etc) mandatory?

Thanks! Have just updated the comment above with a property constraint note (Mandatory/Recommended/Optional)

[aidig]

View comparison of properties DCAT-AP 1.1 vs. DCAT-AP 2.0 here:
https://docs.google.com/presentation/d/16kFrtpNIXpFpe2EuWELEr4_MOVyJrHvF_pA8EQzpt7k/edit?usp=sharing

[aidig]

DCAT v.2.0 is now a W3C Recommendation (4th Feb 2020) and DCAT 1.0 has been superseeded.

DCAT 2 supersedes DCAT [VOCAB-DCAT-20140116], but it does not make it obsolete. DCAT 2 maintains the DCAT namespace as its terms preserve backward compatibility with DCAT [VOCAB-DCAT-20140116]. DCAT 2 relaxes constraints and adds new classes and properties, but these changes do not break the definition of previous terms.
Any new implementation is expected to adopt DCAT 2, while the existing implementations do not need to upgrade to it, unless they want to use the new features. In particular, current DCAT deployments that do not overlap with the DCAT 2 new features (e.g., data services, time and space properties qualified relations, packaging) don't need to change anything to remain in conformance with DCAT 2.

https://www.w3.org/TR/vocab-dcat-2/

[dr-shorthair]

Note that dct:spatial is now a recommended property (instead of optional)
Note that dct:temporal is now a recommended property (instead of optional)

But please note that 'recommended' only means 'use this property when a dataset has a spatial/temporal extent'.

We also did a bit of work to develop recommendations about how to represent time and space in the values of dct:temporal and dct:spatial - see https://www.w3.org/TR/vocab-dcat-2/#time-and-space for examples

[aidig]

Anyone out there planning to "jump in" and create a profile for DCAT-AP 2.0? :-)

[bertvannuffelen]

Question: are dataServices also supported? Or will that not be considered?

[alphiephalphie]

Anyone out there planning to "jump in" and create a profile for DCAT-AP 2.0? :-)

My organization aims to implement DCAT within CKAN, and our product owners are requesting to use one of the DCAT 2 properties, dct.creator. I understand the properties must be added to profiles.py, but where, and how?

Additionally, there are three properties(accrualPeriodicity, license, accessRights) in the DCAT-AP v1.1 that we are unable to harvest/parse from datasets, but I see them in profile.py... is this a known issue?

[seitenbau-govdata]

Additionally, there are three properties(accrualPeriodicity, license, accessRights) in the DCAT-AP v1.1 that we are unable to harvest/parse from datasets, but I see them in profile.py... is this a known issue?

@ar-woodbury I suggest to open a new issue targeting these problem. There you can explain it in more detail, e.g. how the properties looks in the graph which will be imported and what is the result.

[seitenbau-govdata]

We plan to update the current euro_dcat_ap profile and adding some properties from the DCAT-AP version 2.1. Our approach is to update the current profile and not to create a new profile. As far as it is possible the existent properties will still be supported. Soon we will create a PR in draft mode and update it regularly.

[amercader]

@seitenbau-govdata does this mean that the same default profile will be valid for both DCAT-AP 1.1 and 2.1?

[seitenbau-govdata]

@amercader Yes. New properties we can just add to parsing and serializing. But it will be a challenge for changed properties, as dct:temporal for example. Here we have thought about that it could be a feasible solution to add duplicate the value(s) in the properties dcat:startDate/dcat:endDate and schema:startDate/schema:endDate. As if they were two separate profiles called one after the other. At the end the challenge with the serializing will be the same implementing it in one or in a separate profile. At the moment we see now advantage to implement it in a separate profile. What's your opinion on this?

[amercader]

@seitenbau-govdata you mean that DCAT-AP 1.1 expects dcat:startDate/dcat:endDate and DCAT 2.1 expects schema:startDate/schema:endDate (or viceversa) and you suggest that the serialized output contains both?

Perhaps if there are just a few of these backwards incompatible changes we can follow this approach but if there are more perhaps a separate profile (that inherits all logic from the current DCAT 1.1 based one) seems cleaner? I just don't know how many of these changes are. Is there a good changelog document available?

@aidig kindly linked to a document that doesn't seem to be available anymore:

View comparison of properties DCAT-AP 1.1 vs. DCAT-AP 2.0 here:
https://docs.google.com/presentation/d/16kFrtpNIXpFpe2EuWELEr4_MOVyJrHvF_pA8EQzpt7k/edit?usp=sharing

[seitenbau-govdata]

@amercader I think https://www.w3.org/TR/vocab-dcat-2/#changes-since-20140116 should describe the changes between DCAT-AP 1.x and DCAT-AP 2.x. We plan to add support for some of the changes, not all. We will list the properties in the pull request.

Do you think it`s a problem if there are some additional properties which are unknown in one of the DCAT-AP version?
We are really struggling with the decision to update the existent profile or create a new profile for DCAT-AP 2.1. A separate profile is cleaner, of course. But this has its advantages and disadvantages. The advantage is to keep the DCAT-AP 1.1 profile without DCAT-AP 2.1, the disadvantage is that you have to add/change the profile in the CKAN configuration to activate the DCAT-AP 2.1 support. And probably a little bit more work with the documentation.

We have just started to expand the existing profile. But with the thought that this can be split into a new profile at any time.

[amercader]

@seitenbau-govdata I really think that keeping the two profiles separate will make maintenance easier in the future and make things easier for site maintainers to transition from one to the other.

I think it's a fair ask to update a config setting to change the default profile (we can introduce the 2.1 profile but keep the 1.1 profile as default or a version or too to not break existing setups, and then switch to 2.1 as default). We can announce this change and add it to the changelog. (We can even make the setting configurable via the UI if that helps)

This is not exposed in the app endpoints but the parser/serializer support passing the desired profiles, so potentially the same site could expose the same metadata on both profiles to keep compatibility (if there's a use case for that)

All in all I think it makes sense to create a separate profile.

[seitenbau-govdata]

@amercader All right, let's create a new profile for DCAT-AP 2. 😃 We propose euro_dcat_ap_2 following the existing profile euro_dcat_ap. Should we use the class EuropeanDCATAPProfile as super class and calling the super methods of parse_dataset, graph_from_dataset and graph_from_catalog at the beginning? Or should we use the super class RDFProfile to make it more independent from the old profile by moving the logic for every attribute or attribute group into private methods and calling them explicitly, as has already been done in the class SchemaOrgProfile?

In the end the new profile need to support mostly of the new and changed attributes defined in DCAT-AP 2, but also the still valid attributes from DCAT-AP 1. Or could it be a valid option that only the new and changed attributes in DCAT-AP 2 are handled in the new DCAT-AP 2 profile and both profiles have to be called one after the other to get the complete DCAT-AP 2?

What do you prefer?

[amercader]

@seitenbau-govdata Without being familiar with the DCAT-AP 2.x changes, my gut feeling is that for a first implementation extending EuropeanDCATAPProfile and adding the new fields on top (or modifying those where behaviour has changed between the two profiles) will be the easiest.

[seitenbau-govdata]

@amercader Yes. I agree. We have just updated the pull request.