Implement notification information in the Description Resource

[jaxoncreed]

Feature description

Currently, this is the description resource

http://localhost:3001/example/profile/card.meta

[
      {
        "@id": "http://localhost:3001/example/profile/card",
        "http://www.w3.org/ns/ma-ont#format": [
          {
            "@value": "text/turtle"
          }
        ],
        "@type": [
          "http://www.w3.org/ns/ldp#Resource"
        ],
        "http://purl.org/dc/terms/modified": [
          {
            "@value": "2023-04-04T20:33:34.000Z",
            "@type": "http://www.w3.org/2001/XMLSchema#dateTime"
          }
        ]
      },
      {
        "@id": "http://purl.org/dc/terms/",
        "http://purl.org/vocab/vann/preferredNamespacePrefix": [
          {
            "@value": "dc"
          }
        ]
      },
      {
        "@id": "http://www.w3.org/ns/ldp#",
        "http://purl.org/vocab/vann/preferredNamespacePrefix": [
          {
            "@value": "ldp"
          }
        ]
      },
      {
        "@id": "http://www.w3.org/ns/posix/stat#",
        "http://purl.org/vocab/vann/preferredNamespacePrefix": [
          {
            "@value": "posix"
          }
        ]
      },
      {
        "@id": "http://www.w3.org/2001/XMLSchema#",
        "http://purl.org/vocab/vann/preferredNamespacePrefix": [
          {
            "@value": "xsd"
          }
        ]
      }
    ]

This is missing fields defined in the notifications spec (https://solid.github.io/notifications/protocol#description-resource-data-model) specifically the subscription and channel fields.

These should be added to the description resource.

Open spec questions: Is there a specific context/frame that all discovery document MUST use when being requested with application/json+ld? I think there should be. I've opened another issue for this question here: solid/notifications#165

[jaxoncreed]

Could someone with admin access assign me to this feature? I plan to work on it unless there are people already working on it.

[elf-pavlik]

Based on text from #1572

Discovery is done through the storage description resource(s). The server returns the same triples for every such resource as the notification subscription URL is always located in the root of the server.

I think is up to the server to decide if they provide information in storage description, description resource or both.

[joachimvh]

The notification spec does not require discovery through the specific description resources. It just states that if servers want to allow discovery through those which triples need to be present. For the CSS implementation I decided to use the alternative option of allowing discovery through the storage description resource instead, as we needed to add that one anyway for the 0.10 version of the Solid protocol spec.

I'm not against adding the other option as well but the reason it was not done yet is because there currently isn't a generic system in place in CSS to add dynamic metadata to resources. This would have to be added first, so it could then be used to dynamically add the notification discovery metadata. Probably through some sort of wrapping DataAccessor that modifies the results of the getMetadata call. If that is something you want to add we can discuss more about the solution here.

As was mentioned in the linked issue, there is no JSON-LD framing. For description resources this would be even more complicated than just for the notification-specific resources I think, as there are several specifications that have opinions on what should be in a description resource. But even for those specifically of the notification specification there are no such requirements, so to be safe all responses of the related APIs should be interpreted as RDF and not as a JSON document with specific fields.

[csarven]

To be clear, the Solid Notifications Protocol is not claiming the entirety of Description Resource. It is merely describing the data model that's used as part of a Description Resource (or a Storage Description Resource) to advertise Subscription Services and Notification Channels.

Description Resource and Storage Description Resource are not merely interchangeable either. The purpose of the former is resource-specific whereas the latter is intended for all resources in a storage. Granted that the Storage Description Resource could technically describe specific Subscription Services/ and Notification Channels available to each of the resources in a storage - again, possible, but not a good idea for obvious reasons - and Discovery Clients could even discover a topic resource of interest, but none of that is required for conformance with respect to Storage Description Resource. Moreover, the intended discovery of Subscription Services and Notification Channels for a topic resource is the associated Description Resource.

So again, if CSS wants to use the same Subscription Service and Notification Channel for all resources in a storage, Storage Description Resource is indeed the way to go. Discovery Clients will indeed use Storage Description Resource for that purpose.

[elf-pavlik]

IMO storage description is perfectly fine for the discovery of Subscription Services.
Description Resource comes more useful when we try to discover pre-established Notification Channels which use the described resource as its topic.
Since such pre-established channels only make sense for either public-read resources (eg. WebID Documents) or protected resources where the channel type supports authentication/authorization directly on receiveFrom (e.g., #1574) I wouldn't personally prioritize work on Solid Notifications discovery via Description Resource until there is a way to create pre-established notification channels.

[jaxoncreed]

Okay, this makes sense. So, is a discovery client supposed to fetch both the storage description and the resource description? What if they both have notification information? Is there one that should be favored over the other?

[elf-pavlik]

Given that CSS currently only implements storage description, I would start with trying that and if not successful only then try the Description Resource. For sendTo based channel types (like Webhook), which always look for a Subscription Service, the storage description-based discovery (if available) should always be sufficient.

EDIT: Let's be careful not to use this issue in the CSS repo for discussions that would be better served in the notifications spec repo or in a client lib repo.

[jaxoncreed]

Hey, sorry again. I'm having a hard time finding where the subscription/notification information is under the storage resource. Here's what I've tried:

The notifications spec says:

Resource Servers that want to enable Discovery Clients to discover subscription services and notification channels available to a storage in which a given resource is in MUST advertise the associated resources describing the subscription services and notification channels by responding to an HTTP request including a Link header with the rel value of http://www.w3.org/ns/solid#storageDescription [SOLID-PROTOCOL] and the Description Resource as link target [RFC8288].

I can't find a request that yields a link header with <http://somepod.com/somestoragedescription.meta>; rel="http://www.w3.org/ns/solid#storageDescription"

Here are some the requests I've tried:

HEAD http://localhost:3000/example/profile/card: This provides a describedby link, but not a storage link. This could make sense because the card itself is not storage.

So, I moved up the URI path hierarchy as described in the Solid Protocol (https://solidproject.org/TR/protocol#storage-resource) until I found the storage resource

HEAD http://localhost:3000/example/. This also didn't include <http://somepod.com/somestoragedescription.meta>; rel="http://www.w3.org/ns/solid#storageDescription".

Maybe I'm missing something. Could you provide the requests I should make to discover the notifications/subscription information?

Note: Yeah, I'll open a spec issue next time to discuss off topic things.

[joachimvh]

I can't find a request that yields a link header with <http://somepod.com/somestoragedescription.meta>; rel="http://www.w3.org/ns/solid#storageDescription"

Which version of the server are you testing this on? As this was only added in the v6 branch (which the notification PR is also targeting). I just tested it out myself and got the expected header:

Link: <http://localhost:3000/joachimvh/.well-known/solid>; rel="http://www.w3.org/ns/solid/terms#storageDescription"

Every resource that you send a HEAD/GET request to should give you that result, even if the resource itself is not a storage.

[csarven]

https://solidproject.org/TR/2022/protocol-20221231#server-storage-description

Aside: Interesting that CSS is still using the unregistered scheme .well-known/solid. Moreover, it is supposed to be used at the beginning of the URI path, so /joachimvh/.well-known/solid is a bit strange. Granted at the end of the day it is just some characters in the URI path but why not commit to something less confusing, if anything but visually. I suppose you can wait for the dust around storage description resource to settle but then again FWIW, NSS's /.meta naming convention is generally short and sweet (maybe vague, but not exciting).

[joachimvh]

Aside: Interesting that CSS is still using the unregistered scheme .well-known/solid. Moreover, it is supposed to be used at the beginning of the URI path, so /joachimvh/.well-known/solid is a bit strange. Granted at the end of the day it is just some characters in the URI path but why not commit to something less confusing, if anything but visually. I suppose you can wait for the dust around storage description resource to settle but then again FWIW, NSS's /.meta naming convention is generally short and sweet (maybe vague, but not exciting).

Having read your reply makes me realize that I probably misinterpreted that part of the specification. I read it as the "storage description resource" being a specific type of resource that contains the necessary information, which is separate from the description resource of the storage. But I see now that those should be the same thing 😅. Will have to look into how to merge those as that does change things a bit. Might have to implement the idea I suggested initially on this issue anyway to be able to do this.

[csarven]

Right. I've realised that storage description resource and the description resource of the storage resource could be different but practically can be the same resource when I implemented 1) the server requirement (experimental in NSS) and 2) application making use of it (dokieli). I've shared some of those findings in solid/specification#355 (comment) and requesting others with their implementation feedback.

At this time there is not enough data/experience to suggest that the specification should require or advise that those two resources should be distinct or hold non-overlapping information. So, at this time, it is an implementation detail in that servers can use the same resource (for the target of both of the link relations) or use separate resources for storage's auxiliary description.

[joachimvh]

Currently they differ in CSS, hence the .well-known/solid as I just needed any kind of relative path to reference it. Doesn't really matter what it is as this is just a string that can be configured. CSS does use .meta for description resources as that is what people are more used to. I might still look into merging them into the same resource as I can see the logic behind it.

[csarven]

I should add that the intention of storageDescription was/is to refer to "storage description of a storage in which this resource is in" (as mentioned in the comments). Practically speaking thus far, x and y are the same resource, and need not be different:

</foo> storageDescription <x> .
</> describedby <y> .

(</> storageDescription <x> . probably need not be stated... unless storage should be considered to contain itself? heh)

[jaxoncreed]

Which version of the server are you testing this on?

Yep that was it. v6 works fine. Thanks!