Replies: 2 comments 8 replies
-
|
Well there's Commons RDF but that's essentially a dead project because none of the contributing projects could agree on anything 🤣 Turning this around could we think about inverting the control flow here? Part of the overhead right now AFAIK is that we have to hand full control over to Titanium and then convert its final output data model back into Jena's data model. If instead Titanium had an output interface in the vein of Jena's StreamRDF then we could hand a Jena specific implementation of that interface to Titanium. Then Titanium call the relevant methods on its interface as it produces triples which Jena can then immediately convert to its own types in a streaming fashion. I don't know if this is feasible within the JSON-LD processing model but perhaps worth considering. WDYT? |
Beta Was this translation helpful? Give feedback.
-
|
Ok, not sure what aspect of Note that the lack of ability to stream process JSON-LD doesn't mean that that style of output interface isn't a possibility to consider, ultimately JSON-LD, like any RDF serialisation should just emit a list of triples when parsed. If you have some idea of what this hypothethical interface looks like in your head could you perhaps sketch it out as psuedo-code or equivalent so we aren't talking at cross purposes? |
Beta Was this translation helpful? Give feedback.
-
|
My goal for now, is to collect ideas, level of an interest, and requirements, then we can sketch the interface. Having an interface that allows you to pull out quads in a stream-like way is one option. I'm not saying it's a bad idea. The first question is, do we commit to it? Second, what are the requirements? And third, what should it look like? |
Beta Was this translation helpful? Give feedback.
-
|
Retro-fitting interface to existing code runs up against the fact that then there two ways to doing things. i.e. it is a significant impact on the existing system. Expanding on Rob's point - Having the parser stream out would be very good - no whole-parse temporary copy. Could the parser take functions (with generics?) to call to deliver the parts of URIs, literals, bnodes? Then Titanium user can create the RDF terms using native objects and stream as quads/triples? It's easier to work in "generic RDF" - a triple is 3 RDF terms, no restrictions on predicate must be URIs. This may be a part of why CommonsRDF does not get traction. At lower levels of systems, the asymmetric subject/predicate/object is a nuisence. |
Beta Was this translation helpful? Give feedback.
-
|
I've made a fork of Titanium where I use generics. I'll try to see if I can post some code to show how this helps with my use in RDF4J. |
Beta Was this translation helpful? Give feedback.
-
|
Here are some of the changes I've made to Titanium: When converting JSON-LD to RDF I've added generics, a consumer and a value factory. Here is how I use it from RDF4J: When writing data from RDF4J to JSON-LD I pass in my own dataset adapter: And I've made adapter classes for converting between the RDF4J representation and the Titanium representation of triples/quads/iris/etc. These could be optimized a lot when converting back and froth by keeping track of the internal RDF4J object. |
Beta Was this translation helpful? Give feedback.
-
|
Current Titanium API works as a supplier (PULL), and accepts only complete data (PULL). requirements
PUSH: Titanium -> Jena// Jena calls toRDF
void JsonLd.toRDF(jsonld).options(...).produce(consumer);
//Titanium writes RDF to the provided consumer inside produce method
loop [
// Jena provided code called inside produce method
consumer.dataset(...);
consumer.accept(...)
]
// Jena gets back full control and might close the consumer
consumer.end(); // can be dropped, as toRDF's done writingPUSH: Jena -> Titanium// Jena calls fromRDF
<RDFConsumer, Supplier<JsonArray>> consumer = JsonLd.fromRDF().options(...).consumeAndSupply();
// Jena writes RDF to the consumer
loop [
consumer.dataset(...).accept(...).accept(...).accept(...) // Flow?
.dataset(...).accept(...);
]
// Jena closes the consumer
consumer.end();
// Jena gets JSON-LD
JsonArray jsonld = consumer.get(); // could act as end()APIinterface RDFConsumer {
dataset(name);
accept(resource, predicate, value, datatype);
accept(resource, predicate, value);
// end can be dropped to keep it minimal
end();
}Just brainstorming, thank you for the comments. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah I think this looks like a good direction to explore. For the Does this mean there's multiple overloads for accept(String resource, URI predicate, URI value);
accept(URI resource, URI predicate, String value, URI datatype);
accept(URI resource, URI predicate, URI value);The main possible confusion I could see there is how you send Blank Nodes. For the subject position it's kinda obvious that I think the other possibility that was suggested was something like the following: interface RDFConsumer<T> {
// Converters from simple types to the consumers strong type
T uri(String uri);
T blankNode(String id);
T literal(String value, String lang, String datatype);
// Output
dataset(T name);
accept(T resource, T predicate, T value);
end();
}And then your loop becomes something like the following: loop [
// Jena provided code called inside produce method
T graph = ...;
T subject = consumer.uri(u);
// etc.
consumer.dataset(...);
consumer.accept(...)
]Makes the logic on Titanium's side slightly more complex but its allowing the consumer to control the target types directly. For going the other way it gets more complex, so maybe you need |
Beta Was this translation helpful? Give feedback.
-
|
Creating Java URI objects is very expensive, performance wise. I recommend avoiding that at all cost. Generics is preferable for RDF4J. |
Beta Was this translation helpful? Give feedback.
-
Let's collaborate on a new repository with RDF dataset interfaces that would allow seamless integration between Jena & Titanium and maybe other libs, like URDNA2015, N-QUADS Reader/Writer (bundled with Titanium now) etc.
The interface(s) should be minimal and single purpose, to exchange RDF dataset, i.e. reading. It is hard, and there are existing attempts, but something like micro-jars, containing only minimum necessary, single purpose, interfaces could be a way to deal with limitations/features coming from strongly typed language and dependencies linking enforcing strong consistency. In comparison to the JavaScript world where compatibility is declared.
Beta Was this translation helpful? Give feedback.
All reactions