Replies: 2 comments
-
I currently think FilterPattern only having a StatementPattern as child should be merged into a new operator a query optimizer and that such a FilteredStatementPattern should then have it's own evaluation steps. Implementations that can really push the filter down into the storage (getStatements) layer or just add the pushed down filter into the handleFilter method of StatementPatternQueryEvaluationStep. FYI: The FedX does the new kinds operator approach to query evaluation. |
Beta Was this translation helpful? Give feedback.
-
Thanks @JervenBolleman. This is indeed the route we're walking: pushing the Filter into a data access node (which in our case often is already a combination of several The issue I'm also trying to tackle here is that while the filter may be pushed down, it creates the dependency here that the right-hand side of a join 'requires' bindings to be provided, and I want to make this dependency explicit. I.e., when pushing down a filter that references fields from both the left and right-hand side of a join into e.g. the right-hand side, evaluation of the right-hand side is only possible if solutions from the left-hand side are available. |
Beta Was this translation helpful? Give feedback.
-
I'm working on an RDF4J implementation over our existing databases (HBase and Elasticsearch). We're pushing down filters into Elasticsearch such as comparisons and the like. Our current approach is a little course, and I'd like some feedback on how this could be improved.
Queries such as this are fairly trivial:
However, we also want to support queries such as:
For the later query, the basic algebra is:
Conceptually, this optimisation isn't all that difficult, but it relies on knowledge about the join strategy of the join between the artist patterns (yielding
?album
and?birth_date
). In case of a nested loop join (JoinIterator
), bindings for these variables are pushed into the?album :created ?release_date
statement pattern. So if instead the Filter operator would instead be the right argument of that join (and the?album :created ?release_date
statement pattern be the argument of the filter), we've got exactly what we want.The
Filter
andStatementPattern
would be replaced by an operator that pushes the filtering into an Elasticsearch index.Now, query planning in RDF4J does not really cover the physical aspect of operators (at least not with
QueryOptimizer
s). Is this something that should be part of the pre-compilation / prepare 'phase'? Or are there other routes that can be taken? At least one option I've been thinking about is changing the Join operator with a NestedLoop (extends Join) operator (mixing logical and physical optimisations in the same pipeline).Beta Was this translation helpful? Give feedback.
All reactions