GH-4232 performance issue in VALUES clause query #4330

abrokenjester · 2022-12-18T03:15:26Z

GitHub issue resolved: #4232

Briefly describe the changes proposed in this PR:

ensure VALUES clause always covers entire group graph pattern. This is the generically correct way to parse VALUES clauses. An optimizer can potentially look at the ordering in the algebra to push the values clause down into the join tree (by inspecting which parts of the tree have variables bound in the VALUES clause).
adding benchmarks

PR Author Checklist (see the contributor guidelines for more details):

my pull request is self-contained
I've added tests for the changes I made
I've applied code formatting (you can use mvn process-resources to format from the command line)
I've squashed my commits where necessary
every commit message starts with the issue number (GH-xxxx) followed by a meaningful description of the change

...df/src/test/java/org/eclipse/rdf4j/sail/nativerdf/benchmark/SPARQLValuesClauseBenchmark.java

hmottestad · 2022-12-18T09:35:35Z

Results from my laptop:

Benchmark                                          Mode  Cnt    Score   Error  Units
SPARQLValuesClauseBenchmark.simpleEquivalentQuery  avgt    5    0.606 ± 0.007  ms/op
SPARQLValuesClauseBenchmark.valuesOptionalQuery    avgt    5  298.065 ± 6.825  ms/op

The query with the VALUES clause uses ~300ms per query while the other query uses ~0.6ms.

abrokenjester · 2022-12-23T21:05:05Z

On my laptop:

Benchmark                                          Mode  Cnt    Score    Error  Units
SPARQLValuesClauseBenchmark.simpleEquivalentQuery  avgt    5    0.956 ±  0.093  ms/op
SPARQLValuesClauseBenchmark.valuesOptionalQuery    avgt    5  452.636 ± 23.386  ms/op

Looks like you have a slightly better laptop :)

abrokenjester · 2022-12-23T21:11:31Z

Query explanation simple query:

Projection (resultSizeActual=505)
╠══ProjectionElemList
║     ProjectionElem "parent"
║     ProjectionElem "child"
╚══LeftJoin (LeftJoinIterator) (resultSizeActual=505)
   ├──StatementPattern (resultSizeEstimate=504, resultSizeActual=504)
   │     Var (name=parent)
   │     Var (name=_const_f5e5585a_uri, value=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, anonymous)
   │     Var (name=_const_7e619dcc_uri, value=http://www.w3.org/2002/07/owl#Class, anonymous)
   └──Join (JoinIterator) (resultSizeActual=2)
      ╠══StatementPattern (costEstimate=50, resultSizeEstimate=9.9K, resultSizeActual=2)
      ║     Var (name=child)
      ║     Var (name=_const_4592be07_uri, value=http://www.w3.org/2000/01/rdf-schema#subClassOf, anonymous)
      ║     Var (name=parent)
      ╚══StatementPattern (costEstimate=1, resultSizeEstimate=504, resultSizeActual=2)
            Var (name=child)
            Var (name=_const_f5e5585a_uri, value=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, anonymous)
            Var (name=_const_7e619dcc_uri, value=http://www.w3.org/2002/07/owl#Class, anonymous)

Query explanation values clause query:

Projection (resultSizeActual=505)
╠══ProjectionElemList
║     ProjectionElem "parent"
║     ProjectionElem "child"
╚══LeftJoin (LeftJoinIterator) (resultSizeActual=505)
   ├──Join (JoinIterator) (resultSizeActual=504)
   │  ╠══BindingSetAssignment ([[type=http://www.w3.org/2002/07/owl#Class]]) (costEstimate=1, resultSizeEstimate=1, resultSizeActual=1)
   │  ╚══StatementPattern (costEstimate=504, resultSizeEstimate=504, resultSizeActual=504)
   │        Var (name=parent)
   │        Var (name=_const_f5e5585a_uri, value=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, anonymous)
   │        Var (name=type, value=http://www.w3.org/2002/07/owl#Class)
   └──Join (JoinIterator) (resultSizeActual=2)
      ╠══StatementPattern (costEstimate=11, resultSizeEstimate=508, resultSizeActual=254.0K)
      ║     Var (name=child)
      ║     Var (name=_const_f5e5585a_uri, value=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, anonymous)
      ║     Var (name=type)
      ╚══StatementPattern (costEstimate=1, resultSizeEstimate=9.9K, resultSizeActual=2)
            Var (name=child)
            Var (name=_const_4592be07_uri, value=http://www.w3.org/2000/01/rdf-schema#subClassOf, anonymous)
            Var (name=parent)

abrokenjester · 2022-12-23T22:55:21Z

I still can't see exactly what goes wrong, other than that it just seems to be a pathological case in terms of the data shape. I'm starting to wonder if we should have an optimizer that just duplicates the bindingsetassignment clause into the right arg of the left join. We can't pre-bind values in the right arg of the left join, but perhaps if we optimize to something like this:

Projection (resultSizeActual=505)
╠══ProjectionElemList
║     ProjectionElem "parent"
║     ProjectionElem "child"
╚══LeftJoin (LeftJoinIterator) (resultSizeActual=505)
   ├──Join (JoinIterator) (resultSizeActual=504)
   │  ╠══BindingSetAssignment ([[type=http://www.w3.org/2002/07/owl#Class]]) (costEstimate=1, resultSizeEstimate=1, resultSizeActual=1)
   │  ╚══StatementPattern (costEstimate=504, resultSizeEstimate=504, resultSizeActual=504)
   │        Var (name=parent)
   │        Var (name=_const_f5e5585a_uri, value=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, anonymous)
   │        Var (name=type, value=http://www.w3.org/2002/07/owl#Class)
   └──Join (JoinIterator) (resultSizeActual=2)
      ╠══ Join
           -> BindingSetAssignment ([[type=http://www.w3.org/2002/07/owl#Class]]) (costEstimate=1, resultSizeEstimate=1, resultSizeActual=1)
           ╠══StatementPattern (costEstimate=11, resultSizeEstimate=508, resultSizeActual=254.0K)
           ║     Var (name=child)
           ║     Var (name=_const_f5e5585a_uri, value=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, anonymous)
           ║     Var (name=type)
           ╚══StatementPattern (costEstimate=1, resultSizeEstimate=9.9K, resultSizeActual=2)
                   Var (name=child)
                  Var (name=_const_4592be07_uri, value=http://www.w3.org/2000/01/rdf-schema#subClassOf, anonymous)
                 Var (name=parent)

(ugly manual editing by me to get the idea across)

I'm not yet sure if that is something that is generally legal tbh. Just recording my thoughts for the next time I revisit.

hmottestad · 2022-12-26T05:55:47Z

In this case I think it's an issue with the join optimizer. The join optimizer doesn't really need to be limited by scoping issues when it calculates cardinality, as long as any optimizations are legal.

The join optimizer could assume that the binding set assignment will apply and use it to both calculate the size and to ignore the positive effect that binding the ?type variable early will have.

This is the generically correct way to parse VALUES clauses. An optimizer can potentially look at the ordering in the algebra to push the values clause down into the join tree (by inspecting which parts of the tree have variables bound in the VALUES clause).

This benchmark uses generated data conforming to the query pattern, and executes performance tests on both the variant with a VALUES clause, and (as a baseline) the simple equivalent query. Unfortunately, sofar I have been unable to reproduce any significant performance difference.

abrokenjester · 2023-01-20T23:09:11Z

In this case I think it's an issue with the join optimizer. The join optimizer doesn't really need to be limited by scoping issues when it calculates cardinality, as long as any optimizations are legal.

The join optimizer could assume that the binding set assignment will apply and use it to both calculate the size and to ignore the positive effect that binding the ?type variable early will have.

I don't quite understand what you're getting at. As far as I can tell the problem is that the join optimizer calculates an initial cost for the (?child, type, ?type) pattern that is lower than it should be due to its base cardinality being lower than that of the (?child, subClassOf, ?parent) pattern. I'm struggling to see how to accurately compensate here though, to be honest. How can we inform it at the time of cost calculation that ?type is different from ?parent because the former is set by a BSA?

hmottestad reviewed Dec 18, 2022

View reviewed changes

...df/src/test/java/org/eclipse/rdf4j/sail/nativerdf/benchmark/SPARQLValuesClauseBenchmark.java Show resolved Hide resolved

abrokenjester and others added 3 commits January 21, 2023 10:08

GH-4232 updated benchmark to reproduce the issue

818fe45

abrokenjester force-pushed the GH-4232-inlinedata-parsing branch from 60e2402 to 33ce6d6 Compare January 20, 2023 21:08

GH-4232 added regression test for debugging purposes

7794d0f

abrokenjester force-pushed the GH-4232-inlinedata-parsing branch from 33ce6d6 to 7794d0f Compare January 20, 2023 23:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH-4232 performance issue in VALUES clause query #4330

GH-4232 performance issue in VALUES clause query #4330

abrokenjester commented Dec 18, 2022

hmottestad commented Dec 18, 2022

abrokenjester commented Dec 23, 2022

abrokenjester commented Dec 23, 2022

abrokenjester commented Dec 23, 2022

hmottestad commented Dec 26, 2022

abrokenjester commented Jan 20, 2023 •

edited

GH-4232 performance issue in VALUES clause query #4330

Are you sure you want to change the base?

GH-4232 performance issue in VALUES clause query #4330

Conversation

abrokenjester commented Dec 18, 2022

hmottestad commented Dec 18, 2022

abrokenjester commented Dec 23, 2022

abrokenjester commented Dec 23, 2022

abrokenjester commented Dec 23, 2022

hmottestad commented Dec 26, 2022

abrokenjester commented Jan 20, 2023 • edited

abrokenjester commented Jan 20, 2023 •

edited